Systems and methods for inspection and analysis of mechanical equipment using broad frequency acoustic imaging

Abstract

Broad frequency acoustic imaging techniques for inspection and analysis of mechanical equipment are disclosed. In some instances an acoustic imaging device may obtain acoustic response data regarding acoustic signals emitted by mechanical equipment across a plurality of frequencies. The frequencies may be selected based on a characteristic of the mechanical equipment, or may be broadly selected across an acoustic frequency range. The acoustic signals may be compared to baseline acoustic signals emitted by mechanical equipment, and analyzed to determine at least one of a severity of degradation or a type of degradation.

Description

SYSTEMS AND METHODS FOR INSPECTION AND ANALYSIS OF MECHANICAL EQUIPMENT USING BROAD FREQUENCY ACOUSTIC IMAGINGBACKGROUND

[0001] Mechanical systems, such as rotating components (bearings, pulleys, and the like) may emit acoustic signals that change over time as the systems degrade. Mechanical system degradation and failure may, therefore, be detected through acoustic imaging, for example, by capturing acoustic signature data associated with a mechanical device at two or more different times and comparing those acoustic signatures to detect changes indicative of degradation.

[0002] Current attempts to monitor mechanical system degradation or failure using acoustic analysis typically involve monitoring a decibel reading at a single, predetermined frequency, e.g., 30 Hz. However, the acoustic response of mechanical system degradation is often not limited to or reliably detectable at a single frequency.

[0003] Furthermore, traditional analysis of acoustic signals emitted by mechanical equipment is based on differences in acoustic signal strength between an initial acoustic signal at a selected frequency and a current acoustic signal at the same frequency. Predetermined thresholds are set (e.g., at 8 dB, 12 dB, 16 dB, 35 dB, and the like) to define the extent of degradation of the mechanical equipment. However, such basic classification at a particular frequency fails to accurately capture degradation that may differ in frequency depending on severity, and does not provide any information regarding the type of degradation that might be occurring.SUMMARY

[0004] The present disclosure generally relates to systems and methods for inspection and analysis of mechanical equipment using broad frequency acoustic imaging techniques. In some instances, an acoustic imaging device may obtain acoustic response data regarding acoustic signals emitted by mechanical equipment across various frequencies. The frequencies may be selected based on various characteristics of the mechanical equipment (e.g., its rotational speed, design type, construction materials dimensions, type of moving / rolling elements, and the like), or may be broadly selected across an acoustic frequency range. The acoustic signals may be compared to baseline acoustic signals emitted by mechanical equipment, and analyzed to determine at least one of a severity of degradation or a type of degradation. Various models or classification techniques for determining severity and / or type of degradation are provided.

[0005] In a first aspect, an acoustic imaging system is disclosed. The system includes an acoustic sensor array, an optional camera system, an optional display, a processing system, and a memory. The processing system is communicatively connected to the acoustic sensor array, thecamera system, and the display, and the memory is communicatively connected to the processing system. The memory stores instructions which, when executed by the processing system, cause the processing system to: capture acoustic response data regarding acoustic signals emitted by mechanical equipment; compare the acoustic response data to baseline acoustic response data across a plurality of frequencies; and based on the comparison, analyzing and classifying the acoustic response data of the mechanical equipment as to at least one of degradation severity or degradation type.

[0006] In a second aspect, an acoustic imaging system is disclosed. The system includes an acoustic sensor array, an optional camera system, an optional display, a processing system, and a memory. The processing system is communicatively connected to the acoustic sensor array, the camera system, and the display, and the memory is communicatively connected to the processing system. The memory stores instructions which, when executed by the processing system, cause the processing system to: capture acoustic response data representative of acoustic signals emitted by a mechanical equipment across a plurality of frequencies; based on the acoustic response data, identify a mode of operation of the acoustic imaging system corresponding to mechanical equipment acoustic imaging from among a plurality of different modes of operation; and in accordance with the mode of operation, analyzing the acoustic response data of the mechanical equipment as to at least one of degradation severity or degradation type.

[0007] In a third aspect, a method of analyzing acoustic response data is disclosed. The method includes receiving, at a computing system, acoustic response data representative of acoustic signals emitted by a mechanical equipment, and comparing, at the computing system, the acoustic response data to baseline acoustic response data across a plurality of frequencies. The method further includes determining, at the computing system, a degradation profile based on the comparison, wherein the degradation profile includes at least one of a degradation type or a degradation severity.

[0008] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Non-limiting and non-exhaustive examples are described with reference to the following figures:

[0010] Fig. 1 is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure may be implemented.

[0011] Fig. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure may be implemented.

[0012] Fig. 3 is a schematic rear plan view of the acoustic imaging device of Fig. 2.

[0013] Fig. 4 is a schematic plan view of an acoustic imaging sensor array that may be integrated into an acoustic imaging device, according to example aspects of the present disclosure.

[0014] Fig. 5 is a schematic rear plan view of an alternative embodiment of the acoustic imaging device of Fig. 2.

[0015] Fig. 6 is a schematic rear plan view of a further alternative embodiment of the acoustic imaging device of Fig. 2.

[0016] Fig. 7 is a flowchart of a method of performing broad frequency acoustic imaging to obtain classification of type or severity of mechanical equipment degradation, according to an example embodiment.

[0017] Fig. 8 is a flowchart of a method of enabling interaction with acoustic imaging data and classification information to allow iterative improvement of degradation analysis, according to an example embodiment.

[0018] Fig. 9A is a flowchart illustrating a first portion of a method of performing broad frequency acoustic imaging-based detection of mechanical equipment degradation type and severity, in accordance with an example embodiment.

[0019] Fig. 9B is a flowchart illustrating a second portion of the method of Fig. 9A.

[0020] Fig. 10 is a diagram illustrating an example impact score analysis that may be performed in conjunction with aspects of the method of Figs. 9A-9B.

[0021] Fig. 11 is a diagram illustrating an example diagnostics analysis that may be performed in conjunction with aspects of the method of Figs. 9A-9B.

[0022] Fig. 12 is a frequency-domain diagram illustrating of acoustic data example types of mechanical degradation that may be detected in accordance with the analyses described in Figs.9A-9B.

[0023] Fig. 13 is a time-domain diagram illustrating of acoustic data example types of mechanical degradation that may be detected in accordance with the analyses described in Figs.9A-9B.

[0024] Fig. 14 is a flowchart of a method of performing constrained linear classification analysis to determine degradation severity of mechanical equipment.

[0025] Fig. 15 illustrates a method of determining degradation severity of mechanical equipment based on an area under curve analysis relating to changes in acoustic signal strength across a broad frequency range.

[0026] Fig. 16 illustrates a frequency and power level analysis associated with a plurality of different degradation types and severities, according to example experimentation.

[0027] Fig. 17 illustrates a frequency and power level analysis associated with a plurality of different degradation types and severities, according to example experimentation.

[0028] Fig. 18 illustrates a method of determining degradation severity of mechanical equipment based on a power spectral density analysis across a plurality of frequency ranges.

[0029] Fig. 19 illustrates a method of processing a power spectral density.

[0030] Fig. 20 illustrates an example power spectral density within a particular frequency range.

[0031] Fig. 21 illustrates a methodology of selection of overlapping frequency ranges for detection of degradation, according to an example embodiment.

[0032] Fig. 22 illustrates a method of determining degradation severity based on a frequency bin analysis, according to an example embodiment.

[0033] Fig. 23 illustrates a frequency and power level analysis within a plurality of frequency bins to determine relative degradation of mechanical equipment, according to example experimentation.

[0034] Fig. 24 illustrates a frequency and power level analysis within a plurality of frequency bins to determine relative degradation of mechanical equipment, according to example experimentation.

[0035] Fig. 25 is a method of performing a thresholding analysis of spectral power data to determine severity of degradation, according to an example embodiment.

[0036] Fig. 26 illustrates a frequency and power level analysis using thresholding on spectral power data to determine relative degradation of mechanical equipment, according to example experimentation.

[0037] Fig. 27 illustrates an example of peak analysis on spectral power data to determine relative degradation of mechanical equipment, according to example experimentation.

[0038] Fig. 28 is a method of analyzing periodicity of spectral data to determine severity of degradation, according to an example embodiment.

[0039] Fig. 29 illustrates a series of time-domain acoustic signals indicating various severities of degradation relative to a baseline for particular mechanical equipment.

[0040] Figs. 30A and 30B illustrate periodic components of acoustic data obtained from mechanical equipment illustrating different types of degradation, according to example experimentation.

[0041] Fig. 31 illustrates additional sub-audio periodic components of acoustic data obtained from mechanical equipment illustrating degradation, according to example experimentation.

[0042] Fig. 32 is a block diagram of an example machine learning framework in which acoustic data may be analyzed to determine a severity and / or classification of degradation of mechanical equipment, according to an example implementation.

[0043] Fig. 33 illustrates an example decision tree generated from a classifier model useable within the context of the framework of Fig. 32 to determine severity and / or classification of degradation of mechanical equipment, according to an example implementation.

[0044] Fig. 34 is a schematic view of a multi-layer perceptron that may be used in predicting severity and / or classification of degradation of mechanical equipment from received acoustic data, according to an example implementation.

[0045] Fig. 35 is a schematic process flow diagram of a method of creating a spectrogram and analyzing such a spectrogram using a deep learning model, such as a convolutional neural network within the context of the machine learning framework of Fig. 32, to determine severity and / or classification of degradation of mechanical equipment, according to an example implementation.

[0046] Fig. 36 is an example user interface illustrating selection of a rotational speed input mechanism for use in acoustic imaging of mechanical equipment, according to example embodiments.

[0047] Fig. 37 is an example user interface illustrating selection of broad frequency analysis for acoustic imaging of mechanical equipment, according to example embodiments.

[0048] Fig. 38 is an example user interface depicting detected rotational speed and acoustic frequency analysis at a selected frequency range, in accordance with example embodiments.

[0049] Fig. 39 is an example user interface depicting receipt of user-entered degradation severity, in accordance with example embodiments.

[0050] Fig. 40 is an example user interface depicting predicted degradation severity and type, in accordance with example embodiments.

[0051] Fig. 41 illustrates phases of impact score computation, in accordance with example embodiments.

[0052] Fig. 42 illustrates a way to combine the impact and friction scores, in accordance with example embodiments.DETAILED DESCRIPTION

[0053] As briefly described above, embodiments of the present disclosure are directed to methods and systems for acoustic imaging analysis across a broad frequency spectrum. In some examples, an acoustic imaging device may obtain acoustic response data regarding acoustic signals emitted by mechanical equipment across a plurality of frequencies. The frequencies may be selected based on a rotational speed of the mechanical equipment, or may be broadly selectedacross an acoustic frequency range. The acoustic imaging of the present disclosure is usable to detect mechanical system degradation or failure using broad frequency acoustic signature analysis. This analysis falls into two categories: severity and categorization (e.g., the extent of degradation or failure, and the type of degradation or failure).

[0054] Regarding severity, a broad spectrum of frequencies is analyzed to determine degradation severity. In an example implementation, acoustic data is captured from known good, known degraded, and known failed mechanical systems at varying severity. A classification model may be used that generates an overall severity score. The overall severity score may be derived from a set of component values represented as a difference from a baseline “known good” mechanical system. Each component value may be weighted according to experimental determination of relative importance or extent to which it indicates likely degradation or failure. Example components may include: a comparison to baseline of a ratio of frequency components marked as periodic to overall frequency components; a comparison to baseline of the spectral power in decibels of the periodic components detected across the broad spectrum; a comparison to baseline of spectral power in a particular lower end frequency band (e.g., 15 to 20 kHz); a comparison to baseline of spectral power in a particular higher and frequency band (e.g., 35 to 40 kHz); and an inverse of a detected mechanical component rotation speed (e.g., bearing rotation speed).

[0055] To obtain many of the analysis components, a plurality of analyses on acoustic data may be performed. For example, by analyzing a broad frequency range, a periodicity analysis may be performed to determine the portions of the acoustic signal which corresponds to periodic components and the portions of the acoustic signal that are aperiodic. Ratios of periodic to aperiodic signals, and more particularly a change in the ratio of periodic to aperiodic signals within acoustic data, may be indicative of changes in state.

[0056] In further examples, more or fewer components may be used depending on their detected contribution or indication of degradation severity; the specific selection of such components, and weighting thereof, may be derived experimentally and may be supplemented using data regarding different types of mechanical systems and mechanical system failures as such data is captured.

[0057] In still further examples, an overall severity score may be defined as an integer within a range of values, with a final severity score being assigned a severity classification from among two or more severity classifications (e.g., good, pre-failure, beginning of failure, advanced failure, imminent catastrophic failure) depending on preset thresholds that are based on experimental data.

[0058] Regarding classification, experimental data has shown that different types of failures of bearings may result in different acoustic signatures across a frequency range. For example, inthe case of a mechanical bearing, a good bearing will have a particular acoustic signature, while a bearing having metal flakes may have a similar acoustic signature but with greater effects of periodic components as acoustic noise is emitted at various rotational harmonics. A degreased bearing may have a slightly higher acoustic signature compared to a good bearing across low and mid frequency ranges. A rusted bearing may have a higher baseline acoustic signature at lower frequencies, as would a bearing having significant chemical etching or acid etching. Similar techniques to those used for severity classification may be implemented for failure type classification, e.g., looking at power or decibel level of acoustic data at different frequency windows, analysis of periodic versus aperiodic components, and the like.

[0059] As to both severity and categorization, the specific implementation used may vary in a few ways. For example, acoustic data may be captured via an acoustic imaging device, and transferred to a remote computing system for analysis at a later time. Such analysis may be performed and results displayed on another computing system, or returned to the acoustic imaging device for display.

[0060] Alternatively, acoustic data may be captured via an acoustic imaging device, and that data may be analyzed using particular algorithms or models implemented on that acoustic imaging device. This arrangement would allow for near real-time feedback regarding mechanical system degradation severity or type. The algorithms or models on the acoustic imaging device could be updated occasionally or periodically to improve accuracy or detect more types of degradation. Such updates may occur at the acoustic imaging device, or remotely, and returned to the acoustic imaging device for storage and use.

[0061] In a still further implementation, the analysis or modeling on the acoustic imaging device may be adaptive, and over time may adjust to accommodate new types of mechanical system failures. In this implementation, a user of the acoustic imaging device would provide annotating feedback on the device regarding known good or known faulty mechanical systems, and that information would be combined with the acoustic signature information to retrain or adapt models or equations used to determine degradation type or severity.

[0062] In accordance with the above general description and the following disclosure, it is recognized that the broad spectrum acoustic analysis described in the present application has a variety of advantages. In particular, by analyzing acoustic response data across a wide range of frequencies, a more accurate determination of severity of degradation of mechanical equipment may be obtained, because attention is paid to frequencies that might otherwise be overlooked but which may include signatures indicative of degradation. Still further, by performing different types of acoustic analysis across broad frequency ranges, particular types of damage or degradation may be detected, for example those which are periodic versus others which are aperiodic. Furthermore,the various available analysis techniques described herein may be applied alone or in combination, thereby providing flexibility as to amount of computational resources required, detail of degradation assessment, and the like.

[0063] At least some of these advantages are realized in accordance with the below described acoustic imaging system and techniques for performing analysis on acoustic response data as described herein.I. Operating Environment and Example Acoustic Imaging Systems

[0064] Referring first to Figs. 1-6, example acoustic imaging systems in which the user interfaces and methods of use and operation may be performed are described. The systems described herein should be considered exemplary, in that the user interfaces may be presented on a wide variety of types of systems and in various contexts, as is apparent from the details of those interfaces themselves.

[0065] Different than some broad frequency acoustic imaging technologies that use a controlled, active acoustic generator (e.g., ultrasound in medical imaging) to evaluate a target, various embodiments of the presently disclosed technology do not use or require any acoustic generator other than the target under evaluation itself (e.g., a rolling element that makes noise) and are based on passive capturing of acoustic signals emitted from the target under evaluation.

[0066] Referring initially to Fig. 1, an example acoustic imaging system 100 is depicted. The acoustic imaging system may include, in the example shown, an acoustic imaging device 102, optionally communicatively connected to one or more remote computing systems, such as remote system 10 and one or more rotational speed sensor systems 180.

[0067] The acoustic imaging device 102 may be, in various embodiments, a handheld device, a robotic or self-propelled device (e.g., either ground-based or airborne, as in the case of a drone), or a stationary device positioned to receive acoustic signals. In general terms, an acoustic imaging device, also referred to herein as an acoustic camera or acoustic imaging system, visualize sound waves and create images or maps of sound fields in a given area. It is designed to detect and display sound sources and their distribution, for example in real time. In the example shown, the acoustic imaging device 102 includes a processing system 110 communicatively connected to a memory 112, as well as to an acoustic sensor array 120, a camera system 125, a display 130, input devices 132, a power subsystem 140, and a communication interface 150. As illustrated, the acoustic imaging device 102 may also include, in some examples, a rotational speed sensor 145.

[0068] In the example shown the processing system 110 can include one or more programmable or special-purpose execution circuits capable of executing computing instructions. The memory 112 may be volatile or nonvolatile memory, such as read-only memory ("ROM"),random access memory ("RAM"), EEPROM, flash memory, or other memory technology. Those of ordinary skill in the art and others will recognize that memory 112 typically stores data or program modules that are immediately accessible to or currently being operated on by the processing system 110. In this regard, the processing system 110, including one or more processors, may serve as a computational center of the acoustic imaging device 102 by supporting the execution of instructions.

[0069] The acoustic sensor array 120 may include a plurality of spaced-apart acoustic sensors positioned to determine, based on the time and phase of receipt of acoustic signals, the direction, distance, and magnitude of acoustic signals emitted from an acoustic source. For example, in some implementations, the acoustic sensor array 120 may include up to 64 or more acoustic sensors spaced apart from each other, and configured to passively detect and capture acoustic signals in a frequency band of between 2 kHz and 90 kHz at up to, or exceeding, 70 meters. Other frequency bands may be used as well, including those below 2 kHz and up to or exceeding about 100 kHz. Each sensor within the array is responsible for detecting, capturing, and measuring the acoustic signals at a specific location. Each acoustic sensor within the array may be positioned in a specific spatial configuration, typically in a planar arrangement. The acoustic sensor array may include various signal amplifiers and / or analog to digital converter circuits, as well as a signal processing unit useable to convert or otherwise process the captured acoustic signals into digital form and extract such relevant data from the sensor array. In some examples, the acoustic sensor array may include such a signal processing unit, while in other examples, the processing system 110 may perform signal conversion or other processing. An example of such an acoustic sensor array 120 is depicted in Figs. 3-6, below.

[0070] The camera system 125 is positioned to capture an imaging field of view relative to the acoustic imaging device 102. The camera system captures image data, typically in a same direction of orientation as the acoustic sensor array 120, thereby allowing for display of image data with a location-corresponding overlay of acoustic signal data (also referred to herein as acoustic response data) as well as related analysis outcomes as described herein. In examples, the camera system 125 may be a digital still image camera, a video camera, or may include a plurality of camera devices. In example implementations, the camera system may include a digital camera configured to capture images at greater than 1 megapixel in image quality, and may have digital and / or optical zoom capabilities.

[0071] The display 130 may be any of a variety of display devices adapted for display of the image and acoustic information captured using the acoustic sensor array 120 and the camera system 125. In example implementations, the display 130 may be an LCD display and may be capable ofreceiving user input. In some examples, the display 130 is a touchscreen display, such as a capacitive touchscreen display.

[0072] The input devices 132 may include various additional input buttons or switches that are provided on the acoustic imaging device 102 beyond the touchscreen display. For example, in some instances, the input devices 132 may include a power button, an image capture button useable for image capture or to start / stop video capture, and the like.

[0073] The power subsystem 140 may include one or more power sources, such as a battery capable of providing electrical energy to the other components of the acoustic imaging device 102. In examples, the power subsystem 140 may include a rechargeable battery, such as a lithium-ion battery. Other battery types or power sources may be provided as well, such as a wired power connection.

[0074] The communication interface 150 may include one or more components for communicating with other devices, for example via a direct wired connection or over a network. Embodiments of the present disclosure may access basic services that utilize the communication interface 150 to perform communications using common network protocols. The communication interface 150 may correspond to a general purpose wired connection, such as a USB, Firewire, or analogous data connection, and / or may also include a wireless network interface configured to communicate via one or more wireless communication protocols, such as WiFi, 2G, 3G, 4G, 5G, LTE, WiMAX, Bluetooth, or the like.

[0075] In the example shown, a rotational speed sensor 145 may be integrated within the acoustic imaging device 102. The rotational speed sensor 145 may take a variety of forms. In one example embodiment, the rotational speed sensor 145 is implemented as a stroboscope. Such an example is seen in the embodiment illustrated in Fig. 4, below. In a further example embodiment, the rotational speed sensor 145 is implemented as a laser tachometer. Such an example is seen in the embodiment illustrated in Fig. 5, below. In still further example embodiments, the rotational speed sensor 145 may comprise circuitry used to process image data captured from camera system 125, for example to detect, from image frame analysis, a rotational speed of an object that is captured via an image sensor across a plurality of frames. By analyzing the phase and rotational position of the mechanical object, combined with the image of the item, rotational speed may be determined. An example of such rotational speed determinations based on image analysis is described in U.S. Patent No. 10,062,411, entitled “Apparatus and Method for Visualizing Periodic Motions in Mechanical Components”, the disclosure of which is hereby incorporated by reference in its entirety.

[0076] In the example shown, the acoustic imaging device 102 is communicatively connectable, via the communication interface 150, to remote system 10. The remote system 10includes a processing system 20, memory 22, display 30, input devices 32, and a communication interface 50, by way of example.

[0077] The remote system 10 may be implemented as a computing system, such as a desktop, server (virtual or physical), laptop, or handheld portable computing system (e.g., a tablet, cellular telephone, or other mobile device). The processing system 20 and memory 22 are analogous to those described above as included in the acoustic imaging device 102. Display 30 may be an LED, LCD, OLED, or other type of display, and may be implemented as a touchscreen or nontouchscreen display as well. The display 30 may be configured to present the various user interfaces described herein when image and acoustic data are received at the remote system 10 from the acoustic imaging device 102. Input devices 32 generally may include one or more buttons, touch inputs, and the like, which are dependent on the form factor of the remote system 10. The input devices 32 may include, for example, a keyboard, mouse, stylus, and the like.

[0078] The communication interface 50 is configured to provide wired and / or wireless communication with other devices, and can include, for example, a complementary connection to the communication interface 150 above. Via the communication interfaces 50, 150, the acoustic imaging device 102 and remote system 10 may exchange data captured via the acoustic sensor array 120 and camera system, and instructions from remote system 10, to allow for, e.g., local or remote storage of current and historical acoustic test data including acoustic data and image data, as well as remote control of the acoustic imaging device 102. The storage of current and historical acoustic test data may include storage of location information, acoustic signal levels, image data, test settings, and the like, and may be used, either alone or in combination with other test data, to determine or estimate a probability of degradation or failure of a target object that is the subject of acoustic imaging.

[0079] In accordance with aspects of the present disclosure, it is noted that some or all of the features of the acoustic imaging device 102 and / or remote system 10 may or may not be present in all implementations, and that such devices may include other functionalities and features (e.g., remote control features, mobility features, and the like) not described here. Generally speaking, the acoustic imaging device 102 and remote system 10 may be configured with a display capable of depicting the user interfaces described herein, either in real-time as part of control of the acoustic imaging device 102 or based on stored data and / or data transmitted from the acoustic imaging device 102 to the remote system 10. It is also noted that the acoustic imaging device and remote system may be referred to as a first computing device and / or a second computing device in aspects of the present disclosure and claims appended hereto.

[0080] In further examples, the acoustic imaging device 102 and remote system 10 may communicate with or through one or more cloud computing resources, such as cloud 70. The cloud70 may be used to implement detailed analysis of acoustic data captured by the acoustic imaging device 102, in accordance with the various broad frequency range analyses described herein to determine a severity and / or type of degradation of mechanical equipment. In examples, the cloud 70 may be used to store acoustic data retrieved from one or more acoustic imaging devices including the acoustic imaging device 102, and may be used for, e.g., calculating parameters and / or thresholds used in the deterministic analyses described herein, or training and / or execution of predictive models, including various machine learning models that may be deployed in the cloud or on an acoustic imaging device or remote system for analysis and classification of degree or severity of degradation. Details regarding each of these analyses are described in further detail below.

[0081] Additionally, in some example embodiments, the acoustic imaging device 102 may be communicatively connected to other types of devices, such as external rotational speed sensors and the like. Examples of such external rotational speed sensors, and methods of use thereof, are described in co-pending Patent Cooperation Treaty (PCT) Application No. PCT / US24 / 55065, entitled “Acoustic Imaging System With Rotational Speed Detection”, the disclosure of which is hereby incorporated by reference in its entirety.

[0082] In accordance with example aspects, the acoustic data obtained and stored may be associated with various metadata based on user assignment of labels to captured acoustic data, or based on automatic detection of features associated with the acoustic data. Such user-assigned or automatically detected data may include information about rotational speed, and may also include other information about objects detected in images that correspond to the captured acoustic data. For example, identification of specific mechanical equipment, the specific acoustic imaging device used, a time / date of acoustic data capture, and various other information may be appended as metadata to acoustic data. Use of such data to select particular modes of analysis of acoustic data is described below.

[0083] Figs. 2-3 are schematic front and rear views of an acoustic imaging device 200 on which example aspects of the present disclosure may be implemented. The acoustic imaging device 200 is an example of a physical implementation of the acoustic imaging device 102 of Fig. 1, for example when implemented as a manual handheld unit.

[0084] As seen in Fig. 2, the acoustic imaging device 200 includes a touchscreen display 230 positioned within a housing 202. The touchscreen display may be controlled via manual touch operations on user interface elements, such as those described below. A power button 232a activates the acoustic imaging device 200, and a capture button 232b initiates capture of concurrent image and acoustic data, either in still / instantaneous capture mode or a video / streaming mode.

[0085] On a rear side of the housing 202, an acoustic array 220 is mounted to the housing 202. The acoustic array 220 includes a plurality of acoustic sensors 222, for example miniaturized microphones, which are spaced apart from one another along two dimensions on the acoustic array 220. The plurality of acoustic sensors 222 determine, collectively, a direction and signal strength of an acoustic signal. The acoustic sensors 222 may also be used, in some cases, to determine a distance of the acoustic signal from the acoustic imaging device 200. This can be performed, for example, either alone or in combination with image data obtained by a camera 225. The camera 225 is positioned to be oriented in a direction normal to the plane defined by the plurality of acoustic sensors, such that the camera 225 captures a field of view in a direction from which acoustic signals may be sensed. In the example shown, the camera 225 is positioned in the center of the acoustic array 220; however, in other implementations, the camera may be located in other locations. In the example shown, a speaker / vent system 235 is also provided on the rear side of the housing 202, provides air communication into the housing 202, and allows emission of audible feedback, e.g., from a speaker.

[0086] In use, the acoustic imaging device 200 may be positioned or oriented toward an object of interest, for example such that the acoustic array 220 and the camera 225 are aimed at an object. A user may press a capture button 232b to initiate an acoustic sampling process, for example to capture image data and acoustic signals, and again to terminate capture (e.g., in video or streaming mode). Captured data may be displayed on the display 230, for example for manipulation and viewing.

[0087] Fig. 4 is a schematic plan view of an acoustic imaging sensor array 400 that may be integrated into an acoustic imaging device, according to example aspects of the present disclosure. The acoustic imaging sensor array 400 includes acoustic array 220, including acoustic sensors 222 as described above. The acoustic imaging sensor array 400 further includes a camera 225, also described above. In this example, a connector 402 allows for connection of the acoustic imaging sensor array 400 other electronic systems, for example as may be incorporated into a static mounted sensor system or onto a mobile device, such as a land-based mobile unit, airborne drone, or the like.

[0088] As noted above, an acoustic imaging device 200 or acoustic imaging sensor array 400 may be usable in a variety of applications. For example, acoustic signals may be emitted by pneumatic devices (e.g. detection of air leaks and the like), electrical devices (e.g. detection of sparks, periodic noise generated by electrical signals, and the like) and in some embodiments as described herein mechanical devices (e.g., bearings, rollers, or various other reciprocating or moving systems, such as motors, conveyors, gears and gearboxes, couplings fans, compressors, and mechanical robots.). In the particular case of mechanical devices, detection of acoustic signalsemitted by a particular mechanical device may allow a user to determine the likely operational state that mechanical device. For example, operational mechanical devices may have a particular acoustic signature, while mechanical devices in varying states of operational wear or failure may exhibit other characteristics. For example, failing bearings or rotors may emit acoustic noise at higher frequencies that may be difficult to hear audibly, but which may be detected via the acoustic array 220. Other mechanical devices may exhibit wear or failure modes via other types of acoustic output. Additionally, electrical or pneumatic systems may also indicate the presence of failure by way of acoustic signals at various frequencies. Each of these acoustic characteristics may be difficult for even a skilled and highly-trained individual user of an acoustic imaging system to readily identify using the devices and systems currently available, in which acoustic response at an individual frequency is analyzed.

[0089] Referring to Figs. 5-6, schematic views of alternative embodiments of the acoustic imaging device of Fig. 2 are illustrated. The embodiments of Figs. 5-6 reflect incorporation of different types of rotational speed sensors into a handheld acoustic imaging device, to provide a compact, convenient device capable of capturing rotational speed information alongside image data and acoustic data associated with a particular field of view of the device. That is, Figs. 5-6 illustrate integration of a rotational speed sensor to allow for convenient, improved analysis of mechanical equipment by allowing for automatic detection of rotational speed and subsequent selection of one or more frequencies at which acoustic data should be captured and analyzed.

[0090] In the example shown in Fig. 5, a schematic rear plan view of acoustic imaging device 200 is shown. In this example, the rotational speed sensor 145 is implemented as a stroboscope 245. A stroboscope is a device used to make a cyclically moving object appear to move in slow motion or to appear stationary. The stroboscope 245 operates by emitting short, intense bursts of light at regular intervals, synchronized with the cyclic motion of the mechanical system being observed. The periodic flashing of light creates the appearance that the object is either moving slowly or not moving at all, depending on the settings of the stroboscope. The stroboscope 245 includes, in some embodiments, a laser sensor, and will generate strobing signals across frequency ranges until the laser sensor determines that the strobing rate is synchronized with the mechanical system being observed, thereby detecting rotational speed of such a system. This detected frequency may be used in combination with acoustic and image data for subsequent analysis, as discussed in further detail below.

[0091] In the example shown in Fig. 6, a schematic rear plan view of acoustic imaging device 200 is shown according to a still further embodiment. In this example, the rotational speed sensor 145 is implemented using a laser tachometer 345. A laser tachometer is a measuring device used to measure the rotational speed (RPM) of a rotating object, such as a motor, engine, conveyor belt,or any other machinery with a rotating component. It utilizes a laser beam to determine the RPM by measuring reflected light from a reflective target on the rotating object (e.g., on the mechanical equipment under test).

[0092] The acoustic imaging device 200 of Figs. 5-6 is otherwise similar to that of Fig. 3, and may be used to implement the acoustic imaging device of Fig. 1. As such, the camera 225 (along with analysis circuitry), stroboscope 245, or laser tachometer 345 represent examples of a rotational speed sensor 145 depicted therein, while other aspects of the acoustic imaging device may be common among those embodiments. However, in some examples, other features or functionality may be included within such an acoustic imaging device 102, 200 as described in Figs. 1-5 to implement acoustic imaging functionality without affecting the scope of the present disclosure. For example, in some instances, the acoustic imaging device 200 may lack a rotational speed sensor at all, and may rely on external devices or data to receive rotational speed. For example, a wired or wireless connection via the communication interface 150 to an external rotational speed sensor, or directly to mechanical equipment that exposes its rotational speed, may allow for similar analysis of acoustic data as described in Part II, below.II. Broad Frequency Acoustic Imaging Analysis Processes

[0093] Referring to Figs. 7-30, as noted above, the acoustic imaging systems described herein may be used to perform a variety of types of analysis on acoustic imaging data captured from mechanical devices, across a broad acoustic frequency range (e.g., including one or more of infrasound sub-range, audible sound sub-range, and ultrasound sub-range). The various analyses may be used in different contexts, and may be individually selectable by a user based on the type of mechanical equipment or expected failure that might be recognized. Additionally, multiples of these analyses may be performed, either in near-realtime on a device such as acoustic imaging device 102, or on a system separate from such a device such as remote system 10 or within cloud 70.

[0094] Referring first to Figs. 7-8, a high-level description of methods of performing at least some types of analysis of acoustic data is provided. Fig. 7 illustrates a flowchart of a method 700 of performing broad frequency acoustic imaging to obtain classification of type or severity of mechanical equipment degradation, according to an example embodiment.

[0095] In the example shown, the method 700 includes obtaining acoustic response data across a broad frequency range (step 702). Obtaining the acoustic response data may be performed with an acoustic imaging device, such as those described in Part I, above. The frequency range may encompass an entire acoustic frequency range, or may be a sub-range within the entire acoustic frequency range. For example, in some instances, a sub-range may be manually selectedby a user, or a sub-range may be automatically selected, dependent on particular characteristics of a device under analysis. In the instance that the device under analysis is mechanical equipment, such as rotational mechanical equipment (e.g., bearings, pulleys, and the like), the sub- range of an acoustic range to be analyzed, or captured, may be dependent upon a rotational speed of the mechanical equipment. Such a rotational speed may be automatically detected using a rotational speed sensor as described above, and may indicate one or more frequency ranges within an overall acoustic frequency range.

[0096] In example embodiments, obtaining acoustic response data includes determining a distance from a mechanical device emitting acoustic signals. The distance from the mechanical device may be used to determine an acoustic signal strength, or acoustic power, at the device being tested (in this case, mechanical equipment). This allows for comparison of acoustic signal data captured at different times (e.g., comparing current acoustic response data to baseline acoustic response data captured from a test of mechanical equipment known to be operating properly). Because different acoustic imaging tests may be performed by positioning an acoustic imaging device at a slightly different position relative to the mechanical equipment being tested between testing iterations, a determination of acoustic power at the different locations might result in far different acoustic power calculations due to the exponential decrease in acoustic power as distance from an acoustic source increases. As such, calculating acoustic power at the location of the mechanical equipment being tested allows for reliable comparison between test instances.

[0097] In example embodiments, acoustic data may include information about an acoustic response (e.g., acoustic power levels) at different frequencies, or may be time domain data indicative of acoustic signals captured during a predefined period across an acoustic frequency range. The acoustic data may include additional metadata defining, for example, a location at which the acoustic data was captured by an acoustic imaging device; an image captured alongside the acoustic response levels; one or more labels identifying specific mechanical equipment under analysis, or a type of mechanical equipment under analysis, or the like. In the case of identifying information regarding mechanical equipment, such identifying information may be obtained from a visible tag on the equipment (e.g., a user-readable tag including a serial number and / or model number, or a machine readable tag, such as a QR code or RFID tag).

[0098] In the example shown, the method 700 includes determining a mode of analysis of the captured acoustic data (step 704). Determining a mode of analysis may correspond to determining a mode of operation of an acoustic imaging device performing analysis subsequent to data capture, or may refer to selection of a particular analysis model or type of analysis to be performed. Determining the mode of analysis may be performed after receipt of the acoustic response data, or may be performed prior to capture of the acoustic response data.

[0099] In example embodiments, determining a mode of analysis may include receiving user selection of a particular mode, such as selection of a mechanical equipment analysis mode, or selection of a particular type of analysis (from among the various analyses described below) within a user interface of an acoustic imaging device. Determining a mode of analysis may alternatively include automatically selecting such a mode, for example based on information identifying the mechanical equipment or type of mechanical equipment included with the acoustic data. For example, determining a mode of analysis may include performing object detection to identify a particular object within an image captured by the acoustic imaging device, detection of characteristics of such an object (e.g., rotational speed of mechanical equipment, or other identified characteristics of such a device).

[0100] Still further, determining a mode of analysis may include determining a type of analysis, or selection of which analysis models, are to be performed. For example, this may include identifying whether to analyze severity of degradation of mechanical equipment being analyzed, or to determine a classification of types of degradation experienced by the mechanical equipment. In examples, one or more analysis models selected from those described below may be used, including performing a linear classification analysis; a constrained linear classification analysis, or other non-linear regression or classification analysis; frequency binning and / or area under the curve analyses for acoustic spectral power data; peak analysis; periodic and sub-periodic component analyses; thresholding analyses, or use of one or more machine learning models for classification of acoustic data, such as a decision tree model (e.g. a random forest) or multi-layer perceptron model, or use of a deep learning model using spectrogram data. The selection of the mode of analysis (e.g., the higher level determination of using a mechanical mode of analysis, or selection of specific analyses) may be based on any of the information included in the acoustic data, which may include the acoustic data itself, or any image data or metadata associated with the acoustic signal. For example, mechanical equipment may be detected, and a mode of analysis may be selected, based on any of the following:• Visual obj ect recognition (via data comparison or machine learning or artificial intelligence systems): The acoustic imaging system described above may be adapted to automatically, or semi-automatically (with user confirmation or override) determine the most appropriate analysis model to use for that particular type of component detected.• Acoustic signature recognition (via data comparison or machine learning or artificial intelligence systems): The acoustic imaging system may be adapted to automatically, or semi-automatically (with user confirmation or override) determine the most appropriate analysis model to use for that particular type of component detected.• Optical character recognition: Based in part, on part of a visible image captured as part of the acoustic imaging process at an acoustic imaging device, or in a secondary visible image, where the information “read” therein contains a useful identifier(s) (such as a serial number or model number, or component specification), the acoustic imaging system may be adapted to, in response to identification of the identifier, automatically, or semi- automatically (with user confirmation or override) determine the most appropriate analysis model to use for that particular type of component.• QR Code recognition / reading: In part of the visible image, or in a secondary visible image where the information “read” therein contains some type of useful identified s) (such as a serial number or model number, or component specification), the acoustic imaging system may be adapted to automatically, or semi-automatically (with user confirmation or override) determine the most appropriate analysis model to use for that particular type of component.• RFID tag communication: Based on information read from an RFID tag by an acoustic imaging device or other accompanying device, a useful identifier(s) (such as a serial number or model number, or component specification) for mechanical equipment may be obtained that would allow the system to automatically, or semi-automatically (with user confirmation or override) determine the most appropriate analysis model to use for that particular type of component.Other mode determinations are possible as well.

[0101] In the example shown, the method 700 further includes comparing obtained acoustic response data to prior data (step 706). Specific methods of comparing acoustic response data to prior data may vary depending on the type of analysis to be performed. For example, the prior data may include previously captured acoustic response data associated with the same mechanical equipment, operating in a “known good” state. In such a case, the prior data may represent a baseline of acoustic signals emitted by the mechanical equipment. By comparing this prior data to later acquired acoustic response data, the changes in acoustic response may indicate changes in operational health of the mechanical equipment. The comparison may take the form of obtaining a difference in acoustic response across a broad frequency range, and determining where a current acoustic response differs from a baseline acoustic response across all or a portion of an acoustic frequency range. In other examples, comparing acoustic response data to prior data may include using a machine learning model trained using prior data to analyze and classify the acoustic data to generate a likelihood of the presence of mechanical degradation.

[0102] In the example shown, the method 700 further includes classifying a type or severity of degradation of the mechanical equipment (step 708). Classifying type or severity of degradationof mechanical equipment may vary depending on the type of analysis performed. In example embodiments described herein, analyses may include a constrained linear classification analysis to determine degradation severity, an area under curve analysis across a broad frequency range to determine degradation severity, a frequency and power level analysis to determine degradation type or severity, a power spectral density analysis to determine degradation severity, thresholding analyses to determine degradation severity, peak analysis to determine relative degradation, periodicity analysis to determine degradation type or severity, or various machine learning processes usable to classify acoustic signals in accordance with predefined type or severity detection models. Details regarding each of these are provided below.

[0103] Referring now to Fig. 8, a flowchart of a method 800 of enabling interaction with acoustic response data and classification information to allow iterative improvement of degradation analysis is provided. The method 800 may be performed at a device receiving acoustic response data, such as an acoustic imaging device, a remote device, or a combination of such a device with one or more other devices (such as a cloud or server system). The method 800 may be used to receive user feedback regarding predictions or calculations of type or severity of degradation in mechanical equipment, and may be used to refine subsequent analysis of acoustic response data.

[0104] In the example shown, the method 800 includes receiving acoustic response data, including any predictions or analysis associated with that acoustic response data (step 802). Receiving the acoustic response data may include capture of such data at an acoustic imaging device, as well as performance of analysis on that acoustic response data to obtain a type or severity classification of degradation of mechanical equipment. Similarly, the acoustic response data and analysis data may be received at a remote system, separate from an acoustic imaging device.

[0105] In the example shown, the method includes displaying the acoustic response data on a display of a system, such as an acoustic imaging device (step 804). Examples of such a display are provided below in conjunction with Part III. the method also includes receiving user annotations at a user interface of a device used to display the acoustic response data and analysis (step 806). The user annotations may include an addition of a user entered severity or type classification regarding degradation of mechanical equipment associated with the captured acoustic response data. The user annotations may be newly added annotations, or may be corrections of previous annotations that are automatically or manually added to the acoustic response data and which were received in step 802. The newly added user annotations may be stored in conjunction with the acoustic response data within an acoustic analysis system, such as seen in Fig. 1.

[0106] In the example shown, in some embodiments the method 800 includes updating settings associated with a classifier of degradation severity or type (step 808). Updating settingsmay take a variety of forms, depending on the type of analysis used to classify degradation severity or type. For example, in instances where analysis is performed based on predetermined or adaptive thresholds (e.g., differences in acoustic signal strength across one or more frequency ranges), a historical set of annotated acoustic data may be used to define thresholds or ranges of acoustic signals indicative of particular thresholds of degradation severity, or as indicative of degradation type. In instances where analysis is performed using machine learning systems, the historical annotated acoustic data may be used as training data that updates a classifier model used to classify severity or type of degradation.

[0107] Figs. 9A-9B and 10-13 illustrate a particular sequence of analyses, and details thereof, that may be performed using an acoustic imaging system as described above. The particular sequence described in these figures represents one possible set of steps and analyses that may be performed; alternative types of analysis are described further in subsequent Figs. 14-35.

[0108] In the particular example of Figs. 9-13, a method 900 is illustrated that may be performed on any of the devices described above, for example on the acoustic imaging device, remote system, or cloud or server systems described in conjunction with Figs. 1-6. In this example, the method 900 includes obtaining a spectral power of periodic frequency components of acoustic response data (step 902). The spectral power of periodic frequency components may be assessed across an overall acoustic frequency range (e.g., from about 10 kHz to 90 kHz or above). In some alternative examples, this spectral power may be obtained only within selected portions of the acoustic frequency range, which may be selected based on a determined or entered rotational speed of mechanical equipment being assessed.

[0109] In the example shown, the method 900 includes determining whether there exists baseline acoustic response data for the specific mechanical equipment being tested (step 904). This may include determining that acoustic data associated with the same mechanical equipment has been stored within the acoustic imaging system described herein (e.g., at an acoustic imaging device or remote therefrom). The baseline acoustic response data may be identified by a user, or may be detected based on matching metadata or parameters associated with captured acoustic response datasets (e.g., location, time, orientation, etc. associated with an acoustic imaging device used to capture the acoustic response data, or object recognition associated with images included with the acoustic response data).

[0110] If baseline data exists, the method 900 may proceed with the steps described below in conjunction with Fig. 9B. However, if no baseline data exists, the method proceeds to determine if acoustic response peaks exceed a predetermined threshold (step 906). In some examples, the predetermined threshold may be set at a predetermined value, such as a known scalar value. In some examples, the predetermined value may be a scalar multiple of a noise floor observed in theacoustic response data. In the particular example shown, the scalar multiple corresponds to twice a scalar value a which corresponds to the noise floor.

[0111] If, at step 906, the scalar multiple is not exceeded, it can be determined that no defects are detected (step 908). However, if peak values of the acoustic response data exceed the predetermined value, it is likely that a bearing defect is observed. Accordingly, it is next determined whether particular fault frequencies and the rotational speed of the mechanical equipment are both known (step 910). If both values are not known, a severity assessment is performed (step 912). If both values are known, further diagnostics may be performed to determine one or both of a type of defect and a severity of the defect (step 914). An example analysis to determine severity of a defect is illustrated in Fig. 10; an example analysis to determine a type of defect and / or perform other advanced diagnostics is illustrated and described in conjunction with Fig. H.

[0112] Referring to Fig. 9B, if it is determined that baseline data exists, a determination (at step 950) of whether to perform a defect analysis or a lubrication analysis is performed. If a defect analysis is desired, a similar analysis is performed as is described in Fig 9A. In particular, the acoustic response data is analyzed to determine if peaks in acoustic response data fall outside of a predetermined value, such as the scalar multiple of a noise floor value as described above (step 952). If such a value is not exceeded, no defects are considered to have been detected (step 954). However, if peaks above the predetermined value are detected, if peak values of the acoustic response data exceed the predetermined value, it is likely that a bearing defect is observed. Accordingly, it is next determined whether particular fault frequencies and the rotational speed of the mechanical equipment are both known (step 956). If such fault frequencies are known, some advanced diagnostics may be performed, in accordance with the various analyses described herein (step 958). An example analysis to determine a type of defect and / or perform other advanced diagnostics is illustrated and described in conjunction with Fig. 11.

[0113] If fault frequencies or rotational speed of the mechanical equipment are not known, a trend in an impact score may be determined, and may indicate the existence of a potential defect in mechanical equipment (step 960). In some examples, the impact score, designated Is, is calculated as a sum of the values that exceed the noise floor, multiplied by a weighted mapping of those peak values to respective severity levels. In a particular example, impact score may be calculated using the following equation:In which p(n) corresponds to a list of n values in the frequency response that exceed the noise floor value a, a represents a scalar multiple of the noise floor used to create the severity level thresholds(threshold levels 1 to M), s(n) is a function that maps the peak values in p(n) to respective severity levels, and w(m) corresponds to a weight vector that describes how much weight to apply to each of the M severity levels. In this context the severity levels, s(n) can be characterized as follows:

[0114] In the particular calculation described in step 960, a current or trending impact score is analyzed to determine if it is greater than a particular integer multiple of a baseline impact score. If it is determined to be higher than the scalar multiple, a severity of the impact may be calculated and determined (step 962). If it is not higher than a predetermined scalar multiple of the baseline, no defect is deemed to have been detected (step 964).

[0115] In the example shown, if a lubrication analysis is desired at step 950, a friction score is calculated, and compared against a baseline friction score to determine if such a lubrication issue exists (step 970). In examples, the friction score may correspond to an ultrasonic spectral power (e.g., frequencies over 20 kHz). If the friction score is not greater than the scalar multiple of the baseline friction score, it is determined that no lubrication issue has been detected (step 972). If a current or trending friction score is greater than a particular scalar multiple of a baseline friction score, a potential lubrication issue may be detected (step 972). Examples of such a lubrication analysis are described below in conjunction with Figs. 12-13.

[0116] Referring to Fig. 10, a peak analysis chart 1000 illustrating an example impact score analysis that may be performed in conjunction with aspects of the method of Figs. 9A-9B is illustrated. The analysis as described herein may be performed, for example, at steps 912 and / or 960 described previously. As illustrated in the chart 1000, a set of N peak values is detected in acoustic response data. The acoustic response data may correspond to data captured within an environment including mechanical equipment. The mechanical equipment may emit baseline acoustic signals, generally corresponding to environmental noise or baseline acoustic response, or a combination thereof. As illustrated in the chart 1000, at particular frequencies, an acoustic response may be observed that is above a predefined threshold, typically set above the baseline or noise acoustic signal level. In the example as illustrated, a value a represents a scalar multiple of this baseline acoustic noise. N separate peaks, or spikes, at different frequencies, are observed. As noted above, the peak values may be mapped to respective severity levels s(n), which corresponds to rounded integer values indicative of a multiple of the threshold value a. Separate weights may be applied to each of the identified N values, allowing for customization and emphasis of particular acoustic signatures at specific frequencies. The weighted severity levels for each value identified above the threshold may be summed to obtain an overall reference acoustic signature Is in the manner described above.1

[0117] In some embodiments, the acoustic imaging device or system described herein determines that the background or environmental noise is excessive (e.g., exceeding a threshold), which can trigger a warning to indicate that the acoustic data captured is not reliable. The warning can be displayed via a user interface, such as those shown in Figs. 36-40, where locationcorresponding overlay of acoustic signal data as well as related analysis outcomes may or may not be currently displayed with the warning.

[0118] Fig. 11 is a diagram 1100 illustrating an example diagnostics analysis that may be performed in conjunction with aspects of the method of Figs. 9A-9B. The diagnostics analysis may be performed, for example, at step 914, 958 as previously described.

[0119] In general, the diagnostics process may be performed if rotational speed of the mechanical equipment is known, since at that point particular frequencies may be searched for a specific frequency response. The diagram 1100 represents an example analysis of a mechanical bearing having nine ball bearings, and rotating at 25 Hz. The diagram 1100 illustrates an instance of damage on an outer race of the mechanical bearing. In this illustration, and indicative bearing failure frequency may be calculated based on the shaft speed of the mechanical bearing and number of ball bearings included. For example, a bearing failure frequency (BPFO) may be approximated as 0.4 * shaft speed * number of ball bearings, or 0.4 * 25 Hz * 9 = approximately 90 Hz. Once this value is obtained, and acoustic response may be analyzed for the presence of the 90 Hz fundamental frequency, as well as harmonics in the frequency response to determine evidence of outer race damage. In the diagram 1100, a frequency response at multiples of the bearing frequency indicate the presence of a defect in the outer race.

[0120] Although the diagram 1100 is particular to a specific frequency and rotational speed of mechanical equipment, and is selected to identify an outer race defect, other types of defects may be determined in a similar manner based on a known rotational speed. The diagram 1100 is merely for illustration.

[0121] Figs. 12-13 illustrate diagrams showing analyses usable to detect friction or lubrication issues in mechanical equipment. Such analyses may be performed on acoustic response data using an acoustic imaging system such as described herein, and may be performed as described at step 970 of Fig. 9B.

[0122] In the example illustrated, a friction score Fsmay be calculated or determined based on a root mean squared (RMS) value or ultrasonic spectral power value determined from acoustic data. It is noted that the RMS value alone may be susceptible to background noise, and as such in some examples may be used in combination with other metrics to determine the friction score. If the ultrasonic spectral power is used, as illustrated in diagrams 1200 of Fig. 12 and 1300 of Fig.13, it can be seen that a lack of lubrication, roasts, or other types of corrosion may affect a race ofthe bearing, causing increased friction. This increased friction can be seen as an increase in acoustic signal response across all frequencies. In examples, a percentage increase in the RMS value or in the ultrasonic spectral power range (e.g., in a range of 20 kHz to about 60 kHz) may indicate such a friction or lack of lubrication issue. As specifically seen in the diagram 1200, different types of lubrication issues (decreased bearings, rusted bearings, and chemically etched bearings) are compared to a baseline good bearing, and it is illustrated that in a range of up to 60 kHz, the damaged mechanical equipment (bearings) generally have higher acoustic signal strengths. This is validated in the RMS values illustrated in the diagram 1300 showing comparison of a bearing under different operating conditions, and different corresponding performance states. In the specific example shown in diagram 1300, an acoustic response according to specific conditions relating to a good bearing, an underlubricated (degreased) bearing, rusted (oxidized and corroded) bearing, and exposed and degraded (etched) bearing showing the relative RMS values is illustrated. Other conditions may be compared as part of the analyses described herein.

[0123] Generally, defects of rotating mechanical equipment or components (e.g. rolling element bearing defect) can be divided into two categories: impact defect and friction defect. The impact defect can be a defect that causes rolling elements to come into abrupt contact with a defect such as a contaminate or a small hole. The friction defect can be a defect that uniformly increases the overall fiction between the rolling elements and the races, which is typically associated with a lubrication issue. Various embodiments of the impact score and the friction score can be used to measure these two defects respectively.

[0124] The impact score can be originated from the fact that when there is an impacting defect it will show up in the acoustic data as an amplitude modulated signal. The carrier signal can be from typical ball race contact and the modulating signal can be from the impacts. The goal of computing the impact score is to extract this modulating signal and quantify it. In various embodiments, the impact score computation does not require comparison with prior acoustic data or baseline data.

[0125] Fig. 41 illustrates phases of impact score computation in accordance with some embodiments. As illustrated, some embodiments of impact score computation include 3 main phases: envelope extraction, peak detection, and impact score calculation.

[0126] Envelope extraction is a filtering process to extract the modulating signal. Envelope extraction can include a sub-phase of Kurtosis optimized bandpass filtering. Kurtosis optimized bandpass filtering uses a bandpass filter that has a passband chosen from a pre-defined set of bands that maximizes kurtosis subj ect to a constraint. Such a constraint may be, for example, that a certain amount of the original signal energy is preserved. Due to the nature of mechanical systems having a low acoustic signal to noise ratio, directly attempting to envelope the acoustic data may producepoor results. Typically, the impacting modulation has a higher frequency carrier wave. A direct search process can be used to find a frequency band that maximizes Kurtosis while maintaining a certain percentage of the original signal’s energy. More specifically, the modulating signal from the impacts can naturally have a larger Kurtosis than the total signal, filtering out this part of the signal can also remove undesirable signals from non-mechanical noise sources, and energy constraint is required to avoid extracting trivial “non-existent” frequency components, such as components that are negligible energy-wise (e.g., 0.000001% of the energy) but happen to have very large Kurtosis values.

[0127] The envelope extraction can include a sub-phase of a Hilbert transformation. The envelope of the signal can be computed by taking the absolute value of the analytic signal of the output from the Kurtosis optimized bandpass filtering. This process is a robust way of extracting the modulating component from an amplitude modulated signal.

[0128] Final sub-phase(s) of the envelope extraction can include lowpass filtering followed by mean subtraction, to remove any high frequency components that are not likely from any kind of impacting and to center the signal around 0 to remove any direct current (DC) component from the signal.

[0129] Peak detection constitutes the search for impact signatures in the enveloped signal. This phase can include down sampling, fast Fourier transformation (FFT), and peak extraction subphases. Down sampling works to remove noise and not double count peaks. FFT takes the Fourier transform of the down sampled enveloped signal to obtain its frequency response. Peak extraction identifies components in the frequency response that exceed a threshold.

[0130] Impact score calculation is a phase that quantifies the impact signatures (e.g., peak values) from the peak detection phase into a single value, for example, as illustrated and described in accordance with Fig. 10.

[0131] Impact score can be either calculated as a standalone measurement or can be computed as a difference with respect to a known good reference sample. Various thresholds or ranges can be applied to the impact score to classify the impact into different categories, such as, “Good”, “Moderate”, “Severe”, “Critical’, or the like.

[0132] In various embodiments, the friction score can be calculated as the difference in sound pressure level (SPL) dB in a frequency band, such as [20kHz, 60kHz] (ultrasonic power), between the sample under measurement (e.g., the presently captured acoustic data) and a reference sample (e.g., prior or baseline acoustic data). While friction effects are audible, the computing of friction score can omit the audible components to make the friction score less sensitive to background machine / human noise. Similar to the impact score, various thresholds or ranges can be applied tothe impact score to classify the impact into different categories, such as, “Good”, “Moderate”, “Severe”, “Critical’, or the like.

[0133] In various embodiments, the impact score and the friction score can be used either separately or in combination to evaluate mechanical equipment / component and determine the severity of degradation or defect(s). Fig. 42 illustrates a way to combine the impact and friction scores. Since both scores can be classified or discretized into the same classification categories, a single severity value (e.g., a “Combined Score”, “Total Score”, “Matrixed Score”, “Simplified Score”, or the like) describing their combination can be generated. In the example shown in Fig.42, the combination is achieved by taking the “worst” value from the two. In various other embodiments, the combining method(s) can be chosen based on the relative importance and likely immediate effects of either friction or impact defects. The applicable approach can be variable on the type of equipment in which the rolling element bearing is integrated, as well as the total system dependent on the proper operation of the bearing.

[0134] Referring to Figs. 14-35, additional types of analyses are illustrated that may be performed on acoustic data captured, for example, using an acoustic imaging system such as described herein. The analysis may be performed individually, or in combination, to determine either type of degradation, severity of degradation, or a combination thereof.

[0135] Fig. 14 is a flowchart of a method 1400 of performing constrained linear classification analysis to determine degradation severity of mechanical equipment. The method 900 may be used, for example, to determine degradation severity of mechanical equipment, according to example embodiments, via the acoustic imaging systems as described herein. The method 1400 may represent, for example, a variation on the peak analysis described above in conjunction with Figs 9A and 10.

[0136] In the example shown, the method 900 includes obtaining spectral power of periodic frequency components (step 1402). The spectral power may be obtained by determining peak or sustained power at varying frequency levels of the captured acoustic signal. Particular periodic frequency components of the acoustic response data may be captured, including:xi : a ratio of frequency components marked as periodic to a total number of frequency components, as compared to a baseline ratio of frequency components;X2: a spectral power (dB) of periodic components of acoustic response, as compared to a baseline spectral power;xs: a spectral power in the band of 15-20 kHz (e.g., a “lower” frequency band), as compared to a baseline of spectral power in the same frequency range;X4: a spectral power in the band of 35-40 kHz (e.g., a “higher” frequency band), as compared to a baseline of spectral power in the same frequency range; andxs: an inverse of the rotational speed of mechanical equipment ( 1 / (speed in Hz) ).

[0137] The method 1400 further includes applying weighting factors to the periodic frequency components. The weighting factors may vary depending on the type of degradation historically experienced by the particular type of mechanical equipment, and may be adjusted to be better attuned to determine existence or severity of degradation. Example weighting factors may include:wi: weighting factor for a ratio of frequency components marked as periodic to a total number of frequency components, as compared to a baseline ratio of frequency components;W2: weighting factor for spectral power (dB) of periodic components of acoustic response, as compared to a baseline spectral power;W3: weighting factor for spectral power in the band of 15-20 kHz, as compared to a baseline of spectral power in the same frequency range;W4: weighting factor for spectral power in the band of 35-40 kHz, as compared to a baseline of spectral power in the same frequency range; andws: weighting factor for inverse of the rotational speed of mechanical equipment ( 1 / (speed in Hz)).

[0138] In examples, as described above, a separate weighting factor may be applied to each periodic frequency component, depending on the relative importance of that periodic frequency component to an overall classification of degradation severity. The specific values used in the weighting factors may be determined in a variety of ways. For example, a least squares linear regression may be used, with final weights analyzed to determine whether constraints are met. Alternatively, a brute force search over a range of reasonable values for each weight may be used. Still further, a nonlinear regression model may be used as weight constraints built into a loss function. Generally speaking, regardless of the manner in which weights are determined, constraints may be applied to those wastes such that the waiting must be a positive value, and less than some predetermined upper bound such that no individual weight controls the overall severity analysis.

[0139] In the example shown, a frequency score is then generated (step 1406). The frequency score may be generated as a sum of the weighted periodic frequency components, or some other aggregation or accounting of such components. In accordance with the above variables xi-xs and wi-ws, an integer-value severity score may be calculated as follows:Severity = int (wixi + W2X2 + W3X3 + W4X4 + wsxs)

[0140] The severity score may then be classified according to one or more thresholds (step 1408). The one or more thresholds may be user-defined, and may correspond to differing levels of severity that generally corresponds to a need for service or replacement of the mechanicalequipment. For example, low severity, moderate severity (requiring service), high severity (requiring immediate service), or imminent failure may correspond to example classifications used. In some examples, a scaling of a severity score may be applied as follows:Severity < 3 — > Score = 0 (Good Bearing)4 < Severity < 7 — > Score = 1 (Pre-Failure)8 < Severity < 11 — Score = 2 (Beginning of Failure)12 < Severity < 15 — > Score = 3 (Advanced Failure Condition)16 < Severity —> Score = 4 (Warning of Catastrophic Failure)

[0141] Such severity scores and / or messages may be delivered, for example via presentation on a user interface (as described in Part III, below) or communicated to a remote system to notify a user of required maintenance.

[0142] In further examples, the severity scores used, or in particular the weights applied, may be determined by way of a linear regression approach. In such instances, such best fit weights may be known, experimentally to be inaccurate. For example, a particular weight, such as W4, may be determined to be negative using a linear regression analysis, but experimentally it may be determined that degradation increases in conjunction with acoustic spectral power increasing in the 30-45 kHz range. The fact of a negative value for a weight might be driven by overfitting occurring by way of the linear regression. Accordingly, constraints may be applied on regression values used to determine such weights. For example, weight W4 may be forced to be a non-negative value to ensure positive correlation between spectral power in that range and increased degradation.

[0143] Fig. 15 illustrates a method 1500 of determining degradation severity of mechanical equipment based on an area under curve analysis relating to changes in acoustic signal strength across a broad frequency range. The method 1500 may be used as an alternative method of classifying overall severity compared to the method 1400 described previously, or other methods described herein. In this example, the spectral power of acoustic signals is obtained across a frequency range (step 1502). An overall area under the curve may be calculated for the acoustic signals, where the curve is plotted on a spectral power versus frequency chart (step 1504). The area under the curve may be calculated within an entire frequency range of the captured acoustic signals, or may be within selected frequency ranges. For example, the selected frequency ranges may be chosen to depend, at least in part, on a rotational speed of the mechanical equipment being observed and tested.

[0144] In the example shown, the method 1500 further includes comparing the calculated area under the curve to a baseline area under the curve across the selected frequency ranges (step 1506). The baseline area under the curve may be obtained from previous testing, for example of knowngood mechanical equipment. Based on a magnitude of the overall difference between the baseline and current spectral power, degradation of mechanical equipment may be detected. In particular, the magnitude may correspond to a severity score, and the method 1500 may include classification of degradation according to a particular severity score (step 1508) based on the comparison between current and baseline spectral power.

[0145] An example of comparison of baseline and degraded mechanical equipment using an area under the curve analysis is illustrated in the diagram 1600 of Fig. 16. In that diagram, a spectral power of acoustic signals is plotted relative to frequency for mechanical equipment experiencing different degradation types and severities. In the particular example shown, spectral power of acoustic signals for a known good mechanical bearing, a bearing exposed to nitric acid, a bearing experiencing metal flaking, and a bearing experiencing major metal flaking were each tested. As can be seen, an overall aggregate spectral power for the good bearing is significantly lower than that of the degraded bearing types, such that at least severity of degradation may be determined based on comparison of aggregate area under the respective curves shown.

[0146] Fig. 17 illustrates a further diagram 1700 depicting frequency and power level analysis associated with a plurality of different degradation types and severities, according to example experimentation. The diagram 1700 illustrates various other types of degradation that may be experienced by mechanical equipment such as a mechanical bearing. In the specific examples shown, and acoustic signal response of a good bearing is compared to a degreased bearing, a rusted bearing, a chemically etched bearing, a bearing having further chemical etching over an extended period (overnight), a bearing exposed to nitric acid, and a bearing experiencing metal flaking. As illustrated in the diagram 1700, while the different types of degradation exhibit different power response at different frequencies given the way in which the particular degradation type manifests in the mechanical equipment, and overall spectral power level is increased over at least a relatively broader range of the acoustic signal spectrum (e.g. between approximately 20 kHz and 60 to 80 kHz). As such, such an area under the curve analysis may be amenable to determining degradation severity or presence, but may be less useful in determining degradation type. In varying use cases, different portions of the overall frequency range may be selected that are observed to be highly correlative to acoustic signals that illustrate mechanical degradation.

[0147] Referring now to Figs. 18-21, a further analysis, based on power spectral density, is illustrated which may be used in assessing severity and / or type of degradation of mechanical equipment based on acoustic response data. Fig. 18 illustrates a general method 1800 of performing such analysis, according to an example implementation. As above, the method 1800 may be performed on any of a variety of systems included within an acoustic imaging system suchas described above in conjunction with Figs. 1-6, and may be performed alone or in conjunction with other types of acoustic signal analysis described herein.

[0148] In the example shown, time-domain acoustic response data is received and divided into a plurality of frames (step 1802). A windowing function may be applied to form time segments of acoustic response data (step 1804). The windowing function may involve establishing overlapping windows or non-overlapping windows of time domain acoustic response data. In example embodiments, the windows are of equal size, but the specific size may be selected based on the number of windows desired and computational complexity of performing analysis within each created window. In the example shown, the method 1800 includes calculating a Fourier transform for each frame (step 1806), thereby converting the frame into a frequency domain signal. For each frame, and average amplitude of the acoustic signal is calculated (step 1808). The calculation of an average amplitude for each frame may be performed, for example, according to the process described below in conjunction with Fig. 19, and may involve obtaining a squared magnitude of each frequency point, and averaging each for each frame.

[0149] In the example shown, the calculated amplitudes are then normalized to fall within a standard range (step 1810). Normalization may involve dividing the squared magnitudes for each frame by a sample rate. The various frames may then be analyzed according to a variety of characteristics (step 1812). For example, each frame may be analyzed to determine the extent of frequency excitation, a peak power level, a peak distribution of frequency components, and presence or absence of varying harmonics. In doing so, a particular acoustic signature may be detected that is associated with particular types of degradation. By comparison to baseline acoustic analysis performed in a similar fashion, characterization of either the type or severity of degradation may be performed.

[0150] Fig. 19 is a schematic illustration of a method 1900 of processing a power spectral density. In the example shown, each established frame, after a Fourier transform is performed, may be converted to a power spectrum graph. Each frame may the use a squared Fourier transformer, and each of the squared power spectrum graphs may be averaged relative to each other. The average of the power spectrum graphs across each of the frames may be representative of the overall power level of acoustic signals emitted by mechanical equipment.

[0151] Fig. 20 illustrates an example power spectral density graph 2000 illustrating an acoustic signal strength within a particular frequency range. The power spectral density graph, in the example shown, shows an average acoustic signal strength based on the squared, averaged signals of Fig. 19. In this example, the overall power spectral density is normalized to a single frequency, and the average squared FFT (shown in the power spectral density graph 2000, as an output result of the process of Fig. 19) is divided by the bandwidth to be analyzed. This results ina power spectral density value that may be analyzed. By comparing this power level to a baseline power level, severity of degradation may be determined, for example using various thresholding as previously described.

[0152] Fig. 21 illustrates a diagram 2100 that depicts a methodology of selection of overlapping frequency ranges for detection of degradation, according to an example embodiment. The use of overlapping frequency ranges to form the frames used in power spectral density analysis may provide advantages in some scenarios. For example, the use of overlapping frequency ranges, although not required, may enable coverage of more original acoustic response data, and may generate a larger number of degrees of freedom of the analysis performed.

[0153] Overall, the power spectral density analysis described in connection with Figs. 18-21 may be varied in a number of ways. For example, the frame size may vary across embodiments, with narrower frames resulting in greater computational power requirements because more frames would be required to analyze the same overall frequency range. Similarly, use of overlapping frames may increase accuracy at the expense of additional computational power.

[0154] Furthermore, the use of the above power spectral density analysis overall has the effect of generating a value for an average energy at a single frequency over a period of time. While the value may experience some instability as initial data is gathered, as more data is gathered over time, the overall variance will decrease and accuracy of the average energy value may increase. This analysis may be used to perform a variety of types of subsequent analyses, including analyzing frequency excitation at a defined period within an acoustic signature, or a peak decibel or power level of each frequency in a windowed frame. Furthermore, analysis regarding distribution of peaks in the acoustic signature or harmonic content of the acoustic signature may be analyzed as well to detect various types or severities of degradation.

[0155] Fig. 22 illustrates a method 2200 of determining degradation severity based on a frequency bin analysis, according to an example embodiment. The method 2200 may be performed using the acoustic imaging system described above, and as with other analyses described herein, may be performed either alone or in combination with other analyses to determine severity or type of different types of mechanical equipment degradation.

[0156] In the example shown, the method 2200 includes obtaining the spectral power of acoustic signals across a frequency range (step 2202). Acoustic response data may then be separated into visions, with each bin representing a different frequency range (step 2204). The specific frequency ranges associated with each bin may vary, and may be selected based on past observed acoustic response of mechanical equipment degradation. In some instances, the specific frequency ranges may be selected based on a user-provided, or automatically detected, rotational speed of mechanical equipment being tested.

[0157] The method 2200 further includes determining a power level of the acoustic data within each of the selected frequency bins (step 2206). The power level may be an average power level of the acoustic response data across the frequencies included in the bin. For each bin, it is determined whether the power level is above a particular baseline power level (step 2206). The baseline power level may be a power level determined from known good mechanical equipment. The comparison of a current power level to a baseline power level may be to determine whether the current power level exceeds the baseline power level (e.g., in terms of dB) by a predetermined threshold. The threshold selected may be specific to the bin or the frequency range that is being analyzed, as acoustic signal power differences within different frequency ranges may typically experience different magnitudes of change in response to mechanical equipment degradation.

[0158] In the example shown, after all bins are analyzed, an overall severity score may be determined (step 2210). The overall severity store may be based on individual severity determinations within each bin (e.g., whether the signal is above a baseline by a predetermined threshold that is associated with a bin-specific severity score), aggregated across the bins and optionally normalized to arrive at an overall severity score.

[0159] Referring now to Fig. 23, a particular example of such a bin analysis is illustrated. In this example, a diagram 2300 illustrating power level and frequency analysis is illustrated. As shown, a set of four bins are defined (labeled 1-4, respectively), and four different power level (dB) to frequency plots are illustrated, for mechanical equipment experiencing different levels of degradation. In the example shown, the plots relate to a mechanical bearing with no significant degradation, a bearing with metal flaking, a bearing experiencing significant metal flaking, and a bearing exposed to nitric acid, (labeled plots A-D, respectively). In this instance, to illustrate the analysis performed, we assume that a term Al represents power or dB level of acoustic data for the “A” plot (baseline) in bin 1, A2 represents power or dB level of acoustic data for the “A” blot in bin 2, and so on (through A4), and B1-B4, C1-C4, and D1-D4 are similarly assigned, a comparison across plots may be determined by comparing plots within each bin:If Bl > Al + xl (first threshold, bin 1) = Severity BlIf Cl > Al + yl (second threshold, bin 1) = Severity ClIf DI > Al + zl (third threshold, bin 1) = Severity DIIf B2 > A2 + x2 (first threshold, bin 2) = Severity BlIf C2 > A2 + y2 (second threshold, bin 2) = Severity ClIf D2 > A2 + z2 (third threshold, bin 2) = Severity DI(repeated for each bin of bins 1-4)Then an overall severity score for plot B would be: Severity Bl + Severity B2 + Severity B3 + Severity B4. Similar calculations could be made to obtain an overall severity for plots C and D, from bin-specific severities C1-C4 and D1-D4, respectively.

[0160] Fig, 23 illustrates a further example diagram 2400 of a frequency and power level analysis within a plurality of frequency bins to determine relative degradation of mechanical equipment, according to example experimentation. In this example three bins are selected, and plots are illustrated for seven different degradation types: a baseline (good) hearing, a degreased bearing, a rusted bearing, a chemically etched bearing, a prolonged chemically etched bearing (overnight), a bearing exposed to nitric acid, and a bearing experiencing metal flaking, respectively. As can be seen in the diagram 2400, selection of three frequency bins in the range of 15-20 kHz, in an area around 30 kHz, and approximately 45-50 kHz allows differences between the power levels of acoustic signals to be detected across different frequency ranges. In this way, an acoustic signal that has a consistently higher power level as compared to a baseline across multiple frequency bins may indicate a higher severity degradation, while degradation may be detected in any one of the selected frequency bins. In this way, a relatively broad set of frequencies may be analyzed against baseline acoustic data, while severity is affected by the extent to which acoustic signal levels remain consistently above the baseline level in selected frequency ranges.

[0161] Referring now to Figs 25-27, additional types of analyses are described that may be performed on acoustic response data, either alone or in combination with the above analyses. In particular, a thresholding analysis and a peak signal analysis are described.

[0162] Figs. 25-26 in particular illustrate a thresholding analysis that may be used to determine severity of degradation. As illustrated in Fig. 25, a method 2500 of performing such a thresholding analysis includes obtaining acoustic response data across a broad frequency range (step 2502). The acoustic response data may be analyzed at discrete frequency levels (step 2504). The selected, discrete frequency levels may be individual frequency levels spread across a broad frequency spectrum, and may represent either the acoustic signal power at that particular frequency or within a range of frequencies around that frequency. In example implementations, the discrete frequency levels may be selected such that a discrete power is calculated at 5 kHz increments (e.g., as seen in Fig. 26). In the example as shown, a power level of the acoustic response data each discrete frequency may be compared to one or more thresholds indicative of different severities of degradation. In some examples, different thresholds may be selected at different frequency levels, for example based on historical observed degradation severity and acoustic response. Based on the comparisons to thresholds, the acoustic response of mechanical equipment may be classified according to the selected one or more degradation levels associated with the threshold (step 2508).

[0163] In some example embodiments, as known degradation is correlated with additional acoustic response data, the thresholds used at each frequency may be adjusted (at step 2510) to ensure that in subsequent analysis, the severity thresholds are selected appropriately to accurately classify degradation severity.

[0164] Fig. 26 illustrates a diagram 2600 that shows an example of the thresholding analysis described above in conjunction with Fig. 25. In this example, a set of three threshold levels, depicted by the horizontal lines at approximately 38 dB, 45 dB, and 50-53 dB are illustrated, and power levels for mechanical equipment experiencing different types of degradation are tested. In particular, a baseline (good) hearing, a degreased bearing, a rusted bearing, a chemically etched bearing, a prolonged chemically etched bearing (overnight), a bearing exposed to nitric acid, and a bearing experiencing metal flaking, respectively, are tested. In this example, rather than obtaining an average power within a frequency range, power at a number of different individual frequencies may be used (in this instance, at 5 kHz increments). It can be seen by the number of signals that exceed the thresholds where relative degradation may be occurring, as those frequencies where power is above the threshold are more common for mechanical equipment experiencing greater degradation.

[0165] Fig. 27 illustrates a diagram 2700 of a peak analysis that may be performed. In this instance, peaks in acoustic signal strength may be identified, and may be compared against baseline (known good equipment) data to determine the extent to which such peaks are indicative of degradation. In particular, a baseline (good) hearing, a degreased bearing, a rusted bearing, a chemically etched bearing, a prolonged chemically etched bearing (overnight), a bearing exposed to nitric acid, and a bearing experiencing metal flaking, respectively, are tested. In this example, three local peaks are detected in the acoustic signal strength associated with the bearing experiencing metal flaking, while other degraded mechanical equipment experiences overall higher acoustic signal levels, but without significant changes in periodic components. Accordingly, by combining such peak analysis with other types of analyses described herein, further information indicative of a type of degradation may be obtained, which may lead to predictions regarding both type and severity of degradation of mechanical equipment when analyzing acoustic response data relative to baseline data across a broad range of frequencies.

[0166] Referring to Figs. 28-31, details regarding performing analysis of periodicity of spectral data are provided, according to further embodiments. The periodicity analysis described herein may provide further information regarding type or severity of degradation, and may be used by an acoustic imaging system such as the one described above in conjunction with other types of analyses described herein.

[0167] In particular, Fig. 28 illustrates a flowchart of a method 2800 of analyzing periodicity of spectral data to determine severity of degradation, according to an example embodiment. As illustrated in Fig. 28, the method 2800 of performing such a thresholding analysis includes obtaining acoustic response data across a broad frequency range (step 2802). The method 2800 may also include analyzing both periodic and aperiodic signals in the acoustic response data, as well as in baseline data (step 2804). Analysis of this data may include obtaining characteristics of the acoustic response, such as a periodicity ratio and aperiodic components of the signal, as well as baseline noise and other signal characteristics.

[0168] In the example shown, the method 2800 includes comparing aperiodic and periodic components between the acoustic response data and baseline data (step 2806). In particular, a ratio of a periodic components of the acoustic response data to the baseline data may be analyzed, as well as a ratio between the periodic components of the acoustic response data to the baseline data. Based on this comparison, and optionally considering whether any increases in a periodic or periodic components are above a particular, defined threshold, the acoustic response data may be classified according to one or more degradation levels or types (step 2808).

[0169] An example of such periodicity analysis is illustrated in Figs. 29-30. Fig. 29 depicts a chart 2900 that illustrates acoustic response, in a time domain, of a known good mechanical bearing. In particular, baseline 2902 is captured, as well as current acoustic signals for the mechanical equipment. In this example, the bearing has an outer race, and an inner race, and acoustic response of the outer race and in a race are depicted separately as current time domain signals 2904, 2906. A combined signal 2908 represents current acoustic response of the mechanical bearing, which may be compared to baseline 2902.

[0170] Figs. 30A and 30B illustrate the periodic and aperiodic components of acoustic data obtained from mechanical equipment illustrating different types of degradation, according to example experimentation. Fig. 30A illustrates charts 3000, 3010 that illustrate aperiodic signal response of a chemically etched bearing and a rusted bearing. As can be seen, the general envelope within which the aperiodic components reside for both the rusted and chemically etched bearings may be analyzed, and compared against a similar characteristic of baseline data. Similarly, Fig.30B illustrates charts 3050, 3060 that illustrate periodic signal response of the chemically etched and rusted bearing. As can be seen, the chemically etched bearing illustrates a response having generally higher frequency periodicity and stronger periodicity, while the rusted bearing has a more muted, slower frequency periodicity. Based on the amplitude of the signals, and periodicity, one or both of degradation type or severity may be obtained using the overall combination of periodic and a periodic component analysis of the acoustic signal response.

[0171] Referring to Fig. 31, it is noted that in some cases a sub audio periodic acoustic frequency analysis might be preferable in detecting degradation of mechanical equipment. In the example shown, charts 3100, 3110 illustrate a time domain envelope and a frequency response of a good bearing, while charts 3120, 3130 illustrate a time domain envelope and a frequency response of a bearing experiencing degradation due to metal flaking. In this illustration, it can be seen that a relatively significant sub- audio frequency response at 15 Hz occurs and is indicative of the metal flaking type of degradation. It is noted that this is consistent with the peak analysis described above, which determined the presence of periodic acoustic signal peaks in the acoustic response data associated with metal flaking.

[0172] Referring to Figs. 32-35, it is noted that in addition to the deterministic or numerical analyses that are described herein, additional predictive analytics may be used to identify type or severity of degradation, for example using various machine learning and / or artificial intelligence techniques. A description of a general framework, as well as example analyses that may be performed alone or in combination with the above analysis techniques, is provided.

[0173] In particular, Fig. 32 illustrates a block diagram of an example machine learning framework 3200 in which acoustic data may be analyzed to determine a severity and / or classification of degradation of mechanical equipment, according to an example implementation. The machine learning framework generally may be implemented within an acoustic imaging system such as described above. In some implementations, portions of the framework may be located on an acoustic imaging device, such as acoustic imaging device 102, 200. In further implementations, all of the machine learning frameworks, including both training and execution, may be performed on such an acoustic imaging device. In alternative examples, one or both of the training more analysis components of a machine learning framework may be located elsewhere within an acoustic imaging system, such as at a remote device or within a cloud or server system.

[0174] In the example framework 3200 as shown, an acoustic imaging device 102, 200 may include an acoustic imaging component 3210, which contains acoustic response data from mechanical equipment 3250. Such acoustic imaging may be accomplished as described above with respect to Figs. 1-6. The acoustic response data may be used for both predicting type or severity of degradation using trained model 3220. Trained model 3220 may return one or both of severity or type of degradation based on output from trained model 3220. The output from the trained model may take a variety of forms. For example, the output of the trained model may include a classifier output identifying a type or severity of degradation, as well as a confidence level associated with that prediction. The acoustic imaging device 102, 200 may then display, on a user interface 3230, the acoustic imaging data as well as the prediction from trained model 3220. For example, an image of the mechanical equipment may be depicted with an overlay indicating concentration of acousticsignals received. Further messages may be overlaid on a user interface indicating type or severity of degradation represented by the overlay of acoustic signals. An example of such a user interface is depicted in Fig. 40.

[0175] In some instances, a user of the acoustic imaging device 102, 200 may provide user inputs at the user interface 3230. The user inputs may take a variety of forms. In some instances, user inputs may include adding or adjusting labels indicative of type or severity of degradation based on user observation of the mechanical equipment. Such user inputs may result in additional labels or metadata being added to the acoustic response data, which may be stored at the device.

[0176] In the example shown, a model training environment 3240 may also receive the acoustic response data and any user added annotations. This acoustic response data and user input may be used to train, or retrain one or more models that may be used for generating type or severity predictions. The retrained models may be provided back to the acoustic imaging device 102, 204, e.g., replacing trained model 3220. As such, as the acoustic imaging device obtains additional acoustic response signals and a user continues to provide feedback regarding degradation corresponding to those acoustic response signals, the acoustic imaging system will continue to improve and evolve in terms of its accuracy in detecting type or severity of degradation, as well as ability to detect different types of degradation.

[0177] In the example shown, the model training environment 3240 is located remote from the acoustic imaging device 102, 200. This may be preferred in some instances where a centralized location for retraining of models is desired. In such instances, multiple acoustic imaging devices may provide training data to the model training environment 3240, thereby increasing the amount of available training data and overall accuracy of prediction performed by each of the trained models 3220 associated with each of the contributing acoustic imaging devices. Of course, in offline environments, a model training environment 3240 may be employed on an acoustic imaging device, and may be configured to execute periodically (e.g., daily, weekly or the like) to gradually improve predictive capabilities of that acoustic imaging device.

[0178] Figs. 33-34 illustrate specific predictive models that may be used within the framework 3200 of Fig. 32. In particular, Fig, 33 illustrates an example decision tree 3300 generated from a classifier model. The decision tree 3300 may be used to predict a classification of the acoustic data according to severity and / or classification of degradation of mechanical equipment. In the example shown, the classifier model is provided some pre-analyzed data describing the acoustic response of mechanical equipment. For example, a calculated periodicity and acoustic signal strength at varying frequencies may be used to perform classifications of samples. Taking a few possible paths through the decision tree, acoustic response data having a periodicity less than 4.352, a comparative power level at 25 kHz greater than 3.665 and also greater than 6.884 mayindicate degradation, while a signal having similar characteristics but a power level at 25 kHz less than 6.884 and a comparative power level at 40 kHz that is less than -3.059 may be indicative of no degradation. Various other options and outcomes are possible as well, and different classifications may be provided.

[0179] Fig. 34 illustrates an example multi-layer perceptron network 3400 that may be used within the machine learning framework 3200 of Fig. 32. In this instance, the multi-layer perceptron network 3400 may include an input layer that receives an input vector x (depicted as xi to xnx, in which each variable in the input vector represents a variable such as those obtained from the constrained linear classification described above in conjunction with Figs. 9-11 and 14. An output vector Y, consisting of yi to ynyincludes the probabilities that the input vector corresponds to one of the values useable as a final severity score. In this example, such a perceptron arrangement may be constructed using multiple layers of interconnected artificial neurons, with the layers including one or more hidden layers between the input layer and output layer. The multi-layer perceptron network 3400 is trained using gradient descent and backpropagation to adjust model weights in order to minimize the error between the predicted output and the actual target values.

[0180] To train such a multi-layer perceptron network, a set of input data is provided to the network at the input layer, and that data is transformed as it passes through each hidden layer by applying weighted sums and activation functions. The output layer produces the network prediction, which is then used in a loss calculation to determine a difference between the predicted output and actual target output. The error generated by the loss calculation is then backpropagated through the network by adjusting the weights at one or more nodes within the network to minimize loss. This may include computing gradients of the loss with respect to weights and biases of the perceptron network, and updating the weights and biases at a predetermined learning rate. Such a process may be performed iteratively until training data is exhausted or a particular training criteria (e.g., an observed accuracy level) is reached.

[0181] Although in the above description, a gradient loss function with back propagation is used, in alternative examples, an Adam optimizer may be used for training. The selection of which type of optimization is largely a matter of design choice.

[0182] Referring now to Fig. 35, further details regarding use of a deep learning model, in this case a convolutional neural network, are provided. In particular, Fig. 35 illustrates a methodology 3500 for creating and using a spectrogram representation of the acoustic response data in a convolutional neural network for prediction of mechanical equipment degradation.

[0183] In the example shown, the methodology 3500 includes obtaining acoustic data associated with mechanical equipment, and segmenting the acoustic data into discrete time segments (step 3002). The methodology 3500 also includes performing a Fourier transform (e.g.,a Fast Fourier Transform, or FFT) on each segmented acoustic data set (e.g., per time segment) (step 3004). This will create a frequency spectrum representation for each time segment. In the example shown, the methodology 3500 further includes forming a spectrogram from the segment Fourier transforms (step 3006). The spectrogram represents a frequency response over time, as illustrated.

[0184] In the example shown, the method includes providing the spectrogram to a model to generate classifying outputs (step 3008). In the example as illustrated, the model can be a convolutional neural network (CNN), and the classifying outputs may be an identification or prediction of severity or type of defect seen in the spectrogram. In this example, the convolutional neural network, or other model, is trained using prior experimental data associated with both the known good and labeled, degraded mechanical equipment.

[0185] Referring to Figs. 7-35 generally, it is noted that the varying types of analyses that may be performed, and the differing type or severity detection that is possible with each, suggests that in example implementations various combinations of analyses may be used to provide automated classification of acoustic response data. The specific selection may be based on the type of mechanical equipment, the rotational speed of the mechanical equipment, historical predictive effectiveness of a given analysis type, computational complexity of a given analysis type, among other factors.III. Acoustic Imaging Systems and Interfaces - Methods of Use and Operation

[0186] Referring now to Figs. 36-40, methods of use and operation of acoustic imaging systems and the user interfaces that may be generated and interacted with are described. The user interfaces of Figs. 36-40 may be generated and displayed, for example, using an acoustic imaging device, a system that includes such an acoustic imaging device, or a computing device included within such a system. Examples of such devices and systems are provided above in conjunction with Figs. 1-6.

[0187] Fig. 36 is a first example user interface 3600 illustrating selection of a rotational speed input mechanism for use in acoustic imaging of mechanical equipment, according to example embodiments.

[0188] In the example shown, the user interface 3600 includes a field of view region 3601 and a frequency selection region 3604. The acoustic imaging user interface 3600 may be presentable on a display, such as displays 30, 130, 230. The field of view region 3601 displays image data associated with an image captured by a camera system of an acoustic imaging device, as well as an overlay of acoustic signals within a frequency range selected by a user within the frequency selection region 3604. In particular, the field of view region 3601 includes a focus indicator 3606.The focus indicator 3606 illustrates an area in which an acoustic imaging device is configured to focus image data and in which acoustic sensor data is likely most accurate (e.g., by being centered within the acoustic sensor array). The field of view region 3601 can reflect, in real-time, different areas of, perspectives against, or distances from the mechanical equipment under evaluation, responsive to repositioning or reorientation of the acoustic imaging device (e.g., by a user) to have a different focus of the focus indicator 3606.

[0189] In a particular implementation, an operation selection menu 3602 is displayed as a menu bar, and presents a plurality of operating characteristics as options for selection. In the example shown, the operation selection menu 3602 allows for control of a mode (e.g., a mechanical mode in this case), memory management of on-device memory, acoustic signal settings, as well as annotation settings including a color palette usable for the acoustic overlay, markers that may be applied as part of the acoustic overlay and the like.

[0190] In the example shown, a menu 3622 may be presented in response to selection of the acoustic imaging option within the operation selection menu 3602 described previously. In this example, the menu 3622 presents a plurality of capture modes. A capture mode that is a subject of the present disclosure corresponds to a mechanical device acoustic response capture mode, abbreviated as a “MecQ” mode. Other modes may include a mechanical degradation detection and analysis mode, an electrical discharge detection and analysis mode, and an air leak detection and analysis mode.

[0191] Fig. 37 illustrates a mode settings selection user interface 3700 displayable on a device such as described herein. The mode settings selection user interface 3700 includes and acoustic settings menu sub screen 3702. The acoustic settings menu sub screen 3702 presents a plurality of acoustic settings selectable by a user in response to selection of the mechanical device acoustic response capture mode in the menu 3622 of Fig. 36. In this example, a user may select to manually or automatically set a minimum decibel level and maximum decibel level considered within the graphical display, as well as enable one or more known acoustic profiles. Additionally, the user may enable detection of high frequency events and multiple acoustic sources.

[0192] In addition, in the example shown, the acoustic settings menu sub screen 3702 allows a user to select a particular mode of operation within the mechanical device acoustic response capture mode. The modes of operation may include a fixed 30 kHz mode, a user selectable frequency mode (between 2 and 100 kHz), and a mixed multi-mode option, in which a set of discrete, preselected frequencies and / or frequency sub-ranges (either of which referred to herein as discrete frequencies for simplicity) are individually enabled or disabled as part of the overlaid display of acoustic response.

[0193] In the example of the mode settings selection user interface 3700 provided, with the mixed multi-mode option selected a plurality of selectable, discrete frequency indicators 3710a-e (referred to collectively as frequency indicators 710) are displayed. The selectable frequency indicators 3710a-e are each associated with a different, discrete frequency, or different, discrete range of frequencies different from one another, and which is preselected in accordance with the mechanical device acoustic response capture mode. In particular, in the example shown, discrete frequencies of 15 kHz, 20 kHz, 30 kHz, 40 kHz, and 60 kHz are presented in association with frequency indicators 710a-e, respectively. In alternative implementations, other numbers of frequency indicators can be displayed, and other frequencies used either by default or as preselected by a user. Further examples of use of such frequency indicators, and additional examples of the mode options, are provided in further detail below. Details regarding selection and display of particular frequencies and related acoustic responses, and other methods of manipulation of a user interface associated with an acoustic imaging system are provided in U.S. Provisional Patent Application No. 63 / 506,563, filed on June 6, 2023, and entitled “User Interface for Acoustic Imaging Systems”, the disclosure of which is hereby incorporated by reference in its entirety.

[0194] Referring to Fig. 38, a further user interface 3800 is depicted that shows initial capture of acoustic response data at an acoustic imaging device. In the example shown, the field of view region 3601 depicts a scene including a mechanical device 3802, such as a conveyor belt having a plurality of rollers 3804 each associated with bearing is allowing for rotation of the roller. In some examples, a user may select, using a mani pulable frequency selection slider 3610, a range of frequencies along a frequency range. As noted in the comparison between Figs. 37 and 38, one or more such frequency selection features (e.g., including the manipulable frequency selection slider 3610 and / or user manipulable frequency indicators 710a-e). The range of frequencies identified by the slider 3610 correspond to the frequencies for which an overlay within the field of view region 3601 is generated. In the example as shown, overlay 3611 depicts acoustic response at a plurality of different threshold levels (in the example shown, at three different threshold levels of different colors or shades, indicating comparative intensity of acoustic signal response). In some embodiments, the overlay 3611 depicts acoustic response as a color-coded or shade-coded heat map. In some cases, the heat map reflects a continuous color-space or shade-space corresponding to the range of frequencies identified by the slider 3610, without threshold delineations inside the overlay.

[0195] Fig. 39 is an example user interface 3900 depicting receipt of user-entered degradation severity, in accordance with example embodiments. In this example, a severity selection menu 3902 may be presented, and allows a user to manually enter a severity score based on manualinspection of the mechanical equipment under test. The user may select a particular severity score (e.g., low, medium, high, severe, etc.) to be assigned to the captured acoustic response data; this selected score may be a new score, or may replace a previous predicted, calculated, or user-entered score. It is recognized that other types of menus may be presented as well, in further embodiments. For example, a classification menu may be presented that allows a user to select and enter a type of degradation observed by the user. Entry of a severity score, or a classification of degradation type, may represent example user inputs that may be used for purposes of retraining models as described in the framework 3200 of Fig. 32, or may be used in adjusting one or more thresholds used in the deterministic analyses described in connection with Figs. 9-31.

[0196] Fig. 40 is an example user interface 4000 depicting degradation severity and type, in accordance with example embodiments. The degradation severity and type may be presented in an overlay message 4002, and may correspond to a predicted degradation severity and type, or may be presented in response to user entry. Different degradation severity or type can be reflected by displaying the focus indicator 3606 in a distinct color or distinct shade corresponding to the particular severity or type. It is noted that one or both of degradation severity and type may be presented, while another may be excluded. For example, where it may be difficult to classify degradation type, but the presence of degradation is apparent, a degradation severity may be displayed without display of the degradation type. Other variants on such a display are possible as well.

[0197] Referring generally to Figs. 1-40, functionality of computing devices described herein may be implemented in computing logic embodied in hardware or software instructions, which can be written in a programming language, such as C, C++, COBOL, JAVA™, PHP, Perl, HTML, CSS, JavaScript, PythonScript, VBScript, ASPX, Microsoft .NET™ languages such as C#, or the like. Computing logic may be compiled into executable programs or written in interpreted programming languages. Generally, functionality described herein can be implemented as logic modules that can be duplicated to provide greater processing capability, merged with other modules, or divided into sub-modules. The computing logic can be stored in any type of computer-readable medium (e.g., a non-transitory medium such as a memory or storage medium) or computer storage device and be stored on and executed by one or more general-purpose or specialpurpose processors, thus creating a special-purpose computing device configured to provide functionality described herein.

[0198] Many alternatives to the systems and devices described herein are possible. For example, individual modules or subsystems can be separated into additional modules or subsystems or combined into fewer modules or subsystems. As another example, modules or subsystems can be omitted or supplemented with other modules or subsystems. As anotherexample, functions that are indicated as being performed by a particular device, module, or subsystem may instead be performed by one or more other devices, modules, or subsystems. Although some examples in the present disclosure include descriptions of devices comprising specific hardware components in specific arrangements, techniques and tools described herein can be modified to accommodate different hardware components, combinations, or arrangements. Further, although some examples in the present disclosure include descriptions of specific usage scenarios, techniques and tools described herein can be modified to accommodate different usage scenarios. Functionality that is described as being implemented in software can instead be implemented in hardware, or vice versa.

[0199] At least some embodiments of the presently disclosed technology are represented in accordance with the following examples and features, alone or in any combination.

[0200] An example acoustic imaging system may include an acoustic sensor array; optionally a camera system; optionally a display; one or more processors communicatively connected to the acoustic sensor array, the optional camera system, and the optional display; and a memory communicatively connected to the one or more processors. The memory stores instructions which, when executed by the one or more processors, cause the acoustic imaging system to perform actions. Such actions may include capturing, via the camera system, a field of view including a view of at least one rotating mechanical element; capturing, via the acoustic sensor array, acoustic signals emitted by the at least one rotating mechanical element; converting the acoustic signals into acoustic response data in digital form; analyzing the acoustic response data across a plurality of frequencies by at least extracting and quantifying a modulating signal as an impact measure of the at least one rotating mechanical element coming into abrupt contact with one or more defects; determining a degradation severity of the at least one rotating mechanical element based at least partly on the impact measure; and optionally presenting, via the display, the degradation severity in accordance with the field of view captured via the camera system.

[0201] The plurality of frequencies may optionally include at least one of an infrasound frequency, an audible sound frequency, or an ultrasound frequency. The actions optionally comprise presenting, via the display, an overlay depicting an acoustic signal strength of the acoustic signals emitted by the at least one rotating mechanical element in accordance with the field of view. The extracting and quantifying the modulating signal as the impact measure is optionally performed without requiring comparison with prior acoustic response data or baseline acoustic response data. The extracting of the modulating signal optionally is based at least partly on Kurtosis optimized bandpass filtering and Hilbert transformation. The determining the degradation severity of the at least one rotating mechanical element is optionally further based on a friction measure indicating fiction between the at least one rotating mechanical element and oneor more other elements. Optionally, the friction measure is determined based at least partly on a comparison in sound pressure level (SPL) between the acoustic response data and at least one of prior acoustic response data or baseline acoustic response data.

[0202] As another example, a non-transitory computer-readable medium stores contents which, when executed by one or more processors, cause actions to be performed. Such actions may include causing capturing of acoustic signals emitted by at least one rotating mechanical element across at least a subset of the plurality of frequencies; causing conversion of the acoustic signals into acoustic response data in digital form; analyzing the acoustic response data by at least extracting and quantifying a modulating signal as an impact measure of the at least one rotating mechanical element coming into abrupt contact with one or more defects; determining a degradation severity of the at least one rotating mechanical element based at least partly on the impact measure; and causing presentation of the degradation severity in accordance with a field of view including a view of the at least one rotating mechanical element.

[0203] The plurality of frequencies optionally includes at least one of an infrasound frequency, an audible sound frequency, or an ultrasound frequency. The actions optionally comprise causing presentation of an overlay depicting an acoustic signal strength of the acoustic signals emitted by the at least one rotating mechanical element in accordance with the field of view. The extracting of the modulating signal is optionally based at least partly on Kurtosis optimized bandpass filtering and Hilbert transformation. The determining the degradation severity of the at least one rotating mechanical element is optionally further based on a friction measure indicating fiction between the at least one rotating mechanical element and one or more other elements. The friction measure optionally is determined based at least partially on a comparison in sound pressure level (SPL) between the acoustic response data and at least one of prior acoustic response data or baseline acoustic response data.

[0204] An example computer-implemented method may include causing capturing of acoustic signals emitted by at least one rotating mechanical element across a plurality of frequencies; causing conversion of the acoustic signals into acoustic response data in digital form; analyzing the acoustic response data by at least extracting and quantifying a modulating signal as an impact measure of the at least one rotating mechanical element coming into abrupt contact with one or more defects; determining a degradation severity of the at least one rotating mechanical element based at least partly on the impact measure; and causing presentation of the degradation severity in accordance with a field of view including a view of the at least one rotating mechanical element.

[0205] As with previous example, the plurality of frequencies optionally includes at least one of an infrasound frequency, an audible sound frequency, or an ultrasound frequency. The method optionally includes causing presentation of an overlay depicting an acoustic signal strength of theacoustic signals emitted by the at least one rotating mechanical element in accordance with the field of view. The extracting and quantifying of the modulating signal as the impact measure optionally is performed without requiring comparison with prior acoustic response data or baseline acoustic response data. The extracting of the modulating signal optionally is based at least partly on Kurtosis optimized bandpass filtering and Hilbert transformation. The determining of the degradation severity of the at least one rotating mechanical element optionally is further based on a friction measure indicating fiction between the at least one rotating mechanical element and one or more other elements. The friction measure optionally is determined based at least partially on a comparison in sound pressure level (SPL) between the acoustic response data and at least one of prior acoustic response data or baseline acoustic response data.

[0206] Another example acoustic imaging system may include an acoustic sensor array; optionally a camera system; optionally a display; a processing system communicatively connected to the acoustic sensor array, the optional camera system, and the optional display; and a memory communicatively connected to the processing system, the memory storing instructions which, when executed by the processing system, cause the processing system to: capture acoustic response data regarding acoustic signals emitted by mechanical equipment across a plurality of frequencies; compare the acoustic response data to baseline acoustic response data; and based on the comparison, analyze and classify the acoustic response data of the mechanical equipment as to at least one of degradation severity or degradation type.

[0207] Classifying the acoustic response data of the mechanical equipment as to degradation severity optionally includes generating a severity score based on the comparison. The severity score optionally is generated using a classification model including a plurality of weighted features. The classification model optionally comprises a linear classification model, optionally wherein weights associated with each of the plurality of weighted features are determined using a linear regression across a historical dataset of acoustic response data. The classification model optionally comprises a non-linear model implementing one or more constraints on the plurality of weighted features, the one or more constraints optionally based on observations regarding known correspondence between mechanical degradation and acoustic response in a historical dataset of acoustic response data. The plurality of weighted features optionally includes two or more features selected from the group consisting of a comparison to a baseline ratio of frequency components marked as periodic to overall frequency components; a comparison to a baseline of a spectral power of periodic components of the acoustic response data detected across a broad acoustic frequency spectrum; a comparison to a baseline of spectral power in a particular lower end frequency band; a comparison to a baseline of spectral power in a particular higher end frequency band; and an inverse of a detected mechanical component rotation speed. The plurality of weightedfeatures optionally are each assigned a corresponding weight, wherein each corresponding weight is determined using a least squares linear regression over a dataset representing acoustic response data for a plurality of mechanical systems at varying levels of degradation severity. The indication of degradation severity optionally comprises a severity classification from among a plurality of severity classifications, optionally wherein determining the indication of degradation severity includes comparing the severity score against one or more severity threshold values. Classifying a degradation of the mechanical equipment as to degradation severity optionally includes classifying the comparison of the acoustic response data to the baseline acoustic response data into a degradation classification from among a plurality of degradation classifications using a decision tree model.

[0208] An example acoustic imaging system may include generating a spectrogram representative of the acoustic response data, optionally wherein classifying degradation of the mechanical equipment as to degradation severity includes classifying the spectrogram at a deep learning model. The deep learning model optionally comprises a neural network model trained using a plurality of spectrograms representing acoustic response information indicative of known degradation classifications.

[0209] Optionally, the acoustic response data is captured at a first location relative to the mechanical equipment and the baseline acoustic response data is captured at a second location relative to the mechanical equipment, and optionally the acoustic imaging system is configured to: determine a first distance from the mechanical equipment at the first location; determine a second distance from the mechanical equipment at the second location; and calculate, based on acoustic attenuation, acoustic strength of the acoustic response data and the baseline acoustic response data at the mechanical equipment.

[0210] Another example acoustic imaging system may include an acoustic imaging device, optionally wherein the acoustic imaging device includes the processor, the memory, the acoustic sensor array, the optional camera system, and the optional display. The acoustic imaging device optionally comprises a user interface including the display, and wherein the instructions, when executed, cause the acoustic imaging system to: display, on the display, an image depicting mechanical equipment including an overlay depicting an acoustic signal strength of acoustic signals emitted by the mechanical equipment at one or more tested acoustic frequencies; and receive, at the user interface, one or more user annotations of known degradation severity associated with the mechanical equipment. The one or more user annotations and acoustic data representative of the acoustic signals optionally are provided as training data to update a model used in determining the classification of degradation severity. The model optionally comprises at least one of a regression model or a classifier model. The one or more user annotations and theacoustic data optionally are provided to a remote device to be used as training data to update the model, optionally wherein the acoustic imaging device is configured to receive an updated model from the remote device.

[0211] Another example acoustic imaging system may include an acoustic sensor array; optionally a camera system; optionally a display; a processing system communicatively connected to the acoustic sensor array, the camera system, and the optional display; and a memory communicatively connected to the processing system, the memory storing instructions which, when executed by the processing system, cause the processing system to: capture acoustic response data representative of acoustic signals emitted by a mechanical equipment; based on the acoustic response data across a plurality of frequencies, identify a mode of operation of the acoustic imaging system corresponding to mechanical equipment acoustic imaging from among a plurality of different modes of operation; and in accordance with the mode of operation, analyzing the acoustic response data of the mechanical equipment as to at least one of degradation severity or degradation type.

[0212] The example acoustic imaging system may further comprise an acoustic imaging device that includes the acoustic sensor array, the optional camera system, the optional display, the processing system, and the memory, the acoustic imaging device comprising a user interface including the display, and optionally wherein the instructions, when executed, cause the acoustic imaging system to: display, on the display, an image depicting mechanical equipment including an overlay depicting an acoustic signal strength of acoustic signals emitted by the mechanical equipment at one or more tested acoustic frequencies; and receive, at the user interface, one or more user annotations of a known degradation type associated with the mechanical equipment. Analyzing the acoustic response data of the mechanical equipment optionally includes classifying degradation of the mechanical equipment according to a predetermined degradation type includes at least one of: predicting, at a classification model, the predetermined degradation type based on the acoustic response data across the plurality of frequencies; or matching an acoustic signature represented by the acoustic response data to one or more known acoustic response datasets to identify a similarity between the acoustic response data and at least some of the one or more known acoustic response datasets having a known degradation type. Capturing the acoustic response data across the plurality of frequencies optionally includes selecting a plurality of analysis frequencies based, at least in part, on a detected rotational speed of the mechanical equipment.

[0213] Another example of a method of analyzing acoustic response data may include receiving, at a computing system, acoustic response data representative of acoustic signals emitted by a mechanical equipment; comparing, at the computing system, the acoustic response data to baseline acoustic response data across a plurality of frequencies; and determining, at the computingsystem, a degradation profile based on the comparison, wherein the degradation profile includes at least one of a degradation type or a degradation severity. The computing system optionally comprises an acoustic imaging device used to capture the acoustic response data. Optionally, the method further comprises analyzing the acoustic response data to identify a mode of operation of the acoustic imaging system corresponding to mechanical equipment acoustic imaging from among a plurality of different modes of operation. The plurality of different modes of operation optionally includes a mechanical degradation detection mode, an electrical discharge detection mode, and an air leak detection mode. The degradation severity optionally is selected from among a plurality of predefined degradation severity levels and the degradation type is selected from among a plurality of predetermined degradation types. The computing system optionally comprises a system remote from an acoustic imaging device, and optionally receiving the acoustic response data includes receiving the acoustic response data via a network connection from the acoustic imaging device. The example method may further comprise sending information regarding the degradation profile to the acoustic imaging device for display at a user interface of the acoustic imaging device.

[0214] Comparing the acoustic response data to baseline acoustic response data optionally includes one or more of comparing acoustic response across the plurality of frequencies between the acoustic response data and the baseline acoustic response data to determine the degradation severity; comparing sound power density calculations between the acoustic response data and the baseline acoustic response data to determine the degradation severity; comparing frequency spikes in the acoustic response data and the baseline acoustic response data to determine the degradation severity; comparing ratios of periodicity and aperiodicity within the acoustic response data to determine the degradation type or the degradation severity; or comparing an acoustic signature representing the acoustic response data with one or more known or calculated natural frequencies and / or harmonics used as the baseline acoustic response data. The example method may further comprise suppressing acoustic data within at least a portion of the acoustic response data that is unrelated to degradation type or degradation severity.

[0215] Many alternatives to the techniques described herein are possible. For example, processing stages in the various techniques can be separated into additional stages or combined into fewer stages. As another example, processing stages in the various techniques can be omitted or supplemented with other techniques or processing stages. As another example, processing stages that are described as occurring in a particular order can instead occur in a different order. As another example, processing stages that are described as being performed in a series of steps may instead be handled in a parallel fashion, with multiple modules or software processes concurrently handling one or more of the illustrated processing stages. As another example, processing stagesthat are indicated as being performed by a particular device or module may instead be performed by one or more other devices or modules.

[0216] The principles, representative embodiments, and modes of operation of the present disclosure have been described in the foregoing description. However, aspects of the present disclosure which are intended to be protected are not to be construed as limited to the particular embodiments disclosed. Further, the embodiments described herein are to be regarded as illustrative rather than restrictive. It will be appreciated that variations and changes may be made by others, and equivalents employed, without departing from the spirit of the present disclosure. Accordingly, it is expressly intended that all such variations, changes, and equivalents fall within the spirit and scope of the claimed subject matter.

Claims

CLAIMS1. An acoustic imaging system comprising:an acoustic sensor array;a camera system;a processing system communicatively connected to the acoustic sensor array and the camera system;a memory communicatively connected to the processing system, the memory storing instructions that, when executed by the processing system, cause the processing system to:capture acoustic response data regarding acoustic signals emitted by mechanical equipment;conduct a comparison of the acoustic response data and baseline acoustic response data across a plurality of frequencies; andclassify, based on the comparison, the acoustic response data of the mechanical equipment as to at least one of degradation severity or degradation type.

2. The acoustic imaging system of claim 1, wherein classifying the acoustic response data of the mechanical equipment as to degradation severity includes generating a severity score based on the comparison, andoptionally wherein the severity score is generated using a classification model including a plurality of weighted features; oroptionally wherein the classification of degradation severity comprises a severity classification from among a plurality of severity classifications, wherein determining the classification of degradation severity includes comparing the severity score against one or more severity threshold values.

3. The acoustic imaging system of claim 2, wherein the classification model comprises a linear classification model, and wherein weights associated with each of the plurality of weighted features is determined using a linear regression across a historical dataset of acoustic response data; orwherein the classification model comprises a non-linear model implementing one or more constraints on the plurality of weighted features, wherein the one or more constraints are based on observations regarding known correspondence between mechanical degradation and acoustic response in a historical dataset of acoustic response data.

4. The acoustic imaging system of claim 2 or claim 3, wherein the plurality of weighted features includes two or more features of:a comparison to a baseline ratio of frequency components marked as periodic to overall frequency components;a comparison to a baseline of a spectral power of periodic components of the acoustic response data detected across a broad acoustic frequency spectrum;a comparison to a baseline of spectral power in a particular lower end frequency band; a comparison to a baseline of spectral power in a particular higher end frequency band; or an inverse of a detected mechanical component rotation speed;or optionally wherein the weighted features in the plurality of weighted features are each assigned a corresponding weight, wherein each corresponding weight is determined using a least squares linear regression over a dataset representing acoustic response data for a plurality of mechanical systems at varying levels of degradation severity.

5. The acoustic imaging system of any one of claims 1 to 4, wherein classifying the acoustic response data of the mechanical equipment as to degradation severity alternatively includes classifying the comparison of the acoustic response data to the baseline acoustic response data into a degradation classification from among a plurality of degradation classifications using a decision tree model; orwherein the memory stores instructions that, when executed by the processing system, alternatively cause the processing system to generate a spectrogram representative of the acoustic response data, wherein classifying the acoustic response data of the mechanical equipment as to degradation severity includes classifying the spectrogram at a deep learning model, optionally wherein the deep learning model comprises a neural network model trained using a plurality of spectrograms representing acoustic response information indicative of known degradation classifications.

6. The acoustic imaging system of any one of claims 1 to 5, wherein the acoustic response data is captured at a first location relative to the mechanical equipment and the baseline acoustic response data is captured at a second location relative to the mechanical equipment, and wherein the acoustic imaging system is configured to:determine a first distance from the mechanical equipment at the first location; determine a second distance from the mechanical equipment at the second location; and determine, based on acoustic attenuation, acoustic strength of the acoustic response data and the baseline acoustic response data at the mechanical equipment.

7. The acoustic imaging system of any one of claims 1 to 6, wherein the acoustic imaging system includes an acoustic imaging device, and wherein the acoustic imaging device includes the processing system, the memory, the acoustic sensor array, and the camera system; and optionally a user interface, wherein the instructions, when executed, cause the acoustic imaging system to:display, on a display, an image depicting the mechanical equipment including an overlay depicting an acoustic signal strength of acoustic signals emitted by the mechanical equipment at one or more tested acoustic frequencies; andreceive, at the user interface, one or more user annotations of known degradation severity associated with the mechanical equipment.

8. The acoustic imaging system of claim 7, wherein the one or more user annotations and acoustic data representative of the acoustic signals are provided as training data to update a model used in determining the degradation severity, andoptionally wherein the one or more user annotations and the acoustic data are provided to a remote device to be used as training data to update the model, wherein the acoustic imaging device is configured to receive an updated model from the remote device.

9. The acoustic imaging system of any one of claims 1 to 8, wherein the memory stores instructions that, when executed by the processing system, cause the processing system to:based on the acoustic response data, identify a mode of operation of the acoustic imaging system corresponding to mechanical equipment acoustic imaging from among a plurality of different modes of operation; andin accordance with the mode of operation, analyze the acoustic response data of the mechanical equipment as to at least one of degradation severity or degradation type.

10. The acoustic imaging system of any one of claims 1 to 9, wherein classifying the acoustic response data of the mechanical equipment comprises:predicting, via a classification model, a predetermined degradation type based on the acoustic response data across the plurality of frequencies; oridentifying, via acoustic signature matching, a known degradation type based on a similarity between the acoustic response data and a known acoustic response dataset having the known degradation type.

11. A method of analyzing acoustic response data, the method comprising:receiving, at a computing system, acoustic response data representative of acoustic signals emitted by a mechanical equipment;conducting, at the computing system, a comparison of the acoustic response data and baseline acoustic response data across a plurality of frequencies; anddetermining, at the computing system, based on the comparison, a degradation type or a degradation severity.

12. The method of claim 11, further comprising analyzing the acoustic response data to identify a mode of operation of the computing system corresponding to mechanical equipment acoustic imaging from among a plurality of different modes of operation, andoptionally wherein the plurality of different modes of operation includes a mechanical degradation detection mode, an electrical discharge detection mode, and an air leak detection mode.

13. The method of claim 11 or claim 12, wherein comparing the acoustic response data to baseline acoustic response data includes one or more of:comparing acoustic response across a plurality of frequencies between the acoustic response data and the baseline acoustic response data to determine the degradation severity; comparing sound power density calculations between the acoustic response data and the baseline acoustic response data to determine the degradation severity;comparing frequency spikes in the acoustic response data and the baseline acoustic response data to determine the degradation severity;comparing ratios of periodicity and aperiodicity within the acoustic response data to determine the degradation type or the degradation severity; orcomparing an acoustic signature representing the acoustic response data with one or more known or calculated natural frequencies or harmonics used as the baseline acoustic response data; andoptionally the method further comprising suppressing acoustic data within at least a portion of the acoustic response data that is unrelated to degradation type or degradation severity.

14. The method of any one of claims 11 to 13, wherein the mechanical equipment includes a rotating mechanical element, the method further comprising:analyzing the acoustic response data by extracting and quantifying a modulating signal as an impact measure of the rotating mechanical element coming into contact with a defect;determining a degradation severity of the rotating mechanical element based at least partly on the impact measure; andcausing presentation of the degradation severity in accordance with a field of view including a view of the rotating mechanical element;or optionally wherein the plurality of frequencies includes at least one of an infrasound frequency, an audible sound frequency, or an ultrasound frequency.

15. The method of claim 14, wherein determining the degradation severity of the rotating mechanical element is further based on a friction measure indicating fiction between the rotating mechanical element and one or more other elements,the friction measure being optionally determined based on a comparison in sound pressure level between the acoustic response data and at least one of prior acoustic response data or baseline acoustic response data.

Images