Acoustic imaging system for visual diagnostics

Abstract

An acoustic device and related user interface design are provided. The acoustic device enables visual diagnosis of potential problems from a sound source, e.g., based on visually presenting one type of graphical element indicative of a frequency band and another type of graphical element indicative of a noise level of the sound source in a scene at the frequency band.

Description

BACKGROUND

[0001] Acoustic imaging devices can be used to visualize sound waves, including those in the range of frequencies that are audible to humans and those outside of that range, including infrasound and ultrasound. Such devices can create an image or map of a sound field in a given area. Such devices can be used to detect and display sound sources and their distribution in a scene. Typically, acoustic imaging devices will include an array of sensors for detecting sound waves received from various directions. The device can then convert the sound waves into electrical signals that are processed to create a visual representation of the sound field.

[0002] Acoustic imaging devices have applications in different fields. They are often used in industrial applications, such as industrial and commercial equipment inspection and monitoring, equipment troubleshooting, pressure vessel inspection, vehicle inspection, quality testing, environmental noise monitoring, industrial noise control, architectural acoustics, product development, acoustic problem fault diagnosis, and the like. They can help identify noise sources, locate unwanted sound, assess sound propagation patterns, and optimize sound-related designs or configurations. SUMMARY

[0003] Generally, the present disclosure relates to acoustic imaging systems for visual diagnosis of a scene. An improved user interface can be used in conjunction with an acoustic imaging system for visual diagnosis of acoustic problems. Such a user interface can be presented directly on the acoustic imaging system or remotely on another computing device that is communicatively connected to the acoustic imaging system. The user interface presents a number of features that simplify the identification and display of acoustic responses captured at the acoustic imaging system. Such features can include, but are not limited to, the presentation of a number of selectable concurrent frequencies, and the adjustability of the selected frequencies, the display of severity indicators based on detected acoustic information and / or changes thereof. Additionally, various selectable graphical displays can be provided.

[0004] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0005] Non-limiting and non-exhaustive examples are described with reference to the following figures: FIG. 1 is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front plan view of an acoustic imaging device on which example aspects of the present disclosure can be implemented.

[0007] is a block diagram illustrating components of an example acoustic imaging system in which aspects of the present disclosure can be implemented.

[0006] FIG. 2 is a schematic front planFIG. 3 yes FIG. 2 A schematic rear plan view of an acoustic imaging device.

[0008] FIG. 4 This is a schematic plan view of an acoustic imaging sensor array that can be integrated into an acoustic imaging device according to an example aspect of this disclosure.

[0009] FIG. 5 An acoustic imaging user interface, including a field of view and a frequency selection area, is illustrated on a device such as that described herein.

[0010] FIG. 6 An example of a menu user interface that can be displayed on a device such as the one described herein is shown.

[0011] FIG. 7 An example of a mode selection user interface that can be displayed on a device such as the one described herein is shown.

[0012] FIG. 8 This is an example acoustic imaging user interface that displays multiple selectable frequencies simultaneously on a device such as the one described in this article.

[0013] FIG. 9 It is implemented based on the example. FIG. 8 An example of an acoustic imaging user interface, where the selected frequency is enabled and other frequencies are disabled.

[0014] FIG. 10 It is an acoustic imaging user interface that displays user-selectable frequencies on devices such as those described in this article.

[0015] FIG. 11 It is an acoustic imaging user interface that displays multiple optional severity indicators within the field of view, according to an example implementation.

[0016] FIG. 12 It is an acoustic imaging user interface that displays multiple optional severity indicators within the field of view, according to another example implementation.

[0017] FIG. 13 It is an acoustic imaging user interface that displays multiple selected frequencies and includes a warning indicator associated with one of the selected frequencies, according to another example implementation.

[0018] FIG. 14 It is an acoustic imaging user interface according to an example implementation that includes a time-domain acoustic signal graph that can be displayed next to the field of view.

[0019] FIG. 15It is an acoustic imaging user interface according to an example implementation, including a frequency selection area and a time-domain acoustic signal graph that can be displayed next to the field of view area.

[0020] FIG. 16 It is an acoustic imaging user interface according to the example implementation, which includes an octave / decade logarithmic frequency map that can be displayed in conjunction with the field of view area.

[0021] FIG. 17 It is an acoustic imaging user interface according to an example implementation that includes a logarithmic frequency response scale that can be displayed in conjunction with the field of view area.

[0022] FIG. 18 It is an acoustic imaging user interface according to another example implementation, which includes a logarithmic frequency response scale that can be displayed in conjunction with the field of view area.

[0023] FIG. 19 This is a flowchart of a method for performing acoustic imaging on a mechanical system according to an exemplary aspect of this disclosure.

[0024] FIG. 20 This is a flowchart illustrating a sub-method for performing acoustic imaging on a mechanical system according to a selected operating mode, based on an exemplary aspect of this disclosure.

[0025] FIG. 21 This is a flowchart of a method for generating a user interface for display at a computing system by performing acoustic imaging on a mechanical system according to an exemplary aspect of this disclosure.

[0026] FIG. 22 This is a flowchart of a method for analyzing acoustic data and diagnosing a sound source based on an example aspect of this disclosure, using a first graphical indicator based on an indicating frequency band and a second graphical indicator indicating the noise level of the sound source in that frequency band. Detailed Implementation

[0027] Mechanical systems, such as rotating components (bearings, pulleys, etc.), can emit acoustic signals that change over time as the system degrades. Therefore, degradation and failure of mechanical systems can be detected by using acoustic imaging, for example, by capturing acoustic feature data associated with the mechanical equipment at two or more different times and comparing these acoustic features to detect changes indicating degradation.

[0028] Previous attempts to use acoustic analysis to monitor mechanical system degradation or failure typically involved monitoring decibel readings at a single predetermined frequency (e.g., 30 Hz). However, the acoustic response of mechanical system degradation is usually not limited to a single frequency or can be reliably detected at a single frequency.

[0029] In at least some applications of acoustic imaging systems, including those for testing and / or detecting mechanical systems (e.g., industrial noise or noise from rotating, reciprocating, and other moving objects), ease of use remains a challenge. For example, in acoustically challenging or noisy environments, such devices may struggle to identify relevant acoustic signals. Furthermore, users of such devices may find it difficult to select appropriate frequencies where relevant signals might be detected. For instance, a user may not know the specific frequencies at which acoustic noise in a particular mechanical system might be present. The user might need to perform iterative tests at different frequencies and must rely on their judgment regarding which acoustic signal responses might be relevant. Alternatively, such a user might choose to monitor a wide range of frequencies, but in acoustically challenging environments, acoustic noise could drown out any signals of interest that might again be present within or outside the audible acoustic frequency range. Therefore, the ease of use of existing acoustic imaging devices for effectively isolating, displaying, and identifying potential problems or degradation in mechanical systems can be challenging. These challenges may also exist in other scenarios where acoustic imaging systems are employed.

[0030] Generally, this disclosure relates to systems and methods for inspecting and analyzing mechanical equipment using broadband acoustic imaging techniques. In some cases, the acoustic imaging device can acquire acoustic response data regarding acoustic signals emitted by the mechanical equipment at multiple frequencies. These frequencies can be selected based on various characteristics of the mechanical equipment (e.g., its rotational speed, design type, construction material dimensions, type of moving / rolling elements, etc.), or can be selected broadly within the acoustic frequency range. The acoustic signal can be compared with a baseline acoustic signal emitted by the mechanical equipment, and the acoustic signal can be analyzed to determine at least one of the severity or type of degradation. Various models or classification techniques are provided for determining the severity and / or type of degradation.

[0031] In one aspect, an acoustic imaging system is disclosed. The system includes an acoustic sensor array, a camera system, a display, a processing system, and a memory. The processing system is communicatively connected to the acoustic sensor array, the camera system, and the display, and the memory is communicatively connected to the processing system. The memory stores instructions that, when executed by the processing system, cause the processing system to: capture acoustic response data relating to acoustic signals emitted by mechanical equipment at multiple frequencies; compare the acoustic response data with baseline acoustic response data; and, based on the comparison, analyze and classify the acoustic response data of the mechanical equipment regarding at least one of degradation severity or degradation type.

[0032] In another aspect, an acoustic imaging system is disclosed. The system includes an acoustic sensor array, a camera system, a display, a processing system, and a memory. The processing system is communicatively connected to the acoustic sensor array, the camera system, and the display, and the memory is communicatively connected to the processing system. The memory stores instructions that, when executed by the processing system, cause the processing system to: capture acoustic response data representing acoustic signals emitted by mechanical equipment at multiple frequencies; identify, based on the acoustic response data, an operating mode of the acoustic imaging system corresponding to acoustic imaging of the mechanical equipment from multiple different operating modes; and analyze the acoustic response data of the mechanical equipment according to the operating mode, with respect to at least one of degradation severity or degradation type.

[0033] In another aspect, a method for analyzing acoustic response data is disclosed. The method includes receiving acoustic response data representing acoustic signals emitted by mechanical equipment at multiple frequencies at a computing system, and comparing the acoustic response data with baseline acoustic response data at the computing system. The method also includes determining a degradation profile at the computing system based on the comparison, wherein the degradation profile includes at least one of degradation type or degradation severity.

[0034] As briefly described above, embodiments of this disclosure relate to methods and systems for acoustic imaging analysis over a wide frequency spectrum. In some examples, the acoustic imaging device can acquire acoustic response data about acoustic signals emitted by mechanical equipment at multiple frequencies. The frequencies can be selected based on the rotational speed of the mechanical equipment, or can be selected broadly within the acoustic frequency range. The acoustic imaging of this disclosure can be used to detect degradation or failure of mechanical systems using broadband acoustic characterization. This analysis is divided into two categories: severity and classification (e.g., the degree of degradation or failure, and the type of degradation or failure).

[0035] Regarding severity, a wide spectrum is analyzed to determine the degree of degradation. In an example implementation, acoustic data is captured from mechanical systems of known good, known degradation, and known failure at different degrees of severity. A classification model that generates an overall severity score can be used. The overall severity score can be derived from a set of component values ​​represented as the difference from a baseline “known good” mechanical system. Each component value can be weighted based on experimental determination of its relative importance or degree of possible degradation or failure. Example components may include: a comparison with a baseline of the ratio of frequency components labeled as periodic to the overall frequency components; a comparison with a baseline of the spectral power, in decibels, of the periodic components detected across the entire wide spectrum; a comparison with a baseline of the spectral power in a specific low-end band (e.g., 15 kHz to 20 kHz); a comparison with a baseline of the spectral power in a specific high-end band (e.g., 35 kHz to 40 kHz); and the reciprocal of the detected rotational speed of a mechanical component (e.g., bearing rotational speed).

[0036] To obtain numerous analytical components within the analytical components, multiple analyses can be performed on the acoustic data. For example, by analyzing a wide frequency range, a periodic analysis can be performed to determine the portions of the acoustic signal corresponding to periodic components and the aperiodic portions of the acoustic signal. The ratio of the periodic signal to the aperiodic signal, and more specifically, changes in the ratio of the periodic signal to the aperiodic signal within the acoustic data, can indicate changes in state.

[0037] In another example, more or fewer components can be used based on the indication of their detected contribution or the severity of degradation; the specific selection of such components and their weighting can be derived experimentally and can be supplemented with different types of data on mechanical systems and mechanical system failures (when such data is captured).

[0038] In another example, the overall severity score can be defined as an integer within a range of values, where the final severity score is assigned to a severity category (e.g., good, pre-failure, failure onset, advanced failure, impending catastrophic failure) based on a preset threshold based on experimental data.

[0039] Regarding classification, experimental data has shown that different types of bearing failures can lead to different acoustic characteristics across the frequency range. For example, in the case of mechanical bearings, a good bearing will have specific acoustic characteristics, while a bearing with metal flakes may have similar acoustic characteristics but with a greater influence of the periodic component because the acoustic noise is emitted in various rotating harmonics. Grease-free bearings may have slightly higher acoustic characteristics in the low and mid-frequency range compared to good bearings. Rusty bearings may have higher baseline acoustic characteristics at lower frequencies, as will bearings with significant chemical or acid etching. Fault type classification can be implemented using techniques similar to those used for severity classification, such as examining the power or decibel levels of acoustic data at different frequency windows, analyzing periodic versus non-periodic components, etc.

[0040] Regarding both severity and classification, the specific implementation methods used can vary in several ways. For example, acoustic data can be captured via an acoustic imaging device and transmitted to a remote computing system for later analysis. Such analysis can be performed, and the results displayed on another computing system, or the results can be returned to the acoustic imaging device for display.

[0041] Alternatively, acoustic data can be captured via an acoustic imaging device and analyzed using specific algorithms or models implemented on that device. This arrangement allows for near real-time feedback on the severity or type of degradation in the mechanical system. The algorithms or models on the acoustic imaging device can be updated occasionally or periodically to improve accuracy or detect more types of degradation. Such updates can occur at the acoustic imaging device or remotely and are returned to the device for storage and use.

[0042] In another specific implementation, the analysis or modeling on the acoustic imaging device can be adaptive and can be adjusted over time to accommodate new types of mechanical system failures. In this implementation, the user of the acoustic imaging device will provide annotated feedback on the device regarding known good or known faulty mechanical systems, and this information will be combined with acoustic characterization information to retrain or adjust the model or equations used to determine the type or severity of degradation.

[0043] Based on the foregoing general description and the following disclosure, it is recognized that the wide-spectrum acoustic analysis described in this application has various advantages. Specifically, by analyzing acoustic response data over a wide frequency range, a more accurate determination of the severity of degradation of mechanical equipment can be obtained, because frequencies that might otherwise be overlooked but may include indicative features of degradation are taken into account. Furthermore, by performing different types of acoustic analysis over a wide frequency range, specific types of damage or degradation can be detected, such as periodic damage or degradation compared to other non-periodic damage or degradation. Moreover, the various available analytical techniques described herein can be applied individually or in combination, thus providing flexibility regarding the amount of computational resources required, the details of degradation assessment, etc.

[0044] In another aspect, a user interface design with unique graphical elements is disclosed to enable visual diagnosis of potential problems associated with a sound source. Given this user interface design, a method for enabling acoustic diagnosis may include: in response to an acoustic device entering an operating mode, presenting a plurality of selectable graphical indicators corresponding to respective frequency bands in a frequency scale on the graphical user interface; determining the noise level of the sound source at a frequency band in the plurality of frequency bands; and simultaneously presenting graphical indicators of a first type indicating the frequency band and a second type indicating the noise level of the sound source at the frequency band along the frequency scale to achieve visual diagnosis of the sound source.

[0045] The acoustic level or noise level of sound is typically measured in decibels (dB). Advantageously, by visually representing the dB level of a sound source at a specific frequency band, the disclosed system can often help users diagnose potential problems. In this disclosure, a frequency band can be simply described as a frequency. Sometimes, a frequency band refers to a frequency plus or minus 1 kHz; for example, 30 kHz in the various figures may refer to the frequency band from 29 kHz to 31 kHz.

[0046] The method for achieving acoustic diagnostics may include additional steps such as locking multiple frequency bands presented on a frequency scale in a graphical user interface, or fixing the spatial configuration of multiple frequency bands relative to a frequency scale in a graphical user interface. For visual diagnostics of a sound source, multiple second-type graphical indicators indicating the corresponding noise level of the sound source at corresponding frequency bands within the multiple frequency bands may be presented. Therefore, acoustic information of the source, particularly the dB level of the sound source at a specific frequency band, can be continuously stored or analyzed in a time-series manner, for example, for trend analysis based on the development of potential mechanical problems.

[0047] Simultaneously, this method for achieving acoustic diagnostics may include the following steps: floating multiple frequency bands represented on a frequency scale, thereby making the frequency range depicted on the frequency scale adjustable. In this way, the user can reconfigure the desired frequency band for acoustic diagnostics.

[0048] The method may further include: receiving user input via a graphical user interface associated with one of a plurality of selectable graphical indicators to select or deselect a frequency band among the plurality of frequency bands; causing a visual appearance change of one of the plurality of selectable graphical indicators in response to the user input; or causing a second type of graphical indicator indicating the dB level of a sound source in that frequency band to appear or disappear in response to the user input.

[0049] The method may further include selecting the plurality of frequency bands in response to a user selection of an operating mode for analyzing acoustic data in that operating mode. The method may further include: performing a visual diagnosis of the sound source based on storing the frequency bands and noise levels at the frequency bands as metadata of an acoustic image associated with the sound source; or performing a visual diagnosis of the sound source based on retrieving first information about the frequency bands and second information about the noise levels at the frequency bands from the metadata of the acoustic image, and analyzing the first information about the frequency bands and the second information about the noise levels at the frequency bands in light of other acoustic images associated with the sound source (such as in a time-series view analysis); or performing a visual diagnosis of the sound source based on a severity indicator associated with the sound source, the severity indicator being presented on a graphical user interface together with a second type of graphical indicator indicating the noise level of the sound source at that frequency band.

[0050] An exemplary acoustic device may include: a processor; and a memory storing instructions executable by the processor, wherein, when executed, these instructions cause the processor to determine a frequency band for analyzing acoustic data in an operating mode of the acoustic device; analyze the acoustic data based on the frequency band; and simultaneously present, along a frequency scale, a first type of graphical indicator indicating the frequency band and a second graphical indicator indicating the noise level of the sound source in the frequency band. The disclosed system may, for example, measure the dB level of the sound source in the frequency band based on the distance to the sound source. Advantageously, even if the distance from the acoustic device to the sound source changes, the dB level of the sound source in the frequency band can be continuously measured and should remain substantially the same. In some embodiments, in response to changes in the distance between the acoustic device and the sound source, the second graphical indicator indicating the noise level of the sound source in the frequency band will remain substantially stable. "Substantially stable" means that the dB level or the reading of the dB level remains unchanged or varies within a small range, for example, by a few dB.

[0051] As briefly described above, embodiments of this disclosure relate to an improved user interface and method of use thereof, which can be used in conjunction with an acoustic imaging system. Such a user interface can be presented directly on the acoustic imaging system or remotely on another computing device communicatively connected to the acoustic imaging system. The user interface presents several features that simplify the identification and display of acoustic responses captured at the acoustic imaging system.

[0052] In an example implementation, the user interface may include a field of view and one or more other areas displaying control features associated with the display of the captured acoustic data. For example, a frequency selection area may be presented. The frequency selection area may include multiple predefined or custom frequencies displayed simultaneously. The user can manually select the frequencies presented within the frequency selection area to enable or disable the display of acoustic signals captured at those frequencies. The user can also manually adjust the frequencies, for example, moving one or more frequencies within the frequency range displayed in the frequency selection area. The superposition of the detected acoustic signals presented within the field of view in the user interface can be updated based on the selected or enabled frequencies.

[0053] In terms of examples, the user interface may include one or more severity indicators graphically presented within a field of view area, a frequency selection area, or some combination thereof. For instance, user interface elements may be color-coded to indicate the severity or likelihood of a mechanical component failure based on historical data or current acoustic signal levels, or some combination thereof. Additionally or alternatively, one or more text-based indicators may be used to convey severity, such as using predefined severity levels (e.g., acceptable, low risk, moderate risk, high risk, etc.) or using a rating of the likelihood of the failure (e.g., a 0 to 100 scale). Furthermore, other graphical elements may be used to convey the severity of performance degradation, such as traffic signals, icons, etc.

[0054] In another example, the user interface can be adjustable to display one or more other types of graphs or plots to illustrate the six acoustic signal response or as a control mechanism. For example, time-domain acoustic signal graphs, octave-decade frequency logarithmic plots, logarithmic frequency response scales, or some combination of these graphs can be used next to the frequency selection area.

[0055] In some implementations, the user interface may be displayed on a monitor (such as the monitor of an acoustic imaging device). In some specific examples, the monitor may be a touchscreen monitor. In other examples, the monitor may be displayed on a device communicatively connected to the acoustic imaging device. In this case, the device may be a personal computing system (e.g., a desktop computer, laptop, or tablet) or a mobile device, and may include a touchscreen monitor or other types of displays and user input devices.

[0056] In summary, and referring to the example user interfaces described herein and methods of interacting with such user interfaces, it is evident that the user interface features described herein offer significant ease-of-use advantages to individual users interacting with acoustic imaging systems. Specifically, the user interface features described herein enhance the ability of technicians or individuals to inspect target objects through the use of broadband acoustic imaging technology. Such systems, which allow for both individual and simultaneous multi-frequency analysis, provide faster and more accurate diagnosis of potential degradation and / or failure problems, offering users greater value, efficiency, speed, and confidence. This, in turn, improves an organization's ability to successfully inspect and maintain mechanical components and equipment, resulting in longer uptime, improved technician safety, lower training intensity, better technician decision-making and communication, and ultimately reduced overall operating costs.

[0057] In one aspect, a computing device is provided. The computing device includes a display screen, a processor, and a memory storing instructions executable by the processor. When executed, these instructions cause the computing device to display a user interface on the display screen, including a field of view area and a frequency selection area. The frequency selection area displays a frequency scale and includes a plurality of selectable frequency indicators that can be displayed simultaneously thereon. The plurality of selectable frequency indicators includes at least a first selectable frequency indicator and a second selectable frequency indicator, the first and second selectable frequency indicators being selectable independently of each other and indicating different frequencies on the frequency scale. The field of view area displays a field-of-view image including a superposition of detected acoustic signal intensities, the superposition of detected acoustic signal intensities being at least partially based on the selected frequency indicator from the plurality of selectable frequency indicators identified in the frequency selection area.

[0058] In one aspect, the acoustic imaging apparatus includes an acoustic sensor array, a camera system, a touchscreen display, and a processing system communicatively connected to the acoustic sensor array, the camera system, and the touchscreen display. The acoustic imaging apparatus also includes a memory communicatively connected to the processing system, the memory storing instructions that, when executed by the processing system, cause the processing system to generate a graphical user interface (GUI) that can be displayed on the touchscreen display. The GUI includes a frequency selection area that displays a frequency scale and includes a plurality of selectable frequency indicators that can be displayed simultaneously thereon, the plurality of selectable frequency indicators including at least a first selectable frequency indicator and a second selectable frequency indicator, the first and second selectable frequency indicators being selectable independently of each other and indicating different frequencies on the frequency scale. The GUI also includes a field of view area displaying a field of view image of the camera system, the field of view image including a superposition of detected acoustic signal intensities obtained from the acoustic sensor array, the superposition of detected acoustic signal intensities being at least partially based on the selected frequency indicator from the plurality of selectable frequency indicators identified in the frequency selection area.

[0059] In one aspect, a method for performing acoustic imaging of a mechanical system is provided. The method includes: receiving a selection of an acoustic imaging mode for the mechanical device at a display screen associated with an acoustic imaging apparatus; and, in response to the selection of the acoustic imaging mode, receiving a selection of an option from a plurality of options displayed on the display screen, wherein the option is selected from a single-frequency option, a multi-frequency option, and a customizable frequency option. The method includes, in response to receiving a selection of the multi-frequency option, displaying a user interface on the display screen, the user interface including a frequency selection area and a field of view area. The frequency selection area displays a frequency scale and includes a plurality of selectable frequency indicators that can be simultaneously displayed thereon, the plurality of selectable frequency indicators including a plurality of predefined frequencies, each of the predefined frequencies being selectable via the display screen to be enabled or disabled within the frequency selection area. The field of view area displays a field of view image represented in image data, the field of view image including a superposition of detected acoustic signal intensities obtained from acoustic sensor data. The method further includes receiving a selection of one or more of a plurality of selectable frequency indicators to enable a subset of predefined frequencies, wherein the superposition of detected acoustic signal intensities is based at least in part on the enabled frequency indicator among the plurality of selectable frequency indicators.

[0060] In specific examples, the inclusion of multiple preset, selectable, or deactivated frequencies—selectable at frequencies where mechanical devices and systems typically exhibit faulty behavior—allows users to more quickly identify frequencies from which responses are received from specific mechanical components and to rapidly identify and compare acoustic responses from that component across multiple evaluations. Furthermore, by defining thresholds for acoustic signal levels, or by comparing them to historical acoustic signal levels, severity indicators can more easily identify the likelihood or probability of component failure or degradation to the user without requiring extensive user training to identify such problems. Various types of graphical displays can also better illustrate the characteristics of acoustic signals emitted by mechanical systems; as reflected in the frequency response, the amplitude of such signals at various periods can visually depict the performance (performance degradation) of such systems. Other advantages are also apparent and are reflected in the following description.

[0061] I. Operating Environment and Example Acoustic Imaging System First refer to FIGS. 1-4 This paper describes an example acoustic imaging system in which a user interface and methods of use and operation can be implemented. The system described herein should be considered exemplary, as user interfaces can be presented on various types of systems and in various scenarios, as is evident from the details of those interfaces themselves.

[0062] First refer to FIG. 1 An example acoustic imaging system 100 is depicted. In the illustrated example, the acoustic imaging system may include an acoustic imaging device 102, which may optionally be communicatively connected to one or more remote computing systems, such as remote system 10.

[0063] In various implementations, the acoustic imaging device 102 may be a handheld device, a robotic device, or a self-propelled device (e.g., ground-based or airborne, such as in the case of a drone), or a fixed device positioned to receive acoustic signals. Generally, an acoustic imaging device (also referred to herein as an acoustic camera or acoustic imaging system) visualizes sound waves and creates an image or map of a sound field in a given area. It is designed to detect and display sound sources and their distribution, for example, in real time. In the example shown, the acoustic imaging device 102 includes a processing system 110 communicatively connected to a memory 112 and an acoustic sensor array 120, a camera system 125, a display 130, an input device 132, a power subsystem 140, and a communication interface 150.

[0064] In the illustrated example, processing system 110 may include one or more programmable or dedicated execution circuits capable of executing computational instructions. Memory 112 may be volatile or non-volatile memory, such as read-only memory (“ROM”), random access memory (“RAM”), EEPROM, flash memory, or other memory technologies. Those skilled in the art and others will recognize that memory 112 typically stores data or program modules that are readily accessible or currently in operation by processing system 110. In this respect, processing system 110, including one or more processors, can serve as the computational center of acoustic imaging device 102 by supporting instruction execution.

[0065] Acoustic sensor array 120 may include a plurality of spaced-apart acoustic sensors positioned to determine the direction, distance, and magnitude of a signal emitted from a sound source based on the time and phase of acoustic signal reception. For example, in some embodiments, acoustic sensor array 120 may include up to 64 or more acoustic sensors spaced apart from each other, and these acoustic sensors are configured to detect acoustic signals in a frequency band of 2 kHz to 90 kHz at a range of up to or greater than 70 meters. Other frequency bands may also be used, including bands below 2 kHz and up to or greater than about 100 kHz. Each sensor within the array is responsible for detecting and measuring the acoustic signal at a specific location. Each acoustic sensor within the array may be positioned in a specific spatial configuration, typically in a planar arrangement. The acoustic sensor array may include various signal amplifiers and / or analog-to-digital converter circuitry, as well as signal processing units capable of extracting relevant data from the sensor array. In some examples, the acoustic sensor array may include such signal processing units, while in other examples, the processing system 110 may perform signal processing. (See below...) FIGS. 3-4 An example of such an acoustic sensor array 120 is depicted in the figure.

[0066] Camera system 125 is positioned to capture an imaging field of view relative to acoustic imaging device 102. The camera system typically captures image data in the same orientation direction as acoustic sensor array 120, thereby allowing the display of image data having superimposed acoustic signal data as described herein. In examples, camera system 125 may be a digital still image camera, a video camera, or may include multiple camera devices. In an example embodiment, the camera system may include a digital camera configured to capture images with an image quality greater than 1 megapixel and may have digital and / or optical zoom capabilities.

[0067] The display 130 can be any of a variety of display devices suitable for displaying images and acoustic information captured using the acoustic sensor array 120 and camera system 125. In an example embodiment, the display 130 can be an LCD display and can be capable of receiving user input. In some examples, the display 130 is a touchscreen display, such as a capacitive touchscreen display.

[0068] Input device 132 may include various additional input buttons or switches provided on acoustic imaging device 102 that extend beyond the touchscreen display. For example, in some cases, input device 132 may include a power button, an image capture button that can be used for image capture or to start / stop video capture, etc.

[0069] The power subsystem 140 may include one or more power sources, such as batteries capable of supplying power to other components of the acoustic imaging device 102. In this example, the power subsystem 140 may include a rechargeable battery, such as a lithium-ion battery. Other battery types or power sources, such as wired power connections, may also be provided.

[0070] Communication interface 150 may include one or more components for communicating with other devices, for example, via a direct wired connection or over a network. Embodiments of this disclosure provide access to basic services for performing communication using public network protocols via communication interface 150. Communication interface 150 may correspond to a common wired connection such as USB, FireWire, or a similar data connection, and / or may also include a wireless network interface configured to communicate via one or more wireless communication protocols, such as WiFi, 2G, 3G, 4G, 5G, LTE, WiMAX, Bluetooth, etc.

[0071] In the example shown, the acoustic imaging device 102 is communicatively connected to the remote system 10 via a communication interface 150. As an example, the remote system 10 includes a processing system 20, a memory 22, a display 30, an input device 32, and a communication interface 50.

[0072] The remote system 10 can be implemented as a computing system, such as a desktop computer, laptop computer, or handheld portable computing system (e.g., tablet computer, cellular phone, or other mobile device). The processing system 20 and memory 22 are similar to those described above included in the acoustic imaging device 102. The display 30 can be an LED, LCD, OLED, or other type of display, and can also be implemented as a touchscreen display or a non-touchscreen display. The display 30 can be configured to present various user interfaces described herein when image and acoustic data are received from the acoustic imaging device 102 at the remote system 10. The input device 32 typically includes one or more buttons, touch inputs, etc., depending on the form factor of the remote system 10. The input device 32 can include, for example, a keyboard, mouse, stylus, etc.

[0073] Communication interface 50 is configured to provide wired and / or wireless communication with other devices and may include, for example, a complementary connection to communication interface 150 described above. Through communication interfaces 50 and 150, acoustic imaging device 102 and remote system 10 can exchange data captured via acoustic sensor array 120 and camera system, as well as instructions from remote system 10, to allow, for example, local or remote storage of current and historical acoustic test data including acoustic and image data, and remote control of acoustic imaging device 102. Storage of current and historical acoustic test data may include storage of location information, acoustic signal levels, image data, test settings, etc., and may be used alone or in combination with other test data to determine or estimate the probability of degradation or failure of the target object as the object of acoustic imaging.

[0074] According to various aspects of this disclosure, note that some or all of the features of the acoustic imaging device 102 and / or the remote system 10 may or may not be present in all specific implementations, and such devices may include other functions and features not described herein (e.g., remote control features, mobility features, etc.). Generally, the acoustic imaging device 102 and the remote system 10 may be configured with a display capable of depicting the user interface described herein in real time or based on stored data and / or data transmitted from the acoustic imaging device 102 to the remote system 10 as part of the control of the acoustic imaging device 102. It should also be noted that, in various aspects of this disclosure and the appended claims, the acoustic imaging device and the remote system may be referred to as a first computing device and / or a second computing device.

[0075] FIGS. 2-3 These are schematic front and rear views of an acoustic imaging device 200 on which exemplary aspects of this disclosure may be implemented. The acoustic imaging device 200 is... FIG. 1 Examples of physical implementations of the acoustic imaging device 102, such as when it is implemented as a manually handheld unit.

[0076] like FIG. 2 As seen, the acoustic imaging device 200 includes a touchscreen display 230 positioned within a housing 202. The touchscreen display can be controlled via manual touch operations on user interface elements (such as those described below). A power button 232a activates the acoustic imaging device 200, and a capture button 232b initiates the capture of concurrent image and acoustic data in either a still / instant capture mode or a video / streaming mode.

[0077] At the rear of housing 202, an acoustic array 220 is mounted to housing 202. The acoustic array 220 includes a plurality of acoustic sensors 222, such as miniaturized microphones, spaced apart from each other along two dimensions on the acoustic array 220. The plurality of acoustic sensors 222 collectively determine the direction and signal strength of the acoustic signal. In some cases, the acoustic sensors 222 can also be used to determine the distance of the acoustic signal from the acoustic imaging device 200. This can be performed, for example, individually or in combination with image data acquired by camera 225. Camera 225 is positioned in a direction perpendicular to the plane defined by the plurality of acoustic sensors, such that camera 225 captures a field of view in the direction in which the acoustic signal can be sensed. In the example shown, camera 225 is positioned at the center of acoustic array 220; however, in other specific implementations, the camera may be located at other locations. In the example shown, a speaker / ventilation system 235 is also disposed on the rear of housing 202, providing air communication into housing 202 and allowing auditory feedback, for example, from a speaker.

[0078] In use, the acoustic imaging device 200 can be positioned or oriented toward an object of interest, for example, by aiming the acoustic array 220 and camera 225 at the object. A user can press the capture button 232b to initiate the acoustic sampling process, for example, to capture image data and acoustic signals, and then terminate the capture again (e.g., in video or streaming mode). The captured data can be displayed on the display 230, for example, for manipulation and viewing.

[0079] FIG. 4 This is a schematic plan view of an acoustic imaging sensor array 400 that can be integrated into an acoustic imaging device according to an example aspect of this disclosure. The acoustic imaging sensor array 400 includes an acoustic array 220 that includes acoustic sensors 222 as described above. The acoustic imaging sensor array 400 also includes a camera 225, also described above. In this example, a connector 402 allows the acoustic imaging sensor array 400 to be connected to other electronic systems, for example, it can be incorporated into a statically mounted sensor system or into a mobile device (such as a land-based mobile unit, an airborne drone, etc.).

[0080] As described above, the acoustic imaging device 200 or the acoustic imaging sensor array 400 can be used in a variety of applications. For example, acoustic signals can be emitted by pneumatic devices (e.g., detecting air leaks, etc.), electrical devices (e.g., detecting sparks, periodic noise generated by electrical signals, etc.), and mechanical devices (e.g., bearings, rollers, or various other reciprocating or moving systems such as motors, conveyors, gears and gearboxes, couplings, fans, compressors, and mechanical robots) in some embodiments described herein. In specific cases of mechanical devices, the detection of acoustic signals emitted by a particular piece of mechanical equipment can allow a user to determine the possible operating state of that equipment. For example, operating mechanical equipment may have specific acoustic characteristics, while mechanical equipment in different states of wear or failure may exhibit other characteristics. For example, a faulty bearing or rotor may emit acoustic noise at higher frequencies that may be difficult to hear but can be detected via the acoustic array 220. Other mechanical equipment may exhibit wear or failure modes via other types of acoustic output. Additionally, electrical or pneumatic systems can also indicate the presence of faults through acoustic signals at various frequencies. Each of these acoustic characteristics may be difficult for a skilled and trained individual user of an acoustic imaging system to readily identify using currently available equipment and systems for analyzing the acoustic response at a single frequency.

[0081] II. Acoustic Imaging User Interface Now for reference FIGS. 5-18 This document describes and illustrates various user interfaces that can be presented on a display associated with an acoustic imaging system. The user interface can be presented on the display of the acoustic imaging device (such as display 130, 230) or on the display of a remote system (such as display 30 of remote system 10) that receives images and acoustic data from such device. The user interface described herein can be used to control the acoustic imaging device or to analyze captured data, for example, to evaluate the acoustic response of the analyzed system in real time or after image capture has occurred.

[0082] FIG. 5An acoustic imaging user interface 500 is illustrated, which includes a field of view 501 and a frequency selection area 504. The acoustic imaging user interface 500 can be displayed on a display (such as displays 30, 130, 230). The field of view 501 displays image data associated with images captured by the camera system of the acoustic imaging device and a superposition of acoustic signals within a frequency range selected by the user in the frequency selection area 504. Specifically, the field of view 501 includes a focus indicator 506. The focus indicator 506 illustrates an area where the acoustic imaging device is configured to focus image data and where the acoustic sensor data is likely to be most accurate (e.g., by centering within the acoustic sensor array).

[0083] In the example shown, the field of view 501 depicts a scene including mechanical equipment 550, such as a conveyor belt with multiple rollers 552, each roller associated with a bearing that allows the roller to rotate. The user selects a range of frequencies along a frequency range using a manipulated frequency selection slider 510. The frequency range identified by slider 510 corresponds to the frequencies superimposed within the field of view 501 for which it is generated. In the example shown, the superposition 511 depicts the acoustic response at multiple different threshold levels (in the example shown, three different threshold levels indicate a comparison of the intensity of the acoustic signal response).

[0084] While the user interface 500 allows the user to view the acoustic response within the field of view 501, it provides the user with a limited ability to identify all potential frequencies of the signal emitted by the mechanical system. Because the mechanical system may have failure modes that produce acoustic signals at different frequencies, it is advantageous to provide the user with the additional ability to select multiple frequencies for analysis. Within the user interface 500, the user may be able to create a wide frequency window using the slider 510 to attempt to view the acoustic response over a wide frequency range. However, the user will need to set a relatively wide frequency range or move the slider 510 so that it will only display different, narrowing ranges at different times, rather than showing an aggregation of acoustic responses from a specific mechanical component. If a relatively wide frequency range is set, acoustic noise may interfere with the display of the acoustic signal emitted by the mechanical component. Therefore, other graphical display and operating modes are needed.

[0085] Furthermore, due to inherent equipment design, unique degradation characteristics, environmental factors, and variations in the sensitivity of acoustic sensors, single-frequency analysis and / or comparisons with previous test or analytical data can be significantly affected by errors. For comparative purposes, measuring the total dynamic range of sound (infrasound, audible sound, and ultrasound) over a wide frequency range can better represent the condition of the target object (e.g., bearings or mechanical equipment). This analysis can help avoid the limitations of single-frequency methods.

[0086] In a specific implementation, the operation selection menu 502 is displayed as a menu bar and presents multiple operation features as options for selection. In the example shown, the operation selection menu 502 allows control modes (e.g., mechanical mode in this case), memory management of the device's memory, acoustic signal settings, and annotation settings, including a color palette that can be used for acoustic overlay, markers that can be applied as part of the acoustic overlay, etc.

[0087] FIG. 6 A menu user interface 600 capable of being displayed on a device such as those described herein is illustrated. The menu user interface 600 can be displayed on a monitor such as the monitors 30, 130, and 230 described above. The menu user interface 600 can be displayed in response to the selection of an acoustic imaging option within the previously described operation selection menu 502. In this example, menu 602 presents multiple capture modes. The capture mode that is the subject of this disclosure corresponds to the mechanical device acoustic response capture mode, abbreviated as "MecQ" mode. Other modes such as image capture mode, video capture mode, leak capture mode, pulse capture ("PDQ") mode, etc., are also available.

[0088] FIG. 7 An example is illustrated of a mode setting selection user interface 700 that can be displayed on a device such as those described herein. The mode setting selection user interface 700 includes an acoustic settings menu sub-screen 702. The acoustic settings menu sub-screen 702 presents a display that can be adjusted by the user in response to... FIG. 6 The user interface 600 allows selection of multiple acoustic settings through the choice of a mechanical device acoustic response capture mode. In this example, the user can choose to manually or automatically set the minimum and maximum decibel levels considered within the graphical display, and enable one or more known acoustic profiles. Additionally, the user can enable detection of high-frequency events and multiple sound sources.

[0089] Furthermore, in the example shown, the acoustic settings menu subscreen 702 allows the user to select a specific operating mode within the mechanical device acoustic response capture mode. Operating modes may include a fixed 30kHz mode, a user-selectable frequency mode (between 2kHz and 100kHz), and a hybrid multi-mode option where a set of discrete, pre-selected frequencies and / or frequency subranges (either referred to herein as discrete frequencies for simplicity) are individually enabled or disabled as part of an overlay display of the acoustic response.

[0090] In the example of the provided mode setting selection user interface 700, when the hybrid multi-mode option is selected, multiple selectable discrete frequency indicators 710a to 710e (collectively referred to as frequency indicators 710) are displayed. Each of the selectable frequency indicators 710a to 710e is associated with a different discrete frequency or a different discrete frequency range (and can be pre-selected according to the acoustic response capture mode of the mechanical device). Specifically, in the example shown, discrete frequencies of 15kHz, 20kHz, 30kHz, 40kHz, and 60kHz are presented respectively associated with the frequency indicators 710a to 710e. In alternative embodiments, other numbers of frequency indicators may be displayed, and other frequencies may be used by default or according to a user-preselected method. Further examples of using such frequency indicators and additional examples of mode options are provided below in greater detail.

[0091] FIG. 8 This is an example acoustic imaging user interface 800 that simultaneously displays multiple selectable frequencies on a device such as those described herein. The acoustic imaging user interface 800 can respond to inputs such as... FIG. 7 The selection of the described hybrid multi-mode options is displayed within the acoustic settings menu subscreen 702. In the illustrated example, multiple frequency indicators 710a to 710e are again displayed. In this case, each frequency indicator is associated with acoustic signal level indicators 711a to 711e, respectively. Each acoustic signal level indicator 711 displays the acoustic signal level of the signal captured at that specific frequency. Therefore, the user can easily determine the frequencies at which significant acoustic signal components contribute to the overall signal strength reflected in the acoustic signal superposition. In the illustrated example, the acoustic signal level indicator 711 is presented close to the corresponding frequency indicator in the frequency indicator 710, for example, on opposite sides of the frequency scale bar. The acoustic signal level indicator 711 can be displayed as... FIG. 8 The acoustic signal level indicator 711 is positioned entirely or partially within the field of view 501 as depicted in the diagram. In an alternative embodiment, the acoustic signal level indicator 711 may be positioned entirely or partially within the frequency selection region 504.

[0092] In the example shown, an overlay 802 is presented within the field of view 501 of the acoustic imaging user interface 800. The overlay 802 can be presented as multiple colors or regions representing the relative intensity / strength of the acoustic signal, as determined by the acoustic sensor array previously described. Specifically, the overlay 802 can reflect the general intensity of the acoustic signal within selected frequencies identified by frequency indicators 710a to 710e.

[0093] In addition to overlay, text value 804 can be displayed as near overlay within focus indicator 506. Text value 804 can present a total acoustic reading representing the overall signal strength or maximum signal strength of the aggregated acoustic signal represented by overlay 802. In some embodiments, total signal strength refers to the dB level at the sensor of the acoustic device. Therefore, text value 804 can be updated according to distance as the distance from the acoustic device to the sound source changes, since the dB level decreases with increasing distance.

[0094] The text value 804 can also be toggled to individually depict the signal strength associated with each of the selected frequencies chosen for inclusion in the user analysis. This toggling can be performed in various orders: for example, by increasing or decreasing frequencies, by increasing or decreasing signal strength (in dB levels), or by switching by sound source (where multiple sound sources fall within the focus indicator area).

[0095] Additionally, in the illustrated example, a distance indicator 806 may be presented. Distance indicator 806 illustrates the distance between the acoustic imaging device used to capture and overlay the depicted image and the device within focus indicator 506. In the illustrated example, a distance of 3.1 m is depicted, illustrating the calculation of such a distance between the acoustic imaging device and the target object (e.g., a roller emitting an acoustic signal presented on a conveyor belt arrangement). The distance depicted in distance indicator 806 can be a manually entered distance, or it can be a distance calculated based on the use of an integrated optical measurement device included as part of a camera system as an acoustic imaging tool, or using the acoustic signal itself. Details regarding the capture or input of distances to a target are further described in U.S. Prelicense Publication No. 2022 / 0170780, entitled "Portable Acoustic Imaging Tool with Scanning and Analysis Capability," the disclosure of which is incorporated herein by reference in its entirety.

[0096] In an example implementation, the text value 804 of the signal strength can be the signal strength detected at the acoustic sensor array, such as the signal strength provided by the acoustic imaging device 102 described above. In other implementations, the text value 804 can represent the signal strength of a signal emitted from a target object. This value can be an extrapolated or calculated value based on the inverse square relationship between the signal strength detected at the acoustic sensor array and an automatically determined or manually entered distance value. In some examples, the text value 804 can also switch between the signal strength at the target object and the signal strength received at the acoustic imaging device, or it can display both values. For example, such a switching feature can be provided for previously captured or real-time displayed data.

[0097] Diagnostic analyses in the field of equipment reliability and maintenance can be performed in various ways, depending on the circumstances. For example, diagnostic analyses can be performed using data and information collected in situ at a single point in time. In other cases, time-based monitoring and trends of equipment over multiple time points can be used. Both types of analysis are valuable to reliability and maintenance practitioners.

[0098] Acoustic imaging tools and systems may be more susceptible to challenges from poor data collection than other tools. Poor data collection, caused by inconsistent parameter settings and inconsistent correlations of key data points, can lead to erroneous analysis and misdiagnosis of equipment performance. Parameters that need to be addressed and continuously managed may include frequency levels, frequency spans, distance to the target, and others. Parameter management is typically performed manually by the user of the acoustic imaging equipment, and human error is frequent because even a change in a single parameter can alter the measured and calculated data.

[0099] Furthermore, most acoustic imaging systems are currently configured to collect data and metadata at a time, either at one frequency or at a time, across one frequency band (span / level). While this may be useful in many cases for providing a snapshot of sound (including ultrasound) in that particular frequency band, it does not always provide diagnostic utility and direction. Meanwhile, the ability to continuously compare decibel level measurements at one frequency with decibel level measurements at different frequencies or multiple different frequencies provides significantly more information from which in-situ analysis can be performed.

[0100] For example, in some mechanical inspection applications, there is a correlation between decibel levels at certain frequencies and the severity of performance degradation in mechanical systems, where the progression of degradation follows a decreasing shift in the frequency of sounds and ultrasound emitted from the mechanical system. This observation supports the need for visualization across multiple frequencies within a given range and for data preservation. Similarly, the ability to compare decibel levels at a selected frequency at one point in time with decibel levels at the same frequency at multiple points in time provides users with increased diagnostic and potential predictive value through time-based trends in equipment performance. Consistency in this measurement and analysis is crucial for the accurate assessment of trends. If data collected at frequency levels and spans at one time differs from data collected at frequency levels and spans at another time, variability can lead to unacceptable errors in trend analysis, at least beyond a certain limit. Furthermore, the ability to consistently assess trends across multiple frequencies (or bands) over time provides the product of these two aspects, thereby elevating the ability to perform condition-based analysis to another performance level. Moreover, for both of these aspects, analysis and diagnosis based on the collected data can be performed automatically, semi-automatically, or manually. In both automated and semi-automated execution, the data-collecting device can provide real-time graphical notifications of in-situ acoustic analysis and diagnostics, and / or graphical representations of acoustic trend analysis and diagnostics.

[0101] FIG. 9 Further examples of an acoustic imaging user interface 900 implemented according to the example are illustrated. The acoustic imaging user interface 900 typically corresponds to... FIG. 8 The user interface 800 illustrates how selecting individual frequency indicators 710a to 710e can enable or disable those frequency indicators, resulting in a change in the superposition 902 of the intensity of the detected acoustic signal. Users can easily select or deselect frequency indicators by touching indicators on the screen, manipulating physical user interface elements (e.g., function buttons), or through other means of user input to the acoustic device.

[0102] In the example shown, frequency indicators 710a and 710e are selected as disabled, while frequency indicators 710b to 710d are selected as enabled. Therefore, the overlay 902 and the text value 904 indicating the overall signal strength or maximum signal strength are updated to reflect only those frequencies currently enabled. Thus, the user can quickly select or deselect specific frequencies to identify the component frequencies that contribute the most to the overall acoustic signal strength of the detected acoustic signal. This allows the user to quickly isolate frequencies of interest and easily exclude those frequencies experiencing significant interference.

[0103] Note that in the specific implementation of the example, the number of acoustic signal level indicators 711 and frequency indicators 710 can vary, and the frequency indicator 710 can be selectively displayed only when the corresponding acoustic signal level indicator is selected and active. Furthermore, the specific frequency level at which the frequency indicator 710 is placed can vary based on preset user settings, historical usage patterns, etc.

[0104] FIG. 10 Another example of an acoustic imaging user interface 1000 is illustrated. In this example, a user-selectable frequency is chosen for display. Specifically, the frequency indicator 1010 can be manually selected by entering a frequency value on a touchscreen display or by user input on a device (such as the previously described remote system 10 or acoustic imaging device 102). The frequency indicator 1010 can be manually manipulated (e.g., via a touchscreen display, by sliding the selected indicator along a frequency range presented within the frequency selection area 504). As the frequency indicator 1010 moves, the acoustic signal level indicator 1011 can be updated using the acoustic signal level at the selected frequency. Additionally, the overlay 1002 can be updated to reflect the acoustic signal strength and location associated with the selected frequency. The text value 1004 can also be updated to reflect the acoustic signal strength at the selected frequency. Note that in the acoustic imaging user interface 1000, a single frequency indicator 1010 is depicted. However, as will be apparent from this disclosure, two or more such manipulateable frequency indicators can be included within the user interface. Each frequency indicator in the frequency indicator can be moved independently along the frequency range presented within the frequency selection area 504, where the overall signal strength reflected in the overlay 1002 and the text value 1004 are updated accordingly. When the frequency indicator becomes movable, the user can then reconfigure the appropriate frequency band to visually diagnose acoustic problems.

[0105] In the example shown, graphical manipulation can be applied to the frequency selection area 504 to adjust the scaling of the displayed frequencies. For example, as illustrated, a "pinch" action 1012 along the scaling axis can adjust the overall scaling of the displayed frequencies (e.g., to cause a narrower or wider range of frequencies to be displayed). In response, a revised version of the frequency selection area 504 can be displayed. Such rescaling operations can also occur in other graphical areas (including time-domain and other frequency-domain graphical displays, such as those described below). FIGS. 14-18 It is possible in those described.

[0106] FIG. 11 Another example of an acoustic imaging user interface 1100 is illustrated. In this example, the acoustic imaging user interface 1100 can respond to... FIG. 7The selection of hybrid multi-mode options, as illustrated in the example, is presented. However, the acoustic imaging user interface 1100 includes additional notifications for the user to provide further insight into potential issues associated with the mechanical device emitting the acoustic signal, which is the object of analysis using the acoustic imaging system.

[0107] Specifically, in the example shown, the user interface is similar to FIG. 8 The user interface seen includes multiple predefined selectable frequency indicators 710a to 710e and associated acoustic signal level indicators 711a to 711e. However, in this example, additional indicators may be present to identify potential degradation of the mechanical system under test. For example, text box 1102 may include a notification indicating the potential degradation level of the device. The degradation level may be based on some combination of the currently detected acoustic signal level at a predefined frequency, previous tests of the acoustic signal level of the same component (and the change over time comparing the previous tests with the currently detected acoustic signal level), and the known signal response of the faulty equipment. The known signal level response of the faulty component may be based on the determination of other mechanical components from the failure mode. This failure prediction mode may be based, for example, on the high-frequency response (e.g., at 60 kHz) indicating a failure of other mechanical equipment in a similar location.

[0108] In the example shown, text box 1102 presents a text message indicating a “moderate” probability of component failure based on the detected acoustic signal level. This can be based on the current signal level, past signal levels, or changed signal levels within a predefined threshold. Other levels indicating different probabilities of component failure (no risk, low risk, moderate, high, extreme) can be based on the use of other current and / or past acoustic signal responses.

[0109] Additionally, in the example shown, the focus indicator 1106 can be configured to have different appearances depending on the likelihood of component failure of the mechanical components located within the focus indicator 1106 in the field of view. FIG. 11 In this context, dashed lines are used to depict the focus indicator. However, in the example implementation, the focus indicator can be a color-coded graphic element, where the color coding is based on the risk or urgency of a failure (e.g., green, yellow, orange, and red reflect an increased risk of component failure). Similarly, the basis for the color variation or selection of the focus indicator 1106 can be based on the current or past signal level compared to a threshold or known failure mode (e.g., the presence of a high-frequency signal within the frequency response).

[0110] In the example implementation, one or both of text box 1102 or focus indicator 1106 may be included within a given user interface. For simplicity, both text box 1102 and focus indicator 1106 are depicted in user interface 1100; however, in some cases, only one of the two types of signal classification notifications may be included. As described herein, text box 1102 and focus indicator 1106 may be referred to as severity indicators given their intended use.

[0111] FIG. 12 Another example of an acoustic imaging user interface 1200 that displays multiple additional optional severity indicators within a field of view 501 is illustrated. The acoustic imaging user interface 1200 generally corresponds to the user interface 1100, but includes a severity score indicator 1202 and a severity icon 1204.

[0112] In the example shown, the severity score indicator 1202 presents a numerical score on a scale that indicates the severity of a known maintenance or fault problem detected based on the acoustic response. In the example shown, the severity score corresponds to an estimated problem severity and is normalized on a scale of 0 to 100; other scales (e.g., 0 to 10, percentage-based, etc.) can also be used. The severity score can be calculated or estimated based on the acoustic signal level versus a threshold or as a comparison with past scores of previously described known fault modes.

[0113] In the illustrated example, severity icon 1204 is depicted as a traffic signal indicator with multiple distinct light indicators that can be selectively eliminated to indicate the severity of the detected problem. For example, a white light may indicate no risk, a green light may indicate low risk, a yellow light may indicate moderate risk, and a red light may indicate high risk. In the illustrated example, a severity score of 68 out of 100 within severity score indicator 1202 corresponds to moderate risk or the elimination of the yellow light in the traffic signal indicator used as severity icon 1204. Other correlations between severity scores and colors can also be defined.

[0114] and FIG. 11 Similar to the severity indicators illustrated, although both the severity score indicator 1202 and the severity icon 1204 are depicted in the same user interface 1200, it should be understood that only one of these indicators may be used in a given user interface. Additionally, FIGS. 11-12 The combinations of severity indicators shown in the examples can be used together.

[0115] FIG. 13Another example of an acoustic imaging user interface 1300 is illustrated. In this example, the acoustic imaging user interface 1300 displays a plurality of selected frequency indicators 1310a to 1310c with associated acoustic signal level indicators 1311a to 1311c. In this example, the frequency indicators 1310a to 1310c can be manipulated individually, allowing them to move along a frequency range depicted within the frequency selection area 504.

[0116] In this example, the individual acoustic signal level indicators in acoustic signal level indicator 1311 may be color-coded or have a changed appearance based on the current detected acoustic signal level at the corresponding frequency being outside a predetermined threshold. For example, in the user interface 1300 shown, an acoustic response at 95.2 kHz selected using frequency indicator 1310a results in a negative decibel level, thus causing a warning indicator to be displayed in the corresponding acoustic signal level indicator 1311a. For example, indicator 1311a may have a warning appearance, such as a changed color of the indicator's background or the text itself (e.g., red, orange, yellow, etc.), may flash, or may be depicted in a more visible manner (bold, larger size, etc.). Other acoustic signal level indicators 1311b to 1311c may have a normal appearance (e.g., indistinguishable from each other in size, color, etc.). Text box 1304 may depict the overall acoustic signal level, or, if particular attention should be paid to the selected frequency, may depict a selected acoustic signal level among the various acoustic signal levels of the selected frequency.

[0117] General reference FIGS. 8-13 The user interface, note that in some implementations, presents acoustic signal strength readings at the signal source, not at the acoustic imaging device. That is, the acoustic signal strength is calculated using a combination of the detected acoustic signal strength and the distance between the acoustic imaging device and the object being analyzed. As previously noted, this distance can be entered manually or detected automatically.

[0118] Using the acoustic signal strength at the signal source allows for improved comparison between current and previous measurements of acoustic signals emitted from the same object. Because users of acoustic imaging equipment can position the device at slightly different locations each time a test is performed, and because acoustic signal strength is inversely proportional to the square of distance, using the acoustic signal strength at the source improves the reliability of comparisons with past measurements and with known thresholds. Therefore, the severity of the problem, as indicated by the severity indicator in the previously described user interface, and warnings associated with individual frequencies can be presented with improved reliability.

[0119] Note that since obtaining the signal strength at the signal source requires calculations based on the received signal strength, both the signal strength at the source and the received signal strength can be captured. Furthermore, the user interface presented herein can be configured to allow the user (e.g., within the acoustic settings menu subscreen 702 or another screen generated in response to selecting an option from the operation selection menu 502) to switch between the signal strength at the source and the received signal strength.

[0120] Now for reference FIGS. 14-18 It presents additional user interface features, which allow, for example, through... FIGS. 5-13 Various additional window regions are presented alongside the field of view region 501 to flexibly analyze the received acoustic signal. Specifically, in addition to the frequency selection region 504 presenting a linear frequency range for evaluating the acoustic signal, or as an alternative, FIGS. 14-18 Other methods for analysis are illustrated.

[0121] In the first specific example, FIG. 14 The image depicts an acoustic imaging user interface 1400, which includes a time-domain acoustic signal graph 1404 that can be displayed alongside a field of view 501. FIG. 15 A similar acoustic imaging user interface 1500 is depicted, in which a similar time-domain acoustic signal graph 1504 is positioned alongside the field of view 501—but as an addition to, rather than a replacement for, the frequency selection area 504. The time-domain acoustic signal graphs 1404, 1504 can be useful for users who want to see the general amplitude and time scale of the detected acoustic signal. Such graphs can be manipulated by the user to zoom in and out of the time domain within the available data to view the overall signal response or the response within a specific time window.

[0122] While this disclosure envisions including such additional graphic regions, such as time-domain acoustic signal graphics 1404, 1504, the specific positioning and location of such graphics are intended only as examples. Such graphics may be positioned at other locations and / or orientations relative to the field of view, or in some cases may be presented as standalone graphics for analysis without showing such a field of view region.

[0123] FIGS. 16-18 Another example of a diagram that can be displayed next to the field of view region 501 is shown, and optionally further displayed next to the frequency selection region 504 (but not depicted here for simplicity). FIG. 16An acoustic imaging user interface 1600 according to yet another example is illustrated. The acoustic imaging user interface 1600 includes an octave / decade logarithmic frequency plot 1604 adjacent to a field of view region 501. The octave / decade logarithmic frequency plot can be used to identify periodically repeating signals that would otherwise be difficult to see within the available frequency range of the frequency selection region 504. In this example, the octave / decade logarithmic frequency plot 1604 uses a logarithmic scale and can present the acoustic signal at a specific frequency to allow the user to see the periodicity.

[0124] Similarly, FIG. 17 An acoustic imaging user interface 1700 is illustrated, which includes a logarithmic frequency response scale 1704 that can be displayed in conjunction with a field of view region 501. The logarithmic frequency response scale 1704 allows for the simultaneous display of both a wide frequency range of signals and a view of the amplitude or signal strength of such signals, thereby allowing the user to see a wider range of signal responses better than the frequency selection region 504. FIG. 18 An acoustic imaging user interface 1800 is illustrated, which includes an additional version of a logarithmic frequency response scale 1804 that can be displayed in conjunction with a field of view region 501. In this example, the logarithmic frequency response scale exemplifies the captured generalized frequency response and has a scale (logarithmic scale) of 30 dB to -30 dB across the entire frequency range.

[0125] refer to FIGS. 16-18 Note that the diagrams depicted in these user interfaces can also be included in different orientations, and can be included not only next to the field of view area, but also next to the frequency selection area as previously described. Additionally, and generally refer to... FIGS. 14-18 As described above, the detected acoustic signal level can be optionally set to the signal level at the source of the acoustic signal, or as the signal level received at the acoustic imaging device. Furthermore, the depicted graphical elements are intended as exemplary and not limiting, and reflect a way in which a user can more easily navigate between a wide frequency range that may identify acoustic response signals of interest. Such a user interface has particular applicability in the context of the acoustic output of a mechanical system, but can also be applied in a variety of other contexts in a manner consistent with this disclosure.

[0126] III. Acoustic Imaging System and Interface - Methods of Use and Operation Now for reference FIGS. 19-22 It describes the methods of using and operating the acoustic imaging system, as well as the user interface that can be generated and interacted with. FIGS. 19-22 The method can be performed, for example, using acoustic imaging equipment, a system including such acoustic imaging equipment, or a computing device included within such a system. (The above is combined with...) FIGS. 1-4 Examples of such devices and systems are provided.

[0127] In a specific example, FIG. 19 A flowchart illustrating a method 1900 for performing acoustic imaging on a mechanical system according to an example aspect of this disclosure is provided. Method 1900 includes receiving a mode selection (step 1902) in a user interface of the acoustic imaging system. The mode selection may define an acoustic response capture mode for the mechanical device (e.g., receiving a selection for the “MecQ” mode as described above). Other modes such as image capture mode, video capture mode, leak capture mode, pulse capture (“PDQ”) mode, etc., are also available and can be selected according to this disclosure.

[0128] In the example shown, the selection of the chosen mode is determined (at operation 1904). If the mechanical device acoustic response capture mode is selected, the operation is performed within the method described below; if another mode is selected, the acoustic imaging system optionally follows a different process (omitted here for brevity). Note that other modes may use a similar user interface and / or process.

[0129] For example, if the mechanical device acoustic response capture mode is selected, a selection of one or more frequencies or frequency ranges can be received (step 1906). The selection of one or more frequencies may include a selection of a hybrid multi-mode option in which multiple predefined frequencies are enabled. These multiple predefined frequencies may be discontinuous with each other, displayed simultaneously, and a set of frequencies may be selected simultaneously for analysis and display of acoustic signal strength. The selection of one or more frequencies may also include a selection of another mode that allows the user to manually define one or more frequencies and / or adjust such frequencies by moving a frequency indicator along a frequency scale within the user interface described herein. The method also includes displaying the acoustic signal strength at the selected frequencies in response to the selection of the one or more frequencies or frequency ranges (step 1908). Displaying the acoustic signal strength at the selected frequencies may include displaying a superposition reflecting the overall acoustic signal strength at the selected frequencies. Displaying the acoustic signal strength may also include displaying the individual acoustic signal strength at the selected frequencies. Optionally, displaying the acoustic signal strength at the selected frequencies may include adjustments to the user interface, which displays those frequencies based on an analysis of the detected signal strength relative to known problems that may occur at specific frequencies and signal strengths, for example, to generate warnings and / or severity indicators that can serve as user guides (as described below). FIGS. 11-13 (Example shown below). FIG. 21 An example describing the method for generating this display.

[0130] FIG. 20This is a flowchart illustrating a method 2000 for performing acoustic imaging on a mechanical system according to a selected operating mode, based on an exemplary aspect of this disclosure. Method 2000 may be based, for example, on the selection of an acoustic response capture mode for the mechanical device in steps 1902 to 1904 above, and as... FIG. 7 The options for frequency selection, as illustrated in the example, are presented to execute the selection.

[0131] In the example shown, method 2000 includes determining at operation 2002 that the user is in the user interface (e.g., FIG. 7 The user selects which of the displayed frequencies at the user interface. The selected options may include a hybrid multi-mode option, a fixed frequency option, and a user-adjustable selected frequency option. If the user-adjustable selected frequency option is selected, method 2000 includes receiving a further selection of one or more frequencies or frequency ranges (step 2004). The one or more frequencies or frequency ranges may be displayed simultaneously on a frequency scale depicted in the frequency selection area and may be discontinuous with each other. The one or more frequencies may be adjustable, for example, via user touch or touch and drag options on a touchscreen display of a computing device (such as remote system 10 or acoustic imaging device 102).

[0132] If the hybrid multi-mode option is selected, the method includes receiving a selection or deselection of one or more pre-selected frequencies (step 2006). In the example, multiple discrete frequencies or narrow frequency bands are presented, and the user (e.g., by touching a frequency indicator) can select each frequency band to enable or disable those bands to be included in the acoustic signal strength analysis.

[0133] In all operating modes, operation continues with the selected individual or combined acoustic signal strength displayed at a frequency selected by the user (step 2008). Such a display can be presented within the field of view and can superimpose the acoustic signal strength onto an image of the field of view captured by the acoustic imaging device. The display may include additional indicators, such as the distance between the acoustic imaging system and the target object, severity indicators, or alarms predicting or determining the severity of target object degradation or the likelihood of failure. The display may include any of a variety of auxiliary analysis displays, such as... FIGS. 14-18 Those exemplified in the text.

[0134] Now for reference FIG. 21A flowchart of a method 2100 for generating a user interface to be displayed at a computing system is provided. Method 2100 can be performed based on acoustic imaging of a target object, such as a mechanical system. The user interface can be displayed on a display such as displays 30, 130, 230 as described above, as part of an acoustic imaging system 100 including acoustic imaging device 102 and / or remote system 10.

[0135] In the example shown, method 2100 includes determining the sound level at a frequency selected by the user within the user interface (step 2102). The manner in which such a frequency is selected depends on the operating mode of the acoustic imaging system as previously described. Method 2100 also includes displaying an acoustic response overlay (step 2104). The acoustic response overlay may include a graphical representation of the acoustic signal intensity superimposed on an image captured by the acoustic imaging device, such as... FIGS. 5-18 exemplified.

[0136] In the illustrated example, method 2100 includes optionally performing a score (step 2106) on one or more acoustic signal intensities (and in some cases all acoustic signal intensities) at selected frequencies. The scoring of the acoustic signal intensities at selected frequencies can take various forms. In one example, the scoring may include determining the overall signal strength and comparing the overall signal strength to a predefined threshold to determine the likelihood of degradation or failure of a target object (e.g., a mechanical component) within the field of view of the acoustic imaging device. The comparison relative to the threshold may produce a numerical score, or it may produce a determination of a predefined severity level that can be depicted in text, graphics (e.g., colors and / or icons), etc. The comparison may be performed on aggregated signals or on individual acoustic signal intensities at discrete frequencies. The determination of how severity can be assessed may vary depending on the specific implementation and may be based on historical failure modes and associated known signal responses.

[0137] In the example shown, method 2100 includes comparing one or more levels or scores determined through analysis with a threshold (at operation 2108). If no frequency or score indicates a problem (e.g., within a threshold for problematic operation), one or more display messages indicating appropriate operation of the target object may be presented (step 2110). However, if one or more frequencies or scores, or the aggregation of signal strength at selected frequencies, indicate a potential fault, a display indicating one or more alarms associated with the problematic acoustic signal, frequency, or within the field of view associated with the object or an overlay displayed thereon may be generated (step 2112).

[0138] As previously noted, in cases where historical signal levels or changes in signal levels over time are used to determine potential faults or degradation of a target object, in some examples, the signal strength at the target object is used compared to the received signal strength. This comparison improves the reliability of indications that can be generated and displayed via the user interface described herein.

[0139] Now for reference FIG. 22 A flowchart of a method 2200 for acoustic data analysis and diagnosis is described. This method includes the analysis of acoustic data and the diagnosis of sound sources based on various examples described above. FIG. 22 As shown, method 2200 includes step 2202, wherein a frequency band is determined in order to analyze acoustic data obtained from acoustic signals received from a region or scene.

[0140] As step 2204, acoustic data is analyzed based on frequency bands, as described herein. This analysis enables diagnosis of the sound source at step 2206. According to an example aspect of this disclosure, in this example, such diagnosis is performed based on a first graphical indicator showing the frequency band as shown in the user interface and a second graphical indicator showing the noise level of the sound source at (or within) the frequency band as shown in the user interface.

[0141] In this disclosure, an exemplary computing device may include a display screen; a processor; or a memory storing instructions executable by the processor, wherein, when executed, these instructions cause the computing device to: display a user interface on the display screen, the user interface including a field of view and a frequency selection area, the frequency selection area displaying a frequency scale and including a plurality of selectable frequency indicators that can be displayed simultaneously thereon, the plurality of selectable frequency indicators including at least a first selectable frequency indicator or a second selectable frequency indicator, the first selectable frequency indicator and the second selectable frequency indicator being selectable independently of each other and indicating different frequencies on the frequency scale. Furthermore, the field of view may display a field-of-view image including a superimposed superposition of detected acoustic signal intensities, the superposition of detected acoustic signal intensities being at least partially based on the selected frequency indicator among the plurality of selectable frequency indicators identified in the frequency selection area.

[0142] The plurality of selectable frequency indicators can be associated with a plurality of predefined frequencies, each of which can be selected via a display to be enabled or disabled within a frequency selection area. The superposition of detected acoustic signal intensities can be based on a subset of the plurality of predefined frequencies enabled within the frequency selection area. Each of the plurality of selectable frequency indicators can be associated with a frequency subrange on a frequency scale and is adjustable within the frequency range depicted on the frequency scale. The computing device may include an acoustic imaging device, which may include an acoustic sensor array and a camera system. The computing device can be communicatively connected to a second computing device, which may include the acoustic imaging device, which includes an acoustic sensor array and a communication interface. The field of view may include a focus indicator, wherein the superposition is at least partially located within the focus indicator.

[0143] The user interface may include a severity indicator based at least in part on the detected acoustic signal. The severity indicator may include at least one of the following: a severity score indicating the estimated severity of a problem associated with the detected acoustic signal; a color-coded graphical element having an appearance based on the estimated severity of the problem associated with the detected acoustic signal; or a proximity-overlay text message indicating the level of the estimated severity of the problem associated with the detected acoustic signal. The color-coded graphical element may include at least one of a focus indicator, a traffic signal indicator, or an acoustic reading display associated with one or more of a plurality of selectable frequency indicators.

[0144] The user interface may also include at least one of a time-domain acoustic signal graph or a logarithmic frequency response scale. Each of the first selectable frequency indicator and the second selectable frequency indicator may be associated with a continuous frequency range and can be adjusted by the user via the user interface. The first frequency range associated with the first selectable frequency indicator may be discontinuous with the second frequency range associated with the second selectable frequency indicator. The user interface may also include a menu bar that includes acoustic imaging options, wherein selecting an acoustic imaging option may cause the display of multiple capture modes, including a mechanical device acoustic imaging mode.

[0145] An exemplary acoustic imaging device may include: an acoustic sensor array; a camera system; a touchscreen display; a processing system communicatively connected to the acoustic sensor array, the camera system, and the touchscreen display; or a memory communicatively connected to the processing system. The memory stores instructions that, when executed by the processing system, cause the processing system to: generate a graphical user interface (GUI) capable of being displayed on the touchscreen display. The GUI may include: a frequency selection area displaying a frequency scale and including a plurality of selectable frequency indicators capable of being displayed simultaneously thereon, the plurality of selectable frequency indicators including at least a first selectable frequency indicator and a second selectable frequency indicator, the first and second selectable frequency indicators being selectable independently of each other and indicating different frequencies on the frequency scale; or a field of view displaying a field of view image of the camera system, the field of view image including a superposition of detected acoustic signal intensities obtained from the acoustic sensor array, the superposition of detected acoustic signal intensities being at least partially based on the selected frequency indicator from the plurality of selectable frequency indicators identified in the frequency selection area. The multiple selectable frequency indicators can be associated with multiple predefined frequencies, each of which can be selected via a touchscreen display to be enabled or disabled within a frequency selection area. During the acoustic sampling process, an acoustic signal level indicator can be displayed within the user interface for each of the enabled frequency indicators among the multiple selectable frequency indicators. The acoustic signal level indicator can be displayed in either a warning or normal appearance, the warning appearance being based on whether the acoustic signal emitted by the mechanical component at the frequency associated with the selected frequency indicator and detected via an acoustic sensor array indicates a possible malfunction of the mechanical component. The user interface may include a severity indicator that can be displayed within the field of view, the severity indicator being at least partially based on the detected acoustic signal. The severity indicator may include at least one of the following: a severity score indicating the estimated severity of a problem associated with a detected acoustic signal; a color-coded graphic element having an appearance based on the estimated severity of a problem associated with a detected acoustic signal; or a near-overlay text message indicating the level of the estimated severity of a problem associated with a detected acoustic signal.

[0146] An exemplary method for performing acoustic imaging on a mechanical system may include one or more steps, such as: receiving a selection of an acoustic imaging mode for the mechanical device at a display screen associated with an acoustic imaging device; in response to the selection of the acoustic imaging mode for the mechanical device, receiving a selection of an option from a plurality of options displayed on the display screen, the option being selected from a single-frequency option, a multi-frequency option, and a customizable frequency option; and in response to receiving a selection of the multi-frequency option, displaying a user interface on the display screen, the user interface including a frequency selection area and a field of view area, wherein: the frequency selection area displays a frequency scale and includes a plurality of options that can be displayed simultaneously thereon. The selectable frequency indicators include a plurality of predefined frequencies, each of which can be selected via a display to be enabled or disabled within a frequency selection area, and the field of view is displayed in image data representing a field of view image including a superposition of detected acoustic signal intensities obtained from acoustic sensor data; and receiving selection of one or more of the plurality of selectable frequency indicators to enable a subset of predefined frequencies, wherein the superposition of detected acoustic signal intensities is based at least in part on the enabled frequency indicators among the plurality of selectable frequency indicators.

[0147] General reference FIGS. 1-22 The computing device described in this article can be implemented in computing logic embodied in hardware or software instructions, which can be implemented using programming languages ​​such as C, C++, COBOL, and JAVA. ™ , PHP, Perl, HTML, CSS, JavaScript, PythonScript, VBScript, ASPX, Microsoft.NET ™ The computational logic can be written in languages ​​such as C#. It can be compiled into an executable program or written in an interpreted programming language. Generally, the functionality described herein can be implemented as a logical module, which can be: copied to provide greater processing power; merged with other modules; or divided into submodules. The computational logic can be stored in any type of computer-readable medium (e.g., non-transitory media such as memory or storage media) or computer storage device, and can be stored on and executed by one or more general-purpose or special-purpose processors, thereby creating a dedicated computing device configured to provide the functionality described herein.

[0148] Many alternatives to the systems and devices described herein are possible. For example, individual modules or subsystems may be divided into additional modules or subsystems, or combined into fewer modules or subsystems. As another example, a module or subsystem may be omitted or supplemented with other modules or subsystems. As another example, a function indicated to be performed by a particular device, module, or subsystem may alternatively be performed by one or more other devices, modules, or subsystems. Although some examples in this disclosure include descriptions of devices comprising specific hardware components in a particular arrangement, the techniques and tools described herein may be modified to accommodate different hardware components, combinations, or arrangements. Furthermore, although some examples in this disclosure include descriptions of specific use cases, the techniques and tools described herein may be modified to accommodate different use cases. Functions described as implemented in software may alternatively be implemented in hardware, and vice versa.

[0149] Many alternatives to the techniques described herein are possible. For example, processing phases in various techniques can be divided into additional phases or combined into fewer phases. As another example, processing phases in various techniques can be omitted or supplemented with other techniques or processing phases. As another example, processing phases described as occurring in a specific order can alternatively occur in a different order. As another example, processing phases described as being performed in a series of steps can alternatively be processed in parallel, wherein multiple modules or software processes process one or more of the illustrated processing phases simultaneously. As another example, processing phases indicated to be performed by a specific device or module can alternatively be performed by one or more other devices or modules.

[0150] The principles, representative embodiments, and operating modes of this disclosure have been described in the foregoing description. However, the aspects of this disclosure intended to be protected should not be construed as limited to the specific embodiments disclosed. Furthermore, the embodiments described herein should be considered illustrative rather than restrictive. It should be understood that variations and modifications, and equivalent solutions, can be made by others without departing from the spirit of this disclosure. Therefore, it is expressly intended that all such modifications, variations, and equivalents fall within the spirit and scope of the claimed subject matter.

[0151] This application claims the benefits and priority of U.S. Provisional Application No. 63 / 506,563, filed June 6, 2023, and U.S. Provisional Application No. 63 / 607,938, filed December 8, 2023, the entire contents of which are incorporated herein by reference.

Claims

1. A method for enabling acoustic diagnosis, the method comprising: in response to an acoustic device entering an operational mode, causing a plurality of selectable graphical indicators corresponding to respective frequency bands of a plurality of frequency bands to be presented on a graphical user interface in a frequency scale; determining a noise level of a sound source at a frequency band of the plurality of frequency bands; and based on a first type of graphical indicator indicative of the frequency band and a second type of graphical indicator indicative of the noise level of the sound source at the frequency band being simultaneously presented along the frequency scale, enabling a visual diagnosis of the sound source.

2. The method of claim 1, further comprising: locking the plurality of frequency bands presented on the graphical user interface in the frequency scale.

3. The method of claim 1, further comprising: fixing a spatial configuration of the plurality of frequency bands relative to the frequency scale on the graphical user interface.

4. The method of claim 1, further comprising: floating the plurality of frequency bands presented in the frequency scale, such that a frequency range delineated on the frequency scale is adjustable.

5. The method of claim 1, further comprising: further based on a plurality of second types of graphical indicators indicative of respective noise levels of the sound source at respective frequency bands of the plurality of frequency bands, enabling the visual diagnosis of the sound source.

6. The method of claim 1, further comprising: receiving, via the graphical user interface, a user input associated with one of the plurality of selectable graphical indicators to select or deselect the frequency band of the plurality of frequency bands.

7. The method of claim 6, further comprising: in response to the user input, causing a visual appearance of the one of the plurality of selectable graphical indicators to change.

8. The method of claim 6, further comprising: in response to the user input, causing the second type of graphical indicator indicative of the noise level of the sound source at the frequency band to appear or disappear.

9. The method of claim 1, further comprising: in response to a user selection of the operational mode, selecting the plurality of frequency bands to analyze acoustic data in the operational mode.

10. The method of claim 1, further comprising: further based on saving first information of the frequency band and second information of the noise level at the frequency band as metadata of an acoustic image related to the sound source, enabling the visual diagnosis of the sound source.

11. The method of claim 10, further comprising: further based on retrieving the first information of the frequency band and the second information of the noise level at the frequency band from the metadata of the acoustic image, and analyzing the first information of the frequency band and the second information of the noise level at the frequency band in view of other acoustic images related to the sound source, enabling the visual diagnosis of the sound source.

12. The method of claim 1, further comprising: ​ implementing the visual diagnosis of the sound source further based on a severity indicator associated with the sound source, the severity indicator being presented on the graphical user interface along with the second type of graphical indicator indicative of the noise level of the sound source at the frequency band.

13. The method of claim 12, wherein the severity indicator comprises at least one of: a severity score indicative of an estimated problem severity associated with the sound source; a color-coded graphical element having an appearance based on the estimated problem severity associated with the sound source; or a text message indicative of a level of the estimated problem severity associated with the sound source.

14. An acoustic device, the acoustic device comprising: a processor; and a memory storing instructions executable by the processor, wherein the instructions, when executed, cause the processor to: determine a frequency band for analyzing acoustic data in an operating mode of the acoustic device; analyze the acoustic data based on the frequency band; and implement a diagnosis of a sound source based on a first type of first graphical indicator indicative of the frequency band being simultaneously presented along a frequency scale with a second type of second graphical indicator indicative of a noise level of the sound source at the frequency band.

15. The acoustic device of claim 14, wherein the instructions, when executed, further cause the processor to: hold the second graphical indicator indicative of the noise level of the sound source at the frequency band substantially stable in response to a change in a distance between the acoustic device and the sound source.

16. The acoustic device of claim 14, wherein the instructions, when executed, further cause the processor to: generate a third type of graphical indicator representing the sound source and cause the third type of graphical indicator to be superimposed on an image showing a field of view of the acoustic device.

17. The acoustic device of claim 16, wherein the instructions, when executed, further cause the processor to: update the third type of graphical indicator representing the sound source in response to a change in a distance between the acoustic device and the sound source, wherein the third type of graphical indicator comprises a component representing a noise level at an acoustic sensor of the acoustic device.

18. The acoustic device of claim 14, further comprising: an acoustic imaging device comprising an array of acoustic sensors and a camera system.

19. The acoustic device of claim 14, wherein the acoustic device is communicatively connected to a second computing device comprising an acoustic imaging device comprising an array of acoustic sensors and a communication interface.

20. The acoustic device of claim 14, further comprising: a display for presenting the first graphical indicator indicative of the frequency band and the second graphical indicator indicative of the noise level of the sound source at the frequency band. ​

Images