System and method for representing acoustic features from a target scene

Abstract

An acoustic imaging system can include an acoustic sensing array, an electromagnetic imaging tool, a display, and an audio device. A processor can receive data from the acoustic sensor array and the electromagnetic imaging tool to generate a display image that combines acoustic image data and electromagnetic image data. The system can include an audio device that receives audio output from the processor and outputs an audio feedback signal to a user. The audio feedback signal can represent an acoustic signal from an acoustic scene. The system can provide the display image including acoustic image data to the user, and the user can select an acoustic signal that corresponds to the audio output to provide to the audio device. The audio output and the display image can dynamically change in response to a change in a pointing of the acoustic sensing array, such as by changing a stereo audio output.

Description

[0001] Related matters

[0002] This application claims priority to U.S. Patent Application No. 62 / 702,716, filed July 24, 2018, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Currently available acoustic imaging devices include acoustic sensor array configurations with various frequency sensitivity limitations due to a wide range of factors. For example, some acoustic imaging devices are designed to respond to an acoustic frequency range between approximately 20 Hz and approximately 20 kHz. Other devices (e.g., ultrasound devices) are designed to respond to an acoustic frequency range between approximately 38 kHz and approximately 45 kHz.

[0004] However, acoustic imaging devices typically designed to operate in the 20 Hz to 20 kHz frequency range are ineffective at detecting or imaging higher frequencies, such as those up to or above approximately 50 kHz. Similarly, acoustic or ultrasonic devices designed to operate in the 20 kHz to 50 kHz frequency range are ineffective at detecting and / or imaging lower frequencies, such as those at or below 20 kHz. This can be for a variety of reasons. For example, sensor arrays optimized for lower (e.g., audible) frequencies often contain individual sensors that are more spaced apart than sensor arrays optimized for higher (e.g., ultrasonic) frequencies.

[0005] Aside from hardware considerations or as alternatives, different computational algorithms and methods for acoustic imaging are often better suited to acoustic signals with different frequencies and / or different distances to the target, making it difficult to determine how best to acoustically image a scene, especially in the absence of inexperienced users.

[0006] Such differences in imaging different frequency ranges are partly due to the physical phenomena underlying the propagation of sound waves of different frequencies and wavelengths through the air. Certain array orientations, array sizes, and computational methods are often better suited to sound signals with different frequency characteristics (e.g., audible frequencies, ultrasonic frequencies, etc.).

[0007] Similarly, different array properties and / or computation methods can be better suited to acoustic scenarios at different distances from the target. For example, near-field acoustic holography for targets at very close range, and various acoustic beamforming methods for targets at greater distances.

[0008] Accordingly, acoustic inspections using acoustic arrays (e.g., for acoustic imaging) can require a wide range of equipment, e.g., for analyzing acoustic signals having different frequency ranges, and expertise in knowing when different hardware and computing techniques are suitable for performing acoustic analysis. This can make acoustic inspections both time consuming and expensive, and can require a specialist to perform such inspections.

[0009] For example, a user can be forced to manually select various hardware and / or software for performing acoustic analysis. However, an inexperienced analyst can not know the preferred combination of hardware and software for a given acoustic analysis and / or acoustic scene. Additionally, isolating sounds of interest from within a scene can present its own challenges, particularly in a cluttered scene, and can be tedious and frustrating for an inexperienced user. For example, a given acoustic scene (particularly in a noisy environment) can include acoustic signals that include any number of frequencies, intensities, or other characteristics that can obscure acoustic signals of interest.

[0010] Conventional systems tend to require a user to manually identify various acoustic parameters of interest prior to an inspection in order to analyze sounds of interest. However, an inexperienced user can not know how to best isolate and / or identify various sounds of interest.

[0011] Additionally, when multiple imaging techniques (e.g., visible light, infrared, ultraviolet, acoustic, or other imaging techniques) are used in tandem while inspecting the same object or scene, the physical layout and / or other settings (e.g., focal orientation) of the tools used to perform the different imaging techniques can impact analysis. For example, different positions and / or focal orientations of each imaging device can result in parallax errors, where the resulting images can be misaligned. This can result in an inability to properly locate areas of interest and / or problem areas within a scene, documentation errors, and incorrect diagnoses of problems. For example, with respect to acoustic image data, it can be difficult to identify the location or source of acoustic signals of interest if the acoustic image data is misaligned with respect to image data from other imaging techniques (e.g., visible light and / or infrared image data).

[0012] Existing ultrasonic testing and inspection tools employ ultrasonic sensor(s) with or without a parabolic dish to help focus sound toward the receiving sensor(s).

[0013] When a sound of a particular frequency is detected, it is typically displayed as a rising or falling numerical value, or on a frequency or decibel level graph on the device’s display. This can be very confusing and unintuitive for many users.

[0014] However, isolating, locating, and analyzing specific sounds can be a tedious process and can be confusing to many end users. Complex and unintuitive interfaces between the device and the person can be a barrier to effective use of the device and / or even require additional training to operate basic functions on the device.

[0015] Additionally, some of these devices utilize a set of optional audio headphones to amplify and broadcast the actual sound or heterodyne related to the actual sound being detected. The intensity of the sound to the listener's ear typically increases when the sensor(s) are aimed closer to the source or sound, or when the sensor(s) are brought closer to the sound.

[0016] However, isolating specific sounds can be a tedious process and can be confusing to many end users. It is often difficult for the user to coordinate the audible or heterodyne increase in sound intensity with the rapid changes in values on the display screen. SUMMARY

[0017] Some aspects of the present disclosure relate to an acoustic analysis system comprising an acoustic sensor array having a plurality of acoustic sensor elements, and an electromagnetic imaging tool configured to receive electromagnetic radiation from a target scene and output electromagnetic image data representative of the received electromagnetic radiation. The system can include a display and an audio device configured to output an audio feedback signal. In some examples, the audio device can include a loudspeaker or headphones and can be in wired or wireless communication with other system components.

[0018] The system can include a processor in communication with the acoustic sensor array, the electromagnetic imaging tool, the display, and the audio device. In some examples, the processor is configured to receive electromagnetic image data from the electromagnetic imaging tool and acoustic data from the acoustic sensor array. The processor can be configured to generate acoustic image data based on the received acoustic data and generate a display image including the combined acoustic image data and electromagnetic image data.

[0019] In some embodiments, the processor is further configured to generate an audio output representative of the received acoustic data and output the audio output to the audio device. In examples, the processor is configured to detect an acoustic signal and the generated audio output includes a representation of the detected acoustic signal. For example, the processor can determine one or more acoustic parameters associated with the detected acoustic signal and generate an audio output having the one or more acoustic parameters as the detected acoustic signal.

[0020] In some examples, the processor is configured to generate a stereo audio output having a first audio output to a first audio device and a second audio output to a second audio device. In some such examples, the first audio device outputs a first audio feedback signal and the second audio device outputs a second audio feedback signal. The processor can be configured to determine a location of the acoustic signal in the acoustic scene and generate the first audio output and the second audio output based on the determined location of the acoustic signal in the acoustic scene.

[0021] In some examples, the processor can detect multiple acoustic signals within the scene. The processor can output an audio output representing a combination of each acoustic signal. The processor can output multiple audio outputs, each audio output representing one of the multiple acoustic signals, e.g., in sequence, cycling through the corresponding multiple audio outputs. In some examples, a user can select an acoustic signal observed in the display image, such as via a touchscreen input. The processor can output an audio output corresponding to the selected acoustic signal.

[0022] In some examples, the processor can be configured to provide an output representing a dynamic change in the acoustic signal.

[0023] For example, the intensity of the audio output can increase as the acoustic signal moves toward the center of the acoustic scene. Similarly, in a stereo embodiment having first and second audio outputs, the intensity of the audio feedback signal representing the acoustic signal in the first audio output and the intensity of the audio feedback signal representing the acoustic signal in the second audio output vary as the pointing of the acoustic sensor array is adjusted. For example, an acoustic signal on the left-hand side of the scene can be more strongly represented in the left-hand audio feedback signal than in the right-hand audio feedback signal.

[0024] Additionally or alternatively, the acoustic image data can vary in intensity depending on the pointing of the acoustic sensor array. For example, in some embodiments, an indicator of the acoustic signal in the acoustic image data can have a variable transparency in the display image, where the further the location of the acoustic signal is from the center of the acoustic scene, the more transparent the indicator becomes. BRIEF DESCRIPTION OF DRAWINGS

[0025] FIG. 1A and 1B Front and rear views of an example acoustic imaging device are shown.

[0026] FIG. 2 is a functional block diagram illustrating components of an example of an acoustic analysis system.

[0027] FIG. 3A , 3B and 3C show schematic diagrams of example acoustic sensor array configurations within an acoustic analysis system.

[0028] FIG. 4A and 4B A schematic illustration of parallax error in the generation of frames of visible light image data and acoustic image data is shown.

[0029] FIG. 5A and 5B Parallax correction between visible light images and acoustic images is shown.

[0030] FIG. 5C and 5D is FIG. 5A and 5B a color version of

[0031] FIG. 6 is a process flow diagram illustrating an exemplary method for generating a final image that combines acoustic image data and electromagnetic image data.

[0032] FIG. 7 is a process flow diagram illustrating an exemplary process for generating acoustic image data from received acoustic signals.

[0033] FIG. 8 An exemplary lookup table for determining the appropriate algorithm and sensor array to use during an acoustic imaging process is shown.

[0034] FIG. 9A is an exemplary plot of the frequency content of received image data in an acoustic scene over time.

[0035] FIG. 9B An exemplary scene including multiple locations from which acoustic signals are emitted is shown.

[0036] FIG. 9C Multiple combined acoustic and visible light image data at multiple predefined frequency ranges is shown.

[0037] FIG. 10A and 10B is an exemplary display image including combined visible light image data and acoustic image data.

[0038] FIG. 11A and 11B An exemplary plot of the frequency of acoustic data in an acoustic scene versus time is shown.

[0039] FIG. 12A , 12B and 12C show various exemplary ways of comparing acoustic image data to historical acoustic image data stored in a database.

[0040] FIG. 13is a process flow diagram showing exemplary operations of comparing received acoustic image data to a database for object diagnosis.

[0041] FIG. 14 Visualization of acoustic data using a gradient palettization scheme is shown.

[0042] FIG. 15 Visualization of acoustic data using multiple shaded concentric circles is shown.

[0043] FIG. 16 An exemplary visualization is shown that includes both non-numeric information and alphanumeric information.

[0044] FIG. 17 Another example visualization is shown that includes both non-numeric information and alphanumeric information.

[0045] FIG. 18 Another example visualization is shown that includes both non-numeric information and alphanumeric information.

[0046] FIG. 19 An exemplary visualization is shown that shows different sized and colored indicators representing different acoustic parameter values.

[0047] FIG. 20 An exemplary visualization is shown that shows multiple indicators with different colors indicating the severity indicated by the acoustic signals from the corresponding locations.

[0048] FIG. 21 A scene is shown that includes indicators at multiple locations within the scene that show acoustic signals that meet predetermined conditions in a differentiated manner.

[0049] FIG. 22 A display image is shown that includes multiple icons positioned within the display image that indicate identified acoustic profiles within a scene.

[0050] FIG. 23 Another exemplary display image is shown that shows acoustic data via multiple indicators, using concentric circles and alphanumeric information representing the acoustic intensity associated with each acoustic signal.

[0051] FIG. 24 An example display image is shown that has an indicator and additional alphanumeric information associated with the represented acoustic signal.

[0052] FIG. 25A A system is shown that includes a display in which an indicator within a display image is selected and a laser pointer emits a laser toward a scene.

[0053] FIG. 25B A display is shown in the system view of FIG. 25.

[0054] FIG. 26 A display image is shown that includes an indicator having a gradient color scheme representing acoustic image data and including an acoustic image blending control.

[0055] FIG. 27 A display image is shown that includes an indicator having a concentric circle color scheme representing acoustic image data and including an acoustic image blending control.

[0056] FIG. 28 A display image is shown that includes an indicator having a gradient color that indicates locations in a scene that satisfy one or more filtering conditions.

[0057] FIG. 29 A display image is shown that includes an indicator having a concentric circle color that indicates locations in a scene that satisfy one or more filtering conditions.

[0058] FIG. 30 A display image is shown that includes two indicators each having a gradient color that indicates locations in a scene that satisfy different filtering conditions.

[0059] FIG. 31 A display interface is shown that includes a display image and a virtual keyboard.

[0060] FIG. 32 A display is shown that is embedded into eyewear that can be worn by a user and that displays a display image.

[0061] FIG. 33A and 33B A dynamic display image is shown that includes an indicator having a dynamic intensity that is based on a pointing of an acoustic sensor array.

[0062] FIG. 34A , 34B , 34C and 34D show various types of audio devices in communication with an acoustic imaging device in an acoustic imaging system.

[0063] FIG. 35 An example embodiment is shown in which an audio device provides audio feedback to represent displayed acoustic image data.

[0064] FIG. 36 An example embodiment is shown in which an audio device provides selective audio feedback corresponding to a selected acoustic signal source.

[0065] FIG. 37Another example embodiment is shown in which the audio device provides audio feedback to represent the displayed acoustic image data including a plurality of acoustic signals.

[0066] FIG. 38 An example embodiment is shown in which the audio device provides stereo audio feedback to represent the displayed acoustic image data. DETAILED DESCRIPTION

[0067] FIG. 1A And 1B Front and rear views of an example acoustic imaging device are shown. FIG. 1A A front side of an acoustic imaging device 100 is shown having a housing 102 supporting an acoustic sensor array 104 and an electromagnetic imaging tool 106. In some embodiments, the acoustic sensor array 104 includes a plurality of acoustic sensor elements each configured to receive acoustic signals from an acoustic scene and output acoustic data based on the received acoustic signals. The electromagnetic imaging tool 106 can be configured to receive electromagnetic radiation from a target scene and output electromagnetic image data representative of the received electromagnetic radiation. The electromagnetic imaging tool 106 can be configured to detect electromagnetic radiation in one or more of a plurality of wavelength ranges, such as visible light, infrared, ultraviolet, and the like.

[0068] In the illustrated example, the acoustic imaging device 100 includes an ambient light sensor 108 and a position sensor 116, such as a GPS. The device 100 includes a laser pointer 110, which in some embodiments includes a laser rangefinder. The device 100 includes a torch 112, which can be configured to emit visible light radiation toward a scene, and an infrared illuminator 118, which can be configured to emit infrared radiation toward a scene. In some examples, the device 100 can include an illuminator for illuminating a scene in any wavelength range. The device 100 further includes a projector 114, such as an image reprojector, which can be configured to project a generated image onto a scene, such as a color image, and / or a dot projector, which is configured to project a series of dots onto a scene, for example, to determine a depth profile of the scene.

[0069] FIG. 1BThe back of the acoustic imaging device 100 is shown. As shown, the device includes a display 120, which can present images or other data. In some examples, the display 120 includes a touch screen display. The acoustic imaging device 100 includes a speaker, which can provide audio feedback signals to a user, and a wireless interface 124, which can enable wireless communication between the acoustic imaging device 100 and external devices. The device further includes controls 126, which can include one or more buttons, knobs, dials, switches, or other interfacing components to enable a user to interact with the acoustic imaging device 100. In some examples, the controls 126 and the touch screen display combine to provide a user interface for the acoustic imaging device 100.

[0070] In various embodiments, the acoustic imaging device need not include every element shown in the embodiments of FIG. 1A and 1B One or more of the illustrated components can be excluded from the acoustic imaging device. In some examples, one or more components shown in the embodiments of FIG. 1A and 1B may be included as part of the acoustic imaging system, but included separately from the housing 102. Such components can communicate with other components of the acoustic imaging system, for example, using the wireless interface 124, via wired or wireless communication techniques.

[0071] FIG. 2 is a functional block diagram illustrating components of an example of an acoustic analysis system 200. FIG. 2 The example acoustic analysis system 200 can include a plurality of acoustic sensors, such as microphones, MEMS, transducers, etc., arranged in an acoustic sensor array 202. Such an array can be one-dimensional, two-dimensional, or three-dimensional. In various examples, the acoustic sensor array can define any suitable size and shape. In some examples, the acoustic sensor array 202 includes a plurality of acoustic sensors arranged in a grid pattern, such as, for example, an array of sensor elements arranged in longitudinal columns and transverse rows. In various examples, the acoustic sensor array 202 can include arrays of vertical columns of horizontal rows, for example, 8x8, 16x16, 32x32, 64x64, 128x128, 256x256, etc. Other examples are possible, and various sensor arrays need not necessarily include the same number of rows as columns. In some embodiments, such sensors can be positioned on a substrate, for example, such as a printed circuit board (PCB) substrate.

[0072] In FIG. 2In the illustrated configuration, the processor 212 in communication with the acoustic sensor array 202 can receive acoustic data from each of the plurality of acoustic sensors. During exemplary operation of the acoustic analysis system 200, the processor 212 can be in communication with the acoustic sensor array 202 to generate acoustic image data. For example, the processor 212 can be configured to analyze data received from each of the plurality of acoustic sensors arranged in an acoustic sensor array and determine an acoustic scene by "back-propagating" the acoustic signals to their sources. In some embodiments, the processor 212 can generate a digital "frame" of acoustic image data by identifying various source locations and intensities of acoustic signals across a two-dimensional scene. By generating frames of acoustic image data, the processor 212 captures an acoustic image of a target scene at a substantially given point in time. In some examples, a frame includes a plurality of pixels that make up an acoustic image, where each pixel represents a portion of the source scene to which an acoustic signal has been back-propagated.

[0073] The components described as processors within the acoustic analysis system 200, including the processor 212, can be implemented as one or more processors, singly or in any suitable combination of two or more, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic circuits, and the like. The processor 212 can also include memory that stores program instructions and related data that, when executed by the processor 212, cause the acoustic analysis system 200 and the processor 212 to perform the functions attributed to them in this disclosure. The memory can include any fixed or removable magnetic, optical, or electrical media, such as RAM, ROM, CD-ROM, hard or floppy magnetic disks, EEPROM, and the like. The memory can also include removable memory portions that can be used to provide memory updates or increases in memory capacity. The removable memory can also allow acoustic image data to be readily transferred to another computing device, or removed before the acoustic analysis system 200 is used in another application. The processor 212 can also be implemented as a system on a chip that integrates some or all of the components of a computer or other electronic system into a single chip. The processor 212 (processing circuitry) can be configured to communicate processed data to the display 214 or other output / control devices 218.

[0074] In some embodiments, the acoustic sensors in the acoustic sensor array 202 generate a series of signals corresponding to the acoustic signals received by each acoustic sensor to represent an acoustic image. When the signals from each acoustic sensor are obtained by scanning all of the rows that make up the acoustic sensor array 202, a "frame" of acoustic image data is generated. In some examples, the processor 212 can acquire acoustic image frames at a rate sufficient to generate a video representation of the acoustic image data (e.g., 30 Hz or 60 Hz). Independent of particular circuitry, the acoustic analysis system 200 can be configured to manipulate acoustic data representative of the acoustic profile of a target scene to facilitate providing output that can be displayed, stored, transmitted, or otherwise utilized by a user.

[0075] In some embodiments, "back-propagating" the received acoustic signals to generate acoustic image data includes analyzing the received signals at the plurality of acoustic sensors in the acoustic sensor array 202, e.g., via the processor. In various examples, performing back-propagation is a function of one or more parameters, including the distance to the target, the frequency, the sound intensity (e.g., decibel level), the size / configuration of the sensor array, e.g., including the spacing and arrangement of individual sensors within one or more arrays, etc. In some embodiments, such parameters can be preprogrammed into the system, e.g., in memory. For example, the properties of the acoustic sensor array 202 can be stored in memory, such as internal memory or memory specifically associated with the acoustic sensor array 202. Other parameters, such as the distance to the target, can be received in a variety of ways. For example, in some examples, the acoustic analysis system 200 includes a distance measurement tool 204 in communication with the processor 212. The distance measurement tool can be configured to provide distance information representative of the distance from the distance measurement tool 204 to a particular location in the target scene. Various distance measurement tools can include a laser rangefinder or other known distance measurement device, such as other optical or audio distance measurement devices. Additionally or alternatively, the distance measurement tool can be configured to generate three-dimensional depth data such that each portion of the target scene has an associated value for the distance to the target. Thus, in some examples, a measurement of the distance to the target as used herein can correspond to the distance to each location within the target scene. Such three-dimensional depth data can be generated, e.g., via a plurality of imaging tools with different views of the target scene, or via other known distance scanning tools. In general, in various embodiments, the distance measurement tool can be used to perform one or more distance measurement functions, including but not limited to: laser distance measurement, active acoustic wave distance measurement, passive ultrasonic distance measurement, LIDAR distance measurement, RADAR distance measurement, millimeter wave distance measurement, etc.

[0076] Distance information from the distance measurement tool 204 can be used in the back- propagation calculations. Additionally or alternatively, the system 200 can include a user interface 216 into which a user can manually enter a parameter of the distance to the target. For example, if the distance to a component suspected of producing an acoustic signal is known or difficult to measure with the distance measurement tool 204, the user can enter a value for the distance to the target into the system 200.

[0077] In the illustrated embodiment, the acoustic analysis system 200 includes an electromagnetic imaging tool 203 for generating image data representative of a target scene. An exemplary electromagnetic imaging tool can be configured to receive electromagnetic radiation from a target scene and generate electromagnetic image data representative of the received electromagnetic radiation. In some examples, the electromagnetic imaging tool 203 can be configured to generate electromagnetic image data representative of a particular range of wavelengths within the electromagnetic spectrum, such as infrared radiation, visible light radiation, and ultraviolet radiation. For example, in some embodiments, the electromagnetic imaging tool 203 can include one or more camera modules, such as, for example, a visible light camera module 206, configured to generate image data representative of a particular range of wavelengths within the electromagnetic spectrum.

[0078] Visible light camera modules are generally known. For example, various visible light camera modules are included in smartphones and numerous other devices. In some embodiments, the visible light camera module 206 can be configured to receive visible light energy from a target scene and focus the visible light energy on a visible light sensor for generating visible light energy data, which can be displayed on a display 214 and / or stored in memory, for example, in the form of a visible light image. The visible light camera module 206 can include any suitable components for carrying out the functions attributed to the module herein. In some embodiments, the visible light camera module 206 can include a lens, a visible light sensor, and / or other components. FIG. 2In the example of visible light camera module 206, it is illustrated as including a visible light lens assembly 208 and a visible light sensor 210. In some such embodiments, visible light lens assembly 208 includes at least one lens that captures visible light energy emitted by a target scene and focuses the visible light energy on visible light sensor 210. Visible light sensor 210 can include a plurality of visible light sensor elements, such as, for example, CMOS detectors, CCD detectors, PIN diodes, avalanche photodiodes, and the like. Visible light sensor 210 responds to the focused energy by generating electrical signals that can be converted and displayed as a visible light image on display 214. In some examples, visible light module 206 is configurable by a user and can provide output, for example, to display 214 in a variety of formats. Visible light camera module 206 can include compensation functions for various lighting or other operating conditions or user preferences. Visible light camera module can provide a digital output including image data that can include data in a variety of formats (e.g., RGB, CYMK, YCbCr, etc.).

[0079] In operation of certain example visible light camera module 206, light energy received from a target scene can pass through visible light lens assembly 208 and be focused on visible light sensor 210. When the light energy strikes the visible light sensor elements of visible light sensor 210, photons within the photodetector can be released and converted into a detection current. Processor 212 can process the detection current to form a visible light image of the target scene.

[0080] During use of the acoustic analysis system 200, the processor 212 can control the visible light camera module 206 to generate visible light data from a captured target scene for use in creating a visible light image. The visible light data can include luminosity data indicative of color(s) associated with different portions of the captured target scene and / or magnitude values of light associated with different portions of the captured target scene. The processor 212 can generate a "frame" of visible light image data by measuring the response of each visible light sensor element of the acoustic analysis system 200 at a single time. By generating frames of visible light data, the processor 212 captures a visible light image of the target scene at a given point in time. The processor 212 can also repeatedly measure the response of each visible light sensor element of the acoustic analysis system 200 in order to generate a dynamic visible light image (e.g., a video representation) of the target scene. In some examples, the visible light camera module 206 can include its own dedicated processor or other circuitry (e.g., ASIC) capable of operating the visible light camera module 206. In some such embodiments, the dedicated processor communicates with the processor 212 for providing visible light image data (e.g., RGB image data) to the processor 212. In alternative embodiments, the dedicated processor for the visible light camera module 206 can be integrated into the processor 212.

[0081] In instances where each sensor element of the visible light camera module 206 functions as a sensor pixel, the processor 212 can generate a two-dimensional image or picture representation of visible light from the target scene by converting the electrical response of each sensor element into a time-multiplexed electrical signal, which can be processed, e.g., for visualization on the display 214 and / or storage in memory.

[0082] The processor 212 can control the display 214 to display at least a portion of the visible light image of the captured target scene. In some examples, the processor 212 controls the display 214 such that the electrical response of each sensor element of the visible light camera module 206 is associated with a single pixel on the display 214. In other examples, the processor 212 can increase or decrease the resolution of the visible light image such that more or fewer pixels than are present in the visible light camera module 206 are displayed on the display 214. The processor 212 can control the display 214 to display the entire visible light image (e.g., the entirety of the portion of the target scene captured by the acoustic analysis system 200) or less than the entire visible light image (e.g., a less portion of the entire target scene captured by the acoustic analysis system 200).

[0083] In some embodiments, the processor 212 can control the display 214 to concurrently display at least a portion of a visible light image captured by the acoustic analysis system 200 and at least a portion of an acoustic image generated via the acoustic sensor array 202. Such concurrent display can be useful because an operator can reference features displayed in the visible light image to help view sources of acoustic signals concurrently displayed in the acoustic image. In various examples, the processor 212 can control the display 214 to display the visible light image and the acoustic image in a side-by-side arrangement, in an image-in-image arrangement (where one of the images surrounds the other of the images), or in any other suitable arrangement in which the visible light image and the acoustic image are concurrently displayed.

[0084] For example, the processor 212 can control the display 214 to display the visible light image and the acoustic image in the form of a combined arrangement. In such an arrangement, for a pixel or set of pixels in the visible light image that represents a portion of the target scene, there is a corresponding pixel or set of pixels in the acoustic image that represents substantially the same portion of the target scene. In various embodiments, the size and / or resolution of the acoustic image and the visible light image need not be the same. Accordingly, there can be a set of pixels in one of the acoustic image or the visible light image that correspond to a single pixel in the other of the acoustic image or the visible light image, or a set of pixels of different sizes. Similarly, there can be an image in one of the visible light image or the acoustic image that corresponds to a set of pixels in the other image. Thus, as used herein, correspondence does not require a one-to-one pixel relationship, but can include a mismatched size of pixels or groups of pixels. Various combination techniques for mismatched size image regions can be implemented, such as upsampling or downsampling one of the images, or combining pixels with an average value of a corresponding set of pixels. Other examples are known and are also within the scope of the present disclosure.

[0085] Accordingly, corresponding pixels need not have a direct one-to-one relationship. Rather, in some embodiments, a single acoustic pixel has multiple corresponding visible light pixels, or a visible light pixel has multiple corresponding acoustic pixels. Additionally or alternatively, in some embodiments, not all visible light pixels have a corresponding acoustic pixel, or vice versa. Such embodiments can indicate, for example, a picture-in-picture type of display as previously discussed. Thus, a visible light pixel will not necessarily have the same pixel coordinates within the visible light image as a corresponding acoustic pixel has. Accordingly, as used herein, a corresponding pixel generally refers to a pixel from any image (e.g., a visible light image, an acoustic image, a combined image, a display image, etc.) that includes information from substantially the same portion of the target scene. Such pixels need not have a one-to-one relationship between images, and need not have similar coordinate locations within their respective images.

[0086] Similarly, images having corresponding pixels (i.e., pixels representing the same portion of the target scene) can be referred to as corresponding images. Thus, in some such arrangements, corresponding visible light and acoustic images can be superimposed on one another at corresponding pixels. An operator can interact with the user interface 216 to control the transparency or opacity of one or both of the images displayed on the display 214. For example, the operator can interact with the user interface 216 to adjust the acoustic image between full transparency and full opacity, and also to adjust the visible light image between full transparency and full opacity. Such an exemplary combined arrangement (which can be referred to as an alpha blending arrangement) can allow the operator to adjust the display 214 to display only the acoustic image, only the visible light image, or any overlapping combination of the two images between the extremes of only the acoustic image and only the visible light image. The processor 212 can also combine scene information with other data, such as alert data, and so on. In general, the alpha blended combination of the visible light and acoustic images can include any case from 100% acoustic and 0% visible light to 0% acoustic and 100% visible light. In some embodiments, the amount of blending can be adjusted by a user of the camera. Thus, in some embodiments, the blended image can be adjusted between 100% visible light and 100% acoustic.

[0087] Additionally, in some embodiments, the processor 212 can interpret and execute commands from the user interface 216 and / or output / control device 218. Furthermore, the processing of visible light and / or acoustic image data occurring in the processor 212 can be altered using input signals.

[0088] An operator can interact with the acoustic analysis system 200 via a user interface 216, which can include buttons, keys, or another mechanism for receiving input from a user. The operator can receive output from the acoustic analysis system 200 via a display 214. The display 214 can be configured to display acoustic images and / or visible light images in any acceptable color palette or color scheme, and the color palette can be varied, for example, in response to user controls. In some embodiments, acoustic image data can be presented in a color palette so as to represent varying magnitudes of acoustic data from different locations in a scene. For example, in some examples, the display 214 is configured to display acoustic images in a monochrome color palette, such as grayscale. In other examples, the display 214 is configured to display acoustic images in a color color palette, such as, for example, amber, iron bow, blue-red, or other high-contrast color schemes. Combinations of grayscale and color palette displays are also contemplated. In some examples, a display configured to display such information can include processing capabilities for generating and presenting such image data. In other examples, being configured to display such information can include the ability to receive image data from other components, such as the processor 212. For example, the processor 212 can generate values for each pixel to be displayed (e.g., RGB values, grayscale values, or other display options). The display 214 can receive such information and map each pixel into a visual display.

[0089] While the processor 212 can control the display 214 to display at least a portion of an acoustic image and at least a portion of a visible light image simultaneously in any suitable arrangement, the side-by-side arrangement of the figures can help an operator to easily focus on and / or interpret the acoustic image by displaying corresponding visible images of the same scene in an adjacent alignment.

[0090] A power source (not shown) delivers operating power to the various components of the acoustic analysis system 200. In various examples, the power source can include a rechargeable or non-rechargeable battery and power generation circuitry, an AC power source, an inductive power pick-up, a photovoltaic power source, or any other suitable component for providing power to the device. Combinations of power source types are also possible.

[0091] During operation of the acoustic analysis system 200, the processor 212 controls the acoustic sensor array 202 and the visible light camera module 206, with the aid of instructions associated with program information stored in memory, to generate visible light images and acoustic images of a target scene. The processor 212 further controls the display 214 to display the visible light images and / or acoustic images generated by the acoustic analysis system 200.

[0092] As noted above, in some cases, it can be difficult to identify and distinguish real-world (visible) features of a target scene in an acoustic image. In some embodiments, in addition to supplementing the acoustic image with visible light information, it can be useful to emphasize visible edges within a target scene. In some embodiments, known edge detection methods can be carried out on a visible light image of a target scene. Due to the correspondence between the acoustic image and the visible light image, visible light pixels determined to represent visible edges in the target scene correspond to acoustic pixels that also represent visible edges in the acoustic image. It will be appreciated that, as used herein, an "edge" does not necessarily refer to a physical boundary of an object, but can refer to any sufficiently sharp gradient in a visible light image. Examples can include physical boundaries of objects, color changes within objects, shadows across a scene, and the like.

[0093] Although generally described with reference to FIG. 2 Although described as including a visible light camera module 206, in some examples, the electromagnetic imaging tool 203 of the acoustic analysis system 200 can additionally or alternatively include imaging tools capable of generating image data representative of a wide variety of wave spectrums. For example, in various examples, the electromagnetic imaging tool 203 can include one or more tools capable of generating infrared image data, visible light image data, ultraviolet image data, or any other useful wavelength or combination thereof. In some embodiments, the acoustic imaging system can include an infrared camera module having an infrared lens assembly and an infrared sensor array. Additional components for interfacing with, for example, the infrared camera module can be included, such as those described in U.S. Patent Application No. 14 / 837,757, entitled "EDGE ENHANCEMENT FOR THERMAL-VISIBLE COMBINED IMAGES AND CAMERAS," and filed August 27, 2015, which is assigned to the assignee of the present application and is hereby incorporated by reference in its entirety.

[0094] In some examples, two or more data streams can be mixed for display. For example, including the visible light camera module 206, the acoustic sensor array 202, and an infrared camera module (not shown) FIG. 2An exemplary system (not shown) can be configured to produce an output image that includes a blend of visible light (VL) image data, infrared (IR) image data, and acoustic image data. In an exemplary blending scheme, the display image can be represented as: a x IR + β x VL + γ x Acoustic, where a + β + γ = 1. In general, any number of data streams can be combined for display. In various embodiments, the blending ratios such as a, β, and γ can be set by a user. Additionally or alternatively, the set display program can be configured to include different image data streams based on an alert condition (e.g., one or more values in one or more data streams meet a predetermined threshold) or other condition, for example, as described in U.S. Patent No. 7,538,326, entitled "VISIBLE LIGHT AND IR COMBINED IMAGE CAMERA WITH A LASER POINTER," assigned to the assignee of the present application and incorporated herein in its entirety by this reference.

[0095] with respect to FIG. 2 One or more components of the acoustic analysis system 200 described can be included in a portable (e.g., handheld) acoustic analysis tool. For example, in some embodiments, a portable acoustic analysis tool can include a housing 230 configured to house the components in the acoustic analysis tool. In some examples, one or more components of the system 200 can be located outside of the housing 230 of the acoustic analysis tool. For example, in some embodiments, the processor 212, the display 214, the user interface 216, and / or the output control device 218 can be located outside of the housing of the acoustic analysis tool and can communicate with various other system components, for example, via wireless communication (e.g., Bluetooth communication, Wi-Fi, etc.). Such components outside of the acoustic analysis tool can be provided, for example, via an external device such as a computer, a smart phone, a tablet device, a wearable device, and the like. Additionally or alternatively, other test and measurement or data acquisition tools configured to act as a master or a slave device with respect to the acoustic analysis tool can similarly provide various components of the acoustic analysis system outside of the acoustic analysis tool. The external device can communicate with the portable acoustic analysis tool via wired and / or wireless connections and can be used to carry out various processing, display, and / or interface steps.

[0096] In some embodiments, such external devices can provide redundant functionality as components housed in the portable acoustic analysis tool. For example, in some embodiments, the acoustic analysis tool can include a display for displaying acoustic image data, and can be further configured to transfer image data to an external device for storage and / or display. Similarly, in some embodiments, a user can interface with the acoustic analysis tool via an application ("app") running on a smart phone, tablet device, computer, or the like, to carry out one or more functions that can also be carried out with the acoustic analysis tool itself.

[0097] FIG. 3A is a schematic diagram of an exemplary configuration of an acoustic sensor array within an acoustic analysis system. In the illustrated example, the acoustic sensor array 302 includes a plurality of first acoustic sensors (shown in white) and a plurality of second acoustic sensors (hatched). The first acoustic sensors are arranged into a first array 320, and the second acoustic sensors are arranged into a second array 322. In some examples, the first array 320 and the second array 322 can be selectively used to receive acoustic signals for generating acoustic image data. For example, in some configurations, the sensitivity of a particular acoustic sensor array to a particular acoustic frequency is a function of the distance between the acoustic sensor elements.

[0098] In some configurations, sensor elements spaced more closely together (e.g., the second array 322) can better resolve high frequency acoustic signals (e.g., sounds with frequencies greater than 20 kHz, such as ultrasonic signals between 20 kHz and 100 kHz) than sensor elements spaced farther apart (e.g., the first array 320). Similarly, sensor elements spaced farther apart (e.g., the first array 320) can be better suited for detecting low frequency acoustic signals (e.g., < 20 kHz) than sensor elements spaced more closely together (e.g., the second array 322). Various acoustic sensor arrays having sensor elements spaced apart from one another can be provided for detecting acoustic signals having various frequency ranges, such as infrasonic frequencies (< 20 Hz), audible frequencies (approximately between 20 Hz and 20 kHz), ultrasonic frequencies (between 20 kHz and 100 kHz). In some embodiments, portions of the array (e.g., every other acoustic sensor element from array 320) can be used to optimize detection of particular frequency bands.

[0099] Additionally, in some examples, some acoustic sensor elements may be better suited to detecting acoustic signals with different frequency characteristics (such as low-frequency or high-frequency). Therefore, in some embodiments, an array configured to detect low-frequency acoustic signals (such as a first array 320 with more widely spaced sensor elements) may include first acoustic sensor elements better suited to detecting low-frequency acoustic signals. Similarly, an array configured to detect higher-frequency acoustic signals (such as a second array 322) may include second acoustic sensor elements better suited to detecting high-frequency acoustic signals. Therefore, in some examples, the first array 320 and the second array 322 of acoustic sensor elements may include different types of acoustic sensor elements. Alternatively, in some embodiments, the first array 320 and the second array 322 may include the same type of acoustic sensor elements.

[0100] Therefore, in an exemplary embodiment, the acoustic sensor array 302 may include multiple arrays of acoustic sensor elements, such as a first array 320 and a second array 322. In some embodiments, the arrays may be used individually or in combination. For example, in some examples, a user may choose to use the first array 320, the second array 322, or both the first array 320 and the second array 322 simultaneously to perform an acoustic imaging process. In some examples, a user may select which array(s)(s) to use via a user interface. Additionally or alternatively, in some embodiments, the acoustic analysis system may automatically select which array(s)(s) to use based on the analysis of received acoustic signals or other input data (such as expected frequency ranges, etc.). Although FIG. 3A The configuration shown typically includes two arrays (first array 320 and second array 322) generally arranged in a rectangular grid, but it will be appreciated that multiple acoustic sensor elements can be grouped into any number of discrete arrays of any shape. Furthermore, in some embodiments, one or more acoustic sensor elements can be included in multiple distinct arrays that can be selected for operation. As described elsewhere herein, in various embodiments, the process for backpropagating acoustic signals to establish acoustic image data from a scene is implemented based on the arrangement of the acoustic sensor elements. Therefore, the arrangement of the acoustic sensors can be known or otherwise accessible to a processor in order to implement acoustic image generation techniques.

[0101] FIG. 3A The acoustic analysis system further includes: a distance measuring tool 304 and a camera module 306 positioned within an acoustic sensor array 302. The camera module 306 may represent the camera module of an electromagnetic imaging tool (e.g., 203) and may include a visible light camera module, an infrared camera module, an ultraviolet camera module, etc. Additionally, although in FIG. 3Anot shown, but the acoustic analysis system can include one or more additional camera modules of the same or different type as the camera module 306. In the illustrated example, the distance measurement tool 304 and the camera module 306 are positioned within the lattices of acoustic sensor elements in the first array 320 and the second array 322. While shown as being disposed between lattice sites within the first array 320 and the second array 322, in some embodiments one or more components (e.g., the camera module 306 and / or the distance measurement tool 304) can be positioned at corresponding one or more lattice sites in the first array 320 and / or the second array 322. In some such embodiments, the component(s) can be positioned at the lattice sites in place of acoustic sensor elements that would ordinarily be in such positions according to the lattice arrangement.

[0102] As described elsewhere herein, the acoustic sensor array can include acoustic sensor elements arranged in any of a wide variety of configurations. FIG. 3B and FIG. 3C are schematic diagrams illustrating example acoustic sensor array configurations. FIG. 3B An acoustic sensor array 390 is shown that includes a plurality of acoustic sensor elements uniformly spaced apart in an approximately square lattice. A distance measurement tool 314 and a camera array 316 are positioned within the acoustic sensor array 390. In the illustrated example, the acoustic sensor elements in the acoustic sensor array 390 are sensors of the same type, although in some embodiments different types of acoustic sensor elements can be used in the array 390.

[0103] FIG. 3C A plurality of acoustic sensor arrays is shown. Acoustic sensor arrays 392, 394, and 396 each include a plurality of acoustic sensor elements arranged in arrays of different shapes. In FIG. 3C In the illustrated example, the acoustic sensor arrays 392, 394, and 396 can be used individually or together in any combination to create sensor arrays of various sizes. In the illustrated embodiment, the sensor elements of the array 396 are more closely spaced together than the sensor elements of the array 392. In some examples, the array 396 is designed to sense high frequency acoustic data, while the array 392 is designed to sense low frequency acoustic data.

[0104] In various embodiments, the arrays 392, 394, and 396 can include acoustic sensor elements of the same or different types. For example, the acoustic sensor array 392 can include sensor elements having a lower frequency operating range than the frequency operating range of the sensor elements of the acoustic sensor array 396.

[0105] As described elsewhere herein, in some examples different acoustic sensor arrays (e.g., 392, 394, 396) can be selectively turned off and on during various modes of operation (e.g., different desired frequency spectrums to be imaged). Additionally or alternatively, various acoustic sensor elements (e.g., some or all acoustic sensor elements in one or more sensor arrays) can be enabled or disabled depending on the desired system operation. For example, in some acoustic imaging processes, while data from a large number of sensor elements (e.g., sensor elements arranged in high density, such as in sensor array 396) slightly improves the resolution of the acoustic image data, this comes at the cost of the required processing to extract the acoustic image data from the data received at each sensor element. That is, in some examples the increased processing requirements (e.g., in terms of cost, processing time, power consumption, etc.) required to process a large number of input signals (e.g., signals from a large number of acoustic sensor elements) is negative compared to any additional signal resolution provided by the additional data stream. Thus, in some embodiments it can be worthwhile to disable or ignore data from one or more acoustic sensor elements depending on the desired acoustic imaging operation.

[0106] Similar to the system of FIG. 3A and FIG. 3B , the system of FIG. 3C includes distance measurement tools 314 and camera arrays 316 positioned within acoustic sensor arrays 392, 394, and 396. In some examples, additional components such as additional camera arrays (e.g., used to image different portions of the electromagnetic spectrum from camera arrays 316) can similarly be positioned within acoustic sensor arrays 392, 394, and 396. It will be appreciated that while shown in FIG. 3C as positioned within one or more acoustic sensor arrays, distance measurement tools and / or one or more imaging tools (e.g., visible light camera modules, infrared camera modules, ultraviolet sensors, etc.) can be located outside the acoustic sensor array(s). In some such examples, distance measurement tools and / or one or more imaging tools located outside the acoustic sensor array(s) can be supported by the acoustic imaging tool, e.g., by a housing that houses the acoustic sensor array(s), or can be located outside the housing of the acoustic imaging tool. FIG. 3A-2

[0107] In some examples, general misalignment of acoustic sensor arrays and imaging tools such as camera modules can result in misalignment in the corresponding image data generated by the acoustic sensor arrays and imaging tools. FIG. 4A ​A diagram showing a parallax error in the generation of frames of visible light image data and acoustic image data is shown. Generally, the parallax error can be vertical, horizontal, or both. In the illustrated embodiment, the acoustic sensor array 420 and the imaging tool includes a visible light camera module 406. The visible light image frame 440 is shown as being captured from a field of view 441 of the visible light camera module 406, while the acoustic image frame 450 is shown as being captured from a field of view 451 of the acoustic sensor array 420.

[0108] As shown, the visible light image frame 440 and the acoustic imaging frame 450 are not aligned with each other. In some embodiments, a processor (e.g., the processor 212 of the FIG. 2 is configured to manipulate one or both of the visible light image frame 440 and the acoustic image frame 450 in order to align the visible light image data and the acoustic image data. Such manipulation can include shifting one image frame relative to the other image frame. The amount by which the image frames are shifted relative to each other can be determined based on a variety of factors, including, for example, a distance from the visible light camera module 406 and / or the acoustic sensor array 420 to the target. Such distance data can be determined, for example, using the distance measurement tool 404 or receiving distance values via a user interface (e.g., 216).

[0109] FIG. 4B is a diagram similar to FIG. 4A , but including a visible light image of the scene. In the example of FIG. 4B , the visible light image 442 shows a scene with multiple power lines and a support tower. The acoustic image 452 includes multiple locations 454, 456, 458 that indicate high value acoustic data from such locations. As shown, both the visible light image 442 and the acoustic image 452 are displayed simultaneously. However, observation of the two images shows at least one acoustic image local maximum at location 458 that does not appear to coincide with any particular structure in the visible light image 442. Thus, a person observing both images can conclude that there is a misalignment (e.g., a parallax error) between the acoustic image 452 and the visible light image 442.

[0110] FIG. 5A and FIG. 5B shows parallax correction between a visible light image and an acoustic image. Similar to FIG. 4B , FIG. 5A shows a visible light image 542 and an acoustic image 552. The acoustic image 552 includes local maxima at locations 554, 556, and 558. As can be seen, the maxima at locations 554 and 558 do not appear to coincide with any structure in the visible light image. In FIG. 5BIn the example of FIG. 5A, the visible light image 542 and the acoustic image 552 are registered with respect to one another. Now, the local maxima at locations 554, 556, and 558 in the acoustic image appear to coincide with respective locations in the visible light image 542.

[0111] In use, an operator can view the representation in FIG. 5B and determine approximate locations in the visible scene 542 that are likely to be sources of the received acoustic signals. Such signals can be further processed in order to determine information about acoustic signatures of various components in the scene. In various embodiments, acoustic parameters such as frequency content, periodicity, amplitude, and the like can be analyzed with respect to various locations in the acoustic image. When overlaid onto the visible light data so that such parameters can be associated with various system components, the acoustic image data can be used to analyze various properties of objects in the visible light image (e.g., performance characteristics).

[0112] FIG. 5C and FIG. 5D are FIG. 5A and FIG. 5B color versions of FIG. 5A and FIG. 5B As shown in FIG. 5C and FIG. 5D it is easier to see in the color representations of FIG. 5A-5D that the circular gradients of locations 554, 556, and 558 are shown in color. As described elsewhere herein, acoustic image data can be visually represented according to a color scheme in which each pixel of the acoustic image data is colored based on the acoustic intensity at the corresponding location. Thus, in the example representation of

[0113] It will be appreciated that although the example illustrations in FIG. 4A , 4B 5A-5D are described with respect to acoustic image data and visible light image data, such processes can similarly be carried out with a wide variety of electromagnetic image data. For example, as described elsewhere herein, in various embodiments, various such processes can be carried out using acoustic image data in combination with one or more of visible light image data, infrared image data, ultraviolet image data, and the like.

[0114] As described elsewhere herein, in some embodiments, backpropagating the acoustic signals to form an acoustic image can be based on a value of a distance to the target. That is, in some examples, the backpropagation calculation can be based on the distance, and can include determining a two-dimensional acoustic scene that is located at the distance from the acoustic sensor array. In the case of a given two-dimensional imaging plane, a cross-section of a spherical acoustic wave emanating from a source in the plane will typically exhibit a circular shape, with the intensity decaying in the radial direction, as FIG. 5A-5B illustrated in FIG. 3.

[0115] In some such examples, the portion of the acoustic scene representing data at the distance to the target that is not used in the backpropagation calculation will result in an error in the acoustic image data, such as an inaccurate location of one or more sounds in the scene. When the acoustic image is displayed simultaneously (e.g., blended, combined, etc.) with other image data (e.g., electromagnetic image data, such as visible, infrared, or ultraviolet image data), such errors can result in parallax errors between the acoustic image data and the other image data. Accordingly, in some embodiments, some techniques for correcting parallax errors (e.g., as illustrated in FIG. 5A and FIG. 5B illustrated) include adjusting the value of the distance to the target used in the backpropagation calculation for generating the acoustic image data.

[0116] In some cases, the system can be configured to perform a backpropagation process using a first value of the distance to the target, and to display a display image such as FIG. 5A illustrated, in which the acoustic image data and the other data stream can be misaligned. Subsequently, the acoustic analysis system can adjust the value of the distance to the target used by the backpropagation, perform the backpropagation again, and update the display image with the new acoustic image data. This process can be repeated, with the acoustic analysis system cycling through a plurality of values of the distance to the target while a user observes the resulting display images on the display. As the values of the distance to the target are varied, the user can observe a transition from a display image such as FIG. 5A illustrated to a display image such as FIG. 5Bgradual transition of the displayed image. In some such cases, the user can visually observe when the acoustic image data appears to be properly registered with another data stream, such as electromagnetic image data. The user can send a signal to the acoustic analysis system indicating that the acoustic image data appears to be properly registered, thereby indicating to the system that the value of the distance to the target that was used to perform the most recent back propagation is approximately correct, and the distance value can be saved to memory as the correct distance to the target. Similarly, when the display image is updated using a new distance value in an updated back propagation process, the user can manually adjust the value of the distance to the target until the user observes that the acoustic image data is properly registered. The user can choose to save the current distance to the target in the acoustic analysis system as the current distance to the target.

[0117] In some examples, correcting parallax error can include adjusting the orientation of the acoustic image data relative to other image data (e.g., electromagnetic image data) by a predetermined amount and in a predetermined direction based on the data of the distance to the target. In some embodiments, such adjustment is independent of the generation of the acoustic image data by back propagating the acoustic signals to the identified distance to the target.

[0118] In some embodiments, the value of the distance to the target can be used to make other determinations in addition to being used to generate acoustic image data and reduce parallax error between acoustic image data and other image data. For example, in some examples, the processor (e.g., 212) can use the value of the distance to the target in order to focus or assist a user in focusing an image, such as an infrared image, as described in U.S. Patent No. 7,538,326, which is incorporated by reference. As described therein, this can similarly be used to correct parallax error between visible light image data and infrared image data. Thus, in some examples, the distance value can be used to register acoustic image data with electromagnetic imaging data, such as infrared image data and visible light image data.

[0119] As described elsewhere herein, in some examples, the distance measurement tool (e.g., 204) is configured to provide distance information that can be used by the processor (e.g., 212) to generate and register acoustic image data. In some embodiments, the distance measurement tool includes a laser rangefinder configured to emit light onto a target scene at a location whose distance is being measured. In some such examples, the laser rangefinder can emit light in the visible spectrum so that a user can view the laser point in the physical scene to ensure that the rangefinder is measuring the distance to the desired portion of the scene. Additionally or alternatively, the laser rangefinder is configured to emit light in a spectrum to which one or more imaging components (e.g., camera modules) are sensitive. Thus, a user viewing the target scene via the analysis tool (e.g., via display 214) can observe the laser point in the scene to ensure that the laser is measuring the distance to the correct location in the target scene. In some examples, the processor (e.g., 212) can be configured to generate a reference marker in the displayed image representative of the location at which the laser point would be located in the acoustic scene based on the current distance value (e.g., based on a known distance-based parallax relationship between the laser rangefinder and the acoustic sensor array). The location of the reference marker can be compared to the location of the actual laser marker (e.g., graphically on the display and / or physically in the target scene) and the scene can be adjusted until the reference marker and the laser coincide. Such a process can be carried out similar to the infrared registration and focusing techniques described in U.S. Patent No. 7,538,326, which is incorporated by reference.

[0120] FIG. 6 is a process flow diagram illustrating an example method for generating a final image that combines acoustic image data and electromagnetic image data. The method includes the steps of receiving acoustic signals via an acoustic sensor array (680) and receiving distance information (682). The distance information can be received, for example, via a distance measurement device and / or a user interface, such as via manual input, or as a result of a distance adjustment process by which distances are determined based on observed registration.

[0121] The method further includes backpropagating the received acoustic signals to determine acoustic image data representative of an acoustic scene (684). As described elsewhere herein, backpropagation can include analyzing a plurality of acoustic signals received at a plurality of sensor elements in the acoustic sensor array in conjunction with received distance information to determine a source pattern for the received acoustic signals.

[0122] FIG. 6The method of further includes the steps of: capturing electromagnetic image data (686), and registering the acoustic image data with the electromagnetic image data (688). In some embodiments, registering the acoustic image data with the electromagnetic image data is performed as part of the backpropagation step for generating the acoustic image data (684). In other examples, registering the acoustic image data with the electromagnetic image data is performed separately from the generation of the acoustic image data.

[0123] FIG. 6 The method of includes the step of: combining the acoustic image data with the electromagnetic image data to generate a display image (690). As described elsewhere herein, combining the electromagnetic image data and the acoustic image data can include alpha blending the electromagnetic image data and the acoustic image data. Combining the image data can include overlaying one image data set onto another image data set, such as in a picture-in-picture mode or in locations that meet certain conditions (e.g., alert conditions). The display image can be presented to a user, for example, via a display supported by a housing that supports the acoustic sensor array, and / or via a display separate from the sensor array, such as a display of an external device (e.g., a smartphone, tablet device, computer, etc.).

[0124] Additionally or alternatively, the display image can be saved in local (e.g., on-board) memory and / or remote memory for future viewing. In some embodiments, the saved display image can include metadata that allows future adjustment of display image properties, such as blending ratios, backpropagation distances, or other parameters used to generate the image. In some examples, the raw acoustic signal data and / or the electromagnetic image data can be saved with the display image for subsequent processing or analysis.

[0125] While shown as a method for generating a final image that combines acoustic image data and electromagnetic image data, it will be appreciated that, FIG. 6 The method of can be used to combine acoustic image data with one or more image data sets spanning any portion of the electromagnetic spectrum, such as visible light image data, infrared image data, ultraviolet image data, and so on. In some such examples, multiple image data sets, such as visible light image data and infrared image data, can be combined with acoustic image data to generate a display image via similar methods as described with respect to FIG. 6

[0126] In some examples, receiving acoustic signals via a sensor array (680) can include the step of: selecting an acoustic sensor array with which to receive the acoustic signals. As described, for example, with respect to FIG. 3A ​As described above with respect to FIG. 3, the acoustic analysis system can include multiple acoustic sensor arrays that can be adapted to analyze acoustic signals having varying frequencies. Additionally or alternatively, in some examples, different acoustic sensor arrays can be used to analyze acoustic signals propagating from different distances. In some embodiments, different arrays can be nested inside one another. Additionally or alternatively, portions of the arrays can be selectively used to receive acoustic image signals.

[0127] For example, FIG. 3A A first array 320 and a second array 322 nested inside the first array are shown. In example embodiments, the first array 320 can include a sensor array configured (e.g., spaced apart) to receive acoustic signals and generate acoustic image data for frequencies within a first frequency range. The second array 322 can include, for example, a second sensor array configured to be used separately, or in combination with all or a portion of the first array 320, to generate acoustic image data for frequencies within a second frequency range.

[0128] Similarly, FIG. 3C A first array 392, a second array 394 at least partially nested within the first array 392, and a third array 396 at least partially nested within the first array 392 and the second array 394 are shown. In some embodiments, the first array 392 can be configured to receive acoustic signals and generate acoustic image data for frequencies within a first frequency range. The second array 394 can be used with all or a portion of the first array 392 to receive acoustic signals and generate acoustic image data for frequencies within a second frequency range. The third array 396 can be used separately, with all or a portion of the second array 394, and / or with all or a portion of the first array 392 to receive acoustic signals and generate acoustic image data for frequencies within a third frequency range.

[0129] In some embodiments, in a nested array configuration, acoustic sensor elements from one array can be positioned between acoustic sensor elements, such as elements of the third array 396 are generally located between elements of the first array 392. In some such examples, acoustic sensor elements in a nested array (e.g., the third array 396) can be positioned in the same plane as, in front of, or behind acoustic sensor elements in the array in which it is nested (e.g., the first array 392).

[0130] In various implementations, arrays used to sense higher frequency acoustic signals generally require smaller distances between individual sensors. Accordingly, with respect to FIG. 3CFor example, the third array 396 can be better suited to perform an acoustic imaging process involving high frequency acoustic signals. Other sensor arrays (e.g., the first array 392) can be sufficient to perform an acoustic imaging process involving lower frequency signals, and can be used when compared to the array 396 in order to reduce the computational demand of processing signals from a smaller number of acoustic sensor elements. Thus, in some examples, a high frequency sensor array can be nested within a low frequency sensor array. Such arrays can generally be operated individually (e.g., via switching between active arrays) or together, as described elsewhere herein.

[0131] In addition to or as an alternative to selecting an appropriate sensor array based on an expected / desired spectrum for analysis, in some examples, different sensor arrays can be better suited to perform an acoustic imaging process at different distances to a target scene. For example, in some embodiments, if the distance between an acoustic sensor array and a target scene is small, then outer sensor elements in the acoustic sensor array can receive significantly less useful acoustic information from the target scene than sensor elements located more centrally.

[0132] On the other hand, if the distance between an acoustic sensor array and a target scene is large, then closely spaced acoustic sensor elements can not individually provide useful information. That is, if a first and second acoustic sensor element are close together, and the target scene is generally far away, then the second acoustic sensor element can not provide any information that is different in a meaningful sense from the first acoustic sensor element. Thus, the data stream from such a first sensor element and second sensor element can be redundant, and unnecessarily consume processing time and resources for analysis.

[0133] As described elsewhere herein, in addition to affecting which sensor arrays can be best suited to perform acoustic imaging, the distance to a target can be used to perform back propagation for determining acoustic image data from received acoustic signals. However, in addition to being an input value to a back propagation algorithm, the distance to a target can be used to select an appropriate back propagation algorithm to use. For example, in some examples, at a large distance, a spherical propagating acoustic wave can be approximated as substantially planar compared to the size of an acoustic sensor array. Thus, in some embodiments, when the distance to a target is large, back propagation of received acoustic signals can include an acoustic beamforming calculation. However, when closer to the source of the acoustic wave, the planar approximation of the acoustic wave can not be appropriate. Thus, a different back propagation algorithm can be used, such as near-field acoustic holography.

[0134] As described, the measure of distance to the target can be used in a variety of ways in the acoustic imaging process, such as determining the active sensor array(s), determining the backpropagation algorithm, implementing the backpropagation algorithm, and / or registering the resulting acoustic image with electromagnetic image data (e.g., visible light, infrared, etc.). FIG. 7 is a process flow diagram illustrating an example process for generating acoustic image data from received acoustic signals.

[0135] FIG. 7 The process of includes receiving distance information (780), such as via a user interface, for example, from a distance measurement device or inputted distance information. The method further includes the step of selecting one or more acoustic sensor arrays for implementing acoustic imaging based on the received distance information (782). As described, in various examples, the selected array(s) can include a single array, a combination of multiple arrays, or a portion of one or more arrays.

[0136] FIG. 7 The method of further includes the step of selecting a processing scheme for implementing acoustic imaging based on the received distance information (784). In some examples, selecting a processing scheme can include selecting a backpropagation algorithm for generating acoustic image data from acoustic signals.

[0137] After selecting the acoustic sensor array (782) and the processing scheme for implementing acoustic imaging (784), the method includes the step of receiving acoustic signals via the selected acoustic sensor array (786). The received acoustic signals are then backpropagated using the distance and the selected processing scheme to determine acoustic image data (788).

[0138] In various embodiments, FIG. 7 The steps of can be implemented by a user, an acoustic analysis system (e.g., via processor 212), or a combination thereof. For example, in some embodiments, the processor can be configured to receive distance information (780) via a distance measurement tool and / or user input. In some examples, for example, if the distance to the object is known and / or difficult to analyze via a distance measurement tool (e.g., the object size is small and / or the distance to the target is large, etc.), a user can input a value to override the measured distance to be used as distance information. The processor can be further configured to automatically select an appropriate acoustic sensor array for implementing acoustic imaging based on the received distance information, for example, using a lookup table or other database. In some embodiments, selecting an acoustic sensor array includes enabling and / or disabling one or more acoustic sensor elements in order to obtain the desired acoustic sensor array.

[0139] Similarly, in some examples, the processor can be configured to automatically select a processing scheme (e.g., a backpropagation algorithm) for carrying out acoustic imaging based on the received distance information. In some such examples, this can include selecting one from a plurality of known processing schemes stored in memory. Additionally or alternatively, selecting a processing scheme can be equivalent to adjusting portions of a single algorithm to obtain a desired processing scheme. For example, in some embodiments, a single backpropagation algorithm can include a plurality of terms and variables (e.g., based on distance information). In some such examples, selecting a processing scheme (784) can include defining one or more values in the single algorithm, such as adjusting coefficients of one or more terms (e.g., setting various coefficients to zero or one, etc.).

[0140] Accordingly, in some embodiments, an acoustic imaging system can automate several steps of the acoustic imaging process by suggesting and / or automatically implementing a selected acoustic sensor array and / or processing scheme (e.g., a backpropagation algorithm) based on received distance data. This can speed up, improve, and simplify the acoustic imaging process, thereby eliminating the requirement for an acoustic imaging specialist to perform the acoustic imaging process. Accordingly, in various examples, an acoustic imaging system can automatically implement such parameters, notify a user that such parameters are about to be implemented, request permission from a user to implement such parameters, suggest such parameters for manual input by a user, etc.

[0141] The automatic selection and / or suggestion of such parameters (e.g., processing schemes, sensor arrays) can be useful for optimizing the localization of acoustic image data relative to other forms of image data, processing speed, and analysis of acoustic image data. For example, as described elsewhere herein, accurate backpropagation determinations (e.g., using appropriate algorithms and / or accurate distance metrics) can reduce parallax errors between acoustic image data and other (e.g., electromagnetic waves, such as visible light, infrared, etc.) image data. Additionally, utilizing appropriate algorithms and / or sensor arrays, such as can be automatically selected or suggested by an acoustic analysis system, can optimize the accuracy of thermal image data, thereby allowing for analysis of received acoustic data.

[0142] As described, in some examples, an acoustic analysis system can be configured to automatically select an algorithm and / or sensor array for carrying out an acoustic imaging process based on received distance information. In some such embodiments, the system includes a lookup table, e.g., stored in memory, for determining which of a plurality of backpropagation algorithms and acoustic sensor arrays to use to determine acoustic image data. FIG. 8 An exemplary lookup table for determining appropriate algorithms and sensor arrays to use during an acoustic imaging process is shown.

[0143] In the illustrated example, FIG. 8The lookup table consists of N columns, each representing a different array: array 1, array 2, ..., array N. In various examples, each array comprises a unique set of arranged acoustic sensor elements. Different arrays may include sensor elements arranged in a grid (e.g., ...). FIG. 3C (Arrays 392 and 396 in the lookup table). Arrays within the lookup table may also include combinations of sensor elements from one or more such grids. Generally, in some embodiments, each of the arrays: array 1, array 2, ..., array N corresponds to a unique combination of acoustic sensor elements. Some of these combinations may include the entire set of sensor elements arranged in a particular grid, or may include a subset of sensor elements arranged in a particular grid. Any of the various combinations of acoustic sensor elements is a possible option for use as a sensor array in the lookup table.

[0144] FIG. 8 The lookup table further comprises M rows, each representing a different algorithm: Algorithm 1, Algorithm 2, ..., Algorithm M. In some examples, the different algorithms may include different procedures for performing backpropagation analysis on the received acoustic signal. As described elsewhere in this document, in some examples, some different algorithms may be similar to each other while having different coefficients and / or terms for modifying the backpropagation results.

[0145] FIG. 8 An exemplary lookup table includes M×N entries. In some embodiments, an acoustic analysis system utilizing such a lookup table is configured to analyze received distance information and classify the distance information into one of M×N bins, wherein each bin corresponds to FIG. 8 The lookup table contains entries. In such an example, when the acoustic analysis system receives distance information, it can find the entry (i, j) corresponding to the bin containing the distance information in the lookup table and determine the appropriate algorithm and sensor array for use during the acoustic imaging process. For example, if the received distance information corresponds to the bin associated with entry (i, j), the acoustic analysis system can automatically utilize or suggest the use of algorithm i and array j for the acoustic imaging process.

[0146] In various such examples, the distance information bins can correspond to uniformly sized distance ranges, e.g., a first bin corresponding to distances within one foot, a second bin corresponding to distances between one foot and two feet, etc. In other examples, the bins need not correspond to uniformly sized distance spans. Additionally, in some embodiments, fewer than M x N bins can be used. For example, in some embodiments, there can be an algorithm (e.g., Algorithm x) that is never used for a particular array (e.g., Array y). Thus, in such examples, there would be no corresponding distance information bin in the M x N lookup table corresponding to entry (x, y).

[0147] In some embodiments, statistical analysis of the populated distance bins can be used to identify the most common distance or range of distances within the target scene. In some such embodiments, the distance bin with the highest number of corresponding locations (e.g., the highest number of locations with acoustic signals) can be used as the distance information in the process of FIG. 7 In some embodiments, the acoustic sensor array and / or processing scheme utilized can be implemented and / or recommended based on statistical analysis of the distance distribution of various objects in the target scene. This can increase the likelihood that the sensor array and / or processing scheme used to acoustic image the scene is appropriate for the largest number of locations within the acoustic scene.

[0148] Additionally or alternatively, parameters other than distance information can be used to select an appropriate sensor array and / or processing scheme to use in generating acoustic image data. As described elsewhere herein, various sensor arrays can be configured to be sensitive to certain frequencies and / or frequency bands. In some examples, different reverse propagation calculations can be used similarly depending on different acoustic signal frequency content. Thus, in some examples, one or more parameters can be used to determine a processing scheme and / or acoustic sensor array.

[0149] In some embodiments, the acoustic analysis system can be used to initially analyze various parameters of the received acoustic signal processing / analysis. Referring back to FIG. 7 , the method for generating acoustic image data can include the step of, after receiving the acoustic signal (786), analyzing the frequency content of the received signal (790). In some such examples, if the acoustic sensor array(s) and / or processing scheme have been selected (e.g., via steps 782 and / or 784, respectively), the method can include the step of updating the selected array(s) and / or updating the selected processing scheme (792), e.g., based on the analyzed frequency content.

[0150] After the sensor array(s) and / or processing scheme are updated, the method can use the updated parameters to carry out various actions. For example, if the selected sensor array(s) are updated based on the analyzed frequency content (790), new acoustic signals can be received from the (recently) selected acoustic sensor array (786), which can then be back-propagated to determine acoustic image data (788). Alternatively, if the processing scheme is updated at 792, already captured acoustic signals can be back-propagated according to the updated processing scheme to determine updated acoustic image data. If both the processing scheme and the sensor array(s) are updated, new acoustic signals can be received using the updated sensor array, and the new acoustic signals can be back-propagated according to the updated processing scheme.

[0151] In some embodiments, the acoustic analysis system can receive frequency information (778) without analyzing the frequency content of the received acoustic signals (790). For example, in some examples, the acoustic analysis system can receive information about a desired or expected frequency range for future acoustic analysis. In some such examples, the desired or expected frequency information can be used to select one or more sensor arrays and / or a processing scheme that is best suited for that frequency information. In some such examples, the step(s) of selecting acoustic sensor array(s) (782) and / or selecting a processing scheme (784) can be based on the received frequency information in addition to or instead of the received distance information.

[0152] In some examples, received acoustic signals (e.g., received via acoustic sensor elements) can be analyzed, e.g., via a processor (e.g., 210) of the acoustic analysis system. Such analysis can be used to determine one or more properties of the acoustic signals, such as frequency, intensity, periodicity, apparent proximity (e.g., distance estimated based on the received acoustic signals), measured proximity, or any combination thereof. In some examples, acoustic image data can be filtered, e.g., to show only acoustic image data representing acoustic signals with particular frequency content, periodicity, etc. In some examples, any number of such filters can be applied simultaneously.

[0153] As described elsewhere herein, in some embodiments, a series of acoustic image data frames can be captured over time, similar to acoustic video data. Additionally or alternatively, even without repeated generation of acoustic image data, in some examples, acoustic signals are repeatedly sampled and analyzed. Thus, with or without repeated acoustic image data generation (e.g., video), parameters of the acoustic data (such as frequency) can be monitored over time.

[0154] FIG. 9A is an exemplary plot of the frequency content of image data received over time in an acoustic scene. As shown, FIG. 9A The plot of represents an acoustic scene that typically includes four frequencies that are persistent over time, labeled as frequency 1, frequency 2, frequency 3, and frequency 4. The frequency data, such as the frequency content of the target scene, can be determined via processing of the received acoustic signals, for example, using a Fast Fourier Transform (FFT) or other known frequency analysis method.

[0155] FIG. 9B An exemplary scene is shown that includes multiple locations from which acoustic signals are emitted. In the illustrated image, acoustic image data is combined with visible light image data, and acoustic signals present at locations 910, 920, 930, and 940 are shown. In some embodiments, the acoustic analysis system is configured to display acoustic image data belonging to any detected frequency range. For example, in an exemplary embodiment, location 910 includes acoustic image data containing frequency 1, location 920 includes acoustic image data containing frequency 2, location 930 includes acoustic image data containing frequency 3, and location 940 includes acoustic image data containing frequency 4.

[0156] In some such examples, displaying acoustic image data representative of a frequency range is a selectable mode of operation. Similarly, in some embodiments, the acoustic analysis system is configured to display acoustic image data representative of frequencies within only a predetermined frequency band. In some such examples, displaying acoustic image data representative of a predetermined frequency range includes selecting one or more acoustic sensor arrays for receiving acoustic signals from which the acoustic image data is generated. Such arrays can be configured to receive selective frequency ranges. Similarly, in some examples, one or more filters can be employed to limit the frequency content used to generate acoustic image data. Additionally or alternatively, in some embodiments, acoustic image data including information representative of a broad frequency range can be analyzed and shown on a display only if the acoustic image data meets predetermined conditions, for example, falls within a predetermined frequency range.

[0157] FIG. 9CThe multiple combined acoustic and visible light image data at multiple predefined frequency ranges is shown. The first image includes acoustic image data at a first location 910 that includes frequency content for frequency 1. The second image includes acoustic image data at a second location 920 that includes frequency content for frequency 2. The third image includes acoustic image data at a third location 930 that includes frequency content for frequency 3. The fourth image includes acoustic image data at a fourth location 940 that includes frequency content for frequency 4.

[0158] In example embodiments, a user can select various frequency ranges, such as a range that includes frequency 1, frequency 2, frequency 3, or frequency 4, for filtering acoustic image data that represents frequency content other than the selected frequency range. Thus, in such examples, any of the first, second, third, or fourth images can be displayed as a result of the user selecting a desired frequency range.

[0159] Additionally or alternatively, in some examples, the acoustic analysis system can cycle between multiple display images, each having different frequency content. For example, with respect to FIG. 9C In example embodiments, the acoustic analysis system can display the first, second, third, and fourth images in sequence, such as indicated by the arrows in FIG. 9C

[0160] In some examples, the display images can include text or other displays that represent the frequency content displayed in the images, so that a user can observe which locations in the images include acoustic image data that represents certain frequency content. For example, with respect to FIG. 9C Each image can show a textual representation of the frequencies represented in the acoustic image data. With respect to FIG. 9B The image showing multiple frequency ranges can include an indication of the frequency content at each location, including acoustic image data, in some such examples, a user can select a location in the image, e.g., via a user interface, for which the frequency content present at that location is to be viewed in the acoustic scene. For example, the user can select the first location 910, and the acoustic analysis system can present the frequency content of the first location (e.g., frequency 1). Thus, in various examples, a user can use the acoustic analysis system in order to analyze the frequency content of an acoustic scene, such as by viewing what in the scene corresponds to particular frequency content, and / or by viewing what frequency content is present at various locations.

[0161] ​During example acoustic imaging operations, filtering acoustic image data by frequency can help reduce image clutter from, for example, background or other unimportant sounds. During example acoustic imaging procedures, a user can wish to eliminate background sounds, such as a noise floor in an industrial environment. In some such cases, the background noise can primarily include low frequency noise. Accordingly, a user can select to show acoustic image data representative of acoustic signals greater than a predetermined frequency (e.g., 10 kHz). In another example, a user can wish to analyze a particular object that typically emits acoustic signals within a certain range, such as a corona discharge from a transmission line (e.g., as shown in FIG. 5D). In such an example, a user can select a particular frequency range (e.g., between 11 kHz and 14 kHz for a corona discharge) for acoustic imaging. FIG. 5A

[0162] In some examples, an acoustic analysis system can be used to analyze and / or present information associated with the intensity of received acoustic signals. For example, in some embodiments, backpropagating received acoustic signals can include determining acoustic intensity values at multiple locations in an acoustic scene. In some examples, similar to the frequency information described above, acoustic image data can only be included in a display image if the intensity of acoustic signals meets one or more predetermined requirements.

[0163] In various such embodiments, a display image can include acoustic image data representative of acoustic signals above a predetermined threshold (e.g., 15 dB), below a predetermined threshold (e.g., 100 dB), or within a predetermined intensity range (e.g., between 15 dB and 40 dB). In some embodiments, a threshold can be based on a statistical analysis of an acoustic scene, such as above or below a standard deviation from an average acoustic intensity.

[0164] Similar to the descriptions above regarding frequency information, in some embodiments, limiting acoustic image data to represent acoustic signals that meet one or more intensity requirements can include filtering received acoustic signals such that only received signals that meet predetermined conditions are used to generate acoustic image data. In other examples, acoustic image data is filtered to adjust which acoustic image data is displayed.

[0165] Additionally or alternatively, in some embodiments, acoustic intensity at a location within an acoustic scene can be monitored over time (e.g., in conjunction with a video acoustic image representation or via background analysis, without necessarily updating a display image). In some such examples, a predetermined requirement for displaying acoustic image data can include an amount of change or a rate of change in acoustic intensity at a certain location in an image.

[0166] FIG. 10A and 10B ​It is an exemplary display image that includes the combined visible light image data and acoustic image data. FIG. 10A A display image including acoustic image data is shown, illustrated at multiple locations 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090. In some examples, intensity values ​​may be color-coded, for example, where a color is assigned to the sound intensity values ​​based on a predetermined color scheme. In an exemplary embodiment, intensity values ​​may be categorized according to intensity ranges (e.g., 10 dB–20 dB, 20 dB–30 dB, etc.). Each intensity range may be associated with a specific color according to the color scheme. The acoustic image data may include multiple pixels, wherein each pixel is colored with a color associated with the intensity range to which the intensity represented by the pixel of the acoustic image data falls. In addition to or as an alternative to color differentiation, different intensities may be distinguished according to other attributes, such as transparency (e.g., in image overlays where acoustic image data is superimposed on other image data), etc.

[0167] Additional parameters, such as the rate of change of sound intensity, can also be color-tuned. Similar to intensity, the rate of change of changing sound intensity can be color-tuned so that scene sections exhibiting different rates and / or amounts of sound intensity change are displayed in different colors.

[0168] In the illustrated example, the acoustic image data is color-coded according to an intensity palette, so that acoustic image data representing different acoustic signal intensities are displayed with different colors and / or shading. For example, the acoustic image data at positions 1010 and 1030 shows a trayed representation of the first intensity, positions 1040, 1060, and 1080 show a trayed representation of the second intensity, and positions 1020, 1050, 1070, and 1090 show a trayed representation of the third intensity. FIG. 10A The exemplary representation shown illustrates a circular pattern with a color gradient extending outwards from the center at each location of the color representation of the acoustic image data. This is likely due to the attenuation of sound intensity as the signal propagates from the sound source.

[0169] exist FIG. 10A In the example, acoustic image data is combined with visible light image data to generate a display image, which can be presented to the user, for example, via a display screen. The user can view... FIG. 10A The system displays an image to show which locations in the visible scene are generating sound signals, and the intensity of these signals. Therefore, users can quickly and easily observe the locations where sound is being generated and compare the intensity of sound from various locations in the scene.

[0170] Similar to that described with respect to frequencies elsewhere herein, in some embodiments, acoustic image data can only be presented if the corresponding acoustic signal satisfies a predetermined intensity condition. FIG. 10B An exemplary display image showing an image similar to FIG. 10A and including visible light image data and acoustic image representing acoustic signals above a predetermined threshold. As shown, of the locations 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090 in FIG. 10A only locations 1020, 1050, 1070, and 1090 include acoustic image data representing acoustic signals satisfying a predetermined condition.

[0171] In an exemplary scenario, FIG. 10A may include all acoustic image data above a noise floor threshold at each of locations 1010-990, while FIG. 10B shows the same scenario as FIG. 10A but only shows acoustic image data having an intensity greater than 40 dB. This can help a user identify which sound sources in an environment (e.g., the loudest sounds in a target scenario of FIG. 10A and FIG. 10B ).

[0172] As described elsewhere herein, in addition to or as an alternative to directly comparing to an intensity threshold (e.g., 40 dB), in some such examples, the predetermined requirement for displaying acoustic image data can include an amount of change or rate of change of acoustic intensity at a certain location in the image. In some such examples, acoustic image data can only be presented if the rate of change or amount of change of acoustic intensity at a given location satisfies a predetermined condition (e.g., greater than a threshold, less than a threshold, within a predetermined range, etc.). In some embodiments, the amount of change or rate of change of acoustic intensity can be colorized and displayed as or in conjunction with intensity acoustic image data. For example, in an exemplary embodiment, when the rate of change is used as a threshold to determine which locations include acoustic image data, the acoustic image data can include a colorized intensity rate of change metric for display.

[0173] In some examples, the user can manually set intensity requirements (e.g., minimum, maximum, range, rate of change, amount of change, etc.) for the acoustic image data to be displayed. As discussed elsewhere in this document, this can be achieved during acoustic image data generation (e.g., via filtering the received acoustic signal), and / or by not displaying generated acoustic image data representing acoustic signals that do not meet (one or more) the set requirements. In some such examples, filtering of the displayed image based on intensity values ​​can be performed after the acoustic image data and visible light image data have been captured and stored in memory. That is, the data stored in memory can be used to generate a displayed image that includes any number of filtering parameters, such as showing only acoustic image data that meets predefined intensity conditions.

[0174] In some examples, setting a lower intensity limit in an acoustic image (e.g., displaying only acoustic image data representing acoustic signals above a predetermined intensity) can eliminate the inclusion of unwanted background or ambient sounds and / or sound reflections from the acoustic image data. In other cases, setting an upper intensity limit in an acoustic image (e.g., displaying only acoustic image data representing acoustic signals below a predetermined intensity) can eliminate the inclusion of expected loud sounds in the acoustic image data, allowing observation of acoustic signals that would normally be masked by such loud sounds.

[0175] Several display functions are possible. For example, similar to "About" FIG. 9C In some examples of the frequency analysis / display discussed, the acoustic analysis system can cycle through multiple display images, each showing acoustic image data that meets different intensity requirements. Similarly, in some examples, the user can scroll through a range of sound intensity to view the locations within the acoustic image data that have sound intensities within a given range.

[0176] Another parameter that can be used to analyze acoustic data is the periodicity of the acoustic signal. FIG. 11A and 11B An exemplary plot of frequency comparison time for acoustic data in an acoustic scene is shown. FIG. 11A As shown in the diagram, the acoustic data includes: a signal with a frequency of X having a first periodicity, a signal with a frequency of Y having a second periodicity, and a signal with a frequency of Z having a third periodicity. In the example shown, the acoustic signals with different frequencies may also include different periods in the acoustic signals.

[0177] In some such examples, in addition to or instead of frequency content, acoustic signals can be filtered based on periodicity. For example, in some examples, multiple acoustic signal sources in an acoustic scene can produce acoustic signals at a particular frequency. If a user wishes to isolate one such acoustic source for acoustic imaging, the user can select to include acoustic image data in a final display image, or to exclude acoustic image data from a final display image, based on the periodicity associated with the acoustic data.

[0178] FIG. 11B A plot of frequency versus time for an acoustic signal is shown. As shown, the frequency increases approximately linearly over time. However, as shown, the signal includes a periodicity that is approximately constant over time. Thus, depending on selected display parameters, such a signal can or can not appear in an acoustic image. For example, the signal can satisfy the frequency criteria for display at some points in time, but at other points in time, be outside the frequency range displayed. However, a user can select to include such a signal in acoustic image data, or to exclude such a signal from acoustic image data, based on the periodicity of the signal, independent of the frequency content.

[0179] In some examples, extracting acoustic signals having a particular periodicity can facilitate analysis of a particular portion of a target scene (e.g., a particular piece of equipment or type of equipment that typically operates with a particular periodicity). For example, if an object of interest operates with a certain periodicity (e.g., once per second), excluding signals having a different periodicity from this can improve acoustic analysis of the object of interest. For example, with reference to FIG. 11B If the object of interest operates with periodicity 4, isolating signals having periodicity 4 for analysis can result in improved analysis of the object of interest. For example, the object of interest can emit a sound having periodicity 4, but increasing frequency, such as FIG. 11B as shown. This can mean that the properties of the object can be changing (e.g., increasing torque or load, etc.), and should be checked.

[0180] In an example acoustic imaging process, background noise (e.g., a noise floor in an industrial environment, wind in an outdoor environment, etc.) is typically not periodic, while certain objects of interest in the scene emit periodic acoustic signals (e.g., machinery operating at regular intervals). Thus, a user can select to exclude non-periodic acoustic signals from the acoustic image in order to remove background signals and more clearly present acoustic data of interest. In other examples, a user can be looking for a constant tone source, and thus the user can select to exclude periodic signals from the acoustic image data that can obscure the view of the constant tone. In general, a user can select to include acoustic signals in the acoustic image data that are above a certain periodicity, below a certain periodicity, or within a desired range of periodicities. In various examples, periodicity can be identified by a length of time between periodic signals or a frequency of occurrence of periodic signals. Similarly to FIG. 11B the frequency shown in FIG. 25, an analysis of intensity at a given periodicity (e.g., due to an object of interest operating at that periodicity) can similarly be used to track how acoustic signals from the object change over time. In general, in some embodiments, periodicity can be used to perform rate of change analysis on a variety of parameters such as frequency, intensity, etc.

[0181] As described elsewhere herein, in some examples, various portions of a target scene can be associated with different distances to the acoustic imaging sensor. For example, in some embodiments, distance information can include three-dimensional depth information about various portions of the scene. Additionally or alternatively, a user can be able to measure (e.g., with a laser distance tool) or manually input distance values associated with multiple locations in the scene. In some examples, such different distance values for various portions of the scene can be used to adjust the backpropagation calculations at such locations to account for the particular distance value at that location.

[0182] Additionally or alternatively, if different portions of the scene are associated with different distance values, proximity to the acoustic sensor array (e.g., measured proximity and / or apparent proximity) can be another distinguishable parameter between such portions. For example, with respect to FIG. 10B , locations 1020, 1050, 1070, and 1090 are each associated with different distance values. In some examples, similar to frequency or periodicity discussed elsewhere herein, a user can select a particular distance range according to which to include acoustic image data on the display. For example, a user can select to display acoustic image data representative of acoustic signals that are closer than a predetermined distance, farther than a predetermined distance, or within a predetermined distance range.

[0183] Further, in some embodiments, similar to the frequency or periodicity discussed with respect to FIG. 9CAs described in the frequency description, an acoustic analysis system can be configured to cycle through multiple distance ranges, thereby displaying only acoustic image data representing acoustic signals emitted from locations within the target scene that satisfy the current distance range. Such cycling through various displays can help users visually distinguish information between different acoustic signals. For example, in some cases, an object may appear close to the line of sight from an associated electromagnetic imaging tool (e.g., a visible light camera module), and therefore the acoustic image data combined with the electromagnetic image data of such an object may be difficult to distinguish. However, if the objects are separated by depth differences, cycling through different depth ranges of acoustic image data can be used to isolate each acoustic data source from one another.

[0184] Generally, acoustic analysis systems can be configured to apply various settings to include and / or exclude acoustic image data representing acoustic signals that satisfy one or more predefined parameters. In some examples, an acoustic analysis system can be used to select multiple conditions that an acoustic signal must satisfy in order to display acoustic image data representing such signals, for example, in a display image.

[0185] For example, regarding FIG. 10A and 10B Only FIG. 10B It shows FIG. 10A In scenarios where acoustic signals exceed a threshold intensity, additional or alternative limitations are possible. For example, in some embodiments, the user can additionally filter the acoustic image data such that the acoustic image data is displayed only for acoustic signals having frequency content within a predetermined frequency range and / or having a predetermined periodicity. In exemplary embodiments, limited to the predetermined frequencies and / or periodicity of interest, the acoustic image data can be eliminated from additional locations such as 1020 and 1090.

[0186] Generally, users can apply any number of acoustic data requests to include or exclude acoustic image data from the displayed image, including parameters such as intensity, frequency, periodicity, apparent proximity, measured proximity, sound pressure, particle velocity, particle displacement, sound power, sound energy, sound energy density, sound exposure, pitch, amplitude, brilliance, harmonics, and the rate of change of any such parameters. Additionally, in some embodiments, users can combine requests using any suitable logical combination such as AND, OR, XOR, etc. For example, a user might want to display only sound signals with (intensity above a predetermined threshold) AND (frequency within a predetermined range).

[0187] Additionally or alternatively, the acoustic analysis system can be configured to cycle through one or more parameter ranges to illustrate different portions of the target scene, such as with respect to the multiple frequencies illustrated in FIG. 9C In general, one or more parameters can be cycled through in this manner. For example, a parameter (e.g., intensity) can be divided into multiple ranges (e.g., 10 dB - 20 dB and 20 dB - 30 dB), and the acoustic analysis system can cycle through such ranges, displaying all acoustic image data falling within a first range, then all acoustic image data falling within a second range, and so on.

[0188] Similarly, in some embodiments, the acoustic analysis system can be configured to combine parameter requirements by cycling through nested ranges. For example, in an exemplary embodiment, acoustic image data satisfying a first intensity range AND a first frequency range can be displayed. The displayed frequency range can be cycled through while limiting the displayed acoustic image data to acoustic signals satisfying the first intensity range. After cycling through the frequency range, the intensity range can be updated to a second intensity range, such that the displayed acoustic image data satisfies the second intensity range and the first frequency range. Similar to the process incorporating the first intensity range, the frequency range can be similarly cycled through while maintaining the second intensity range. This process can continue until all combinations of frequency ranges and intensity ranges have been satisfied. A similar such process can be performed on any of the multiple parameters.

[0189] Additionally or alternatively, in some embodiments, the acoustic analysis system can be configured to identify and distinguish multiple sounds in an acoustic scene. For example, with respect to FIG. 9B , the acoustic analysis system can be configured to identify four discrete sounds at locations 910, 920, 930, and 940. The system can be configured to cycle through multiple displays, each display showing acoustic image data at a single discrete location, similar to that illustrated in FIG. 9C , although not necessarily dependent on any parameter values. Similarly, such cycling between acoustic image data at various locations can be performed after one or more parameter requirements limit the displayed acoustic image data.

[0190] For example, with respect to FIG. 10A and FIG. 10B , the acoustic analysis system can cycle through multiple acoustic image scenes (e.g., as a display image including acoustic image scenes with visible light image data) prior to applying an intensity threshold, with each scene including acoustic image data at a single location. In some embodiments, the acoustic image data can be cycled through according to FIG. 10Athe illustrated example, the cycle has 10 separate images, each including image data at a different location in locations 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090. However, according to some embodiments, after applying an intensity filter such that only locations with an intensity greater than a threshold are displayed (e.g., as in FIG. 10B FIG. 6B), the acoustic analysis system can update the cycling process to cycle through only images corresponding to locations that satisfy the filtering threshold. That is, with respect to FIG. 10B FIG. 6B, the cycling process can be updated to cycle only between four images, each showing discrete acoustic image data at locations 1020, 1050, 1070, and 1090, respectively.

[0191] Thus, in various embodiments, each location in the target scene that includes acoustic image data is shown in one of a plurality of cycled-through display images, either before or after applying one or more filters to limit which acoustic image data is shown. This cycled display of individual sound source locations can help a user viewing the images to identify sources of particular sounds. In some embodiments, each image in the cycle includes only a single source of acoustic data, and in some such embodiments, further includes one or more parameters of the acoustic data, such as frequency content, intensity, periodicity, apparent proximity, and the like.

[0192] In addition to, or as an alternative to, cycling between images showing acoustic image data that satisfies certain conditions, in some examples, locations of acoustic signal sources can be detected in the acoustic image data, and the locations of the acoustic signal sources can be displayed in the acoustic image data in isolation from other acoustic signals. For example, with respect to FIG. 10A in some embodiments, acoustic image data representative of acoustic signals emanating from each of locations 1010-990 can be identified and cycled through. For example, in an example operational process, a display image including acoustic image data at one of locations 1010-990 can be cycled through, either automatically or under the direction of a user, for individual analysis of each acoustic signal source. In various embodiments, the order in which different locations of acoustic image data are displayed while cycling can depend on a variety of parameters, such as by location, proximity, intensity, frequency content, and the like.

[0193] Additionally or alternatively, in some examples, acoustic image data from various locations can be cycled through after applying one or more filters to isolate only acoustic image data that satisfies one or more predetermined conditions. For example, with respect to FIG. 10B, locations 1020, 1050, 1070, and 1090 are shown as including acoustic image data representing acoustic signals that satisfy a predetermined intensity requirement. In some embodiments, such a display requirement can be applied to individual loops through the source locations of acoustic signals. For example, with further reference to FIG. 10B , the display image includes image data from only one of locations 1020, 1050, 1070, and 1090 that satisfy the acoustic intensity condition, which can be looped through for individual analysis at each location.

[0194] In the exemplary process with reference to FIG. 10A and FIG. 10B , acoustic image data collected from the scene can generally be shown at locations 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090 in FIG. 10A . Such locations can include acoustic image data representing acoustic signals having a wide variety of acoustic parameters, such as a wide variety of intensities, frequency content, periodicity, and so on.

[0195] As described elsewhere herein, a user can wish to isolate acoustic signals having one or more particular acoustic parameters, such as acoustic signals having a minimum acoustic intensity. Acoustic image data representing acoustic signals that do not satisfy such a condition can be excluded from the image, for example, as shown in FIG. 10B , leaving acoustic image data at locations 1020, 1050, 1070, and 1090. However, the user can wish to further identify the source of particular sounds that satisfy a display condition (e.g., having an intensity above a threshold). Accordingly, the user can select to display the acoustic image data associated with locations 1020, 1050, 1070, and 1090 one at a time, in order to view the source location of each sound and analyze each sound individually. In various embodiments, the user can select to manually loop through such locations, or the processor can automatically update the display image to sequentially display the acoustic image data of individual locations. This can help the user to further eliminate and ignore acoustic signals that are not of interest, but happen to satisfy one or more filtering parameters applied to the image.

[0196] Although with respect to intensity and FIG. 10A and 10BHaving been described, generally speaking, the display images including acoustic image data from a single location selected from among a plurality of locations can be cycled through one after another for individual analysis. The plurality of locations including representative acoustic image data can be the entire set of locations corresponding to acoustic signal sources in the acoustic scene, or can be a subset of such locations, e.g., including only locations having acoustic signals satisfying one or more conditions. Such conditions can depend on any one or more acoustic parameters, such as intensity, frequency content, periodicity, proximity, and so on, and can be satisfied based on various parameters being below a predetermined value, above a predetermined value, or within a predetermined range of values.

[0197] In various examples, modifying the display images to selectively include acoustic image data in the display images can be accomplished in a variety of ways. In some embodiments, the display images (e.g., including electromagnetic image data and acoustic image data) can be real-time images, with the electromagnetic image data and acoustic image data being continuously updated to reflect changes in the scene. In some examples, when using certain conditions to determine whether to include acoustic image data in the display images, the received acoustic signals are analyzed to determine whether acoustic image data is to be included at various locations in the updated real-time images. That is, as new display images are generated based on recently received acoustic signals and electromagnetic radiation, the construction of the display images can depend on the analysis of the acoustic signals to determine which acoustic signals satisfy any particular conditions placed on the display images (e.g., intensity threshold, etc.). The display images including acoustic image data can then be generated only as appropriate according to such conditions.

[0198] In other examples, the display images can be generated from data stored in memory, such as previously captured acoustic data and electromagnetic image data. In some such examples, the previously acquired acoustic data is analyzed with respect to various conditions to be placed on the acoustic image data, and combined with the electromagnetic image data in locations where the previously captured acoustic data satisfies such conditions. In such embodiments, a single scene can be viewed in a number of ways, e.g., by analyzing different acoustic parameters. The display images representative of the previously captured acoustic image data can be updated based on any updated conditions placed on the display images for whether to include acoustic image data in various locations in the display images.

[0199] In some embodiments, one or more acoustic parameters used to selectively include acoustic image data in a displayed image can be used to modify the displayed image and / or image capture techniques. For example, in a real-time imaging example, various conditions for determining whether acoustic image data is included in the display may include distance to the target (e.g., apparent distance or measured distance) and / or frequency content. As described elsewhere herein, some of these parameters can be used to select an acoustic sensor array and / or a processing scheme for generating the acoustic image data. Thus, in some such examples, when acoustic image data is represented solely based on such parameters that satisfy one or more predetermined conditions, such conditions can be used to select an acoustic sensor array and / or a processing scheme for generating the acoustic image data.

[0200] For example, in an exemplary embodiment, if the acoustic image data is included only in real-time images at locations where the corresponding acoustic signal comprises frequency content within a first frequency range, one or more acoustic sensor arrays can be selected to acquire the acoustic signal best suited for the first frequency range. Similarly, if the acoustic image data is included only in real-time images at locations where the acoustic signal source is within a first distance range, one or more acoustic sensor arrays can be selected to acquire the acoustic signal best suited for acoustic imaging within the first distance range. Additionally or alternatively, as shown, for example, with reference to... FIG. 6 As described, the processing scheme for generating acoustic image data can be selected based on desired frequency or distance conditions. Such selected acoustic imaging sensor array(s) and processing scheme(s) can then be used to receive acoustic signals and generate acoustic image data for updated real-time display images, in order to optimize the included acoustic image data.

[0201] Similarly, in some embodiments where a display image is generated from historical data previously stored in memory, various conditions for determining which locations in the display image include acoustic image data can be used to update the acoustic image data representing the acoustic scene. For example, in some embodiments, the data stored in memory includes raw acoustic data received by one or more acoustic sensor arrays from the time the acoustic signal was received. Based on the conditions used to determine whether acoustic image data is included at various locations in the display image (e.g., desired distances and / or frequency ranges), a processing scheme (e.g., a backpropagation algorithm) can be selected to use the raw data stored in memory to generate acoustic image data optimized for desired parameters for display.

[0202] It will be understood that, although visible light image data and acoustic image data are commonly used for description and illustration, reference... FIG. 9A- The processes described in C, 10A, and 10B can be used with any of a variety of electromagnetic image data. For example, in various embodiments, similar processes can be carried out with infrared image data or ultraviolet image data instead of visible light image data. Additionally or alternatively, combinations of the electromagnetic spectrum can be used in such processes, such as mixing infrared image data and visible light image data. In general, in various examples, acoustic image data can be selectively shown in conjunction with any combination of electromagnetic image data (e.g., acoustic image data included when the corresponding acoustic signals satisfy one or more predetermined parameters).

[0203] In some embodiments, the acoustic analysis system is configured to store one or more acoustic signals and / or acoustic image data in a database (e.g., in local memory) and / or accessible from an external or remote device. Such acoustic signals can include acoustic image data representative of acoustic scenes during normal operation and / or other parameters associated with acoustic scenes, such as frequency data, intensity data, periodicity data, and the like. In various examples, the database can include acoustic image data and / or other acoustic parameters (e.g., intensity, frequency, periodicity, etc.) representative of broad scenes (e.g., a factory) and / or more specific scenes (e.g., a particular object).

[0204] In some embodiments, the database scenes can be generic to a particular type of device (such as a particular device model). Additionally or alternatively, the database scenes can be specific to an individual object, even if different such objects are different instances of the same object (e.g., two separate machines of the same model). Similarly, the database scenes can be more specific, e.g., including particular operating states of the object. For example, if a particular object has multiple modes of operation, the database can include multiple scenes with such an object, one for each mode of operation.

[0205] In various embodiments, the database scene can be a single acoustic image and / or associated acoustic parameters. In other examples, the database scene can include composite data formed from multiple previously captured acoustic images and / or associated parameters. Generally, the database scene (e.g., acoustic image and / or parameters) can include an acoustic representation of the scene during normal operation. In some examples, the database can include other elements associated with the scene, such as corresponding visible light images, infrared images, ultraviolet images, or combinations thereof. In some embodiments, database generation and / or comparison can be carried out similarly to that described in U.S. Patent Application No. 15 / 190,792, filed June 23, 2016, and entitled "THERMAL ANOMALY DETECTION," which is assigned to the assignee of the present application and incorporated by reference herein in its entirety. In some embodiments, the database can be generated by capturing acoustic image data and / or one or more associated acoustic parameters (e.g., frequency, intensity, periodicity, etc.) of a scene while the objects within the scene are operating correctly. In some such examples, a user can tag the captured database images to associate the images with one or more objects, locations, scenes, and so forth, so that in the future the captured acoustic images and / or associated parameter(s) can be identified for database analysis and comparison.

[0206] The recently generated acoustic image data can be compared to acoustic image data stored in the database to determine whether the acoustic profile of the acoustic scene is within typical operating standards. Additionally or alternatively, acoustic parameters (such as intensity, frequency, periodicity, and so forth) from the live acoustic scene and / or the recently generated acoustic image can be compared to similar parameters in the database.

[0207] The comparison of the current acoustic image data to historical acoustic image data stored in the database (e.g., previously captured images, composite images generated from multiple previously captured images, expected images provided by a factory, etc.) can be carried out in a variety of ways. FIG. 12A-12C A variety of exemplary ways for comparing acoustic image data to historical acoustic image data stored in a database are shown. FIG. 12AAn acoustic imaging tool 1200 is shown, which includes an acoustic sensor array 1202 having an acoustic field of view 1212 and an electromagnetic imaging tool 1204 having an electromagnetic field of view 1214. As shown, the electromagnetic field of view 1214 and the acoustic field of view 1212 include a target scene 1220, which includes an object of interest 1222. In some embodiments, the acoustic imaging tool 1200 is permanently fixed in a location such that the object of interest 1222 is in the electromagnetic field of view 1214 and the acoustic field of view 1212. In some embodiments, the acoustic imaging tool 1200 can be powered via inductive or parasitic power, can be wired into an AC mains power in a building, or be configured to continuously monitor the object 1222.

[0208] The fixed acoustic imaging tool 1200 can be configured to periodically capture acoustic and / or electromagnetic image data of the object 1222. Because the acoustic imaging tool 1200 is typically fixed in place, images captured at different times will be approximately from the same vantage point. In some examples, acoustic image data captured via the acoustic imaging tool 1200 can be compared to a database of acoustic image data representative of approximately the same scene, e.g., to detect anomalies or abnormalities in the acoustic scene. This can be carried out as described in U.S. Patent Application No. 15 / 190,792, which is incorporated by reference, for example.

[0209] FIG. 12B An example display on a handheld acoustic imaging tool is shown. The display 1230 includes two portions—1232 and 1234. In the illustrated example, portion 1234 shows a database image 1244 of the object of interest, while portion 1232 includes a live display 1242 of real-time acoustic image data of the object. In such a side-by-side view, a user can compare the live image 1242 to the database image 1244 to see any differences between the typical acoustic signals (e.g., as shown in the database image 1244) and the current real-time image 1242. Similarly, a user can compare whether the live image 1242 approximately matches the database image 1244. If so, the user can capture the live acoustic image for further analysis and / or comparison to the database image 1244.

[0210] FIG. 12C Another example display on a handheld acoustic imaging tool is shown. FIG. 12C The display 1250 shows a database image 1254 and a live image 1256 on the same display 1252. In this example, the database image 1254 is shown in a top portion of the display 1252, while the live image 1256 is shown in a bottom portion of the display 1252. In some examples, the database image 1254 can be shown in a top portion of the display 1252, while the live image 1256 is shown in a bottom portion of the display 1252. In some examples, the database image 1254 can be shown in a left portion of the display 1252, while the live image 1256 is shown in a right portion of the display 1252. In some examples, the database image 1254 can be shown in a right portion of the display 1252, while the live image 1256 is shown in a left portion of the display 1252. FIG. 12CIn the example, the user can similarly compare the acoustic image data in live image 1256 with the acoustic image data in database image 1254 to view the differences. Additionally, the user can adjust the alignment of the acoustic imaging tool to align the objects in live image 1256 with the objects in database image 1254 for further analysis and comparison.

[0211] As FIG. 12A-12C The result of this process can compare live and / or recently captured acoustic images with previous acoustic image data, such as from a database. In some examples, such a process can be used to register live and / or recently captured acoustic images with database images for automated comparison. Other processes that can be used to “recapture” acoustic image data as database images from similar points of interest are described in U.S. Patent Application No. 13 / 331,633, filed December 20, 2011, entitled “THERMAL IMAGING CAMERA FOR INFRARED REPHOTOGRAPHY”, U.S. Patent Application No. 13 / 331,644, filed December 20, 2011, entitled “THERMAL IMAGING CAMERA FOR INFRARED REPHOTOGRAPHY”, and U.S. Patent Application No. 13 / 336,607, filed December 23, 2011, each of which is assigned to the assignee of this application and incorporated herein by reference in its entirety.

[0212] Comparing real-time acoustic image data and / or acoustic features with corresponding acoustic images and / or acoustic features of comparable scenes / objects can be used to provide a quick and simplified analysis of the operational state of a scene / object. For example, a comparison can indicate that certain locations within an acoustic scene are emitting acoustic signals with different intensities or spectra than during typical operation, which can indicate a problem. Similarly, locations in a scene may be emitting acoustic signals that are normally silent. Additionally or alternatively, a comparison of the overall acoustic features of live and historical scenes from a database can generally indicate variations in acoustic parameters within the scene, such as frequency content, sound intensity, etc.

[0213] In some examples, the acoustic analysis system is configured to compare the recent / real-time acoustic scene to a database. In some embodiments, the acoustic analysis system is configured to characterize differences between the recent / real-time scene and the database scene, and diagnose one or more possible problems in the current scene based on the comparison. For example, in some embodiments, a user can pre-select an object or target scene of interest to compare to the acoustic database. The acoustic analysis system can analyze the scene based on the selected object / scene, comparing database images and / or other parameters to recent / current images and / or other parameters. Based on the object / scene selected from the database, the acoustic analysis system can be able to identify one or more differences between the database images / parameters and the recent / current images / parameters, and correlate the identified difference(s) to possible causes of the difference(s).

[0214] In some examples, the acoustic analysis system can be pre-programmed with a plurality of diagnostic information, e.g., associating various differences between database images / parameters and recent / current images / parameters to possible causes and / or solutions to the causes. Additionally or alternatively, a user can load such diagnostic information, e.g., from a repository of diagnostic data. Such data can be provided, e.g., by a manufacturer of the acoustic analysis system, a manufacturer of the object of interest, etc. In still further examples, the acoustic analysis system can additionally or alternatively learn diagnostic information, e.g., via one or more machine learning processes. In some such examples, a user can diagnose one or more problems in a target scene after observing acoustic deviations of the scene from typical scenes, and can input data representative of the one or more problems and / or one or more solutions to the acoustic analysis system. The system can be configured to learn, over time and via a plurality of data entries, to associate different differences between recent / current images and / or parameters and those stored in the database to certain problems and / or solutions. In diagnosing a problem and / or determining a proposed solution, the acoustic analysis system can be configured to output the suspected problem and / or the proposed solution to a user, e.g., via a display. Such a display can be on a handheld acoustic inspection tool or a remote device (e.g., a user’s smartphone, tablet device, computer, etc.). Additionally or alternatively, such a display indicating a potential problem and / or solution can be transmitted to a remote site, such as an off-site operator / system monitor, e.g., via a network.

[0215] In some example diagnostic characterizations, the acoustic analysis system can observe a particular periodic squeak, which indicates that additional lubrication is needed on the operating machine. Similarly, a constant high-pitched tone can indicate a gas or air leak in the target scene. Other issues can similarly have identifiable acoustic signatures, such as bearing damage within the object being analyzed, such that viewing the acoustic signature via an acoustic imaging system (e.g., a handheld acoustic imaging tool) can assist in diagnosing any abnormalities in the system or object.

[0216] An acoustic analysis system that is able to compare received acoustic signals to a baseline (e.g., acoustic image data and / or parameters from a database) and carry out diagnostic information and / or suggest corrective action can eliminate the need for an experienced expert to analyze acoustic data of a scene. Rather, acoustic inspection and analysis can be carried out by a system operator with limited or no experience in analyzing acoustic data.

[0217] FIG. 13 is a process flow diagram illustrating example operations of comparing received acoustic image data to a database for object diagnosis. The method includes receiving a selection of a target of interest (1380) and retrieving baseline acoustic image and / or acoustic parameters of the target of interest from a database (1382). For example, a user can wish to carry out acoustic analysis on a particular object of interest and can select such object from a predefined list of objects having available baseline acoustic image and / or parameters available in the database.

[0218] The method further includes capturing acoustic image data and associated parameters representative of the target of interest, e.g., using a handheld acoustic imaging tool (1384). After capturing acoustic image data and associated parameters (1384), the method includes comparing the captured acoustic image data and / or associated parameters to the retrieved baseline image and / or parameters (1386).

[0219] FIG. 13 The method of further includes diagnosing an operational problem of the target of interest based on the comparison if the captured acoustic image data and / or parameters deviate sufficiently from the baseline (1388). The method can further include displaying an indication of the likely problem and / or corrective action to the user (1392). In some embodiments, a comparison display can additionally or alternatively be displayed to the user, e.g., a difference image showing differences between the current acoustic image data and the baseline acoustic image data.

[0220] In some such examples, determining whether there is a deviation from the baseline (1388) includes comparing one or more acoustic parameters of the captured data to similar parameters in the baseline data and determining whether the difference between the captured and baseline parameters exceeds a predetermined threshold. In various examples, different parameters can include different thresholds, and such thresholds can be absolute thresholds, statistical thresholds, and the like. In some embodiments, the comparison can be made on a location-by-location basis, and the comparison can be made for a subset of locations within the scene.

[0221] For example, with reference to FIG. 9B it is possible to analyze only locations on the object (e.g., locations 910 and 940) that include acoustic image data and appear on the object with respect to the operation of the object. In such examples, different acoustic parameters at each location to be compared (e.g., 910 and 940) are compared individually between the captured image and the database image. For example, with reference to FIG. 9B Comparing the captured data and / or associated parameters to those from the database can include comparing the frequency, intensity, and periodicity of location 910 in the captured image to the frequency, intensity, and periodicity of location 910 in the database image, respectively. Similar comparisons can be made between the captured image and the database image at location 940. As described, each comparison can include different metrics for determining whether there is a sufficient deviation from the baseline (1388).

[0222] Diagnosing operational problems (1390) and displaying an indication of possible problems and / or corrective actions (1392) can be made based on a combination of comparisons between the captured image data and baseline image data and / or parameters. In some examples, such diagnosis can include multi-dimensional analysis, such as a combined comparison of multiple parameters at a given location. For example, in an example embodiment, a certain condition can be indicated by both a frequency deviation from the baseline greater than a first threshold and an intensity deviation from the baseline greater than a second threshold.

[0223] In some examples, even after displaying an indication of possible problems and / or corrective actions (1392), the process can include capturing new acoustic image data and associated parameters (1384) and repeating the comparison and diagnosis process. Thus, a user can observe whether any taken corrective actions effectively change the acoustic characteristics of the object in order to correct the identified problems and / or to bring the acoustic characteristics of the object in line with the baseline.

[0224] In some embodiments, if there is not enough deviation from the baseline after comparing the captured data to the baseline data (1386) (1388), the process can end (1394) with a conclusion that the object is functioning properly based on the current acoustic characteristics of the object. Additionally or alternatively, new acoustic image data and associated parameters of the object of interest can be captured (1384), and the comparison and diagnostic process can be repeated. In some examples, the continuous repeated analysis can be performed using a fixed acoustic analysis system, for example, including FIG. 12A acoustic imaging tool 1200 in

[0225] The comparison of acoustic data (e.g., image data and / or other acoustic parameters) can help the user more easily identify whether the object is functioning properly, and if not, diagnose the problem with the object. In some examples, the comparison to the baseline can help the user ignore "normal" sounds in the scene, such as expected operational sounds or noise floor / background sounds, which can not be related to an operational problem with the object.

[0226] During operation, the observation of acoustic image data and / or associated acoustic parameters, or the comparison of the current acoustic scene to the database acoustic scene, can indicate locations of interest for the user to focus on for further inspection. For example, a comparison acoustic image that shows deviation from the database image can indicate one or more locations in the scene that are operating abnormally. Similarly, viewing an acoustic image that has acoustic characteristics at one or more unexpected locations can indicate locations of interest for the user. For example, referring to FIG. 10B , the user observes on the display of the acoustic imaging system FIG. 10B which can recognize that a particular location (e.g., 1020) is unexpectedly emitting acoustic signals, or similarly, the comparison to the baseline image indicates unexpected parameters of the acoustic signals at that location (e.g., unexpected frequency, intensity, etc.).

[0227] In some such examples, the user can move closer to such a location in order to more closely inspect the location for abnormalities. In moving closer to the object, the value of the distance to the target can be updated to reflect the new distance between the acoustic array and the target location. The acoustic sensor array and / or the backpropagation algorithm can be updated based on the updated distance to the target. Additionally or alternatively, the updated acoustic analysis from the closer location can yield a different analysis of the acoustic signals from the target. For example, high frequency acoustic signals (e.g., ultrasonic signals) tend to decay within a relatively short distance to the source of the acoustic signals. Thus, when the user moves closer to the target for further inspection, the acoustic sensor array can see additional signals (e.g., high frequency signals). Such a marked change in the observable scene can also result in adjustments to the acoustic sensor array and / or adjustments to the backpropagation algorithm used to perform acoustic imaging.

[0228] Thus, as the user moves closer to an object or region of interest, the sensor array and / or the backpropagation algorithm used to perform acoustic imaging can be updated one or more times. Each update can provide additional details about the object or region of interest that can not be observable from a greater distance using a different sensor array and / or backpropagation algorithm. For example, moving closer to an object or region of interest can also increase the acoustic intensity of the acoustic signals of interest relative to background sounds in the environment based on an initial observation of a wider scene.

[0229] In some embodiments, an acoustic analysis system (e.g., a handheld acoustic imaging tool) can prompt a user to move closer to an object or region of interest within a scene. For example, in comparing a current acoustic image to a baseline database image, the acoustic analysis system can identify one or more locations in the scene that deviate from the baseline. The acoustic analysis system can highlight such one or more locations to the user, e.g., via a display, and suggest that the user move closer to the identified location(s) for further analysis. In some examples, the acoustic analysis system can classify the identified location, such as an object within the environment or a subcomponent of a particular object, as having its own baseline profile stored in a database. The system can be configured to suggest and / or implement such a profile of the classified location to facilitate further analysis of the identified location as the user moves closer for additional inspection.

[0230] The systems and processes described herein can be used to improve the speed, efficiency, accuracy, and thoroughness of acoustic inspections. Various automated actions and / or suggestions (e.g., of sensor arrays, backpropagation algorithms, etc.) can increase the ease of inspection to the point that an inexperienced user can conduct a thorough acoustic inspection of an acoustic scene. Moreover, such processes can be used to analyze a wide range of scenes, such as entire systems, individual objects, and subcomponents of individual objects. Predefined and / or user-generated profiles of baseline acoustic data of acoustic scenes can even help inexperienced users identify anomalies in captured acoustic data.

[0231] Registration of acoustic image data with other data streams, such as visible, infrared, and / or ultraviolet image data, can provide additional context and detail as to which objects are emitting the acoustic signals represented in the acoustic image data. Combining an acoustic sensor array and a distance measurement tool (e.g., a laser rangefinder) can help a user quickly and easily determine an appropriate value for the distance to a target for use during an acoustic imaging process. In various examples, an acoustic sensor array, a distance measurement tool, a processor, a memory, and one or more additional imaging tools (e.g., a visible light camera module, an infrared camera module, etc.) can be supported by a single housing in a handheld acoustic imaging tool, which can provide efficient acoustic analysis of multiple scenes. Such a handheld acoustic imaging tool can be moved from one scene to another for quick analysis of multiple objects of interest. Similarly, using a handheld tool, a user can move closer to a location of interest within a scene for further inspection or analysis.

[0232] Various systems and methods for conducting acoustic imaging and generating and displaying acoustic image data are described herein. An example system can include an acoustic sensor array including a plurality of acoustic sensor elements configured to receive acoustic signals from an acoustic scene and output acoustic data based on the received acoustic signals.

[0233] A system can include an electromagnetic imaging tool configured to receive electromagnetic radiation from a target scene and output electromagnetic image data representative of the received electromagnetic radiation. Such an imaging tool can include an infrared imaging tool, a visible light imaging tool, an ultraviolet imaging tool, and the like, or combinations thereof.

[0234] The system can include a processor in communication with the acoustic sensor array and the electromagnetic imaging tool. The processor can be configured to receive electromagnetic image data from the electromagnetic imaging tool and acoustic data from the acoustic sensor array. The processor can be configured to generate acoustic image data of the scene based on the received acoustic data and received distance information representative of a distance to the target, e.g., via backpropagation calculations. The acoustic image data can include a visual representation of the acoustic data, such as by a color palette or color scheme, such as described elsewhere herein.

[0235] The processor can be configured to combine the generated acoustic image data and the received electromagnetic image data to generate a display image including both acoustic image data and electromagnetic image data, and to communicate the display image to a display. Combining the acoustic image data and the electromagnetic image data can include correcting for parallax errors between the acoustic image data and the electromagnetic image data, e.g., based on the received distance information.

[0236] In some examples, the distance information can be received from a distance measurement tool in communication with the processor. The distance measurement tool can include, for example, an optical distance measurement device, such as a laser distance measurement device, and / or an acoustic distance measurement device. Additionally or alternatively, the user can manually input the distance information, e.g., via a user interface.

[0237] The system can include a laser pointer to help identify a location of a point of interest, such as a sound or sound profile, based on selected parameters, such as frequency, decibel level, periodicity, distance, etc., or combinations thereof. Such a laser pointer can be used to pinpoint and align a field of view of the scene with the appropriate sound visualization displayed on the display. This can be useful in environments where the object being inspected is at a distance from the acoustic imaging device, or where the location of the visualization of the sound on the display is not clear relative to the actual scene.

[0238] In some examples, the laser pointer can be visualized on the display. Such a visualization can include generating a laser pointer dot on the display representative of the laser pointer in the actual scene, e.g., via the processor. In some examples, the position of the laser pointer on the display can be augmented, e.g., with an icon or another aligned display marker representative of the laser pointer in the scene, to better determine the location on the display relative to the actual scene.

[0239] As described elsewhere herein, the thermal imaging system can be configured to create a false color (e.g., tinted), symbolic, or other non-numerical visual representation of the acoustic data generated by the one or more acoustic sensors, such as by creating acoustic image data. Additionally or alternatively, the system can provide audio feedback to the user, such as via a speaker, headphones, wired or remote communication earpieces, and the like. The transmission of such audio or heterodyne audio can be synchronized with the detected and displayed visual representation of sound.

[0240] In various examples, the acoustic data can be visualized in a variety of ways, for example, to facilitate understanding of such data, and to prevent viewers from making false assumptions about the nature of the sound being visualized. In some examples, different types of visualizations can provide intuitive understanding of the sound being visualized.

[0241] In some embodiments, the generated display includes non-numerical visual representations with contextual numerical and / or alphanumeric data, to provide a thorough presentation of information about the sound being visualized, which can assist the user in determining and / or implementing one or more appropriate courses of action.

[0242] Various display features can be combined, including various non-numerical graphical representations (e.g., symbols, tints, etc.) and alphanumeric information. In some embodiments, the display features present in a given representation of a scene can be customized by the user, for example, from a plurality of selectable settings. Additionally or alternatively, a preset combination of display features can be selected by the user to automatically include a desired combination of information in the display image. In various embodiments, aspects of the display image can be adjusted by the user, for example, via a virtual interface (e.g., provided via a touchscreen) and / or via physical controls.

[0243] FIG. 14 Visualizations of acoustic data using a gradient tinting scheme are shown. As shown, an acoustic parameter (e.g., intensity) is shown via a gradient tinting scheme. In an example gradient tinting scheme, according to the tinting scheme, parameter values will have a unique color associated with them. Changes in parameter values at a given pixel will typically result in a change in the color associated with that pixel to represent the new parameter value. As shown in the example of FIG. 2, the acoustic parameter values appear to change radially from a center position in the acoustic signal at positions 232, 234, 236. Other examples of gradient tinting are described in U.S. Patent Application No. 15 / 802,153, filed November 2, 2017, which is assigned to the assignee of the present application. FIG. 14

[0244] FIG. 15 ​Visualizations of acoustic data using multiple shaded concentric circles are shown. In some such examples, as opposed to having a gradient color scheme with colors associated with parameter values, FIG. 15 Each solid-colored concentric circle shown can represent pixels having acoustic parameter values within a range of values associated with that color. In the illustrated example, the acoustic parameter (e.g., intensity) associated with the acoustic signals at locations 332, 334, 336 changes radially from the center of the acoustic signal. In the example color scheme, pixels shown in red represent acoustic parameter values within a first range of parameter values, pixels shown in yellow represent acoustic parameter values within a second range of parameter values, and pixels shown in green represent acoustic parameter values within a third range of parameter values, however, other display techniques are possible, including additional or alternative colors, patterns, and so forth. In various embodiments, the ranges of values can correspond to absolute ranges, such as intensity values between 10 dB and 20 dB, or can be relative ranges, such as intensity values between 90% and 100% of the maximum intensity.

[0245] As described elsewhere herein, in some embodiments, a display image including electromagnetic image data and acoustic image data can include both visual indications of acoustic signals and alphanumeric representations of one or more parameters associated with the acoustic signals. FIG. 16 An example visualization is shown that illustrates both non-numerical information (e.g., via coloration of parameter value ranges) and alphanumeric information. In the illustrated example, an alphanumeric sound intensity value label is associated with each of three locations having colorized acoustic image data (e.g., intensity data). As shown, the acoustic signals have corresponding visual indicators representing the acoustic parameters (1602, 1604, 1606) associated therewith as well as alphanumeric information (1612, 1614, 1616, respectively). In example embodiments, the alphanumeric information can provide a numerical value associated with a location at which colorized acoustic data is displayed, such as a maximum intensity value. In some examples, a user can select one or more locations to display coloration and / or alphanumeric data. For example, a user can select to annotate a display image with alphanumeric representations of acoustic parameters associated with one or more acoustic signals within a scene.

[0246] In some examples, the alphanumeric information can represent multiple parameters (e.g., acoustic parameters) associated with acoustic signals at a given location in a scene. FIG. 17An example visualization is shown that includes non-numerical information (e.g., via shading of parameter value ranges) and alphanumeric information. In the illustrated example, sound intensity values and corresponding frequency values (e.g., average frequency or peak frequency) are shown in alphanumeric information 1702, 1704, 1706 associated with each of three locations with shaded acoustic data (e.g., intensity data). Similar to the discussion above FIG. 16 As discussed above, a user can initiate the inclusion of various such data at various locations. For example, a user can select to annotate a display image with alphanumeric representations of one or more acoustic parameters associated with one or more acoustic signals within the scene.

[0247] FIG. 18 Another example visualization is shown that illustrates both non-numerical information (e.g., via shading of parameter value ranges) and alphanumeric information. In the illustrated example, distance measurements are included with alphanumeric information 1812, 1814, 1816 associated with each of three locations with shaded acoustic data (e.g., intensity data). Similar to the discussion above FIG. 16 As discussed above, a user can select to include various such data at various locations, e.g., as part of annotation of a display image.

[0248] In some examples, non-numerical representations can be used to convey information related to multiple acoustic parameters. For example, FIG. 19 An example visualization is shown that illustrates indicators 1902, 1904, 1906 (in this case, circles) of different sizes and colors that represent different acoustic parameter values. In example embodiments, the size of an indicator corresponds to the intensity of an acoustic signal at a given location, while the color of an indicator corresponds to a peak or average frequency. In example embodiments, the indicator size can illustrate relative values such that comparing the size of one indicator to the size of another indicator represents a relative difference between the acoustic parameter values represented at the locations associated with the indicators. Additionally or alternatively, alphanumeric information can be included to provide absolute or relative acoustic parameter values.

[0249] In some embodiments, colored indicators can be used to represent the severity of one or more detected acoustic signals and / or associated acoustic parameters, such as the amount of deviation from a baseline parameter. FIG. 20An example visualization is shown that illustrates a plurality of indicators 2002, 2004, 2006 having different colors that indicate the severity indicated by the acoustic signals from the corresponding locations. For example, in example embodiments, a red indicator indicates a critical severity based on one or more acoustic parameters (e.g., when compared to a baseline such as a baseline of typical operating conditions), a yellow indicator indicates a moderate severity, and a green indicator represents a lesser severity. In other examples, other color schemes or appearance characteristics (e.g., indicator transparency, indicator size, etc.) can be used to visually distinguish the severity of the acoustic signals. In the illustrated example, indicator 2004 represents the highest level of severity, indicator 2006 represents a lesser severity, and indicator 2002 represents the least severe acoustic signal.

[0250] As described elsewhere herein, in various embodiments, one or more acoustic parameters can be displayed on a visual representation of an acoustic scene, e.g., by a color or grayscale tinting. In some embodiments, the system can be configured to identify one or more locations in the scene that satisfy one or more acoustic conditions, such as a range of frequencies, a range of intensities, a range of distances, and so on, that are identified. In some examples, various locations corresponding to acoustic profiles can be identified (e.g., that satisfy a particular set of conditions or parameters). Such identified locations can be presented in a manner that is distinguishable from the tinting scheme used otherwise with the acoustic image data in creating the display image. For example, FIG. 21 A scene is shown that includes indicators at a plurality of locations within the scene. Indicators 2101, 2102, 2103, 2104, and 2105 are positioned within the scene. Indicators 2103, 2104, and 2105 include, e.g., tinted acoustic image data that represents values of one or more acoustic parameters corresponding to scale 2110. Indicators 2101 and 2102 are shown with a unique presentation scheme that is distinguishable from the tinting scheme that appears at locations 2103, 2104, and 2105. In such embodiments, a user can quickly and easily identify those locations in the image that satisfy one or more desired conditions. In some such examples, the user can select one or more desired conditions, e.g., from a scale such as 2110, based on a selection of a range of values, for display in a distinguishable manner.

[0251] Additionally or alternatively, locations that satisfy conditions of a particular sound profile, such as a corresponding acoustic profile, can be presented with an icon that is representative of the satisfied condition. For example, FIG. 22 A plurality of icons 2202, 2204, 2206, 2208 are shown positioned within the display image that indicate identified acoustic profiles within the scene. As FIG. 22The example profiles shown include bearing wear, air leak, and arcing. Such profiles can be identified by satisfying a set of one or more parameters associated with such profiles in order to be classified as such profiles.

[0252] FIG. 23 Another example display is shown, which shows acoustic data using concentric circles and alphanumeric information representing acoustic intensity associated with each acoustic signal via a plurality of indicators 2302, 2304, and 2306. As described elsewhere herein, in some examples, the size of the indicators can represent one or more acoustic parameters present at the corresponding location. In some embodiments, the indicators can be monochromatic, and indicate acoustic parameters in one or more other ways, such as by indicator size, line width, line type (e.g., solid, dashed, etc.).

[0253] In some examples, the display can include alphanumeric information based on selections made by the user. For example, in some embodiments, the system (e.g., via the processor) can include information representing one or more acoustic parameters of the acoustic signal located at a particular location in response to selection of an indicator on the display by the user at such location (e.g., via a user interface). FIG. 24 An example display image is shown, with an indicator and additional alphanumeric information associated with the represented acoustic signal. In the example, the indicator 2402 on the display can be selected (e.g., via a crosshair representation, which can indicate a selection such as a touch screen input selection) for further analysis. The display shows alphanumeric information 2404, which includes a list of data associated with the location corresponding to the indicator, including peak intensity and corresponding frequency, frequency range, distance to location measurement, and criticality level indicated by the acoustic signal from the location.

[0254] In some examples, the display image can include a plurality of indicators representing a corresponding plurality of acoustic signals in the scene. In some embodiments, in such cases, the user can select one or more indicators (e.g., selected via a touch screen or other user interface), and in response to detecting the selection, the processor can present additional information about the acoustic signal. Such additional information can include alphanumeric of one or more acoustic parameters. In some examples, such additional information can be displayed simultaneously for a plurality of acoustic signals. In other examples, such additional information for a given acoustic signal is hidden when another acoustic signal is selected.

[0255] As described elsewhere in this document, in some examples, the system may include a laser pointer. In some examples, the laser pointer may have a fixed orientation or may have an adjustable pointing, for example, controllable via a processor. In some examples, the system may be configured to align the laser pointer with a location in the target scene associated with a selected location in the image. FIG. 25A A system including a display (in some examples, embodied as a handheld tool) is shown, the display being such as... FIG. 24 The display shown allows selection of an indicator 2502. A laser pointer 2504 emits a laser beam 2506 toward the scene, creating a laser point 2508 in the scene corresponding to the position of the selected indicator 2502 in the image. This helps the user visualize the location of the selected and / or analyzed acoustic signal in the environment. In some embodiments, the laser point 2508 may be detected by an electromagnetic imaging tool and is visible on the display along with the displayed indicator 2502 and alphanumeric information 2512 including acoustic parameter information. In some examples, the acoustic imaging system is configured to detect or predict the position of a laser in the scene and provide a visual indication 2510 of the laser position. FIG. 25B It shows things like FIG. 25A The image shown in the system view.

[0256] In embodiments where the laser pointer has a fixed orientation, the user can view a display image with a visual indication of the laser position as feedback, allowing the user to adjust the direction of the laser to match the selected acoustic signal.

[0257] As described elsewhere herein, in some embodiments, acoustic image data may be combined with electromagnetic image data for presentation in a displayed image. In some examples, the acoustic image data may include adjustable transparency so that various aspects of the electromagnetic image data are not completely obscured. FIG. 26 The diagram illustrates acoustic image data at a location in a scene, indicated by indicator 2602, where indicator 2602 includes a gradient color scheme. The system may include a display device, which may be integrated with or separate from the acoustic imaging tool, configured to present display data including electromagnetic image data and acoustic image data.

[0258] In some embodiments, the device (e.g., a handheld acoustic imaging tool) may include physical hybrid controls 2614 (e.g., one or more buttons, knobs, sliders, etc., which may be included as part of the user interface) and / or virtual hybrid controls 2604 (such as an interface implemented via a touchscreen or other virtual means). In some embodiments, such functionality may be provided by an external display device such as a smartphone, tablet, computer, etc.

[0259] FIG. 27 A virtual and / or physical hybrid control tool for displaying images is shown that includes a partially transparent concentric circle color scheme. Similar to that described with respect to FIG. 26 the indicator 2702 can represent acoustic signals within a scene. The acoustic imaging system can include a physical hybrid control 2714 and / or a virtual hybrid control 2704 that can be used to adjust the transparency of acoustic image data (e.g., the indicator 2702) within the displayed image.

[0260] Additionally or alternatively, a physical and / or virtual interface can be used to adjust one or more display parameters. For example, in some embodiments, one or more filters can be applied to selectively display acoustic image data that satisfies one or more conditions, such as described elsewhere herein. FIG. 28 A scene is shown that includes an indicator 2802 having a gradient color that indicates locations within the scene that satisfy one or more filters (e.g., have one or more acoustic or other parameters that satisfy one or more corresponding thresholds or predetermined conditions). In various examples, the filter can be selected and / or adjusted via a physical control 2814 (e.g., via one or more buttons, knobs, switches, etc.) and / or a virtual control 2804 (e.g., a touchscreen). In the illustrated example, the filter includes displaying acoustic image data for only those acoustic signals having acoustic parameters (e.g., frequencies) that fall within a predefined range 2806 of acoustic parameters. As shown, the predefined range 2806 is a subset of a possible filter range 2816. In some examples, a user can adjust the limits of the predefined range 2806, e.g., via the virtual 2804 or physical 2814 controls, to adjust the effect of the filter.

[0261] FIG. 29 A virtual and / or physical filter adjustment for a displayed image is shown that includes a partially transparent concentric circle color scheme. As shown, indicators 2902 are shown within a scene based on acoustic parameters that fall within a predetermined range 2906 based on the value of the filter. The filter can be adjustable within a range of values 2916, e.g., via virtual 2904 and / or physical 2914 controls.

[0262] In some embodiments, multiple filters can be utilized to customize a displayed image that includes colorized acoustic image data. FIG. 30 A displayed image is shown that illustrates a first indicator and a second indicator. As described elsewhere herein, one or more filters can be applied to the displayed image (e.g., via physical filter controls and / or virtual filter controls) to customize the displayed data. In FIG. 30In the illustrated example, filtering includes establishing a first filter range 3006 and a second filter range 3008. In some examples, the filter ranges can be adjustable within a range of values 3016, e.g., via virtual 3004 and / or physical 3014 controls.

[0263] Such filter ranges can represent any of a variety of parameters, such as frequency, amplitude, proximity, etc. As illustrated, both the first and second filter ranges are associated with a color (which can be adjusted by the user in some examples), and the indicators 3002, 3012 are positioned in the image at locations where the corresponding acoustic signals satisfy the filter condition(s) associated with each filter range. As illustrated, the first indicator 3002 represents acoustic signals that satisfy the first filter range 3006 (shown in dark shading), while the second indicator 3012 represents acoustic signals that satisfy the second filter range 3008 (shown in lighter shading). Thus, the user can be able to quickly identify locations in the scene with acoustic data that satisfies a variety of conditions at once, while also identifying which conditions are satisfied at which locations.

[0264] In some examples, a display device, such as an acoustic imaging tool or an external display device, can include a virtual keyboard as an input device, such as FIG. 31 as illustrated. FIG. 31 A display interface is shown that includes an indicator 3102 representing one or more acoustic parameters of acoustic signals in a scene. A virtual keyboard 3110 is included in the display, which can be used to add alphanumeric information 3112 to the displayed image. Utilizing such a virtual keyboard can allow the user to input custom annotations, such as various inspection notes, labels, date / time stamps, or other data that can be stored with the image. In various examples, the virtual keyboard can be used to add text that is included in the image data and / or attached to the image data, such as by being stored in metadata associated with the displayed image.

[0265] A variety of devices can be used to present a display image that includes various combinations of acoustic image data and other data, such as alphanumeric data, image data from one or more electromagnetic spectrums, symbols, and so on. In some examples, a handheld acoustic imaging tool can include a built-in display for presenting the display image. In other examples, the information to be displayed or data processed for generating the display (e.g., raw sensor data) can be transmitted to an external device for display. Such external devices can include, for example, a smartphone, a tablet device, a computer, a wearable device, and so on. In some embodiments, the display image is presented in conjunction with real-time electromagnetic image data (e.g., visible light image data) in an augmented reality type display.

[0266] FIG. 32 A display embedded in eyewear 3210 that can be worn by a user is shown. In some examples, the eyewear can include one or more embedded imaging tools, such as described in U.S. Patent Publication No. 20160076937, entitled "DISPLAY OF IMAGES FROM AN IMAGING TOOL EMBEDDED OR ATTACHED TO A TEST AND MEASUREMENT TOOL," and assigned to the assignee hereof, the relevant portions of which are incorporated herein by reference. In some such examples, the integrated display can show a live display image 3220. For example, the display can show electromagnetic image data (e.g., visible light image data) representative of a scene that the user is facing, and can simultaneously display (e.g., via mixing, overlaying, etc.) one or more additional data streams, such as acoustic image data (e.g., including indicator 3202), and so on, to provide added information to the user. In some embodiments, such as the eyewear shown in FIG. 32, the display includes a transparent display screen, such that when no display image is provided to the display, the user can view the scene directly with his or her eyes through the eyewear, rather than being presented with live visible light image data. In some such examples, additional data, such as acoustic image data, alphanumeric data, and so on, can be displayed on the display that is otherwise transparent in the user's field of view, such that the user views such data in addition to his or her scene view through the display. FIG. 32

[0267] As described elsewhere herein, in various examples, the various data presented in the display image can be combined in a variety of ways, including mixing with other data streams (e.g., mixing acoustic image data with visible light image data). In some examples, the intensity of the mixing can vary between different locations within a single display image. In some embodiments, the user can manually adjust the mixing ratio for each of a plurality of locations (e.g., each of a plurality of indicators of detected acoustic signals). Additionally or alternatively, the mixing can be a function of one or more parameters, such as frequency, amplitude, proximity, and so on.

[0268] In some embodiments, the acoustic imaging tool can be configured to identify a degree to which the sensor array is pointed at each of a plurality of locations from which a detected acoustic signal is emitted, and to mix corresponding acoustic image data with, for example, visible light image data accordingly. FIG. 33A An example display is shown that includes a first indicator 3302 and a second indicator 3304 representing acoustic signals in an acoustic scene. In this example, the first indicator 3302 is shown as a circle, and the second indicator 3304 is shown as a triangle. In some embodiments, the display can be configured to show a live display image, such as described elsewhere herein. In some such examples, the display can be configured to mix acoustic image data with visible light image data, such as described elsewhere herein. In some embodiments, the display can be configured to mix acoustic image data with visible light image data in a manner that varies between different locations within a single display image. In some embodiments, the display can be configured to mix acoustic image data with visible light image data in a manner that is a function of one or more parameters, such as frequency, amplitude, proximity, and so on. FIG. 33A ​In the display scheme of FIG. 3B, the first indicator 3302 is more prominently displayed than the second indicator 3304 (e.g., with a higher blending factor or lower transparency). Conversely, in the display scheme of FIG. 3C, the second indicator 3304 is more prominently displayed than the first indicator 3302 (e.g., with a higher blending factor or lower transparency). Generally speaking, in some embodiments, the acoustic imaging system can determine a metric indicative of the degree to which a sensor is pointed at a given location (e.g., corresponding to an indicator in the acoustic image data), and adjust the blending ratio corresponding to such a location accordingly (e.g., greater degree of pointing corresponds to a higher blending ratio). FIG. 33A FIG. 33B In the display scheme of FIG. 3B, the first indicator 3302 is more prominently displayed than the second indicator 3304 (e.g., with a higher blending factor or lower transparency). Conversely, in the display scheme of FIG. 3C, the second indicator 3304 is more prominently displayed than the first indicator 3302 (e.g., with a higher blending factor or lower transparency). Generally speaking, in some embodiments, the acoustic imaging system can determine a metric indicative of the degree to which a sensor is pointed at a given location (e.g., corresponding to an indicator in the acoustic image data), and adjust the blending ratio corresponding to such a location accordingly (e.g., greater degree of pointing corresponds to a higher blending ratio). FIG. 33B

[0269] In some embodiments, the acoustic imaging system can include one or more audio devices to provide audio feedback based on acoustic signals received from the acoustic scene, e.g., in addition to or as an alternative to visualizing acoustic image data. The processor of the acoustic imaging system can be configured to generate an audio output representative of the received acoustic data, and to communicate the audio output to an audio device, which outputs an audio feedback signal to the user based on the generated audio output.

[0270] The one or more audio devices can include speakers, earphones, headphones, or other wired or wireless audio devices. In some examples, the audio devices can provide an audio or heterodyne representation of the acoustic image data displayed on a display. For example, in some examples, the audio devices can be configured to reproduce acoustic signals emanating from one or more locations in the acoustic scene as audio feedback signals, and can provide such audio feedback signals individually or in combination, e.g., to reproduce a combination of sounds.

[0271] In some embodiments, the audio feedback can be provided to the user concurrently with a visual representation of the acoustic scene (e.g., a display image including acoustic image data), to provide the user with multiple modalities of analyzing the acoustic scene.

[0272] In some embodiments, a user or technician can aim a device based on the audio feedback from the audio devices, e.g., to emphasize or focus on a sound.

[0273] ​​In some examples, where multiple acoustic signals from corresponding multiple locations in the acoustic scene, the acoustic imaging system including the audio device can cycle through the imaged sounds and provide audio feedback for each imaged sound independently. In some embodiments, the acoustic imaging system can provide audio feedback to the user via the audio device corresponding to the imaged sound that matches the criteria (e.g., based on one or more selected parameters such as frequency, decibel level, periodicity, distance, etc.). Additionally or alternatively, the system can be configured to provide a collective audio feedback signal representative of all detected acoustic signals.

[0274] In some examples, the audio device can provide a mono sound representation of the scene or object of interest. Additionally or alternatively, the audio device can provide a stereo sound representation of the scene or object of interest. In some embodiments, the audio device can switch between mono and stereo sound representations.

[0275] FIG. 34A 、 34B Figures 34C and 34D illustrate various types of audio devices in communication with the acoustic imaging device in the acoustic imaging system. Various examples include wired headphones (e.g., in-ear or over-ear headphones), wireless headphones / speakers (e.g., Bluetooth communication with the acoustic imaging device), wired and / or wireless speakers in communication with the acoustic imaging device, or acoustic imaging devices with built-in speakers on the sensor side and / or display / interface side. Combinations of such devices are possible, and in some such examples, the user can select one or more of the multiple possible audio devices for outputting the audio feedback.

[0276] FIG. 34A Figure 34A illustrates an acoustic imaging device 3402 with a port 3404 for interfacing with a wired headphone such as 3406. FIG. 34B Figure 34B illustrates an acoustic imaging device 3412 including a wireless interface 3414 for wireless communication with one or more wireless audio devices such as wireless speakers or headphones 3416. In some examples, the acoustic imaging device includes an integrated speaker. FIG. 34C Figure 34C illustrates an acoustic imaging device 3432 including an integrated speaker 3434 integrated to the sensor side of the device. FIG. 34D Figure 34D illustrates an acoustic imaging device 3442 including an integrated speaker 3444 integrated to the display side of the device. In general, the acoustic analysis system can include one or more audio devices configured to output an audio feedback signal to the user. Any combination of audio devices can be used, such as integrated speakers and wireless headphones, etc.

[0277] FIG. 35Exemplary embodiments are shown in which an audio device provides audio feedback to represent displayed acoustic image data. During exemplary operation, acoustic imaging device 3502 can initially receive acoustic data and electromagnetic data from a scene and generate a display including acoustic image data and electromagnetic image data, as described elsewhere herein. A user or technician can direct the acoustic imaging device so that a sound of interest is located near a center point of the display screen. The center point can be represented in a variety of ways, including a crosshair, a box, a circle, other shapes, and so on. When the visualized sound is located at or near the center point of the display, the acoustic imaging device can provide audio feedback to the user or technician through an audio device, such as through headphones 3506 wired to acoustic imaging device 3502 via headphone jack 3504.

[0278] In the illustrated example, acoustic imaging device 3502 detects sound 3510 in a scene and outputs audio feedback 3520 based on the detected sound 3510. Audio feedback 3520 provided via audio device 3506 can include a reproduction of the centered sound, including reproduction of one or more acoustic parameters such as frequency, periodicity, and so on. In some examples, such parameters can be normalized to a more suitable scale for presentation to the user, such as providing audio feedback of ultrasonic information in the audible frequency range and / or adjusting intensity to a comfortable and / or audible level.

[0279] FIG. 36 Exemplary embodiments are shown in which an audio device provides selective audio feedback corresponding to a selected acoustic signal source. In some examples, a user or technician can select which of a plurality of visualized sounds is to be used to generate audio feedback through an audio device. In various examples, the user or technician can select one or more visualized sounds via a touchscreen on the display, via one or more physical interfaces (e.g., buttons, etc.), via an external device, and so on. In some examples, the user can draw or otherwise position one or more shapes or markers on the screen to identify one or more locations from which to generate audio feedback.

[0280] In the illustrated example, acoustic imaging device 3602 shows an acoustic image including indicators 3622, 3624, 3626 corresponding to sounds 3612, 3614, 3616, respectively. As shown in the example, a user selects sound 3614 by, for example, selecting indicator 3624 via a touchscreen. Audio device 3606, shown as a wired headset, outputs audio feedback signal 3634 representative of the selected sound 3614. Acoustic imaging device 3602 can be configured so that if the user selects a different sound, for example, by selecting a different indicator on the display, a corresponding audio feedback signal is output via audio device 3606. FIG. 36 In the illustrated example, acoustic imaging device 3602 shows an acoustic image including indicators 3622, 3624, 3626 corresponding to sounds 3612, 3614, 3616, respectively. As shown in the example, a user selects sound 3614 by, for example, selecting indicator 3624 via a touchscreen. Audio device 3606, shown as a wired headset, outputs audio feedback signal 3634 representative of the selected sound 3614. Acoustic imaging device 3602 can be configured so that if the user selects a different sound, for example, by selecting a different indicator on the display, a corresponding audio feedback signal is output via audio device 3606.

[0281] FIG. 37 Another example embodiment is shown in which the audio device provides audio feedback to represent the displayed acoustic image data. Similar to FIG. 36 , the acoustic scene includes sounds 3712, 3714, and 3716, and corresponding acoustic image data is shown with corresponding indicators 3722, 3724, and 3726, respectively. In some examples, the user or technician can receive audio feedback for all of the sounds in the acoustic image data, or only for sounds that meet certain criteria described herein.

[0282] In some embodiments, the user or technician receives, via the audio device 3706, audio feedback signals that simultaneously represent each of the sounds 3712, 3714, and 3716. Alternatively, the acoustic imaging device 3702 can be configured to cycle through the sounds and locations, providing audio feedback for such locations one at a time. For example, in an example implementation, the user or technician first receives 3622, then 3624, then 3626, audio feedback. The acoustic imaging device can then return to sound 3612 to repeat the cycle, provide audio feedback for all of the sounds, and so on. Thus, the acoustic imaging device can be used to sequentially output audio feedback signals 3732, 3734, and 3736 in response to the detected sounds 3712, 3714, and 3716, respectively.

[0283] In various examples, the user can select from one or more settings corresponding to how to provide audio feedback signals in response to a scene that includes multiple sounds, including from one or more of: combining the multiple sounds into a single audio feedback signal, cycling through multiple audio feedback signals, each corresponding to one of the multiple sounds, providing an audio feedback signal corresponding to a single selected sound. As described, in some examples, the multiple sounds that are combined and / or cycled through are a subset of the multiple sounds in the acoustic scene that meet one or more predetermined requirements. Additionally or alternatively, in some examples, the acoustic imaging system can cycle through the multiple audio feedback signals in an order that is based on one or more parameters, such as in an order of decreasing intensity. In some embodiments, the acoustic imaging system can indicate on the displayed image which of the multiple sound signals in the acoustic scene is being represented by the currently provided audio feedback signal.

[0284] FIG. 38 An example embodiment is shown in which the audio device provides stereo audio feedback to represent the displayed acoustic image data. For example, FIG. 38 An acoustic imaging device 3802 and an audio device 3806 connected thereto are shown. Sounds 3812 and 3826 are present in the acoustic scene, and are shown via indicators 3822 and 3826, respectively.

[0285] In some examples, the acoustic imaging device 3802 is configured to provide stereophonic audio feedback to a user or technician, e.g., where sound signals from a left-hand portion of the acoustic scene are presented as audio feedback to a left-hand audio device 3850, and sounds on a right-hand portion of the scene are presented as audio feedback to a right-hand audio device 3860. In the illustrated example, sound 3816 (and corresponding indicator 3826) is shown on the left-hand side of the acoustic scene, while sound 3812 (and corresponding indicator 3822) is shown near the center of the scene. In some systems employing stereophonic audio feedback, audio feedback corresponding to sound 3812 is provided in both speakers (because it is near the middle of the display), and audio feedback corresponding to sound 3816 is provided to the left speaker 3850 more than the right speaker 3860 (because it is closer to the left side of the display). In some examples, audio feedback corresponding to sound 3816 can be heard in only the left ear.

[0286] In some embodiments, the acoustic scene can be divided into regions that indicate how sound signals therein are represented in the audio feedback signals. For example, in an example, sound signals from a particular region, such as the leftmost quarter of the scene, are included in the audio feedback signal that is output only from the left speaker, or has a greater magnitude in the audio feedback signal that is output from the left speaker than the right speaker. Similarly, sound signals from a particular region, such as the rightmost quarter of the scene, are included in the audio feedback signal that is output only from the right speaker, or has a greater magnitude in the audio feedback signal that is output from the right speaker than the left speaker. Sound signals that are more important to the scene can be represented in the audio feedback signals from both the left and right speakers.

[0287] In some embodiments, the acoustic imaging system is configured to receive sound signals from a scene, and generate acoustic image data representative of such sound signals. In some embodiments, the acoustic image data can be processed to determine how to balance the audio feedback signals between, e.g., a left channel and a right channel, when creating stereophonic audio feedback. For example, in some examples, pixel locations of the acoustic image data can help determine the relative positioning of multiple sounds in the scene, and can be used to balance the stereophonic audio feedback signals.

[0288] In the illustrated example, the left speaker 3850 outputs a first audio feedback signal 3852, and the right speaker 3860 outputs a second audio feedback signal 3862. In the example, the first audio feedback signal 3852 represents only sound 3816, while the second audio feedback signal represents both sounds 3816 and 3812.

[0289] While the illustrated example includes two speakers 3850 and 3860, in some examples, the system can include any number of audio output devices that can output an audio feedback signal representative of received acoustic data. The audio feedback signal can be constructed based on the location of any number of acoustic signals detected within the scene.

[0290] In some examples, if the pointing of the acoustic imaging device is to be changed such that, for example, sound 3812 is moved toward the right side of the scene, the audio output will dynamically change such that only audio feedback signal 3862 will represent sound 3812 on the right hand side of the screen. Similarly, as sound 3816 moves toward the center of the screen, audio feedback signals 3852 and 3862 can represent sound 3852.

[0291] Additionally or alternatively, as described elsewhere herein, dynamic output based on the pointing of the acoustic imaging device can include adjusting various aspects of the displayed image. For example, as described with reference to FIG. 33A and 33B transparency or blending ratio associated with acoustic image data can be adjusted based on the relative pointing of the acoustic imaging tool, such as the pointing of the acoustic sensor array. In some such embodiments, with respect to FIG. 38 In the pointing shown in the figure, indicator 3826 can be more transparent than indicator 3822 because sound 3812 is more central in the scene. However, if the pointing of the acoustic imaging system is changed such that sound 3816 moves toward the center and sound 3812 moves toward the right hand side of the scene, indicator 3822 can become more transparent than indicator 3826.

[0292] Thus, in some embodiments, the acoustic imaging system can produce visual and / or audible output having a dynamic change in intensity based on the relative pointing of the system, such as the pointing direction of the acoustic sensor array, electromagnetic imaging tool, or both.

[0293] The various processes described herein can be embodied as non-transitory computer-readable media comprising executable instructions for causing one or more processors to perform such processes. A system can comprise one or more processors configured to carry out such processes, for example, based on instructions stored in a memory integral to the processor or external to the processor. In some cases, various components can be distributed throughout the system. For example, a system can comprise multiple distributed processors each configured to perform at least a portion of the overall process performed by the system. Additionally, it will be appreciated that various features and functions described herein can be combined in a single acoustic imaging system, for example, embodied as a handheld acoustic imaging tool or a distributed system having various separate and / or separable components.

[0294] Various functions of the components described herein can be combined. In some embodiments, features described in this application can be combined with features described in PCT Application entitled “SYSTEMS AND METHODS FOR PROJECTING AND DISPLAYING ACOUSTIC DATA,” attorney docket number 56581.178.2, filed July 24, 2019, which is assigned to the assignee of the present application and incorporated by reference herein. In some embodiments, features described in this application can be combined with features described in PCT Application entitled “SYSTEMS AND METHODS FOR TAGGING AND LINKING ACOUSTIC IMAGES,” attorney docket number 56581.179.2, filed July 24, 2019, which is assigned to the assignee of the present application and incorporated by reference herein. In some embodiments, features described in this application can be combined with features described in PCT Application entitled “SYSTEMS AND METHODS FOR DETACHABLE AND ATTACHABLE ACOUSTIC IMAGING SENSORS,” attorney docket number 56581.180.2, filed July 24, 2019, which is assigned to the assignee of the present application and incorporated by reference herein. In some embodiments, features described in this application can be combined with features described in PCT Application entitled “SYSTEMS AND METHODS FOR ANALYZING AND DISPLAYING ACOUSTIC DATA,” attorney docket number 56581.181.2, filed July 24, 2019, which is assigned to the assignee of the present application and incorporated by reference herein.

[0295] Various embodiments have been described. Such examples are non-limiting and do not in any way delimit or limit the scope of the present application.

Claims

1. An acoustic analysis system comprising: an acoustic sensor array comprising a plurality of acoustic sensor elements that receive acoustic signals from a scene and output acoustic data based on the acoustic signals; an audio device configured to output an audio feedback signal; and a processor in communication with the acoustic sensor array and the audio device, the processor configured to: receive the acoustic data from the acoustic sensor array; generate acoustic image data of the scene based on the acoustic data, the acoustic image data comprising a visual representation of one or more acoustic signals in the scene; generate a display image comprising the acoustic image data; and provide a visual indication representing an acoustic signal within the scene, the visual indication having a dynamically changing overall transparency as a pointing of the acoustic sensor array is adjusted relative to a location of the acoustic signal within the scene, such that when the acoustic sensor array is more directly pointed at a location of a first acoustic signal than a second acoustic signal, a visual indication corresponding to the first acoustic signal is more prominently displayed than a visual indication corresponding to the second acoustic signal, and a color tint of the visual indication changes radially from a center of the visual indication based on an acoustic parameter of the acoustic signal; wherein the overall transparency of the visual indication of the acoustic signal varies as a function of a location of the acoustic signal within the scene, such that as the location of the acoustic signal moves away from a center of the scene, the overall transparency of the visual indication increases.

2. The acoustic analysis system of claim 1, wherein the acoustic signal is a first acoustic signal and the output further represents a second acoustic signal within the scene; the display image comprises a first visual indication of the first acoustic signal and a second visual indication of the second acoustic signal, each of the first and second visual indications having a variable overall transparency in the display image; and the overall transparency of the first visual indication and the overall transparency of the second visual indication increase as the first visual indication and the second visual indication move away from a center of the scene in the display image, respectively.

3. The acoustic analysis system of claim 1, wherein the display image comprises a mixture of the acoustic image data and electromagnetic image data representing electromagnetic radiation from the scene, and wherein changing the overall transparency of the visual indication within the display image comprises adjusting a mixing ratio of the acoustic image data and the electromagnetic image data.

Images