Systems and methods for removable and attachable acoustic imaging sensors

Abstract

Some systems include an electromagnetic imaging tool configured to receive electromagnetic radiation, a communication interface, a processor in communication with the electromagnetic imaging tool and the communication interface, and a housing. A system may include a first sensor head having a first plurality of acoustic sensor elements arranged in a first acoustic sensor array. The communication interface may provide communication between the processor and the sensor head via wired or wireless communication. The communication interface may include a docking port in communication with the processor and configured to removably receive a corresponding docking mechanism of the first sensor head. Some systems may include a second sensor head having a second plurality of acoustic sensor elements. The second sensor head may be interchangeably connectable to the communication interface and / or the first sensor head.

Description

[0001] This application is a divisional application of an application with a filing date of July 24, 2019, an application number of 201980062594.X and an invention name of “Systems and methods for detachable and attachable acoustic imaging sensors”.

[0002] Related matters

[0003] This application claims priority to U.S. Patent Application No. 62 / 702,716, filed on July 24, 2018, the entire contents of which are incorporated herein by reference. Background Art

[0004] Currently available acoustic imaging devices include acoustic sensor array configurations that have various frequency sensitivity limitations due to a variety of factors. For example, some acoustic imaging devices are designed to respond to an acoustic frequency range between approximately 20 Hz and approximately 20 kHz. Other devices (e.g., ultrasound devices) are designed to respond to an acoustic frequency range between approximately 38 kHz and approximately 45 kHz.

[0005] However, acoustic imaging devices that are typically designed to operate in the 20 Hz to 20 kHz frequency range are unable to effectively detect or image higher frequencies, such as up to or above approximately 50 kHz. Similarly, acoustic or ultrasonic devices that are designed to operate in the 20 kHz to 50 kHz frequency range are unable to effectively detect and / or image lower frequencies, such as at or below 20 kHz. This may be for a variety of reasons. For example, a sensor array optimized for lower (e.g., audible) frequencies typically contains individual sensors that are farther apart than a sensor array optimized for higher (e.g., ultrasonic) frequencies.

[0006] In addition to or as an alternative to hardware considerations, different computational algorithms and methods for acoustic imaging are often better suited to acoustic signals having different frequencies and / or different distances to targets, making it difficult to determine how best to acoustically image a scene, especially without an inexperienced user.

[0007] Such differences in imaging different acoustic frequency ranges are due in part to the physics behind the propagation of acoustic waves of different frequencies and wavelengths through the air. Certain array orientations, array sizes, and computational methods may generally be better suited for acoustic signals with different frequency characteristics (e.g., audible frequencies, ultrasonic frequencies, etc.).

[0008] Similarly, different array properties and / or computational methods may be better suited to acoustic scenarios at different distances to the target, e.g., near-field acoustic holography for targets at very close distances, various acoustic beamforming methods for targets at greater distances.

[0009] Therefore, acoustic inspections using acoustic arrays (e.g., for acoustic imaging) may require a wide range of equipment, for example, for analyzing acoustic signals having different frequency ranges, and expertise in understanding when different hardware and computing techniques are suitable for performing acoustic analysis. This may make acoustic inspections time-consuming and expensive, and may require experts to perform such inspections.

[0010] For example, a user may be forced to manually select various hardware and / or software for performing the acoustic analysis. However, an inexperienced analyst may not know the preferred combination of hardware and software for a given acoustic analysis and / or acoustic scene. Additionally, isolating the sounds of interest from within a scene may provide its own challenges, particularly in cluttered scenes, and may be tedious and frustrating to inexperienced users. For example, a given acoustic scene (particularly in a noisy environment) may include an acoustic signal that includes any number of frequencies, intensities, or other characteristics that may obscure the acoustic signal of interest.

[0011] Conventional systems often require users to manually identify various acoustic parameters of interest prior to inspection in order to analyze the sounds of interest. However, inexperienced users may not know how to best isolate and / or identify the various sounds of interest.

[0012] Additionally, when multiple imaging techniques (e.g., visible light, infrared, ultraviolet, acoustic, or other imaging techniques) are used in tandem while inspecting the same object or scene, the physical layout and / or other settings (e.g., focus orientation) of the tools used to implement the different imaging techniques may affect the analysis. For example, the different positions and / or focus orientations of each imaging device may result in parallax errors, wherein the resulting images may be misaligned. This may result in the inability to properly locate areas of interest and / or problem areas within a scene, documentation errors, and misdiagnosis of problems. For example, with respect to acoustic image data, if the acoustic image data is misaligned with respect to image data from other imaging techniques (e.g., visible light and / or infrared image data), it may be difficult to identify the location or source of an acoustic signal of interest.

[0013] Existing ultrasonic testing and inspection tools employ (one or more) ultrasonic sensors, with or without a parabolic dish, to help focus the sound toward the receiving sensor(s). When a sound of a particular frequency is detected, it is typically displayed as a rising or falling numerical value, or displayed on a frequency or decibel level graph on the device's display. This can be very confusing and unintuitive for many users. No visualization of the image or sound of the live scene is available.

[0014] Isolating, locating and analyzing specific sounds can be a tedious process and can be confusing to many end users. Additional contextual information is often required in order to conduct appropriate analysis and reporting activities. Advanced acoustic imaging equipment has the ability to produce a false-color visual representation of the sound that is integrated with a static or live visual image of the scene. However, different sensor configurations and numbers of sensors are often preferred for different applications or when different levels of performance are required.

[0015] Technicians who wish to perform different applications, including near-field acoustic holography, far-field beamforming, sonic inspection, infrasound inspection, and / or ultrasonic inspection, must use acoustic imagers with different sensor configurations. For example, a user may need a device for sonic inspection using near-field acoustic holography and a separate device for ultrasonic inspection using far-field acoustic beamforming. This is both expensive and time-consuming for the user. Summary of the invention

[0016] Some aspects of the present disclosure relate to an acoustic imaging system.The system may include an electromagnetic imaging tool configured to receive electromagnetic radiation from a target scene and output electromagnetic image data representative of the received electromagnetic radiation.

[0017] The system may also include a communication interface, a processor in communication with the electromagnetic imaging tool and the communication interface, and a housing. In some systems, the communication interface may provide communication between the first sensor head and the processor. In some systems, the housing may support the processor and the communication interface.

[0018] The system may also include a first sensor head having a first plurality of acoustic sensor elements arranged into a first acoustic sensor array. In some systems, the communication interface includes a docking port that communicates with the processor and is integrated into the housing and is configured to removably receive the first sensor head. The first sensor head may include a docking mechanism that is configured to removably dock with the docking port of the communication interface. In some examples, the sensor head may include an electromagnetic imaging tool, a neck having a flexible gooseneck, a neck having a telescoping portion, and / or a wireless communication interface.

[0019] Some systems may include additional sensor heads, such as a second sensor head having a second plurality of acoustic sensor elements arranged in a second acoustic sensor array. The second sensor head may be interchangeably connectable to the communication interface. In some systems, the second sensor head may be interchangeably connectable to the first sensor head.

[0020] In some systems, the processor can be configured to receive electromagnetic image data from an electromagnetic tool and to receive acoustic data from an acoustic sensor array (e.g., a first acoustic sensor array associated with a first sensor head, and / or a second sensor array associated with a second sensor head). The processor can also be configured to generate acoustic image data based on the received acoustic data, and to combine the generated acoustic image data with the received electromagnetic image data to generate a display image including the acoustic image data and the electromagnetic data.

[0021] The details of one or more examples are set forth below in the accompanying drawings and the description.Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A and 1B Front and rear views of an exemplary acoustic imaging device are shown.

[0023] Figure 2 is a functional block diagram illustrating components of an example of an acoustic analysis system.

[0024] Figure 3A , 3B 3C and 3C show schematic diagrams of exemplary acoustic sensor array configurations within an acoustic analysis system.

[0025] Figure 4A and 4B A schematic diagram showing parallax errors in the generation of frames of visible light image data and acoustic image data is shown.

[0026] Figure 5A and 5B The parallax correction between the visible light image and the acoustic image is shown.

[0027] Figure 5C and 5D yes Figure 5A and 5B Color version of .

[0028] Figure 6 is a process flow diagram illustrating an exemplary method for generating a final image combining acoustic image data and electromagnetic image data.

[0029] Figure 7 is a process flow diagram illustrating an exemplary process for generating acoustic image data from received acoustic signals.

[0030] Figure 8 An exemplary lookup table for determining an appropriate algorithm and sensor array to use during an acoustic imaging procedure is shown.

[0031] Fig.9A is an exemplary plot of the frequency content of received image data over time in an acoustic scene.

[0032] Fig. 9B An exemplary scenario including multiple locations where acoustic signals are emitted is shown.

[0033] Fig. 9C A plurality of combined acoustic and visible light image data at a plurality of predefined frequency ranges are shown.

[0034] Fig. 10A and 10B is an exemplary display image including combined visible light image data and acoustic image data.

[0035] Fig.11A and 11B An exemplary plot of frequency versus time for acoustic data in an acoustic scene is shown.

[0036] Fig. 12A , 12B 12C illustrate various exemplary ways to compare acoustic image data with historical acoustic image data stored in a database.

[0037] Fig.13 is a process flow diagram illustrating exemplary operations of comparing received acoustic image data to a database for subject diagnosis.

[0038] Fig.14 Exemplary embodiments of an acoustic imaging tool and a sensor head including a plurality of acoustic sensor elements are shown.

[0039] Fig.15 An exemplary sensor head and acoustic imaging tool configured for slide-in attachment are shown.

[0040] Fig.16 An exemplary acoustic sensor head is shown having a central sensor portion and a plurality of folded auxiliary sensor portions connected to the central sensor portion via attachment mechanisms.

[0041] Fig.17An exemplary acoustic sensor is shown having a central sensor portion including one or more attachment mechanisms that may be configured to removably receive one or more corresponding auxiliary sensor portions.

[0042] Fig.18 A sensor head assembly including a flexible "gooseneck" section is shown.

[0043] Fig.19 A wand-type sensor head assembly is shown, comprising a handle supporting a sensor array via an extension portion.

[0044] Fig. 20 An example system including an acoustic imaging device and an acoustic sensor head is shown. DETAILED DESCRIPTION

[0045] Figure 1A and 1B Front and rear views of an example acoustic imaging device are shown. Figure 1A The front side of an acoustic imaging device 100 is shown, which has a housing 102 supporting an acoustic sensor array 104 and an electromagnetic imaging tool 106. In some embodiments, the acoustic sensor array 104 includes a plurality of acoustic sensor elements, each of which is configured to receive an acoustic signal from an acoustic scene and output acoustic data based on the received acoustic signal. The electromagnetic imaging tool 106 can be configured to receive electromagnetic radiation from a target scene and output electromagnetic image data representing the received electromagnetic radiation. The electromagnetic imaging tool 106 can be configured to detect electromagnetic radiation in one or more of a plurality of wavelength ranges, such as visible light, infrared light, ultraviolet light, and the like.

[0046] In the illustrated example, the acoustic imaging device 100 includes: an ambient light sensor 108 and a position sensor 116, such as a GPS. The device 100 includes: a laser pointer 110, which in some embodiments includes a laser rangefinder. The device 100 includes: a torch 112, which can be configured to emit visible light radiation toward a scene; and an infrared illuminator 118, which can be configured to emit infrared radiation toward the scene. In some examples, the device 100 may include an illuminator for illuminating the scene in any wavelength range. The device 100 further includes: a projector 114, such as an image reprojector, which can be configured to project a generated image onto a scene such as a color image; and / or a dot projector, which is configured to project a series of dots onto a scene, for example, to determine a depth profile of the scene.

[0047] Figure 1BThe back of the acoustic imaging device 100 is shown. As shown, the device includes: a display 120, which can present images or other data. In some examples, the display 120 includes a touch screen display. The acoustic imaging device 100 includes: a speaker that can provide audio feedback signals to the user, and a wireless interface 124 that can enable wireless communication between the acoustic imaging device 100 and an external device. The device further includes a control 126, which can include one or more buttons, knobs, dials, switches, or other docking components to enable the user to interact with the acoustic imaging device 100. In some examples, the control 126 and the touch screen display are combined to provide a user interface for the acoustic imaging device 100.

[0048] In various embodiments, the acoustic imaging device need not include Figure 1A and 1B Each element shown in the embodiments of the present invention may be excluded from one or more of the illustrated components of the acoustic imaging device. In some examples, Figure 1A and 1B One or more components shown in the embodiments of may be included as part of the acoustic imaging system, but separately from the housing 102. Such components may communicate with other components of the acoustic imaging system, for example using the wireless interface 124, via wired or wireless communication techniques.

[0049] Figure 2 is a functional block diagram illustrating components of an example of an acoustic analysis system 200 . Figure 2 An exemplary acoustic analysis system 200 may include: a plurality of acoustic sensors, such as microphones, MEMS, transducers, etc., arranged in an acoustic sensor array 202. Such an array may be one-dimensional, two-dimensional, or three-dimensional. In various examples, the acoustic sensor array may define any suitable size and shape. In some examples, the acoustic sensor array 202 includes a plurality of acoustic sensors arranged in a grid pattern, such as, for example, an array of sensor elements arranged in longitudinal columns and transverse rows. In various examples, the acoustic sensor array 202 may include an array of vertical columns of horizontal rows in, for example, 8×8, 16×16, 32×32, 64×64, 128×128, 256×256, etc. Other examples are possible, and the various sensor arrays do not necessarily have to include the same number of rows as columns. In some embodiments, such sensors may be positioned on a substrate, such as, for example, a printed circuit board (PCB) substrate.

[0050] exist Figure 2In the configuration shown, a processor 212 in communication with the acoustic sensor array 202 can receive acoustic data from each of the plurality of acoustic sensors. During exemplary operation of the acoustic analysis system 200, the processor 212 can communicate with the acoustic sensor array 202 to generate acoustic image data. For example, the processor 212 can be configured to analyze data received from each of the plurality of acoustic sensors arranged in an acoustic sensor array, and determine an acoustic scene by "back propagating" the acoustic signal to the source of the acoustic signal. In some embodiments, the processor 212 can generate a digital "frame" of acoustic image data by identifying various source locations and intensities of acoustic signals across a two-dimensional scene. By generating a frame of acoustic image data, the processor 212 captures an acoustic image of a target scene at a substantially given point in time. In some examples, a frame includes a plurality of pixels constituting an acoustic image, wherein each pixel represents a portion of a source scene to which the acoustic signal has been back propagated.

[0051] The components described as processors within the acoustic analysis system 200, including the processor 212, may be implemented as one or more processors, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic circuits, and the like, either alone or in any suitable combination. The processor 212 may also include a memory that stores program instructions and associated data that, when executed by the processor 212, cause the acoustic analysis system 200 and the processor 212 to perform the functions attributed to them in the present disclosure. The memory may include any fixed or removable magnetic, optical, or electrical medium, such as RAM, ROM, CD-ROM, hard or floppy disks, EEPROM, and the like. The memory may also include a removable memory portion that may be used to provide memory updates or increase memory capacity. The removable memory may also allow the acoustic image data to be easily transferred to another computing device, or to be removed before the acoustic analysis system 200 is used in another application. The processor 212 may also be implemented as a system on a chip that integrates some or all of the components of a computer or other electronic system into a single chip. The processor 212 (processing circuit) may be configured to communicate processed data to a display 214 or other output / control device 218 .

[0052] In some embodiments, the acoustic sensors in the acoustic sensor array 202 generate a series of signals corresponding to the acoustic signals received by each acoustic sensor to represent an acoustic image. When the signal from each acoustic sensor is obtained by scanning all the rows that make up the acoustic sensor array 202, a "frame" of acoustic image data is generated. In some examples, the processor 212 can acquire acoustic image frames at a rate sufficient to generate a video representation of the acoustic image data (e.g., 30 Hz or 60 Hz). Independent of the specific circuitry, the acoustic analysis system 200 can be configured to manipulate acoustic data representing the acoustic profile of a target scene so as to provide an output that can be displayed, stored, transmitted, or otherwise utilized by a user.

[0053] In some embodiments, "back propagating" the received acoustic signal to generate acoustic image data includes: analyzing the received signal at multiple acoustic sensors in the acoustic sensor array 202, for example, via a processor. In various examples, performing back propagation is a function of one or more parameters, including the distance to the target, the frequency, the sound intensity (e.g., decibel level), the size / configuration of the sensor array, such as the spacing and arrangement of individual sensors within one or more arrays, etc. In some embodiments, such parameters can be pre-programmed into the system, for example, in a memory. For example, the properties of the acoustic sensor array 202 can be stored in a memory, such as an internal memory or a memory specifically associated with the acoustic sensor array 202. Other parameters, such as the distance to the target, can be received in a variety of ways. For example, in some examples, the acoustic analysis system 200 includes a distance measurement tool 204 that communicates with the processor 212. The distance measurement tool can be configured to provide distance information representing the distance from the distance measurement tool 204 to a specific location in the target scene. Various distance measurement tools can include a laser rangefinder or other known distance measurement devices, such as other optical or audio distance measurement devices. Additionally or alternatively, the distance measurement tool can be configured to generate three-dimensional depth data so that each part of the target scene has an associated value of the distance to the target. Therefore, in some examples, the measurement result of the distance to the target as used in this article can correspond to the distance to each position in the target scene. Such three-dimensional depth data can be generated, for example, via multiple imaging tools with different views of the target scene or via other known distance scanning tools. In general, in various embodiments, the distance measurement tool can be used to implement one or more distance measurement functions, including but not limited to: laser distance measurement, active acoustic wave distance measurement, passive ultrasonic distance measurement, LIDAR distance measurement, RADAR distance measurement, millimeter wave distance measurement, etc.

[0054] The distance information from the distance measurement tool 204 can be used in the back propagation calculation. Additionally or alternatively, the system 200 can include a user interface 216 into which a user can manually input a parameter of the distance to the target. For example, if the distance to the component suspected of generating the acoustic signal is known or difficult to measure using the distance measurement tool 204, the user can enter the value of the distance to the target into the system 200.

[0055] In the illustrated embodiment, the acoustic analysis system 200 includes an electromagnetic imaging tool 203 for generating image data representative of a target scene. An exemplary electromagnetic imaging tool may be configured to receive electromagnetic radiation from a target scene and generate electromagnetic image data representative of the received electromagnetic radiation. In some examples, the electromagnetic imaging tool 203 may be configured to generate electromagnetic image data representative of a specific wavelength range within the electromagnetic spectrum, such as infrared radiation, visible light radiation, and ultraviolet radiation. For example, in some embodiments, the electromagnetic timing tool 203 may include one or more camera modules, such as, for example, a visible light camera module 206, which is configured to generate image data representative of a specific wavelength range in the electromagnetic spectrum.

[0056] Visible light camera modules are generally known. For example, various visible light camera modules are included in smart phones and numerous other devices. In some embodiments, visible light camera module 206 can be configured to receive visible light energy from a target scene and focus the visible light energy on a visible light sensor for generating visible light energy data, which can be displayed on display 214 and / or stored in memory, for example, in the form of a visible light image. Visible light camera module 206 may include any suitable components for implementing the functionality attributed to the module herein. Figure 2In the example of , visible light camera module 206 is illustrated as including visible light lens assembly 208 and visible light sensor 210. In some such embodiments, visible light lens assembly 208 includes at least one lens that captures visible light energy emitted by a target scene and focuses the visible light energy on visible light sensor 210. Visible light sensor 210 may include a plurality of visible light sensor elements, such as, for example, a CMOS detector, a CCD detector, a PIN diode, an avalanche photodiode, and the like. Visible light sensor 210 responds to the focused energy by generating an electrical signal, which may be converted and displayed as a visible light image on display 214. In some examples, visible light module 206 is user-configurable and may provide output in a variety of formats, such as to display 214. Visible light camera module 206 may include compensation functionality for various lighting or other operating conditions or user preferences. The visible light camera module may provide a digital output including image data, which may include data in a variety of formats (e.g., RGB, CYMK, YCbCr, etc.).

[0057] In operation of an exemplary visible light camera module 206, light energy received from a target scene can pass through visible light lens assembly 208 and be focused on visible light sensor 210. When the light energy impinges on a visible light sensor element of visible light sensor 210, photons within the photodetector can be released and converted into a detection current. Processor 212 can process the detection current to form a visible light image of the target scene.

[0058] During use of the acoustic analysis system 200, the processor 212 may control the visible light camera module 206 to generate visible light data from the captured target scene for creating a visible light image. The visible light data may include photometric data indicating the color(s) associated with different portions of the captured target scene and / or the magnitude of light associated with different portions of the captured target scene. The processor 212 may generate a "frame" of visible light image data by measuring the response of each visible light sensor element of the acoustic analysis system 200 at a single time. By generating a frame of visible light data, the processor 212 captures a visible light image of the target scene at a given point in time. The processor 212 may also repeatedly measure the response of each visible light sensor element of the acoustic analysis system 200 in order to generate a dynamic visible light image (e.g., a video representation) of the target scene. In some examples, the visible light camera module 206 may include its own dedicated processor or other circuitry (e.g., an ASIC) capable of operating the visible light camera module 206. In some such embodiments, the dedicated processor communicates with the processor 212 for providing visible light image data (e.g., RGB image data) to the processor 212. In an alternative embodiment, a dedicated processor for visible light camera module 206 may be integrated into processor 212 .

[0059] With each sensor element of visible light camera module 206 functioning as a sensor pixel, processor 212 may generate a two-dimensional image or pictorial representation of visible light from a target scene by converting the electrical response of each sensor element into a time-multiplexed electrical signal that may be processed, for example, for visualization on display 214 and / or storage in memory.

[0060] Processor 212 may control display 214 to display at least a portion of the captured visible light image of the target scene. In some examples, processor 212 controls display 214 such that the electrical response of each sensor element of visible light camera module 206 is associated with a single pixel on display 214. In other examples, processor 212 may increase or decrease the resolution of the visible light image such that more or fewer pixels are displayed on display 214 than there are sensor elements in visible light camera module 206. Processor 212 may control display 214 to display the entire visible light image (e.g., the entire portion of the target scene captured by acoustic analysis system 200) or less than the entire visible light image (e.g., a lesser portion of the entire target scene captured by acoustic analysis system 200).

[0061] In some embodiments, processor 212 may control display 214 to simultaneously display at least a portion of a visible light image captured by acoustic analysis system 200 and at least a portion of an acoustic image generated via acoustic sensor array 202. Such concurrent display may be useful because an operator may reference features displayed in the visible light image to help view the source of an acoustic signal simultaneously displayed in the acoustic image. In various examples, processor 212 may control display 214 to display the visible light image and the acoustic image in a side-by-side arrangement, in a picture-in-picture arrangement (in which one image surrounds another image in an image), or in any other suitable arrangement in which the visible light image and the acoustic image are displayed simultaneously.

[0062] For example, the processor 212 can control the display 214 to display the visible light image and the acoustic image in a combined arrangement. In such an arrangement, for a pixel or a set of pixels representing a portion of a target scene in the visible light image, there is a corresponding pixel or a set of pixels representing substantially the same portion of the target scene in the acoustic image. In various embodiments, the size and / or resolution of the acoustic image and the visible light image do not need to be the same. Accordingly, there may be a set of pixels in one of the acoustic image or the visible light image, which corresponds to a single pixel in the other of the acoustic image or the visible light image, or a set of pixels of different sizes. Similarly, there may be an image corresponding to a set of pixels in the other image in one of the visible light image or the acoustic image. Therefore, as used herein, correspondence does not require a one-to-one pixel relationship, but may include mismatched sizes of pixels or pixel groups. Various combination techniques for image regions of mismatched sizes may be implemented, such as upsampling or downsampling one of the images, or combining pixels with the average value of corresponding pixel sets. Other examples are known and are also within the scope of the present disclosure.

[0063] Therefore, corresponding pixels do not need to have a direct one-to-one relationship. Instead, in some embodiments, a single acoustic pixel has multiple corresponding visible light pixels, or a visible light pixel has multiple corresponding acoustic pixels. Additionally or alternatively, in some embodiments, not all visible light pixels have corresponding acoustic pixels, or vice versa. Such an embodiment may indicate a display of the type of figure in the figure discussed previously, for example. Therefore, the visible light pixel will not necessarily have the same pixel coordinates as the pixel coordinates of the corresponding acoustic pixel within the visible light image. Accordingly, as used herein, corresponding pixels generally refer to pixels from any image (e.g., visible light image, acoustic image, combined image, display image, etc.) that include information from substantially the same part of the target scene. Such pixels do not need to have a one-to-one relationship between images, and do not need to have similar coordinate orientations within their respective images.

[0064] Similarly, images with corresponding pixels (i.e., pixels representing the same portion of the target scene) may be referred to as corresponding images. Thus, in some such arrangements, corresponding visible light images and acoustic images may be superimposed on each other at corresponding pixels. An operator may interact with the user interface 216 to control the transparency or opacity of one or both of the images displayed on the display 214. For example, an operator may interact with the user interface 216 to adjust the acoustic image between fully transparent and fully opaque, and also adjust the visible light image between fully transparent and fully opaque. Such an exemplary combination arrangement (which may be referred to as an alpha blending arrangement) may allow an operator to adjust the display 214 to display only an acoustic image, only a visible light image, or any overlapping combination of the two images between the extremes of only an acoustic image and only a visible light image. The processor 212 may also combine scene information with other data, such as alarm data and the like. In general, an alpha blending combination of a visible light image and an acoustic image may include anything from 100% acoustic and 0% visible light to 0% acoustic and 100% visible light. In some embodiments, the amount of blending can be adjusted by the user of the camera. Thus, in some embodiments, the blended image can be adjusted between 100% visible light and 100% acoustic.

[0065] Additionally, in some embodiments, processor 212 may interpret and execute commands from user interface 216 and / or output / control device 218. Furthermore, input signals may be used to alter the processing of visible light and / or acoustic image data that occurs in processor 212.

[0066] The operator can interact with the acoustic analysis system 200 via a user interface 216, which may include buttons, keys, or another mechanism for receiving input from the user. The operator can receive output from the acoustic analysis system 200 via a display 214. The display 214 can be configured to display an acoustic image and / or a visible light image with any acceptable palette or color scheme, and the palette can change, for example, in response to user control. In some embodiments, the acoustic image data can be presented with a palette to represent the varying magnitude of acoustic data from different locations in the scene. For example, in some examples, the display 214 is configured to display an acoustic image with a monochrome palette (such as grayscale). In other examples, the display 214 is configured to display an acoustic image with a color palette such as, for example, amber, iron red (ironbow), blue-red or other high-contrast color schemes. A combination of grayscale and color palette displays is also conceivable. In some examples, a display configured to display such information may include processing capabilities for generating and presenting such image data. In other examples, being configured to display such information may include the ability to receive image data from other components, such as processor 212. For example, processor 212 may generate a value (e.g., RGB value, grayscale value, or other display option) for each pixel to be displayed. Display 214 may receive such information and map each pixel into a visual display.

[0067] Although processor 212 can control display 214 to simultaneously display at least a portion of the acoustic image and at least a portion of the visible light image in any suitable arrangement, the image-in-image arrangement can help an operator easily focus and / or interpret the acoustic image by displaying corresponding visible images of the same scene in adjacent alignment.

[0068] A power source (not shown) delivers operating power to the various components of the acoustic analysis system 200. In various examples, the power source can include a rechargeable or non-rechargeable battery and power generation circuit, an AC power source, an inductive power pickup, a photovoltaic power source, or any other suitable power supply component. Combinations of power supply components are also possible, such as a rechargeable battery and another component configured to provide power to operate the device and / or charge the rechargeable battery.

[0069] During operation of the acoustic analysis system 200, the processor 212, with the help of instructions associated with program information stored in the memory, controls the acoustic sensor array 202 and the visible light camera module 206 to generate visible light images and acoustic images of the target scene. The processor 212 further controls the display 214 to display the visible light images and / or acoustic images generated by the acoustic analysis system 200.

[0070] As described above, in some cases, it may be difficult to identify and distinguish real-world (visible) features of the target scene in the acoustic image. In some embodiments, in addition to supplementing the acoustic image with visible light information, it may be useful to emphasize visible edges within the target scene. In some embodiments, known edge detection methods can be implemented on a visible light image of the target scene. Due to the correspondence between the acoustic image and the visible light image, visible light pixels determined to represent visible edges in the target scene correspond to acoustic pixels that also represent visible edges in the acoustic image. It will be appreciated that, as used herein, "edge" does not necessarily refer to the physical boundaries of an object, but can refer to any sufficiently sharp gradient in a visible light image. Examples may include physical boundaries of objects, color changes within an object, shadows across scenes, and the like.

[0071] Although usually referred to Figure 2 Described as including a visible light camera module 206, in some examples, the electromagnetic imaging tool 203 of the acoustic analysis system 200 may additionally or alternatively include: an imaging tool capable of generating image data representing a wide variety of spectra. For example, in various examples, the electromagnetic imaging tool 203 may include one or more tools capable of generating infrared image data, visible light image data, ultraviolet image data, or any other useful wavelength or combination thereof. In some embodiments, the acoustic imaging system may include an infrared camera module having an infrared lens assembly and an infrared sensor array. Additional components for docking with, for example, an infrared camera module may be included, such as those described in U.S. Patent Application No. 14 / 837,757, entitled “EDGE ENHANCEMENT FOR THERMAL-VISIBLE COMBINED IMAGES AND CAMERAS” and filed on August 27, 2015, which is assigned to the assignee of the present application and is incorporated herein by reference in its entirety.

[0072] In some examples, two or more data streams may be mixed for display. For example, a visible light camera module 206, an acoustic sensor array 202, and an infrared camera module ( Figure 2An exemplary system (not shown in the drawings) can be configured to produce an output image including a mixture of visible light (VL) image data, infrared (IR) image data, and acoustic (Acoustic) image data. In an exemplary mixing scheme, the display image can be represented as: α×IR+β×VL+γ×Acoustic, where α+β+γ=1. In general, any number of data streams can be combined for display. In various embodiments, the mixing ratios such as α, β, and γ can be set by the user. Additionally or alternatively, the set display program can be configured to include different image data streams based on an alarm condition (e.g., one or more values ​​in one or more data streams meet a predetermined threshold) or other conditions, for example, as described in U.S. Patent No. 7,538,326, entitled “VISIBLELIGHT AND IR COMBINED IMAGE CAMERA WITH A LASER POINTER”, which is assigned to the assignee of the present application and is incorporated herein by reference in its entirety.

[0073] Relative to Figure 2 One or more components of the described acoustic analysis system 200 may be included in a portable (e.g., handheld) acoustic analysis tool. For example, in some embodiments, the portable acoustic analysis tool may include a housing 230 configured to accommodate the components in the acoustic analysis tool. In some examples, one or more components of the system 200 may be located outside the housing 230 of the acoustic analysis tool. For example, in some embodiments, the processor 212, display 214, user interface 216, and / or output control device 218 may be located outside the housing of the acoustic analysis tool and may communicate with various other system components, for example, via wireless communications (e.g., Bluetooth communications, Wi-Fi, etc.). Such components outside the acoustic analysis tool may be provided, for example, via external devices such as computers, smart phones, tablet devices, wearable devices, etc. In addition or alternatively, other test and measurement or data acquisition tools configured to act as a master or slave device relative to the acoustic analysis tool may similarly provide various components of the acoustic analysis system outside the acoustic analysis tool. The external device may communicate with the portable acoustic analysis tool via a wired and / or wireless connection and may be used to perform various processing, display and / or interfacing steps.

[0074] In some embodiments, such an external device can provide redundant functionality as a component housed in the portable acoustic analysis tool. For example, in some embodiments, the acoustic analysis tool can include a display for displaying acoustic image data, and can be further configured to transmit the image data to an external device for storage and / or display. Similarly, in some embodiments, a user can interface with the acoustic analysis tool via an application ("app") running on a smart phone, tablet device, computer, etc., to perform one or more functions that can also be performed with the acoustic analysis tool itself.

[0075] Figure 3A is a schematic diagram of an exemplary configuration of an acoustic sensor array within an acoustic analysis system. In the illustrated example, the acoustic sensor array 302 includes: a plurality of first acoustic sensors (shown in white) and a plurality of second acoustic sensors (shaded). The first acoustic sensors are arranged into a first array 320, and the second acoustic sensors are arranged into a second array 322. In some examples, the first array 320 and the second array 322 can be selectively used to receive acoustic signals for generating acoustic image data. For example, in some configurations, the sensitivity of a particular acoustic sensor array to a particular acoustic frequency is a function of the distance between the acoustic sensor elements.

[0076] In some configurations, sensor elements that are spaced closer together (e.g., second array 322) are better able to resolve high frequency acoustic signals (e.g., sounds with frequencies greater than 20 kHz, such as ultrasonic signals between 20 kHz and 100 kHz) than sensor elements that are spaced further apart (e.g., first array 320). Similarly, sensor elements that are spaced further apart (e.g., first array 320) may be better suited for detecting low frequency acoustic signals (e.g., <20 kHz) than sensor elements that are spaced more closely together (e.g., second array 322). Various acoustic sensor arrays having sensor elements spaced apart from one another may be provided for detecting acoustic signals having various frequency ranges, such as infrasonic frequencies (<20 Hz), audible frequencies (between approximately 20 Hz and 20 kHz), ultrasonic frequencies (between 20 kHz and 100 kHz). In some embodiments, a portion of the array (e.g., each other acoustic sensor element from array 320) may be used to optimize detection of a particular frequency band.

[0077] Additionally, in some examples, some acoustic sensor elements may be more suitable for detecting acoustic signals with different frequency characteristics (such as low frequency or high frequency). Therefore, in some embodiments, an array configured to detect low-frequency acoustic signals (such as a first array 320 having sensor elements spaced farther apart) may include a first acoustic sensor element that is more suitable for detecting low-frequency acoustic signals. Similarly, an array configured to detect higher-frequency acoustic signals (such as a second array 322) may include a second acoustic sensor element that is more suitable for detecting high-frequency acoustic signals. Therefore, in some examples, the first array 320 and the second array 322 of acoustic sensor elements may include different types of acoustic sensor elements. Alternatively, in some embodiments, the first array 320 and the second array 322 may include the same type of acoustic sensor elements.

[0078] Thus, in an exemplary embodiment, the acoustic sensor array 302 may include multiple arrays of acoustic sensor elements, such as a first array 320 and a second array 322. In some embodiments, the arrays may be used individually or in combination. For example, in some examples, a user may choose to use the first array 320, the second array 322, or both the first array 320 and the second array 322 to perform an acoustic imaging process. In some examples, a user may select which array(s) to use via a user interface. Additionally or alternatively, in some embodiments, the acoustic analysis system may automatically select the array(s) to use based on an analysis of the received acoustic signal or other input data (such as an expected frequency range, etc.). Although Figure 3A The configuration shown in generally includes: two arrays (first array 320 and second array 322) generally arranged in a rectangular grid, but it will be appreciated that multiple acoustic sensor elements can be grouped into any number of discrete arrays of any shape. In addition, in some embodiments, one or more acoustic sensor elements can be included in multiple distinguished arrays that can be selected for operation. As described elsewhere herein, in various embodiments, a process for backpropagating acoustic signals to establish acoustic image data from a scene is implemented based on the arrangement of acoustic sensor elements. Therefore, the arrangement of acoustic sensors can be known or otherwise accessible to the processor in order to implement the acoustic image generation technique.

[0079] Figure 3A The acoustic analysis system further includes: a distance measurement tool 304 and a camera module 306 positioned within the acoustic sensor array 302. The camera module 306 may represent a camera module of an electromagnetic imaging tool (e.g., 203), and may include a visible light camera module, an infrared camera module, an ultraviolet camera module, and the like. Figure 3A306. Although not shown, the acoustic analysis system may include one or more additional camera modules of the same type or a different type than the camera module 306. In the illustrated example, the distance measurement tool 304 and the camera module 306 are positioned within a grid of acoustic sensor elements in the first array 320 and the second array 322. Although shown as being disposed between grid sites within the first array 320 and the second array 322, in some embodiments, one or more components (e.g., the camera module 306 and / or the distance measurement tool 304) may be positioned at corresponding one or more grid sites in the first array 320 and / or the second array 322. In some such embodiments, the component(s) may be positioned at the grid sites in place of acoustic sensor elements that would normally be in such locations according to the grid arrangement.

[0080] As described elsewhere herein, an acoustic sensor array may include acoustic sensor elements arranged in any of a wide variety of configurations. Figure 3B and Figure 3C is a schematic diagram illustrating an exemplary acoustic sensor array configuration. Figure 3B An acoustic sensor array 390 is shown that includes a plurality of acoustic sensor elements evenly spaced apart in an approximately square grid. A distance measurement tool 314 and a camera array 316 are positioned within the acoustic sensor array 390. In the illustrated example, the acoustic sensor elements in the acoustic sensor array 390 are the same type of sensor, although in some embodiments, different types of acoustic sensor elements may be used in the array 390.

[0081] Figure 3C Multiple acoustic sensor arrays are shown. Acoustic sensor arrays 392, 394, and 396 each include multiple acoustic sensor elements arranged in arrays of different shapes. Figure 3C In the example of the present invention, acoustic sensor arrays 392, 394, and 396 can be used individually or together in any combination to create sensor arrays of various sizes. In the illustrated embodiment, the sensor elements of array 396 are spaced more closely together than the sensor elements of array 392. In some examples, array 396 is designed to sense high-frequency acoustic data, while array 392 is designed to sense low-frequency acoustic data.

[0082] In various embodiments, arrays 392, 394, and 396 may include the same or different types of acoustic sensor elements. For example, acoustic sensor array 392 may include sensor elements having a lower frequency operating range than sensor elements of acoustic sensor array 396.

[0083] As described elsewhere herein, in some examples, different acoustic sensor arrays (e.g., 392, 394, 396) can be selectively turned off and on during various operating modes (e.g., different desired frequency spectra to be imaged). Additionally or alternatively, various acoustic sensor elements (e.g., some or all of the acoustic sensor elements in one or more sensor arrays) can be enabled or disabled depending on the desired system operation. For example, in some acoustic imaging processes, while data from a large number of sensor elements (e.g., sensor elements arranged in a high density, such as in sensor array 396) slightly improves the resolution of the acoustic image data, this is at the expense of the processing required to extract the acoustic image data from the data received at each sensor element. That is, in some examples, the increased processing requirements (e.g., cost, processing time, power consumption, etc.) required to process a large number of input signals (e.g., signals from a large number of acoustic sensor elements) are negative compared to any additional signal resolution provided by the additional data streams. Therefore, in some embodiments, it may be worthwhile to disable or ignore data from one or more acoustic sensor elements depending on the desired acoustic imaging operation.

[0084] and Figure 3A and Figure 3B Similar to the system, Figure 3C The system includes a distance measurement tool 314 and a camera array 316 positioned within the acoustic sensor arrays 392, 394, and 396. In some examples, additional components such as additional camera arrays (e.g., used to image different portions of the electromagnetic spectrum from the camera array 316) may be similarly positioned within the acoustic sensor arrays 392, 394, and 396. It will be appreciated that although in Figures 3A-3C , but the distance measurement tool and / or one or more imaging tools (e.g., a visible light camera module, an infrared camera module, an ultraviolet sensor, etc.) may be located outside the acoustic sensor array(s). In some such examples, the distance measurement tool and / or one or more imaging tools located outside the acoustic sensor array(s) may be supported by the acoustic imaging tool, for example, by a housing that houses the acoustic sensor array(s), or may be located outside the housing of the acoustic imaging tool.

[0085] In some examples, general misalignment of the acoustic sensor array and an imaging tool (such as a camera module) can result in misalignment in the corresponding image data generated by the acoustic sensor array and the imaging tool. Figure 4AA schematic diagram of parallax error in the generation of frames of visible light image data and acoustic image data is shown. In general, parallax error can be vertical, horizontal, or both. In the illustrated embodiment, acoustic sensor array 420 and imaging tool include visible light camera module 406. Visible light image frame 440 is shown as being captured according to field of view 441 of visible light camera module 406, while acoustic image frame 450 is shown as being captured according to field of view 451 of acoustic sensor array 420.

[0086] As shown, visible light image frame 440 and acoustic imaging frame 450 are not aligned with each other. In some embodiments, the processor (e.g., Figure 2 Processor 212 of the system is configured to manipulate one or both of visible light image frame 440 and acoustic image frame 450 so as to align the visible light image data and the acoustic image data. Such manipulation may include shifting one image frame relative to the other image frame. The amount by which the image frames are shifted relative to each other may be determined based on a variety of factors, including, for example, the distance from visible light camera module 406 and / or acoustic sensor array 420 to the target. Such distance data may be determined, for example, using distance measurement tool 404 or receiving a distance value via a user interface (e.g., 216).

[0087] Figure 4B is similar to Figure 4A , but includes a visible light image of the scene. Figure 4B In the example of , visible light image 442 shows a scene with multiple power lines and support towers. Acoustic image 452 includes multiple locations 454, 456, 458 that indicate high magnitude acoustic data from such locations. As shown, both visible light image 442 and acoustic image 452 are displayed simultaneously. However, observation of the two images shows at least one acoustic image local maximum at location 458 that does not appear to be consistent with any particular structure in visible light image 442. Therefore, a person observing the two images can conclude that there is a misalignment (e.g., a parallax error) between acoustic image 452 and visible light image 442.

[0088] Figure 5A and Figure 5B The parallax correction between the visible light image and the acoustic image is shown. Figure 4B , Figure 5A Visible light image 542 and acoustic image 552 are shown. Acoustic image 552 includes local maxima at locations 554, 556, and 558. As can be seen, the maxima at locations 554 and 558 do not appear to coincide with any structure in the visible light image. Figure 5BIn the example of , visible light image 542 and acoustic image 552 are registered with respect to each other. Now, the local maxima at locations 554, 556, and 558 in the acoustic image appear to coincide with respective locations in visible light image 542.

[0089] During use, the operator can view (e.g., via display 214) Figure 5B , and determine approximate locations in the visible scene 542 that are likely to be sources of the received acoustic signals. Such signals may be further processed to determine information about the acoustic signatures of various components in the scene. In various embodiments, acoustic parameters such as frequency content, periodicity, amplitude, etc. may be analyzed relative to various locations in the acoustic image. When overlaid onto the visible light data so that such parameters can be associated with various system components, the acoustic image data may be used to analyze various properties (e.g., performance characteristics) of objects in the visible light image.

[0090] Figure 5C and Figure 5D yes Figure 5A and Figure 5B A color version of Figure 5A and Figure 5B shown in and Figure 5C and Figure 5D As can be more easily seen in the color representation of , positions 554, 556, and 558 show a circular gradient in color. As described elsewhere herein, the acoustic image data can be visually represented according to a color scheme in which each pixel of the acoustic image data is colored based on the acoustic intensity at the corresponding position. Thus, in Figures 5A-5D In the exemplary representation of , the circular gradients at locations 554 , 556 , 558 generally represent gradients in acoustic intensity in the imaging plane based on the back-propagating received acoustic signal.

[0091] It will be appreciated that although the description is given with respect to acoustic image data and visible light image data, Figure 4A , 4B , 5A-5D, but such processes may be similarly performed using a variety of electromagnetic image data. For example, as described elsewhere herein, in various embodiments, various such processes may be performed using a combination of acoustic image data with one or more of visible light image data, infrared image data, ultraviolet image data, etc.

[0092] As described elsewhere herein, in some embodiments, backpropagating the acoustic signal to form an acoustic image can be based on a value of the distance to the target. That is, in some examples, the backpropagation calculation can be based on the distance and can include determining a two-dimensional acoustic scene that is located at that distance from the acoustic sensor array. Given a two-dimensional imaging plane, a cross-section of a spherical acoustic wave emanating from a source in the plane will generally appear circular, with the intensity decaying in the radial direction, such as Figure 5A-5B as shown in .

[0093] In some such examples, portions of the acoustic scene representing data at distances to targets that are not used in the back-propagation calculations will result in errors in the acoustic image data, such as inaccurate locations of one or more sounds in the scene. When the acoustic image is displayed simultaneously (e.g., mixed, combined, etc.) with other image data (e.g., electromagnetic image data, such as visible light, infrared, or ultraviolet image data), such errors may result in parallax errors between the acoustic image data and the other image data. Therefore, in some embodiments, some techniques for correcting parallax errors (e.g., such as Figure 5A and Figure 5B ) includes: adjusting the value of the distance to the target used in the back-propagation calculation for generating the acoustic image data.

[0094] In some cases, the system can be configured to use the value of the first distance to the target to perform a back-propagation process and display values ​​such as Figure 5A , where the acoustic image data and the other data stream may be misaligned. The acoustic analysis system may then adjust the value of the distance-to-target used for backpropagation, perform backpropagation again, and update the display image with the new acoustic image data. This process may be repeated, with the acoustic analysis system cycling through multiple distance-to-target values ​​while the user observes the resulting display image on the display. As the value of the distance-to-target changes, the user may observe Figure 5A The displayed image is shown to Figure 5BA gradual transition of the displayed image shown. In some such cases, the user can visually observe when the acoustic image data appears to be correctly aligned with another data stream (such as electromagnetic image data). The user can send a signal to the acoustic analysis system indicating that the acoustic image data appears to be correctly aligned, thereby indicating to the system that the value of the distance to the target used to implement the most recent back-propagation is approximately correct, and the distance value can be saved to the memory as the correct distance to the target. Similarly, when the displayed image is updated with a new distance value during the updated back-propagation process, the user can manually adjust the value of the distance to the target until the user observes that the acoustic image data is correctly aligned. The user can choose to save the current distance to the target as the current distance to the target in the acoustic analysis system.

[0095] In some examples, correcting for parallax error can include adjusting the orientation of the acoustic image data relative to other image data (e.g., electromagnetic image data) by a predetermined amount and in a predetermined direction based on the distance to the target data. In some embodiments, such adjustment is independent of the generation of the acoustic image data by back-propagating the acoustic signal to the identified distance to the target.

[0096] In some embodiments, in addition to being used to generate acoustic image data and reduce parallax errors between the acoustic image data and other image data, the value of the distance to the target can also be used to make other determinations. For example, in some examples, as described in U.S. Patent No. 7,538,326, a processor (e.g., 212) can use the value of the distance to the target to focus or help a user focus an image such as an infrared image, which is incorporated by reference. As described therein, this can similarly be used to correct parallax errors between visible light image data and infrared image data. Therefore, in some examples, the distance value can be used to align acoustic image data with electromagnetic imaging data (such as infrared image data and visible light image data).

[0097] As described elsewhere herein, in some examples, the distance measurement tool (e.g., 204) is configured to provide distance information that can be used by the processor (e.g., 212) to generate and register acoustic image data. In some embodiments, the distance measurement tool includes a laser rangefinder that is configured to emit light onto the target scene at the location where the distance is measured. In some such examples, the laser rangefinder can emit light in the visible spectrum so that a user can view the laser point in the physical scene to ensure that the rangefinder is measuring the distance to the desired portion of the scene. Additionally or alternatively, the laser rangefinder is configured to emit light in a spectrum to which one or more imaging components (e.g., a camera module) are sensitive. Therefore, a user viewing the target scene via the analysis tool (e.g., via display 214) can observe the laser point in the scene to ensure that the laser is measuring the distance to the correct location in the target scene. In some examples, the processor (e.g., 212) can be configured to generate a reference marker in the displayed image that represents the location where the laser point will be located in the acoustic scene based on the current distance value (e.g., based on a known distance-based parallax relationship between the laser rangefinder and the acoustic sensor array). The location of the reference marker can be compared to the location of the actual laser marker (e.g., graphically on the display and / or physically in the target scene), and the scene can be adjusted until the reference marker and the laser are consistent. Such a process can be implemented similar to the infrared registration and focusing techniques described in U.S. Patent No. 7,538,326, which is incorporated by reference.

[0098] Figure 6 6 is a process flow diagram illustrating an exemplary method for generating a final image combining acoustic image data and electromagnetic image data. The method includes the following steps: receiving an acoustic signal via an acoustic sensor array (680); and receiving distance information (682). The distance information may be received, for example, via a distance measurement device and / or a user interface, such as via manual input, or as a result of a distance adjustment process by which the distance is determined based on an observed registration.

[0099] The method further includes back-propagating the received acoustic signals to determine acoustic image data representative of the acoustic scene (684). As described elsewhere herein, back-propagating can include analyzing a plurality of acoustic signals received at a plurality of sensor elements in the acoustic sensor array in combination with the received distance information to determine a source pattern of the received acoustic signals.

[0100] Figure 6The method further includes the steps of capturing electromagnetic image data (686), and registering the acoustic image data with the electromagnetic image data (688). In some embodiments, registering the acoustic image data with the electromagnetic image data is performed as part of the back-propagation step used to generate the acoustic image data (684). In other examples, registering the acoustic image data with the electromagnetic image data is performed separately from generating the acoustic image data.

[0101] Figure 6 The method includes the steps of combining acoustic image data with electromagnetic image data to generate a display image (690). As described elsewhere herein, combining the electromagnetic image data and the acoustic image data may include alpha blending the electromagnetic image data and the acoustic image data. Combining the image data may include superimposing one image data set on another image data set, such as in a picture-in-picture mode or in a position where certain conditions (e.g., an alarm condition) are met. The display image may be presented to a user, for example, via a display supported by a housing that supports the acoustic sensor array, and / or via a display separate from the sensor array, such as a display of an external device (e.g., a smart phone, a tablet device, a computer, etc.).

[0102] Additionally or alternatively, the display image can be saved in local (e.g., onboard) memory and / or remote memory for future viewing. In some embodiments, the saved display image can include metadata that allows for future adjustment of display image properties, such as mixing ratio, back propagation distance, or other parameters used to generate the image. In some examples, raw acoustic signal data and / or electromagnetic image data can be saved with the display image for subsequent processing or analysis.

[0103] Although shown as a method for generating a final image combining acoustic image data and electromagnetic image data, it will be appreciated that Figure 6 The method can be used to combine acoustic image data with one or more image data sets spanning any portion of the electromagnetic spectrum, such as visible light image data, infrared image data, ultraviolet image data, etc. In some such examples, the acoustic image data can be combined with one or more image data sets spanning any portion of the electromagnetic spectrum, such as visible light image data, infrared image data, ultraviolet image data, etc. Figure 6 In a similar manner to the method described, multiple sets of image data, such as visible light image data and infrared image data, are combined with acoustic image data to generate a display image.

[0104] In some examples, receiving an acoustic signal via a sensor array (680) can include the following steps: selecting an acoustic sensor array with which to receive the acoustic signal. Figure 3A-C, the acoustic analysis system may include multiple acoustic sensor arrays, which may be suitable for analyzing acoustic signals with varying frequencies. Additionally or alternatively, in some examples, different acoustic sensor arrays may be used to analyze acoustic signals propagating from different distances. In some embodiments, different arrays may be nested within each other. Additionally or alternatively, portions of the array may be selectively used to receive acoustic image signals.

[0105] For example, Figure 3A A first array 320 and a second array 322 nested inside the first array are shown. In an exemplary embodiment, the first array 320 may include a sensor array configured (e.g., spaced apart) to receive acoustic signals and generate acoustic image data for frequencies within a first frequency range. The second array 322 may include, for example, a second sensor array configured to be used alone or in combination with all or part of the first array 320 to generate acoustic image data for frequencies within a second frequency range.

[0106] Similarly, Figure 3C A first array 392, a second array 394 at least partially nested within the first array 392, and a third array 396 at least partially nested within the first array 392 and the second array 394 are shown. In some embodiments, the first array 392 can be configured to receive acoustic signals and generate acoustic image data for frequencies within a first frequency range. The second array 394 can be used with all or part of the first array 392 to receive acoustic signals and generate acoustic image data for frequencies within a second frequency range. The third array 396 can be used alone, with all or part of the second array 394, and / or with all or part of the first array 392 to receive acoustic signals and generate acoustic image data for frequencies within a third frequency range.

[0107] In some embodiments, in a nested array configuration, acoustic sensor elements from one array may be positioned between acoustic sensor elements, such as elements of third array 396 being generally positioned between elements of first array 392. In some such examples, acoustic sensor elements in a nested array (e.g., third array 396) may be positioned in the same plane as, in front of, or behind acoustic sensor elements in the array in which they are nested (e.g., first array 392).

[0108] In various implementations, arrays used to sense higher frequency acoustic signals generally require smaller distances between individual sensors. Figure 3C, for example, the third array 396 may be better suited for conducting acoustic imaging processes involving high frequency acoustic signals. Other sensor arrays (e.g., the first array 392) may be sufficient to conduct acoustic imaging processes involving lower frequency signals, and other sensor arrays may be used when compared to the array 396 in order to reduce the computational requirements of processing signals from a smaller number of acoustic sensor elements. Thus, in some examples, a high frequency sensor array may be nested within a low frequency sensor array. As described elsewhere herein, such arrays may generally be operated individually (e.g., via switching between active arrays) or together.

[0109] In addition to or as an alternative to selecting an appropriate sensor array based on an expected / desired spectrum for analysis, in some examples, different sensor arrays may be more suitable for conducting the acoustic imaging process at different distances to the target scene. For example, in some embodiments, if the distance between the acoustic sensor array and the target scene is small, then the outer sensor elements in the acoustic sensor array may receive significantly less useful acoustic information from the target scene than the more centrally located sensor elements.

[0110] On the other hand, if the distance between the acoustic sensor array and the target scene is large, the closely spaced acoustic sensor elements may not provide useful information individually. That is, if the first and second acoustic sensor elements are close together, and the target scene is generally far away, the second acoustic sensor element may not provide any information that is meaningfully different from the first acoustic sensor element. Therefore, the data streams from such first and second sensor elements may be redundant and unnecessarily consume processing time and resources for analysis.

[0111] As described elsewhere herein, in addition to influencing which sensor arrays may be best suited for performing acoustic imaging, the distance to the target can also be used to perform back propagation for determining acoustic image data based on the received acoustic signal. However, in addition to being an input value to the back propagation algorithm, the distance to the target can be used to select an appropriate back propagation algorithm to be used. For example, in some examples, at long distances, spherically propagated sound waves can be approximated as being substantially planar compared to the size of the acoustic sensor array. Therefore, in some embodiments, when the distance to the target is large, the back propagation of the received acoustic signal may include an acoustic beamforming calculation. However, when closer to the source of the sound wave, the planar approximation of the sound wave may not be appropriate. Therefore, different back propagation algorithms, such as near-field acoustic holography, can be used.

[0112] As described, the measure of distance to a target can be used in a variety of ways in an acoustic imaging process, such as determining (one or more) active sensor arrays, determining a back-propagation algorithm, implementing a back-propagation algorithm, and / or aligning the resulting acoustic image with electromagnetic image data (e.g., visible light, infrared, etc.). Figure 7 is a process flow diagram illustrating an exemplary process for generating acoustic image data from received acoustic signals.

[0113] Figure 7 The process includes receiving distance information (780), such as from a distance measuring device or input, such as via a user interface. The method further includes the step of selecting one or more acoustic sensor arrays for performing acoustic imaging based on the received distance information (782). As described, in various examples, the selected (one or more) arrays may include a single array, a combination of multiple arrays, or a portion of one or more arrays.

[0114] Figure 7 The method further includes the step of selecting a processing scheme for performing acoustic imaging based on the received distance information (784). In some examples, selecting a processing scheme may include selecting a back propagation algorithm for generating acoustic image data from an acoustic signal.

[0115] After selecting an acoustic sensor array (782) and a processing scheme (784) for performing acoustic imaging, the method includes the steps of receiving an acoustic signal via the selected acoustic sensor array (786) and then backpropagating the received acoustic signal using the distance and the selected processing scheme to determine acoustic image data (788).

[0116] In various embodiments, Figure 7 The steps of may be performed by a user, an acoustic analysis system (e.g., via processor 212), or a combination thereof. For example, in some embodiments, the processor may be configured to receive distance information (780) via a distance measurement tool and / or user input. In some examples, for example, if the distance to the object is known and / or difficult to analyze via a distance measurement tool (e.g., the object size is small and / or the distance to the target is large, etc.), the user may enter a value to override the measured distance for use as the distance information. The processor may be further configured to automatically select an appropriate acoustic sensor array for performing acoustic imaging based on the received distance information, for example, using a lookup table or other database. In some embodiments, selecting the acoustic sensor array includes enabling and / or disabling one or more acoustic sensor elements in order to obtain the desired acoustic sensor array.

[0117] Similarly, in some examples, the processor can be configured to automatically select a processing scheme (e.g., a back propagation algorithm) for performing acoustic imaging based on the received distance information. In some such examples, this can include selecting one from a plurality of known processing schemes stored in a memory. Additionally or alternatively, selecting a processing scheme can be equivalent to adjusting portions of a single algorithm to obtain a desired processing scheme. For example, in some embodiments, a single back propagation algorithm can include multiple terms and variables (e.g., based on distance information). In some such examples, selecting a processing scheme (784) can include defining one or more values ​​in a single algorithm, such as adjusting coefficients of one or more terms (e.g., setting various coefficients to zero or one, etc.).

[0118] Thus, in some embodiments, the acoustic imaging system can automate several steps of the acoustic imaging process by suggesting and / or automatically implementing a selected acoustic sensor array and / or processing scheme (e.g., a back-propagation algorithm) based on the received distance data. This can speed up, improve, and simplify the acoustic imaging process, thereby eliminating the requirement for an acoustic imaging expert to perform the acoustic imaging process. Thus, in various examples, the acoustic imaging system can automatically implement such parameters, notify a user that such parameters are about to be implemented, request permission from a user to implement such parameters, suggest such parameters for manual input by a user, and the like.

[0119] Automatic selection and / or suggestion of such parameters (e.g., processing schemes, sensor arrays) may be useful for optimizing the positioning of acoustic image data relative to other forms of image data, processing speed, and analysis of acoustic image data. For example, as described elsewhere herein, accurate back-propagation determination (e.g., using appropriate algorithms and / or accurate distance metrics) can reduce parallax errors between acoustic image data and other (e.g., electromagnetic waves, such as visible light, infrared, etc.) image data. Additionally, the accuracy of thermal image data can be optimized using appropriate algorithms and / or sensor arrays, such as may be automatically selected or suggested by an acoustic analysis system, thereby allowing analysis of received acoustic data.

[0120] As described, in some examples, the acoustic analysis system can be configured to automatically select an algorithm and / or sensor array for performing an acoustic imaging process based on the received distance information. In some such embodiments, the system includes a lookup table, such as stored in a memory, for determining which of a plurality of back-propagation algorithms and acoustic sensor arrays to use to determine acoustic image data. Figure 8 An exemplary lookup table for determining the appropriate algorithm and sensor array to use during an acoustic imaging procedure is shown.

[0121] In the example shown, Figure 8The lookup table includes N columns, each column representing a different array: Array 1, Array 2, ..., Array N. In various examples, each array includes a unique set of arranged acoustic sensor elements. Different arrays may include sensor elements arranged in a grid (e.g., Figure 3C Arrays 392 and 396 in the lookup table). An array within the lookup table may also include combinations of sensor elements from one or more such grids. Generally speaking, in some embodiments, each of the arrays: Array 1, Array 2, ..., Array N corresponds to a unique combination of acoustic sensor elements. Some of such combinations may include an entire set of sensor elements arranged in a particular grid, or may include a subset of sensor elements arranged in a particular grid. Any of a variety of combinations of acoustic sensor elements are possible options for use as a sensor array in the lookup table.

[0122] Figure 8 The query table further includes M rows, each row representing a different algorithm: Algorithm 1, Algorithm 2, ..., Algorithm M. In some examples, the different algorithms may include different processes for performing back propagation analysis on the received sound signal. As described elsewhere herein, in some examples, some different algorithms may be similar to each other, while having different coefficients and / or terms for modifying the back propagation results.

[0123] Figure 8 An exemplary lookup table of includes M×N entries. In some embodiments, an acoustic analysis system utilizing such a lookup table is configured to analyze the received distance information and classify the distance information into one of M×N bins, wherein each bin corresponds to Figure 8 In such an example, when the acoustic analysis system receives the distance information, the system can find the entry (i, j) in the lookup table corresponding to the bin in which the distance information is located, and determine the appropriate algorithm and sensor array for use during the acoustic imaging process. For example, if the received distance information corresponds to the bin associated with entry (i, j), the acoustic analysis system can automatically utilize or suggest the use of algorithm i and array j for the acoustic imaging process.

[0124] In various such examples, the distance information bins may correspond to uniformly sized distance ranges, e.g., a first bin corresponding to distances within one foot, a second bin corresponding to distances between one foot and two feet, and so on. In other examples, the bins need not correspond to uniformly sized distance spans. Additionally, in some embodiments, fewer than M×N bins may be used. For example, in some embodiments, there may be an algorithm (e.g., Algorithmx) that has never been used for a particular array (e.g., Arrayy). Thus, in such an example, there would be no corresponding distance information bin corresponding to entry (x, y) in the M×N lookup table.

[0125] In some embodiments, statistical analysis of the populated distance bins can be used to identify the most common distances or distance ranges within the target scene. In some such embodiments, the distance bins with the highest number of corresponding locations (e.g., locations with the highest number of acoustic signals) can be used as Figure 7 That is, in some embodiments, the acoustic sensor array and / or processing scheme utilized can be implemented and / or recommended based on a statistical analysis of the distance distribution of various objects in the target scene. This can increase the likelihood that the sensor array and / or processing scheme used to acoustically image the scene is suitable for the maximum number of locations within the acoustic scene.

[0126] Additionally or alternatively, parameters other than distance information can be used to select an appropriate sensor array and / or processing scheme to use when generating acoustic image data. As described elsewhere herein, various sensor arrays can be configured to be sensitive to certain frequencies and / or frequency bands. In some examples, different back propagation calculations can be similarly used based on different acoustic signal frequency content. Therefore, in some examples, one or more parameters can be used to determine a processing scheme and / or acoustic sensor array.

[0127] In some embodiments, an acoustic analysis system may be used to initially analyze various parameters of the received acoustic signal processing / analysis. Figure 7 , a method for generating acoustic image data may include the steps of: after receiving an acoustic signal (786), analyzing the frequency content of the received signal (790). In some such examples, if (one or more) acoustic sensor arrays and / or processing schemes have been selected (e.g., performed via steps 782 and / or 784, respectively), the method may include the steps of: updating (one or more) selected arrays and / or updating the selected processing scheme (792), for example, based on the analyzed frequency content.

[0128] After updating the sensor array(s) and / or processing scheme, the method may perform various actions using the updated parameters. For example, if the selected sensor array(s) (792) are updated based on the analyzed frequency content (790), a new acoustic signal may be received from the (most recently) selected acoustic sensor array (786), which may then be back-propagated to determine acoustic image data (788). Alternatively, if the processing scheme is updated at 792, the already captured acoustic signals may be back-propagated according to the updated processing scheme to determine updated acoustic image data. If both the processing scheme and the sensor array(s) are updated, a new acoustic signal may be received using the updated sensor array, and the new acoustic signal may be back-propagated according to the updated processing scheme.

[0129] In some embodiments, the acoustic analysis system may receive frequency information (778) without analyzing the frequency content (790) of the received acoustic signal. For example, in some examples, the acoustic analysis system may receive information about a desired or expected frequency range for future acoustic analysis. In some such examples, the desired or expected frequency information may be used to select one or more sensor arrays and / or a processing scheme that best suits the frequency information. In some such examples, the step(s) of selecting (one or more) acoustic sensor arrays (782) and / or selecting a processing scheme (784) may be based on the received frequency information in addition to or in lieu of the received distance information.

[0130] In some examples, the received acoustic signal (e.g., an acoustic signal received via an acoustic sensor element) can be analyzed, for example, via a processor (e.g., 210) of an acoustic analysis system. Such analysis can be used to determine one or more properties of the acoustic signal, such as frequency, intensity, periodicity, apparent proximity (e.g., a distance estimated based on the received acoustic signal), measured proximity, or any combination thereof. In some examples, the acoustic image data can be filtered, for example, to show only acoustic image data representing acoustic signals having a particular frequency content, periodicity, etc. In some examples, any number of such filters can be applied simultaneously.

[0131] As described elsewhere herein, in some embodiments, a series of acoustic image data frames may be captured over time, similar to acoustic video data. Additionally or alternatively, even if acoustic image data is not generated repeatedly, in some examples, acoustic signals are repeatedly sampled and analyzed. Thus, parameters of acoustic data (such as frequency) may be monitored over time, with or without repeated acoustic image data generation (e.g., video).

[0132] Fig.9A is an exemplary plot of the frequency content of image data received over time in an acoustic scene. As shown, Fig.9A The acoustic scene represented by the plot of typically includes four frequencies that persist over time, labeled Frequency 1, Frequency 2, Frequency 3, and Frequency 4. Frequency data, such as the frequency content of the target scene, can be determined by processing the received sound signal, for example using a Fast Fourier Transform (FFT) or other known frequency analysis methods.

[0133] Fig. 9B An exemplary scene including multiple locations where acoustic signals are emitted is shown. In the illustrated image, acoustic image data is combined with visible light image data, and acoustic signals present at locations 910, 920, 930, and 940 are shown. In some embodiments, the acoustic analysis system is configured to display acoustic image data belonging to any detected frequency range. For example, in an exemplary embodiment, location 910 includes acoustic image data including frequency 1, location 920 includes acoustic image data including frequency 2, location 930 includes acoustic image data including frequency 3, and location 940 includes acoustic image data including frequency 4.

[0134] In some such examples, displaying acoustic image data representing a frequency range is an optional operating mode. Similarly, in some embodiments, the acoustic analysis system is configured to display acoustic image data representing frequencies only within a predetermined frequency band. In some such examples, displaying acoustic image data representing a predetermined frequency range includes: selecting one or more acoustic sensor arrays for receiving acoustic signals from which acoustic image data is generated. Such an array can be configured to receive a selective frequency range. Similarly, in some examples, one or more filters can be used to limit the frequency content used to generate acoustic image data. Additionally or alternatively, in some embodiments, acoustic image data including information representing a wide frequency range can be analyzed and shown on a display only if the acoustic image data meets a predetermined condition (e.g., falls within a predetermined frequency range).

[0135] Fig. 9CA plurality of combined acoustic and visible light image data at a plurality of predefined frequency ranges are shown. The first image includes acoustic image data at a first position 910, the first position 910 including frequency content at frequency 1. The second image includes acoustic image data at a second position 920, the second position 920 including frequency content at frequency 2. The third image includes acoustic image data at a third position 930, the third position 930 including frequency content at frequency 3. The fourth image includes acoustic image data at a fourth position 940, the fourth position 940 including frequency content at frequency 4.

[0136] In an exemplary embodiment, a user may select various frequency ranges, such as a range including frequency 1, frequency 2, frequency 3, or frequency 4, for filtering acoustic image data representing frequency content other than the selected frequency range. Thus, in such an example, any of the first, second, third, or fourth images may be displayed due to the user selecting a desired frequency range.

[0137] Additionally or alternatively, in some examples, the acoustic analysis system can cycle between multiple display images, each display image having a different frequency content. Fig. 9C In an exemplary embodiment, the acoustic analysis system may display the first, second, third, and fourth images in sequence, such as Fig. 9C As indicated by the arrow in .

[0138] In some examples, the displayed image may include text or other displays representing the frequency content displayed in the image, so that a user can observe which locations in the image include acoustic image data representing certain frequency content. Fig. 9C , each image may show a textual representation of the frequencies represented in the acoustic image data. Fig. 9B , an image showing multiple frequency ranges can include an indication of the frequency content at each location, including acoustic image data. In some such examples, a user can select, for example, via a user interface, a location in the image for which frequency content present at the location is to be viewed in the acoustic scene. For example, a user can select a first location 910, and the acoustic analysis system can present the frequency content of the first location (e.g., frequency 1). Thus, in various examples, a user can use the acoustic analysis system to analyze the frequency content of an acoustic scene, such as by seeing where in the scene corresponds to a particular frequency content, and / or by seeing what frequency content is present at various locations.

[0139] During an exemplary acoustic imaging operation, filtering the acoustic image data by frequency can help reduce image clutter, for example, from background or other insignificant sounds. During an exemplary acoustic imaging process, a user may wish to eliminate background sounds, such as a noise floor in an industrial environment. In some such cases, the background noise may consist primarily of low frequency noise. Thus, a user may select to display acoustic image data that represents acoustic signals greater than a predetermined frequency (e.g., 10 kHz). In another example, a user may wish to analyze specific objects that typically emit acoustic signals within a certain range, such as corona discharges emanating from transmission lines (e.g., as Figures 5A-5D In such an example, a user may select a specific frequency range (e.g., between 11 kHz and 14 kHz for corona discharge) for acoustic imaging.

[0140] In some examples, the acoustic analysis system can be used to analyze and / or present information associated with the intensity of the received acoustic signal. For example, in some embodiments, backpropagating the received acoustic signal can include determining acoustic intensity values ​​at multiple locations in the acoustic scene. In some examples, similar to the above-mentioned frequency, the acoustic image data is included in the displayed image only if the intensity of the acoustic signal meets one or more predetermined requirements.

[0141] In various such embodiments, the displayed image may include acoustic image data representing acoustic signals above a predetermined threshold (e.g., 15 dB), acoustic signals below a predetermined threshold (e.g., 100 dB), or acoustic signals within a predetermined intensity range (e.g., between 15 dB and 40 dB). In some embodiments, the threshold may be based on a statistical analysis of the acoustic scene, such as above or below a standard deviation from the average acoustic intensity.

[0142] Similar to what is described above with respect to frequency information, in some embodiments, limiting the acoustic image data to represent acoustic signals that meet one or more intensity requirements may include filtering the received acoustic signals so that only received signals that meet predetermined conditions are used to generate the acoustic image data. In other examples, filtering the acoustic image data adjusts which acoustic image data is displayed.

[0143] Additionally or alternatively, in some embodiments, the acoustic intensity at a location within the acoustic scene may be monitored over time (e.g., in conjunction with a video acoustic image representation or via background analysis without having to update a displayed image). In some such examples, the predetermined requirements for displaying acoustic image data may include an amount or rate of change of the acoustic intensity at a location in the image.

[0144] Fig. 10A and 10Bis an exemplary display image including combined visible light image data and acoustic image data. Fig. 10A A display image including acoustic image data shown at a plurality of locations 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090 is shown. In some examples, the intensity values ​​may be toned, for example, where a color is assigned to the acoustic intensity values ​​based on a predetermined toning scheme. In an exemplary embodiment, the intensity values ​​may be classified according to intensity ranges (e.g., 10 dB–20 dB, 20 dB–30 dB, etc.). Each intensity range may be associated with a particular color according to the toning scheme. The acoustic image data may include a plurality of pixels, wherein each pixel is colored with a color associated with an intensity range in which the intensity represented by the pixel of the acoustic image data falls. In addition to or as an alternative to distinguishing by color, different intensities may be distinguished according to other attributes, such as transparency (e.g., in an image overlay that overlays acoustic image data onto other image data), and the like.

[0145] Additional parameters can also be colored, such as the rate of change of sound intensity. Similar to intensity, the rate of change of changing sound intensity can be colored so that parts of the scene showing different rates of change of sound intensity and / or amounts of change of sound intensity are displayed in different colors.

[0146] In the illustrated example, the acoustic image data is color-coded according to the intensity palette, so that the acoustic image data representing different acoustic signal intensities are shown in different colors and / or shades. For example, the acoustic image data at positions 1010 and 1030 show a palletized representation of a first intensity, positions 1040, 1060, and 1080 show a palletized representation of a second intensity, and positions 1020, 1050, 1070, and 1090 show a palletized representation of a third intensity. Fig. 10A As shown in the exemplary representation in , each location showing a color-toned representation of acoustic image data shows a circular pattern with a color gradient extending outward from the center. This may be due to the attenuation of acoustic intensity as the signal propagates from the acoustic signal source.

[0147] exist Fig. 10A In an example, the acoustic image data is combined with the visible light image data to generate a display image, which can be presented to a user, for example, via a display. The user can view Fig. 10A The display image can be used to see which positions in the visible scene are generating acoustic signals, as well as the strength of such signals. Thus, the user can quickly and easily observe where the sound is being generated, and compare the strength of the sound from various positions in the scene.

[0148] Similar to what is described with respect to frequencies elsewhere herein, in some embodiments, acoustic image data may be presented only if the corresponding acoustic signal satisfies a predetermined intensity condition. Fig. 10B Shows something like Fig. 10A An exemplary display image of a display image including visible light image data and an acoustic image representing an acoustic signal above a predetermined threshold. As shown, in the display image including the acoustic image data Fig. 10A Among the positions 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090, only the positions 1020, 1050, 1070, and 1090 include acoustic image data representing acoustic signals satisfying a predetermined condition.

[0149] In an exemplary scenario, Fig. 10A All acoustic image data above the noise floor threshold at each of the positions 1010-990 may be included, and Fig. 10B Shown with Fig. 10A The same scene, but it only shows the acoustic image data with an intensity greater than 40dB. This can help the user identify the environment (e.g. Fig. 10A and Fig. 10B which sound sources in a target scene) are contributing certain sounds (e.g., the loudest sounds in the scene).

[0150] As described elsewhere herein, in addition to or as an alternative to comparing directly to an intensity threshold (e.g., 40 dB), in some such examples, the predetermined requirement for displaying acoustic image data may include an amount of change or rate of change of acoustic intensity at a certain location in the image. In some such examples, acoustic image data may be presented only if the rate of change or amount of change of acoustic intensity at a given location satisfies a predetermined condition (e.g., greater than a threshold, less than a threshold, within a predetermined range, etc.). In some embodiments, the amount of change or rate of change of acoustic intensity may be toned and displayed as intensity acoustic image data, or displayed in conjunction with intensity acoustic image data. For example, in an exemplary embodiment, when the rate of change is used as a threshold to determine which locations include acoustic image data, the acoustic image data may include a toned intensity rate of change metric for display.

[0151] In some examples, a user can manually set intensity requirements (e.g., minimum value, maximum value, range, rate of change, amount of change, etc.) for the acoustic image data to be displayed. As discussed elsewhere herein, including only acoustic image data that meets the intensity requirements can be implemented during the generation of the acoustic image data (e.g., via filtering the received acoustic signal), and / or can be implemented by not displaying the generated acoustic image data that represents an acoustic signal that does not meet (one or more) set requirements. In some such examples, filtering the displayed image according to the intensity value can be implemented after the acoustic image data and the visible light image data have been captured and stored in the memory. That is, the data stored in the memory can be used to generate a displayed image including any number of filtering parameters, such as only showing acoustic image data that meets predefined intensity conditions, etc.

[0152] In some examples, setting a lower intensity limit in the acoustic image (e.g., only displaying acoustic image data representing acoustic signals above a predetermined intensity) can eliminate the inclusion of undesirable background or ambient sounds and / or acoustic reflections from the acoustic image data. In other cases, setting an upper intensity limit in the acoustic image (e.g., only displaying acoustic image data representing acoustic signals below a predetermined intensity) can eliminate the inclusion of expected loud sounds in the acoustic image data in order to observe acoustic signals that would normally be masked by such loud sounds.

[0153] Several display functions are possible. For example, similar to Fig. 9C In the frequency analysis / display discussed, in some examples, the acoustic analysis system can cycle through multiple display images, each display image showing acoustic image data that meets different intensity requirements. Similarly, in some examples, the user can scroll through a series of acoustic intensity ranges to view locations in the acoustic image data that have acoustic intensities within a given range.

[0154] Another parameter that can be used to analyze acoustic data is the periodicity value of the acoustic signal. Fig.11A and 11B An exemplary plot of frequency versus time of acoustic data in an acoustic scene is shown. Fig.11A As shown in the plot, the acoustic data includes: a signal with a frequency X having a first periodicity, a signal with a frequency Y having a second periodicity, and a signal with a frequency Z having a third periodicity. In the illustrated example, the acoustic signals with different frequencies may also include different periodicities in the acoustic signals.

[0155] In some such examples, acoustic signals may be filtered based on periodicity in addition to or in lieu of frequency content. For example, in some examples, multiple acoustic signal sources in an acoustic scene may generate acoustic signals at a particular frequency. If a user wishes to isolate one such acoustic source for acoustic imaging, the user may choose to include the acoustic image data in a final display image, or exclude the acoustic image data from a final display image based on the periodicity associated with the acoustic data.

[0156] Fig. 11B A plot of frequency versus time for an acoustic signal is shown. As shown, the frequency increases approximately linearly over time. However, as shown, the signal includes a periodicity that is approximately constant over time. Therefore, depending on the selected display parameters, such a signal may or may not appear in the acoustic image. For example, a signal may meet the frequency criteria for being displayed at certain points in time, but be outside the displayed frequency range at other points in time. However, a user can choose to include such a signal in the acoustic image data or exclude such a signal from the acoustic image data based on the periodicity of the signal, regardless of the frequency content.

[0157] In some examples, extracting acoustic signals with a particular periodicity can help analyze specific parts of the target scene (e.g., specific equipment items or types of equipment that typically operate with a particular periodicity). For example, if the object of interest operates with a certain periodicity (e.g., once per second), excluding signals with a periodicity different from this can improve the acoustic analysis of the object of interest. Fig. 11B If the object of interest operates with a periodicity of 4, isolating the signal with a periodicity of 4 for analysis may produce an improved analysis of the object of interest. For example, the object of interest may emit a sound with a periodicity of 4, but with an increased frequency, such as Fig. 11B This could mean that properties of the object may be changing (e.g. increased torque or load, etc.) and should be checked.

[0158] In an exemplary acoustic imaging process, background noise (e.g., noise floor in an industrial environment, wind in an outdoor environment, etc.) is typically not periodic, while certain objects of interest in the scene emit periodic acoustic signals (e.g., machinery operating at regular intervals). Therefore, a user may choose to exclude non-periodic acoustic signals from the acoustic image in order to remove the background signal and more clearly present the acoustic data of interest. In other examples, the user may be looking for a constant tone source, and therefore the user may choose to exclude periodic signals from the acoustic image data that may obscure the view of the constant tone. In general, a user may choose to include acoustic signals above a certain periodicity, below a certain periodicity, or within a desired range of periodicities in the acoustic image data. In various examples, periodicity can be identified by the length of time between periodic signals or the frequency of occurrence of periodic signals. Similar to Fig. 11B , analysis of intensity at a given periodicity (e.g., because the object of interest operates with that periodicity) can similarly be used to track how the acoustic signal from the object changes over time. In general, in some embodiments, periodicity can be used to perform rate of change analysis on a variety of parameters such as frequency, intensity, etc.

[0159] As described elsewhere herein, in some examples, various portions of a target scene can be associated with different distances to an acoustic imaging sensor. For example, in some embodiments, the distance information can include three-dimensional depth information about various portions in the scene. Additionally or alternatively, a user may be able to measure (e.g., using a laser distance tool) or manually enter distance values ​​associated with multiple locations in the scene. In some examples, such different distance values ​​for various portions of the scene can be used to adjust the back propagation calculations at such locations to accommodate a specific distance value at that location.

[0160] Additionally or alternatively, if different portions of the scene are associated with different distance values, then proximity (e.g., measured proximity and / or apparent proximity) to the acoustic sensor array can be another distinguishable parameter between such portions. Fig. 10B , positions 1020, 1050, 1070, and 1090 are each associated with a different distance value. In some examples, similar to the frequency or periodicity discussed elsewhere herein, a user can select a specific distance range according to which acoustic image data is included on the display. For example, a user can choose to display only acoustic image data representing acoustic signals that are closer than a predetermined distance, farther than a predetermined distance, or within a predetermined distance range.

[0161] In addition, in some embodiments, similar to Fig. 9CAs described by the frequencies in the image, the acoustic analysis system can be configured to cycle through multiple distance ranges to only show acoustic image data representing acoustic signals emitted from locations in the target scene that satisfy the current distance range. Such cycling through various displays can help a user visually distinguish information between different acoustic signals. For example, in some cases, an object may appear close together in line of sight from an associated electromagnetic imaging tool (e.g., a visible light camera module), and thus acoustic image data combined with electromagnetic image data of such an object may be indistinguishable. However, if the objects are separated by depth differences, cycling through different depth ranges of acoustic image data can be used to isolate each acoustic data source from each other.

[0162] In general, the acoustic analysis system can be configured to apply various settings to include and / or exclude acoustic image data representing acoustic signals that satisfy one or more predefined parameters. In some examples, the acoustic analysis system can be used to select a plurality of conditions that an acoustic signal must satisfy in order to display acoustic image data representing such a signal, for example, in a display image.

[0163] For example, about Fig. 10A and 10B , only in Fig. 10B It is shown in Fig. 10A However, additional or alternative limitations are possible. For example, in some embodiments, the user may additionally filter the acoustic image data so that the acoustic image data is only shown for acoustic signals having frequency content within a predetermined frequency range and / or having a predetermined periodicity. In an exemplary embodiment, the acoustic image data may be eliminated from additional locations such as 1020 and 1090, limited to the predetermined frequencies and / or periodicities of interest.

[0164] In general, a user may apply any number of acoustic data requirements to include or exclude acoustic image data from a displayed image, including parameters such as intensity, frequency, periodicity, apparent proximity, measured proximity, acoustic pressure, particle velocity, particle displacement, acoustic power, acoustic energy, acoustic energy density, acoustic exposure, pitch, amplitude, brilliance, harmonics, rate of change of any such parameter, etc. Additionally, in some embodiments, a user may combine requirements using any appropriate logical combination such as AND, OR, XOR, etc. For example, a user may wish to display only acoustic signals having (intensity above a predetermined threshold) AND (frequency within a predetermined range).

[0165] Additionally or alternatively, the acoustic analysis system can be configured to cycle through one or more parameter ranges to illustrate different portions of the target scene, such as with respect to Fig. 9C , which is shown in FIG. 1 , in which the acoustic image data is cycled through multiple frequencies. In general, one or more parameters can be cycled through in this manner. For example, a parameter (e.g., intensity) can be divided into multiple ranges (e.g., 10dB-20dB and 20dB-30dB), and the acoustic analysis system can cycle through such ranges, displaying all acoustic image data that falls within a first range, then all acoustic image data that falls within a second range, and so on.

[0166] Similarly, in some embodiments, the acoustic analysis system can be configured to combine parameter requirements by looping through nested ranges. For example, in an exemplary embodiment, acoustic image data that satisfies a first intensity range AND a first frequency range can be displayed. The displayed frequency range can be looped through while limiting the displayed acoustic image data to acoustic signals that satisfy the first intensity range. After looping through the frequency range, the intensity range can be updated to a second intensity range so that the displayed acoustic image data satisfies the second intensity range and the first frequency range. Similar to the process of incorporating the first intensity range, the frequency range can be similarly looped through while maintaining the second intensity range. This process can continue until all combinations of frequency ranges and intensity ranges have been met. A similar such process can be implemented for any of a plurality of parameters.

[0167] Additionally or alternatively, in some embodiments, the acoustic analysis system can be configured to identify and distinguish multiple sounds in an acoustic scene. Fig. 9B , the acoustic analysis system can be configured to identify four discrete sounds at locations 910, 920, 930, and 940. The system can be configured to cycle through multiple displays, each display showing acoustic image data at a single discrete location, similar to Fig. 9C As shown, although not necessarily dependent on any parameter value. Similarly, such cycling between acoustic image data at various locations may be performed after one or more parameter requirements limit the acoustic image data displayed.

[0168] For example, about Fig. 10A and Fig. 10B Before applying the intensity threshold, the acoustic analysis system can cycle through a plurality of acoustic image scenes (e.g., as displayed images including acoustic image scenes having visible light image data), wherein each scene includes acoustic image data at a single location. Fig. 10A, with a cycle of 10 separate images, each image comprising image data at a different position in positions 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090. However, according to some embodiments, after applying an intensity filter so that only positions with intensities greater than a threshold are displayed (e.g., as Fig. 10B ), the acoustic analysis system can update the loop process to only loop through the images corresponding to the positions that meet the filtering threshold. Fig. 10B , the loop process can be updated to loop between only four images, each image showing discrete acoustic image data at positions 1020, 1050, 1070 and 1090, respectively.

[0169] Thus, in various embodiments, each location in the target scene that includes acoustic image data is shown in one of a plurality of displayed images that are cycled through, before or after one or more filters are applied to limit which acoustic image data are shown. This cyclic display of the various sound source locations can help a user viewing the image to identify the source of a particular sound. In some embodiments, each image in the cycle includes only a single source of acoustic data, and in some such embodiments, further includes one or more parameters of the acoustic data, such as frequency content, intensity, periodicity, apparent proximity, and the like.

[0170] In addition to or as an alternative to cycling between images showing acoustic image data that satisfy certain conditions, in some examples, the location of the acoustic signal source may be detected in the acoustic image data and the location of the acoustic signal source may be displayed in the acoustic image data in isolation from other acoustic signals. Fig. 10A , in some embodiments, acoustic image data representing acoustic signals emanating from each of the locations 1010-990 may be identified and cycled through. For example, during an exemplary operation, a display image including acoustic image data at one of the locations 1010-990 may be cycled through automatically or under the direction of a user for analyzing each acoustic signal source individually. In various embodiments, the order in which different locations of acoustic image data are displayed while cycling may depend on a variety of parameters, such as by location, proximity, intensity, frequency content, and the like.

[0171] Additionally or alternatively, in some examples, acoustic image data from various locations may be looped through after applying one or more filters to isolate only acoustic image data that meets one or more predetermined conditions. Fig. 10B, positions 1020, 1050, 1070, and 1090 are shown as including acoustic image data representing acoustic signals that meet predetermined intensity requirements. In some embodiments, such display requirements can be applied to each loop through the source position of the acoustic signal. For example, further reference Fig. 10B , a display image including image data from only one of the locations 1020, 1050, 1070, and 1090 that satisfies the acoustic intensity condition may be cycled through for individual analyses at each location.

[0172] In reference Fig. 10A and Fig. 10B In an exemplary process, acoustic image data collected from a scene may be typically obtained in Fig. 10A 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, and 1090. Such locations may include acoustic image data representing acoustic signals having various acoustic parameters, such as various intensities, frequency contents, periodicities, and the like.

[0173] As described elsewhere herein, a user may wish to isolate acoustic signals having one or more specific acoustic parameters, such as an acoustic signal having a minimum acoustic intensity. Acoustic image data representing acoustic signals that do not meet such conditions may be excluded from the image, e.g., Fig. 10B As shown, the acoustic image data at positions 1020, 1050, 1070, and 1090 are left. However, the user may wish to further identify the source of a specific sound that meets the display condition (e.g., has an intensity above a threshold). Therefore, the user can choose to display the acoustic image data associated with positions 1020, 1050, 1070, and 1090 one by one in order to view the source location of each sound and analyze each sound separately. In various embodiments, the user can choose to manually cycle through such positions, or the processor can automatically update the display image to sequentially display the acoustic image data of each position. This can help the user further eliminate and ignore acoustic signals that are not of interest but happen to meet one or more filter parameters applied to the image.

[0174] Despite the strength and Fig. 10A and 10BIt is described that, in general, a display image including acoustic image data from a single location selected from a plurality of locations can be cycled through one after another for separate analysis. The plurality of locations including representative acoustic image data can be an entire set of locations corresponding to acoustic signal sources in an acoustic scene, or can be a subset of such locations, for example, including only locations having acoustic signals that satisfy one or more conditions. Such conditions can depend on any one or more acoustic parameters, such as intensity, frequency content, periodicity, proximity, etc., and can be satisfied based on various parameters being below a predetermined value, above a predetermined value, or within a predetermined range of values.

[0175] In various examples, modifying the display image to selectively include acoustic image data in the display image can be accomplished in a variety of ways. In some embodiments, the display image (e.g., including electromagnetic image data and acoustic image data) can be a real-time image, where the electromagnetic image data and acoustic image data are continuously updated to reflect changes in the scene. In some examples, when certain conditions are used to determine whether to include acoustic image data in the display image, the received acoustic signal is analyzed to determine whether to include acoustic image data at various locations in the updated real-time image. That is, as a new display image is generated based on the most recently received acoustic signals and electromagnetic radiation, the construction of the display image can depend on the analysis of the acoustic signals to determine which acoustic signals meet any specific conditions placed on the display image (e.g., intensity thresholds, etc.). A display image including acoustic image data can then be generated based on such conditions and only in appropriate circumstances.

[0176] In other examples, the display image can be generated from data stored in memory, such as previously captured acoustic data and electromagnetic image data. In some such examples, the previously acquired acoustic data is analyzed with respect to various conditions to be placed on the acoustic image data, and is combined with the electromagnetic image data in locations where the previously captured acoustic data satisfies such conditions. In such embodiments, a single scene can be viewed in many ways, for example, by analyzing different acoustic parameters. The display image representing the previously captured acoustic image data can be updated based on any updated conditions placed on the display image for whether to include acoustic image data in various locations in the display image.

[0177] In some embodiments, one or more acoustic parameters used to selectively include acoustic image data in a displayed image can be used to modify the displayed image and / or image capture technology. For example, in a real-time imaging example, various conditions for determining whether to include acoustic image data in a display can include distance to a target (e.g., apparent distance or measured distance) and / or frequency content. As described elsewhere herein, some such parameters can be used to select an acoustic sensor array and / or a processing scheme for generating acoustic image data. Therefore, in some such examples, when acoustic image data is represented based only on such parameters that satisfy one or more predetermined conditions, an acoustic sensor array and / or a processing scheme can be selected based on such conditions for generating acoustic image data.

[0178] For example, in an exemplary embodiment, if acoustic image data is only included in real-time images at locations where the corresponding acoustic signals include frequency content within a first frequency range, one or more acoustic sensor arrays may be selected for acquiring acoustic signals that are most suitable for the first frequency range. Similarly, if acoustic image data is only included in real-time images at locations where the acoustic signal sources are within a first distance range, one or more acoustic sensor arrays may be selected for acquiring acoustic signals that are most suitable for acoustic imaging within the first distance range. Additionally or alternatively, as for example with reference to Figure 6 As described, a processing scheme for generating acoustic image data may be selected based on desired frequency or distance conditions. Such (one or more) selected acoustic imaging sensor arrays and processing schemes may then be used to receive acoustic signals and generate acoustic image data for an updated real-time display image to optimize the included acoustic image data.

[0179] Similarly, in some embodiments where a display image is generated from historical data previously stored in memory, various conditions that determine which locations in the display image include acoustic image data can be used to update the acoustic image data representing the acoustic scene. For example, in some embodiments, the data stored in the memory includes raw acoustic data received by (one or more) acoustic sensor arrays from the time the acoustic signal was received. Based on the conditions for determining whether to include acoustic image data at various locations in the display image (e.g., a desired distance and / or frequency range), a processing scheme (e.g., a back propagation algorithm) can be selected for use with the raw data stored in the memory to generate acoustic image data to be displayed that is optimized for desired parameters.

[0180] It will be appreciated that although typically described and illustrated using visible light image data and acoustic image data, reference Fig.9A-C, 10A, and 10B may be used with any of a variety of electromagnetic image data. For example, in various embodiments, similar processes may be performed using infrared image data or ultraviolet image data instead of visible light image data. Additionally or alternatively, combinations of the electromagnetic spectrum may be used in such processes, such as mixing infrared image data and visible light image data. In general, in various examples, acoustic image data (e.g., acoustic image data included when the corresponding acoustic signal satisfies one or more predetermined parameters) may be selectively shown in conjunction with any combination of electromagnetic image data.

[0181] In some embodiments, the acoustic analysis system is configured to store one or more acoustic signals and / or acoustic image data in a database (e.g., in local memory) and / or accessible from an external or remote device. Such acoustic signals may include: acoustic image data representing an acoustic scene during normal operation, and / or other parameters associated with the acoustic scene, such as frequency data, intensity data, periodicity data, etc. In various examples, the database scene may include acoustic image data and / or other acoustic parameters (e.g., intensity, frequency, periodicity, etc.) representing a broad scene (e.g., a factory) and / or a more specific scene (e.g., a specific object).

[0182] In some embodiments, a database scenario can be generic to a particular type of device (such as a particular device model). Additionally or alternatively, a database scenario can be specific to a single object, even if different such objects are different instances of the same object (e.g., two separate machines of the same model). Similarly, a database scenario can be more specific, for example, including a particular operating state of an object. For example, if a particular object has multiple operating modes, a database can include multiple scenarios with such an object, one scenario for each operating mode.

[0183] In various embodiments, the database scene can be a single acoustic image and / or associated acoustic parameters. In other examples, the database scene can include synthetic data formed by multiple previously captured acoustic images and / or associated parameters. In general, the database scene (e.g., acoustic image and / or parameter) can include an acoustic representation of the scene during normal operation. In some examples, the database may include other elements associated with the scene, such as corresponding visible light images, infrared images, ultraviolet images, or a combination thereof. In some embodiments, the database generation and / or comparison can be implemented similar to the database generation and comparison of infrared image data described in U.S. Patent Application No. 15 / 190,792, filed on June 23, 2016 and entitled "THERMAL ANOMALY DETECTION", which is assigned to the assignee of the present application and is incorporated herein by reference as a whole. In some embodiments, the database can be generated by capturing the acoustic image data of the scene and / or one or more associated acoustic parameters (e.g., frequency, intensity, periodicity, etc.) while the object in the scene is operating correctly. In some such examples, a user may tag a captured database image to associate the image with one or more objects, locations, scenes, and the like, such that the captured acoustic image and / or associated parameter(s) may be identified in the future for database analysis and comparison.

[0184] The most recently generated acoustic image data can be compared with the acoustic image data stored in the database to determine whether the acoustic profile of the acoustic scene is within typical operating standards. Additionally or alternatively, acoustic parameters (such as intensity, frequency, periodicity, etc.) from the live acoustic scene and / or the most recently generated acoustic image can be compared with similar parameters in the database.

[0185] Comparing current acoustic image data with historical acoustic image data stored in a database (eg, a previously captured image, a composite image generated from multiple previously captured images, an expected image provided by a factory, etc.) can be performed in a variety of ways. Figures 12A-12C Various exemplary approaches are shown for comparing acoustic image data with historical acoustic image data stored in a database. Fig. 12AAn acoustic imaging tool 1200 is shown that includes an acoustic sensor array 1202 having an acoustic field of view 1212 and an electromagnetic imaging tool 1204 having an electromagnetic field of view 1214. As shown, the electromagnetic field of view 1214 and the acoustic field of view 1212 include a target scene 1220 that includes an object of interest 1222. In some embodiments, the acoustic imaging tool 1200 is permanently fixed in a location such that the object of interest 1222 is in the electromagnetic field of view 1214 and the acoustic field of view 1212. In some embodiments, the acoustic imaging tool 1200 can be powered via inductive or parasitic power, can be wired into an AC mains power source in a building, or can be configured to continuously monitor the object 1222.

[0186] The fixed acoustic imaging tool 1200 can be configured to periodically capture acoustic and / or electromagnetic image data of the object 1222. Because the acoustic imaging tool 1200 is typically fixed in place, images captured at different times will be approximately from the same vantage point. In some examples, the acoustic image data captured via the acoustic imaging tool 1200 can be compared with a database of acoustic image data representing approximately the same scene, for example, to detect anomalies or abnormalities in the acoustic scene. For example, this can be implemented as described in U.S. Patent Application No. 15 / 190,792, which is incorporated by reference.

[0187] Fig. 12B An exemplary display, such as on a handheld acoustic imaging tool, is shown. Display 1230 includes two portions, 1232 and 1234. In the illustrated example, portion 1234 shows a database image 1244 of an object of interest, while portion 1232 includes a live display 1242 of real-time acoustic image data of the object. In such a side-by-side view, a user can compare the live image 1242 with the database image 1244 to view any differences between a typical acoustic signal (e.g., as shown in the database image 1244) and the current real-time image 1242. Similarly, a user can compare whether the live image 1242 approximately matches the database image 1244. If so, the user can capture the live acoustic image for further analysis and / or comparison with the database image 1244.

[0188] Fig. 12C Another exemplary display, such as on a handheld acoustic imaging tool, is shown. Fig. 12C Display 1250 shows a database image 1254 and a live image 1256 on the same display 1252. Fig. 12CIn the example of , the user can similarly compare the acoustic image data in the live image 1256 with the acoustic image data in the database image 1254 to view the differences. Additionally, the user can adjust the alignment of the acoustic imaging tool to align objects in the live image 1256 with objects in the database image 1254 for additional analysis and comparison.

[0189] As Figures 12A-12C The result of the process in can be compared with the acoustic image of live and / or recent capture and previous acoustic image data such as from a database. In some examples, such a process can be used to align the acoustic image of live and / or recent capture with the database image for automated comparison. Other processes that can be used to “recapture” acoustic image data from a similar vantage point as a database image are described in U.S. patent application Ser. No. 13 / 331,633, filed on Dec. 20, 2011, and entitled “THERMAL IMAGING CAMERA FOR INFRARED REPHOTOGRAPHY,” U.S. patent application Ser. No. 13 / 331,644, filed on Dec. 20, 2011, and entitled “THERMAL IMAGING CAMERA FOR INFRARED REPHOTOGRAPHY,” and U.S. patent application Ser. No. 13 / 336,607, filed on Dec. 23, 2011, and entitled “THERMAL IMAGING CAMERA FOR INFRARED REPHOTOGRAPHY,” each of which is assigned to the assignee of the present application and is incorporated by reference in its entirety.

[0190] Comparing real-time acoustic image data and / or acoustic features with corresponding acoustic images and / or acoustic features of comparable scenes / objects can be used to provide a quick and simplified analysis of the operational state of the scene / object. For example, the comparison can indicate that certain locations within the acoustic scene are emitting acoustic signals with different intensities or spectra than during typical operation, which can indicate a problem. Similarly, locations in the scene may be emitting acoustic signals that are normally silent. Additionally or alternatively, a comparison of the overall acoustic features of live scenes and historical scenes from a database can generally indicate changes in acoustic parameters in the scene, such as frequency content, sound intensity, and the like.

[0191] In some examples, the acoustic analysis system is configured to compare a recent / real-time acoustic scene with a database. In some embodiments, the acoustic analysis system is configured to characterize the differences between the recent / real-time scene and the database scene, and to diagnose one or more possible problems in the current scene based on the comparison. For example, in some embodiments, a user can pre-select an object of interest or a target scene to compare with the acoustic database. The acoustic analysis system can analyze the scene by comparing the database image and / or other parameters with the recent / current image and / or other parameters based on the selected object / scene. Based on the object / scene selected from the database, the acoustic analysis system may be able to identify one or more differences between the database image / parameters and the recent / current image / parameters, and associate the identified difference(s) with one or more possible causes of the difference(s).

[0192] In some examples, the sound analysis system can be pre-programmed with multiple diagnostic information, for example, various differences between database images / parameters and recent / current images / parameters are associated with possible causes and / or solutions to the causes. Additionally or alternatively, the user can load such diagnostic information, for example, from a repository of diagnostic data. Such data can be provided, for example, by the manufacturer of the acoustic analysis system, the manufacturer of the object of interest, and the like. In still other examples, the acoustic analysis system can additionally or alternatively learn diagnostic information, for example, via one or more machine learning processes. In some such examples, the user can diagnose one or more problems in the target scene after observing the acoustic deviation of the scene from the typical scene, and data representing the one or more problems and / or one or more solutions can be input into the acoustic analysis system. The system can be configured to learn over time and via multiple data entries to associate different differences between recent / current images and / or parameters and those stored in the database with certain problems and / or solutions. When diagnosing a problem and / or determining a proposed solution, the acoustic analysis system can be configured to output suspected problems and / or proposed solutions to the user, for example, via a display. Such a display may be on a handheld acoustic inspection tool or a remote device (e.g., a user's smartphone, tablet device, computer, etc.). Additionally or alternatively, such a display indicating potential problems and / or solutions may be transmitted to a remote site, such as an off-site operator / system monitor, for example, via a network.

[0193] In some example diagnostic characterizations, an acoustic analysis system may observe a specific periodic squeaking sound that indicates a need for additional lubrication on an operating machine. Similarly, a constant high pitched signal may indicate a gas or air leak in the target scene. Other problems may similarly have identifiable acoustic signatures, such as bearing damage within the object being analyzed, such that viewing the acoustic signature via an acoustic imaging system (e.g., a handheld acoustic imaging tool) may help diagnose any anomalies in the system or object.

[0194] An acoustic analysis system that is capable of comparing received acoustic signals to a baseline (e.g., acoustic image data and / or parameters from a database) and implementing diagnostic information and / or recommending corrective action can eliminate the need for experienced experts to analyze the acoustic data of a scene. Instead, the acoustic inspection and analysis can be performed by a system operator with limited or no experience in analyzing acoustic data.

[0195] Fig.13 1380 ; and retrieving a baseline acoustic image and / or acoustic parameters for the object of interest from the database (1382). For example, a user may wish to perform an acoustic analysis on a particular object of interest and may select such an object from a predefined list of objects having available baseline acoustic images and / or parameters available in the database.

[0196] The method further includes the steps of capturing acoustic image data and associated parameters representing the target of interest, for example using a handheld acoustic imaging tool (1384). After capturing the acoustic image data and associated parameters (1384), the method includes comparing the captured acoustic image data and / or associated parameters to the retrieved baseline image and / or parameters (1386).

[0197] Fig.13 The method further includes: if the captured acoustic image data and / or parameters deviate sufficiently from the baseline (1388), diagnosing an operational problem with the target of interest based on the comparison (1390). The method may further include the step of: displaying an indication of possible problems and / or corrective actions to a user (1392). In some embodiments, a comparison display may additionally or alternatively be displayed to the user, for example, a difference image showing the difference between the current acoustic image data and the baseline acoustic image data.

[0198] In some such examples, determining whether there is a deviation from the baseline (1388) includes comparing one or more acoustic parameters of the captured data to similar parameters in the baseline data, and determining whether the difference between the captured and baseline parameters exceeds a predetermined threshold. In various examples, different parameters can include different thresholds, and such thresholds can be absolute thresholds, statistical thresholds, etc. In some embodiments, the comparison can be made on a location-by-location basis, and the comparison can be made for a subset of locations within a scene.

[0199] For example, refer to Fig. 9B , it is possible to analyze only the locations (e.g., locations 910 and 940) that include acoustic image data and appear on the object with respect to the operation of the object. In such an example, different acoustic parameters at each location (e.g., 910 and 940) to be compared are compared separately between the captured image and the database image. For example, referring to Fig. 9B , comparing the captured data and / or associated parameters with those from the database may include comparing the frequency, intensity, and periodicity of location 910 in the captured image with the frequency, intensity, and periodicity of location 910 in the database image, respectively. A similar comparison may be performed between the captured image and the database image at location 940. As described, each comparison may include a different metric for determining whether there is a sufficient deviation from the baseline (1388).

[0200] Diagnosing an operational problem (1390) and displaying an indication of a possible problem and / or corrective action (1392) may be performed based on a combination of comparisons between the captured image data and baseline image data and / or parameters. In some examples, such a diagnosis may include a multi-dimensional analysis, such as a combined comparison of multiple parameters at a given location. For example, in an exemplary embodiment, a condition may be indicated by both a frequency deviation from a baseline greater than a first threshold and an intensity deviation from a baseline greater than a second threshold.

[0201] In some examples, even after the indication of possible problems and / or corrective actions is displayed (1392), the process can include capturing new acoustic image data and associated parameters (1384) and repeating the comparison and diagnosis process. Thus, the user can observe whether any corrective actions taken effectively change the acoustic characteristics of the object so as to correct the identified problems and / or bring the acoustic characteristics of the object into line with the baseline.

[0202] In some embodiments, if after comparing the captured data to the baseline data (1386), there is not sufficient deviation from the baseline (1388), the process can end (1394) with the conclusion that the object is functioning normally based on the object's current acoustic characteristics. Additionally or alternatively, new acoustic image data and associated parameters of the object of interest can be captured (1384), and the comparison and diagnostic process can be repeated. In some examples, the continuous repetitive analysis can be performed using a fixed acoustic analysis system, such as a fixed acoustic analysis system including Fig. 12A Acoustic imaging tool 1200 in.

[0203] Comparison of acoustic data (e.g., image data and / or other acoustic parameters) can help a user more easily identify whether an object is functioning properly, and if not, diagnose a problem with the object. In some examples, comparison to a baseline can help a user ignore "normal" sounds in a scene, such as expected operating sounds or noise floor / background sounds, which may not be related to an operating problem with the object.

[0204] During operation, observation of acoustic image data and / or associated acoustic parameters, or observation of comparisons between the current acoustic scene and a database acoustic scene, can indicate locations of interest to the user for further inspection. For example, a comparative acoustic image showing deviations from a database image can indicate one or more locations in the scene that are operating abnormally. Similarly, viewing an acoustic image that has acoustic features at one or more unexpected locations can indicate locations of interest to the user. For example, referring to Fig. 10B , the user observes on the display of the acoustic imaging system Fig. 10B , which may recognize that a particular location (e.g., 1020) is unexpectedly emitting an acoustic signal, or similarly, comparison with a baseline image indicates an unexpected parameter (e.g., unexpected frequency, intensity, etc.) of the acoustic signal at that location.

[0205] In some such examples, a user may move closer to such a location to more closely examine whether there is an anomaly at that location. When moving closer to the object, the value of the distance to the target may be updated to reflect the new distance between the acoustic array and the target location. The acoustic sensor array and / or the back propagation algorithm may be updated based on the updated distance to the target. Additionally or alternatively, the updated acoustic analysis from a closer location may produce a different analysis of the acoustic signal from the target. For example, high-frequency acoustic signals (e.g., ultrasonic signals) tend to attenuate within a relatively short distance to the source of the acoustic signal. Therefore, when the user moves closer to the target for further inspection, the acoustic sensor array may see additional signals (e.g., high-frequency signals). Such significant changes in the observable scene may also result in adjusting the acoustic sensor array and / or adjusting the back propagation algorithm used for acoustic imaging.

[0206] Thus, as the user moves closer to an object or area of ​​interest, the sensor array and / or back propagation algorithm used to perform acoustic imaging may be updated one or more times. Each update may provide additional details about the object or area of ​​interest that may not be observed from a greater distance using a different sensor array and / or back propagation algorithm. For example, moving closer to an object or area of ​​interest may also increase the acoustic intensity of the acoustic signal of interest relative to the background sounds in the environment based on an initial observation of a broader scene.

[0207] In some embodiments, an acoustic analysis system (e.g., a handheld acoustic imaging tool) can prompt a user to move closer to an object or area of ​​interest within a scene. For example, when comparing a current acoustic image to a baseline database image, the acoustic analysis system can identify one or more locations in the scene that deviate from the baseline. The acoustic analysis system can, for example, highlight such one or more locations to the user via a display, and suggest that the user move closer to the identified (one or more) locations for further analysis. In some examples, the acoustic analysis system can classify the identified locations (such as objects within an environment or subcomponents of a particular object) as having their own baseline profile stored in a database. The system can be configured to suggest and / or implement such profiles of the classified locations to facilitate further analysis of the identified locations when the user moves closer for additional inspection.

[0208] The systems and processes described herein can be used to improve the speed, efficiency, accuracy, and thoroughness of acoustic inspections. Various automated actions and / or suggestions (e.g., sensor arrays, back-propagation algorithms, etc.) can improve the convenience of inspections to the extent that an inexperienced user can perform a thorough acoustic inspection of an acoustic scene. In addition, such processes can be used to analyze a wide range of scenarios, such as entire systems, individual objects, and sub-components of individual objects. Predefined and / or user-generated profiles of baseline acoustic data for an acoustic scene can even help inexperienced users identify anomalies in the captured acoustic data.

[0209] Registration of acoustic image data with other data streams (such as visible light, infrared and / or ultraviolet image data) can provide additional context and details for which objects are emitting the acoustic signals represented in the acoustic image data. Combining an acoustic sensor array and a distance measurement tool (e.g., a laser rangefinder) can help a user quickly and easily determine an appropriate distance to a target value for use during an acoustic imaging process. In various examples, an acoustic sensor array, a distance measurement tool, a processor, a memory, and one or more additional imaging tools (e.g., a visible light camera module, an infrared camera module, etc.) can be supported by a single housing in a handheld acoustic imaging tool that can provide effective acoustic analysis of multiple scenes. Such a handheld acoustic imaging tool can be moved from one scene to another for rapid analysis of multiple objects of interest. Similarly, using a handheld tool, a user can move closer to a location of interest within a scene for further inspection or analysis.

[0210] Various acoustic imaging systems can be configured to create a pseudo-color, symbolic, or other non-digital visual representation of an acoustic signal (ultra-sonic, acoustic, and / or ultrasonic) detected by a sensor in one or more sensor arrays. Some such systems can provide audio feedback of the received acoustic signal to the user via a speaker, earphones, headphones (wired or remotely transmitted), etc. In some such instances, the audio feedback includes: a reproduction of the received audio signal or a heterodyne signal based on the received acoustic signal. The transmission of the actual audio or heterodyne audio is synchronized with the visual representation of the detected and displayed sound.

[0211] The exemplary acoustic imaging system may employ any number of different methods to display, locate, describe and analyze detected sounds. Visualization methods may include various types of colored shapes, icons, with various levels of transparency adjusted to fit the visible background with which they are displayed.

[0212] As described elsewhere herein, in some embodiments, an acoustic imaging system may include a plurality of acoustic sensor elements arranged in an array. In some cases, different array shapes and sizes, and different acoustic sensor elements may more effectively sense different types of acoustic signals, for example, based on frequency content, distance to a target, etc. Thus, in some applications, an acoustic imager that is sensitive to specific frequencies is desired, and / or using a specific sensor configuration will produce better results.

[0213] In some embodiments, the acoustic imaging system can include multiple acoustic sensor heads, each having multiple acoustic sensing elements that can be interchangeably integrated into the system. The acoustic sensor heads can be configured to communicate wirelessly (e.g., via a wireless radio module) and / or via wired communication with the acoustic imaging tool. In some embodiments, the acoustic imaging tool can be designed and configured to accept different sensor heads, e.g., for different applications and uses. This can provide greater flexibility for the user of the tool, allowing for better data acquisition and analysis capabilities, and reducing the time and expense that would occur if a completely separate tool were used.

[0214] In some examples, one or more sensor heads can be directly attached to a main acoustic imaging device. Additionally or alternatively, one or more sensor heads can be used independently as an accessory to the main device. In some such instances, the (one or more) sensor heads and the main device can communicate via a wired connection and / or wireless communication via one or more wireless radio elements in the main device and (one or more) sensor heads. In some embodiments, the radio device in the sensor head assembly can communicate wirelessly with a wired (e.g., USB-based) radio "dongle", which can be connected to the main acoustic imaging device.

[0215] Fig.14 An exemplary embodiment of an acoustic imaging tool 1412 and a sensor head 1422 including a plurality of acoustic sensor elements 1430 is shown. The acoustic imaging tool 1412 includes a docking port 1410 for receiving a corresponding docking mechanism 1420 of the sensor head 1422. Attaching the docking mechanism 1420 to the docking port 1410 can provide communication between the acoustic sensor elements 1430 and the acoustic imaging tool 1412, such as a processor.

[0216] Fig.14 A circular sensor head is depicted having a docking mechanism 1420 on a first side of the sensor head and an acoustic sensor element 1430 on a second side opposite the first side. Although the sensor head 1422 is shown as circular, the sensor head may be any of a variety of shapes, such as rectangular, circular, triangular, or any other suitable shape.

[0217] In some examples, the sensor head 1422 and / or the acoustic imaging tool 1412 may include one or more attachment elements to fix the sensor head to the tool. Such attachment elements may include latches, locks, friction fit elements, screws, clips, hook and loop fasteners, or other suitable elements for fixing the sensor head to the acoustic imaging tool. In some examples, the docking mechanism on the acoustic sensor head is used as one of the one or more attachment elements configured to engage with the docking port of the acoustic imaging tool, which can act as a corresponding attachment element of the docking mechanism for receiving the acoustic sensor head. In various embodiments, the attachment of the acoustic sensor head to the acoustic imaging tool may include inserting an element of the acoustic sensor head (e.g., docking mechanism 1420) into a socket (e.g., docking port 1410) of the acoustic imaging tool. In other examples, the attachment may include sliding the acoustic sensor head into place from one side of the housing of the acoustic imaging device. The communication between the acoustic imaging tool and the sensor head may include a wired connection (e.g., an electrical connection) or a wireless connection (e.g., Bluetooth).

[0218] Fig.15 An exemplary sensor head 1522 and acoustic imaging tool 1512 configured for slide-in attachment are shown. As shown, the acoustic imaging tool housing can include: one or more slide rails 1517 configured to receive edges of one or more corresponding mating features 1527 of the acoustic sensor head 1522. In some embodiments, the acoustic imaging tool 1512 can also include a docking port 1510 configured to receive a corresponding docking mechanism 1520 on the acoustic sensor head when the acoustic sensor head is slidably received via the slide rail(s) of the acoustic imaging tool housing.

[0219] Fig.15 A square sensor head is depicted having a docking mechanism 1520 and mating features 1527 on a first side, and an acoustic sensor element 1530 on a second side opposite the first side. Although the sensor head 1522 is shown as a rectangle, the sensor head may be any of a variety of shapes, such as rectangular, circular, triangular, or any other suitable shape.

[0220] In some examples, the attachable acoustic sensor head can be expandable, for example, to increase the size and / or shape of the acoustic sensor element array, and / or to incorporate additional array configurations. Changing the size and / or shape of the acoustic sensor element array can be beneficial, such as when changing location, environment, or looking for sounds with different acoustic parameters (e.g., frequency content, periodicity, amplitude). Fig.16An exemplary acoustic sensor head is shown having a central sensor portion 1622 and a plurality of folded auxiliary sensor portions 1632a-d connected to the central sensor portion 1622 via attachment mechanisms 1629a-d. In some examples, the attachment mechanisms 1629a-d may include folding hinges or other flexible, foldable, or otherwise compliant members.

[0221] As shown, the central sensor portion 1622 includes a docking mechanism 1620, for example, for engaging a corresponding docking port of an acoustic imaging device. In some embodiments, attachment mechanisms 1629a-d provide for communication between the auxiliary sensor portions 1632a-d and the docking mechanism 1620. Additionally or alternatively, in some embodiments, the auxiliary sensor portions 1632a-d can communicate wirelessly with the central sensor portion 1622 and / or the acoustic imaging device.

[0222] In some embodiments, the auxiliary sensor may be attachable, such as by any of the various attachment mechanisms discussed herein. Fig.17 An exemplary acoustic sensor is shown having a central sensor portion 1722 including one or more attachment mechanisms 1729a-d that can be configured to removably receive one or more corresponding auxiliary sensor portions 1732a-d via corresponding complementary attachment mechanisms 1728a-d.

[0223] As shown, the central sensor portion 1722 includes a docking mechanism 1720, e.g., for engaging a corresponding docking port of an acoustic imaging device. In some embodiments, when (one or more) auxiliary sensor portions are attached to the central sensor portion via such an attachment mechanism, corresponding attachment mechanisms 1728a-d and 1729a-d provide communication between the auxiliary sensor portions 1732a-d and the docking mechanism 1720. Additionally or alternatively, in some embodiments, the auxiliary sensor portions 1732a-d can communicate wirelessly with the central sensor portion 1722 and / or the acoustic imaging device.

[0224] In general, in various examples, a detachable acoustic sensor head may include a main sensor portion and one or more auxiliary sensor portions that may be optionally used during sensor operation. Such auxiliary sensor portions may be removably attached to the main sensor portion, may be folded outward from the main sensor portion, and / or may extend from the main sensor portion. In some examples, when attached and / or folded into place, such a sensor is automatically able to provide acoustic information, for example, to a system processor. In other embodiments, a user may manually select to activate one or more such auxiliary sensor portions. In various examples, any number of auxiliary sensor portions may be attached to the main sensor portion and / or folded out from the main sensor portion.

[0225] In some embodiments, the attachable sensor head may include a docking mechanism for attaching to the acoustic imaging tool housing, wherein the sensor array may be movable relative to the acoustic imaging tool housing when the sensor head is attached to the acoustic imaging tool housing via the docking mechanism. Fig.18 A sensor head assembly is shown that includes a flexible "gooseneck" portion 1824 that attaches a docking mechanism 1820 to a sensor array 1822. The "gooseneck" portion 1824 can allow for increased operability of the sensor array 1822 when the sensor head assembly is attached to an acoustic imaging tool (e.g., engaging a docking port of the acoustic imaging tool via the docking mechanism 1820).

[0226] In some cases, it may be preferable to increase the distance between the sensor head and the user's handheld acoustic imaging tool, such as for safety in high voltage or high heat applications, or to access hard to reach places. Fig.19 A wand-type sensor head assembly is shown including a handle 1978 that supports a sensor array 1922 via an extension 1945. In some examples, the extension may be telescoping.

[0227] As described elsewhere herein, in some examples, the acoustic sensor head may be configured to communicate with an acoustic imaging tool via wired or wireless communications. Fig. 20 An example system including an acoustic imaging device and an acoustic sensor head is shown. As shown, the acoustic imaging device 2012 includes a docking port 2010 and a wireless communication interface 2014 .

[0228] The acoustic sensor head 2050 includes a handle 2078, a sensor array 2022 including a plurality of acoustic sensor elements, and an extension 2045 extending between the handle and the sensor array. A cable 2003 connects the handle 2078 to a docking mechanism 2020 configured to dock with a docking port 2010. The sensor head 2050 includes a wireless communication interface 2080 configured to wirelessly communicate with a wireless communication interface 2014 of an acoustic imaging device 2012.

[0229] During operation, the acoustic imaging device 2012 may receive acoustic data from acoustic sensor elements on the acoustic sensor array 2022, for example, via the docking mechanism 2020 and the docking port 2010, or via the wireless communication interfaces 2080 and 2014. The acoustic imaging device may operate as described elsewhere herein, for example, to generate acoustic image data.

[0230] In various examples, an acoustic imaging system includes a plurality of interchangeable sensor heads, eg, having different acoustic sensing capabilities and characteristics.

[0231] In some examples, one or more acoustic sensor heads include an electromagnetic imaging tool integrated therein. Wired or wireless communication between the sensor array of the sensor head and the acoustic imaging device can be configured to transmit acoustic data and electromagnetic image data to the imaging device. Additionally or alternatively, in some examples, the acoustic imaging device is configured to communicate with a sensor head (e.g., of a plurality of sensor heads) via wired or wireless communication, which may include the electromagnetic imaging tool necessary for the whole.

[0232] The various processes described herein may be embodied as non-transitory computer-readable media including executable instructions for causing one or more processors to perform such processes. The system may include one or more processors configured to implement such processes, for example, based on instructions stored in a memory integral to the processor or external to the processor. In some cases, various components may be distributed throughout the system. For example, the system may include multiple distributed processors, each configured to execute at least a portion of the entire process performed by the system.

[0233] Various embodiments have been described. Such examples are non-limiting and do not define or limit the scope of the invention in any way. Additionally, it will be appreciated that the various features and functions described herein can be combined into a single acoustic imaging system, for example, embodied as a handheld acoustic imaging tool or a distributed system having various separate and / or separable components.

[0234] The various functions of the components described herein may be combined. In some embodiments, the features described in this application may be combined with the features described in PCT application entitled “SYSTEMS AND METHODS FOR PROJECTING AND DISPLAYING ACOUSTIC DATA”, with attorney docket number 56581.178.2 filed on July 24, 2019, assigned to the assignee of this application, and incorporated herein by reference. In some embodiments, the features described in this application may be combined with the features described in PCT application entitled “SYSTEMS AND METHODS FORTAGGING AND LINKING ACOUSTIC IMAGES”, with attorney docket number 56581.179.2 filed on July 24, 2019, assigned to the assignee of this application, and incorporated herein by reference. In some embodiments, the features described in this application may be combined with features described in PCT application entitled “SYSTEMS AND METHODS FOR ANALYZING AND DISPLAYING ACOUSTIC DATA,” attorney docket number 56581.181.2, filed on July 24, 2019, assigned to the assignee of this application, and incorporated herein by reference. In some embodiments, the features described in this application may be combined with features described in PCT application entitled “SYSTEMS AND METHODS FOR REPRESENTING ACOUSTIC SIGNATURES FROM ATARGET SCENE,” attorney docket number 56581.182.2, filed on July 24, 2019, assigned to the assignee of this application, and incorporated herein by reference.

[0235] Various embodiments have been described. Such examples are non-limiting and do not define or limit the scope of the invention in any way.

Claims

1. A system for acoustic imaging, comprising: An acoustic imaging tool having an acoustic sensor head, the acoustic sensor head comprising: a main sensor section having a first acoustic sensor array; and a folding hinge configured to removably attach the auxiliary sensor portion to the main sensor portion, wherein the auxiliary sensor portion is selectively positionable between a first position and a second position via the folding hinge when the auxiliary sensor portion is attached to the main sensor portion; a communication interface; and a processor configured to communicate with the acoustic sensor head via the communication interface and receive first acoustic data from a first acoustic sensor array, Wherein the acoustic sensor head is configured to receive second acoustic data from the auxiliary sensor portion.

2. The system of claim 1 , wherein in a first position, the auxiliary sensor portion is folded inwardly toward the main sensor portion and covers at least a portion of the main sensor portion, and in a second position, the auxiliary sensor portion is unfolded outwardly from the main sensor portion and positioned adjacent to the main sensor portion.

3. The system of claim 1, wherein the auxiliary sensor portion has a second acoustic sensor array and is configured to transmit second acoustic data to the acoustic sensor head for transmission by the acoustic sensor head to the processor.

4. The system of claim 1 , wherein the acoustic sensor head comprises a plurality of folding hinges configured to removably attach a plurality of auxiliary sensor portions to the main sensor portion, and the acoustic sensor head is configured to receive second acoustic data from the plurality of auxiliary sensor portions. 5 . The system of claim 4 , wherein the acoustic sensor head is configured to transmit second acoustic data from the plurality of auxiliary sensor portions to the processor via the communication interface.

6. The system of claim 1, wherein the auxiliary sensor portion has a second acoustic sensor array comprised of acoustic sensor elements having a spacing therebetween that is different from that of the acoustic sensor elements in the first acoustic sensor array of the main sensor portion.

7. The system of claim 1, wherein the acoustic sensor head is a first acoustic sensor head, the system further comprising a second acoustic sensor head, wherein the first acoustic sensor head and the second acoustic sensor head are interchangeably connectable to the acoustic imaging tool and are configured to communicate with the processor via the communication interface.

8. The system of claim 1, further comprising: an electromagnetic imaging tool configured to transmit electromagnetic image data to said processor, Wherein the processor is configured to combine the electromagnetic image data with acoustic image data generated from at least first acoustic data and generate a display image including the acoustic image data and the electromagnetic image data.

9. The system of claim 8, wherein the acoustic sensor head comprises an electromagnetic imaging tool.

10. The system according to claim 3, wherein: The processor is operable to receive first acoustic data from the main sensor portion in response to acoustic sensor elements in a first acoustic sensor array receiving a plurality of first acoustic signals from a first scene, and the processor is operable to receive second acoustic data from the auxiliary sensor portion in response to acoustic sensor elements in a second acoustic sensor array receiving a plurality of second acoustic signals from a second scene.

11. A system for acoustic imaging, comprising: An acoustic imaging tool having an acoustic sensor head, the acoustic sensor head comprising: a main sensor portion having a first acoustic sensor array; and an auxiliary sensor portion having a second acoustic sensor array, wherein the auxiliary sensor portion is removably attached to the main sensor portion; communication interface; and a processor configured to communicate with the acoustic sensor head via the communication interface and receive: first acoustic data from a first acoustic sensor array; and Second acoustic data from the auxiliary sensor portion via the acoustic sensor head.

12. The system of claim 11, wherein the acoustic sensor head further comprises a folding hinge by which the auxiliary sensor portion is removably attached to the main sensor portion.

13. A system according to claim 12, wherein the auxiliary sensor part can be positioned between a first position and a second position, in which the auxiliary sensor part is folded inward and covers at least a portion of the main sensor part, and in the second position, the auxiliary sensor part is unfolded outward from the main sensor part and is positioned adjacent to the main sensor part.

14. The system of claim 11, wherein the acoustic sensor head comprises a plurality of auxiliary sensor portions removably attached to the main sensor portion, and Wherein the acoustic sensor head is configured to receive second acoustic data from the plurality of auxiliary sensor portions.

15. The system of claim 11, wherein the acoustic sensor head has a first array of acoustic sensor elements and a second array of acoustic sensor elements, wherein the first array of acoustic sensor elements is different than the second array of acoustic sensor elements.

16. The system of claim 15, wherein: A distance between adjacent acoustic sensor elements in the first acoustic sensor element array is smaller than a distance between adjacent acoustic sensor elements in the second acoustic sensor element array.

17. A method for acoustic imaging, comprising: removably attaching an auxiliary sensor portion to a main sensor portion of an acoustic sensor head, the main sensor portion having a first acoustic sensor array and the auxiliary sensor portion having a second acoustic sensor array; receiving, by a processor of the acoustic imaging tool, first acoustic data generated by a first acoustic sensor array of the primary sensor portion; as well as receiving, by a processor of the acoustic imaging tool, second acoustic data generated by a second acoustic sensor array of the auxiliary sensor portion, Wherein the auxiliary sensor portion is configured to transmit the second acoustic data to the acoustic sensor head, and the acoustic sensor head is configured to transmit the first acoustic data and the second acoustic data to a processor of the acoustic imaging tool.

18. The method according to claim 17, further comprising: generating first acoustic image data based on first acoustic data generated by a first acoustic sensor array; combining the first acoustic image data and the first electromagnetic image data from the imaging device to generate a first displayable image; generating second acoustic image data based on second acoustic data generated by a second acoustic sensor array; as well as The second acoustic image data and the second electromagnetic image data from the imaging device are combined to generate a second displayable image.

19. The method according to claim 17, further comprising: The auxiliary sensor portion is deployed from the main sensor portion of the acoustic sensor head, wherein the auxiliary sensor portion is selectively positionable in a first position and a second position, wherein the auxiliary sensor portion covers at least a portion of the main sensor portion and wherein the auxiliary sensor portion is oriented outwardly from the main sensor portion.

20. The method of claim 17, further comprising: separating the auxiliary sensor portion from the main sensor portion of the acoustic sensor head; as well as A different auxiliary sensor part having a different second acoustic sensor array is removably attached to the main sensor part.

Images