Acoustic imaging system for a portable computing device

Abstract

A portable acoustic imaging system includes a housing selectively attachable to a portable computer device and a plurality of microphones disposed in the housing. The microphones are arranged in a defined array for capturing acoustic data.

Description

[0001] Cross-references to related applications This application claims priority and interest in U.S. Provisional Application No. 63 / 624,835, filed January 25, 2024, the contents of which are incorporated herein by reference in their entirety for all purposes. Background Technology

[0002] Acoustic imaging equipment may include an acoustic sensor for detecting an acoustic signal of interest and a visual sensor for capturing a visual image of an area to be examined with the acoustic signal of interest. The acoustic imaging equipment may overlay the acoustic image onto the visual image to assist in identifying the location or source of the acoustic signal of interest within the area to be examined.

[0003] While conventional acoustic imaging systems can provide detailed acoustic images, they also have drawbacks. For example, conventional acoustic imaging systems are typically dedicated systems incompatible with typical consumer electronic devices, such as smartphones or tablets. Therefore, due to their large form factor and complex integration requirements, conventional acoustic imaging systems may be difficult to couple with consumer electronic devices. Furthermore, the portability and accessibility of conventional acoustic imaging systems may be limited by their large form factor and complex integration requirements.

[0004] Therefore, there are technical challenges in developing acoustic imaging systems that can be seamlessly integrated with consumer electronic devices while maintaining portability and ease of use without compromising the quality of the generated acoustic images. Summary of the Invention

[0005] The challenges mentioned above can be addressed using acoustic imaging systems or array accessories that include small microphone arrays and can be seamlessly integrated with standard portable computing devices such as smartphones or tablets. Array accessories can facilitate the generation of detailed acoustic images by utilizing a combination of microphone arrays with a built-in camera on the portable computing device or an integrated camera within the array accessory. This solution can advantageously provide a high-quality acoustic imaging system compatible with consumer portable computing devices, thus increasing accessibility and portability without compromising the quality of the generated acoustic images.

[0006] By providing auxiliary or accessory-type solutions for portable computing devices, acoustic imaging applications can be more readily available to the average consumer, thanks to the significantly lower cost of array accessories compared to dedicated conventional acoustic imaging systems, coupled with the smaller and more portable form factor of array accessories.

[0007] The array accessory may or may not include an integrated camera located on the array accessory. In an example embodiment where the array accessory does not include an integrated camera, the array accessory may utilize a built-in camera on a portable computer device to capture a visual image to be merged with audio data from a microphone array to generate an acoustic image. When the array accessory is equipped with an integrated camera, the array accessory may utilize image data captured from the integrated camera to merge with audio data from a microphone array to generate an acoustic image. In an example embodiment where the array accessory is equipped with an integrated camera, the array accessory may utilize image data captured from the integrated camera on the array accessory and utilize a built-in camera on a portable computer device to capture a visual image to be merged with audio data from a microphone array to generate an acoustic image. Therefore, in some example embodiments, both the integrated camera on the array accessory and the built-in camera on the portable computer device can be used to capture visual data, and visual data from two (or more) cameras can be combined with audio data from a microphone array to generate an acoustic image.

[0008] Array attachments can also vary in terms of onboard computing resources. Thus, for example, an array attachment may include a wide variety of numbers, sizes, and capabilities of processors and memory located onboard within the array attachment. In some example embodiments, the array attachment may include minimal onboard computing resources, for example, to enable audio data to be captured by the array attachment and then transmitted to a portable computing device for processing and visualization. In other example embodiments, the array attachment may include full onboard computing resources, for example, to enable acoustic images to be generated by the array attachment and then transmitted to a portable computing device for display to a user.

[0009] Array accessories can adopt various shape factors. For example, an array accessory can have a sleeve shape factor, such that a portable computer device can be attached within the sleeve shape of the array accessory. As another example, an array accessory can have a bottom-mounting shape factor, such that the array accessory can be attached to the bottom of the portable computer device. As yet another example, an array accessory can have a hinge shape factor, such that the array accessory can be pivotally attached to the portable computer device. Other shape factors can also be used depending on the desired arrangement. Therefore, array accessories can advantageously be flexible in design to meet different user needs.

[0010] In an example embodiment, the array accessory can be configured for calibration (such as parallax correction) either manually by the user interacting with an application running on a portable computing device or automatically when the array accessory has an integrated camera. Calibration can advantageously ensure the accuracy of the generated acoustic images, which can therefore provide a highly efficient solution for integrating acoustic imaging capabilities with portable computing devices.

[0011] These and other features, aspects, and advantages of this disclosure will become more readily understood with reference to the following description and the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure. Attached Figure Description

[0012] The complete and implementable disclosure of this disclosure, including its best mode, is set forth in the description with reference to the accompanying figures.

[0013] Figure 1 This is an isometric view of a portable acoustic imaging system according to an exemplary embodiment of this subject matter, attached to a portable computer device.

[0014] Figure 2 yes Figure 1 Another isometric view of the example portable acoustic imaging system, where the example portable acoustic imaging system is shown as being unloaded from a portable computer device.

[0015] Figure 3 yes Figure 1 A rear view of an example portable acoustic imaging system.

[0016] Figure 4 This is an isometric view of a portable acoustic imaging system according to another exemplary embodiment of this subject matter.

[0017] Figure 5 This is an isometric view of a portable acoustic imaging system according to another exemplary embodiment of the subject matter, attached to a portable computer device.

[0018] Figure 6 This is an isometric view of a portable acoustic imaging system according to another exemplary embodiment of the subject matter, attached to a portable computer device.

[0019] Figure 7 yes Figure 6 Another isometric view of the example portable acoustic imaging system, where the example portable acoustic imaging system is shown unloaded from a portable computer device.

[0020] Figure 8This is an isometric view of a portable acoustic imaging system according to another exemplary embodiment of the subject matter, attached to a portable computer device.

[0021] Figure 9 This is a schematic view of specific components of an acoustic imaging system according to an example embodiment of this subject matter.

[0022] Figures 10 to 12 The illustration shows various parts of a parallax correction process for a portable acoustic imaging system according to an example embodiment of this subject matter.

[0023] Figure 13 This is a flowchart of an acoustic imaging method according to an example embodiment of this subject matter.

[0024] Reference numerals are used repeatedly in this specification and drawings to indicate the same or similar features or elements of the invention. Detailed Implementation

[0025] This disclosure generally relates to a portable acoustic imaging system that can be selectively attached to a portable computing device, such as a smartphone or tablet. Thus, in one example embodiment, a portable acoustic imaging system includes: a housing selectively attachable to a portable computing device; and a plurality of microphones deployed within the housing. The microphones are arranged in a defined array for capturing acoustic data.

[0026] Selective attachability refers to the ability of a portable acoustic imaging system to easily and interchangeably connect and disconnect from various types of portable computing devices, such as smartphones or tablets, at the user's discretion. This feature provides users with the flexibility to choose when to integrate acoustic imaging capabilities with their chosen devices and the ability to remove the system when not in use. This contrasts with permanent or semi-permanent attachment to a single device or mount. Selective attachability promotes enhanced portability, convenience, and adaptability in a variety of use cases.

[0027] In some embodiments, a unique aspect of the portable acoustic imaging system lies in its ability to selectively attach to a portable computing device to form a single handheld physical structure. For example, unlike alternative embodiments that can be detached from the computing system but operatively connected (e.g., using a communication cable), some example embodiments of the portable acoustic imaging system can be attached to the portable device not only using a communication cable but also through a physical integration that creates a single combined structure that can be easily held, moved, and manipulated by the user. This selective attachment, used to create a single combined structure, not only facilitates data communication but also physically couples the acoustic imaging system to the portable device, thereby enhancing its portability, handling, and user interaction. The combined structure allows for streamlined operation, providing the user with the ability to simultaneously capture acoustic and visual data while manipulating the device in real time, further enhancing the system's usability and ergonomics. This unique feature distinguishes these example embodiments of the proposed device from alternative acoustic imaging embodiments that are plugged into a computer without providing an integrated handheld form factor.

[0028] In an example embodiment, the portable acoustic imaging system does not include an integrated camera. Therefore, the portable acoustic imaging system can utilize the camera of a portable computing device to capture visual images, which can be combined with audio data from a microphone to generate an acoustic image. The camera of the portable computing device may be offset from a defined array of microphones. Furthermore, the focus center of the camera on the portable computing device may be offset from the center of the defined array of microphones. In other example embodiments, the portable acoustic imaging system may include an integrated camera. Therefore, the portable acoustic imaging system can utilize the camera of the portable acoustic imaging system and / or the camera of the portable computing device to capture visual images, which can be combined with audio data from a microphone to generate an acoustic image.

[0029] In some example embodiments, when both the integrated camera of the portable acoustic imaging system and the camera of the portable computer device are utilized, the integrated camera on the portable acoustic imaging system can be of relatively lower resolution relative to the camera of the portable computer device. In some of these embodiments, the integrated camera on the portable acoustic imaging system may be used primarily to assist in the calibration of the microphone used for acoustic imaging, and the camera of the portable computer device may be utilized for acoustic imaging. For example, if the deviation between the camera of the portable computer device and the (lower resolution) integrated camera is known, the lower resolution image can be pixel-mapped to a higher resolution image from the portable computer device (e.g., using various known computer vision image registration techniques), and the acoustic image can then be correspondingly mapped from the lower resolution to the higher resolution visual image using the determined mapping. Alternatively or additionally, if the geometric relationship between the camera and microphone array of the portable computer device is known, the acoustic image can be directly mapped to the higher resolution camera image (e.g., via parallax registration methods described in further detail elsewhere herein).

[0030] In example embodiments, a portable acoustic imaging system may include a variety of onboard computing resources. Thus, for example, a portable acoustic imaging system may include a wide variety of processors and memories of varying numbers, sizes, and capabilities onboard. For instance, a portable acoustic imaging system may include a limited set of acoustic imaging processors and memories onboard, and audio data captured by microphones on the portable acoustic imaging system may be transmitted to a portable computer device for processing and visualization. As another example, a portable acoustic imaging system may include full acoustic imaging capabilities, and associated processors and memory may be onboard to generate acoustic images on the portable acoustic imaging system and then transmit the acoustic images to a portable computer device for display to a user.

[0031] In example embodiments, the portable acoustic imaging system can have various shape factors. For example, the portable acoustic imaging system can have a sleeve shape factor, a bottom-mounted shape factor, or an articulated shape factor. Other shape factors may also be utilized.

[0032] In example embodiments, a portable acoustic imaging system can be configured for calibration of the portable acoustic imaging system, such as parallax correction. Thus, for example, a user can interact with an application running on a portable computing device to calibrate image / audio data, such as when the portable acoustic imaging system does not have an integrated camera. In some embodiments, when the portable acoustic imaging system includes an integrated camera, the portable acoustic imaging system can perform an automated calibration process for the alignment of the camera and microphone of the portable computing device. For example, a pixel mapping between a lower-resolution image generated by the integrated camera and a higher-resolution image generated by the camera of the portable computing device can be determined. This mapping can then be applied to data generated by the microphone array to align the data with the higher-resolution camera. Alternatively or additionally, the parallax registration methods described elsewhere herein can be used to directly map the data generated by the microphone array to the higher-resolution camera.

[0033] Those skilled in the art will understand that this discussion is merely a description of exemplary embodiments and is not intended to limit the broader aspects of this disclosure.

[0034] When describing elements of this disclosure or its preferred embodiments, the quantifiers “a,” “an,” and the pronouns “the” and “the” are intended to refer to one or more of the elements present. As used herein, the terms “comprising” and “including” are intended to be inclusive in a manner similar to the term “comprising.” Similarly, the term “or” is generally intended to be inclusive (i.e., “A or B” is intended to mean “A or B or both”). Approximate language as used throughout the specification and claims is applied to modify any quantitative expression that may be permissibly varied without altering its associated essential function. Thus, values ​​modified by one or more terms such as “about,” “approximate,” and “substantially” are not limited to the specified precise values. In at least some cases, approximate language may correspond to the precision of the instrument used to measure the value. For example, approximate language may refer to a margin of ten percent (10%).

[0035] The term “attachment” refers to direct attachment unless otherwise indicated herein.

[0036] Portable acoustic imaging system: Reference Figures 1 to 3 The portable acoustic imaging system 200 according to an example embodiment of this subject matter will now be described in more detail. Figure 1 In the image, the portable acoustic imaging system 200 is shown attached to or mounted on the portable computer device 100. Figure 2 In the image, the portable acoustic imaging system 200 is shown as being removed or unloaded from the portable computer device 100. (As shown...) Figure 1 and Figure 2As shown, the portable acoustic imaging system 200 can be selectively attached to the portable computing device 100. Therefore, for example, when a user uses the portable acoustic imaging system 200 to generate an acoustic image, the user can attach the portable acoustic imaging system 200 to the portable computing device 100, and when the user is not using the portable acoustic imaging system 200 to generate an acoustic image, the user can remove the portable acoustic imaging system 200 from the portable computing device 100.

[0037] Therefore, for example, a user can selectively attach the portable acoustic imaging system 200 to the portable computing device 100, and thus, depending on the desired arrangement... Figure 1 and Figure 2 The portable computing device 100 and the portable acoustic imaging system 200 are arranged in a configuration shown. The portable acoustic imaging system 200 can define a vertical direction V, a longitudinal direction L, and a lateral direction T that are mutually perpendicular and form an orthogonal system.

[0038] Portable computing device 100 can be any suitable portable computing device. For example, such as... Figure 1 and Figure 2 As shown, portable computing device 100 can be a smartphone. In other example embodiments, portable computing device 100 can be a tablet or other portable computing device, such as a smartwatch or a single-board computer. As shown, portable computing device 100 can include a housing 110, for example, housing 110 has a top portion 112 and a bottom portion 114 ( Figure 2 Between (e.g., when portable computing device 100 such) Figure 1 and Figure 2 As shown in the diagram, when oriented vertically, it extends along the vertical direction (V). The portable computing device 100 may include multiple cameras 120, which are mounted to the housing 110, for example, at the top portion 112 of the housing 110. The multiple cameras 120 may be positioned, for example, opposite the display 111 of the portable computing device 100 in the lateral direction T. Therefore, the multiple cameras 120 may be positioned and oriented to capture visual images in a direction opposite to the display 111 of the portable computing device 100. As will be understood, a user of the portable computing device 100 can utilize the display 111 of the portable computing device 100 to orient the multiple cameras 120 of the portable computing device 100 for capturing visual images of a desired area.

[0039] The portable acoustic imaging system 200 may include a housing 210 that houses various components of the portable acoustic imaging system 200 and facilitates attachment of the portable acoustic imaging system 200 to the portable computing device 100. When the portable acoustic imaging system 200 is as... Figure 1 When attached to the portable computing device 100 as shown, the housing 210 can also protect the portable computing device 100.

[0040] Housing 210 may be located between top portion 212 and bottom portion 214 (e.g., when a portable acoustic imaging system 200 is used). Figure 1 and 2 As shown, when oriented vertically, it extends along the vertical direction (V). Figure 1 and Figure 2 As shown, in a particular example embodiment, housing 210 may have a sleeve-shaped form factor. Therefore, for example, housing 210 may include a pair of sidewalls 211, a bottom wall 213, and a front wall 215. The sidewalls 211 may be spaced apart along the longitudinal direction L and / or positioned along the longitudinal direction L on opposite sides of housing 210. The bottom wall 213 may be positioned at the bottom portion 214 of housing 210, and the front wall 215 may extend along the vertical direction V between the top portion 212 and the bottom portion 214 of housing 210. The sidewalls 211, bottom wall 213, and front wall 215 may collectively define a slot 202 configured to receive a portable computing device 100. Furthermore, the top portion 212 and / or the rear portion of housing 210 may be open to allow the portable computing device 100 to be inserted into the slot 202. Therefore, a user can slide the portable computing device 100 into the slot 202 to attach the portable acoustic imaging system 200 to the portable computing device 100. The user can also slide the portable computing device 100 from the slot 202 to unload the portable acoustic imaging system 200 from the portable computing device 100. When the portable acoustic imaging system 200 is attached to the portable computing device 100, a portion of the housing 210 of the portable acoustic imaging system 200 can contact and / or couple with a portion of the housing 110 of the portable computing device 100. As an example, some or all of the bottom wall 213 and side walls 211 may be included at the outer edge and extend into the slot 202. These edges(s) can securely hold the portable computing device 100 within the slot 202.

[0041] In an example embodiment, the height of the front wall 215 along the vertical direction V can be approximately equal to the height of the housing 110 of the portable computing device 100 along the vertical direction V, and the width of the front wall 215 along the longitudinal direction L can be approximately equal to the width of the housing 110 of the portable computing device 100 along the longitudinal direction L. Therefore, the front wall 215 can be generally complementary to the shape of the housing 210. The housing 210 may include an opening or transparent window 220 at the top portion 212 of the housing 210. The window 220 can be aligned with the cameras(s)120 of the portable computing device 100, and when the portable acoustic imaging system 200 is attached to the portable computing device 100, it allows the cameras(s)120 to capture visual images through the housing 210. The height of the side wall 211 along the vertical direction V can be less than the height of the front wall 215. When the portable acoustic imaging system 200 is attached to the portable computing device 100, the reduced height of the side wall 211 allows the user to access inputs (such as buttons, switches, etc.) on the portable computing device 100. Furthermore, as noted above, the rear portion of the housing 210, opposite the front wall 215, can be open. Therefore, when the portable acoustic imaging system 200 is attached to the portable computing device 100, the display 111 of the portable computing device 100 can be visible and accessible to the user. In an example embodiment, the housing 210 can be flexible, including resilient padding, or deformable against compression of the portable computing device 100 and assisting in coupling the portable acoustic imaging system 200 to the portable computing device 100 and / or other features that assist in coupling the portable acoustic imaging system 200 to portable computing devices of various sizes.

[0042] like Figure 1 and Figure 2As shown, the portable acoustic imaging system 200 also includes a plurality of microphones 230. The microphones 230 can be deployed within the housing 210. For example, the microphones 230 can be deployed within the front wall 215 of the housing 210. Furthermore, the housing 210 (e.g., the front wall 215) can define a plurality of openings, and the microphones 230 can be deployed within the housing 210 such that each of the microphones 230 is aligned with one or more corresponding openings to facilitate sound transmission through the housing 210 to the microphone 230. In an example embodiment, the openings can be positioned at the front surface 216 of the housing 210, for example, when the portable acoustic imaging system 200 is attached to the portable computing device 100, with the front surface 216 facing away from the portable computing device 100. The microphones 230 can be positioned and oriented to capture and record sound in a direction opposite to the display 111 of the portable computing device 100. Thus, for example, the microphones 230 can be positioned and oriented to capture audio data in a direction opposite to the display 111 of the portable computing device 100. In an example embodiment, microphone 230 may be positioned and oriented to capture audio data in substantially the same direction as the cameras(s)120 of portable computing device 100.

[0043] Microphone 230 may include a suitable number of microphones. For example, microphone 230 may include five, six, seven, eight, nine, ten, fifteen, twenty, or more microphones. Microphone 230 may be arranged in a defined array. The defined array may have a suitable shape, such as circular, elliptical, square, rectangular, or other shapes. The defined array may be generally planar, for example, such that the microphones 230 are arranged coplanarly with each other. Microphone 230 may be a suitable transducer for converting sound into electrical signals. For example, microphone 230 may be one or more of MEMS (Micro-Electro-Mechanical Systems) microphones, condenser microphones, dynamic microphones, piezoelectric microphones, fiber optic microphones, etc.

[0044] The position of each microphone 230 can be known to assist beamforming during acoustic imaging. In an example embodiment, the position data for each microphone 230 can be stored as an M × 3 matrix, where M is the number of microphones 230, and each row contains (x, y, z) triples describing the point in three-dimensional space where the m-th microphone is located. The origin can be defined relative to a reference location, such as the geometric center of a defined array.

[0045] Figure 3 This is a rear view of the portable acoustic imaging system 200. (As shown) Figure 3As shown, the portable acoustic imaging system 200 may include a data transmission and power delivery plug 240. The plug 240 may be mounted to the housing 210, for example, at the bottom portion 214 of the housing 210. In an example embodiment, the plug 240 may be positioned on the bottom wall 213 of the housing 210. The plug 240 may be configured to be received within a socket (not shown) of the portable computing device 100 when the portable acoustic imaging system 200 is attached to the portable computing device 100. The plug 240 and the socket may be configured with a suitable standard, such as Universal Serial Bus (USB), Thunderbolt, Lightning, or other connectors for electronic devices. The plug 240 may thus facilitate signal communication and / or power delivery between the portable acoustic imaging system 200 and the portable computing device 100. It will be understood that in other example embodiments, the portable computing device 100 and the portable acoustic imaging system 200 may be configured for wireless communication, such as via Bluetooth or other suitable communication protocols.

[0046] exist Figures 1 to 3 In the example embodiment shown, the portable acoustic imaging system 200 does not include a camera integrated within the portable acoustic imaging system 200. Therefore, as described in more detail below, the cameras(s)120 on the portable computer device 100 can be used to capture visual images for forming a combined image with an acoustic image having acoustic data from the microphone 230. However, as described in more detail below, in other example embodiments, the portable acoustic imaging system 200 may include an integrated camera for capturing visual images. Visual images from the integrated camera on the portable acoustic imaging system 200 can be used as a substitute for or supplement to visual images from the cameras(s)120 on the portable computer device 100.

[0047] What will be understood is... Figures 1 to 3 The arrangement of the portable acoustic imaging system 200 shown is provided by way of example. In other exemplary embodiments, the portable acoustic imaging system 200 may include other components and / or different arrangements. The following... Figures 4 to 12 Other example aspects of the portable acoustic imaging system 200 will be described in more detail in the context of [the previous sentence].

[0048] like Figure 4As shown, the portable acoustic imaging system 200 may include a camera 250 for capturing visual images. The camera 250 may be deployed within a housing 210. Therefore, the camera 250 may be integrated within the portable acoustic imaging system 200. For example, the camera 250 may be deployed within the front wall 215 of the housing 210. Furthermore, the housing 210 (e.g., the front wall 215) may define an opening, and the camera 250 may be deployed within the housing 210 such that the camera 250 is aligned with the corresponding opening to assist in the camera 250's exposure to light. In an example embodiment, the opening may be positioned at the front surface 216 of the housing 210, for example, when the portable acoustic imaging system 200 is attached to the portable computing device 100, with the front surface 216 facing away from the portable computing device 100. The camera 250 may be positioned and oriented to capture visual images in a direction opposite to the display 111 of the portable computing device 100. Therefore, for example, camera 250 can be positioned and oriented to capture visual images in an orientation opposite to the display 111 of portable computing device 100. In an example embodiment, camera 250 can be positioned and oriented to capture visual images in substantially the same orientation as the (multiple) cameras 120 and / or microphones 230 of portable computing device 100.

[0049] In an example embodiment, camera 250 may be deployed within the outer perimeter of the defined array of microphones 230. Thus, for example, camera 250 may be positioned between adjacent microphones 230 within the defined array. In particular, camera 250 may be deployed at the center (e.g., centroid) of the defined array of microphones 230. In an example embodiment, camera 250 may be spaced no more than five centimeters (5 cm) from the defined array of microphones 230. Therefore, camera 250 on the portable acoustic imaging system 200 may be positioned, for example, close to microphone 230 relative to camera(s) 120 on the portable computing device 100. Such positioning of camera 250 on the portable acoustic imaging system 200 advantageously allows camera 250 to assist in calibrating the alignment of microphone 230 with camera 250 and / or camera 120. In particular, it is advantageous for camera 250 to be located at the centroid of the array of microphones 230 because any deviation will then be independent of the distance from the target (i.e., no parallax effect). If the camera 250 is not at the center of mass, it is still advantageous to place it as close to the center of mass as possible. The closer it is, the smaller the parallax deviation will be.

[0050] As noted above, visual images from camera 250 on portable acoustic imaging system 200 can be used as a substitute for or supplement to visual images from camera(s)120 on portable computer device 100. Therefore, in the example embodiment, camera 250 on portable acoustic imaging system 200 may have a lower resolution than camera(s)120 on portable computer device 100. The lower-resolution camera 250 on portable acoustic imaging system 200 can be used to assist in calibrating the alignment of the higher-resolution camera(s)120 and microphone 230 on portable computer device 100, while the higher-resolution camera(s)120 on portable computer device 100 can be used for acoustic imaging. For example, if the deviation between camera 120 and camera 250 is known, the lower-resolution image from camera 250 can be pixel-mapped to the higher-resolution image from camera 120 (e.g., using various known computer vision image registration techniques), and then data obtained using microphone 230 can be correspondingly mapped from the lower resolution to the higher-resolution visual image using the determined mapping. Alternatively or additionally, if the geometric relationship between camera 120 and microphone 230 is known, data from microphone 230 can be directly mapped to camera 120 (e.g., via a parallax registration method described in further detail elsewhere in this document).

[0051] In other example embodiments, the camera 250 on the portable acoustic imaging system 200 may have the same or greater resolution as the cameras 120(on) on the portable computer device 100, and the camera 250 on the portable acoustic imaging system 200 may be used to assist in calibrating the alignment of the higher-resolution cameras 120(on) and microphone 230 on the portable computer device 100, as well as in acoustic imaging. Generally, the camera 250 may be used to capture visual images, and visual images from the camera 250 on the portable acoustic imaging system 200 may be used as a substitute for or supplement to visual images from the cameras 120(on) on the portable computer device 100.

[0052] Adding a camera 250 to the portable acoustic imaging system 200 allows for additional capabilities for acoustic imaging, such as higher-quality supplementary photographs from multiple cameras 120 on the portable computer device 100 that can be paired with the acoustic image via attachment. For a portable acoustic imaging system 200 without a camera 250... Figures 1 to 3For the purposes of this embodiment, the positions of the cameras(s)120 on the portable computer device 100 relative to each microphone 230 must be known in order to perform parallax correction and properly superimpose the visual and acoustic images from the cameras(s)120 on the portable computer device 100. In an example embodiment: (1) the positions of the cameras(s)120 on the portable computer device 100 relative to the microphones 230 may be stored on the portable acoustic imaging system 200 and / or the portable computer device 100, for example, via a database accessible via a network; or (2) the user may manually adjust the FOV and window of the visual image until the visual image is properly superimposed with the acoustic mapping, as discussed in more detail below. In the second case, it is not necessary to know the positions of the cameras on the computing device.

[0053] like Figure 5 As shown in the illustration, in another example embodiment, the housing 210 of the portable acoustic imaging system 200 may have a reduced height shape factor. For example, the height of the front wall 215 along the vertical direction V may be less than the height of the housing 110 of the portable computing device 100 along the vertical direction V. Furthermore, when the portable acoustic imaging system 200 is attached to the portable computing device 100, the top of the front wall 215 may be positioned along the vertical direction V below the cameras(s) 120. Thus, the front wall 215 may be spaced apart along the vertical direction V from the cameras(s) 120 of the portable computing device 100 to allow the cameras(s) 120 to capture visual images without interference from the housing 210 when the portable acoustic imaging system 200 is attached to the portable computing device 100.

[0054] The size of the housing 210 can be selected to facilitate reception or mounting onto the portable computing device 100. As an example, when the portable computing device 100 is a smartphone, the height of the front wall 215 along the vertical direction V can be no less than 5 cm and no more than 20 cm, and the width of the front wall 215 along the longitudinal direction L can be no less than 5 cm and no more than 12 cm. As an example, when the portable computing device 100 is a tablet, the height of the front wall 215 along the vertical direction V can be no less than 5 cm and no more than 50 cm, and the width of the front wall 215 along the longitudinal direction L can be no less than 5 cm and no more than 35 cm. The thickness of the front wall 215 along the transverse direction T can be no less than 2.5 mm and no more than 3 cm. It will be understood that such sizes are provided by way of example and can be different to account for the portable computing device 100.

[0055] like Figure 6 and Figure 7 As shown in the diagram, in another example embodiment, the housing 210 of the portable acoustic imaging system 200 may have a bottom-mounted shape factor. Therefore, the housing 210 of the portable acoustic imaging system 200 can be attached to the portable computing device 100 at the bottom portion 114 of the housing 110, and the housing 210 of the portable acoustic imaging system 200 can extend downward from the bottom portion 114 of the housing 110. In such an arrangement, the plug 240 can be positioned at the top portion 212 of the housing 210.

[0056] like Figure 8 As shown in the diagram, in another example embodiment, the housing 210 of the portable acoustic imaging system 200 may have an articulated shape factor. Therefore, the housing 210 of the portable acoustic imaging system 200 can be pivotally attached to the portable computing device 100 at the bottom portion 114 of the housing 110, and the housing 210 of the portable acoustic imaging system 200 can extend downward from the bottom portion 114 of the housing 110. For example, a plug 240 on a portion 260 of the housing 210 can be received within a socket (not shown) of the portable computer device 100, and other portions of the housing 210 can pivot relative to portion 260. Figure 8 The pivot direction is indicated by a curved double arrow.

[0057] It will be understood that other form factors may be utilized in other example embodiments. For example, the connection between the portable computer device 100 and the portable acoustic imaging system 200 may be flexible, such as via a gooseneck attached to the portable computer device 100, which may allow the portable acoustic imaging system 200 to move independently relative to the portable computer device 100.

[0058] Portable computer device 100 and portable acoustic imaging system 200 can jointly achieve wide-frequency acoustic imaging via an array of microphones 230 and cameras 120 and / or 250 to visualize and locate acoustic signals. Generally, data signals from microphones 230 can be processed via algorithms to achieve far-field array beamforming of the acoustic data from microphones 230 in the frequency domain. The output of beamforming can be an acoustic map describing the acoustic signal power at a series of locations in space, and the series of locations used for beamforming can cover the same field of view (FOV) as cameras 120 and / or 250, allowing the acoustic map to be superimposed on the visual image, thus allowing the user to locate sound sources. Acoustic imaging can be applied to a wide range of applications, including reducing noise sources in product design, locating compressed air leaks, wildlife location, partial discharge location, and mechanical defect location. While location is the primary use of acoustic imaging, the technology can also be used for quantitative purposes, such as measuring sound pressure level (SPL) and calculating the distance to a noise source to be measured.

[0059] As can be seen from the above, when the portable acoustic imaging system 200 is attached to the portable computer device 100 to provide acoustic imaging, the portable acoustic imaging system 200 can cooperate with the portable computer device 100. In an example embodiment, the portable acoustic imaging system 200 may utilize components of the portable computer device 100, such as the display 111, the camera(s) 120, and / or computing devices, to support the portable acoustic imaging system 200.

[0060] Acoustic imaging system components: Figure 9 This is a schematic view of specific components of the acoustic imaging system 300 according to an example embodiment. The acoustic imaging system 300 is described in more detail below in the context of the portable computing device 100 and the portable acoustic imaging system 200 described above. However, it will be understood that the acoustic imaging system 300 can be used in or alongside other portable acoustic imaging system arrangements.

[0061] like Figure 9 As shown, the acoustic imaging system 300 may also include a processing device or computing device 350, which may typically be configured to facilitate the operation of at least a portion of the acoustic imaging system 300. In this regard, as discussed in more detail below, the microphone 310, the camera(s) 320, the display 330, and / or the battery 340 may be operated and / or controlled by the computing device 350.

[0062] Computing device 350 may typically include one or more processors 352 and one or more memories 354. As used herein, the terms “processing device,” “computing device,” etc., may generally refer to any suitable processing device, such as a general-purpose or special-purpose microprocessor, microcontroller, integrated circuit, application-specific integrated circuit (ASIC), digital signal processor (DSP), field-programmable gate array (FPGA), logic device, one or more central processing units (CPU), graphics processing units (GPU), processing units performing other special-purpose computing, semiconductor devices, etc. Furthermore, these “computing devices” are not necessarily limited to a single element, but may include any suitable number, type, and configuration of processing devices integrated in any suitable manner to facilitate aircraft operation.

[0063] The computing device 350 may include or be associated with one or more memory elements or non-transitory computer-readable storage media (such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, or other suitable memory devices, including combinations thereof). These memory devices may be components separate from the processor or may be included on-board within the processor. Furthermore, these memory devices may store information and / or data accessible to one or more processors, including instructions executable by one or more processors. It should be understood that the instructions may be software written in any suitable programming language or may be implemented in hardware. Additionally or alternatively, instructions may be executed logically and / or virtually using separate threads on one or more processors.

[0064] For example, computing device 350 may be operable to execute programming instructions or microcontroller code associated with the operation of acoustic imaging system 300. In this regard, instructions may be software or any set of instructions that, when executed by a processing device, cause the processing device to perform operations (such as running one or more software applications, adjusting operating parameters of acoustic imaging system 300, etc.). Furthermore, it should be noted that computing device 350 disclosed herein is capable of and operable to perform any method, method step, or part of a method disclosed herein. For example, in some example embodiments, the methods disclosed herein may be embodied in programming instructions stored in memory and executed by computing device 350.

[0065] The computing device 350 may be onboard on the portable computing device 100 and / or the portable acoustic imaging system 200. For example, one or more of the processor 352 and memory 354 may be onboard on the portable computing device 100 (e.g., within housing 110), and one or more of the processor 352 and memory 354 may be onboard on the portable acoustic imaging system 200 (e.g., within housing 210). Thus, for example, the portable computing device 100 and / or the portable acoustic imaging system 200 may include memory elements and processors for performing operations for acoustic image generation. In example embodiments, the portable acoustic imaging system 200 may include various onboard computing resources. Thus, for example, the portable acoustic imaging system 200 may include various numbers, sizes, and capabilities of processors 352 and memory 354 onboard on the portable acoustic imaging system 200. For example, the portable acoustic imaging system 200 may include a limited onboard processor 352 and memory 354, and audio data captured by the microphone 310 on the portable acoustic imaging system 200 may be sent to the portable computer device 100 for processing and visualization. As another example, the portable acoustic imaging system 200 may include full acoustic imaging capabilities, and the processor 352 and memory 354 may be onboard on the portable acoustic imaging system 200 for generating acoustic images on the portable acoustic imaging system 200 and then sending the acoustic images to the portable computer device for presentation to a user on a display 330 (such as display 111 of the portable computer device 100). Other data processing configurations may also be implemented between the portable acoustic imaging system 200 and the portable computer device 100.

[0066] Generally, the computing device 350 may store instructions executable by the processors 352 to perform operations. Operations may include accessing acoustic data from a microphone 310 (such as microphone 230 on a portable acoustic imaging system 200). Thus, for example, the microphone 310 may transmit data corresponding to sound at the microphone 310 to the processors 352 and / or the memory 354. The processors 352 may access the acoustic data from the microphone 310 directly from the microphone 310 and / or from the memory 354. Operations may also include calculating an acoustic image based at least in part on the acoustic data from the microphone 310. Methods used to calculate the acoustic image (such as beamforming) will generally be understood by those skilled in the art and will not be described in more detail herein for the sake of brevity.

[0067] The operation may further include accessing data corresponding to the visual images from (multiple) cameras 320 (such as (multiple) cameras 120 on the portable computer device 100 and / or camera 250 on the portable acoustic imaging system 200). Thus, for example, (multiple) cameras 320 may transfer data corresponding to the visual images at (multiple) cameras 320 to (multiple) processors 352 and / or (multiple) memories 354. (Multiple) processors 352 may directly access the visual images from (multiple) cameras 320 and / or from (multiple) memories 354. The operation may further include calculating a combined image based at least in part on the acoustic and visual images. The methods used to calculate the combined image will generally be understood by those skilled in the art and will not be described in more detail herein for the sake of simplicity. The operation may further include transferring data corresponding to the combined image to a display 330, such as the display 111 of the portable computer device 100.

[0068] Battery 340 can supply power to various components of acoustic imaging system 30, such as microphone 310, (multiple) cameras 320, display 330, and / or computing device 350. Battery 340 can be mounted on portable computing device 100 and / or portable acoustic imaging system 200.

[0069] Depending on the arrangement of the acoustic imaging system 300, the alignment between the acoustic image and the visual image can vary. In an example embodiment where the camera 250 on the portable acoustic imaging system 200 is positioned at the center of a defined array of microphones 230, the alignment of the acoustic and visual images may require: (1) that the scanning area for the microphones 230 has the same horizontal and vertical field of view (FOV) as the camera 250 on the portable acoustic imaging system 200; and (2) that the acoustic image is scaled up to the higher resolution of the camera 250 on the portable acoustic imaging system 200. In an example embodiment where the camera 250 on the portable acoustic imaging system 200 is offset from the center of the defined array of microphones 230, or when using cameras(120) on the portable computer device 100, parallax correction requires: (1) identifying pixels in the visual image corresponding to the center of the acoustic image; and (2) cropping the visual image such that the FOV of the camera matches the acoustic image.

[0070] Parallax correction: Parallax correction can be used to align the acoustic and visual images when the camera 250 on the portable acoustic imaging system 200 is off-center from the defined array of microphones 230, or when multiple cameras 120 on the portable computer device 100 are used for visual images. Automated parallax correction schemes may require knowledge of the physical displacement between the center of the defined array of microphones 230 and the off-center camera. Manual parallax correction schemes can be implemented by the user using guided instructions. Both methods assume that the camera's field of view (FOV) is large enough to completely encompass the FOV of the acoustic image.

[0071] For automated parallax correction solutions Figure 10 Examples of acoustic and visual FOVs in physical space at a specific fixed distance are shown. Given these two FOVs, the goal of parallax correction is to find two coordinates in the visible light image space corresponding to the upper-left and lower-right scan points of the acoustic image, respectively. and Once these two coordinates are found, that portion of the visual image can be cropped and then superimposed on a proportionally enlarged version of the acoustic image to achieve the final image with parallax correction / alignment.

[0072] Without loss of generality, only the z-coordinate will be derived. and This is because the outlined process is simply repeated for the x-coordinate. Assume a vertical displacement between the centers of the defined array of the camera and microphone 130. The angle between two lines of sight can be calculated using the following formula (see also...). Figure 11 ): Where d is the distance between the portable computer device 100 / portable acoustic imaging system 200 and the center of the scanning area used in noise source or beamforming. This value is updated by acoustic distance estimation or external distance measurement hardware if the microphone 230 is focused at a different distance. U.S. Patent No. 11,209,306 describes the use of a laser rangefinder for parallax adjustment purposes in an acoustic imager, and the entire contents of which are incorporated herein by reference.

[0073] Next, calculate the angle between the two lines of sight. The corresponding number of pixels in visible light space. The instantaneous field of view of the VL camera in the z-direction. The description has the following formula: Vertical FOV of a single pixel in a camera unit: in It is the FOV of the camera in the z-direction, and It is the resolution of the visual image in the z-direction (i.e., the number of rows in the visual image).

[0074] use , and angle Pixel offset in the corresponding visual image space It is given by the following formula: , Now, the visual image z-coordinate corresponding to the center of the acoustic FOV in the visual image space. It is given by the following formula: in It is the z-coordinate of the center of the visual field of view (FOV).

[0075] Finally, the clipping height is calculated based on the size of the acoustic FOV in the z-direction, given the VL pixel size. . It is given by the following formula: in Refers to the FOV in the z-direction of the acoustic scanning area.

[0076] Reference Figure 12 The final VL coordinates used for parallax correction can be calculated using the following formula. and : .

[0077] For automated parallax correction schemes, all FOV and position information for the camera can be pre-determined via model or database queries.

[0078] If the FOV and position information for the camera cannot be determined automatically, the user can apply manual parallax correction. For manual correction, the user first inputs the camera's FOV and resolution information so that the following can be calculated: , Next, only the z-coordinate will be exported. and However, for the x-coordinate, the outlined processing is simply repeated.

[0079] use and The portable computer device 100 / portable acoustic imaging system 200 can crop out a portion of the visual image covering the same field of view (FOV) as the scanning area. While the user images the scene using a single, unique noise source, the cropped visual image is superimposed on the acoustic image, allowing the user to manually adjust... and (and it also adjusts automatically) and (This continues) until the noise source on the acoustic image is properly aligned with the physical source object on the visual image.

[0080] Acoustic imaging methods: Figure 13 This is a flowchart of an acoustic imaging method 400 according to an exemplary embodiment of this subject matter. The following describes an acoustic imaging system 300 (… Figure 9 ) and portable computing device 100 and portable acoustic imaging system 200 ( Figures 1 to 3 Method 400 is described in more detail in the context of [the description of the method]. As an example, computing device 350 may be programmed to implement at least a portion of method 400. It will be understood that, in the example embodiment, method 400 may be used in other acoustic imaging systems or consequently.

[0081] At 410, the portable acoustic imaging system is attached to a portable computer device. For example, at 410, a user can attach the portable acoustic imaging system 200 to the portable computing device 100. For example, at 410, a user can slide the portable computing device 100 into the slot 202 to attach the portable acoustic imaging system 200 to the portable computing device 100. As noted above, the portable acoustic imaging system 200 can have other form factors. Thus, for example, at 410, a user can attach the housing 210 of the portable acoustic imaging system 200 to the portable computing device 100 at the bottom portion 114 of the housing 110, or a user can pivotally attach the portable acoustic imaging system 200 to the portable computing device 100 at the bottom portion 114 of the housing 110. At 410, a plug 240 can also be received within a socket of the portable computing device 100, for example, to establish signal communication between the portable acoustic imaging system 200 and the portable computing device 100.

[0082] At 420, audio data from the microphone of the portable acoustic imaging system can be accessed. For example, a user can point the microphone 310 of the portable acoustic imaging system 300 (such as microphone 230 on the portable acoustic imaging system 200) toward a sound source, such as a leak, wildlife, discharge, mechanical defect, etc. At 420, the microphone 310 can transmit data corresponding to the sound at the microphone 310 to the computing device 350. As noted above, the processor(s) 352 and / or memory(s) 354 of the computing device 350 may be mounted on the portable acoustic imaging system 200 or the portable computing device 100.

[0083] At 430, an acoustic image can be calculated at least in part based on audio data from the microphone at 420. For example, at 430, computing device 350 can be configured to calculate an acoustic image at least in part based on acoustic data from microphone 310. In an example embodiment, at 430, computing device 350 can be configured to perform beamforming processing on the audio data in order to calculate the acoustic image.

[0084] At 440, video data corresponding to the visual image from the camera can be accessed. For example, a user can point multiple cameras 320 (such as multiple cameras 120 on portable computer device 100 and / or camera 250 on portable acoustic imaging system 200) toward a sound source, such as a leak, wildlife, discharge, mechanical defect, etc. In a particular example embodiment, the video data can be accessed, captured, acquired, etc., in parallel with audio data from a microphone at 420. At 440, the multiple cameras 320 can transmit the data corresponding to the visual image to computing device 350.

[0085] At 450, a combined image can be calculated, at least in part, based on the acoustic image from 430 and the visual image from 440. For example, computing device 350 can be configured to calculate the combined image by superimposing the acoustic image from 430 with the visual image from 440. Thus, the combined image can include both the acoustic image from 430 and the visual image from 440. Before calculating the combined image, parallax correction (such as the parallax correction described above) can be performed to calibrate any deviations between the acoustic and visual images.

[0086] At 460, a combined image is presented on the display of a portable computer device. For example, at 460, the combined image can be presented on display 330 (such as display 111 of portable computer device 100). The user can view the combined image on display 330 to assist in reducing noise sources in product design, locating compressed air leaks, wildlife, partial discharge, mechanical defects, etc. Thus, for example, the user can see both visual information from visual images from (multiple) cameras 320 and acoustic images on display 330. The acoustic images can visually inform the user about the location, amplitude, direction, etc., of sound captured by microphone 310 at 420.

[0087] After viewing the combined images, the portable acoustic imaging system can be removed from the portable computer device. For example, the user can uninstall the portable acoustic imaging system 200 from the portable computing device 100 after completing method 400, and not until the user subsequently starts method 400 again.

[0088] Figure 13 The steps performed in a specific order are described for illustrative and discussion purposes. Using the disclosure provided herein, those skilled in the art will understand that the steps of any method discussed herein can be adapted, rearranged, extended, omitted, or modified in various ways without departing from the scope of this disclosure. Furthermore, while aspects of method 400 have been explained using acoustic imaging system 300, portable computing device 100, and portable acoustic imaging system 200 as examples, it should be understood that these methods can be applied to other acoustic imaging systems.

[0089] These and other modifications and variations of the invention can be practiced by those skilled in the art without departing from the spirit and scope of the invention as more specifically set forth in the appended claims. Furthermore, it should be understood that aspects of the various embodiments can be interchanged entirely or in part. Moreover, those skilled in the art will appreciate that the foregoing description is by way of example only and is not intended to limit the invention as described herein by such appended claims.

[0090] Example Implementation Further aspects of this disclosure are provided by the subject matter of the following provisions concerning portable acoustic imaging systems and acoustic imaging methods.

[0091] A portable acoustic imaging system optionally includes a housing that is selectively attachable to a portable computer device. Optionally, a plurality of microphones are deployed within the housing, arranged in a defined array for capturing acoustic data.

[0092] Optionally, the portable acoustic imaging system includes one or more processors and one or more non-transitory computer-readable media, the one or more non-transitory computer-readable media storing instructions executable by the one or more processors to perform operations. These operations may optionally include: accessing acoustic data from a microphone; and calculating an acoustic image based at least in part on the acoustic data from the microphone.

[0093] Optionally, one or more processors and one or more non-transitory computer-readable media are deployed within a housing. Alternatively, one or more processors and one or more non-transitory computer-readable media may be deployed within a portable computer device.

[0094] Optionally, the operation further includes: accessing data corresponding to the visual image from the camera; and calculating a combined image based at least in part on the acoustic image and the visual image. Optionally, the operation further includes: transferring data corresponding to the combined image to the display of a portable computer device. Optionally, the operation further includes: implementing automatic calibration for aligning the visual image and the acoustic image. Optionally, the automatic calibration includes parallax correction relative to the center of the camera.

[0095] Optionally, the portable acoustic imaging system includes a camera for capturing visual images. The camera is optionally deployed within a housing. Optionally, the camera is deployed within the outer perimeter of a defined array. Optionally, the camera is spaced no more than five centimeters from the defined array. Alternatively, the camera is optionally deployed within a portable computer device.

[0096] Optionally, the housing defines a plurality of openings on its front surface, with each of the camera and microphone being deployed within the housing at one or more corresponding locations in the openings, the front surface of the housing facing away from the portable computer device when the housing is attached to the portable computer device.

[0097] Optionally, the portable acoustic imaging system includes one or more processors and one or more non-transitory computer-readable media, the one or more non-transitory computer-readable media storing instructions executable by the one or more processors to perform operations. These operations may optionally include: accessing acoustic data from a microphone; calculating an acoustic image based at least in part on the acoustic data from the microphone; accessing data corresponding to a visual image from a camera; and calculating a combined image based at least in part on the acoustic image and the visual image.

[0098] Optionally, the operation further includes: transmitting data corresponding to the combined image to the display of a portable computer device. Optionally, the portable computer device includes a smartphone or tablet. Optionally, the housing includes a sleeve configured to receive the portable computer device. Optionally, the portable acoustic imaging system includes a data transmission and power delivery plug mounted to the housing, the plug being configured to be received within a socket of the portable computer device when the housing is attached to the portable computer device.

[0099] Optionally, the portable acoustic imaging system includes a microphone array accessory configured to be detachably connected to a portable computer device, wherein the microphone array accessory includes a plurality of microphones arranged in a predefined pattern for capturing acoustic data.

[0100] Optionally, the portable acoustic imaging system includes a processor located within a microphone array accessory or a portable computer device, configured to process acoustic data to generate an acoustic image. Optionally, the portable acoustic imaging system includes a camera for capturing visual images, wherein the visual and acoustic images are processed to generate a combined image.

[0101] Optionally, the camera is built into a microphone array accessory or integrated into a portable computer device. Optionally, the portable acoustic imaging system includes a user interface that runs on the portable computer device for displaying combined images and for controlling the operation of the acoustic imaging system.

[0102] Optionally, the processor is configured to perform a calibration process for aligning a visual image from the camera with an acoustic image from the microphone array. Optionally, the calibration process includes parallax correction relative to the center of the camera on the portable computer device.

[0103] Optionally, the portable computer device is a smartphone or tablet. Optionally, the microphone array accessory includes a sleeve shape factor, a bottom-mounted shape factor, or a hinged shape factor.

[0104] Optionally, the acoustic imaging method includes: accessing audio data from a microphone array accessory detachably connected to a portable computer device. The acoustic imaging method may also include: calculating an acoustic image based at least in part on the acoustic data from the microphone array accessory. The acoustic imaging method may also include: accessing video data corresponding to visual images from one or more cameras. The acoustic imaging method may also include: calculating a combined image based at least in part on the acoustic image and the visual image; and transmitting data corresponding to the combined image to a display of the portable computer device.

[0105] Optionally, one or more cameras are deployed in a portable computer device. Optionally, one or more cameras are deployed in a microphone array accessory. Optionally, one or more cameras are deployed in both a portable computer device and a microphone array accessory.

[0106] Optionally, the method further includes: implementing calibration for aligning visual and acoustic images, which may include parallax correction relative to the center of the camera. Optionally, the portable computer device is a smartphone or tablet, and the microphone array accessory includes a sleeve shape factor, a bottom-mounted shape factor, or an articulated shape factor.

[0107] Alternatively, the portable acoustic imaging system is substantially as described herein. Alternatively, the acoustic imaging method is substantially as described herein.

Claims

1. A portable acoustic imaging system, comprising: A housing that can be selectively attached to a portable computer device; and Multiple microphones are deployed within the housing and arranged in a defined array to capture acoustic data.

2. The portable acoustic imaging system of claim 1, further comprising one or more processors and one or more non-transitory computer-readable media, said one or more non-transitory computer-readable media storing instructions executable by said one or more processors to perform operations including: Access acoustic data from the microphone; as well as The acoustic image is calculated at least in part based on acoustic data from the microphone.

3. The portable acoustic imaging system of claim 2, wherein the one or more processors and the one or more non-transitory computer-readable media are deployed in a housing.

4. The portable acoustic imaging system of claim 2, wherein the one or more processors and the one or more non-transitory computer-readable media are deployed in a portable computer device.

5. The portable acoustic imaging system of claim 2, wherein the operation further includes: Access the data corresponding to the visual images from the camera; as well as The combined image is calculated based at least in part on acoustic and visual images.

6. The portable acoustic imaging system of claim 5, wherein the operation further includes: The data corresponding to the combined image is transmitted to the display of the portable computer device.

7. The portable acoustic imaging system of claim 5, wherein the operation further includes: Automatic calibration is implemented for aligning visual and acoustic images.

8. The portable acoustic imaging system of claim 7, wherein automatic calibration includes parallax correction relative to the center of the camera.

9. The portable acoustic imaging system of claim 1, further comprising a camera for capturing visual images.

10. The portable acoustic imaging system of claim 9, wherein the camera is deployed within the housing.

11. The portable acoustic imaging system of claim 10, wherein the camera is deployed within the periphery of a defined array.

12. The portable acoustic imaging system of claim 10, wherein the camera is spaced no more than five centimeters from the defined array.

13. The portable acoustic imaging system of claim 9, wherein the camera is deployed in a portable computer device.

14. The portable acoustic imaging system of claim 9, wherein the housing defines a plurality of openings on the front surface of the housing, each of the camera and microphone being deployed within the housing at one or more corresponding locations in the openings, and the front surface of the housing facing away from the portable computer device when the housing is attached to the portable computer device.

15. The portable acoustic imaging system of claim 9, further comprising one or more processors and one or more non-transitory computer-readable media, said one or more non-transitory computer-readable media storing instructions executable by said one or more processors to perform operations including: Access acoustic data from the microphone; The acoustic image is calculated based at least in part on acoustic data from the microphone; Access the data corresponding to the visual images from the camera; as well as The combined image is calculated based at least in part on acoustic and visual images.

16. The portable acoustic imaging system of claim 15, wherein the operation further includes: The data corresponding to the combined image is transmitted to the display of the portable computer device.

17. The portable acoustic imaging system of claim 1, wherein the portable computer device includes a smartphone or tablet.

18. The portable acoustic imaging system of claim 1, wherein the housing includes a sleeve configured to receive a portable computer device.

19. The portable acoustic imaging system of claim 1, further comprising a data transmission and power delivery plug, the data transmission and power delivery plug being mounted to the housing, the plug being configured to be received in a socket of the portable computer device when the housing is attached to the portable computer device.

20. A portable acoustic imaging system, comprising: A microphone array accessory configured to be detachably connected to a portable computer device, wherein the microphone array accessory includes a plurality of microphones arranged in a predefined pattern for capturing acoustic data.

21. The portable acoustic imaging system of claim 20, further comprising a processor located within a microphone array accessory or a portable computer device, configured to process acoustic data to generate an acoustic image.

22. The portable acoustic imaging system of claim 21, further comprising a camera for capturing visual images, wherein the visual and acoustic images are processed to generate a combined image.

23. The portable acoustic imaging system of claim 22, wherein the camera is built into a microphone array accessory.

24. The portable acoustic imaging system of claim 22, wherein the camera is integrated within a portable computer device.

25. The portable acoustic imaging system of claim 22, further comprising a user interface running on a portable computer device for displaying combined images and for controlling the operation of the portable acoustic imaging system.

26. The portable acoustic imaging system of claim 21, wherein the processor is configured to perform a calibration process for aligning a visual image captured by a camera of a portable computer device with an acoustic image from a microphone array accessory.

27. The portable acoustic imaging system of claim 26, wherein the calibration process includes parallax correction relative to the center of the camera of the portable computer device.

28. The portable acoustic imaging system of claim 20, wherein the portable computer device is a smartphone or tablet.

29. The portable acoustic imaging system of claim 20, wherein the microphone array accessory includes a sleeve shape factor, a bottom-mounted shape factor, or a hinged shape factor.

30. An acoustic imaging method, comprising: Access audio data from a microphone array accessory that is detachably connected to a portable computer device; The acoustic image is calculated at least in part based on audio data from the microphone array attachment; Access video data corresponding to visual images from one or more cameras; The combined image is calculated based at least in part on acoustic and visual images; as well as The data corresponding to the combined image is transmitted to the display of the portable computer device.

31. The method of claim 30, wherein the one or more cameras are deployed in a portable computer device.

32. The method of claim 30, wherein the one or more cameras are deployed in a microphone array accessory.

33. The method of claim 30, wherein the one or more cameras are deployed in both a portable computer device and a microphone array accessory.

34. The method of claim 30, further comprising: Achieve calibration to align visual and acoustic images.

35. The method of claim 34, wherein calibration includes parallax correction relative to the center of a corresponding camera among the one or more cameras.

36. The method of claim 30, wherein the portable computer device is a smartphone or tablet.

37. The method of claim 36, wherein the microphone array accessory includes a sleeve shape factor, a bottom-mounted shape factor, or a hinged shape factor.

Images