A sound source three-dimensional positioning system and method based on binocular acoustic camera

Abstract

The application relates to the technical field of sound source positioning, and discloses a sound source three-dimensional positioning system and method based on a binocular acoustic camera, wherein the positioning system comprises a first acoustic camera and a second acoustic camera, which are used for shooting the same sound source from different positions to obtain an acoustic image; an information processing system is in communication connection with the first acoustic camera and the second acoustic camera, is used for receiving and processing acoustic image data collected by the two acoustic cameras, and calculates the position coordinates of a sound source in a three-dimensional space based on the relative position relationship of the first acoustic camera and the second acoustic camera, a horizontal pixel path, a vertical pixel path and pixel coordinates of the sound source in the acoustic image of the first acoustic camera and the second acoustic camera. The application breaks through the limitation that a traditional single acoustic camera can only be two-dimensionally positioned, the acoustic images are collected by the two cameras from different positions, the information processing system constructs complete logic of multi-source data fusion-three-dimensional coordinate calculation, and a basic solution scheme is provided for complex three-dimensional space sound source positioning.

Description

Technical Field

[0001] This invention relates to the field of sound source localization technology, specifically a sound source three-dimensional localization system and method based on a binocular acoustic camera. Background Technology

[0002] In the field of sound source localization technology, acoustic cameras are the core devices for realizing visual localization of sound sources. They generate a two-dimensional acoustic image corresponding to the sound source by collecting sound field information, and then overlay the acoustic image with the optical image captured by the camera, thereby realizing the location identification of the sound source in a two-dimensional plane (i.e., the photographic image).

[0003] However, in real-world applications with depth (such as locating sound sources in industrial equipment malfunctions and tracing the source of unusual noises in building spaces), a single acoustic camera has significant technical limitations: because a single device can only acquire the pixel coordinates of the sound source in a two-dimensional plane, it cannot establish a correlation between pixel positions and actual spatial depth. Therefore, it cannot calculate the actual distance between the sound source and the camera, nor can it determine the true height of the sound source in physical space. This limitation means that a single acoustic camera can only achieve "planar positioning," failing to meet the need for precise three-dimensional spatial positioning of sound sources, thus restricting the application scope and positioning accuracy of acoustic cameras in complex three-dimensional scenes. Summary of the Invention

[0004] In view of this, in order to solve the problem that a single acoustic camera cannot achieve three-dimensional localization of sound sources, this invention provides a three-dimensional localization system and control method for sound sources based on binocular acoustic cameras, which outputs the three-dimensional position information of the sound source in physical space with parameters of "distance from the two cameras + height from the ground", thus meeting the requirements for accurate localization of sound sources in complex three-dimensional scenes.

[0005] In a first aspect, the present invention provides a sound source three-dimensional localization system based on a binocular acoustic camera, comprising:

[0006] The first acoustic camera and the second acoustic camera are used to capture images of the same sound source from different locations to obtain acoustic images;

[0007] The information processing system is communicatively connected to the first acoustic camera and the second acoustic camera. It is used to receive and process the acoustic image data collected by the two acoustic cameras. Based on the relative positional relationship between the first acoustic camera and the second acoustic camera, the horizontal pixel path, the vertical pixel path, and the pixel coordinates of the sound source in the acoustic image of the first acoustic camera and the second acoustic camera, it calculates the position coordinates of the sound source in three-dimensional space.

[0008] The sound source 3D localization system based on binocular acoustic cameras provided in this invention overcomes the limitation of traditional single acoustic cameras, which can only perform 2D localization, through the architecture of "dual acoustic cameras + information processing system". The dual cameras acquire acoustic images from different locations, providing a foundation for obtaining sound source depth information. The information processing system integrates multi-dimensional data such as relative positional relationships, horizontal / vertical pixel paths, and pixel coordinates to construct a complete logic of "multi-source data fusion - 3D coordinate calculation". This allows for direct derivation of the sound source's spatial location without additional auxiliary equipment, ensuring the realization of the core function of 3D localization. Furthermore, it is adaptable to various scenarios such as industrial fault detection, providing a fundamental solution for locating sound sources in complex 3D spaces.

[0009] In one optional implementation, the position coordinates of the sound source in three-dimensional space include: the distance L1 from the sound source to the first acoustic camera, the distance L2 from the sound source to the second acoustic camera, and the actual height h from the ground.

[0010] This invention, by quantifying the horizontal distance from the sound source to the dual cameras and the vertical height from the ground, transforms the positioning result from an "abstract spatial concept" into "precisely measurable parameters," significantly improving the practicality of the technical solution. At the same time, it provides a standardized data format for subsequent data storage, analysis, and linkage with other detection systems, enhancing the system's adaptability to practical applications.

[0011] In one optional embodiment, the system further includes: a camera bracket for supporting the first acoustic camera and the second acoustic camera; the first acoustic camera and the second acoustic camera are configured to be at the same height, such that the center of the two acoustic cameras is at a height h0 above the ground, and their front faces are perpendicular to the ground, with the included angle between the frontal normals of the two cameras being between 30° and 120°.

[0012] This invention provides a standardized deployment benchmark for system hardware by adding a camera bracket and defining the deployment parameters of the dual cameras. The setup, with both cameras at the same height and perpendicular to the ground, ensures that the pixel coordinates of the audio-visual images acquired by the two cameras have a unified vertical reference, avoiding pixel coordinate misalignment due to height differences and reducing subsequent calculation errors. The normal angle range of 30°-120° balances field-of-view coverage and depth information capture capabilities, preventing insufficient depth data due to excessively small angles and avoiding missed sound source capture due to excessively large angles. This setup allows for standardized hardware deployment without complex calibration tools, lowering the operational threshold on-site, while improving the consistency and reliability of audio-visual data, providing high-quality input data for accurate calculations in the information processing system.

[0013] In one optional implementation, the information processing system includes:

[0014] A coordinate system construction module is used to construct a coordinate system based on the horizontal pixel path of the first acoustic camera and the horizontal pixel path of the second acoustic camera. The coordinate system is based on the relative position of the two acoustic cameras and the trajectory of the horizontal pixel path is used as the coordinate axis reference. A sound source distance calculation module is used to determine the distance L1 of the sound source from the first acoustic camera and the distance L2 of the sound source from the second acoustic camera based on the horizontal coordinate x1 of the sound source in the acoustic image of the first acoustic camera and the horizontal coordinate x2 of the sound image of the second acoustic camera.

[0015] The sound source height calculation module is used to calculate the vertical distance h1 of the sound source relative to the center pixel of the first acoustic camera based on the vertical pixel path of the first acoustic camera and the distance L1, combined with the vertical coordinate y1 of the sound source in the acoustic image of the first acoustic camera, and to calculate the actual height h of the sound source based on the vertical distance h1 of the center pixel of the first acoustic camera and the height h0 of the center of the two acoustic cameras from the ground.

[0016] The coordinate system construction module of this invention establishes a spatial calculation benchmark based on the horizontal pixel path, resolving calculation deviations caused by differences in the positions of the two cameras; the sound source distance calculation module focuses on the horizontal coordinate data, simplifying the horizontal distance derivation process and improving calculation efficiency; the sound source height calculation module combines the vertical pixel path with L1 to fill the gap in vertical dimension positioning. These three modules form a progressive relationship of "first establish benchmark - then calculate horizontal - then calculate vertical," with the output of the preceding module directly serving as the input for the subsequent module. This ensures that data processing is interconnected, reducing redundant calculations and error accumulation, making 3D coordinate calculations more accurate and stable, while also reducing the complexity of system software development.

[0017] In one optional implementation, the horizontal pixel path is a path map formed by connecting points of the same horizontal coordinate at different distances of the acoustic camera, and its boundary is the horizontal viewing angle of the acoustic camera; the vertical pixel path is a path map formed by connecting points of the same vertical coordinate at different distances of the acoustic camera, and its boundary is the vertical viewing angle of the acoustic camera.

[0018] This invention establishes a direct mapping relationship between pixel coordinates and actual spatial positions by defining "lines connecting the same horizontal / vertical coordinate points at different distances," transforming the abstract "path" into a quantifiable spatial trajectory. The limitation of the viewpoint boundary clarifies the effective calculation range of the path, helping information processing systems quickly filter out invalid data and avoid interference from pixel coordinates outside the viewpoint. For example, a horizontal pixel path can accurately correlate the horizontal coordinate with the horizontal distance, and a vertical pixel path can match the vertical coordinate with the vertical height, providing core technical support for coordinate system construction and distance and height calculation. This is a key support for realizing the transformation from "pixel data to spatial parameters."

[0019] In one optional implementation, the actual height h of the sound source is calculated using the formula h = h0 + h1, where h0 is the height of the acoustic camera center from the ground, and h1 is the vertical distance of the sound source relative to the camera center pixel calculated based on the vertical coordinate y1 and the vertical pixel path.

[0020] This invention, through a clearly defined height calculation formula (h = h0 + h1), transforms the sound source height calculation from a "module functional description" into "directly executable quantization logic." h0 represents known hardware deployment parameters, while h1 is derived based on pixel coordinates and vertical pixel paths. The combination of these two factors provides a clear mathematical basis for height calculation, avoiding height errors caused by ambiguity in the calculation logic.

[0021] Secondly, the present invention provides a method for three-dimensional localization of sound sources based on a binocular acoustic camera, comprising the following steps:

[0022] Acoustic images of the same sound source are acquired from different locations using a first acoustic camera and a second acoustic camera, and transmitted to an information processing system. The information processing system calculates the position coordinates of the sound source in three-dimensional space based on the relative positional relationship between the first acoustic camera and the second acoustic camera, the horizontal pixel path, the vertical pixel path, and the pixel coordinates of the sound source in the acoustic images of the first acoustic camera and the second acoustic camera.

[0023] The present invention provides a stereo acoustic camera-based 3D sound source localization method, constructing a complete "data acquisition-computation localization" process. Dual cameras acquire acoustic images to provide raw data for subsequent calculations. The information processing system integrates multi-dimensional data to derive 3D coordinates, achieving synergy and consistency with the overall system solution. This method does not rely on specific hardware models; 3D localization can be achieved through standardized steps, exhibiting strong versatility and portability. In adapting to different brands of acoustic cameras, consistent localization results can be achieved simply by deploying equipment and processing data according to this method, meeting the implementation requirements of different scenarios.

[0024] In one alternative implementation, before acquiring the acoustic image, the first and second acoustic cameras are adjusted to the same height using a camera bracket, so that their front faces are perpendicular to the ground, and the included angle between the frontal normals of the two cameras is between 30° and 120°.

[0025] This invention, by clearly defining the adjustment and deployment parameters of the camera bracket, transforms the method from "abstract steps" into a "practical on-site operation guide." Workers can standardize equipment deployment by following the steps of "adjusting to the same height - ensuring verticality to the ground - controlling the angle range," avoiding deviations in audio-visual data caused by improper deployment. For example, in industrial plant sites, no specialized technicians are required; ordinary maintenance personnel can complete equipment deployment by following these steps, significantly reducing the labor costs and technical barriers to implementation. Simultaneously, it ensures consistent hardware deployment across different sites and by different personnel, providing a stable foundation for subsequent data acquisition and calculation.

[0026] In one optional implementation, the step of calculating and determining the position coordinates of the sound source in three-dimensional space includes: constructing a spatial coordinate system based on the horizontal pixel path of the first acoustic camera and the horizontal pixel path of the second acoustic camera using the coordinate system construction module of the information processing system;

[0027] The sound source distance calculation module determines the distance L1 from the first acoustic camera and the distance L2 from the second acoustic camera based on the pixel coordinates x1, x2 and the mapping relationship of the spatial coordinate system.

[0028] The sound source height calculation module calculates the vertical distance h1 of the sound source relative to the center pixel based on the vertical pixel path, distance L1, and pixel coordinate y1 of the first acoustic camera, and calculates the actual height h of the sound source using the formula h = h0 + h1.

[0029] The coordinate system construction in this embodiment of the invention provides a benchmark for calculation, the distance calculation locks the horizontal orientation, and the height calculation supplements the vertical dimension. The three-step progressive operation makes the complex calculation process clearer and easier to execute. This refined step not only ensures the integrity and rigor of the calculation logic, but also enhances the operability and maintainability of the method, making the three-dimensional positioning method easier to implement and apply.

[0030] In one optional implementation, when calculating distances L1 and L2, the information processing system converts pixel coordinates (x1, x2) into actual spatial horizontal distance parameters based on the mapping relationship of horizontal pixel paths in the spatial coordinate system. When calculating vertical distance h1, the information processing system combines the pixel distribution ratio corresponding to the vertical viewpoint of the first acoustic camera (1) to convert the pixel offset corresponding to pixel coordinate y1 into actual spatial vertical distance h1.

[0031] In this embodiment of the invention, the horizontal direction is transformed based on the coordinate system mapping relationship to convert x1 and x2, ensuring that the pixel offset corresponds accurately to the actual horizontal distance; the vertical direction is transformed based on the pixel distribution ratio of the vertical viewpoint to convert y1, avoiding errors where the same pixel coordinate corresponds to different heights at different distances. For example, in long-distance sound source localization, the actual distance corresponding to the pixel offset can be accurately magnified through the mapping relationship; in short-distance localization, the vertical height calculation is refined based on the pixel distribution ratio, improving the resolution of small-range height differences. This design provides a clear technical basis for the calculation process, reduces subjective estimation errors, further improves the accuracy of 3D localization, and meets the needs of scenarios with high positioning accuracy requirements (such as precision instrument fault detection). Attached Figure Description

[0032] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the module composition of a sound source three-dimensional localization system based on a binocular acoustic camera according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of an acoustic image acquired by a binocular acoustic camera according to an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the horizontal pixel path of an acoustic image acquired by a binocular acoustic camera according to an embodiment of the present invention.

[0036] Figure 4 This is a schematic diagram of another module of a sound source three-dimensional localization system based on a binocular acoustic camera according to an embodiment of the present invention;

[0037] Figure 5 This is a flowchart illustrating a three-dimensional sound source localization method based on a binocular acoustic camera according to an embodiment of the present invention.

[0038] Figure 6 This is another schematic flowchart of a sound source three-dimensional localization method based on a binocular acoustic camera according to an embodiment of the present invention.

[0039] Explanation of reference numerals in the attached figures:

[0040] 101-First acoustic camera, 102-Second acoustic camera, 103-Information processing system, 1041-Horizontal pixel path of the first acoustic camera, 1042-Horizontal pixel path of the second acoustic camera, 105-Vertical pixel path of the first acoustic camera, 106-Sound source, 107-Camera support, 108-Ground, 109-Center pixel. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] This invention provides a sound source three-dimensional localization system based on a binocular acoustic camera, such as... Figure 1 As shown, it includes: a first acoustic camera (101) and a second acoustic camera (102), used to capture images of the same sound source from different locations to obtain acoustic images; wherein the acoustic images are as follows: Figure 2 As shown in (1), the first acoustic camera captures an image of size m1*n1, where (x1, y1) are the pixel coordinates of the sound source on the image, m1 is the maximum horizontal pixel, n1 is the maximum vertical pixel, x1 is the horizontal coordinate of the pixel, and y1 is the vertical coordinate of the pixel, as shown in (1). Figure 2 (2) The second acoustic camera captures an image of m2*n2, where (x2, y2) are the pixel coordinates of the sound source on the image, m2 is the maximum horizontal pixel, n2 is the maximum vertical pixel, x2 is the horizontal coordinate of the pixel, and y2 is the vertical coordinate of the pixel.

[0043] The information processing system (103) is communicatively connected to the first acoustic camera (101) and the second acoustic camera (102), and is used to receive and process the acoustic image data collected by the two acoustic cameras. Based on the relative positional relationship between the first acoustic camera and the second acoustic camera, the horizontal pixel path, the vertical pixel path, and the pixel coordinates of the sound source in the acoustic image of the first acoustic camera and the second acoustic camera, the system calculates the position coordinates of the sound source in three-dimensional space.

[0044] The above horizontal pixel path, such as Figure 3 The horizontal pixel paths (1041) of the first acoustic camera and (1042) of the second acoustic camera shown in the diagram are path maps formed by connecting points with the same horizontal coordinate at different distances from the acoustic cameras. Their boundaries are the horizontal viewing angles of the acoustic cameras; the vertical pixel paths (such as...) Figure 4The vertical pixel path (105) of the first acoustic camera shown is a path diagram formed by connecting points of the same vertical coordinate at different distances of the acoustic camera, and its boundary is the vertical viewing angle of the acoustic camera.

[0045] In this embodiment of the invention, the horizontal pixel path directly associates the horizontal pixel coordinates of the acoustic camera with the horizontal positions at different depths in actual space by connecting the same horizontal coordinate points at different distances. Its boundary matches the horizontal viewing angle of the acoustic camera, clearly defining the spatial coverage range of the pixel coordinates in the horizontal direction. Similarly, the vertical pixel path establishes a correspondence between the vertical pixel coordinates and the vertical height in actual space by connecting the same vertical coordinate points at different distances. The boundary matching the vertical viewing angle ensures the accuracy of spatial coverage in the vertical direction. Together, these methods overcome the limitation of traditional two-dimensional acoustic images where "pixel coordinates cannot be associated with actual spatial positions," providing a direct conversion basis for information processing systems from "pixel data to spatial parameters," and serving as the core technical support for achieving three-dimensional sound source localization.

[0046] In this embodiment of the invention, the boundaries of the horizontal and vertical pixel paths correspond to the horizontal and vertical viewing angles of the acoustic camera, respectively. This characteristic helps the information processing system quickly filter valid data: in the acquired acoustic image, if the sound source pixel coordinates (x1, y1 or x2, y2) exceed the path boundary (i.e., exceed the camera's viewing angle range), the system can directly determine that the data is invalid, without needing further calculation; conversely, if it is within the path boundary, the system can focus on the valid path trajectory for parameter derivation. This boundary filtering mechanism avoids interference from invalid pixel data in the calculation process, reduces redundant calculation steps, significantly improves the data processing efficiency of the information processing system, and ensures that the 3D positioning results can be output quickly.

[0047] Compared to the limitations of traditional single acoustic cameras, which can only generate two-dimensional acoustic images and cannot obtain sound source depth (distance) and actual height, this invention overcomes the limitation of traditional single acoustic cameras only being able to locate in two dimensions through a "dual acoustic camera + information processing system" architecture. The dual cameras acquire acoustic images from different locations, providing a foundation for obtaining sound source depth information. The information processing system integrates multi-dimensional data such as relative positional relationships, horizontal / vertical pixel paths, and pixel coordinates to construct a complete logic of "multi-source data fusion - three-dimensional coordinate calculation," which can directly deduce the spatial location of the sound source without additional auxiliary equipment. This design not only ensures the realization of the core function of three-dimensional positioning but also provides a unified technical framework for subsequent refined optimizations. It can be adapted to various scenarios such as industrial fault detection and building noise tracing, providing a basic solution for sound source localization in complex three-dimensional spaces. It fills the technical gap that single devices cannot achieve three-dimensional localization, significantly improving the spatial integrity and accuracy of sound source localization.

[0048] Furthermore, the position coordinates of the sound source in three-dimensional space obtained through the above system include: the distance L1 of the sound source (106) from the first acoustic camera, the distance L2 from the second acoustic camera, and the actual height h from the ground. By quantifying the horizontal distance of the sound source from the two cameras and the vertical height from the ground, the positioning result is transformed from an "abstract spatial concept" into "precisely measurable parameters," greatly improving the practicality of the technical solution. For example, in industrial equipment maintenance, staff can quickly locate the horizontal position of the sound source based on L1 and L2, and determine the vertical height of the faulty equipment component based on h, avoiding low troubleshooting efficiency caused by ambiguous positioning parameters. At the same time, it provides a standardized data format for subsequent data storage, analysis, and linkage with other detection systems, enhancing the adaptability of the system to practical applications.

[0049] like Figure 4 As shown, the system provided in this embodiment of the invention also includes a camera bracket (107) for supporting the first acoustic camera and the second acoustic camera; the first acoustic camera and the second acoustic camera are set at the same height, so that the center of the two acoustic cameras is at a height of h0 from the ground, and their front faces are perpendicular to the ground (108), and the angle between the front normals of the two cameras is between 30° and 120°. This setting establishes a unified reference for the acquisition angle of the two cameras in the vertical direction, avoiding the problem of misalignment of the reference of the sound image pixel coordinates (x1, y1) and (x2, y2) due to the difference in height and tilt angle of the two cameras, ensuring that the data has a consistent comparison basis when the subsequent information processing system (103) calculates the sound source distance (L1, L2) and vertical distance (h1) based on the pixel coordinates of the two cameras, reducing the positioning error from the hardware deployment level and improving the accuracy of three-dimensional coordinate calculation.

[0050] This invention provides a convenient support and fixing structure for dual-camera height adjustment and vertical ground calibration via a camera bracket. It achieves uniform height and vertical attitude calibration of the dual cameras without the need for additional complex measuring tools. It can effectively resist the influence of slight vibrations in the field environment (such as vibrations from industrial equipment operation, airflow disturbances, etc.) on the camera attitude, and avoid problems such as changes in height reference and deviation of the included angle from the set range caused by attitude shift of the dual cameras, ensuring that the dual cameras are always in a stable acquisition state. At the same time, the uniform height and vertical ground setting makes it less likely for the relative position relationship of the dual cameras to shift due to equipment deformation under different environmental temperature and humidity changes, further ensuring the stability of audio-visual data acquisition and improving the reliable operation capability of the system in complex field environments.

[0051] The angle between the two cameras' frontal normals is set within the range of 30° to 120°. This angle range avoids the problems of excessive overlap in the field of view of the two cameras and insufficient capture of sound source depth information caused by an excessively small angle, while also preventing the situation of misalignment of the field of view and difficulty in simultaneously and accurately capturing the same sound source image due to an excessively large angle. This angle range ensures that the two cameras can effectively cover the target monitoring area while forming a reasonable field of view intersection relationship, enabling the information processing system (103) to quickly establish the association between the pixel coordinates (x1, x2) of the two cameras and the spatial position of the sound source through the horizontal pixel path, efficiently derive the sound source distance parameters, and improve the success rate of capturing the same sound source and the efficiency of positioning calculation. In addition, it takes into account the monitoring needs of different scenarios (such as a smaller angle can be selected for close-range monitoring in small spaces, and a larger angle can be selected for long-range monitoring in large spaces), and this angle range can meet the positioning requirements without high-precision angle control, reducing the operational difficulty of on-site deployment. Whether in industrial plants, building spaces, or other scenarios, staff can quickly complete the standardized deployment of the two cameras, reduce system debugging time, and improve the convenience of practical applications.

[0052] Specifically, the information processing system (103) of this embodiment includes:

[0053] A coordinate system construction module is used to construct a coordinate system based on the horizontal pixel paths of the first acoustic camera and the second acoustic camera. This coordinate system uses the relative positions of the two acoustic cameras as a reference and the trajectory of the horizontal pixel paths as the coordinate axes. This embodiment of the invention uses the relative positions of the two acoustic cameras as a reference, combined with the horizontal pixel path trajectories, to construct a coordinate system, establishing a direct mapping relationship between the abstract "pixel coordinates" and the "actual spatial position." Compared to calculation methods without a clear reference, this module eliminates calculation deviations caused by differences in the positions of the two cameras, providing a unified spatial framework for subsequent distance and height calculations. This ensures logical consistency in the conversion from pixel data to spatial parameters, avoids positioning errors caused by reference misalignment, and provides an underlying guarantee for overall calculation accuracy. A sound source distance calculation module is used to determine the distance L1 from the sound source to the first acoustic camera and the distance L2 from the sound source to the second acoustic camera based on the abscissa x1 of the sound source in the acoustic image of the first acoustic camera and the abscissa x2 of the sound image of the second acoustic camera. The sound source distance calculation module in this embodiment of the invention only needs to rely on the horizontal coordinates (x1, x2) of the sound source in the dual-camera acoustic image to quickly determine the distance L1 from the first acoustic camera and the distance L2 from the second acoustic camera. It eliminates the need for additional sensors (such as laser rangefinders) or complex multi-parameter fitting algorithms, significantly simplifying the distance calculation process through the direct correlation of "horizontal pixel coordinates - horizontal pixel path - spatial distance". Simultaneously, this module focuses on key parameters in the horizontal direction (horizontal coordinates), avoiding interference from irrelevant pixel data (such as vertical coordinates), shortening data processing time, and improving the efficiency and accuracy of horizontal sound source localization.

[0054] The sound source height calculation module is used to calculate the vertical distance h1 of the sound source relative to the center pixel of the first acoustic camera based on the vertical pixel path of the first acoustic camera and the distance L1, combined with the vertical coordinate y1 of the sound source in the acoustic image of the first acoustic camera, and to calculate the actual height h of the sound source (106) based on the vertical distance h1 of the center pixel of the first acoustic camera and the height h0 of the center of the two acoustic cameras from the ground (108). The sound source height calculation module combines the vertical pixel path (105) of the first acoustic camera, the determined distance L1 and the vertical coordinate y1 of the sound source to construct a calculation link of "distance-vertical pixel coordinate-actual height": first, the corresponding spatial line is locked in the vertical pixel path with the distance L1, then the vertical distance h1 of the sound source relative to the center pixel (109) is determined by the vertical coordinate y1, and finally the actual height h is obtained by combining the unified height h0 of the two cameras. This module cleverly utilizes the previously calculated distance L1 as an intermediate reference, avoiding the problem that a single camera cannot calculate the height due to a lack of depth information. It accurately fills the gap in the vertical dimension of traditional two-dimensional positioning, ensuring that the three-dimensional position parameters (distance L1 / L2, height h) of the sound source are complete and reliable.

[0055] The three main modules of the information processing system form a collaborative logic of first establishing the coordinate system, then calculating distance, and finally calculating altitude: the coordinate system construction module provides a benchmark for distance calculation, the result (L1) of the distance calculation module provides key parameters for altitude calculation, and the output of the preceding modules directly serves as the input for the subsequent modules, making the entire data processing process interconnected and mutually verifying. This collaborative mode can effectively reduce the impact of single parameter errors on the final result (such as a small deviation in the horizontal coordinate x1, which can be corrected by associating the coordinate system benchmark with another camera x2), significantly improving the reliability and fault tolerance of the 3D positioning results, and ensuring stable and accurate output of sound source spatial coordinates even in complex sound field environments.

[0056] The actual height h of the sound source calculated by the information processing system is obtained using the formula h = h0 + h1, where h0 is the height of the acoustic camera center from the ground, and h1 is the vertical distance of the sound source relative to the camera center pixel calculated based on the vertical coordinate y1 and the vertical pixel path. Combining these two factors provides a clear mathematical basis for height calculation, avoiding height errors caused by ambiguous calculation logic. In practical applications, such as locating abnormal noises within building ceilings, the height of the noise location from the ground can be quickly calculated using h0 (camera height) and h1 (vertical offset), accurately determining whether the noise originates from the ceiling joists or pipelines, thus improving troubleshooting accuracy. Simultaneously, this formula provides clear algorithmic logic for the information processing system software programming, reducing development difficulty, ensuring consistency of height calculation results across different devices, and enhancing the system's standardization.

[0057] This invention also provides a method for three-dimensional sound source localization based on a binocular acoustic camera, applicable to any of the above-described binocular acoustic camera-based three-dimensional sound source localization systems, such as... Figure 5 As shown, it includes the following steps:

[0058] S1, uses a first acoustic camera and a second acoustic camera to capture acoustic images of the same sound source from different locations and transmits them to the information processing system;

[0059] S2, through the information processing system, based on the relative positional relationship between the first acoustic camera and the second acoustic camera, the horizontal pixel path, the vertical pixel path, and the pixel coordinates of the sound source in the sound image of the first acoustic camera and the second acoustic camera, the position coordinates of the sound source in three-dimensional space are calculated.

[0060] This invention presents a complete "data acquisition-computation positioning" method. Dual cameras acquire acoustic images to provide raw data for subsequent calculations. The information processing system integrates multi-dimensional data to derive three-dimensional coordinates, achieving synergy and consistency with the overall system solution. This method does not rely on specific hardware models; it achieves three-dimensional positioning through standardized steps, exhibiting strong versatility and portability. For example, in adapting to different brands of acoustic cameras, simply deploying equipment and processing data according to this method achieves consistent positioning results. Simultaneously, it provides a basic framework for subsequent method refinement (such as deployment parameters and calculation steps), meeting the implementation needs of different scenarios.

[0061] In one embodiment, as Figure 6 As shown, the procedure before step S1 also includes:

[0062] S0, adjust the first acoustic camera and the second acoustic camera to the same height using the camera bracket, so that their front faces are perpendicular to the ground, and ensure that the angle between the frontal normals of the two cameras is between 30° and 120°.

[0063] This invention transforms the abstract process into a practical, on-site operational guide by clearly defining the adjustment and deployment parameters of the camera bracket. Workers can standardize equipment deployment by following the steps of "adjusting to the same height - ensuring verticality to the ground - controlling the angle range," avoiding deviations in audio-visual data caused by improper deployment. For example, in industrial plants, no specialized technicians are required; ordinary maintenance personnel can complete equipment deployment by following these steps, significantly reducing the labor costs and technical barriers to implementation. Simultaneously, it ensures consistent hardware deployment across different sites and by different personnel, providing a stable foundation for subsequent data acquisition and calculation.

[0064] Further, the step of calculating and determining the position coordinates of the sound source in three-dimensional space in the embodiment of the present invention includes: S211, based on the horizontal coordinate x1 of the sound source in the acoustic image of the first acoustic camera and the horizontal coordinate x2 of the sound source in the acoustic image of the second acoustic camera, and combined with the horizontal pixel paths of the two acoustic cameras, determining the distance L1 of the sound source from the first acoustic camera and the distance L2 from the second acoustic camera.

[0065] S212, based on the vertical coordinate y1 of the sound source in the acoustic image of the first acoustic camera, combined with the vertical pixel path of the first acoustic camera and the distance L1, the actual height h of the sound source is calculated. The actual height h of the sound source is calculated using the formula h = h0 + h1, where h0 is the height of the center of the acoustic camera from the ground, and h1 is the vertical distance of the sound source relative to the center pixel of the camera, calculated based on the vertical coordinate y1 and the vertical pixel path.

[0066] The above process first establishes a spatial coordinate system based on the horizontal pixel path of the dual cameras using a coordinate system construction module. This provides a unified benchmark for subsequent calculations, avoiding distance calculation deviations caused by a lack of spatial reference. Compared to the simplified method of directly associating pixel coordinates with spatial position, this step provides a clear mathematical basis for the transformation from "pixel data to spatial parameters," ensuring a coherent and seamless calculation logic from horizontal distance to vertical height, laying the foundation for overall positioning accuracy. In the distance calculation stage, L1 and L2 can be derived solely through the mapping relationship between the x1 and x2 coordinates and the coordinate system, avoiding interference from irrelevant parameters. In the height calculation stage, h1 is accurately derived by combining the vertical pixel path, L1, and y1, and finally, the actual height h is obtained by superimposing h0. This step-by-step focusing approach of "horizontal-vertical" reduces redundant calculations and provides accurate input for subsequent height calculations through the preceding result (L1), reducing error accumulation and making the calculation results of the three-dimensional coordinates (L1, L2, h) more reliable. Furthermore, by fusing multi-dimensional data such as horizontal pixel paths, vertical pixel paths, and dual-camera coordinates, and through coordinate system association and linkage of distance and height parameters, a mutually verifying computational closed loop is formed. In noisy industrial environments or scenarios where the sound source is far away, single data may contain errors, but the collaborative processing of multi-dimensional data can effectively offset interference, ensuring that even if a certain parameter has a slight deviation, a reliable result can still be derived through overall logical deduction, thus improving the system's stable positioning capability in complex environments.

[0067] To further refine the calculation logic of distance and height, and to achieve the conversion between pixel coordinates and actual distance, in this embodiment of the invention, when calculating distances L1 and L2, the information processing system converts pixel coordinates (x1, x2) into actual spatial horizontal distance parameters based on the mapping relationship of horizontal pixel paths in the spatial coordinate system. When calculating vertical distance h1, the information processing system combines the pixel distribution ratio corresponding to the vertical viewpoint of the first acoustic camera (1) to convert the pixel offset corresponding to pixel coordinate y1 into actual spatial vertical distance h1.

[0068] In the horizontal direction, x1 and x2 are transformed based on coordinate system mapping to ensure that the pixel offset accurately corresponds to the actual horizontal distance. In the vertical direction, y1 is transformed based on the pixel distribution ratio of the vertical viewpoint to avoid errors where the same pixel coordinate corresponds to different heights at different distances. For example, in long-distance sound source localization, the actual distance corresponding to the pixel offset can be accurately magnified through the mapping relationship; in short-distance localization, the vertical height calculation is refined based on the pixel distribution ratio to improve the resolution of small-range height differences. This design provides a clear technical basis for the calculation process, reduces subjective estimation errors, further improves the accuracy of 3D positioning, and meets the needs of scenarios with high positioning accuracy requirements (such as precision instrument fault detection).

[0069] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A sound source three-dimensional localization system based on a binocular acoustic camera, characterized in that, include: The first and second acoustic cameras are used to capture images of the same sound source from different locations to obtain acoustic images. The information processing system is communicatively connected to the first acoustic camera and the second acoustic camera. It is used to receive and process the acoustic image data collected by the two acoustic cameras. Based on the relative positional relationship between the first acoustic camera and the second acoustic camera, the horizontal pixel path, the vertical pixel path, and the pixel coordinates of the sound source in the acoustic image of the first acoustic camera and the second acoustic camera, it calculates the position coordinates of the sound source in three-dimensional space. Wherein, the horizontal pixel path of the first acoustic camera and the horizontal pixel path of the second acoustic camera are path diagrams formed by connecting points of the same horizontal coordinate at different distances of the acoustic camera, and their boundaries are the horizontal viewing angle of the acoustic camera; the vertical pixel path of the first acoustic camera and the vertical pixel path of the second acoustic camera are path diagrams formed by connecting points of the same vertical coordinate at different distances of the acoustic camera, and their boundaries are the vertical viewing angle of the acoustic camera.

2. The system according to claim 1, characterized in that, The position coordinates of the sound source in three-dimensional space include: the distance L1 from the sound source to the first acoustic camera, the distance L2 from the sound source to the second acoustic camera, and the actual height h from the ground.

3. The system according to claim 1 or 2, characterized in that, Also includes: A camera bracket for supporting the first acoustic camera and the second acoustic camera; the first acoustic camera and the second acoustic camera are set at the same height, such that the center of the two acoustic cameras is at a height h0 from the ground, and their front faces are perpendicular to the ground, with the included angle between the front normals of the two cameras being between 30° and 120°.

4. The system according to claim 1, characterized in that, The information processing system includes: A coordinate system construction module is used to construct a coordinate system based on the horizontal pixel path of the first acoustic camera and the horizontal pixel path of the second acoustic camera. The coordinate system takes the relative position of the two acoustic cameras as a reference and the trajectory of the horizontal pixel path as a reference axis. The sound source distance calculation module is used to determine the distance L1 from the sound source to the first acoustic camera and the distance L2 from the sound source to the second acoustic camera based on the horizontal coordinate x1 of the sound source in the acoustic image of the first acoustic camera and the horizontal coordinate x2 of the sound source in the acoustic image of the second acoustic camera. The sound source height calculation module is used to calculate the vertical distance h1 of the sound source relative to the center pixel of the first acoustic camera based on the vertical pixel path of the first acoustic camera and the distance L1, combined with the vertical coordinate y1 of the sound source in the acoustic image of the first acoustic camera, and to calculate the actual height h of the sound source based on the vertical distance h1 of the center pixel of the first acoustic camera and the height h0 of the center of the two acoustic cameras from the ground.

5. The system according to claim 1, characterized in that, The horizontal pixel path is a path map formed by connecting points of the same horizontal coordinate at different distances of the acoustic camera, and its boundary is the horizontal viewing angle of the acoustic camera; the vertical pixel path is a path map formed by connecting points of the same vertical coordinate at different distances of the acoustic camera, and its boundary is the vertical viewing angle of the acoustic camera.

6. The system according to claim 4, characterized in that, The actual height h of the sound source is calculated using the formula h = h0 + h1, where h0 is the height of the acoustic camera center from the ground, and h1 is the vertical distance of the sound source relative to the center pixel of the camera, calculated based on the vertical coordinate y1 and the vertical pixel path.

7. A method for three-dimensional sound source localization based on a binocular acoustic camera, characterized in that, include: Acoustic images of the same sound source are captured from different locations using a first acoustic camera and a second acoustic camera, and then transmitted to the information processing system. The information processing system calculates the position coordinates of the sound source in three-dimensional space based on the relative positional relationship between the first acoustic camera and the second acoustic camera, the horizontal pixel path, the vertical pixel path, and the pixel coordinates of the sound source in the sound image of the first acoustic camera and the second acoustic camera. Wherein, the horizontal pixel path of the first acoustic camera and the horizontal pixel path of the second acoustic camera are path diagrams formed by connecting points of the same horizontal coordinate at different distances of the acoustic camera, and their boundaries are the horizontal viewing angle of the acoustic camera; the vertical pixel path of the first acoustic camera and the vertical pixel path of the second acoustic camera are path diagrams formed by connecting points of the same vertical coordinate at different distances of the acoustic camera, and their boundaries are the vertical viewing angle of the acoustic camera.

8. The method according to claim 7, characterized in that, Before acquiring the acoustic images, the process also includes adjusting the first and second acoustic cameras to the same height using a camera bracket, ensuring that their front faces are perpendicular to the ground, and ensuring that the angle between the frontal normals of the two cameras is between 30° and 120°.

9. The method according to claim 7 or 8, characterized in that, The calculation of the position coordinates of the sound source in three-dimensional space includes: A spatial coordinate system is constructed based on the horizontal pixel path of the first acoustic camera and the horizontal pixel path of the second acoustic camera using the coordinate system construction module of the information processing system. The sound source distance calculation module determines the distance L1 from the first acoustic camera and the distance L2 from the second acoustic camera based on the pixel coordinates x1, x2 and the mapping relationship of the spatial coordinate system. The sound source height calculation module calculates the vertical distance h1 of the sound source relative to the center pixel based on the vertical pixel path, distance L1, and pixel coordinate y1 of the first acoustic camera, and calculates the actual height h of the sound source using the formula h = h0 + h1. Where x1 is the abscissa of the sound source in the acoustic image of the first acoustic camera, x2 is the abscissa of the sound source in the acoustic image of the second acoustic camera, y1 is the ordinate of the sound source in the acoustic image of the first acoustic camera, and h0 is the height of the center of the first acoustic camera and the second acoustic camera from the ground.

10. The method according to claim 8, characterized in that, When calculating distances L1 and L2, the information processing system converts pixel coordinates (x1, x2) into actual spatial horizontal distance parameters based on the mapping relationship of horizontal pixel paths in the spatial coordinate system. When calculating vertical distance h1, the information processing system combines the pixel distribution ratio corresponding to the vertical viewpoint of the first acoustic camera to convert the pixel offset corresponding to pixel coordinate y1 into actual spatial vertical distance h1. Wherein, L1 is the distance between the sound source and the first acoustic camera, and L2 is the distance between the sound source and the second acoustic camera; x1 is the horizontal coordinate of the sound source in the acoustic image of the first acoustic camera, and x2 is the horizontal coordinate of the sound source in the acoustic image of the second acoustic camera; h1 is the vertical distance of the sound source relative to the center pixel of the first acoustic camera; and y1 is the vertical coordinate of the sound source in the acoustic image of the first acoustic camera.

Images