Sound source positioning and intelligent tracking method and system based on multi-sensor array

Abstract

The invention discloses a sound source positioning and intelligent tracking method and system based on a multi-sensor array, and relates to the field of digital signal processing, and the sound source positioning and intelligent tracking method comprises the steps: system starting and environment signal collection; preprocessing and optimizing the signal; sound source positioning calculation; intelligent dynamic tracking; data transmission and real-time display; and data analysis and remote control. According to the invention, high-precision sound source localization is realized through the improved TDOA algorithm and deep learning optimization, and intelligent dynamic tracking of the sound source is ensured by combining fuzzy control and the PID algorithm optimized by the neural network; meanwhile, signal processing is optimized through technologies such as adaptive filtering and automatic gain control, and accurate input is provided; the system supports a multi-mode sensor array, and the tracking precision and adaptability are enhanced in combination with sound source signals and environmental parameters; and the upper computer has remote control and data analysis functions, so that the operability and the data management capability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of digital signal processing, and in particular to a method and system for sound source localization and intelligent tracking based on a multi-sensor array. Background Art

[0002] As an important medium for human-computer interaction, sound signals are widely used in many fields. Sound source localization technology originated in the war period. It was originally used to detect the position of artillery and then gradually expanded to the civilian field. In the 1970s and 1980s, the introduction of microphone arrays promoted the research of speech signal processing and sound source localization. In the 1990s, related processing algorithms became a research hotspot and promoted technological breakthroughs. In recent years, deep learning technology has been applied to the feature extraction and localization of sound signals, and has made significant progress, especially in fusing multi-sensor information to improve robustness and accuracy. However, sound source localization in complex environments still faces challenges in accuracy and stability.

[0003] In China, sound source localization technology has also achieved rapid development. Common methods include positioning methods based on time delay difference of arrival (TDOA), beamforming and high-resolution spatial spectrum estimation. The TDOA method is widely used due to its low computational complexity and high real-time performance, but its effect is limited in strong noise environments. The beamforming method enhances the signal in a specific direction by adjusting the sensor weights, but it requires high computing resources, and the array layout has a great impact on the accuracy. High-resolution spatial spectrum estimation methods such as the MUSIC algorithm achieve high-precision estimation of the sound source position by decomposing the signal and noise subspaces.

[0004] With the rapid development of the Internet of Things and embedded technology, the market demand for lightweight, low-power sound source localization systems with high-precision real-time tracking capabilities has increased. However, existing sound source localization technologies face multiple challenges in complex environments, including low positioning accuracy in noisy environments, high computing resource requirements, a large impact of sensor array layout on accuracy, insufficient real-time and stability, and poor system adaptability. Although multi-sensor information fusion and different positioning algorithms have made some progress, existing technologies still cannot meet the needs of high-precision, real-time tracking in resource-constrained embedded devices, especially in dynamically changing environments. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a sound source localization and intelligent tracking method and system based on a multi-sensor array, which solves the technical problems of the existing sound source localization technology in complex environments, such as insufficient accuracy, noise resistance, real-time performance and adaptability, and is difficult to meet the high-precision real-time tracking requirements of embedded devices.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention provides a sound source localization and intelligent tracking method based on a multi-sensor array, comprising the following steps:

[0008] S1, start the microcontroller, activate the multi-mode sensor array module, and collect environmental sound source signals and related parameters, including noise intensity and environmental characteristics, through the integrated multi-sensor array;

[0009] S2, processing the collected sound source signal, including using an adaptive low-pass filter to dynamically remove high-frequency noise, combining an automatic gain control function to optimize the signal strength, and outputting the optimized signal through an integrated module;

[0010] S3, using the improved TDOA algorithm to calculate the time difference of the sound source reaching each sensor, and determine the relative position of the sound source through the time difference between multiple sensors;

[0011] S4, adjust the pan / tilt rotation angle in real time according to the sound source position, and achieve high-precision dynamic tracking through fuzzy control and neural network optimized PID algorithm to ensure that the target is always within the optimal capture range of the sensor array;

[0012] S5, transmitting the sound source location information and environmental data to the host computer through the wireless transmission module, and displaying the sound field distribution map in real time;

[0013] S6. After receiving the data, the host computer provides remote control functions based on data analysis, including visualization of historical tracking paths and multi-terminal interactive operations.

[0014] As a further solution of the present invention, the specific method steps of step S1 are:

[0015] S21, pre-processing the collected sound source signal, using an adaptive low-pass filter to remove high-frequency noise in the environment, and dynamically adjusting the filter bandwidth to adapt to changes in the spectrum characteristics of the environmental noise;

[0016] S22, optimizing signal strength through an amplifier circuit having an automatic gain control function;

[0017] S23, integrating the filtering circuit and the amplifying circuit into the same module to improve signal processing efficiency;

[0018] S24: Output the optimized sound source signal to provide accurate input for subsequent positioning algorithms.

[0019] As a further solution of the present invention, the improved TDOA algorithm in step S3 includes:

[0020] Based on the arrival time difference of the sound source signal received by the multi-sensor array and the geometric distribution of the multi-sensor, the least square method is used for positioning;

[0021] The sound source position is optimized by combining the sound source correction model based on deep learning;

[0022] In order to deal with noise interference and multipath effects, the calculation accuracy is improved through adaptive weight assignment.

[0023] As a further solution of the present invention, the PID control algorithm in step S4 includes:

[0024] Dynamically adjust the pan / tilt rotation angle based on the sound source localization information;

[0025] Combined with fuzzy control algorithm, PID parameters are adjusted in real time to optimize the dynamic response performance of the PTZ.

[0026] The multi-degree-of-freedom adjustment mechanism enables high-precision positioning and smooth movement of the gimbal in three-dimensional space.

[0027] As a further solution of the present invention, the host computer in step S6 also provides the following remote control function:

[0028] Adjust gimbal motion parameters, including response speed, angle range, and tracking mode;

[0029] Dynamically set the coverage area of ​​sound source tracking based on real-time environmental data and optimize tracking path planning;

[0030] Provides visual analysis of historical data, including sound source trajectory, signal strength changes, and system response time statistics.

[0031] The present invention also provides a sound source localization and intelligent tracking system based on a multi-sensor array, which is used to implement any of the above-mentioned sound source localization and intelligent tracking methods based on a multi-sensor array, comprising:

[0032] Multimodal sensor array for simultaneously collecting sound source signals and environmental parameters;

[0033] A microcontroller, used for preprocessing, feature extraction and sound source localization of the collected signals;

[0034] Intelligent tracking module, used to receive sound source positioning data, generate dynamic tracking path based on PID algorithm optimized by fuzzy control and neural network, and send control signals to the PTZ control module;

[0035] The PTZ control module is used to adjust the PTZ angle according to the positioning information to achieve sound source tracking;

[0036] Wireless communication module, used to transmit the sound source location information and environmental data to the host computer;

[0037] The host computer is equipped with a visual interface and remote control function, which is used to display the sound source location information, analyze data and perform system management.

[0038] As a further solution of the present invention, the multimodal sensor array comprises:

[0039] An evenly distributed microphone array is used to receive the sound source signal and use it for time difference calculation to determine the location of the sound source;

[0040] Noise intensity sensor, used to monitor the ambient noise level in real time;

[0041] Temperature and humidity sensors are used to provide environmental information to assist in signal processing and system parameter optimization.

[0042] As a further solution of the present invention, the wireless transmission module includes:

[0043] ESP8266 Wi-Fi module, used to transmit the sound source location information to the host computer via wireless LAN;

[0044] Bluetooth module, used to achieve short-distance real-time data interaction;

[0045] ZigBee module is used to transmit the location information of sound sources in a low-power environment.

[0046] As a further solution of the present invention, the microcontroller adopts an STM32F103 chip, which has a built-in ADC module and a DSP acceleration unit for fast signal processing.

[0047] As a further solution of the present invention, the intelligent tracking module includes:

[0048] An algorithm unit, used for dynamically generating a tracking path based on the sound source localization data;

[0049] The control unit is used to output control signals to the pan / tilt control module to ensure that the pan / tilt angle adjustment is synchronized with the change of the sound source position.

[0050] Compared with the prior art, the beneficial technical effects of the present invention are:

[0051] 1. The method of the present invention adopts an improved TDOA algorithm combined with a deep learning auxiliary model, which can determine the location of the sound source with high precision, reduce the influence of noise interference and multipath effect, and provide more reliable positioning data. Compared with traditional sound source localization technology, this method can more effectively improve positioning accuracy;

[0052] 2. The method of the present invention combines fuzzy control and neural network optimized PID algorithm. The intelligent tracking module can adjust the pan-tilt rotation angle according to the real-time sound source position to ensure that the target is always within the optimal capture range. The dynamic tracking capability enables the system to cope with complex environmental changes and real-time tracking needs.

[0053] 3. The method of the present invention optimizes the sound source signal by using an adaptive low-pass filter, automatic gain control and a signal integration module, which can effectively remove noise and enhance signal strength, thereby providing more accurate data support for subsequent positioning and tracking;

[0054] 4. The system of the present invention can simultaneously collect sound source signals, environmental noise and other environmental parameters (such as temperature and humidity) through a multi-modal sensor array, so as to better perform signal processing and system optimization in a complex environment. By increasing the types of sensor arrays, the adaptability and robustness of the system are improved;

[0055] 5. The host computer of the system of the present invention provides a visual interface and remote control functions, allowing users to monitor the sound source location information and environmental data in real time, and optimize the tracking path and dynamically adjust the pan-tilt motion parameters through data analysis functions, and support multi-terminal interactive operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flow chart of the steps of the sound source localization and intelligent tracking method of the present invention;

[0057] Figure 2 It is a framework diagram of the sound source localization and intelligent tracking system of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the implementation cases described are only part of the implementation content of the present invention, not all. Based on the implementation content of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0059] Embodiment 1

[0060] See also Figure 1 In an embodiment of the present invention, a method for sound source localization and intelligent tracking based on a multi-sensor array is provided, comprising the following steps:

[0061] S1, start the microcontroller, activate the multi-mode sensor array module, and collect environmental sound source signals and related parameters, including noise intensity and environmental characteristics, through the integrated multi-sensor array;

[0062] S2, processing the collected sound source signal, including using an adaptive low-pass filter to dynamically remove high-frequency noise, combining an automatic gain control function to optimize the signal strength, and outputting the optimized signal through an integrated module;

[0063] S3, using the improved TDOA algorithm to calculate the time difference of the sound source reaching each sensor, and determine the relative position of the sound source through the time difference between multiple sensors;

[0064] S4, adjust the pan / tilt rotation angle in real time according to the sound source position, and achieve high-precision dynamic tracking through fuzzy control and neural network optimized PID algorithm to ensure that the target is always within the optimal capture range of the sensor array;

[0065] S5, transmitting the sound source location information and environmental data to the host computer through the wireless transmission module, and displaying the sound field distribution map in real time;

[0066] S6. After receiving the data, the host computer provides remote control functions based on data analysis, including visualization of historical tracking paths and multi-terminal interactive operations.

[0067] Specifically, the specific method steps of step S2 are:

[0068] S21, pre-processing the collected sound source signal, using an adaptive low-pass filter to remove high-frequency noise in the environment, and dynamically adjusting the filter bandwidth to adapt to changes in the spectrum characteristics of the environmental noise;

[0069] S22, optimizing signal strength through an amplifier circuit having an automatic gain control function;

[0070] S23, integrating the filtering circuit and the amplifying circuit into the same module to improve signal processing efficiency;

[0071] S24: Output the optimized sound source signal to provide accurate input for subsequent positioning algorithms.

[0072] Specifically, the improved TDOA algorithm in step S3 includes:

[0073] Based on the arrival time difference of the sound source signals received by the multi-sensor array and the geometric distribution of multiple sensors, the least squares method is used for positioning; the sound source position is optimized in combination with the sound source correction model based on deep learning; in view of noise interference and multipath effects, the calculation accuracy is improved through adaptive weight assignment, especially under the interference of noise and multipath effects, it can still work stably.

[0074] Specifically, the PID control algorithm in step S4 includes:

[0075] The pan-tilt rotation angle is dynamically adjusted based on the sound source positioning information; the PID parameters are adjusted in real time in combination with the fuzzy control algorithm to optimize the dynamic response performance of the pan-tilt; and the high-precision positioning and smooth movement of the pan-tilt in three-dimensional space are achieved through the multi-degree-of-freedom adjustment mechanism. The combination of these two control strategies improves the accuracy and stability of pan-tilt tracking, can adapt to different environmental changes, and ensure the efficiency of dynamic tracking.

[0076] Specifically, in step S6, the host computer also provides the following remote control functions:

[0077] Adjust the gimbal motion parameters, including response speed, angle range and tracking mode; dynamically set the coverage area of ​​sound source tracking according to real-time environmental data, and optimize the tracking path planning; provide visual analysis function of historical data, including sound source trajectory, signal strength changes and system response time statistics.

[0078] Embodiment 2

[0079] See also Figure 2 In an embodiment of the present invention, a sound source localization and intelligent tracking system based on a multi-sensor array is provided, which is used to implement a sound source localization and intelligent tracking method as described above, and the system includes:

[0080] Multimodal sensor array for simultaneously collecting sound source signals and environmental parameters;

[0081] A microcontroller, used for preprocessing, feature extraction and sound source localization of the collected signals;

[0082] Intelligent tracking module, used to receive sound source positioning data, generate dynamic tracking path based on PID algorithm optimized by fuzzy control and neural network, and send control signals to the PTZ control module;

[0083] The PTZ control module is used to adjust the PTZ angle according to the positioning information to achieve sound source tracking;

[0084] Wireless communication module, used to transmit the sound source location information and environmental data to the host computer;

[0085] The host computer is equipped with a visual interface and remote control function, which is used to display the sound source location information, analyze data and perform system management.

[0086] Specifically, the multimodal sensor array includes:

[0087] The evenly distributed microphone array is used to receive the sound source signal and calculate the time difference to determine the location of the sound source; the noise intensity sensor is used to monitor the ambient noise level in real time; the temperature and humidity sensor is used to provide environmental information to assist signal processing and system parameter optimization. Through the multi-modal sensor array, the solution can not only accurately locate the sound source, but also monitor the ambient noise intensity and environmental parameters such as temperature and humidity in real time, providing more comprehensive data support.

[0088] Specifically, the wireless transmission module includes:

[0089] The ESP8266 Wi-Fi module is used to transmit the sound source location information to the host computer through the wireless LAN; the Bluetooth module is used to achieve short-distance real-time data interaction; the ZigBee module is used to achieve the transmission of sound source location information in a low-power environment. A variety of data transmission methods are provided to meet the needs of different scenarios, including LAN transmission, short-distance real-time interaction, and low-power transmission.

[0090] Specifically, the microcontroller uses the STM32F103 chip, which has a built-in ADC module and a DSP acceleration unit for fast signal processing to ensure the real-time performance and accuracy of the system. The built-in 12-bit high-precision ADC can directly sample analog signals (such as the sound source signal output by the sensor) with high precision and convert them into digital signals, reducing the need for external ADC modules, improving system integration, and reducing costs and space occupancy. The built-in DSP acceleration unit significantly improves signal processing efficiency and speed, reduces power consumption, and enhances real-time performance, making it very suitable for applications with efficient and real-time data processing, such as sound source localization and intelligent tracking.

[0091] Specifically, the intelligent tracking module includes: an algorithm unit for dynamically generating a tracking path based on the sound source positioning data; a control unit for outputting a control signal to the pan / tilt control module to ensure that the pan / tilt angle adjustment is synchronized with the change in the sound source position. The key to this module is to use fuzzy control and neural network to optimize the PID algorithm to ensure high-precision dynamic tracking of the pan / tilt in three-dimensional space.

[0092] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A sound source localization and intelligent tracking method based on a multi-sensor array, characterized in that: The steps include: S1, start the microcontroller, activate the multi-mode sensor array module, and collect environmental sound source signals and related parameters, including noise intensity and environmental characteristics, through the integrated multi-sensor array; S2, processing the collected sound source signal, including using an adaptive low-pass filter to dynamically remove high-frequency noise, combining an automatic gain control function to optimize the signal strength, and outputting the optimized signal through an integrated module; S3, using the improved TDOA algorithm to calculate the time difference of the sound source reaching each sensor, and determine the relative position of the sound source through the time difference between multiple sensors; S4, adjust the pan / tilt rotation angle in real time according to the sound source position, and achieve high-precision dynamic tracking through fuzzy control and neural network optimized PID algorithm to ensure that the target is always within the optimal capture range of the sensor array; S5, transmitting the sound source location information and environmental data to the host computer through the wireless transmission module, and displaying the sound field distribution map in real time; S6. After receiving the data, the host computer provides remote control functions based on data analysis, including visualization of historical tracking paths and multi-terminal interactive operations.

2. A method for sound source localization and intelligent tracking based on a multi-sensor array according to claim 1, characterized in that: The specific method steps of step S2 are: S21, pre-processing the collected sound source signal, using an adaptive low-pass filter to remove high-frequency noise in the environment, and dynamically adjusting the filter bandwidth to adapt to changes in the spectrum characteristics of the environmental noise; S22, optimizing signal strength through an amplifier circuit having an automatic gain control function; S23, integrating the filtering circuit and the amplifying circuit into the same module to improve signal processing efficiency; S24: Output the optimized sound source signal to provide accurate input for subsequent positioning algorithms.

3. The method for sound source localization and intelligent tracking based on a multi-sensor array according to claim 1, characterized in that: The improved TDOA algorithm in step S3 includes: Based on the arrival time difference of the sound source signal received by the multi-sensor array and the geometric distribution of the multi-sensor, the least square method is used for positioning; The sound source position is optimized by combining the sound source correction model based on deep learning; In order to deal with noise interference and multipath effects, the calculation accuracy is improved through adaptive weight assignment.

4. The method for sound source localization and intelligent tracking based on a multi-sensor array according to claim 1, characterized in that: The PID control algorithm in step S4 includes: Dynamically adjust the pan / tilt rotation angle based on the sound source localization information; Combined with fuzzy control algorithm, PID parameters are adjusted in real time to optimize the dynamic response performance of the PTZ. The multi-degree-of-freedom adjustment mechanism enables high-precision positioning and smooth movement of the gimbal in three-dimensional space.

5. The method for sound source localization and intelligent tracking based on a multi-sensor array according to claim 1, characterized in that: In step S6, the host computer also provides the following remote control functions: Adjust gimbal motion parameters, including response speed, angle range, and tracking mode; Dynamically set the coverage area of ​​sound source tracking based on real-time environmental data and optimize tracking path planning; Provides visual analysis of historical data, including sound source trajectory, signal strength changes, and system response time statistics.

6. A sound source localization and intelligent tracking system based on a multi-sensor array, used to implement a sound source localization and intelligent tracking method based on a multi-sensor array as described in any one of claims 1 to 5, characterized in that: include: Multimodal sensor array for simultaneously collecting sound source signals and environmental parameters; A microcontroller, used for preprocessing, feature extraction and sound source localization of the collected signals; Intelligent tracking module, used to receive sound source positioning data, generate dynamic tracking path based on PID algorithm optimized by fuzzy control and neural network, and send control signals to the PTZ control module; The PTZ control module is used to adjust the PTZ angle according to the positioning information to achieve sound source tracking; Wireless communication module, used to transmit the sound source location information and environmental data to the host computer; The host computer is equipped with a visual interface and remote control function, which is used to display the sound source location information, analyze data and perform system management.

7. A sound source localization and intelligent tracking system based on a multi-sensor array according to claim 6, characterized in that: The multimodal sensor array comprises: An evenly distributed microphone array is used to receive the sound source signal and use it for time difference calculation to determine the location of the sound source; Noise intensity sensor, used to monitor the ambient noise level in real time; Temperature and humidity sensors are used to provide environmental information to assist in signal processing and system parameter optimization.

8. The sound source localization and intelligent tracking system based on a multi-sensor array according to claim 6, characterized in that: The wireless transmission module comprises: ESP8266 Wi-Fi module, used to transmit the sound source location information to the host computer via wireless LAN; Bluetooth module, used to achieve short-distance real-time data interaction; ZigBee module is used to transmit the location information of sound sources in a low-power environment.

9. The sound source localization and intelligent tracking system based on a multi-sensor array according to claim 6, characterized in that: The microcontroller adopts an STM32F103 chip and has a built-in ADC module and a DSP acceleration unit for fast signal processing.

10. The sound source localization and intelligent tracking system based on a multi-sensor array according to claim 6, characterized in that: The intelligent tracking module includes: An algorithm unit, used for dynamically generating a tracking path based on the sound source localization data; The control unit is used to output control signals to the pan / tilt control module to ensure that the pan / tilt angle adjustment is synchronized with the change of the sound source position.

Images