Complex environment noise patrol, sound source positioning and tracking system and method based on multi-sensing information fusion

Abstract

This invention discloses a method and system for noise inspection, sound source localization, and tracking in complex environments based on multi-sensor information fusion, belonging to the fields of environmental noise monitoring and the Internet of Things (IoT). The system, with a main control and data processing module at its core, integrates units such as a sound information acquisition module, a sound source localization module, a motion control and drive module, a precise positioning module, and a data processing and analysis center. The method achieves autonomous inspection, real-time precise localization, and dynamic tracking of noise sources in complex environments through a closed-loop process of "initialization – synchronous acquisition – exceeding limits judgment – ​​localization and tracking – coordinate calibration – evidence retention – cloud analysis." This invention breaks through the coverage bottleneck of traditional fixed monitoring, realizes the fusion calibration of acoustic orientation and absolute geographic coordinates, constructs a multi-modal data fusion-based noise event tracing system, and possesses excellent scenario adaptability and functional scalability, providing reliable technical support for intelligent noise pollution prevention and control.

Description

Technical Field

[0001] This invention belongs to the field of environmental noise monitoring and Internet of Things (IoT) technology, and particularly relates to a noise source dynamic monitoring, positioning, and data tracing system based on a mobile platform and its working method. Specifically, this invention provides an intelligent noise inspection system that integrates sound source direction perception, autonomous movement tracking, geographic coordinate calibration, audio data acquisition, and cloud data synchronization, as well as a method for realizing the integrated workflow of "detection-positioning-tracking-recording-uploading" of this system. Background Technology

[0002] With the acceleration of urbanization and industrialization, noise pollution has become a key environmental issue affecting public health and quality of life. In semi-open or complex terrain scenarios such as school campuses, communities, industrial parks, mountainous and hilly areas, and urban canyons, noise sources are characterized by intermittency, mobility, and wide spatial distribution, posing a severe challenge to traditional noise monitoring methods.

[0003] Current mainstream noise monitoring methods are divided into two categories: manual inspection and fixed monitoring equipment. Both have significant shortcomings: manual inspection relies on subjective judgment, has inconsistent monitoring standards, limited coverage, and cannot achieve 24-hour continuous monitoring. In complex terrain, it suffers from insufficient accessibility and high operational risks, and it is even more difficult to quickly respond to and track sudden or moving noise sources. Although fixed monitoring equipment can monitor continuously, its fixed deployment location leads to a limited monitoring range and blind spots. In complex scenarios with large terrain undulations and dense obstructions, the effective monitoring range shrinks significantly and the number of blind spots increases dramatically. It cannot meet the comprehensive monitoring needs of dynamic scenarios and is difficult to achieve precise control of noise pollution.

[0004] Although sound source localization and mobile robot technology have made great strides, and domestic and foreign research has expanded to the field of environmental monitoring using civilian robots and drones equipped with microphone arrays, and progress has been made in sound source localization algorithms and deep learning voiceprint recognition, existing technical solutions mostly focus on single sound source localization or fixed point monitoring, and have not yet formed a comprehensive intelligent solution that deeply integrates functions such as sound source localization, autonomous mobile platform, geographic information calibration, IoT data synchronization and audio source tracing.

[0005] Existing technologies generally suffer from problems such as limited functionality, isolated data, lack of traceability capabilities, and low system integration: it is difficult to achieve deep integration of sound source localization, geographic coordinate transformation, and autonomous navigation; satellite positioning signals are easily blocked in complex terrain, and pose calculation is prone to cumulative errors, further leading to inaccurate matching between acoustic orientation and geographic coordinates; it is impossible to achieve real-time cloud synchronization and remote management of monitoring data; it cannot synchronously store original audio and video data synchronized with location and time to support post-event analysis and voiceprint tracing, and cannot form compliant and effective evidence for law enforcement; and the lack of collaborative optimization between multiple sensors and core controllers results in the need to improve system real-time performance, power consumption, and reliability, making it difficult to meet the noise monitoring needs in complex dynamic environments.

[0006] Therefore, there is an urgent need for an intelligent noise monitoring system that integrates autonomous mobility, real-time sound source localization, precise geographical location recording, synchronous audio evidence collection, and instant upload and management of IoT data. This system would fill the technological gap in dynamic environmental noise monitoring and control, and provide technical support for building a smart and precise noise pollution prevention and control system. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for complex environmental noise inspection, sound source localization and tracking based on multi-sensor information fusion, specifically addressing the following core technical problems existing in the prior art:

[0008] 1. Traditional fixed monitoring equipment has a statically limited coverage area and monitoring blind spots, while manual inspection has insufficient accessibility and cannot provide continuous monitoring. Neither of these two solutions can achieve dynamic tracking and rapid response to mobile and intermittent noise sources with variable propagation paths in complex terrain.

[0009] 2. Existing sound source localization schemes can only calculate the relative orientation of the sound source, and cannot achieve the fusion and calibration of acoustic orientation information and absolute geographic coordinates. In complex terrain, the localization matching is inaccurate, and the localization results cannot directly support the accurate implementation of noise pollution control.

[0010] 3. The monitoring data is isolated and scattered, lacking original audio and video evidence synchronized with the location and time, making it impossible to achieve complete source tracing and post-event analysis of noise events, and even more impossible to form compliant and effective evidence for law enforcement.

[0011] 4. The system has limited functionality and low integration, failing to form a complete control loop of "perception-location-tracking-recording-uploading-analysis". It cannot adapt to the multi-scenario adaptation, multi-sensor linkage, and remote dispatch and control needs of noise inspection in complex terrain.

[0012] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0013] First, the present invention provides a complex environmental noise inspection, sound source localization and tracking system based on multi-sensor information fusion, including a main control and data processing module, and a sound information acquisition module, a sound source localization module, a motion control and drive module, a precise positioning module and a data communication and uploading module respectively connected to the main control and data processing module. It also includes a data processing and analysis center, a visual recognition module and a storage module that are bidirectionally connected to the main control and data processing module.

[0014] The sound information acquisition module is used to acquire noise signals from the target environment in real time, quantify the noise intensity, and send the acquired raw noise signals and noise intensity data to the data processing and analysis center. The sound source localization module is used to calculate the azimuth angle information of the noise source relative to the inspection vehicle in real time and send the calculated sound source azimuth information to the data processing and analysis center. The precise positioning module is used to obtain the absolute geographic coordinates of the current position of the inspection vehicle in real time and transmit the coordinate data back to the data processing and analysis center in real time. The visual recognition module is used to acquire path images during the inspection vehicle's tracking of the sound source. The system identifies obstacles and generates obstacle avoidance commands, and collects on-site environmental video data after the inspection vehicle reaches the sound source location; the motion control and drive module is used to receive control commands from the main control and data processing module, drive the inspection vehicle to perform corresponding movement actions, and complete preset path inspection or sound source directional tracking; the storage module is used to store original audio files, on-site video files, positioning data logs, and complete noise event data packets; the data communication and upload module is used to receive the full noise event data summarized by the main control and data processing module, and upload the data to a remote IoT cloud platform in real time to realize remote management and source tracing analysis of noise data.

[0015] The data processing and analysis center incorporates a multi-source data preprocessing unit, a time-frequency conversion and feature extraction unit, a sound source precise localization unit, a tracking path planning unit, and a multi-modal data fusion unit. Specifically: the multi-source data preprocessing unit filters and reduces noise in the original noise signal, and aligns the noise signal's timestamp with the coordinate data of the precise localization module and the carrier position data of the motion control and drive module to ensure the spatiotemporal synchronization of the multi-source data; the time-frequency conversion and feature extraction unit performs time-frequency conversion and feature extraction on the preprocessed noise signal, compares the noise intensity with a preset threshold to determine if it exceeds the limit, and triggers the sound source localization and tracking process; the sound source precise localization unit is used to combine... The acoustic location information and the geographic coordinate data from the precise positioning module are used to convert the acoustic location information into the absolute geographic coordinates of the noise source, thereby achieving precise spatial positioning of the noise source. The tracking path planning unit is used to plan the optimal tracking route based on the real-time calculated absolute geographic coordinates of the noise source, through the direction-action mapping rules and real-time feedback control logic pre-set within the unit, and generate steering and speed control commands and send them to the main control and data processing module. The multimodal data fusion unit is used to fuse acoustic feature data, geographic coordinate data, on-site video data collected by the visual recognition module, and raw audio data to generate a complete noise event data package containing the event occurrence time, precise coordinates, noise intensity, acoustic features, and audio-visual evidence.

[0016] The main control and data processing module serves as the core control unit of the system, receiving control commands issued by the data processing and analysis center, scheduling the collaborative work of various modules, and completing the entire process control of noise event detection, location, tracking, recording, and uploading.

[0017] Furthermore, the present invention also provides a method for noise inspection, sound source localization and tracking in complex environments based on multi-sensor information fusion, implemented based on the above system, including the following steps:

[0018] Step S1: System deployment and initialization. The inspection carrier equipped with the system is deployed in the target monitoring area. After the system is powered on, the self-test of each module is completed, the communication link is established, and the initial position information of the inspection carrier is obtained and reported.

[0019] Step S2: Real-time data acquisition and synchronization. After the system enters the inspection mode, it continuously collects environmental noise signals through the sound information acquisition module, calculates the location information of the noise source in real time through the sound source localization module, and synchronously obtains the real-time geographical coordinates of the inspection vehicle to complete the timestamp alignment and calibration of multi-source data.

[0020] Step S3: Data preprocessing and noise exceedance judgment. The collected raw noise signal is preprocessed to eliminate environmental interference, the noise intensity value is quantified and calculated, and the noise intensity value is compared with the preset threshold to determine whether the sound source localization and tracking process is triggered.

[0021] Step S4: Sound source localization and autonomous tracking. When the noise intensity is determined to exceed the standard, the system drives the inspection vehicle to autonomously adjust its attitude and travel route according to the real-time calculated sound source location information, and to face and approach the noise source. During the tracking process, the sound source location is updated in real time and the travel strategy is adjusted.

[0022] Step S5: Precise positioning and coordinate upload. When the inspection vehicle approaches the sound source to a preset distance, or when the noise intensity continues to exceed the standard for a preset duration, the inspection vehicle is controlled to stop moving. The precise geographic coordinates of the current location are obtained as noise source location calibration data, and the coordinates and noise intensity data are uploaded to the IoT cloud platform in real time.

[0023] Step S6: Preserve on-site evidence. Simultaneously start audio recording and video acquisition at the location point to generate audio and video evidence files that are spatiotemporally synchronized with the coordinates of the noise source, associate them, store them, and upload them to the cloud platform.

[0024] Step S7: Data aggregation and analysis in the cloud. The cloud platform receives and integrates all noise event data and audio-visual evidence, generates noise event logs, and allows managers to remotely view and trace the source of the noise.

[0025] Compared with the prior art, the present invention has the following outstanding substantive features and significant progress:

[0026] 1. It achieves dynamic and proactive noise monitoring, breaking through the coverage bottleneck of traditional monitoring. Through mobile autonomous inspection, it overcomes the limitations of the static and limited coverage of traditional fixed monitoring equipment, and can proactively detect and track mobile or intermittent noise sources, significantly expanding the effective monitoring area. It also solves the problems of insufficient accessibility and numerous blind spots in manual inspections in complex terrain, improving the flexibility and proactivity of monitoring.

[0027] 2. Improved sound source localization accuracy and response efficiency, achieving a complete closed loop from direction perception to coordinate calibration. By integrating acoustic orientation positioning and multi-mode satellite absolute positioning technologies, the system first rapidly guides the carrier toward the sound source using acoustic methods, and then obtains accurate geographic coordinates through a precise positioning module. This solves the problem of mismatch between acoustic orientation and geographic coordinates in existing technologies, resulting in fast positioning response and accurate results, providing a clear target location for noise control.

[0028] 3. Multimodal information fusion and full-process closed-loop management have been completed, enabling comprehensive noise event tracing capabilities. Through spatiotemporal synchronous calibration of multi-source data, multi-dimensional information such as sound intensity, sound source direction, geographical coordinates, and on-site audio and video has been integrated to generate a uniquely bound tracing data package for noise events. Real-time data uploading, visualization, and centralized management have been achieved through an IoT platform, forming a complete data closed loop of "perception-location-tracking-recording-uploading-analysis," providing comprehensive, three-dimensional, and compliant data support for noise governance decision-making and law enforcement evidence collection.

[0029] 4. Excellent applicability and scalability. This system solution is not only suitable for campus scenarios, but can also be extended to noise monitoring scenarios in semi-open or complex environments such as communities, industrial parks, mountainous and hilly areas, and urban canyons. The modular design facilitates the subsequent integration of other sensors such as air quality monitoring sensors, or the introduction of AI voiceprint recognition algorithms. The system has strong scalability and can adapt to noise monitoring needs in multiple scenarios. Attached Figure Description

[0030] Figure 1 This is a block diagram of the overall architecture of the complex environmental noise inspection, sound source localization and tracking system based on multi-sensor information fusion as described in this invention;

[0031] Figure 2 This is a schematic diagram of the hardware circuit connection of the system described in this invention;

[0032] Figure 3 This is a flowchart illustrating the method for complex environmental noise inspection, sound source localization, and tracking based on multi-sensor information fusion as described in this invention.

[0033] Figure 4 This is a control logic block diagram for sound source localization and autonomous tracking in this invention. Detailed Implementation

[0034] The present invention will now be described in further detail and in complete detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0035] This specific implementation provides a complete scheme for a complex environmental noise inspection, sound source localization and tracking system based on multi-sensor information fusion. The system uses a wheeled noise inspection vehicle as the inspection carrier and is designed for noise monitoring scenarios in semi-open places such as campuses and communities. It can realize the full-process functions of autonomous inspection of noise sources, real-time positioning, dynamic tracking, coordinate calibration, audio evidence collection and cloud data management.

[0036] I. System Overall Architecture

[0037] This system uses the ESP32S3 main controller as the core control unit and constructs a complete closed-loop architecture of "perception-positioning-control-transmission-analysis". It includes a main control and data processing module, as well as a sound information acquisition module, a sound source positioning module, a motion control and drive module, a precise positioning module, a data communication and uploading module, a data processing and analysis center, a visual recognition module, a storage module, and a power supply module, which are respectively connected to the main control and data processing modules.

[0038] The system comprises several modules: a sound information acquisition module and a sound source localization module, which together form the acoustic sensing unit responsible for acquiring environmental noise signals and calculating the direction of sound sources; a motion control and drive module responsible for enabling automatic inspection of the vehicle and directional tracking of sound sources; a precise positioning module responsible for obtaining the absolute geographic coordinates of the vehicle and the noise source; a data communication and upload module responsible for enabling bidirectional communication between the system and the IoT cloud platform; a storage module responsible for locally storing the original audio and video data and location data of noise events; a power supply module providing stable power to the entire system; and a data processing and analysis center built into the main control and data processing modules, responsible for the fusion processing of multi-source data, algorithm execution, and control command generation. The overall system architecture is as follows: Figure 1 As shown, the hardware circuit connection is as follows: Figure 2 As shown.

[0039] II. Detailed Implementation of Core Hardware Modules

[0040] (I) Main Control and Data Processing Module

[0041] In this embodiment, the main control and data processing module adopts the ESP32S3 main controller, which is equipped with a 240MHz dual-core Xtensa LX7 processor, 512KB SRAM, 4MB flash memory, 45 digital I / O pins, 21 channels of 12-bit resolution analog input pins, and integrates 3 UART interfaces, 2 I2C interfaces, 3 SPI interfaces, 2 I2S interfaces, and a built-in 2.4GHz WiFi and Bluetooth 5 LE wireless module.

[0042] The core performance of this main controller is well-suited to the complex task requirements of this system: 8 independent PWM outputs enable parallel motor speed control; dual I2S interfaces can stably process 44.1kHz audio stream data; an 80MHz high-speed SPI bus meets the high-speed storage requirements of SD card audio files; a built-in WiFi module supports the TCP / IP protocol stack, enabling real-time cloud upload of positioning and noise data; rich peripheral interfaces are compatible with all external sensor modules, reducing the complexity of the system's peripheral circuitry; and the overall power consumption is less than 800mW when the system is running at full load. This embodiment uses the Arduino IDE as the development environment, which supports Windows, Mac OS X, and Linux operating systems, is compatible with the C / C++ programming language, and has a rich built-in peripheral driver library, enabling full-function development of the ESP32S3 main controller.

[0043] (II) Motion Control and Drive Module

[0044] This module is used to realize the automatic inspection, sound source directional tracking and obstacle avoidance functions of the inspection car. It consists of three parts: L298N motor drive unit, 4 DC geared motors, and power supply branch.

[0045] The L298N motor drive unit has an input voltage range of 6-12V, is equipped with 4 signal input pins and 2 enable pins, supports PWM speed regulation and digital direction control, and integrates two independent "H" bridge full-bridge circuits. Each "H" bridge consists of 4 switching elements. By controlling the synchronous conduction of the switching elements on the diagonal, the forward and reverse rotation of the motor can be achieved, thereby driving the vehicle to complete forward, backward, and turning actions. In this embodiment, the IN1, IN2, IN3, and IN4 pins of the L298N are connected to the GPIO pins of the ESP32S3 main controller, the ENA and ENB pins are connected to the PWM output channel of the ESP32S3, and OUT1, OUT2 and OUT3, OUT4 are respectively connected to two sets of coaxial DC geared motors. The L298N and the ESP32S3 main controller share a common ground. The main controller realizes stepless speed control of the motor by outputting PWM signals with different duty cycles; and realizes forward and reverse rotation of the motor and steering control of the vehicle by switching the high and low level combinations of the IN1-IN4 pins.

[0046] (III) Sound Source Localization Module

[0047] This module is used to calculate the azimuth information of noise sources in the environment relative to the inspection vehicle in real time. It adopts the AR1105 sound source positioning module, which has a size of 37mm×26mm, an operating current of 28-31mA, supports a wide voltage input of 4-6.5V, and is compatible with the power supply architecture of this system.

[0048] This module integrates a DSP algorithm core and three highly consistent digital silicon microphones with a 10mm pitch, constructing a six-directional detection system covering a 360° range, corresponding to 0°, 60°, 120°, 180°, 240°, and 300° directions. It supports a 16kHz sampling rate, a direction detection period of less than 50ms, an effective pickup range of 10-200cm, and microphone consistency of ±1 dBFS. The module uses a phase difference positioning algorithm to analyze the time delay difference of sound waves arriving at different microphones in real time to calculate the sound source location. When the sound source signal intensity in a certain direction exceeds a preset threshold, the corresponding IO port outputs a 3.3V high-level signal, directly feeding back to the ESP32S3 main controller.

[0049] (iv) Sound information acquisition module (noise intensity detection and recording function)

[0050] This module is used for real-time acquisition of environmental noise signals, noise intensity quantization, and raw audio recording. It uses an INMP441 MEMS digital microphone, measures 3.76mm × 4.72mm, has a frequency response range of 60Hz-15kHz, an operating current of 1.4mA, an operating voltage of 1.8V-3.3V, a sensitivity of -26dBFS, and a built-in automatic gain control function, which can stably acquire sound amplitude within a dynamic range of -26dBFS to -94dBSPL.

[0051] The microphone is equipped with six pins: the SD pin for transmitting digital audio data, the SCK pin for receiving clock signals, the WS pin for receiving frame synchronization signals, the L / R pin for left / right channel selection, the VDD pin for power supply, and the GND pin for ground. The module connects to the ESP32S3 main controller via an I2S interface, converting the acquired sound signals into digital signals for transmission to the main controller, effectively reducing signal interference and data loss. In this embodiment, the module simultaneously implements noise intensity detection and audio recording functions, optimizing system integration through hardware multiplexing: during inspection, the module collects ambient noise at a 44.1kHz sampling rate and 16-bit quantization for real-time noise intensity quantization; when the vehicle arrives at the sound source location and stops, the module simultaneously starts audio recording, generating a WAV format audio file for subsequent source tracing analysis.

[0052] (v) Precision positioning module

[0053] This module is used to obtain the absolute geographic coordinates of the inspection vehicle and to calibrate the spatial location of noise sources. It adopts the ATGM322D 5N-31 multi-mode satellite navigation and positioning module, which is packaged in a compact 12×16mm package. It is equipped with the AT6558 fourth-generation low-power GNSS SOC chip from Zhongke Microelectronics and is fully compatible with six major satellite navigation systems: Beidou, GPS, GLONASS, GALILEO, QZSS, and SBAS. Through 32-channel parallel tracking technology, it can simultaneously receive signals from multiple constellations of satellites, achieving a positioning accuracy of up to 2.5 meters and a tracking sensitivity of up to -162dBm. It can still maintain stable positioning output in weak signal scenarios.

[0054] This module supports the NMEA-0183 standard protocol and connects to the ESP32S3 main controller via the UART interface. It outputs NMEA statements containing positioning information in real time. The main controller extracts and converts the NMEA statements to obtain standard decimal latitude and longitude coordinates, which are used as geographic calibration data for the noise source.

[0055] (vi) Visual Recognition Module

[0056] This module uses an OV2640 camera module, connected to the ESP32S3 main controller via a DVP digital interface. It supports image acquisition up to 2 megapixels and a frame rate of up to 30fps. The module is used to acquire real-time path images during the inspection vehicle's tracking of the sound source, identify obstacles using frame difference analysis, and generate obstacle avoidance commands that are fed back to the main control and data processing modules. It is also used to acquire on-site environmental video data after the inspection vehicle reaches the sound source location, synchronously storing it in conjunction with the audio files to provide complete on-site evidence for noise event tracing.

[0057] (vii) Storage module

[0058] This module is used to locally store the original audio files, location data logs, and complete noise event data packets. It is implemented using a 1GB TF card and an MK012525 Micro SD card reader.

[0059] The TF card operates at DC 3.3V / 5V, with a read / write speed of at least 4MB / s and an operating temperature range of -25℃ to 85℃, meeting the requirements for use in noisy inspection environments. The MK012525 card reader operates at DC 4.5-5.5V, integrates a 3.3V voltage regulator circuit, and is equipped with six pins: ground (GND), power supply (VCC), SPI bus (MISO, MOSI, SCK), and chip select signal (CS). The card reader communicates with the ESP32S3 host controller via the SPI interface, enabling high-speed data transmission between the host controller and the TF card.

[0060] (viii) Power supply module

[0061] This module provides a stable and matched power supply for all modules in the system. It consists of a 7.4V 2000mAh lithium battery and a multi-output voltage regulator module.

[0062] The 7.4V lithium battery serves as the main power source for the vehicle's power system and all sensor modules. The multi-output voltage regulator module measures 4.8cm x 4.8cm, has an input voltage range of 6-24V, and can directly adapt to the 7.4V lithium battery input. It is equipped with six independent 5V and six independent 3.3V outputs, which can simultaneously power multiple modules with different voltage requirements. The module has a built-in main power switch, which can control the system's on / off state with one button. It also has a Vout=Vin direct connection output function, which can provide sufficient power input for the L298N motor drive module.

[0063] (ix) Data Communication and Upload Module

[0064] This module utilizes the built-in 2.4GHz WiFi module of the ESP32S3 main controller, supporting the TCP / IP protocol stack. It establishes a bidirectional communication connection with the Baffars Cloud IoT platform via TCP, enabling real-time uploading and remote management of noise event data. The platform's TCP server address is bemfa.com, and the communication port is 8344. Uploaded data is encapsulated in standard JSON format, allowing for cloud storage, visualization, and export of the data.

[0065] III. Specific Implementation Methods of Core Software and Algorithms

[0066] (I) Sound source direction identification and data reading logic

[0067] The ESP32S3 main controller reads six directional signals from the AR1105 module in real time via its digital input port. Each signal corresponds to an azimuth angle. When a sound source triggers a signal in a certain direction, the corresponding GPIO pin outputs a high level. The main controller obtains the azimuth angle information of the current sound source by cyclically scanning the level states of the six GPIO pins. Only when three consecutive high-level signals in the same direction are detected is it determined to be a valid sound source direction, avoiding misjudgments caused by transient interference. The scan period is 50ms.

[0068] (II) Noise Intensity Quantization Algorithm

[0069] The INMP441 microphone captures ambient sound in real time via the I2S interface at a sampling rate of 44.1kHz. The raw data is temporarily stored in a memory buffer as 32-bit signed integers. The system uses a sliding window RMS algorithm to process the audio data, with a window size of 4096 samples. The specific calculation process is as follows:

[0070] The first step is to read 4096 audio samples each time, extract the high 16 bits of valid data from the left channel, and map the samples to the [-1,1] interval through normalization.

[0071] The second step is to calculate the root mean square (RMS) value of all samples within the window. The calculation method is to divide the sum of the squares of the amplitudes of all samples within the window by the total number of samples, and then take the square root.

[0072] The third step is to convert the RMS value to a decibel value, calculated as 20 × lg(RMS + 10⁻⁻⁶). 6 ), of which 10⁻ 6 To prevent zero input of a minimum value, and to avoid calculation errors caused by zero input;

[0073] The fourth step is to superimpose a 94dB offset to compensate for the sensitivity parameters of the INMP441 microphone and obtain the actual ambient noise intensity value.

[0074] When the system detects noise intensity exceeding the preset threshold of 60dB three times consecutively, a stop command is triggered. The system uses a counter and a timer to determine the condition: the counter increments each time the RMS value exceeds the threshold, and if the continuous exceeding condition is not met within 3 seconds, the counter is reset to zero. Once the condition is met, the motor drive is stopped immediately, and the secondary positioning and recording process is started.

[0075] (III) Sound Source Tracking and Motion Control Logic

[0076] The main controller generates motor control commands based on the real-time acquired effective sound source azimuth angle and a preset direction-action mapping rule. This drives the trolley to move towards the sound source. The sound source direction detection result is updated every 50ms, dynamically adjusting the motor control commands to achieve continuous closed-loop tracking of static or moving sound sources. The control logic is as follows: Figure 4 As shown.

[0077] In this embodiment, the mapping rule between the sound source azimuth angle and the motor control command is as follows: when the sound source azimuth angle is 0°, the left and right wheels move forward at the same speed; when the sound source azimuth angle is 60°, the left wheel speed decreases by 50%, and the right wheel moves forward at a constant speed; when the sound source azimuth angle is 120°, the left wheel stops, and the right wheel moves forward at a constant speed; when the sound source azimuth angle is 180°, the left and right wheels reverse at the same speed; when the sound source azimuth angle is 240°, the right wheel stops, and the left wheel moves forward at a constant speed; when the sound source azimuth angle is 300°, the right wheel speed decreases by 50%, and the left wheel moves forward at a constant speed.

[0078] During the tracking process, the main controller synchronously receives obstacle detection signals from the visual recognition module. When an obstacle is detected at a distance of less than 0.5 meters, an obstacle avoidance command is generated, and the vehicle is controlled to bypass the obstacle and continue moving towards the sound source.

[0079] (iv) GPS positioning data parsing logic

[0080] The ESP32S3 master controller reads the raw NMEA data stream output by the ATGM322D module in real time via the UART serial port, and uses a dedicated function to parse statements starting with $GNGGA. The specific parsing process is as follows:

[0081] The first step is data filtering: traverse the original data stream and only extract statements that start with $GNGGA, ignoring other irrelevant statements;

[0082] The second step is field segmentation: the statement is divided into 15 fields by commas and asterisks, and key information such as the original latitude value, north-south indicator, original longitude value, east-west indicator, and positioning status is extracted.

[0083] The third step is coordinate transformation: convert the original coordinates in DDMM.MMMMM format to standard decimal coordinates. Latitude is converted to degrees by truncating the first two digits, and the remaining part is a fraction. The decimal value is equal to the degree plus the fraction divided by 60. Longitude is converted to degrees by truncating the first three digits, and the remaining part is a fraction. The decimal value is equal to the degree plus the fraction divided by 60.

[0084] Fourth step, sign correction: If it is south latitude S or west longitude W, the decimal value is negative;

[0085] Step 5, validity check: If the location status is 0, it is an invalid location, or the number of fields is insufficient, then discard the data; otherwise, output the valid decimal latitude and longitude coordinates.

[0086] After the car stops at the sound source location, the main controller continuously collects positioning data 5 times and takes the average value as the final calibration coordinates of the noise source to improve positioning accuracy.

[0087] (v) Cloud Data Communication Logic

[0088] The ESP32S3 main controller connects to the Buffalo Cloud IoT platform via its built-in WiFi module. The core communication process is as follows:

[0089] The first step, initialization phase: Configure parameters such as WiFi name and password, platform user unique identifier, device theme, etc., connect to the platform server via TCP protocol, and send device online command;

[0090] The second step is data encapsulation: encapsulate the data such as the time, latitude and longitude coordinates, noise intensity, and device status of the noise event into a standard JSON format;

[0091] The third step is data upload: the encapsulated data packets are uploaded to the platform via TCP message publishing to achieve cloud storage and visualization of the data;

[0092] The fourth step is the reconnection mechanism: if the network connection is interrupted, the reconnection process is automatically triggered to ensure the stability of data transmission.

[0093] (vi) Audio recording and storage logic

[0094] This embodiment reuses the INMP441 microphone to implement audio recording functionality. The specific process is as follows:

[0095] The first step is to reconfigure the I2S interface parameters after the car stops, setting the sampling rate to 44.1kHz, the quantization bit depth to 16 bits, and the mono acquisition.

[0096] The second step is to create a double buffer in memory to temporarily store the real-time audio data and prevent data loss.

[0097] The third step is to use a timer interrupt to write the audio data in the buffer to the TF card at fixed intervals via the SPI interface. According to the WAV file format specification, the file header containing the audio format, sampling rate, and quantization bit depth is written first, and then the audio data is written segment by segment.

[0098] The fourth step is to close audio capture and file writing after completing the preset 10-second recording, release related resources, and name the audio file in the format noise_record_year month day hour minute second.wav. The file storage path is associated with the noise source coordinate information to provide a basis for subsequent source tracing.

[0099] (vii) Backstage acoustic analysis logic

[0100] For WAV format audio files stored on a TF card, acoustic analysis was performed using the Voice-box speech processing toolbox in MATLAB software to generate audio waveforms and spectrograms. The waveforms, with time on the horizontal axis and sound amplitude on the vertical axis, reflect the temporal characteristics of the noise signal. The spectrograms, through Fast Fourier Transform, convert the audio signal into time-frequency domain data, with time on the horizontal axis, frequency on the vertical axis, and color amplitude representing signal energy. This visually reflects the frequency distribution, duration, and other characteristics of the noise signal, enabling sound source type identification and voiceprint feature extraction, providing effective evidence for tracing the source of noise events.

[0101] IV. Complete System Workflow Example

[0102] In this embodiment, the complete workflow of the noise inspection vehicle consists of 7 core steps, as shown in the flowchart below. Figure 3 As shown, the details are as follows:

[0103] The first step is system deployment and initialization: Deploy the inspection vehicle in the target monitoring area, turn on the main power of the system, and after the ESP32S3 main controller starts up, complete the self-test of all peripheral modules, establish WiFi and cloud communication links, obtain the initial latitude and longitude coordinates of the vehicle through the GPS module, upload the initial location information to the Baffars cloud platform, and enter the inspection mode.

[0104] The second step is real-time data acquisition and synchronization: During the inspection, the system simultaneously starts data acquisition from multiple modules. The AR1105 module outputs the sound source direction detection result every 50ms, the INMP441 microphone continuously collects environmental noise signals, and the GPS module updates the real-time coordinates of the vehicle every second. All data are timestamped and aligned to achieve spatiotemporal synchronization of multi-source data, with a time synchronization deviation of no more than 10ms.

[0105] The third step is data preprocessing and noise exceeding the standard judgment: The main controller preprocesses the collected audio data, eliminates environmental interference such as wind noise through adaptive filtering, and quantifies the noise intensity in real time through the sliding window RMS algorithm. When the noise intensity exceeds the preset threshold of 60dB for three consecutive times, it is judged as an excessive noise event, triggering the sound source localization and tracking process.

[0106] The fourth step is sound source localization and autonomous tracking: The main controller generates motor control commands based on the effective sound source azimuth angle output by the AR1105 module through the direction-action mapping rules, driving the car to move towards the sound source direction. The sound source direction is updated every 50ms, and the travel route is dynamically adjusted to achieve closed-loop tracking of the sound source. Obstacle avoidance is completed simultaneously during the tracking process.

[0107] The fifth step is precise positioning and coordinate uploading: When the car approaches the sound source to a preset distance of 2 meters, or when the noise intensity exceeds the standard for more than 10 seconds, the main controller controls the car to stop moving, triggers the GPS module to perform secondary positioning, collects 5 positioning data in a row and takes the average value as the calibration coordinates of the noise source, encapsulates the coordinates, noise intensity and timestamp into JSON format and uploads them to the Baffars cloud platform in real time.

[0108] Step 6: On-site audio and video evidence preservation: After the car stops, the INMP441 microphone is activated to record a 10-second WAV format audio file, and the OV2640 camera is activated to capture on-site video. The audio and video files are stored in the TF card and are associated with the coordinates of the noise source in time and space. The file storage path and key information are uploaded to the cloud simultaneously.

[0109] Step 7: Data Cloud Aggregation and Back-end Analysis: The Baffa Cloud Platform receives and stores all uploaded noise event data and audio / video files, generates noise event logs, and allows administrators to remotely view event information and download raw data through the platform's visual interface. MATLAB is used to perform spectrogram analysis on the recording files, extracting sound source voiceprint features to complete the source tracing analysis of noise events.

[0110] V. Experimental Verification Examples

[0111] (I) Experimental Environment and Equipment

[0112] This experiment was conducted in the campus square of Zhejiang University Ningbo Institute of Technology. The experimental environment was an open and semi-open scene, simulating noise scenarios such as daily construction and crowd activities on campus. The core equipment of the experiment was a prototype noise inspection vehicle made according to the scheme of this embodiment. The experimental sound source was a standard sound source with a frequency of 1kHz and the output intensity of the sound source was stable.

[0113] (II) Experimental Content and Procedures

[0114] The first item is the verification of the initial positioning function: the car is placed at the position with absolute coordinates of 29.816573° North latitude and 121.572030° East longitude, the system power is turned on, and the initial coordinate data uploaded by the car to the Baffars cloud platform is recorded to verify the initial positioning and data upload functions.

[0115] The second item is the verification of sound source localization and tracking function: sound sources are set in different directions around the car, namely, one time directly in front, two times to the right front, and two times to the left front, for a total of five repeated experiments. The accuracy of the car in recognizing the direction of the sound source, the steering response speed, and the stability of tracking are recorded.

[0116] The third item is the positioning accuracy verification: a sound source is set up 1.5 meters away from the initial position of the car, the calibration coordinates of the sound source are recorded after the car stops, and the actual coordinates of the sound source are compared with the coordinates of the sound source to calculate the positioning error.

[0117] The fourth item is the verification of the recording and acoustic analysis functions: record the integrity and audio clarity of the audio file after the car stops, perform spectrogram analysis on the audio file, and verify the accuracy of sound source frequency identification.

[0118] The fifth item is the verification of data transmission stability: record the communication connection status between the car and the cloud platform and the data upload success rate during the experiment.

[0119] (III) Experimental Results and Analysis

[0120] Regarding the initial positioning function, the initial coordinates uploaded by the vehicle were 29.816576° North latitude and 121.572032° East longitude, which are consistent with the actual placement position. The initial positioning and data upload functions are working normally.

[0121] In terms of sound source localization and tracking, in 5 repeated experiments, the car was able to accurately identify the direction of the sound source, automatically adjust its driving direction to move towards the sound source, the motor drive module worked stably, the car turned flexibly, the sound source direction recognition accuracy was 100%, and the direction response delay was less than 50ms.

[0122] Regarding the verification of positioning accuracy, the experimental results show that the positioning error of the car to the noise source is within 2.5 meters, which meets the design positioning accuracy requirements, and the precise parking and secondary positioning functions are reliable.

[0123] In terms of recording and acoustic analysis functions, the recording files are stored completely in the TF card, the audio content is clear, and the sound source information within 10 seconds can be completely recorded; the spectrogram analysis results show that the core frequency of the sound source can be accurately identified as 1kHz, which is consistent with the sound source parameters set in the experiment, and the sound source features can be effectively extracted.

[0124] In terms of data transmission stability, the communication connection between the vehicle and the cloud platform was stable during the experiment, and all noise event data were successfully uploaded with a 100% data upload success rate, enabling remote visual control.

[0125] The comprehensive experimental results show that the noise inspection vehicle in this embodiment can stably realize all the preset functions, and all performance indicators meet the design requirements. It has the engineering application capability for noise monitoring in semi-open places such as campuses and communities.

[0126] VI. Alternative Implementation Methods

[0127] This embodiment is merely a preferred implementation of the present invention and is not intended to limit the invention. The inspection carrier of the present invention is not limited to wheeled vehicles, but can also be replaced by tracked robots, quadruped robots, drones, and other mobile carriers, adapting to different inspection scenarios such as mountains, elevated roads, and utility tunnels; the microphone array can be replaced with an array scheme with a higher number of channels to further improve the angular resolution and positioning accuracy of sound source localization; AI voiceprint recognition algorithms can be introduced to automatically classify sound sources and identify anomalies in the recorded data, further improving the system's intelligence level; it can be expanded to connect to environmental sensors such as meteorological and air quality sensors to achieve multi-parameter environmental monitoring functions.

[0128] Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention shall be included within the scope of protection of this invention.

Claims

1. A complex environmental noise inspection, sound source localization, and tracking system based on multi-sensor information fusion, comprising a main control and data processing module, and a sound information acquisition module, a sound source localization module, a motion control and drive module, a precise positioning module, and a data communication and uploading module respectively communicatively connected to the main control and data processing module, characterized in that, It also includes a data processing and analysis center, a visual recognition module, and a storage module that are bidirectionally connected to the main control and data processing module; The sound information acquisition module is used to acquire noise signals from the target environment in real time, quantify the noise intensity, and send the acquired raw noise signals and noise intensity data to the data processing and analysis center. The sound source localization module is used to calculate the azimuth angle information of the noise source in the environment relative to the inspection carrier in real time, and send the calculated sound source azimuth information to the data processing and analysis center. The precise positioning module is used to obtain the absolute geographic coordinates of the current position of the inspection vehicle in real time and transmit the coordinate data back to the data processing and analysis center in real time. The visual recognition module is used to collect path images to identify obstacles and generate obstacle avoidance commands during the process of the inspection vehicle tracking the sound source, and to collect on-site environmental video data after the inspection vehicle arrives at the sound source location. The motion control and drive module is used to receive control commands from the main control and data processing module, drive the inspection carrier to perform corresponding movement actions, and complete the preset path inspection or sound source directional tracking. The storage module is used to store the original audio files, on-site video files, location data logs, and complete noise event data packets; The data communication and uploading module is used to receive the full data of noise events summarized by the main control and data processing modules, and upload the data to the remote IoT cloud platform in real time to realize remote management and source tracing analysis of noise data. The data processing and analysis center includes a multi-source data preprocessing unit, a time-frequency conversion and feature extraction unit, a sound source precise localization unit, a tracking path planning unit, and a multimodal data fusion unit; wherein: The multi-source data preprocessing unit is used to filter and reduce noise in the original noise signal, and to align the timestamp of the noise signal with the coordinate data of the precise positioning module and the carrier position data of the motion control and drive module at the time origin to ensure the spatiotemporal synchronization of the multi-source data. The time-frequency conversion and feature extraction unit is used to perform time-frequency conversion and feature extraction on the preprocessed noise signal, compare the noise intensity with a preset threshold, determine whether it is excessive noise, and trigger the sound source localization and tracking process. The sound source precise positioning unit is used to combine the sound source orientation information with the geographic coordinate data of the precise positioning module to convert the acoustic orientation information into the absolute geographic coordinates corresponding to the noise source, thereby achieving precise spatial positioning of the noise source. The tracking path planning unit is used to plan the optimal tracking route based on the real-time calculated absolute geographic coordinates of the noise source, through the direction-action mapping rules and real-time feedback control logic pre-set in the unit, generate steering and speed control commands and send them to the main control and data processing module. The multimodal data fusion unit is used to fuse acoustic feature data, geographic coordinate data, on-site video data collected by the visual recognition module, and raw audio data to generate a complete noise event data packet. The main control and data processing module serves as the core control unit of the system, receiving control commands issued by the data processing and analysis center, scheduling the collaborative work of various modules, and completing the entire process control of noise event detection, location, tracking, recording, and uploading.

2. The complex environment noise patrol, sound source positioning and tracking system based on multi-sensing information fusion according to claim 1, characterized in that, The main control and data processing module uses an ESP32S3 main controller; the sound information acquisition module uses an INMP441 MEMS digital microphone, which is connected to the main control and data processing module via an I2S interface, and can generate WAV format audio files associated with the geographic coordinates of the noise source; the sound source localization module uses a six-way microphone array, and calculates the azimuth angle of the noise source through a DSP phase difference localization algorithm, with a direction detection period of no more than 50ms; the precise positioning module uses a multi-mode satellite navigation and positioning module compatible with BeiDou, GPS, GLONASS, and GALILEO, with a positioning accuracy of no more than 2.5 meters. 3.The complex environment noise patrol, sound source positioning and tracking system based on multi-sensing information fusion of claim 1, wherein, The motion control and drive module uses an L298N motor drive module, which supports PWM stepless speed regulation and digital direction control, and can drive the inspection vehicle to perform forward, backward, turning, stair climbing, and slope climbing actions; the vision recognition module uses an OV2640 camera module, and the acquired video data and audio files are synchronously associated and stored; the storage module is connected to the main control and data processing module through an SPI interface, supporting high-speed reading and writing of audio, video and positioning data; the data communication and upload module establishes bidirectional communication with the IoT cloud platform through the TCP protocol, and can upload noise event time, intensity, coordinates, audio and video files and vehicle motion trajectory data.

4. The complex environment noise patrol, sound source positioning and tracking system based on multi-sensing information fusion according to claim 1, characterized in that, The multi-source data preprocessing unit is also used to perform adaptive filtering on the original noise signal to improve the signal-to-noise ratio, and at the same time to complete the time synchronization error calibration of the multi-source data to ensure that the spatiotemporal deviation of the data is no more than 10ms; the time-frequency conversion and feature extraction unit is used to convert the preprocessed noise signal into a time spectrum through fast Fourier transform, and extract the acoustic feature parameters of the spectral peak, bandwidth, and duration of the noise exceeding the standard, as the basis for noise exceeding the standard and sound source type identification.

5. The complex environment noise patrol, sound source localization and tracking system based on multi-sensing information fusion of claim 1, wherein, The sound source precise positioning unit is also used to optimize the sound source azimuth calculation accuracy through triangulation and beamforming algorithms, and at the same time, it combines the noise intensity change trend to determine whether the noise source is in a moving state; the direction-action mapping rule built into the tracking path planning unit is to map the sound source azimuth angle to a standardized control instruction set of the combination of the inspection carrier motor direction and speed; the real-time feedback control logic is to cyclically execute the closed-loop control process of "sound source direction detection → generating motor control instructions → driving carrier movement → detecting sound source direction again", with a control cycle of no more than 50ms.

6. The complex environmental noise inspection, sound source localization and tracking system based on multi-sensor information fusion according to claim 1, characterized in that, The multimodal data fusion unit is also used to encapsulate the generated noise event data packets into a standard JSON format, and at the same time complete the unique identifier binding between the data packets and the noise events; the data communication and uploading module also has an automatic reconnection mechanism for network interruption to ensure the stability of data transmission.

7. A method for complex environment noise patrol, sound source positioning and tracking based on multi-sensing information fusion, characterized in that, The system implementation based on any one of claims 1 to 6 includes the following steps: Step S1: System deployment and initialization. The inspection carrier equipped with the system is deployed in the target monitoring area. After the system is powered on, the self-test of each module is completed, the communication link is established, and the initial position information of the inspection carrier is obtained and reported. Step S2: Real-time data acquisition and synchronization. After the system enters the inspection mode, it continuously acquires environmental noise signals, calculates the noise source location information, obtains the real-time geographic coordinates of the inspection carrier, and completes the timestamp alignment and calibration of multi-source data. Step S3: Data preprocessing and noise exceedance judgment. The collected raw noise signal is preprocessed to eliminate environmental interference, the noise intensity value is quantified and calculated, and compared with the preset threshold to determine whether the sound source localization and tracking process is triggered. Step S4: Sound source localization and autonomous tracking. When the noise intensity exceeds the standard, the system drives the inspection vehicle to autonomously adjust its attitude and travel route according to the real-time calculated sound source location information, and to face and approach the noise source. During the tracking process, the sound source location is updated in real time and the travel strategy is adjusted. Step S5: Precise positioning and coordinate upload. When the inspection vehicle approaches the sound source to a preset distance or the noise intensity continues to exceed the standard for a preset time, the vehicle is controlled to stop moving. The precise geographic coordinates of the current location are obtained as noise source location calibration data, and the relevant data is uploaded to the IoT cloud platform in real time. Step S6: Preserve on-site evidence. Simultaneously start audio recording and video acquisition at the location point to generate audio and video evidence files that are spatiotemporally synchronized with the coordinates of the noise source, associate them, store them, and upload them to the cloud platform. Step S7: Data aggregation and analysis in the cloud. The cloud platform receives and integrates all noise event data and audio-visual evidence, generates noise event logs, and allows managers to remotely view and trace the source of the noise.

8. The complex environment noise patrol, sound source positioning and tracking method based on multi-sensing information fusion according to claim 7, characterized in that, In step S3, the criteria for triggering the sound source localization and tracking process are that the noise intensity exceeds a preset threshold for a certain number of consecutive preset times, or the noise intensity exceeds the standard for a certain duration. The preprocessing includes adaptive filtering and noise reduction of the original noise signal, as well as time synchronization error calibration of multi-source data.

9. The complex environment noise patrol, sound source positioning and tracking method based on multi-sensing information fusion according to claim 7, characterized in that, In step S4, autonomous tracking is achieved through direction-action mapping rules and real-time feedback control logic. The sound source location detection result is updated once every preset period, and the travel direction and speed of the inspection vehicle are dynamically adjusted. At the same time, obstacles on the path are identified in real time and obstacle avoidance actions are performed. In step S5, the accurate geographic coordinates are taken as the average value of the positioning data collected multiple times consecutively as the final calibration coordinates.

10. The complex environment noise patrol, sound source positioning and tracking method based on multi-sensing information fusion according to claim 7, characterized in that, In step S6, the naming of the audio and video files includes the event occurrence time and sound source coordinate information to achieve a unique association with the noise event; in step S7, cloud analysis includes time-frequency conversion and feature extraction of the audio files to identify the sound source type, and comparison with historical noise event data to analyze the activity patterns, frequency of occurrence and intensity change trends of the noise source.

Images