An Integrated Sensor Based on Microphone Array and Stereo Vision

Abstract

The invention discloses an integrated sensor based on a microphone array and stereo vision, including a hardware system and a control system; the hardware system includes a microphone array and a stereo vision device, and the microphone array is responsible for receiving sound information in the environment and generating multi-channel analog audio; the stereo vision The device is responsible for collecting images of the environment. The work of the control system includes audio work area, video work area and video and audio synthesis work area. The audio processing area is responsible for audio denoising, sound source localization, and sound source tracking (through audio processing); the video processing area is responsible for locating and tracking the target sound source. That is, the target sound source is calibrated, positioned and accurately tracked through image processing and image recognition; the video and audio synthesis area is responsible for enhancing the audio signal (improving the signal-to-noise ratio of the audio signal) and acquiring and outputting the fusion signal of the video and audio signal.

Description

technical field [0001] The invention mainly relates to an integrated sensor based on a microphone array and stereo vision, in particular to a sound source positioning and tracking technology based on a microphone array and stereo vision, belonging to the field of integrated sensors. Background technique [0002] In today's era, to record events, scenes, etc., audio and video are undoubtedly the most reductive methods; with the rapid development of Internet technology, more and more communication between people is carried out through the Internet, such as video conferencing . However, in the face of the complex sound field environment and the unpredictable position of the sound source, the traditional audio collection and video recording methods are facing major challenges. The sound field contains not only the target sound source, but also other interference sound sources; therefore, the collected sound includes not only the sound of the target sound source, but also enviro...

AI Technical Summary

Problems solved by technology

[0005] The audio processing involved in the above-mentioned patents is used to track the sound source by continuously scanning the space (or calculating the correlation) in continuous time, which has high time and space complexity, high requirements on the processor, and reduces the sound quality. The real-time nature of source tracking increases the interference factor (environmental interference sound source) when operating the entire space, which reduces the accuracy of positioning tracking; moreover, there are large errors in direct use of the sound source coordinates obtained through audio processing, which is not competent High-precision tracking work

Images