Multi-channel audio-visual linkage devcie

A multi-channel, audio-visual technology, applied in measuring devices, image communication, image data processing, etc., can solve the problems of unfavorable popularization of audio-visual linkage applications, large volume, poor real-time response, etc.

Active Publication Date: 2018-12-25
GUANGZHOU SHENGGUANG MICROELECTRONICS
View PDF16 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the existing solutions have the following disadvantages: poor real-time response, high power consumption, high implementation cost, large size, complex supporting development environment, it is difficult to become a sensor unit for the Internet of Things, and it is not conducive to the popularization of audio-visual linkage applications

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-channel audio-visual linkage devcie
  • Multi-channel audio-visual linkage devcie
  • Multi-channel audio-visual linkage devcie

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0049] figure 1 Embodiment 1 of the present invention provides a multi-channel audio-visual linkage device.

[0050] refer to figure 1 , the multi-channel audio-visual linkage device mainly includes: MLMS (Multi-lens Multi-sensor, multi-lens multi-sensor) integrated circuit chip 200, a multi-lens module 110 connected to the integrated circuit chip 200, a pan-tilt camera 120, and a multi-microphone device 130 and intelligent recognition AI chip 300.

[0051] refer to figure 2 , the multi-lens module 110 includes a plurality of lenses / sensors, the microphone array module 220 includes a plurality of microphone devices, and the plurality of microphone devices are distributed orderly and uniformly in the whole airspace according to the direction of the plurality of lenses / sensors and the pan-tilt camera 120 type installation.

[0052] The integrated circuit chip 200 is integrated with: an audio-visual linkage module 230, the audio-visual linkage module 230 is connected to the ...

Embodiment 2

[0059] The process of the multi-channel audio-visual linkage device performing the audio-fixing task is as follows.

[0060] The multi-image processing module 210 is configured to collect multi-channel first original images through the multi-lens module 110, and perform processing such as distortion correction on the first original images to generate a first panoramic stitching image.

[0061] Specifically, the implementation process for the multi-image processing module 210 to generate the first panoramic stitched image is as follows. The image acquisition unit of the corresponding channel collects and outputs a plurality of first original images in RAW, RGB, YUV and other formats through the connection lens / sensor; the ISP unit performs image signal processing on the first original image to obtain processed image information; distortion The correction unit performs distortion correction and image mapping transformation processing on the processed image information to obtain ...

Embodiment 3

[0073] The process of the multi-channel audio-visual linkage device performing the video-fixed-audio task is as follows.

[0074] The multi-image processing module 210 is configured to collect multi-channel second original images through the multi-lens module 110, and after processing the second original images, generate a second panoramic stitching image, and stitch the second panoramic images according to a preset method. After the sound source object is recognized on the image, a second sound source object is created, and the image detection parameters are assigned to the second sound source object and stored in the object collection queue.

[0075] Specifically, the second panoramic stitching image is also obtained by the image acquisition unit, the ISP unit, the distortion correction unit and the panoramic stitching unit. The image acquisition unit is used to collect a plurality of second original images in formats such as RAW, RGB, and YUV output by the lens / sensor; the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-channel video-audio linkage device, which relates to the technical field of an intelligent edge computing device, comprising a connected multi-lens module, a pan head camera, a multi-microphone device, an intelligent identification AI chip and an integrated circuit chip. The integrated circuit chip comprises an audio-visual linkage module, wherein the audio-visual linkage module is connected with a multi-image processing module through an object acquisition queue, is connected with a microphone array module through an audio acquisition queue, and is connected with a multi-image splicing module through an identification queue. The multi-lens module and the pan camera are connected with the multi-image processing module, and the multi-microphone device is connected with the microphone array module. The intelligent identification AI chip is connected with a multi-image processing module and a microphone array module through an audio-video interface. The invention utilizes integrated circuit chip to increase flexibility of video-audio linkage function combination, to improve video-audio linkage response speed, and utilizes multi-graph splicing module to reduce network transmission flow and development cost, to reduce delay, to reduce power consumption and product volume.

Description

technical field [0001] The invention relates to the technical field of intelligent edge computing devices, in particular to a multi-channel audio-visual linkage device. Background technique [0002] Audio-visual linkage is the concept of visual and acoustic interactive perception of "seeing pictures and following audio" and "hearing sound and watching video". The former is to quickly orientate the sounding coordinates of the object and collect audio data based on the recognized object or pedestrian; the latter is to quickly determine and locate the sound source according to the sounding direction, and simultaneously collect the image data of the sound source. [0003] Generally speaking, the audio-visual linkage device adopts multiple independent cameras and multiple microphone devices, which are connected to the router integration device through the audio-video interface and then packaged synchronously, and then output to a PC (Personal Computer, personal computer) or serve...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04N7/18G06T5/00G06T3/40G01S5/18
CPCG01S5/18G06T3/4038G06T5/006H04N7/181
Inventor 于燕斌张燕生林振雷胡志建
Owner GUANGZHOU SHENGGUANG MICROELECTRONICS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap