Composite audiovisual perception and localization system using acoustic array and multispectral camera
Through the composite audio-visual perception and positioning system of acoustic array and multispectral cameras, combined with acoustic and optical detection, accurate detection and positioning of three-dimensional space is achieved, solving the problem of insufficient accuracy of automated monitoring and positioning in the existing technology, and is suitable for gas Leak and fire detection provide efficient automated monitoring and data upload capabilities.
Patent Information
- Application Number
- CN202420597172.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-26
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2034-03-26
AI Technical Summary
Existing acoustic imagers have not yet fully automated monitoring in gas leakage detection and fire detection, and the accuracy of acoustic positioning and three-dimensional spatial recognition are difficult to achieve, and binocular recognition is inaccurate or false alarms may occur in complex environments.
A composite audio-visual perception and positioning system using acoustic array and multispectral camera, combined with acoustic array components and multispectral camera devices, 360° rotation detection is achieved through the rotating component, combined with acoustic detection and binocular recognition, and a multi-arm spiral microphone array and Infrared, visible, thermal imaging cameras perform composite analysis.
It realizes accurate detection and positioning of three-dimensional space, improves identification and positioning accuracy in complex environments, has fully automated monitoring and efficient data upload functions, saves human resources, and is suitable for application scenarios such as fire safety.
Smart Images

Figure CN221959560U_ABST
Abstract
Description
Technical Field
[0001] The utility model belongs to the field of array signal processing and machine vision, and more specifically, relates to a composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera. Background Art
[0002] In terms of gas leak detection, acoustic detection and optical imaging are usually used in combination at home and abroad, and the main equipment is an acoustic imager. The principle of an acoustic imager is to use a microphone array to measure the distribution of sound fields within a certain range. By processing the sound pressure signal collected by the microphone through an array signal processing algorithm, the position and intensity of the noise source on the surface of the object under test can be obtained, and an intuitive image can be displayed in the form of a cloud map. Modern acoustic cameras are often combined with cameras, which can directly display the distribution of sound sources on the optical image of the object. Modern acoustic cameras are often combined with cameras, which can directly display the distribution of sound sources on the optical image of the object. Chinese utility model patent CN218937673U discloses a portable acoustic imager, which is provided with a plug rod and a support rod, which can be disassembled to make manual use and transportation more convenient; the acoustic imager can be rotated on the plug rod so that the angle can be adjusted; patent application WO2021215595A1 discloses a method and an acoustic camera for visually displaying improved acoustic information of an acoustic camera, which assigns the absolute levels of acoustic parameters with different sizes to color elements with different colors, and improves the visualization method of acoustic detection. The current acoustic imager has not yet achieved fully automated monitoring, which consumes human resources; it is difficult to achieve three-dimensional space recognition, and the accuracy of acoustic positioning needs to be improved.
[0003] In terms of fire detection, the current fire perception and positioning technology based on binocular vision mainly sets visible light binocular cameras or near-infrared binocular cameras at multiple points of the station to collect images of the monitoring area. The collected images are preprocessed and a binary image is obtained by threshold segmentation. Then the circularity, rectangularity and number of sharp corners in the image are calculated, and the image is flame identified based on these features. If there is a flame in the detection area, a fire alarm signal is issued, and the information of temperature, smoke, carbon dioxide, carbon monoxide, oxygen and infrared sensors are integrated to improve the accuracy of the alarm. The fire source is measured by the binocular camera, the fire source is located in combination with the position of the camera, and the location information of the fire source is output to control the fire extinguishing device near the fire source to extinguish the fire. Chinese utility model patent CN215691228U discloses a fire rescue robot for collecting indoor fire scene images using a binocular camera, and then drawing a three-dimensional map of the fire scene based on stereo vision matching. However, using only binocular recognition in complex environments such as temperature, humidity, dust, and interfering light sources is prone to inaccurate positioning or false alarms.
[0004] Based on the above analysis, there is an urgent need to propose a composite audio-visual perception and positioning system to solve the limitations of three-dimensional spatial positioning of acoustic arrays and the limitations of complex environments on binocular recognition, so as to improve the recognition and positioning accuracy in audio-visual perception and positioning application scenarios such as gas leak detection and fire detection. Utility Model Content
[0005] In view of the above defects or improvement needs of the prior art, the utility model provides a composite audio-visual perception and positioning system using an acoustic array and a multispectral camera, the purpose of which is to combine the acoustic array component and the multispectral camera device, thereby solving the technical problems of the limitations of the acoustic array three-dimensional spatial positioning and the restrictive effects of complex environments on binocular recognition.
[0006] To achieve the above-mentioned purpose, according to one aspect of the utility model, a composite audio-visual perception and positioning system using an acoustic array and a multispectral camera is provided, the system comprising a main body, an acoustic array assembly, a multispectral camera device and a rotating assembly; the acoustic array assembly comprises two acoustic array devices of the same structure, and the two acoustic array devices are symmetrically arranged on the top of the main body; the multispectral camera device is arranged in front of the upper part of the main body, the multispectral camera device comprises an infrared camera, a visible camera and a thermal camera, the thermal camera is located at the center of the multispectral camera device, and the infrared camera and the visible camera are symmetrically arranged on both sides of the thermal camera; the bottom of the main body is connected to the rotating assembly.
[0007] Preferably, the two acoustic array devices are respectively welded to the top of the main body through supporting columns, and an angle exists between the two acoustic array devices and the plane where the main body is located.
[0008] Preferably, a plurality of circular holes are provided on the surface of the sound array device, and microphone heads are installed in the circular holes.
[0009] Preferably, the plurality of circular holes are distributed in a multi-arm spiral shape from the center of the surface of the acoustic array device to the surrounding areas.
[0010] Preferably, the angle is 120° to 170°.
[0011] Preferably, the rotating assembly comprises a connecting column and a base, one end of the connecting column is threadedly connected to the bottom of the main body, and the other end of the connecting column is rotatably connected to the base.
[0012] Preferably, circular holes are symmetrically provided on both sides of the main body, and nozzles can be installed inside the circular holes.
[0013] Preferably, a voice alarm device is provided below the multi-spectral camera device, and a spectral sensor is provided below the voice alarm device.
[0014] In general, the above technical solutions conceived by the utility model can achieve the following beneficial effects compared with the prior art:
[0015] 1. The composite audio-visual perception and positioning system using an acoustic array and a multispectral camera proposed in the utility model combines an acoustic array component with a multispectral camera device to realize a composite audio-visual perception and positioning system in which acoustic detection and binocular recognition cooperate with each other, so that the detection of three-dimensional space can be better realized, the recognition and positioning can be more accurate, and the adaptability to various complex environments is stronger.
[0016] 2. The composite audio-visual perception and positioning system using acoustic array and multi-spectral camera proposed in this utility model adopts composite detection mode, and compositely analyzes far-infrared thermal imaging, near-infrared spectral imaging and visible light video images, making it more adaptable to complex environmental conditions and more responsive to identification and positioning. The binocular recognition and positioning technology has the advantages of more efficient measurement, high accuracy of three-dimensional reconstruction and restoration, and simple system.
[0017] 3. The utility model proposes a composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera. The acoustic array component in the system adopts a multi-arm spiral microphone array structure, which has better spatial filtering performance than other array structures and can effectively suppress aliasing effects. At the same time, it can obtain a lower sidelobe level while maintaining a narrow main lobe, thereby achieving better acoustic imaging effects.
[0018] 4. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera proposed in the utility model can be fixedly installed or mounted on an intelligent robot to realize functions such as fully automated monitoring, and can provide timely warnings and upload data to a cloud platform in a timely manner, saving human resources and providing a reliable solution for monitoring in various application scenarios such as fire safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a schematic diagram of the structure of the composite audio-visual perception and positioning system of the utility model using an acoustic array and a multi-spectral camera;
[0020] Figure 2 This is a structural front view of the composite audio-visual perception and positioning system of the utility model using an acoustic array and a multi-spectral camera;
[0021] Figure 3 It is a top view of the structure of the composite audio-visual perception and positioning system of the utility model using an acoustic array and a multi-spectral camera;
[0022] Figure 4 It is a structural left view of the composite audio-visual perception and positioning system of the utility model using an acoustic array and a multi-spectral camera;
[0023] Figure 5 It is an exploded diagram of the composite audio-visual perception and positioning system of the utility model using an acoustic array and a multi-spectral camera.
[0024] In all the drawings, the same reference numerals are used to represent the same elements or structures, wherein: 1-main body; 11-voice alarm device; 12-multi-spectral camera device; 120-infrared camera; 121-thermal imaging camera; 122-visible camera; 13-circular hole; 2-connecting column; 3-acoustic array device; 30-circular hole; 31-support column; 4-base. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solution and advantages of the utility model more clear, the utility model is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the utility model and are not used to limit the utility model. In addition, the technical features involved in each embodiment of the utility model described below can be combined with each other as long as they do not conflict with each other.
[0026] like Figure 1-5 As shown, the utility model proposes a composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera, the system consisting of a main body 1, an acoustic array component, a multi-spectral camera device 12, a spectral sensor 10, a connecting column 2 and a base 4.
[0027] Specifically, the two acoustic array devices 3 are welded to the top of the main body 1 through two inclined support columns 31, and the two acoustic array devices 3 have a certain angle with the plane. As a preferred embodiment of the utility model, the angle is 120° to 170°.
[0028] To further explain, each sound array device 3 has 32 circular holes 30 in a multi-arm spiral shape, and each circular hole has a circular microphone head.
[0029] To further illustrate, the multi-spectral camera device 12 is installed at the upper front position of the main body 1 and faces straight ahead. The multi-spectral camera device 12 is composed of the infrared camera 120 located on both sides, the visible camera 122 and the thermal imaging camera 121 located in the center.
[0030] To further explain, a voice alarm device 11 and a spectrum sensor 10 are provided in front of the main body 1 and below the multi-spectral camera device 12 .
[0031] To further explain, the main body 1 has a circular hole 13 inside, which can be used to install nozzles for spraying water and foam on both sides.
[0032] To further explain, the bottom of the main body 1 is supported by a cylindrical connecting column 2, and the connecting column 2 and the main body 1 are rotationally connected via a thread; the connecting column 2 is installed above the base 4, and the connecting column 2 can rotate around the base 4 to achieve 360° rotation detection; the base 4 can be disassembled or the connecting buckle can be replaced to facilitate installation on the intelligent robot.
[0033] The technical solution of the present utility model is further illustrated by means of specific embodiments below.
[0034] Taking the application scenario of identifying and locating gas leaks or fires as an example, the specific use process of this system includes the following steps:
[0035] Step 1, fix the system or mount the system on the robot;
[0036] Step 2: When the robot is installed in a fixed position, the system can rotate regularly to detect gas leaks, fires, and temperature anomalies; after the robot is loaded, the system can conduct inspections along the robot's fixed route;
[0037] Step 3: When the system detects gas leakage, fire or abnormal temperature, it will issue an alarm and take emergency measures.
[0038] The step 1 is specifically as follows: the system is installed at a suitable location, and it can be fixedly installed or mounted on an intelligent robot, such as a robot dog; in addition, a nozzle device for spraying water and foam can be connected to the through hole;
[0039] Step 2 is as follows: the system will perform rotation detection at regular intervals. When identifying and locating gas leakage or fire, the system will perform rotation detection at regular intervals, collect acoustic and optical information through the combination of acoustic detection and binocular recognition, and identify whether gas leakage, fire, or temperature abnormality has occurred; when the system is mounted on an intelligent robot, it can realize fully automated, large-coverage gas leakage detection and fire detection inspection by setting inspection routes for the intelligent robot;
[0040] The specific details of step 3 are as follows: once fire safety hazards such as gas leakage, fire or abnormal temperature are identified, the system will first issue a voice alarm through the voice alarm device. After connecting to the Internet, the system can upload the abnormal situation to the cloud platform so that relevant manual processing can be carried out in time; if a nozzle that sprays water and foam is connected, when a gas leak is found, foam can be automatically sprayed to the location to isolate harmful gases. When a fire is found, water can be automatically sprayed to the location to extinguish the fire. When the processing is completed, the system continues to rotate and detect to confirm whether the hidden danger has been completed by emergency processing. This makes the handling of safety hazards more timely, saves manpower and ensures safety.
[0041] It is easy for those skilled in the art to understand that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera, characterized in that: The system comprises a main body (1), an acoustic array assembly, a multi-spectral camera device (12) and a rotating assembly; the acoustic array assembly comprises two acoustic array devices (3) of identical structure, the two acoustic array devices (3) being symmetrically arranged on the top of the main body (1); the multi-spectral camera device (12) being arranged directly in front of the upper part of the main body (1), the multi-spectral camera device (12) comprising an infrared camera (120), a visible camera (122) and a thermal imaging camera (121), the thermal imaging camera (121) being located at the center of the multi-spectral camera device (12), the infrared camera (120) and the visible camera (122) being symmetrically arranged on both sides of the thermal imaging camera (121); and the bottom of the main body (1) is connected to the rotating assembly.
2. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 1, characterized in that: The two acoustic array devices (3) are respectively welded to the top of the main body (1) via support columns (31), and an angle exists between the two acoustic array devices (3) and the plane where the main body (1) is located.
3. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 2, characterized in that: A plurality of circular holes (30) are provided on the surface of the sound array device (3), and microphone heads are installed in the circular holes (30).
4. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 3, characterized in that: A plurality of circular holes (30) are distributed in a multi-arm spiral shape from the center of the surface of the acoustic array device (3) to the surrounding areas.
5. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 2 or 4, characterized in that: The angle is 120° to 170°.
6. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 1, characterized in that: The rotating assembly comprises a connecting column (2) and a base (4), one end of the connecting column (2) is threadedly connected to the bottom of the main body (1), and the other end of the connecting column (2) is rotatably connected to the base (4).
7. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 1, characterized in that: Circular holes (13) are symmetrically arranged on both sides of the main body (1), and nozzles can be installed inside the circular holes (13).
8. The composite audio-visual perception and positioning system using an acoustic array and a multi-spectral camera according to claim 1, characterized in that: A voice alarm device (11) is provided below the multi-spectral camera device (12), and a spectrum sensor (10) is provided below the voice alarm device (11).
