Sound source positioning device based on acousto-optic infrared fusion

Abstract

The utility model provides a sound source positioning device based on acousto-optic infrared fusion, which comprises a shell, a spiral arm, a limiting block, a sound source detection assembly, an array panel, an optical camera and an infrared camera. A microphone array is disposed on an array panel fixed to one side of the housing. The optical camera is fixedly connected to the center of the microphone array. The infrared camera is arranged at one end of the rotary arm, the other end of the rotary arm is connected to the shell and can rotate between a working position and a non-working position relative to the shell, in the working position, the infrared camera is located in the center of the microphone array, the rotary arm is limited by the limiting block, and in the non-working position, the infrared camera is located on the other side of the shell. According to the utility model, the infrared camera is introduced, so that images can be collected in a non-illumination or low-illumination environment or under the condition of a specific sound source type to be superposed with an acoustic cloud picture, and the positioning result is more visual; and the conversion between the optical camera and the infrared camera can be quickly completed through the rotating arm, and complicated disassembly and replacement operations are not needed.

Description

Technical Field

[0001] The utility model belongs to the technical field of acoustic signal collection and sound source positioning, and in particular relates to a sound source positioning device based on acoustic, optical and infrared fusion. Background Art

[0002] In many fields, such as speech recognition, sound source localization, sound enhancement, etc., it is necessary to collect, analyze and process sound signals in a directional manner. Microphone array technology refers to an acoustic system composed of two or more microphone units in a certain spatial structure. By processing the correlation between the sound signals collected by the microphones, the sound source information can be separated, thereby realizing the detection and location of the sound source, etc., so it has been widely used in these fields.

[0003] At present, microphone array signal processing technology and acoustic processing algorithms have been relatively complete after long-term development. By applying acoustic algorithms to array signals, a cloud map of the sound source intensity distribution can be obtained, and it can be further superimposed with the real picture of the surrounding environment to obtain the distribution position of the sound source in the real environment. Existing microphone arrays generally use acoustic cloud maps and photos collected by optical cameras for superposition. Due to the need for superposition operations, the optical camera and the microphone array need to be at the same coordinate origin. Therefore, the optical camera is often installed at the origin of the array panel. Utility Model Content

[0004] The applicant of the utility model has discovered that the cloud map superposition of the existing sound source positioning device is limited by the environment, such as it cannot be used in an environment without light or low light, and is also limited by the type of sound source, such as it cannot detect invisible but warm gas. In these cases, it is often necessary to use an infrared camera to collect infrared images, and then further superimpose them with the acoustic cloud map to locate the sound source. The installation position of the infrared camera also needs to be at the same coordinate origin as the microphone array. This leads to a position conflict between the infrared camera and the optical camera. If the type of superimposed image needs to be changed, the camera at the same position needs to be disassembled and replaced. Not only is the operation complicated and the replacement efficiency low, but it may also cause the camera position to shift and be inconsistent during operation, affecting the positioning result.

[0005] Based on the above-mentioned inventive concept, the utility model provides a sound source localization device based on the fusion of sound, light and infrared. In addition to an optical camera installed in the center of the array panel, the device also introduces an infrared camera for use in no-light or low-light environments or specific sound source types (such as invisible but warm gas), thereby expanding the application space of the sound source localization device. At the same time, a conversion arm is added to the infrared camera, and the user can quickly convert the optical camera and the infrared camera without the need for complicated disassembly and replacement processes. This not only simplifies the operation, but also greatly improves the efficiency of converting image acquisition tools.

[0006] In order to realize the above invention concept, the technical solution provided by the utility model is:

[0007] A sound source positioning device based on acoustic, optical and infrared fusion, which is special in that it includes a shell, a rotary arm, a limit block and a sound source detection component, wherein the sound source detection component includes an array panel, an optical camera and an infrared camera;

[0008] The shell is used to support the swing arm, the limit block and the sound source detection assembly;

[0009] A microphone array composed of microphone array elements is arranged on the array panel for collecting acoustic signals, and the array panel is fixed to one side of the shell;

[0010] The optical camera is fixedly connected to the center of the microphone array and is used to collect optical images;

[0011] An infrared camera is arranged at one end of the rotary arm for collecting infrared images. The other end of the rotary arm is connected to the shell and can rotate relative to the shell between a working position and a non-working position. In the working position, the infrared camera is located at the center of the microphone array, and the rotary arm is limited to the working position by a limit block, which is installed on the shell. In the non-working position, the infrared camera is located on the other side of the shell.

[0012] Furthermore, a slot is provided on the rotary arm, and the slot is used to cooperate with the limit block to limit the infrared camera on the rotary arm to its working position.

[0013] Furthermore, the shell includes a shell body and a cover, the shell body is used to install the rotary arm, the limit block and the sound source detection component and to accommodate the cable of the sound source detection component, and the cover is used to close the shell body.

[0014] Furthermore, the sound source localization device also includes a clip, the swing arm includes a joint for connecting the swing arm to the shell body, the joint passes through the shell body, the clip is engaged with the joint in the shell body and contacts with the shell body to achieve the connection between the swing arm and the shell body, and the clip is limited by the cover.

[0015] Furthermore, the rotary arm includes a box-shaped portion for accommodating the infrared camera, and the box-shaped portion is provided with a cable fixer for fixing the cable of the infrared camera.

[0016] Furthermore, a wiring groove for guiding the cables of the infrared camera is provided on the rotary arm.

[0017] Furthermore, the cover is L-shaped and includes a cover plate and an insert plate. Slide grooves for the insert plate to slide are processed on the two side walls of the shell body, and the cover plate covers the side walls of the shell body.

[0018] The advantages of the utility model are:

[0019] 1. The sound source localization device based on acoustic-optical-infrared fusion of the utility model, in addition to the array panel fixed on the shell and the optical camera fixed to the center of the array panel, also includes an infrared camera, which is arranged on a rotary arm, the rotary arm is connected to the shell and can rotate between a working position and a non-working position relative to the shell. In the working position, the infrared camera is located at the center of the microphone array, and can collect infrared images and superimpose the acoustic cloud map. In the non-working position, the infrared camera is located on the other side of the shell, and the optical camera can be switched to collect optical images for image superposition. Therefore, in addition to installing an optical camera at the center of the array panel, the utility model also introduces an infrared camera, so that infrared images can be collected in an environment without light or low light or in the case of a specific sound source type (such as invisible but warm gas) to superimpose with the acoustic cloud map, and the positioning result is more intuitive and clear; at the same time, a conversion rotary arm is added to the infrared camera, which can quickly complete the conversion between the optical camera and the infrared camera without complicated disassembly and replacement operations, and also avoids the camera position offset that may be caused during the operation, ensuring the positioning result.

[0020] 2. A card slot is provided on the swing arm. The limit block and the card slot can more reliably ensure that the infrared camera is at the center coordinate position of the array, ensure the accuracy and uniformity of the image position collected by each detection element, thereby improving the accuracy of sound source positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The features and advantages of the present invention will become more easily understood through the following description with reference to the accompanying drawings, which are not drawn to scale and some features are enlarged or reduced to show details of specific components. In the accompanying drawings:

[0022] Figure 1 It is a schematic stereoscopic diagram of the sound source localization device based on acoustic, optical and infrared fusion of the utility model. In (1), the infrared camera is in a working position, and in (2), the infrared camera is in a non-working position;

[0023] Figure 2It is a schematic three-dimensional diagram of the housing in the utility model;

[0024] Figure 3 is a schematic front view of the array panel in the utility model;

[0025] Figure 4 Schematic stereograms of the rotary arm of the present invention, (1) and (2) respectively show different viewing angles;

[0026] Figure 5 It is a partial enlarged stereogram of the present invention when the rotary arm and the limit block are in cooperation;

[0027] Figure 6 It is a partially enlarged stereoscopic view showing the buckle in the utility model;

[0028] Figure 7 2 is a schematic diagram of the positioning system design when the utility model is used to locate the sound source, (A) and (B) are respectively the positioning results obtained by using the microphone array and the optical camera in an illuminated environment and the positioning results obtained by using the microphone array and the infrared camera in a low-light environment.

[0029] In the figure: 1-shell, 101-shell body, 102-cover, 103-opening, 104-cover plate, 105-plug plate, 106-slide groove; 2-swing arm, 201-slot, 202-box-shaped part, 203-cable holder, 204-wiring groove, 205-annular groove; 3-limiting block; 4-array panel, 401-microphone array element, 402-center hole, 403-screw hole; 5-optical camera; 6-infrared camera; 7-buckle; 8-connector; P-sound source grid plane. DETAILED DESCRIPTION

[0030] The present invention will be described in detail below with reference to the accompanying drawings by means of exemplary embodiments of the present invention. It should be noted that the following detailed description of the present invention is only for the purpose of illustration, and is not intended to limit the present invention.

[0031] The utility model provides a sound source localization device based on acoustic-optical-infrared fusion, which is used to locate the sound source through the combination of a microphone array, an optical camera and an infrared camera. The microphone array collects the sound signal of the sound source to generate an acoustic cloud map, the optical camera is used to capture the optical image and superimpose it with the acoustic cloud map, and the infrared camera can collect images in a dark environment or a specific sound source type, such as invisible but warm gas, to superimpose it with the acoustic cloud map. Therefore, any superposition of the three images of acoustic, optical and infrared can be achieved.

[0032] First, the overall reference Figure 1As an exemplary embodiment of the utility model, a sound source positioning device based on acoustic-optical-infrared fusion includes a shell 1, a swing arm 2, a limit block 3 and a sound source detection component, and the sound source detection component includes an array panel 4, an optical camera 5 and an infrared camera 6, wherein the shell 1 is used to support the swing arm 2, the limit block 3 and the sound source detection component, and a microphone array for collecting sound signals is arranged on the array panel 4. The collected signals can be processed by an FPGA (Field Programmable Gate Array) device and then transmitted to a PC by an Ethernet module and an acoustic image is generated according to an algorithm, and optical image information is collected by an ordinary optical camera 5 or infrared image information is collected by an infrared camera 6, to create an acoustic image superimposed on an ordinary or infrared image and displayed on a PC, thereby realizing visualization of an acoustic fault target (sound source point).

[0033] See now Figure 2 In some embodiments, the shell 1 includes a shell body 101 and a cover 102. The shell body 101 is used to install the swing arm 2, the limit block 3 and the sound source detection component and to accommodate the cables of the sound source detection component. An opening 103 for the cables of the FPGA and the detection component to pass through may be opened on the shell body 101. The cover 102 is used to close the shell body 101.

[0034] In particular, the cover 102 can be L-shaped and include a cover plate 104 and an insert plate 105. The two side walls of the shell body 101 are processed with sliding grooves 106 for the insert plate 105 to slide. The cover plate 104 covers the side walls of the shell body 101. After the various components of the device are installed in place on the shell body 101, the insert plate 105 of the cover 102 can be inserted into the two sliding grooves 106 of the shell body 101 and slid to the bottom of the sliding grooves, and then the cover plate 104 of the cover 102 can be connected to the shell body 101 by threaded connection.

[0035] Combination Figure 3 , the array panel 4 is fixed to one side of the shell 1, in particular, by passing screws through screw holes 403 on the array panel 4 and corresponding holes on the shell 1. A microphone array composed of microphone array elements 401 is arranged on the array panel 4. The microphone array is arranged so that its center position is approximately at the center of the array panel 4. The arrangement structure and number of the microphone array elements 401 are not particularly limited. Preferably, 64 array elements are arranged in a spiral array formation shown in the figure. Random arrays, ring arrays and other microphone array formations may also be used.

[0036] The optical camera 5 is used to collect optical images and is fixedly connected to the center position of the microphone array, and can be optionally installed in the center hole 402 on the array panel 4.

[0037] Now combine Figure 4The infrared camera 6 is arranged at one end of the rotary arm 2 for collecting infrared images. The other end of the rotary arm 2 is connected to the housing 1 and can rotate relative to the housing 1 between a working position and a non-working position. The working position is as follows: Figure 1 As shown in (1), the infrared camera 6 is located at the center of the microphone array, shielding the optical camera 5, and the swing arm 2 is limited to the working position under the action of the limit block 3, and the limit block 3 is installed on the housing 1. The non-working position is as shown in FIG. Figure 1 As shown in (2), the infrared camera 6 is located on the other side of the housing 1, that is, the side of the housing 1 opposite to the side where the array panel 4 is located, and no longer blocks the optical camera 5 and allows it to collect images. The arm 2 is roughly L-shaped, and in the working position, the two legs of the arm 2 are parallel to the two adjacent surfaces of the housing 1.

[0038] In a specific embodiment of the utility model, in order to facilitate the installation of the circuit board of the infrared camera 6, the arm 2 includes a box-shaped portion 202 for accommodating the infrared camera 6, and a cable holder 203 for fixing the cable of the infrared camera 6 is provided on the box-shaped portion 202. In addition, a wiring groove 204 for guiding the cable of the infrared camera 6 can be provided on the arm 2, and the wiring groove 204 can extend from one end of the arm 2 where the infrared camera 6 is installed to the other end of the arm 2, and finally lead to the housing 1, so that all the cables are stored in the housing 1, which is beneficial to the overall aesthetics of the device.

[0039] In another embodiment, Figure 5 As can be seen more clearly in the figure, a slot 201 is provided on the swing arm 2, and the slot 201 is used to cooperate with the limit block 3 to limit the infrared camera 6 on the swing arm 2 to its working position. The slot 201 can be located at the connection between the two legs of the swing arm 2, and the limit block 3 is located at the edge between the adjacent surfaces of the shell 1. In this structure, the slot 201 is easy to engage with the limit block 3, which is convenient for realizing accurate positioning of the swing arm 2 and the infrared camera thereon, thereby ensuring that the infrared camera is in the correct position to collect the infrared image at the center position of the array. Therefore, the limit block and the slot can more reliably ensure that the infrared camera is in the center coordinate position of the array, ensure the accuracy and uniformity of the image collection position of each detection element, and thus improve the accuracy of sound source positioning.

[0040] Further, refer to Figure 6 The sound source localization device of the present invention further includes a buckle 7. The swing arm 2 includes a joint 8 for connecting the swing arm 2 to the shell body 101. The joint 8 passes through the shell body 101. The buckle 7 is engaged with the joint 8 in the shell body 101 and contacts the shell body 101. Figure 4The annular groove 205 shown in (2) of the embodiment is inserted into the annular groove 205 to realize the connection between the swing arm 2 and the housing 1, and the buckle 7 is limited by the cover 102. Specifically, during installation, the connector 8 passes through the hole on the housing 1, and then the buckle 7 is inserted into the annular groove 205, thereby limiting the lateral displacement of the buckle 7 and the swing arm 2. After the buckle 7 is fully inserted, the top is flush with the top of the housing 7. After the cover 102 of the housing 1 is installed, the longitudinal displacement of the buckle 7 and the swing arm 2 can be limited. At this point, the swing arm 2 is connected to the housing 1, and only the degree of freedom of rotation around the housing 1 is retained.

[0041] Last reference Figure 7 , which shows an experiment using the sound source localization device based on acoustic-optical-infrared fusion of the utility model. In the experiment, the sound source uses two speakers to make sounds. In the figure, X, Y, and Z are the coordinate axes of the acoustic algorithm space coordinate system, the origin of the coordinate is located at the center of the microphone array, and P is the sound source plane. The FPGA used in the experiment is the EP4CE75F23I7 model chip of the Cyclone IV E series of Intel. The chip resources are: 75K logic resources, 343KByte RAM memory, 400 9-bit hardware multipliers, 4 PLL phase-locked loops, 20 global clock resources, etc. The microphone array used includes 64 array elements, each of which uses a digital MEMS microphone of model SPH0641LU4H-1 with a frequency response range of 100 to 80kHz and is arranged in a multi-arm spiral array. The ordinary optical camera is the Jierui Microtong H264 industrial camera with an effective pixel of 1920×1080P and a maximum frame rate of 30fps. The infrared camera is the Dali SD series infrared thermal imaging module with a temperature measurement range of -50° to 350° and a viewing angle of 60°*45°.

[0042] Figure 7 The experimental result (A) in the figure is the superposition result of the ordinary optical image and the acoustic cloud map in the illuminated environment, and the arm 2 is rotated to its non-working position in the experiment; the experimental result (B) is the superposition result of the infrared image and the acoustic cloud map in the low-light environment, and the arm 2 is in its working position in cooperation with the limit block 3 in the experiment. It can be seen that the present invention can realize the superposition of the environmental image and the acoustic cloud map, whether in an illuminated environment or in a poor lighting condition, so as to intuitively and clearly indicate the location of the sound source in the environment.

[0043] As described above, in addition to installing an optical camera in the center of the array panel, the utility model also introduces an infrared camera. Therefore, infrared images can be collected in no-light or low-light environments or in specific sound source types (such as invisible but warm gas) to superimpose with the acoustic cloud map, and the positioning results are more intuitive and clear. At the same time, a conversion arm is added to the infrared camera to quickly complete the conversion between the optical camera and the infrared camera without the need for complicated disassembly and replacement operations, and the camera position deviation that may be caused during the operation is avoided, thereby ensuring the positioning result.

[0044] The features mentioned and / or shown in the above description of the exemplary embodiments of the utility model can be combined in the same or similar manner into one or more other embodiments, combined with the features in other embodiments or replace the corresponding features in other implementations. The technical solutions obtained by these combinations or replacements should also be deemed to be included in the protection scope of the utility model.

Claims

1. A sound source localization device based on acoustic, optical and infrared fusion, characterized in that: It includes a shell, a rotary arm, a limit block and a sound source detection component, wherein the sound source detection component includes an array panel, an optical camera and an infrared camera; The shell is used to support the rotary arm, the limit block and the sound source detection assembly; A microphone array composed of microphone array elements is arranged on the array panel for collecting acoustic signals, and the array panel is fixed to one side of the housing; The optical camera is fixedly connected to the center of the microphone array and is used to collect optical images; The infrared camera is arranged at one end of the rotary arm for collecting infrared images. The other end of the rotary arm is connected to the shell and can rotate relative to the shell between a working position and a non-working position. In the working position, the infrared camera is located at the center of the microphone array, and the rotary arm is limited to the working position under the action of the limit block, and the limit block is installed on the shell. In the non-working position, the infrared camera is located on the other side of the shell.

2. The sound source localization device based on acoustic, optical and infrared fusion according to claim 1 is characterized in that: The rotary arm is provided with a slot, and the slot is used to cooperate with the limit block to limit the infrared camera on the rotary arm to its working position.

3. The sound source localization device based on acoustic, optical and infrared fusion according to claim 1 or 2, characterized in that: The shell comprises a shell body and a cover, wherein the shell body is used to install the rotary arm, the limit block and the sound source detection assembly and to accommodate the cable of the sound source detection assembly therein, and the cover is used to close the shell body.

4. The sound source localization device based on acoustic, optical and infrared fusion according to claim 3 is characterized in that: It also includes a buckle, the swing arm includes a joint for connecting the swing arm to the shell body, the joint passes through the shell body, the buckle is engaged with the joint in the shell body and contacts the shell body to achieve the connection between the swing arm and the shell body, and the buckle is limited by the cover.

5. The sound source localization device based on acoustic, optical and infrared fusion according to claim 1 or 2, characterized in that: The rotary arm comprises a box-shaped portion for accommodating the infrared camera, and a cable fixer for fixing the cable of the infrared camera is provided on the box-shaped portion.

6. The sound source localization device based on acoustic, optical and infrared fusion according to claim 1 or 2, characterized in that: The rotary arm is provided with a wiring groove for guiding the cable of the infrared camera.

7. The sound source localization device based on acoustic, optical and infrared fusion according to claim 3 is characterized in that: The cover is L-shaped and comprises a cover plate and an insert plate. Slide grooves for the insert plate to slide are processed on the two side walls of the shell body, and the cover plate covers the side walls of the shell body.

Images