Hearing system based on double-hemisphere array

Abstract

The invention discloses an auditory system based on a double-hemisphere array, relates to the technical field of unmanned system perception, and can realize low-frequency noise broadband orientation processing, so that a single small-scale hemisphere array can realize detection, identification and orientation functions, and target positioning can be further realized through intersection of orientation results of the two hemisphere arrays. The head and the tail of the back of the unmanned system are respectively provided with a miniature hemispherical sound array, on one hand, the array aperture can be increased, and the direction finding stability is improved; and on the other hand, the double arrays are similar to human ears, so that the spatial sound field information can be better sensed, and the positioning performance is improved. The limitation of the Rayleigh limit of a traditional array orientation method is broken through, that is, distortionless broadband beam forming is achieved, and particularly, low-frequency signal orientation in a space of 360 degrees in the horizontal direction and 90 degrees in the pitching direction can be achieved through a small-scale hemisphere array.

Description

Technical Field

[0001] The present invention relates to the field of unmanned system perception technology, and in particular to a hearing system based on a dual-hemispherical array that can realize the detection, recognition and directional ranging functions of 360-degree circumferential target sounds. Background Art

[0002] The microphone array can be used as an intelligent auditory system for unmanned systems to realize detection functions such as acoustic signal detection, recognition, and positioning of typical targets.

[0003] The linear array cannot distinguish the directions on the left and right sides of the linear array, and has good directional performance only within the range of ±60° of the perpendicular midline on one side of the linear array. The array manifold of the uniform linear array has a standard Vandermonde form, so there are a large number of optimized beamforming methods that utilize this feature, and a relatively complete sparse linear array theory has been developed on this basis.

[0004] Planar arrays are widely used. They have good beam performance in the conical area in front of the plane and are usually used for acoustic imaging. Currently, the most common planar arrays are multi-arm logarithmic spiral arrays or sunflower arrays, which have better beam performance in a wider frequency band.

[0005] The spherical array can measure the direction of the horizontal and pitch directions of 360°. The spherical array with approximately uniform distribution can perform modal domain beamforming in the spherical harmonic domain to achieve frequency-independent beam directivity within the analysis band. However, it is usually only necessary to monitor the corresponding space in the upper hemisphere, that is, the hemispherical array can meet the acoustic detection requirements.

[0006] A single acoustic array is usually only used for passive direction finding, and cannot measure distance and position. Traditional unmanned equipment is mostly single array, and traditional single array is only used for voice recognition and voice orientation, and can only measure direction or with the help of other means such as laser ranging. Summary of the invention

[0007] In view of the above problems, the present invention provides a hearing system based on a dual-hemispherical array for overcoming the above problems or at least partially solving the above problems. The system can be used to identify sirens, drones, gunshots, explosions, etc., and orient and locate the sound source.

[0008] The present invention provides the following scheme:

[0009] A hearing system based on a dual-hemisphere array, comprising:

[0010] Two hemispherical microphone arrays, the two hemispherical microphone arrays are arranged at intervals on the back of the unmanned system; the hemispherical microphone arrays include a hemispherical structure body and 16 microphones evenly distributed and sampled on the hemispherical structure body;

[0011] A controller, the controller being communicatively connected to the two hemispherical microphone arrays;

[0012] The controller is used to perform the following operations:

[0013] Using an energy detection algorithm to combine the energy of the sampled signals acquired by one or both of the two hemispherical microphone arrays, determining whether the sampled signals contain a target signal;

[0014] After determining that the target signal exists, using a sound category recognition algorithm to identify the sound type corresponding to the target signal;

[0015] Based on the received signal model of the hemispherical acoustic array, two hemispherical microphone arrays and the sound source are respectively oriented to obtain a first directional result and a second directional result;

[0016] The intersection positioning algorithm is used to combine the first orientation result and the second orientation result to obtain the positioning information of the sound source.

[0017] Preferably, the hemispherical structural body comprises a rigid hemispherical body with a radius of 42 mm.

[0018] Preferably, the maximum modal order of the hemispherical microphone array is N=3, and the corresponding upper limit of the analysis frequency is 3867 Hz.

[0019] Preferably, the energy detection algorithm includes utilizing the energy of the sampled signal to compare with a preset threshold value to determine whether the sampled signal contains the target signal.

[0020] Preferably: the statistics and decision criteria of the energy detection algorithm are expressed by the following formula:

[0021]

[0022] Where: T is the detection statistic, x(n) is the sampled signal, N is the number of sampling points, γ is the decision threshold, H1 is the target signal that exceeds the threshold, and H0 is the target signal that does not exist.

[0023] Preferably: the sound category recognition algorithm includes a deep learning recognition and classification network for typical target sounds obtained by constructing a deep learning network as a basis.

[0024] Preferably, the two hemispherical microphone arrays and the sound source are oriented respectively using spherical harmonic domain based on the received signal model of the hemispherical acoustic array, so as to obtain the first directional result and the second directional result.

[0025] Preferably, the power spectrum of the far-field source positioning in the spherical harmonic domain is expressed by the following formula:

[0026]

[0027] Where: Ψ represents the angle corresponding to the spectrum peak.

[0028] Preferably: the first orientation result includes a first arrival angle, and the second orientation result includes a second arrival angle; a set of overdetermined equations are established using the first arrival angle and the second arrival angle and the least squares method is used to solve them to obtain the positioning information of the sound source.

[0029] Preferably, the overdetermined equation is expressed by the following formula in the manner of Ax=b:

[0030]

[0031] The further z that can be obtained is:

[0032]

[0033] Where: d i (z=z i ) means d i Projected magnitude on the horizontal plane.

[0034] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] The embodiment of the present application provides a hearing system based on a dual-hemispherical array, which can achieve the decoupling of frequency and azimuth information, break through the Rayleigh limit of the traditional array directional method, that is, achieve distortion-free broadband beamforming, and especially can use a small-scale hemispherical array to achieve low-frequency signal directional in a horizontal 360-degree and a pitch 90-degree space. At the same time, the modal domain processing of the hemispherical array can extract the broadband sound signal of the target without distortion, while the spatial domain filtering of the traditional array signal is related to the frequency, resulting in distortion in the broadband signal extraction.

[0036] On this basis, small-scale dual-hemispherical array deployment is realized to achieve acoustic independent positioning function. Combined with the detection and recognition functions of a single acoustic array, intelligent perception of environmental sound field information under a small-scale array is realized. An intelligent auditory system can be provided for unmanned systems to achieve the detection, recognition and directional ranging functions of 360-degree target sounds, and combined with other information such as video modules, the unmanned system can perceive the multi-physical field information of the surrounding environment and provide effective information support for its decision-making and actions.

[0037] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a schematic diagram of a hearing system based on a dual-hemisphere array provided by an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of a dual array positioning effective area provided by an embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of the structure of a hemispherical microphone array provided by an embodiment of the present invention;

[0042] Figure 4 It is a structural block diagram of a sound type recognition algorithm provided by an embodiment of the present invention.

[0043] In the figure: hemispherical microphone array 1, unmanned system 2. DETAILED DESCRIPTION

[0044] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0045] See also Figure 1 , is a hearing system based on a dual-hemisphere array provided by an embodiment of the present invention, such as Figure 1 As shown, the system may include:

[0046] Two hemispherical microphone arrays 1 are arranged at intervals on the back of the unmanned system 2; the hemispherical microphone array 1 includes a hemispherical structure body and 16 microphones evenly sampled and distributed on the hemispherical structure body; in specific implementation, the embodiment of the present application can provide that the hemispherical structure body includes a rigid hemispherical body with a radius of 42 mm. The maximum modal order of the hemispherical microphone array is N=3, and the corresponding upper limit of the analysis frequency is 3867 Hz.

[0047] A controller, the controller being communicatively connected to the two hemispherical microphone arrays 1;

[0048] The controller is used to perform the following operations:

[0049] An energy detection algorithm is used to combine the energy of the sampling signals obtained by one or both of the two hemispherical microphone arrays to determine whether the sampling signal contains the target signal; the energy detection algorithm includes using the energy of the sampling signal to compare with a preset threshold value to determine whether the sampling signal contains the target signal.

[0050] The statistics and decision criteria of the energy detection algorithm are expressed as follows:

[0051]

[0052] Where: T is the detection statistic, x(n) is the sampled signal, N is the number of sampling points, γ is the decision threshold, H1 is the target signal that exceeds the threshold, and H0 is the target signal that does not exist.

[0053] After determining the existence of the target signal, a sound category recognition algorithm is used to identify the sound type corresponding to the target signal; in specific implementation, the embodiment of the present application can provide that the sound category recognition algorithm includes a deep learning recognition and classification network for typical target sounds obtained by constructing a deep learning network as the basis.

[0054] The receiving signal model based on the hemispherical acoustic array orients the two hemispherical microphone arrays and the sound source respectively to obtain the first orientation result and the second orientation result; the receiving signal model based on the hemispherical acoustic array orients the two hemispherical microphone arrays and the sound source respectively using the spherical harmonic domain to obtain the first orientation result and the second orientation result. Further, the power spectrum of the far-field source positioning in the spherical harmonic domain is expressed by the following formula:

[0055]

[0056] Where: Ψ represents the angle corresponding to the spectrum peak.

[0057] The intersection positioning algorithm is used to combine the first orientation result and the second orientation result to obtain the positioning information of the sound source. In specific implementation, the first orientation result includes a first arrival angle, and the second orientation result includes a second arrival angle; a set of overdetermined equations is established using the first arrival angle and the second arrival angle, and the least squares method is used to solve the equations to obtain the positioning information of the sound source. Furthermore, the overdetermined equation is expressed by the following formula in the manner of Ax=b:

[0058]

[0059] The further z that can be obtained is:

[0060]

[0061] Where: d i(z=z i ) means d i Projected magnitude on the horizontal plane.

[0062] The hearing system based on the dual hemispherical array provided in the embodiment of the present application can realize broadband directional processing of low-frequency noise, so that a single small-scale hemispherical array can realize detection, recognition and directional functions, and further, the target positioning can be realized by the intersection of the directional results of the two hemispherical arrays. By installing a miniature hemispherical acoustic array at the head and tail of the back of the unmanned system, on the one hand, the array aperture can be increased and the direction finding stability can be improved; on the other hand, the dual array is similar to the human ears, which can better perceive the spatial sound field information and improve the positioning performance.

[0063] It is understandable that when the directional results of two arrays are used for directional intersection positioning, when the target is close to the direction of the line connecting the two arrays, the positioning deviation is too large. Therefore, it is more suitable for sound source positioning near the direction perpendicular to the direction of the array connection. Figure 2 The sector area shown.

[0064] The hearing system based on the dual-hemispherical array provided in the embodiment of the present application can realize 360° broadband orientation of low-frequency signals based on a small-scale (less than 10 cm) hemispherical array. The orientation of low-frequency signals of traditional arrays requires a large-scale array (meter level). In addition, the dual-hemispherical acoustic array realizes the intersection positioning of circumferential targets under limited aperture. Traditional unmanned equipment mostly uses a single array and can only be oriented or measured by other means such as laser ranging.

[0065] The following is a detailed introduction to an example of a hearing system based on a dual-hemispherical array provided in an embodiment of the present application being installed on a robot dog.

[0066] A spherical array is designed by using a rigid sphere with a radius of 42 mm and 32 microphones distributed approximately evenly. The 16 microphones and hemispherical structure in the upper hemisphere are taken to obtain the required 16-element hemispherical array, as shown in Figure 3 As shown. At this time, the maximum modal order N = 3, and the corresponding upper limit of the analysis frequency is 3867Hz. The main energy of typical target noise is concentrated in the medium and low frequency bands, especially the band below 2000Hz, so the designed hemispherical array meets the acoustic detection requirements. When in use, a miniature hemispherical acoustic array is installed at the head and tail of the robot dog's back.

[0067] When performing target signal detection, the sampling signal of one micro-hemispherical acoustic array can be obtained, or the sampling signals of two micro-hemispherical acoustic arrays can be obtained simultaneously. The energy detection algorithm is combined with the sampling signal to determine whether the adopted signal includes the target signal. The energy detection algorithm is a method that compares the energy of the signal within a specific time with a preset threshold value to determine whether the target signal exists in the signal segment. The statistics and decision criteria of the energy detection algorithm can be expressed as:

[0068]

[0069] Where: T is the detection statistic, x(n) is the sampled signal, N is the number of sampling points, γ is the decision threshold, H1 is the target signal that exceeds the threshold, and H0 is the target signal that does not exist.

[0070] After determining that the target signal is included, it is necessary to identify the sound type of the signal. The embodiment of the present application uses deep learning networks such as convolutional neural networks (CNN) as the basis to build a deep learning recognition and classification network for typical target sounds. The structure diagram of the sound type recognition algorithm is shown in the figure. Figure 4 As shown in the figure, it includes model training and testing. Model training is the core, including modules such as data preparation, feature extraction, model building and training.

[0071] After the sound type is determined, the sound source (target) can be oriented respectively through two hemispherical microphone arrays. After the orientation is completed, two directional results corresponding to the two hemispherical microphone arrays can be obtained. In the specific implementation, the system provided by the present application adopts a receiving signal model based on a hemispherical sound array and uses spherical harmonic domain MVDR (SH-MVDR) for orientation. The MVDR method minimizes the impact of interference from the direction of undesired DOA on the array while maintaining a uniform gain in the observation direction. Based on the same principle, the power spectrum of SH-MVDR far-field source positioning can be expressed as:

[0072]

[0073] Similar to the SH-MUSIC sound source orientation method, SH-MVDR also calculates the spatial spectrum in all directions by traversing the spatial angles. The angle Ψ corresponding to the peak value of the spectrum is the direction of the sound source.

[0074] After obtaining the two directional results, the sound source can be located. Since two directional results are obtained, the intersection positioning method can be used to locate the sound source. In specific implementation, intersection positioning is the most common geometric positioning technology. Here, the angle of arrival (AOA) information of the two hemispherical arrays is used to establish a set of overdetermined equations and solve them using the least square method (LSM). The overdetermined equations containing AOA information can be written as Ax=b:

[0075]

[0076] The further z that can be obtained is:

[0077]

[0078] Where: d i (z=z i ) means d i Projected magnitude on the horizontal plane.

[0079] The problem with this approach is that the estimation of the vertical coordinate z of the sound source is coupled with the errors of x and y, but the advantage is that only two array AOAs are needed to obtain a three-dimensional coordinate estimate. It is worth noting that in order to make A or b have physical meaning when taking any value in the horizontal angle range, the elements of A or b should avoid being represented by tangent or cotangent functions. At the same time, in order to avoid singularity of A or b, |Ψ i |Not all are π / 2.

[0080] In summary, the auditory system based on the dual hemispherical array provided by the present application can realize the decoupling of frequency and azimuth information, break through the Rayleigh limit of the traditional array directional method, that is, realize distortion-free broadband beamforming, especially can use the small-scale hemispherical array to realize the low-frequency signal directional in the horizontal 360 degrees and the pitch 90 degrees space. At the same time, the modal domain processing of the hemispherical array can extract the broadband sound signal of the target without distortion, while the spatial domain filtering of the traditional array signal is related to the frequency, resulting in distortion in the broadband signal extraction.

[0081] On this basis, small-scale dual-hemispherical array deployment is realized to achieve acoustic independent positioning function. Combined with the detection and recognition functions of a single acoustic array, intelligent perception of environmental sound field information under a small-scale array is realized. An intelligent auditory system can be provided for unmanned systems to achieve the detection, recognition and directional ranging functions of 360-degree target sounds, and combined with other information such as video modules, the unmanned system can perceive the multi-physical field information of the surrounding environment and provide effective information support for its decision-making and actions.

[0082] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0083] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0084] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A hearing system based on a dual-hemisphere array, characterized in that: include: Two hemispherical microphone arrays, the two hemispherical microphone arrays are arranged at intervals on the back of the unmanned system; the hemispherical microphone arrays include a hemispherical structure body and 16 microphones evenly distributed and sampled on the hemispherical structure body; A controller, the controller being communicatively connected to the two hemispherical microphone arrays; The controller is used to perform the following operations: Using an energy detection algorithm to combine the energy of the sampled signals acquired by one or both of the two hemispherical microphone arrays, determining whether the sampled signals contain a target signal; After determining that the target signal exists, using a sound category recognition algorithm to identify the sound type corresponding to the target signal; Based on the received signal model of the hemispherical acoustic array, two hemispherical microphone arrays and the sound source are respectively oriented to obtain a first directional result and a second directional result; The intersection positioning algorithm is used to combine the first orientation result and the second orientation result to obtain the positioning information of the sound source.

2. The hearing system based on the dual hemisphere array according to claim 1, characterized in that: The hemispherical structural body comprises a rigid hemispherical body with a radius of 42 mm.

3. The hearing system based on a dual hemisphere array according to claim 1, characterized in that: The maximum modal order of the hemispherical microphone array is N=3, and the corresponding upper limit of the analysis frequency is 3867 Hz.

4. The hearing system based on a dual hemisphere array according to claim 1, characterized in that: The energy detection algorithm includes using the energy of the sampled signal to compare with a preset threshold value to determine whether the sampled signal contains the target signal.

5. The hearing system based on the dual hemisphere array according to claim 4, characterized in that: The statistics and decision criteria of the energy detection algorithm are expressed as follows: Where: T is the detection statistic, x(n) is the sampled signal, N is the number of sampling points, γ is the decision threshold, H1 is the target signal that exceeds the threshold, and H0 is the target signal that does not exist.

6. The hearing system based on a dual hemisphere array according to claim 1, characterized in that: The sound category recognition algorithm includes a deep learning recognition and classification network for typical target sounds that is constructed based on a deep learning network.

7. The hearing system based on a dual hemisphere array according to claim 1, characterized in that: The receiving signal model based on the hemispherical acoustic array uses the spherical harmonic domain to orient two hemispherical microphone arrays and the sound source respectively, so as to obtain a first orientation result and a second orientation result.

8. The hearing system based on dual hemisphere array according to claim 7, characterized in that: The power spectrum of the far-field source positioning in the spherical harmonic domain is expressed by the following formula: Where: Ψ represents the angle corresponding to the spectrum peak.

9. The hearing system based on dual hemisphere array according to claim 1, characterized in that: The first orientation result includes a first arrival angle, and the second orientation result includes a second arrival angle; a set of overdetermined equations is established using the first arrival angle and the second arrival angle, and the least squares method is used to solve them to obtain the positioning information of the sound source.

10. The hearing system based on dual hemisphere array according to claim 9, characterized in that: The overdetermined equation is expressed as follows in the form of Ax=b: The further z that can be obtained is: Where: d i (z=z i ) means d i Projected magnitude on the horizontal plane.

Images