Sound source positioning method and device, and directivity sound pickup method and device thereof

Abstract

The present application relates to the technical field of sound source positioning, and particularly provides a sound source positioning method, device, and a directivity sound pickup method and device, aiming to solve the technical problem of low positioning accuracy of a single-size microphone array for a sound source, inability to obtain complete sound source signals, and difficulty in meeting user requirements. To this end, the sound source positioning method includes the following steps: obtaining a first microphone array and a second microphone array, the aperture of the first microphone array being different from that of the second microphone array; determining a first positioning result corresponding to the first microphone array and a second positioning result corresponding to the second microphone array based on the first microphone array and the second microphone array; and determining a sound source direction based on the first positioning result and the second positioning result.

Description

Technical Field

[0001] This invention relates to the field of sound source localization, and specifically provides a sound source localization method, apparatus, and directional sound pickup method and apparatus. Background Technology

[0002] Microphone array technology is widely used in consumer electronics products. However, due to size limitations, these products typically employ small-aperture arrays for directional sound pickup and other algorithmic operations to process the sound, as seen in devices like smartphones, smart speakers, and smart remote controls. With the development of the automotive industry, the demand for sound pickup in vehicles has increased, leading to greater application of microphone array technology in automobiles. However, current methods for sound source localization in vehicles either use large-aperture or small-aperture microphone arrays. Such single-size (aperture) microphone arrays offer low accuracy in sound source localization, only picking up signals within a certain frequency band and failing to capture the complete sound source signal. This method is insufficient to meet user needs.

[0003] Accordingly, a new sound source localization scheme is needed in this field to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned shortcomings, this invention is proposed to provide a solution, or at least a partial solution, to the technical problems of existing single-size microphone arrays having low accuracy in sound source localization, inability to acquire complete sound source signals, and difficulty in meeting user needs. This invention provides a sound source localization method, apparatus, and a directional sound pickup method and apparatus thereof.

[0005] In a first aspect, the present invention provides a sound source localization method, comprising the following steps: acquiring a first microphone array and a second microphone array, wherein the first microphone array and the second microphone array have different apertures; determining a first localization result corresponding to the first microphone array and a second localization result corresponding to the second microphone array based on the first microphone array and the second microphone array respectively; and determining the sound source direction based on the first localization result and the second localization result.

[0006] In one embodiment, determining the sound source direction based on the first positioning result and the second positioning result includes: determining whether the first positioning result and the second positioning result are consistent; if so, taking the direction corresponding to either the first positioning result or the second positioning result as the sound source direction; if not, determining the sound source direction based on the first positioning result, the second positioning result, the first microphone array, and the second microphone array.

[0007] In one embodiment, determining the sound source direction based on the first positioning result, the second positioning result, the first microphone array, and the second microphone array includes: obtaining a first direction deviation value according to the directions corresponding to the first positioning result and the second positioning result respectively, and using the first direction deviation value as a first difference; determining whether the first difference is less than a preset value; if the first difference is less than the preset value, determining a third positioning result corresponding to the first microphone array based on the first microphone array and a fourth positioning result corresponding to the second microphone array based on the second microphone array, and determining the sound source direction based on the third positioning result and the fourth positioning result; if the first difference is not less than the preset value, performing cross-validation on the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array, obtaining a fifth positioning result corresponding to the first microphone array and a sixth positioning result corresponding to the second microphone array based on the cross-validation, and determining the sound source direction based on the fifth positioning result and the sixth positioning result.

[0008] In one embodiment, determining the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array based on the first microphone array and the second microphone array respectively includes: calculating the angle of arrival (AOA) of the sound source corresponding to each microphone in the first microphone array and the second microphone array respectively based on the DOA algorithm, using the largest AOA among all the calculated AOA of the first microphone array as the first positioning result, and using the largest AOA among all the calculated AOA of the second microphone array as the second positioning result; determining the third positioning result corresponding to the first microphone array based on the first microphone array and the fourth positioning result corresponding to the second microphone array based on the second microphone array includes: calculating the AOA of the sound source corresponding to each microphone in the first microphone array and the second microphone array based on the TDOA algorithm, using the largest AOA among all the calculated AOA of the first microphone array as the third positioning result, and using the largest AOA among all the calculated AOA of the second microphone array as the fourth positioning result; determining the sound source direction based on the third positioning result and the fourth positioning result includes: using the direction corresponding to the largest positioning result among the third positioning result and the fourth positioning result as the sound source direction.

[0009] In one embodiment, cross-validating the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array, and obtaining a fifth positioning result corresponding to the first microphone array and a sixth positioning result corresponding to the second microphone array based on the cross-validation, includes: swapping the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array; calculating the time delay corresponding to the first microphone array and the second microphone array based on the swapped positioning results; and obtaining the sound source arrival angle corresponding to the first microphone array and the second microphone array based on the calculated time delay, so as to obtain the first microphone array and the sixth positioning result corresponding to the second microphone array respectively. The angles of arrival of the sound source corresponding to the two microphone arrays are respectively used as the fifth positioning result corresponding to the first microphone array and the sixth positioning result corresponding to the second microphone array; determining the sound source direction based on the fifth positioning result and the sixth positioning result includes: obtaining a second direction deviation value based on the directions corresponding to the fifth positioning result and the first positioning result respectively, and using the second direction deviation value as a second difference value; obtaining a third direction deviation value based on the directions corresponding to the sixth positioning result and the second positioning result respectively, and using the third direction deviation value as a third difference value; if the second difference value is less than the third difference value, the direction corresponding to the fifth positioning result is used as the sound source direction; if the second difference value is not less than the third difference value, the direction corresponding to the sixth positioning result is used as the sound source direction.

[0010] In a second aspect, the present invention provides a directional sound pickup method, comprising the following steps: determining the direction of a sound source using a sound source localization method; extracting a first sound source spectrum in the direction of the sound source from a first microphone array, and extracting a second sound source spectrum in the direction of the sound source from a second microphone array; and performing data fusion on the first sound source spectrum and the second sound source spectrum to obtain a time-domain signal of the sound source.

[0011] In a third aspect, the present invention provides a sound source localization device, including an acquisition module configured to acquire a first microphone array and a second microphone array, wherein the first microphone array and the second microphone array have different apertures; a determination module configured to determine a first localization result corresponding to the first microphone array and a second localization result corresponding to the second microphone array based on the first microphone array and the second microphone array, respectively; and a localization module configured to determine the sound source direction based on the first localization result and the second localization result.

[0012] In a fourth aspect, the present invention provides a directional sound pickup system, including a sound source localization device configured to determine the direction of a sound source; a spectrum extraction module configured to extract a first sound source spectrum in the direction of the sound source from a first microphone array and extract a second sound source spectrum in the direction of the sound source from a second microphone array; and a fusion module configured to perform data fusion on the first sound source spectrum and the second sound source spectrum to obtain a time-domain signal of the sound source.

[0013] In a fifth aspect, a vehicle is provided, including a vehicle body, at least two microphone arrays with different apertures disposed within the vehicle body, a processor, and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and executed by the processor to cause the vehicle to perform a sound source localization method.

[0014] In a sixth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the vehicle sound source localization method described in any of the preceding claims.

[0015] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0016] This invention determines the direction of the sound source based on the positioning results corresponding to the first and second microphone arrays with different array sizes (apertures), which makes up for the problem that the accuracy of determining the direction of the sound source is low when using a microphone array with a single array size, thus achieving accurate positioning of the sound source and meeting user needs.

[0017] When determining the direction of the sound source based on the positioning results corresponding to the first microphone array and the second microphone array, a first difference is first determined based on the first positioning result and the second positioning result. If the first difference is less than a preset value, the direction of the sound source is further determined based on the third positioning result determined by the first microphone array and the fourth positioning result determined by the second microphone array. If the first difference is not less than the preset value, the first positioning result and the second positioning result are cross-validated to determine the direction of the sound source. In this way, a more accurate direction of the sound source is determined under different conditions, improving the accuracy of sound source positioning.

[0018] Based on the determined sound source localization results, the first microphone array and the second microphone array are further used to perform directional sound pickup of the sound source in the direction of the sound source, thereby obtaining the complete time domain signal of the sound source and realizing ultra-wideband directional sound pickup. Attached Figure Description

[0019] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0020] Figure 1 This is a phase diagram of a large microphone array according to an embodiment of the present invention;

[0021] Figure 2 This is a phase diagram of a small microphone array according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart illustrating a sound source localization method according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of a far-field model and a near-field model according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of a sound source localization device according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the structure of a vehicle according to an embodiment of the present invention;

[0026] Figure 7 This is a schematic diagram illustrating the application of a microphone array in a vehicle according to an embodiment of the present invention;

[0027] List of reference numerals :

[0028] 11: Acquire module; 12: Determine module; 13: Locate module. Detailed Implementation

[0029] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0030] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0031] Currently, traditional methods for locating sound sources in vehicles either use large or small microphone arrays. These single-size microphone arrays have low accuracy in sound source location, only able to pick up sound signals within a certain frequency band, failing to capture the complete sound source signal and thus failing to meet user needs. Therefore, this application provides a sound source location method that determines the sound source direction based on the location results corresponding to a first microphone array and a second microphone array with different array sizes (apertures). This overcomes the problem of low accuracy in determining the sound source direction using a single-size microphone array, achieving precise sound source location and meeting user requirements.

[0032] Generally speaking, such as Figure 1 As shown, in a large microphone array, when the distance between the microphones in the array is greater than half the wavelength λ / 2 of the sound signal frequency, phase aliasing will occur, making it impossible to distinguish the phase difference between the two signals. Therefore, the maximum distance between any two microphones in the microphone array determines the upper limit of the resolvable frequency.

[0033] like Figure 2As shown, for small-sized microphone arrays, when the spacing between microphones in the array is too small, the phase difference between the signals picked up by the two microphones is too small for low-frequency signals. This makes it impossible to accurately calculate the phase difference during phase estimation, resulting in almost no improvement in the signal-to-noise ratio of low-frequency signals during directional sound pickup. Therefore, the minimum spacing of the microphone array determines the lower limit of the frequency range of the sound signals that can be processed. It is evident that a single-sized microphone array has low accuracy in sound source localization, and the results of directional sound pickup are also incomplete. Therefore, this application uses microphone arrays of different sizes to perform sound source localization.

[0034] The following section uses two microphone arrays of different sizes as examples to illustrate sound source localization in detail. (See appendix.) Figure 3 , Figure 3 This is a schematic flowchart illustrating the main steps of a sound source localization method according to an embodiment of the present invention. Figure 3 As shown, the sound source localization method in this embodiment of the invention mainly includes the following steps S101-S103.

[0035] Step S101: Obtain the first microphone array and the second microphone array, wherein the apertures of the first microphone array and the second microphone array are different.

[0036] Specifically, the first microphone array and the second microphone array in this application have different apertures. Here, aperture refers to the distance between any two microphones in the first microphone array or the second microphone array. The aperture of a microphone array is also called the microphone array size. For example, this application will subsequently use the first microphone array as a small-sized microphone array and the second microphone array as a large-sized microphone array to illustrate microphone arrays with different apertures, but it is not limited to this. Of course, the first microphone array can also be a large-sized microphone array, and the second microphone array can also be a small-sized microphone array, but this will not be elaborated here.

[0037] Step S102: Determine the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array based on the first microphone array and the second microphone array, respectively. Specifically, in the process of determining the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array based on the first microphone array and the second microphone array, the angle of arrival (OA) of the sound source corresponding to each microphone in the first microphone array and the second microphone array is calculated based on the DOA algorithm. The largest OA among all the calculated OAs of the first microphone array is used as the first positioning result, and the largest OA among all the calculated OAs of the second microphone array is used as the second positioning result.

[0038] For a small-sized first microphone array, it is equivalent to a far-field model. When one of any two microphones in the first microphone array is taken as the reference microphone, and the sound source arrival angle θ1 corresponding to the other microphone is taken as the positioning result, the sound source arrival angle θ1 can be calculated by the DOA algorithm. The specific calculation formula can be found in the following calculation formula (1). Then, the largest positioning result among all the positioning results determined in the small-sized first microphone array is taken as the first positioning result, that is, the largest one among all the sound source arrival angles θ1 determined in the first microphone array is taken as the first positioning result.

[0039]

[0040] In the above formula, Δτ1 is the time delay from the sound source to any two microphones in the first microphone array, d1 is the distance between any two microphones in the first microphone array, and c is the sound wave speed.

[0041] For a large-sized second microphone array, when one of any two microphones in the second microphone array is used as the reference microphone, it is equivalent to a near-field model. The sound source arrival angle θ2 corresponding to the other microphone is used as the positioning result. The sound source arrival angle θ2 can be calculated by the DOA algorithm. The specific calculation formula can be found in the following calculation formula (2). Then, the largest positioning result among all the positioning results determined in the large-sized second microphone array is used as the second positioning result, that is, the largest one among all the sound source arrival angles θ2 determined in the second microphone array is used as the second positioning result.

[0042]

[0043] Where Δτ2 is the time delay from the sound source to any two microphones in the second microphone array, d2 is the distance between any two microphones in the second microphone array, r1 is the distance between the reference microphone in any two microphones in the second microphone array and the sound source, and c is the sound wave speed.

[0044] Specifically, the number of microphones in the microphone array can be set according to actual needs. For example... Figure 4 As shown, when the number of microphones in both the first and second microphone arrays is two, the first microphone array, consisting of microphones m1 and m2, is a small-sized array, which is the far-field model, while the second microphone array, consisting of microphones m3 and m4, is a large-sized array, which is the near-field model. Therefore, the first sound source localization result corresponding to the first microphone array can be calculated using the above formula (1), and the second localization result corresponding to the second microphone array can be calculated using the above formula (2).

[0045] Step S103: Determine the sound source direction based on the first and second positioning results. Specifically, firstly, it is determined whether the first and second positioning results are consistent. If they are consistent, the direction corresponding to either the first or second positioning result is taken as the sound source direction. If they are inconsistent, the sound source direction is determined based on the first positioning result, the second positioning result, the first microphone array, and the second microphone array. The specific implementation of determining the sound source direction based on the first and second positioning results, the first microphone array, and the second microphone array will be shown in subsequent embodiments and will not be elaborated here.

[0046] Based on steps S101-S103 above, the direction of the sound source is determined based on the positioning results corresponding to the first and second microphone arrays with different array sizes (apertures). This makes up for the problem that the accuracy of the sound source direction determined by a microphone array with a single array size is low, and achieves accurate positioning of the sound source, thus meeting the user's needs.

[0047] In one implementation, during the process of determining the sound source direction based on the first positioning result, the second positioning result, the first microphone array, and the second microphone array, a first direction deviation value is first obtained according to the directions corresponding to the first positioning result and the second positioning result, respectively. This first direction deviation value is used as a first difference value. Next, it is determined whether the first difference value is less than a preset value. If the first difference value is less than the preset value, a third positioning result corresponding to the first microphone array and a fourth positioning result corresponding to the second microphone array are determined based on the first microphone array. The sound source direction is then determined based on the third and fourth positioning results. If the first difference value is not less than the preset value, cross-validation is performed on the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array. Based on the cross-validation, a fifth positioning result corresponding to the first microphone array and a sixth positioning result corresponding to the second microphone array are obtained. The sound source direction is then determined based on the fifth and sixth positioning results.

[0048] When determining the direction of the sound source based on the positioning results corresponding to the first microphone array and the second microphone array, a first difference is first determined based on the first positioning result and the second positioning result. If the first difference is less than a preset value, the direction of the sound source is determined based on the third positioning result determined by the first microphone array and the fourth positioning result determined by the second microphone array. If the first difference is not less than the preset value, the direction of the sound source is determined by cross-validation based on the first positioning result and the second positioning result. In this way, a more accurate direction of the sound source is determined, and the accuracy of sound source positioning is improved.

[0049] The "determining the third positioning result corresponding to the first microphone array based on the first microphone array and the fourth positioning result corresponding to the second microphone array based on the second microphone array" described in the aforementioned embodiment specifically includes: calculating the angle of arrival of the sound source corresponding to each microphone in the first microphone array and the second microphone array based on the TDOA algorithm, taking the largest angle of arrival of the sound source corresponding to all microphones in the first microphone array as the third positioning result, and taking the largest angle of arrival of the sound source corresponding to all microphones in the second microphone array as the fourth positioning result. Specifically, in this embodiment, the TDOA algorithm refers to taking one of any two microphones in the first microphone array or the second microphone array as the reference microphone, and taking the angle of arrival of the sound source θ corresponding to the other microphone as the positioning result. The formula for calculating the angle of arrival of the sound source θ can be found in the following formula (3). Then, the largest positioning result among all positioning results determined based on the first microphone array or the second microphone array is obtained, and the largest positioning result is taken as the third positioning result or the fourth positioning result. That is, the largest one among all angles of arrival of the sound source θ determined by the first microphone array is taken as the third positioning result, and the largest one among all angles of arrival of the sound source θ determined by the second microphone array is taken as the fourth positioning result.

[0050]

[0051] Wherein, Δτ is the time delay from the sound source to any two microphones in the first microphone array or the second microphone array, d is the distance between any two microphones in the first microphone array or the second microphone array, r1 is the distance between the reference microphone in any two microphones in the first microphone array or the second microphone array and the sound source, and c is the sound wave speed.

[0052] In addition, after obtaining the third and fourth localization results, the direction corresponding to the largest localization result between the third and fourth localization results can be taken as the sound source direction, that is, the larger of the third and fourth localization results can be taken as the sound source direction.

[0053] The "cross-validation of the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array, and obtaining the fifth positioning result corresponding to the first microphone array and the sixth positioning result corresponding to the second microphone array based on the cross-validation" described in the aforementioned embodiment includes: exchanging the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array, calculating the time delay corresponding to the first microphone array and the second microphone array based on the exchanged positioning results; obtaining the sound source arrival angle corresponding to the first microphone array and the second microphone array based on the calculated time delay, and using the obtained sound source arrival angles corresponding to the first microphone array and the second microphone array as the fifth positioning result corresponding to the first microphone array and the sixth positioning result corresponding to the second microphone array, respectively. Specifically, after determining the first positioning result corresponding to the first microphone array, the first positioning result is used as the parameter θ2 and substituted into the aforementioned formula (2) to calculate the time delay Δτ2, and the time delay Δτ2 is used as the time delay Δτ1 corresponding to the first microphone array. Then, the time delay Δτ1 corresponding to the first microphone array is substituted into the aforementioned formula (1) to calculate the sound source arrival angle θ1, and the sound source arrival angle θ1 is used as the fifth positioning result corresponding to the first microphone array. Similarly, after determining the second positioning result corresponding to the second microphone array, the second positioning result is used as parameter θ1 and substituted into the aforementioned formula (1) to calculate the time delay Δτ1. The time delay Δτ1 is then used as the time delay Δτ2 corresponding to the second microphone array. Next, the time delay Δτ2 corresponding to the second microphone array is substituted into the aforementioned formula (2) to calculate the sound source arrival angle θ2. The sound source arrival angle θ2 is then used as the sixth positioning result corresponding to the second microphone array.

[0054] After determining the fifth and sixth positioning results, the sound source direction can be determined based on them. Specifically, this involves: first, obtaining a second directional deviation value based on the directions corresponding to the fifth and first positioning results, using this second directional deviation value as the second difference; and second, obtaining a third directional deviation value based on the directions corresponding to the sixth and second positioning results, using this third directional deviation value as the third difference. Then, if the second difference is less than the third difference, the direction corresponding to the fifth positioning result is taken as the sound source direction; otherwise, if the second difference is not less than the third difference, the direction corresponding to the sixth positioning result is taken as the sound source direction. In this embodiment, microphone arrays of different sizes are used for preliminary sound source positioning. Based on this preliminary positioning, cross-validation is performed using the results to obtain the final sound source positioning result, improving the accuracy of sound source positioning. The sound source positioning method in this embodiment can also be used for sound tracking and external sound positioning warnings in driving mode.

[0055] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0056] Note that the number of microphone arrays of different sizes set in this application can also be multiple. For multiple microphone arrays of different sizes, the sound source localization method in the foregoing embodiment can be used to obtain the sound source direction corresponding to any two microphone arrays. Finally, the maximum value of all sound source directions determined by all microphone arrays of different sizes is taken as the final sound source direction, but it is not limited to this.

[0057] Furthermore, this invention also provides a directional sound pickup method. In this directional sound pickup process, the sound source direction is first determined using the sound source localization method described in the previous embodiment. Then, a first sound source spectrum is extracted from the sound source direction by a first microphone array, and a second sound source spectrum is extracted from the sound source direction by a second microphone array. Specifically, the beamforming algorithm can be used to extract the first sound source spectrum from the first microphone array, and similarly, the beamforming algorithm can be used to extract the second sound source spectrum from the second microphone array. Finally, the first and second sound source spectra are fused to obtain the time-domain signal of the sound source. This data fusion can be achieved by first superimposing the first and second sound source spectra, and then performing an inverse Fourier transform on the superimposed sound source spectrum to obtain the time-domain signal of the sound source. By accurately picking up the sound source direction, this method expands the frequency range for locating the sound source, building upon existing methods that utilize single-size microphone arrays for directional sound pickup. It also improves the pickup bandwidth of traditional arrays while enhancing the localization accuracy of broadband signals.

[0058] Furthermore, the present invention also provides a sound source localization device. (See attached document.) Figure 5 , Figure 5 This is a main structural block diagram of a sound source localization device according to an embodiment of the present invention. Figure 5As shown, the sound source localization device in this embodiment of the invention mainly includes an acquisition module 11, a determination module 12, and a positioning module 13. In some embodiments, one or more of the acquisition module 11, determination module 12, and positioning module 13 can be combined into a single module. In some embodiments, the acquisition module 11 can be configured to acquire a first microphone array and a second microphone array, wherein the apertures of the first microphone array and the second microphone array are different. The determination module 12 can be configured to determine a first positioning result corresponding to the first microphone array and a second positioning result corresponding to the second microphone array, respectively, based on the first microphone array and the second microphone array. The positioning module 13 can be configured to determine the sound source direction based on the first positioning result and the second positioning result. In one embodiment, a description of the specific functions can be found in steps S101-S103.

[0059] The aforementioned sound source localization device is used for performing Figure 3 The illustrated sound source localization method embodiments are similar in technical principle, the technical problems they solve, and the technical effects they produce. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the sound source localization device can be found in the embodiments of the sound source localization method, and will not be repeated here.

[0060] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0061] Furthermore, the present invention also provides a directional sound pickup system, including a sound source localization device, a spectrum extraction module, and a fusion module. The sound source localization device can be configured to determine the direction of the sound source. The spectrum extraction module can be configured to extract a first sound source spectrum in the direction of the sound source from a first microphone array and a second sound source spectrum in the direction of the sound source from a second microphone array. The fusion module can be configured to perform data fusion on the first and second sound source spectra to obtain the time-domain signal of the sound source.

[0062] Furthermore, the present invention also provides a vehicle, such as Figure 6 As shown, it includes the vehicle body, processor, and storage device. Figure 6 (Not shown in the image), a first microphone array M1 and a second microphone array M2 are simultaneously installed on the vehicle body. Based on the first microphone array M1 and the second microphone array M2, sound source localization and directional sound pickup can be achieved. Specifically, the placement of the first microphone array M1 and the second microphone array M2 within the vehicle can be arbitrary, and the shape of the microphone arrangement in the first microphone array M1 and the second microphone array M2 can also be arbitrary. For example, as shown... Figure 7 As shown, in practical applications in vehicles, in this embodiment, the four microphones in the first microphone array can be placed corresponding to the four doors of the vehicle, and the three microphones in the second microphone array can be placed corresponding to the rear seat storage panel. The four microphones in the first microphone array are arranged in a rectangular shape, and the three microphones in the second microphone array form an isosceles triangle. However, the placement and shape of the microphones in the first or second microphone array are not limited to these; they can be placed in any other location or in any other shape. Furthermore, this embodiment also further limits the size of the microphone array, such as... Figure 7As shown, the first microphone array M1, composed of three microphones forming an equilateral triangle, is a small-sized array, while the second microphone array M2, composed of four microphones forming a rectangle, is a large-sized array. Therefore, the minimum aperture of the second microphone array is required to be greater than the maximum aperture of the first microphone array. For example, in this embodiment, the minimum aperture of the large-sized second microphone array can be set to 0.5m, and the maximum aperture of the small-sized first microphone array can be set to 0.05m, but this is not limited to this. When locating the sound source, the sounds of other nearby vehicles are approximately near-field signals for the large-sized array and approximately far-field signals for the small-sized array. Based on the combination of near and far fields, the sound source can be further located, thereby improving the location accuracy of the sound source. The storage device is suitable for storing multiple program codes. The storage device can be configured to store a program for executing the sound source location method of the above method embodiment. The processor can be configured to execute the program in the storage device, which includes, but is not limited to, a program for executing the sound source location method of the above method embodiment. For ease of explanation, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0063] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for performing the sound source localization method of the above-described method embodiments. This program can be loaded and run by a processor to implement the sound source localization method described above. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0064] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0065] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0066] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for locating a sound source, characterized in that, Includes the following steps: Acquire a first microphone array and a second microphone array, wherein the apertures of the first microphone array and the second microphone array are different; Based on the first microphone array and the second microphone array, determine the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array, respectively. The direction of the sound source is determined based on the first and second positioning results; The process of determining the sound source direction based on the first positioning result and the second positioning result includes: determining whether the first positioning result and the second positioning result are consistent; if so, then taking the direction corresponding to either the first positioning result or the second positioning result as the sound source direction; if not, then determining the sound source direction based on the first positioning result, the second positioning result, the first microphone array, and the second microphone array. Determining the direction of the sound source based on the first positioning result, the second positioning result, the first microphone array, and the second microphone array includes: A first directional deviation value is obtained based on the directions corresponding to the first positioning result and the second positioning result, and the first directional deviation value is used as the first difference value. Determine whether the first difference is less than a preset value; If the first difference is less than a preset value, a third positioning result corresponding to the first microphone array is determined based on the first microphone array and a fourth positioning result corresponding to the second microphone array is determined based on the second microphone array, and the sound source direction is determined based on the third positioning result and the fourth positioning result. If the first difference is not less than a preset value, cross-validate the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array. Based on the cross-validation, obtain the fifth positioning result corresponding to the first microphone array and the sixth positioning result corresponding to the second microphone array, and determine the sound source direction based on the fifth positioning result and the sixth positioning result.

2. The sound source localization method according to claim 1, characterized in that, The step of determining the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array based on the first microphone array and the second microphone array respectively includes: calculating the angle of arrival of the sound source corresponding to each microphone in the first microphone array and the second microphone array based on the DOA algorithm, taking the largest angle of arrival of the sound source among all the calculated angles of arrival of the sound source in the first microphone array as the first positioning result, and taking the largest angle of arrival of the sound source among all the calculated angles of arrival of the sound source in the second microphone array as the second positioning result; The steps of determining the third positioning result corresponding to the first microphone array based on the first microphone array and determining the fourth positioning result corresponding to the second microphone array based on the second microphone array include: calculating the angle of arrival (AHA) of the sound source corresponding to each microphone in the first microphone array and the second microphone array based on the TDOA algorithm, using the largest AHA of the sound source corresponding to all microphones in the first microphone array as the third positioning result, and using the largest AHA of the sound source corresponding to all microphones in the second microphone array as the fourth positioning result; Determining the sound source direction based on the third and fourth positioning results includes taking the direction corresponding to the largest positioning result among the third and fourth positioning results as the sound source direction.

3. The sound source localization method according to claim 1, characterized in that, Cross-validation is performed on the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array. Based on the cross-validation, the fifth positioning result corresponding to the first microphone array and the sixth positioning result corresponding to the second microphone array are obtained, including: The first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array are swapped; Calculate the time delays corresponding to the first microphone array and the second microphone array based on the swapped positioning results; Based on the calculated time delays corresponding to the first microphone array and the second microphone array, the angles of arrival of the sound sources corresponding to the first microphone array and the second microphone array are obtained respectively. The angles of arrival of the sound sources corresponding to the first microphone array and the second microphone array are respectively used as the fifth positioning result corresponding to the first microphone array and the sixth positioning result corresponding to the second microphone array. Determining the sound source direction based on the fifth and sixth positioning results includes: obtaining a second direction deviation value based on the directions corresponding to the fifth and first positioning results respectively, using the second direction deviation value as a second difference value; and obtaining a third direction deviation value based on the directions corresponding to the sixth and second positioning results respectively, using the third direction deviation value as a third difference value. If the second difference is less than the third difference, the direction corresponding to the fifth positioning result is taken as the sound source direction; if the second difference is not less than the third difference, the direction corresponding to the sixth positioning result is taken as the sound source direction.

4. A directional sound pickup method, characterized in that, Includes the following steps: The sound source direction is determined using the sound source localization method according to any one of claims 1 to 3; A first sound source spectrum in the direction of the sound source is extracted by a first microphone array, and a second sound source spectrum in the direction of the sound source is extracted by a second microphone array; The first sound source spectrum and the second sound source spectrum are fused to obtain the time-domain signal of the sound source.

5. A sound source localization device, characterized in that, include: The acquisition module is configured to acquire a first microphone array and a second microphone array, wherein the first microphone array and the second microphone array have different apertures; The determination module is configured to determine a first positioning result corresponding to the first microphone array and a second positioning result corresponding to the second microphone array based on the first microphone array and the second microphone array, respectively. The positioning module is configured to determine the direction of the sound source based on the first positioning result and the second positioning result; The positioning module is used to determine whether the first positioning result and the second positioning result are consistent. If so, then take the direction corresponding to either the first positioning result or the second positioning result as the sound source direction; If not, obtain a first direction deviation value based on the directions corresponding to the first positioning result and the second positioning result respectively, and use the first direction deviation value as the first difference; determine whether the first difference is less than a preset value; if the first difference is less than the preset value, determine a third positioning result corresponding to the first microphone array based on the first microphone array and a fourth positioning result corresponding to the second microphone array based on the second microphone array, and determine the sound source direction based on the third positioning result and the fourth positioning result; if the first difference is not less than the preset value, perform cross-validation on the first positioning result corresponding to the first microphone array and the second positioning result corresponding to the second microphone array, obtain a fifth positioning result corresponding to the first microphone array and a sixth positioning result corresponding to the second microphone array based on the cross-validation, and determine the sound source direction based on the fifth positioning result and the sixth positioning result.

6. A directional sound pickup system, characterized in that, include: A sound source localization device is configured to determine the direction of a sound source using the sound source localization method according to any one of claims 1 to 3; The spectrum extraction module is configured to extract a first sound source spectrum in the direction of the sound source from a first microphone array and to extract a second sound source spectrum in the direction of the sound source from a second microphone array. The fusion module is configured to perform data fusion on the spectrum of the first sound source and the spectrum of the second sound source to obtain the time-domain signal of the sound source.

7. A vehicle comprising a vehicle body, at least two microphone arrays with different apertures disposed within the vehicle body, a processor, and a storage device, the storage device being adapted to store multiple lines of program code, characterized in that, The program code is adapted to be loaded and run by the processor to cause the vehicle to perform the sound source localization method of any one of claims 1 to 3 or the directional sound pickup method of claim 4.

8. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the sound source localization method of any one of claims 1 to 3 or the directional pickup method of claim 4.

Images