Sound source localization method, device, equipment and storage medium

Abstract

The embodiments of the present application relate to a method, device, equipment and storage medium for sound source localization. The method comprises: obtaining initial sound source data; processing the initial sound source data to generate multiple sound source data groups; generating initial localization data corresponding to each sound source data group; generating target localization data based on the data fluctuation index value of each initial localization data and each initial localization data. Through the above technical solution, sound source localization is achieved during the rotation of the microphone array, and the accuracy of the sound source localization result is improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a sound source localization method, device, equipment and storage medium. Background Art

[0002] Sound source localization technology is a technology that determines the location of a sound source in space. It mainly locates the sound based on the principle of time difference and sound level difference between the sound reaching each microphone.

[0003] At present, the sound source localization algorithms for microphone arrays mainly include time difference of arrival (TDOA), beamforming and holography.

[0004] However, the above-mentioned various sound source localization algorithms are all based on the assumption that the position of the microphone array is fixed. When these algorithms are applied to actual products, the accuracy of the sound source localization results will often be greatly reduced. Summary of the invention

[0005] In order to solve the problem that the accuracy of sound source localization decreases after the above-mentioned sound source localization algorithm is applied to actual products, the present application provides a sound source localization method, device, equipment and storage medium.

[0006] In a first aspect, an embodiment of the present application provides a method for localizing a sound source, the method comprising:

[0007] Obtaining initial sound source data;

[0008] Processing the initial sound source data to generate multiple sound source data groups;

[0009] Generating initial positioning data corresponding to each of the sound source data groups;

[0010] Based on the data fluctuation index value of each of the initial positioning data and each of the initial positioning data, target positioning data is generated.

[0011] In some embodiments, generating target positioning data based on the data fluctuation index value of each of the initial positioning data and each of the initial positioning data includes:

[0012] Based on each of the initial positioning data, determining a data change trend value and a data dispersion index value as the data fluctuation index value;

[0013] Determining a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data dispersion index value;

[0014] The target positioning data is generated based on each of the candidate positioning data.

[0015] In some embodiments, determining the data change trend value and the data dispersion index value based on each of the initial positioning data includes:

[0016] Performing linear fitting on each of the initial positioning data to obtain a linear slope corresponding to each of the initial positioning data as the data change trend value;

[0017] The variance of each of the initial positioning data is determined as the data dispersion indicator value.

[0018] In some embodiments, the determining a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data dispersion index value comprises:

[0019] If it is determined that the absolute value of the linear slope is greater than a preset slope threshold, and the variance is greater than a preset variance threshold, a preset number of the initial positioning data are selected from the initial positioning data in order of priority from high to low as the candidate positioning data; wherein the priority is determined by the time difference between the data acquisition time and the current moment.

[0020] In some embodiments, the determining a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data dispersion index value comprises:

[0021] If it is determined that the absolute value of the linear slope is greater than the preset slope threshold and the variance is less than or equal to the preset variance threshold, or if it is determined that the absolute value of the linear slope is less than or equal to the preset slope threshold, each of the initial positioning data is determined as each of the candidate positioning data.

[0022] In some embodiments, the processing the initial sound source data to generate a plurality of sound source data groups comprises:

[0023] Performing frame processing on the initial sound source data to obtain each single frame of sound source data;

[0024] Windowing is performed on each of the single-frame sound source data to generate each of the sound source data groups.

[0025] In some embodiments, generating initial positioning data corresponding to each sound source data group based on each sound source data group includes:

[0026] Using a preset sound source localization algorithm corresponding to the microphone array, sound source localization processing is performed on each of the sound source data groups to generate the initial localization data corresponding to the corresponding sound source data group.

[0027] In a second aspect, an embodiment of the present application further provides a sound source localization device, the device comprising:

[0028] An initial sound source data acquisition module, used to acquire initial sound source data;

[0029] A sound source data group generating module, used for processing the initial sound source data to generate a plurality of sound source data groups;

[0030] An initial positioning data generating module, used for generating initial positioning data corresponding to each sound source data group based on each sound source data group;

[0031] The target positioning data determination module is used to generate target positioning data based on the data fluctuation index value of each of the initial positioning data and each of the initial positioning data.

[0032] In some embodiments, the target positioning data determination module includes:

[0033] A data fluctuation index value determination submodule, used to determine a data change trend value and a data dispersion index value based on each of the initial positioning data as the data fluctuation index value;

[0034] A candidate positioning data determination submodule, configured to determine a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data discrete index value;

[0035] The target positioning data generating submodule is used to generate the target positioning data based on each of the candidate positioning data.

[0036] In some embodiments, the data fluctuation index value determination submodule is specifically used to:

[0037] Performing linear fitting on each of the initial positioning data to obtain a linear slope corresponding to each of the initial positioning data as the data change trend value;

[0038] The variance of each of the initial positioning data is determined as the data dispersion indicator value.

[0039] In some embodiments, the candidate positioning data determination submodule is specifically used to:

[0040] If it is determined that the absolute value of the linear slope is greater than a preset slope threshold, and the variance is greater than a preset variance threshold, a preset number of the initial positioning data are selected from the initial positioning data in order of priority from high to low as the candidate positioning data; wherein the priority is determined by the time difference between the data acquisition time and the current moment.

[0041] In some embodiments, the candidate positioning data determination submodule is further specifically configured to:

[0042] If it is determined that the absolute value of the linear slope is greater than the preset slope threshold and the variance is less than or equal to the preset variance threshold, or if it is determined that the absolute value of the linear slope is less than or equal to the preset slope threshold, each of the initial positioning data is determined as each of the candidate positioning data.

[0043] In some embodiments, the sound source data group generation module is specifically used to:

[0044] Performing frame processing on the initial sound source data to obtain each single frame of sound source data;

[0045] Windowing is performed on each of the single-frame sound source data to generate each of the sound source data groups.

[0046] In some embodiments, the initial positioning data generating module is specifically used to:

[0047] Using a preset sound source localization algorithm corresponding to the microphone array, sound source localization processing is performed on each of the sound source data groups to generate the initial localization data corresponding to the corresponding sound source data group.

[0048] In a third aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0049] Processor and memory;

[0050] The processor is used to execute the steps of the sound source localization method described in any embodiment of the present application by calling the program or instruction stored in the memory.

[0051] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instructions, wherein the program or the instructions enable a computer to execute the steps of the sound source localization method described in any embodiment of the present application.

[0052] The sound source localization solution provided in the embodiment of the present application processes the initial sound source data to generate multiple sound source data groups, performs sound source localization calculation on each sound source data group to obtain corresponding initial localization data, and based on the data fluctuation index value of each initial localization data, comprehensively calculates each initial localization data to generate final target localization data, thereby realizing sound source localization during the rotation of the microphone array and improving the accuracy of the sound source localization result. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0055] Figure 1 A flow chart of a sound source localization method provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of a flow chart of another sound source localization method provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of the structure of a sound source localization device provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described in detail below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0061] The various sound source localization algorithms currently available are constructed under the assumption that the position of the microphone array is fixed. When the microphone array is applied to actual products, the position of the microphone array will change to a certain extent, resulting in low accuracy of the sound source localization results calculated by these sound source localization algorithms. For example, when the microphone array is placed at the head position of a service robot, it will rotate with the rotation of the robot's head / body. In this case, the sound source localization results obtained by the above sound source localization algorithm are not accurate enough.

[0062] Based on the above situation, a sound source localization solution is provided in an embodiment of the present application to realize more accurate calculation of sound source localization data using the above sound source localization algorithm during the rotation of the microphone array, which can not only take into account the existing sound source localization algorithm, but also improve the accuracy of the sound source localization results.

[0063] The sound source localization method provided in the embodiment of the present application is mainly applicable to the scenario of using a microphone array to localize the sound source. For example, it can be applied to the fields of automobiles, aerospace, heavy equipment, etc. to locate noise, and it can also be applied to the fields of security robots, service robots, and remote conferences to locate speakers.

[0064] The sound source localization method provided in the embodiment of the present application can be performed by a sound source localization device, which can be implemented by software and / or hardware, and can be integrated in an electronic device. The electronic device can be a device with a microphone array installed, such as a mobile phone, a PDA, a tablet computer, a smart wearable device, or a robot used in the fields of security, service, etc.; the electronic device can also be a device that processes sound source data, such as a laptop computer, a desktop computer, a server, etc.

[0065] Figure 1 is a flow chart of a sound source localization method provided by an embodiment of the present application. Figure 1 , the sound source localization method specifically includes:

[0066] S110, obtaining initial sound source data.

[0067] The initial sound source data refers to the sound source data obtained by collecting sound from the sound source for a certain period of time. The initial sound source data is a multi-channel speech signal, and a speech signal of one channel corresponds to one microphone in the microphone array.

[0068] Specifically, the electronic device obtains initial sound source data obtained by the microphone array collecting sound from the sound source for a certain period of time. For example, the electronic device uses the microphone array to collect sound from the sound source in real time to obtain the initial sound source data. For another example, the electronic device obtains the initial sound source data by reading the sound source data corresponding to the microphone array stored in an internal storage medium or an external storage medium.

[0069] S120: Process the initial sound source data to generate multiple sound source data groups.

[0070] Here, the sound source data group refers to the combination of sound source data of each channel.

[0071] Specifically, in the process of processing the speech signal, the speech signal is required to be in a stable state, so it is necessary to judge whether the speech signal is stable to the initial sound source data. If the judgment result is that the speech signal is in a stable state, then other processing can be directly performed on it. If the judgment result is that the speech signal is not stable, then it is necessary to perform frame processing on the initial sound source data so that the speech signal of each frame (i.e., single-frame sound source data) is in a stable state. In addition, according to the algorithm (such as Fourier transform) adopted for processing the speech signal, the sound source data can also be windowed to achieve the purpose of alleviating spectrum leakage. By performing the above-mentioned frame and windowing processing on the sound source data of each channel, multiple sound source data of each channel can be obtained. The sound source data of the same frame of each channel are combined together to obtain multiple sound source data groups.

[0072] In some embodiments, S120 may be implemented as follows: performing frame processing on the initial sound source data to obtain each single frame of sound source data; performing windowing processing on each single frame of sound source data to generate each sound source data group.

[0073] Specifically, according to the above description, the electronic device can perform frame processing on the sound source data of each channel in the initial sound source data according to the preset frame length to obtain multiple single-frame sound source data of each channel. Then, the electronic device can perform windowing operation on each single-frame sound source data according to the preset window length and window shift. That is, for each single-frame sound source data, each time a window shift is moved, the sound source data is multiplied by a preset window function within the window length range after the movement to obtain the windowed sound source data within the window length range. According to the above process, the windowed sound source data within the multiple window length ranges of each channel can be obtained. The windowed sound source data of each channel within the same window length range are combined to obtain each sound source data group.

[0074] The frame length in the above process can be set according to the data volume of the sound source data, or it can be set to a fixed value based on artificial experience (such as 512 speech signal sampling points); the window length can be set according to the stability of the speech signal of the initial sound source data and the number of vibration cycles, or it can be set to a fixed value based on artificial experience (such as 16ms); in order to compensate for the weakening of the speech signals at both ends of the window length range by the window function, the window shift can be set to make a certain proportion (such as half) of the overlap between two adjacent window length ranges, and the window shift can also be set to a fixed value (such as 8ms) based on artificial experience.

[0075] S130: Generate initial positioning data corresponding to each sound source data group.

[0076] The positioning data refers to the sound source positioning result, which may be azimuth, elevation angle, distance, etc. The initial positioning data refers to the sound source positioning result directly calculated according to the sound source positioning algorithm.

[0077] Specifically, the electronic device uses a sound source localization algorithm in the related art to perform sound source localization calculation on each sound source data group, and can obtain initial localization data corresponding to each sound source data group.

[0078] In some embodiments, S130 includes: performing sound source localization processing on each sound source data group using a preset sound source localization algorithm corresponding to the microphone array, and generating initial localization data corresponding to the corresponding sound source data group.

[0079] Specifically, considering that different microphone array structures (such as circular structures, linear structures, square structures, etc.) may be adapted to different sound source localization algorithms, and the structure of the microphone array may change due to the movement of various components in the actual product. Therefore, in an embodiment of the present application, when calculating the initial positioning data, the electronic device first identifies the structure of the microphone array, for example, the position of each microphone is fitted to determine the structure of the microphone array. Then, a sound source localization algorithm adapted to the structure of the microphone array is determined. Finally, the sound source localization algorithm determined is used to perform sound source localization calculations on each sound source data group to obtain the initial positioning data corresponding to each sound source data group. This can improve the accuracy of the initial positioning data and provide a more accurate data basis for the calculation of the final sound source localization result (i.e., target positioning data), thereby further improving the accuracy of the target positioning data.

[0080] S140 : Generate target positioning data based on the data fluctuation index value of each initial positioning data and each initial positioning data.

[0081] The data fluctuation index value is an index value used to characterize the data fluctuation degree of each initial positioning data. The higher the data fluctuation degree, the greater the change of the microphone array, and the more unstable the accuracy of the sound source localization result.

[0082] Specifically, considering that the relative orientation and relative distance between the sound source and the actual product where the microphone array is deployed remain unchanged within a certain period of time, when the microphone array is fixed, the initial positioning data obtained above should be in a state of numerical stability. However, in actual applications, the microphone array will change with the movement of the product, so that the value of each initial positioning data shows a certain volatility, so that the accuracy of each initial positioning data is not high enough. Therefore, in an embodiment of the present application, the electronic device performs certain processing on each initial positioning data according to the data fluctuation index value of each initial positioning data. For example, when the data fluctuation index value is large, that is, when the fluctuation degree of each initial positioning data is strong, the local initial positioning data is selected to calculate the target positioning data; when the data fluctuation index value is small, that is, when the fluctuation degree of each initial positioning data is weak, the overall initial positioning data or a certain initial positioning data is selected to calculate the target positioning data. In this way, the noise caused by data fluctuation can be weakened, and a target positioning data with higher accuracy can be obtained.

[0083] In some embodiments, S140 includes: determining a data change trend value and a data discrete index value based on each initial positioning data as a data fluctuation index value; determining multiple candidate positioning data from each initial positioning data based on the data change trend value and the data discrete index value; and generating target positioning data based on each candidate positioning data.

[0084] The data change trend value is the value of an indicator that characterizes whether the data presents a trend change, for example, it can be a fitting slope, or it can be the comparison result of the size relationship between each two adjacent data. The data dispersion index value refers to the value of an indicator that characterizes the degree of data dispersion, for example, it can be variance, standard deviation, dispersion coefficient, etc. Candidate positioning data is at least one initial positioning data used to calculate target positioning data.

[0085] Specifically, the electronic device calculates the data change trend value and the data discrete index value according to each initial positioning data to characterize the degree of data fluctuation. Then, the electronic device screens each initial positioning data according to the calculated data change trend value and data discrete index value to obtain multiple candidate positioning data. After that, the electronic device performs statistical calculations on each candidate positioning data according to preset statistical indicators, such as mean, median, mode, etc., and the result is the target positioning data. In this way, appropriate candidate positioning data can be selected according to the degree of numerical fluctuation of each initial positioning data to calculate the target positioning data, further weakening the noise caused by the fluctuation of the initial positioning data of the sound source positioning, thereby further improving the accuracy of the target positioning data.

[0086] The sound source localization method provided in the above-mentioned embodiments of the present application processes the initial sound source data to generate multiple sound source data groups, performs sound source localization calculation on each sound source data group to obtain corresponding initial localization data, and based on the data fluctuation index value of each initial localization data, comprehensively calculates each initial localization data to generate final target localization data, thereby realizing sound source localization during the rotation of the microphone array and improving the accuracy of the sound source localization result.

[0087] Figure 2 This is a flow chart of another sound source localization method provided by an embodiment of the present application. Based on the above embodiments, the sound source localization method further optimizes "determining the data change trend value and the data discrete index value based on each initial positioning data"; on this basis, it further optimizes "determining multiple candidate positioning data from each initial positioning data based on the data change trend value and the data discrete index value". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. See Figure 2 , the sound source localization method specifically includes:

[0088] S210: Acquire initial sound source data.

[0089] S220: Perform frame processing on the initial sound source data to obtain each single frame of sound source data.

[0090] S230, performing windowing processing on each single frame of sound source data to generate each sound source data group.

[0091] S240: Perform sound source localization processing on each sound source data group using a preset sound source localization algorithm corresponding to the microphone array to generate initial localization data corresponding to the corresponding sound source data group.

[0092] S250 , performing linear fitting on each initial positioning data to obtain a linear slope corresponding to each initial positioning data as a data change trend value.

[0093] Specifically, in practical applications, the robot equipped with a microphone array mostly rotates in one direction within a certain period of time, so the fluctuation of each initial positioning data generally presents an upward or downward trend. Based on this, the electronic device performs a linear fitting of a linear function on each initial positioning data to obtain a linear function formula, and then obtains the linear slope of the linear function formula. The numerical value of the linear slope can characterize the change trend of each initial positioning data, so it is used as the data change trend value.

[0094] S260: Determine the variance of each initial positioning data as a data dispersion indicator value.

[0095] Specifically, the electronic device calculates the variance of each initial positioning data as a data dispersion index value.

[0096] S270: If it is determined that the absolute value of the linear slope is greater than the preset slope threshold and the variance is greater than the preset variance threshold, a preset number of initial positioning data are selected from the initial positioning data in descending order of priority as candidate positioning data.

[0097] Among them, the preset slope threshold is a preset critical value of the linear slope, for example, it can be any value in tan(0°)~tan(10°). The preset variance threshold is a preset critical value of the variance. The priority is used to characterize the order in which the initial positioning data is selected as the candidate positioning data. The higher the priority of a certain initial positioning data, the higher the order in which it is selected as the candidate positioning data. Optionally, the priority is determined by the time difference between the data acquisition time and the current moment, and the shorter the time difference, the higher the priority. The data acquisition time refers to the moment when the initial positioning data is acquired. The preset number is a preset quantity value, which can be a fixed value set by experience. Optionally, the preset number is determined based on the number of each initial positioning data and a preset data volume ratio (i.e., a preset ratio, such as 1 / 2, 1 / 3 or 1 / 4, etc.).

[0098] Specifically, considering the error of sound source localization, when the electronic device determines that the absolute value of the calculated linear slope is greater than the preset slope threshold, it determines that each initial positioning data has a trend change (linear rise or linear decline). Then, when the electronic device determines that the variance obtained by the above calculation is greater than the preset variance threshold, it determines that the data discreteness of each initial positioning data is relatively strong. So far, it can be explained that the data fluctuation degree of each initial positioning data is relatively strong. Therefore, the electronic device calculates the time difference corresponding to each initial positioning data according to the data acquisition time and the current time of each initial positioning data, and then determines the priority of each initial positioning data. Then, the electronic device selects a preset number of initial positioning data from each initial positioning data in order from high to low priority as each candidate positioning data. For example, with time as the horizontal axis and positioning data as the vertical axis, the trend distribution results of each initial positioning data are obtained, and the preset number of initial positioning data in the latter part of the trend distribution results are selected as candidate positioning data.

[0099] In order to further improve the accuracy of target positioning data, based on a certain amount of data, the smaller the preset number / preset ratio is, the better. For example, the accuracy of the target positioning data obtained when the preset ratio is 1 / 4 is higher than the accuracy of the target positioning data obtained when the preset ratio is 1 / 2.

[0100] S280: If it is determined that the absolute value of the linear slope is greater than the preset slope threshold and the variance is less than or equal to the preset variance threshold, or if it is determined that the absolute value of the linear slope is less than or equal to the preset slope threshold, each initial positioning data is determined as each candidate positioning data.

[0101] Specifically, if the electronic device determines that the absolute value of the linear slope is greater than the preset slope threshold, but the variance does not exceed the preset variance threshold, it means that although the initial positioning data presents a linear change trend, its data discreteness is weak, and after comprehensive determination, the data fluctuation of the initial positioning data is weak. Then, all the initial positioning data can be directly determined as candidate positioning data.

[0102] If the electronic device determines that the absolute value of the linear slope does not exceed the preset slope threshold, it means that the initial positioning data does not show a trend change, and it can be considered that the value fluctuation is weak. At this time, all the initial positioning data can also be directly determined as candidate positioning data.

[0103] S290: Generate target positioning data based on each candidate positioning data.

[0104] The sound source localization method provided in the above embodiment of the present application obtains the linear slope corresponding to each initial positioning data by performing linear fitting on each initial positioning data, and calculates the variance of each initial positioning data, so as to obtain the data fluctuation index value of each initial positioning data by a simpler calculation, thereby improving the calculation speed of correcting the initial positioning data, thereby improving the real-time performance of the sound source localization. By comparing the absolute value of the linear slope of each initial positioning data with a preset slope threshold value to determine whether each initial positioning data presents a trend change, and combining the comparison of the variance with the preset variance threshold value to determine the discrete degree of each initial positioning data, the judgment of the data fluctuation degree of each initial positioning data is realized, and according to the judgment result, each candidate positioning data that is more suitable for calculating the target positioning data is screened from each initial positioning data, providing a more accurate data basis for the calculation of the target positioning data, thereby further improving the accuracy of the target positioning data.

[0105] Figure 3 This is a schematic diagram of the structure of a sound source localization device provided in an embodiment of the present application. Figure 3 As shown, the sound source localization device 300 includes:

[0106] An initial sound source data acquisition module 310 is used to acquire initial sound source data;

[0107] A sound source data group generating module 320 is used to process the initial sound source data to generate a plurality of sound source data groups;

[0108] An initial positioning data generating module 330 is used to generate initial positioning data corresponding to each sound source data group based on each sound source data group;

[0109] The target positioning data determination module 340 is used to generate target positioning data based on the data fluctuation index value of each initial positioning data and each initial positioning data.

[0110] The above-mentioned sound source localization device provided by the present application processes the initial sound source data corresponding to the microphone array to generate multiple sound source data groups, and performs sound source localization calculation on each sound source data group to obtain corresponding initial localization data, and based on the data fluctuation index value of each initial localization data, comprehensively calculates each initial localization data to generate final target localization data, thereby realizing sound source localization during the rotation of the microphone array and improving the accuracy of the sound source localization result.

[0111] In some embodiments, the target location data determination module 340 includes:

[0112] A data fluctuation index value determination submodule is used to determine a data change trend value and a data discrete index value based on each initial positioning data as a data fluctuation index value;

[0113] A candidate positioning data determination submodule is used to determine a plurality of candidate positioning data from each initial positioning data based on a data change trend value and a data discrete index value;

[0114] The target positioning data generation submodule is used to generate target positioning data based on each candidate positioning data.

[0115] In some embodiments, the data fluctuation index value determination submodule is specifically used to:

[0116] Perform linear fitting on each initial positioning data to obtain the linear slope corresponding to each initial positioning data as the data change trend value;

[0117] The variance of each initial positioning data is determined as the data dispersion indicator value.

[0118] In some embodiments, the candidate positioning data determination submodule is specifically used to:

[0119] If it is determined that the absolute value of the linear slope is greater than a preset slope threshold, and the variance is greater than a preset variance threshold, a preset number of initial positioning data are selected from the initial positioning data in descending order of priority as candidate positioning data; wherein the priority is determined by the time difference between the data acquisition time and the current moment.

[0120] In some embodiments, the candidate positioning data determination submodule is further specifically configured to:

[0121] If it is determined that the absolute value of the linear slope is greater than the preset slope threshold and the variance is less than or equal to the preset variance threshold, or if it is determined that the absolute value of the linear slope is less than or equal to the preset slope threshold, each initial positioning data is determined as each candidate positioning data.

[0122] In some embodiments, the sound source data group generation module 320 is specifically used to:

[0123] Performing frame processing on the initial sound source data to obtain each single frame of sound source data;

[0124] Each single frame of sound source data is subjected to windowing processing to generate each sound source data group.

[0125] In some embodiments, the initial positioning data generating module 330 is specifically used to:

[0126] Using the preset sound source localization algorithm corresponding to the microphone array, sound source localization processing is performed on each sound source data group to generate initial localization data corresponding to the corresponding sound source data group.

[0127] The sound source localization device provided in the embodiments of the present application can execute the sound source localization method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0128] It is worth noting that in the embodiment of the above-mentioned sound source localization device, the various modules and sub-modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional modules / sub-modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of this application.

[0129] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0130] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0131] The memory 402 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may run the program instructions to implement the sound source localization method and / or other desired functions provided in any embodiment of the present application. Various contents such as a preset slope threshold, a preset variance threshold, a preset number, and a preset sound source localization algorithm may also be stored in the computer-readable storage medium.

[0132] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside. The output device 404 may include, for example, a display, a speaker, and a communication network and a remote output device connected thereto, etc.

[0133] Of course, to simplify, Figure 4 Only some of the components related to the present application in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 400 may also include any other appropriate components.

[0134] In addition to the above methods and devices, the embodiments of the present application may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps of the sound source localization method provided by any embodiment of the present application.

[0135] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0136] In addition, the embodiments of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the sound source localization method provided by any embodiment of the present application.

[0137] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0138] It should be noted that the terms used in this application are only for describing specific embodiments, rather than limiting the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The term "and / or" includes any and all combinations of one or more related listed items. The term "includes", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method or device. Relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method or device including the elements.

[0139] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A sound source localization method, characterized in that: include: Obtaining initial sound source data; Processing the initial sound source data to generate multiple sound source data groups; Generating initial positioning data corresponding to each of the sound source data groups; generating target positioning data based on the data fluctuation index value of each of the initial positioning data and each of the initial positioning data; The generating target positioning data based on the data fluctuation index value of each of the initial positioning data and each of the initial positioning data comprises: Based on each of the initial positioning data, determining a data change trend value and a data dispersion index value as the data fluctuation index value; Determining a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data dispersion index value; Based on each of the candidate positioning data, generate the target positioning data; Determining the data change trend value and the data dispersion index value based on each of the initial positioning data includes: Performing linear fitting on each of the initial positioning data to obtain a linear slope corresponding to each of the initial positioning data as the data change trend value; Determine the variance of each of the initial positioning data as the data dispersion index value; The determining of a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data discrete index value comprises: If it is determined that the absolute value of the linear slope is greater than a preset slope threshold, and the variance is greater than a preset variance threshold, a preset number of the initial positioning data are selected from the initial positioning data in order of priority from high to low as the candidate positioning data; wherein the priority is determined by the time difference between the data acquisition time and the current moment, and the time difference is inversely proportional to the priority.

2. The method according to claim 1, characterized in that The determining of a plurality of candidate positioning data from each of the initial positioning data based on the data change trend value and the data discrete index value comprises: If it is determined that the absolute value of the linear slope is greater than the preset slope threshold and the variance is less than or equal to the preset variance threshold, or if it is determined that the absolute value of the linear slope is less than or equal to the preset slope threshold, each of the initial positioning data is determined as each of the candidate positioning data.

3. The method according to claim 1, characterized in that The processing of the initial sound source data to generate a plurality of sound source data groups comprises: Performing frame processing on the initial sound source data to obtain each single frame of sound source data; Windowing is performed on each of the single-frame sound source data to generate each of the sound source data groups.

4. The method according to any one of claims 1 to 3, characterized in that: The generating, based on each of the sound source data groups, the initial positioning data corresponding to each of the sound source data groups comprises: Using a preset sound source localization algorithm corresponding to the microphone array, sound source localization processing is performed on each of the sound source data groups to generate the initial localization data corresponding to the corresponding sound source data group.

5. A sound source localization device, characterized in that: include: An initial sound source data acquisition module, used to acquire initial sound source data; A sound source data group generating module, used for processing the initial sound source data to generate a plurality of sound source data groups; An initial positioning data generating module, used for generating initial positioning data corresponding to each sound source data group based on each sound source data group; A target positioning data determination module, used for generating target positioning data based on the data fluctuation index value of each of the initial positioning data and each of the initial positioning data; The target positioning data determination module includes: A data fluctuation index value determination submodule is used to determine a data change trend value and a data discrete index value based on each initial positioning data as a data fluctuation index value; A candidate positioning data determination submodule is used to determine a plurality of candidate positioning data from each initial positioning data based on a data change trend value and a data discrete index value; A target positioning data generation submodule is used to generate target positioning data based on each candidate positioning data; The data fluctuation index value determination submodule is specifically used for: Perform linear fitting on each initial positioning data to obtain the linear slope corresponding to each initial positioning data as the data change trend value; Determine the variance of each initial positioning data as the data dispersion index value; The candidate positioning data determination submodule is specifically used for: If it is determined that the absolute value of the linear slope is greater than a preset slope threshold, and the variance is greater than a preset variance threshold, a preset number of initial positioning data are selected from the initial positioning data in order of priority from high to low as candidate positioning data; wherein the priority is determined by the time difference between the data acquisition time and the current moment, and the time difference is inversely proportional to the priority.

6. An electronic device, characterized in that: The electronic device comprises: Processor and memory; The processor is used to execute the steps of the sound source localization method according to any one of claims 1 to 4 by calling the program or instruction stored in the memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the sound source localization method according to any one of claims 1 to 4.

Images