Sound source positioning method, system and device, sound source positioning equipment and storage medium

Abstract

The invention relates to the technical field of automobiles, and discloses a sound source positioning method, system and device, sound source positioning equipment and a storage medium, and the method comprises the steps: collecting training sound source data which comprises left ear sound source data collected at the left ear position of a passenger in a vehicle and right ear sound source data collected at the right ear position of the passenger; a training sample is obtained based on the training sound source data, and target training is a binaural time difference calculated based on left ear sound source data and right ear sound source data; according to the training sample and the real positioning label of the training sample, a sound source positioning model is obtained through training, and the sound source positioning model is used for positioning the target sound source according to the collected target sound source data. According to the invention, the binaural time difference is used as the training sample, the sound source localization model is obtained through training, the binaural time difference of the target sound source data obtained through calculation is input into the sound source localization model, sound source localization of the target sound source is realized, and accurate localization of the sound source in a small space can be realized.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a sound source localization method, system, device, sound source localization equipment and storage medium. Background Art

[0002] With the continuous advancement of science and technology, the adjustment of audio systems in small spaces (such as vehicle interiors) is no longer limited to the adjustment of loudness, pitch and timbre, but also focuses on the adjustment of the spatial sense of sound in small spaces.

[0003] For example, taking the adjustment of the sound system in the vehicle space as an example, a passenger sits in a certain position in the vehicle and turns on the vehicle's sound system to play music. If the passenger clearly feels that the sound source in the vehicle space is on the left or right, then the vehicle's sound system needs to be adjusted to improve the passenger's vehicle driving experience or riding experience.

[0004] The goal is usually to locate the sound source in the vehicle space and then adjust the sound system in the vehicle space. However, the sound undergoes multiple complex reflections in a small space, which makes it difficult for the existing technology to accurately locate the sound source in the vehicle space based on the collected sound data, and it is impossible to provide a reference for the sound space adjustment of the sound system in the vehicle space. Summary of the invention

[0005] Embodiments of the present invention provide a sound source localization method, system, device, sound source localization equipment and storage medium to solve the problem that it is difficult to localize a sound source in a small space.

[0006] A sound source localization method, the method comprising:

[0007] Collecting training sound source data, the training sound source data including left ear sound source data collected from the left ear position of a passenger in the vehicle and right ear sound source data collected from the right ear position of a passenger;

[0008] Based on the training sound source data, a training sample is obtained, where the training sample is an interaural time difference calculated based on the left ear sound source data and the right ear sound source data;

[0009] A sound source localization model is trained based on the training samples and the real localization labels of the training samples. The sound source localization model is used to locate the target sound source based on the collected target sound source data.

[0010] In the above method, optionally, obtaining a training sample based on the training sound source data includes:

[0011] Converting the training sound source data into a pulse signal;

[0012] Dividing the pulse signal into a preset number of frequency band signals by using a filter;

[0013] Performing rectification and filtering on each of the frequency band signals to obtain an auditory signal;

[0014] Based on the auditory signal, a training sample of the training sound source data is obtained by calculation.

[0015] In the above method, optionally, the step of collecting training sound source data includes:

[0016] Processing the swept frequency signal in a preset frequency range into audio data of different hearing angles;

[0017] The audio data at different hearing angles are played respectively to collect the training sound source data at different hearing angles.

[0018] In the above method, optionally, the real positioning label of the training sample is obtained by:

[0019] Get the test audio data;

[0020] Processing the test audio data into positioning determination audio data at different hearing angles;

[0021] Playing the positioning determination audio data at different hearing angles respectively, and obtaining a true positioning result corresponding to each positioning determination audio data based on a positioning determination rule;

[0022] Based on the mapping relationship between the training sound source data and the positioning determination audio data in terms of auditory perception, the real positioning result of the positioning determination audio data is used as the real positioning label of the training sample.

[0023] In the above method, optionally, the positioning determination rule includes:

[0024] A coordinate system is constructed with the head position of the tester as the origin, wherein the ordinate of the coordinate system points to the front of the tester, and the abscissa of the coordinate system points to the left of the tester;

[0025] When the positioning determination audio data exists within a first coordinate range, determining that the real positioning result of the positioning determination audio data is a first positioning result;

[0026] When the positioning determination audio data exists within a second coordinate range, determining that the real positioning result of the positioning determination audio data is a second positioning result;

[0027] When the positioning determination audio data exists within a third coordinate range, it is determined that the true positioning result of the positioning determination audio data is a third positioning result.

[0028] The above method, optionally, after training the sound source localization model according to the training samples and the real localization labels of the training samples, the method further comprises:

[0029] Collect target sound source data;

[0030] Calculate the interaural time difference of the target sound source data;

[0031] The binaural time difference of the target sound source data is input into a sound source localization model to obtain a target localization result of the target sound source.

[0032] A sound source localization system, the system comprising: a collector, an analyzer, and a decision device;

[0033] A collector, used for collecting target sound source data;

[0034] An analyzer, used for calculating the interaural time difference of the target sound source data;

[0035] The determiner is used to input the binaural time difference of the target sound source data into the sound source localization model to obtain the target localization result of the target sound source.

[0036] A sound source localization device, comprising:

[0037] A sound source collection unit, used to collect training sound source data, wherein the training sound source data includes left ear sound source data collected from the left ear position of a passenger in the vehicle and right ear sound source data collected from the right ear position of a passenger;

[0038] A training sample acquisition unit, configured to obtain a training sample based on the training sound source data, wherein the training sample is an interaural time difference calculated based on the left ear sound source data and the right ear sound source data;

[0039] The model training unit is used to train a sound source localization model according to the training samples and the real localization labels of the training samples, and the sound source localization model is used to locate the target sound source according to the collected target sound source data.

[0040] A sound source localization device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned sound source localization method when executing the computer program.

[0041] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the sound source localization method described above is implemented.

[0042] The above-mentioned sound source localization method, device, sound source localization equipment and storage medium calculate the binaural time difference of the training sound source data collected at the binaural position of the passenger in the vehicle to obtain the binaural time difference of the training sound source data, and use the calculated binaural time difference as a training sample and a manually marked real positioning label to train the sound source localization model, and obtain a sound source localization model for locating the target sound source according to the collected target sound source data. It can be seen that in the present invention, the binaural time difference is used as a training sample to train the sound source localization model, and then the binaural time difference calculated according to the target sound source data is input into the sound source localization model to realize the sound source localization of the target sound source, which can realize accurate positioning of the sound source in a small space. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0044] Figure 1 It is a flowchart of a sound source localization method disclosed in Embodiment 1 of the present invention;

[0045] Figure 2 It is a partial flow chart of a sound source localization method disclosed in Embodiment 1 of the present invention;

[0046] Figure 3 It is a partial flow chart of a sound source localization method disclosed in Embodiment 1 of the present invention;

[0047] Figure 4 It is a partial flow chart of a sound source localization method disclosed in Embodiment 1 of the present invention;

[0048] Figure 5 is a schematic diagram of coordinates of a positioning determination rule disclosed in the first embodiment of the present invention;

[0049] Figure 6 is another flow chart of a sound source localization method disclosed in Embodiment 1 of the present invention;

[0050] Figure 7 is a structural schematic diagram of a sound source localization system disclosed in Embodiment 2 of the present invention;

[0051] Figure 8 is a structural schematic diagram of a sound source localization device disclosed in Embodiment 3 of the present invention;

[0052] Fig. 9It is a structural schematic diagram of a sound source localization device disclosed in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The present invention discloses a sound source localization method, device, sound source localization equipment and storage medium. The binaural time difference of the training sound source data collected at the binaural position of the passenger in the vehicle is calculated to obtain the binaural time difference of the training sound source data. The calculated binaural time difference is used as a training sample and a manually marked real positioning label to train a sound source localization model, and a sound source localization model for locating a target sound source according to the collected target sound source data is obtained. It can be seen that in the present invention, the binaural time difference is used as a training sample to train a sound source localization model, and then the binaural time difference calculated according to the target sound source data is input into the sound source localization model to realize the sound source localization of the target sound source, so that the sound source in a small space can be accurately located. The following is an explanation through specific embodiments.

[0055] The sound source localization method of the present invention can be applied to the sound source localization at any position in the vehicle space, such as the sound source localization at the binaural position of the driver in the driving seat, or the sound source localization at the binaural position of passengers in other positions, etc. The sound spatial sense of the vehicle sound system is adjusted according to the obtained sound source localization result, so that the driver in the driving seat of the vehicle has a better auditory enjoyment when driving the vehicle, or the passengers in other seats in the vehicle have a better auditory enjoyment.

[0056] Embodiment 1

[0057] like Figure 1 FIG. 1 is a flow chart of a sound source localization method disclosed in Embodiment 1 of the present invention, which is specifically as follows:

[0058] S101: Collect training sound source data.

[0059] The training sound source data includes left ear sound source data collected from the left ear position of the passenger in the vehicle and right ear sound source data collected from the right ear position of the passenger.

[0060] In a specific implementation, in this embodiment, the training sound source data can be collected by a binaural collection device (such as a binaural microphone). The training audio is played through the speakers in the vehicle, and then the training audio played by the speakers is collected by the binaural collection device to obtain the training sound source data.

[0061] The binaural acquisition device includes a left ear collector and a right ear collector. When the audio in the vehicle plays the training audio, the left ear collector and the right ear collector respectively collect the training audio played by the audio to obtain paired training sound source data, namely, left ear sound source data and right ear sound source data. In addition, the left ear collector and the right ear collector of the binaural acquisition device can be installed on the left and right sides of the headrest of a designated seat in the vehicle, thereby simulating the sound source data received by the binaural positions of the passengers in the vehicle, or a dummy is placed on the designated seat in the vehicle, and the binaural acquisition device is installed at the left and right ears of the dummy, thereby simulating the sound source data received by the binaural positions of the passengers in the vehicle.

[0062] S102: Obtain training samples based on the training sound source data.

[0063] The training sample is the interaural time difference (ITD) calculated based on the left ear sound source data and the right ear sound source data. That is to say, in this embodiment, the left ear sound source data and the right ear sound source data need to be processed to calculate the interaural time difference between the left ear sound source data and the right ear sound source data, and the calculated interaural time difference is used as a training sample for subsequent training of the sound source localization model.

[0064] It should be understood that the training sound source data in this embodiment may include multiple groups of left ear sound source data and right ear sound source data. That is to say, the training sound source data can be collected multiple times according to different training sound sources. Each time the training sound source data is collected, there will be one more group of left ear sound source data and right ear sound source data. Then, each group of left ear sound source data and right ear sound source data is calculated separately to obtain multiple groups of binaural time differences, and thus multiple training samples are obtained. The number of training samples is not limited in this embodiment.

[0065] S103: According to the training samples and the real positioning labels of the training samples, a sound source localization model is trained.

[0066] The sound source localization model is used to locate the target sound source according to the collected target sound source data. That is to say, the target sound source data is subjected to binaural time difference calculation to obtain the binaural time difference, and the calculated binaural time difference is input into the sound source localization model to obtain the target sound source target localization result, thereby realizing the sound source localization of the target sound source.

[0067] Inputting the training samples and the true positioning labels of the training samples into the sound source localization model can enable the sound source localization model to learn the mapping relationship between the training samples and the true positioning labels, so that the calculated binaural time difference of the target sound source number is input into the sound source localization model to obtain the target positioning result of the target sound source.

[0068] Specifically, when training the sound source localization model based on the training samples and the true positioning labels of the training samples, it is necessary to determine whether the sound source localization model has been fully trained. If the sound source localization model has been fully trained, the training is completed, and the sound source localization model can be used to locate the target sound source according to the target sound source data; if the sound source localization model has not been fully trained, the training samples and the true positioning labels of the training samples are input into the sound source localization model for iterative training until a trained sound source localization model is obtained.

[0069] Whether the sound source localization model has been fully trained is determined based on the preset training end conditions, which can be specifically as follows:

[0070] First, the loss function of the sound source localization model is calculated. If the calculated loss value keeps decreasing, the model parameters of the sound source localization model are continuously updated, and the training samples and the true positioning labels of the training samples are input into the sound source localization model for iterative training. If the calculated loss value no longer decreases, it is determined that the sound source localization model has been fully trained.

[0071] Among them, the loss function can be a cross entropy loss function, as shown below:

[0072]

[0073] Among them, y i Indicates the real positioning result. It represents the predicted positioning result output by the sound source localization model, n represents the number of interaural time differences input into the sound source localization model, and LOSS is the loss value calculated by the loss function.

[0074] Secondly, after the interaural time difference and the actual positioning results corresponding to the interaural time difference are input into the sound source localization model for training, the actual positioning labels of the training samples are compared with the predicted positioning results output by the sound source localization model. When the ratio of the actual positioning labels and the predicted positioning results exceeds the preset ratio threshold, it is determined that the sound source localization model has been fully trained.

[0075] For example, taking the number of training samples as 100, the training samples are input into the sound source localization model to obtain the predicted localization results of these 100 training samples, and the predicted localization results are compared with the true localization labels. When there are more than 80 training samples whose true localization labels are the same as the predicted localization results, it is determined that the sound source localization model has been fully trained.

[0076] In addition, the sound source localization model in this embodiment includes an input layer, a hidden layer and an output layer, wherein the activation function of the hidden layer can be as follows:

[0077]

[0078] Among them, x represents the result obtained by weighted summation of the training samples input into the input layer.

[0079] The tanh(x) function can transform the input training samples into values ​​between -1 and 1 for processing in the output layer.

[0080] The activation function of the output layer can be softmax, which transforms the data set into a probability calculation problem, as shown below:

[0081]

[0082] Among them, xi represents the i-th component of x, and softmax(x) represents the output result of the output layer.

[0083] During the sound source localization model training process, the optimization algorithm may be an adam algorithm to update the sound source localization model parameters to continuously improve the model learning effect.

[0084] It should be understood that the model structure of the above-mentioned sound source localization model and the optimization algorithm of the model are only an exemplary description of the sound source localization model in this embodiment, and do not limit the sound source localization model to be implemented in only such a way, and are not specifically limited in this embodiment.

[0085] In summary, the present invention discloses a sound source localization method, which calculates the binaural time difference of the training sound source data collected at the binaural position of the passenger in the vehicle to obtain the binaural time difference of the training sound source data, and uses the calculated binaural time difference as a training sample and a manually marked real positioning label to train the sound source localization model, thereby obtaining a sound source localization model for locating the target sound source according to the collected target sound source data. It can be seen that in the present invention, the binaural time difference is used as a training sample to train the sound source localization model, and then the binaural time difference calculated according to the target sound source data is input into the sound source localization model to realize the sound source localization of the target sound source, so that the sound source in a small space can be accurately located.

[0086] based on Figure 1 In the specific implementation, step S102 can be implemented by the following steps: Figure 2 As shown:

[0087] S201: Convert training sound source data into pulse signals.

[0088] In a specific implementation, in this embodiment, the training sound source data can be pulse processed to convert the training sound source data into a pulse signal, and subsequent steps are performed according to the obtained pulse signal. The training sound source data is converted into a pulse signal, that is, the left ear training sound source data and the right ear training sound source are pulse processed respectively to obtain a left ear pulse signal and a right ear pulse signal.

[0089] S202: Divide the pulse signal into a preset number of frequency band signals through a filter.

[0090] In a specific implementation, in this embodiment, the pulse signal can be divided into 24 frequency bands according to the bark domain frequency band division rule through the gamatone filter, as shown in Table 3:

[0091] Table 3 Bark domain frequency band division table

[0092]

[0093]

[0094] As we all know, the frequency band where the fundamental frequency of human voice is concentrated is mainly concentrated in the frequency band of 100-770 Hz. Therefore, based on these 24 frequency band signals, the frequency band signals of 100-770 Hz are selected, and finally 7 frequency band signals are obtained.

[0095] It should be understood that the left ear pulse signal is divided by a gamatone filter to obtain seven left ear frequency band signals, and the right ear pulse signal is divided by a gamatone filter to obtain seven right ear frequency band signals.

[0096] S203: Rectify and filter each frequency band signal to obtain an auditory signal.

[0097] In a specific implementation, in this embodiment, the auditory signal can be obtained by rectifying and filtering each frequency band signal, and subsequent steps are performed according to the obtained auditory signal.

[0098] It should be understood that after rectifying and filtering each frequency band signal, a left ear hearing signal and a right ear hearing signal are obtained.

[0099] S204: Based on the auditory signal, an interaural time difference as a training sample is calculated.

[0100] The auditory signal includes a left ear auditory signal and a right ear auditory signal. The obtained left ear auditory signal and right ear auditory signal are input into an interaural time difference calculation formula to calculate the interaural time difference as a training sample.

[0101] Specifically, the interaural time difference calculation formula can be shown as follows:

[0102]

[0103] Among them, t represents the time of a certain moment of the signal, N represents the number of auditory signals, τ represents the number of time lags of binaural acquisition of training sound source data, and b L,m represents the left ear hearing signal, b R,m Represents the right ear hearing signal, the number of m-band signals.

[0104] In one implementation, the training sound source data can be obtained by the following steps: Figure 3 As shown:

[0105] S301: Processing a frequency sweep signal in a preset frequency range into audio data of different hearing angles.

[0106] The frequency sweep signal of the preset frequency range can be set according to actual needs, such as a frequency sweep signal of 20 to 20000 Hz, and the frequency sweep signal is processed into audio signals of different hearing angles by audio processing software, such as hearing angles including but not limited to -40, -20, -10, -5, 0, 5, 10, 20 and 40°. This is not specifically limited in this embodiment.

[0107] The audio processing software includes but is not limited to Adobe audition, Logic pro, etc., and is not specifically limited in this embodiment.

[0108] S302: Play the audio data at different hearing angles to collect training sound source data at different hearing angles.

[0109] The audio data at different hearing angles are played respectively, and then the audio data at different hearing angles are collected by binaural collection devices to obtain training sound source data at different hearing angles. For example, the audio data is played by the speakers in the vehicle, and then the audio data played by the speakers is collected by binaural collection devices to obtain training sound source data.

[0110] To sum up, in this embodiment, the sweep frequency signal is processed into audio data at different hearing angles. By playing the audio data at different hearing angles, training sound source data at different hearing angles is collected, which can effectively reduce the impact of noise data on the training sound source data, and is conducive to speeding up the training speed of the sound source localization model.

[0111] In one implementation, the actual positioning result corresponding to the interaural time difference can be obtained by the following method: Figure 4 As shown:

[0112] S401: Acquire test audio data.

[0113] In a specific implementation, the test audio data in this embodiment can be audio data simulating human voice, and the audio content can be set as needed, such as "Welcome to participate in the subjective evaluation test of sound and image localization", or the test audio data can also be a recitation, music, etc., which is not limited in this embodiment.

[0114] S402: Process the test audio data into positioning determination audio data at different hearing angles.

[0115] In a specific implementation, in this embodiment, the test audio data can be processed into positioning judgment audio data at different listening angles through audio processing software. The listening angles include but are not limited to -40, -20, -10, -5, 0, 5, 10, 20 and 40°, which are not limited in this embodiment.

[0116] S403: Play the positioning determination audio data at different hearing angles respectively, and obtain the real positioning result corresponding to each positioning determination audio data based on the positioning determination rule.

[0117] In a specific implementation, in this embodiment, the positioning determination audio data of different hearing angles can be determined manually. For example, a tester is arranged to sit in the driving position of the vehicle, and the positioning determination audio data of different hearing angles are played through the speakers installed in the vehicle. Then, the tester performs a subjective positioning determination on the positioning determination audio data, and the subjective determination result is the true positioning result of the positioning determination audio data.

[0118] S404: Based on the mapping relationship between the original sound source data and the positioning determination audio data in terms of auditory perception, the real positioning result of the positioning determination audio data is used as the real positioning label of the training sample.

[0119] It should be understood that the positioning judgment audio data used by the tester for positioning judgment has a specified listening angle, and the collected training sound source data also has a specified listening angle. According to the corresponding relationship between the listening angles, the true positioning result of the positioning judgment audio data with the same listening angle is used as the true positioning label of the training sample calculated based on the training sound source data.

[0120] To sum up, in this embodiment, the real positioning result corresponding to the positioning judgment audio data is obtained by manual judgment, and then the correspondence between the positioning judgment audio data and the training sound source data in terms of auditory perception is obtained, and the real positioning label of the training sample is obtained, which can greatly improve the accuracy of the real positioning label corresponding to the training sample and improve the training efficiency of the sound source localization model.

[0121] In one implementation, the positioning determination rule may be as follows:

[0122] Construct a coordinate system with the tester's head position as the origin, such as Figure 5 As shown, the ordinate of the coordinate system points to the front of the tester, and the abscissa of the coordinate system points to the left of the tester.

[0123] When the positioning determination audio data exists within the first coordinate range, the true positioning result of the positioning determination audio data is determined to be the first positioning result.

[0124] In a specific implementation, when the tester hears the positioning determination audio data and determines through subjective judgment that the positioning determination audio data exists within the first coordinate range, the positioning result of the positioning determination audio data is determined to be the first positioning result.

[0125] The first coordinate range can be defined as a range with the center of the human head as the origin, 1.2 meters forward in the vertical coordinate direction, and 10 cm to the left and right of the horizontal coordinate. When the positioning determines that the audio data exists within the first coordinate range, the first determination result can be centered. It should be understood that the above-mentioned first coordinate range and the first determination result are only exemplary descriptions and are not intended to limit the first coordinate range and the first determination result.

[0126] When the positioning determination audio data exists within the second coordinate range, the true positioning result of the positioning determination audio data is determined as the second positioning result.

[0127] In a specific implementation, when the tester hears the positioning determination audio data and determines through subjective judgment that the positioning determination audio data exists within the second coordinate range, the positioning result of the positioning determination audio data is determined to be the second positioning result.

[0128] The second coordinate range can be defined as a range greater than 10 cm to the right of the horizontal coordinate, with the center of the human head as the origin, 1.2 meters forward in the vertical coordinate direction, and when the positioning determines that the audio data exists in the second coordinate range, the second determination result can be biased to the right. It should be understood that the above second coordinate range and the second determination result are only exemplary descriptions and are not intended to limit the second coordinate range and the second determination result.

[0129] When the positioning determination audio data exists within the third coordinate range, it is determined that the true positioning result of the positioning determination audio data is the third positioning result.

[0130] In a specific implementation, when the tester hears the positioning determination audio data and determines through subjective judgment that the positioning determination audio data exists within the third coordinate range, the positioning result of the positioning determination audio data is determined to be the third positioning result.

[0131] The third coordinate range can be defined as a range greater than 10 cm to the left of the horizontal coordinate, with the center of the human head as the origin, 1.2 meters forward in the vertical coordinate direction, and when the positioning determines that the audio data exists within the third coordinate range, the third determination result can be biased to the right. It should be understood that the third coordinate range and the third determination result are merely exemplary descriptions and are not intended to limit the third coordinate range and the third determination result.

[0132] based on Figure 1 In the specific implementation, the following steps may be further included after step S103: Figure 6 As shown:

[0133] S104: Collect target sound source data.

[0134] The target sound source data is the sound source data at the binaural positions of the passenger in the vehicle, that is, the target sound source data includes left ear target sound source data and right ear target sound source data.

[0135] In a specific implementation, in this embodiment, the target sound source data can be collected by a binaural collection device (such as a binaural microphone).

[0136] S105: Calculate and obtain the interaural time difference of the target sound source data.

[0137] In the specific implementation, in this embodiment, it is necessary to screen out different frequency band signals in the human voice fundamental frequency set from the target sound source data, and then process the different frequency band signals into auditory signals respectively, and calculate the binaural time difference between the left ear training sound source data and the right ear training sound source data in the training sound source data based on the auditory signals.

[0138] S106: Input the target sound source data into the sound source localization model to obtain a sound source localization result of the target sound source.

[0139] In a specific implementation, the sound source localization model in this embodiment is used to locate the target sound source according to the target sound source data. That is, the left ear target sound source data and the right ear target sound source data in the target sound source data are calculated to obtain the interaural time difference, and the calculated interaural time difference is input into the sound source localization model to obtain the target localization result of the target sound source.

[0140] In summary, in this embodiment, the sound source localization model can be used to accurately locate the sound source in a small space, providing a reference for adjusting the sound spatial sense of the sound system in the small space.

[0141] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0142] Embodiment 2

[0143] like Figure 7 FIG. 1 is a schematic diagram of the structure of a sound source localization system disclosed in Embodiment 2 of the present invention. The system includes a collector, an analyzer, and a discriminator, as shown below:

[0144] Collector 701, used to collect target sound source data;

[0145] An analyzer 702, configured to calculate the interaural time difference of the target sound source data;

[0146] The decision unit 703 is used to input the binaural time difference of the target sound source data into the sound source localization model to obtain the target localization result of the target sound source.

[0147] For the specific definition of the sound source localization system, please refer to the relevant definition of the sound source localization method in the above text, which will not be repeated here. Each module in the above-mentioned sound source localization system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the sound source localization device in the form of hardware, or can be stored in the memory in the sound source localization device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0148] Embodiment 3

[0149] like Figure 8 FIG. 1 is a schematic diagram of the structure of a sound source localization device disclosed in Embodiment 3 of the present invention, as shown below:

[0150] Specifically, the sound source localization device in this embodiment may include the following units:

[0151] A first sound source collection unit 801 is used to collect training sound source data, wherein the training sound source data includes left ear sound source data collected from the left ear position of a passenger in the vehicle and right ear sound source data collected from the right ear position of a passenger;

[0152] A training sample acquisition unit 802 is used to obtain a training sample based on the training sound source data, where the training sample is an interaural time difference calculated based on the left ear sound source data and the right ear sound source data;

[0153] The model training unit 803 is used to train a sound source localization model according to the training samples and the real localization labels of the training samples, and the sound source localization model is used to locate the target sound source according to the target sound source data.

[0154] In one implementation, the time difference calculation unit 802 may be used to:

[0155] Convert training sound source data into pulse signals;

[0156] Dividing the pulse signal into a preset number of frequency band signals through a filter;

[0157] Rectify and filter the signal of each frequency band to obtain the auditory signal;

[0158] Based on the auditory signal, an interaural time difference as the training sample is calculated.

[0159] In one implementation, collecting training sound source data includes:

[0160] Processing the swept frequency signal in a preset frequency range into audio data of different hearing angles;

[0161] The audio data at different hearing angles are played respectively to collect the training sound source data at different hearing angles.

[0162] In one implementation, the true location labels of the training samples are obtained by:

[0163] Get the test audio data;

[0164] Processing the test audio data into positioning determination audio data at different hearing angles;

[0165] Play the positioning determination audio data at different hearing angles respectively, and obtain the real positioning result corresponding to each positioning determination audio data based on the positioning determination rule;

[0166] Based on the mapping relationship between the training sound source data and the positioning judgment audio data in terms of auditory perception, the true positioning result of the positioning judgment audio data is used as the true positioning label of the training sample.

[0167] In one implementation, the positioning determination rule includes:

[0168] A coordinate system is constructed with the tester's head position as the origin, with the ordinate of the coordinate system pointing to the front of the tester and the abscissa of the coordinate system pointing to the left of the tester;

[0169] When the positioning determination audio data exists within the first coordinate range, determining a true positioning result of the positioning determination audio data as a first positioning result;

[0170] When the positioning determination audio data exists within the second coordinate range, determining the true positioning result of the positioning determination audio data as the second positioning result;

[0171] When the positioning determination audio data exists within the third coordinate range, it is determined that the true positioning result of the positioning determination audio data is the third positioning result.

[0172] In one implementation, the device further includes the following units:

[0173] A second sound source collection unit, used to collect target sound source data;

[0174] A time difference calculation unit, used to calculate the binaural time difference of target sound source data;

[0175] The sound source localization unit inputs the binaural time difference of the target sound source data into the sound source localization model to obtain the target localization result of the target sound source.

[0176] For the specific definition of the sound source localization device, please refer to the relevant definition of the sound source localization method above, which will not be repeated here. Each module in the above-mentioned sound source localization device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the sound source localization device in the form of hardware, or can be stored in the memory in the sound source localization device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0177] Embodiment 4

[0178] Embodiment 4 of the present application discloses a sound source localization device, which may be a server, and its internal structure diagram may be as follows: Fig. 9As shown. The sound source localization device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the sound source localization device is used to provide computing and control capabilities. The memory of the sound source localization device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the sound source localization device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sound source localization method is implemented.

[0179] In one embodiment, a sound source localization device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0180] Collecting training sound source data, where the training sound source data is the sound source data at the binaural positions of passengers in the vehicle;

[0181] Calculate interaural time difference of training sound source data;

[0182] The sound source localization model is trained according to the interaural time difference and the actual localization result corresponding to the interaural time difference, and the sound source localization model is obtained through training. The sound source localization model is used to locate the target sound source according to the collected target sound source data.

[0183] Embodiment 5

[0184] Embodiment 5 of the present application discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor in a sound source localization device, the sound source localization device can perform the steps of any embodiment of a sound source localization method disclosed in the present invention. The computer-readable storage medium can be non-volatile or volatile.

[0185] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0186] Collecting training sound source data, where the training sound source data is the sound source data at the binaural positions of passengers in the vehicle;

[0187] Calculate interaural time difference of training sound source data;

[0188] The sound source localization model is trained according to the interaural time difference and the actual localization result corresponding to the interaural time difference, and the sound source localization model is obtained through training. The sound source localization model is used to locate the target sound source according to the collected target sound source data.

[0189] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0190] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0191] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0192] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0193] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0194] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0195] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0196] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A sound source localization method, characterized in that: The method comprises: Collecting training sound source data, the training sound source data including left ear sound source data collected from the left ear position of a passenger in the vehicle and right ear sound source data collected from the right ear position of a passenger; Based on the training sound source data, a training sample is obtained, where the training sample is an interaural time difference calculated based on the left ear sound source data and the right ear sound source data; A sound source localization model is trained based on the training samples and the real localization labels of the training samples. The sound source localization model locates the target sound source based on the collected target sound source data.

2. The sound source localization method according to claim 1, characterized in that: The step of obtaining a training sample based on the training sound source data comprises: Converting the training sound source data into a pulse signal; Dividing the pulse signal into a preset number of frequency band signals by using a filter; Performing rectification and filtering on each of the frequency band signals to obtain an auditory signal; Based on the auditory signal, an interaural time difference as the training sample is calculated.

3. The sound source localization method according to claim 1, characterized in that: The collecting of training sound source data includes: Processing the swept frequency signal in a preset frequency range into audio data of different hearing angles; The audio data at different hearing angles are played respectively to collect the training sound source data at different hearing angles.

4. The sound source localization method according to claim 1, characterized in that: The true positioning label of the training sample is obtained in the following way: Get the test audio data; Processing the test audio data into positioning determination audio data at different hearing angles; Playing the positioning determination audio data at different hearing angles respectively, and obtaining a true positioning result of each positioning determination audio data based on a positioning determination rule; Based on the mapping relationship between the training sound source data and the positioning determination audio data in terms of auditory perception, the real positioning result of the positioning determination audio data is used as the real positioning label of the training sample.

5. The sound source localization method according to claim 4, characterized in that: The positioning determination rules include: A coordinate system is constructed with the head position of the tester as the origin, wherein the ordinate of the coordinate system points to the front of the tester, and the abscissa of the coordinate system points to the left of the tester; When the positioning determination audio data exists within a first coordinate range, determining that the real positioning result of the positioning determination audio data is a first positioning result; When the positioning determination audio data exists within a second coordinate range, determining that the real positioning result of the positioning determination audio data is a second positioning result; When the positioning determination audio data exists within a third coordinate range, it is determined that the true positioning result of the positioning determination audio data is a third positioning result.

6. The sound source localization method according to claim 1, characterized in that: After the sound source localization model is obtained through training according to the training samples and the real localization labels of the training samples, the method further includes: Collect target sound source data; Calculate the interaural time difference of the target sound source data; The binaural time difference of the target sound source data is input into a sound source localization model to obtain a target localization result of the target sound source.

7. A sound source localization system, characterized in that: The system includes: a collector, an analyzer, and a judge; A collector, used for collecting target sound source data; An analyzer, used for calculating the interaural time difference of the target sound source data; The determiner is used to input the binaural time difference of the target sound source data into the sound source localization model to obtain the target localization result of the target sound source.

8. A sound source localization device, characterized in that: include: A first sound source collection unit is used to collect training sound source data, wherein the training sound source data includes left ear sound source data collected from the left ear position of the passenger in the vehicle and right ear sound source data collected from the right ear position of the passenger; A training sample acquisition unit, configured to obtain a training sample based on the training sound source data, wherein the training sample is an interaural time difference calculated based on the left ear sound source data and the right ear sound source data; The model training unit is used to train a sound source localization model according to the training samples and the real localization labels of the training samples, and the sound source localization model is used to locate the target sound source according to the collected target sound source data.

9. A sound source localization device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the sound source localization method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the sound source localization method according to any one of claims 1 to 6 is implemented.

Images