Audio signal pickup regulation and control method based on relative distance discrimination threshold

Abstract

The invention relates to the technical field of sound source localization, and particularly discloses an audio signal pickup regulation and control method based on a relative distance discrimination threshold value, and the method comprises the steps: constructing spatial audios with different directions, different distances and different stimulation sequences through employing a synthesized near-field HRTF, and carrying out the measurement and analysis of the relative distance discrimination threshold value of a subject; and predicting the difference of the relative distance discrimination thresholds of the sound source direction, the sound source distance and the sound source stimulation sequence of the air conduction and bone conduction equipment in the near-field range by using the analysis result, calculating the relative distance discrimination threshold of the audiences, picking up the audio of the corresponding position by using a spherical harmonic wave beam forming method for the audiences with the relatively large relative distance discrimination threshold, and carrying out sound source stimulation on the audiences with the relatively large relative distance discrimination threshold. And the distance perception of the corresponding position is increased. By means of the method, the distance sensing ability of the hearing aid and the consumer-level bone conduction earphone can be evaluated, and then the hearing equipment which enables audiences to obtain the ability of better judging the object movement is designed accordingly.

Description

Technical Field

[0001] The present application relates to the technical field of sound source localization, and more specifically, to an audio signal pickup control method based on a relative distance discrimination threshold. Background Art

[0002] Hearing plays a key role in our lives, especially in determining the location of objects. Our ability to localize a sound source covers two main aspects: determining the direction of the sound source and estimating the distance of the sound source. In the natural environment, this ability is essential for our spatial orientation and navigation, especially in low light or limited vision, when hearing becomes a key tool for us to judge the distance of a sound source.

[0003] Research has found that the distance localization of a sound source is affected by a variety of factors, including the loudness of the sound, the ratio of direct to reverberant sound, the frequency distribution of the sound, the difference in sound received by both ears, and changes caused by head movement. Among these factors, loudness and the spectral characteristics of the sound mainly provide information about the relative distance of the sound source, while in an indoor environment, the ratio of direct to reverberant sound can provide clues to the absolute distance of the sound source. Among all these factors, loudness is a major factor in judging the distance of a sound source.

[0004] The auditory system's ability to perceive distance changes, namely the minimum perceptible difference (JND), is related to the reference distance. At long distances, the JND remains relatively stable, which means that the sensitivity of our auditory system to distance changes is constant at a certain ratio. The pressure discrimination hypothesis holds that the threshold of distance changes that we can perceive can reach a 5% difference in sound pressure level.

[0005] However, at close range, the JND changes more, mainly because the sound source at close range generates spherical waves, which are different from the plane waves at long range. Therefore, the traditional positioning method based on plane waves is no longer effective in the near field. In addition, when rendering sound effects, the traditional high-precision HRTF has high requirements for computing resources.

[0006] Therefore, a method for adjusting and controlling audio signal pickup based on a relative distance discrimination threshold is provided. Summary of the invention

[0007] In order to solve the above technical problems, the present application is proposed. The present application provides an audio signal pickup control method based on a relative distance discrimination threshold, which includes:

[0008] S1, synthesize near-field HRTF using sound source information and head parameters;

[0009] S2, constructing virtual spatial audio according to the near-field HRTF;

[0010] S3. Use equal loudness matching method to make the stimuli presented through air conduction device and bone conduction device have the same loudness level within the measurement frequency range;

[0011] S4. Use air conduction devices and bone conduction devices to play virtual space audio, and measure the subjects' relative distance discrimination thresholds in different directions and distances using a two-item forced choice method and a 2-up 1-down adaptive method;

[0012] S5. Calculate the relative distance discrimination threshold of the listeners using the relative distance discrimination threshold measurement results of the subjects, and obtain the relative distance discrimination thresholds of the listeners at different positions and distances;

[0013] S6. Setting a relative distance discrimination threshold evaluation standard, and judging whether it is necessary to add a spherical harmonic domain beamforming algorithm for regulation according to the audience's ability to perceive the distance of small movements of objects;

[0014] S7. For listeners who have poor distance perception of tiny object movements, a beamforming algorithm based on spherical harmonics is used to decouple the angle and frequency in the sound field to enhance the distance perception of tiny object movements.

[0015] Preferably, in S1, the distance change function is as follows:

[0016]

[0017] Among them, h l (kr) is the spherical Hankel function of the first kind, with order m, h′ l (ka) is its first derivative at radius a, k = ω / c is the wave number, c is the speed of sound in air, P l is the l-th order Legendre polynomial, θ is the angle between the vector from the center of the sphere to point X on the sphere and the vector from the center of the sphere to the sound source, p n is the pressure at the ear from a sound source at a near-field distance n, p f is the pressure at the ear due to a sound source at far-field distance f.

[0018] Preferably, S2 includes: convolving the synthesized near-field HRTF and Gaussian white noise to obtain spatial audio corresponding to different positions, wherein the Gaussian white noise used is generated by Audition software and is 100 Hz-8000 Hz.

[0019] Preferably, S3 includes: S31, adjusting the air conduction device to ensure that the sound pressure level of the played spatial audio is 60dBSPL; S32, the subject wears Sennheiser IE800 air conduction headphones and Radioear B81 bone conduction vibrator, and takes off the air conduction headphones when controlling the bone conduction vibrator to play the spatial audio signal; S33, the subject adjusts the gain applied to the bone conduction vibrator to match the perceived loudness of the air conduction headphone stimulation.

[0020] Preferably, the S33 includes: controlling the gain of the air conduction headphone stimulation to remain unchanged, comparing the perceived loudness of the bone conduction vibrator stimulation with the perceived loudness of the air conduction headphone stimulation, and if the perceived loudness of the bone conduction vibrator is greater than the perceived loudness of the air conduction headphone stimulation, reducing the perceived loudness of the bone conduction vibrator; if the perceived loudness of the bone conduction vibrator is less than the perceived loudness of the air conduction headphone stimulation, increasing the perceived loudness of the bone conduction vibrator.

[0021] Preferably, S4 comprises: S41, conducting a preliminary experiment for each subject; S42, setting a reference distance of the sound source in the formal experiment, and configuring an initial adaptive virtual distance for the sound source in different directions; S43, playing the sound source in different stimulation orders, and having the subject make judgments on two consecutively played sound stimuli through a binomial forced choice method; S44, using a 2 up 1 down adaptive method, when the subject makes two consecutive correct judgments, the sound source moves away from the reference distance by 1%, and when the subject makes an incorrect judgment, the sound source moves closer to the reference distance by 1%, until the adaptive virtual distance of the sound source is consistent with the reference distance, and stops; wherein, before the first incorrect judgment, the size of each step is set to 3%.

[0022] Preferably, S41 comprises: allowing the subject to listen to surround sound in the horizontal plane through an air conduction device or a bone conduction device to familiarize the subject with the virtual sound field; allowing the subject to listen to stimulation signals from near to far in different azimuths in sequence so that the subject can become familiar with virtual sound sources at different distances; and providing each subject with a brief training consisting of a number of trials with feedback so that they can become familiar with the task and stimulation.

[0023] Preferably, when the loudness factor is eliminated, a compensation coefficient K related to the sound source distance r is used to normalize the stimulus signals at different distances so that the spatial audio at different distances in the same direction presents the same loudness; wherein the calculation formula of the compensation coefficient is as follows:

[0024]

[0025] where d r is the distance from the sound source to the right ear, d l is the distance from the sound source to the left ear;

[0026] S′(n)=K*S(n)

[0027] Where S′(n) is the normalized audio signal, and S(n) is the original audio signal.

[0028] Preferably, in S5, the relative distance discrimination threshold is calculated as follows:

[0029]

[0030] JND r is the relative distance discrimination threshold at the reference distance r, d r is the reference distance, d′ a It is the average of the distances corresponding to the last 5 flip positions.

[0031] Preferably, S6 includes: setting a relative distance discrimination threshold j to evaluate the individual's spatial perception ability. If the relative distance discrimination threshold of the listener is greater than j, it means that the listener's perception ability of the distance of the moving object is poor. Even if the moving distance of the object is large, the listener cannot perceive it, and it is necessary to add a beamforming algorithm in the spherical harmonic domain for enhancement. On the contrary, if the relative distance discrimination threshold of the listener is less than j, it is not necessary to add a beamforming algorithm in the spherical harmonic domain for enhancement.

[0032] Preferably, in S7, the formula for calculating the radial filtering coefficient d is:

[0033] d(k)=(C -1 ) T ·q

[0034] Where q is the response mode corresponding to different orders, C is the radial coefficient matrix predetermined;

[0035] To filter:

[0036]

[0037] where k is the wave number, r s is the distance to the target sound source, r s Satisfy a <r s <r NF , where a is the radius, r NF is the boundary radius of the near field, and its value is r NF ≈N / k.

[0038] Compared with the prior art, the present application provides an audio signal pickup control method based on relative distance discrimination threshold. The method measures the relative distance discrimination threshold in different directions, different reference distances, different stimulation orders and different stimulation types, analyzes the factors affecting the relative distance discrimination threshold when using bone conduction devices, and estimates the audience's ability to perceive the distance of moving objects in different situations, providing a scientific basis for the improvement and optimization of hearing devices such as hearing aids and consumer-grade bone conduction headphones, thereby helping the hearing-impaired to better perceive sound and improve their auditory experience and quality of life. At the same time, the relative distance discrimination threshold is combined with the spherical harmonic beamforming algorithm to minimize the complexity of the algorithm while improving the audience's ability to perceive the movement of sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0040] Figure 1 The present invention is a method flow chart of an example.

[0041] Figure 2 Schematic diagram of the sound source orientation and sound source distance of an example of the present invention.

[0042] Figure 3 Flow chart of step S3 in the example of the present invention.

[0043] Figure 4 Flow chart of step S4 in the example of the present invention.

[0044] Figure 5 It is the average relative distance discrimination threshold result of 10 subjects under fixed stimulation in the example of the present invention.

[0045] Figure 6 It is the average relative distance discrimination threshold result of 9 subjects under normalized stimulation in the example of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present application is further described below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0047] Example:

[0048] like Figure 1As shown, according to the audio signal pickup control method based on the relative distance discrimination threshold according to the embodiment of the present application, it includes: S1, synthesizing near-field HRTF using sound source information and head parameters; S2, constructing virtual spatial audio according to the near-field HRTF; S3, using the equal loudness matching method to make the stimuli presented by the air conduction device and the bone conduction device have the same loudness level within the measurement frequency range; S4, using the air conduction device and the bone conduction device to play the virtual spatial audio, and measuring the relative distance discrimination threshold of the subject in different directions and different distances by using the binomial forced choice method and the 2-up 1-down adaptive method. value; S5. Use the relative distance discrimination threshold measurement results of the subjects to calculate the relative distance discrimination threshold of the audience, and obtain the relative distance discrimination threshold of the audience at different directions and distances; S6. Set the relative distance discrimination threshold evaluation standard, and judge whether it is necessary to increase the spherical harmonic domain beamforming algorithm for regulation according to the audience's distance perception ability of the object's tiny movement; S7. For the audience with poor distance perception ability of the object's tiny movement, use the spherical harmonic domain based on the beamforming algorithm to decouple the angle and frequency in the sound field to enhance the distance perception ability of the object's tiny movement.

[0049] In an embodiment of the present application, the step S1 synthesizes a near-field HRTF using sound source information and head parameters. It should be understood that the sound rendering of the traditional high-precision HRTF (head-related transfer function) needs to simulate the transmission process of sound waves from the sound source to the ears, which includes air filtering, reverberation of the surrounding environment, and scattering and reflection of the human body (torso, head, auricle, etc.). This process involves the calculation of complex physical properties and environmental characteristics, and requires a large number of mathematical operations, so the demand for computing resources is very high. Based on this, in order to reduce this computational burden, a preset method is used to convert the traditional far-field HRTF into an HRTF suitable for a near-field sound source environment using sound source information and head parameters. This method simulates sound source positioning by pre-measuring and establishing a detailed HRTF database, thereby reducing the need for real-time calculations.

[0050] Specifically, in the embodiment of the present application, Figure 2 The two sound source directions (90° and 180°) and two reference distances (50cm and 100cm) of the sound source orientation and sound source distance diagram shown in the figure are used to synthesize near-field HRTF data. Among them, 50 HRTFs are synthesized at intervals of 1cm in the distance range of 100cm to 50cm; 50 HRTFs are synthesized at intervals of 0.5cm in the range of 50cm to 25cm; and 50 HRTFs are synthesized at intervals of 0.25cm in the range of 25cm to 12.5cm. A total of 300 HRTF data are synthesized, covering two directions (90° and 180°).

[0051] In particular, the far-field HRTF database used is the SCUT HRTF database measured by South China University of Technology, which provides HRTF data in the range of 20cm to 100cm, specifically including 20cm, 25cm, 30cm, 40cm, 50cm, 60cm, 70cm, 80cm, 90cm, and 100cm, that is, HRTF data of 10 distances in one direction.

[0052] The HRTF data at 100 cm is selected and combined with the distance variation function (DVF) to synthesize the required near-field HRTF. The distance variation function is as follows:

[0053]

[0054] Among them, h l (kr) is the spherical Hankel function of the first kind, with order m, h′ l (ka) is its first derivative at radius a, k = ω / c is the wave number, c is the speed of sound in air, P l is the l-order Legendre polynomial, and θ is the angle between the vector from the center of the sphere to point X on the sphere and the vector from the center of the sphere to the sound source. By calculating all frequencies of interest, the transfer function for a specific sound source position at point X can be obtained. n is the pressure at the ear from a sound source at a near-field distance n, p f is the pressure at the ear due to a sound source at far-field distance f.

[0055] In particular, the distance change function is a mathematical model designed based on the distance change law between the sound source and the receiver, which can obtain the required near-field HRTF as needed. In this way, we can simulate the pressure of the sound source at the ear at the near-field distance, which matches the pressure of the sound source at the ear at the far-field distance. This method based on a preset database and mathematical model effectively reduces the computational load of real-time processing, so that even on devices with limited computing resources, a relatively accurate sound source localization effect can be achieved.

[0056] In the embodiment of the present application, the step S2 constructs the virtual spatial audio according to the near-field HRTF. It should be understood that in order to reproduce the spatial distribution of sound in the virtual environment, including the direction and distance of the sound, thereby providing a more realistic auditory experience, the synthesized near-field HRTF is further used to construct the virtual spatial audio environment.

[0057] Specifically, step S2 includes: convolving the synthesized near-field HRTF and Gaussian white noise to obtain spatial audio corresponding to different positions, wherein the Gaussian white noise used is generated by Audition software and is 100 Hz-8000 Hz.

[0058] In the embodiment of the present application, the step S3 uses the equal loudness matching method so that the stimulation presented by the air conduction device and the bone conduction device has the same loudness level within the measurement frequency range. It should be understood that due to the different response sensitivities of the human ear to different frequencies, especially in the frequency range of 2000Hz to 5000Hz, the human ear is most sensitive to the intermediate frequency, while at the low and high frequencies, the sensitivity of the human ear decreases. Therefore, in order to maintain the consistency of the auditory experience of sounds of different frequencies, the loudness of sounds of different frequencies is adjusted by the equal loudness matching method so that the loudness perceived by the human ear remains consistent at different volume levels. That is, the equal loudness matching method is used so that the stimulation presented by the air conduction device and the bone conduction device has the same loudness level within the measurement frequency range.

[0059] Specifically, Figure 3 As shown, step S3 includes: S31, adjusting the air conduction device to ensure that the sound pressure level of the played spatial audio is 60dB SPL; S32, the subject wears Sennheiser IE800 air conduction headphones and Radioear B81 bone conduction vibrator, and takes off the air conduction headphones when controlling the bone conduction vibrator to play the spatial audio signal (that is, alternately using the air conduction device and the bone conduction device to play the spatial audio, and testing the loudness perception of the air conduction device and the bone conduction device at the same sound source position respectively); S33, the subject adjusts the gain applied to the bone conduction vibrator to match the perceived loudness of the air conduction headphone stimulation.

[0060] The specific implementation process of step S31 is as follows: wear a Sennheiser IE800 air conduction headset on a head and torso simulator (HATS, B&K 4128); play spatial audio at 180° and 50cm through the air conduction headset, measure the sound pressure level using HATS, and record the device configuration when the sound pressure level is 60dB SPL. This step is to ensure that in the subsequent loudness matching process, the audio played by the air conduction device has a known reference sound pressure level.

[0061] Among them, the specific implementation process of step S33 is: controlling the gain of the air conduction headphone stimulation to remain unchanged, comparing the perceived loudness of the bone conduction vibrator stimulation with the perceived loudness of the air conduction headphone stimulation, if the perceived loudness of the bone conduction vibrator is greater than the perceived loudness of the air conduction headphone stimulation, then reducing the perceived loudness of the bone conduction vibrator; if the perceived loudness of the bone conduction vibrator is less than the perceived loudness of the air conduction headphone stimulation, then increasing the perceived loudness of the bone conduction vibrator.

[0062] In the embodiment of the present application, the step S4 uses an air conduction device and a bone conduction device to play virtual space audio, and measures the relative distance discrimination threshold of the subject in different directions and at different distances by using a two-item forced choice method and a 2-up 1-down adaptive method. It should be understood that in virtual reality and augmented reality applications, understanding and simulating human perception of the distance of the sound source is essential for creating a more realistic auditory experience. Therefore, in order to better predict the performance problems of the bone conduction device in the ability to perceive the tiny movement of objects, and to provide a basis and direction for improving and enhancing performance, the relative distance discrimination threshold of the subject is further measured by using a two-item forced choice method and a 2-up 1-down adaptive method, and the difference in the ability of the air conduction device and the bone conduction device to detect subtle sound movements is analyzed. The results of the analysis are used to predict the performance problems of the bone conduction device in the ability to perceive the tiny movement of objects, and to provide a basis and direction for improving and enhancing performance.

[0063] Specifically, Figure 4 As shown, step S4 includes: S41, conducting a preliminary experiment for each subject; S42, setting a reference distance of the sound source in the formal experiment, and configuring an initial adaptive virtual distance for the sound source in different directions; S43, playing the sound source in different stimulation orders, and using a two-item forced choice method, the subject judges two consecutively played sound stimuli respectively; S44, using a 2-up 1-down adaptive method, when the subject makes two consecutive correct judgments, the sound source moves away from the reference distance by 1%, and when the subject makes an incorrect judgment, the sound source moves closer to the reference distance by 1%, until the adaptive virtual distance of the sound source is consistent with the reference distance, and stops; wherein, before the first incorrect judgment, the size of each step is set to 3%.

[0064] The specific implementation process of step S41 is as follows:

[0065] 1. Have the subjects listen to horizontal surround sound through air conduction or bone conduction devices to familiarize themselves with the virtual sound field.

[0066] Control the air conduction earphone or bone conduction vibrator to play the spatial audio at the same distance, where the spatial audio is the surround sound at 7 azimuth angles (0°, 30°, 60°, 90°, 120°, 150°, 180°) at 25 cm, and then play the surround sound at 50 cm and 100 cm.

[0067] 2. Let the subjects listen to stimulation signals from near to far in different azimuths in turn, so that the subjects can become familiar with the virtual sound sources at different distances.

[0068] Control the air conduction earphone or bone conduction vibrator to play the spatial audio at the same azimuth, where the azimuth of the spatial audio includes 0°, 90° and 180°, the distance is from 20cm to 100cm, and the interval is 10cm (i.e. 20cm, 30cm, 40cm, 50cm, 60cm, 70cm, 80cm, 90cm, 100cm).

[0069] 3. Give each subject a brief training session consisting of several trials with feedback to familiarize them with the task and stimuli. During the formal test, no feedback will be provided to the subject.

[0070] In particular, in the embodiment of the present application, the above formal experiment covers four main variables: two directions (such as 180° and 90°), two reference distances (such as 50cm and 100cm), two stimulation orders (approaching stimulation order and far away stimulation order) and two conduction methods (air conduction device and bone conduction device). The combination of these variables forms a total of 24 experimental conditions. Among them, playing the spatial audio at the reference distance first and then playing the spatial audio at the adaptive position is defined as the approaching stimulation order, and conversely playing the spatial audio at the adaptive distance first and then playing the spatial audio at the reference position is defined as the far away stimulation order.

[0071] Among them, for the sound source in the 90° direction, the initial adaptive virtual distance is set to 70% of the reference distance (for example, when the reference distance is 50cm, the initial adaptive virtual distance is set to 35cm), and a larger initial adaptive virtual distance is set for the sound source in the 180° direction. At the same time, the two spatial audios are combined into an experimental trial, in which the spatial position corresponding to one spatial audio is the position at the reference distance, and the spatial position corresponding to the other spatial audio is the position at the adaptive distance. Among them, the Gaussian white noise used in the construction of the spatial audio is 0.5 seconds long, including a rise time of 30 milliseconds and a fall time of 30 milliseconds. Here, the purpose of setting the rise and fall time is to ensure a smooth transition of the sound signal and avoid sharp sound wave interference caused by the sudden start or end of the sound. This not only helps to improve the accuracy and reliability of the measurement, but also enhances the auditory experience of the subjects and reduces the impact of external noise on the experimental results.

[0072] In particular, during the above experiment, the subjects sat in a comfortable chair and held a transponder. The transponder was equipped with two buttons, the red button was used to indicate that the second sound was closer than the first sound, and the blue button indicated that the second sound was farther than the first sound. The transponder and the sound card were connected to the computer running the experimental program. In order to avoid visual interference, the computer screen was turned off during the experiment to ensure that the subjects could focus on the auditory task. During the experiment, the subjects pressed the corresponding red or blue button in time to give feedback according to the difference in the sound they heard. During the break, the subjects could choose to turn on the computer screen, and the countdown to the next trial block would be displayed on the screen to help them understand the progress of the experiment and prepare. The entire experimental environment was kept quiet to ensure the accuracy of the data. At the same time, the transponder and sound card were checked before the experiment to ensure that all equipment was working properly and the button functions were correct.

[0073] It is worth mentioning that the above-mentioned formal experiment (hereinafter referred to as "Experiment 1") was conducted in the presence of loudness (that is, the synthesized near-field HRTF was synthesized using a distance variation function, and then convolved with Gaussian white noise to obtain spatial audio). Therefore, in order to study the effects of direction, reference distance, stimulation order and conduction method on the relative distance discrimination threshold while eliminating the loudness factor, another experimental method is provided (hereinafter referred to as "Experiment 2"). In this experiment, in order to remove the loudness factor, a compensation coefficient K related to the sound source distance r was used to normalize the stimulus signals at different distances, so that the spatial audio at different distances in the same direction presents the same loudness. In this way, the original audio signal is compensated by the compensation coefficient to obtain the spatial audio required for Experiment 2. The calculation formula for the compensation coefficient is as follows:

[0074]

[0075] where d r is the distance from the sound source to the right ear, d l It is the distance from the sound source to the left ear. The specific usage is as follows:

[0076] S′(n)=K*S(n)

[0077] Where S′(n) is the normalized audio signal, and S(n) is the original audio signal.

[0078] Experiment 2 is similar to Experiment 1. Before conducting Experiment 2, the preparatory experiment of step S41 above is also required. Experiment 2 also covers four main variables: two directions (such as 180° and 90°), two reference distances (such as 50 cm and 100 cm), two stimulation sequences (approaching stimulation sequence and moving away stimulation sequence), and two conduction methods (air conduction device and bone conduction device), resulting in a total of 24 experimental conditions.

[0079] Optionally, after compensating the audio signal to obtain a normalized audio signal, the amplitude of the audio signal is randomized by 15 dB SPL. By randomly adjusting the sound pressure level, a fixed sound pressure level may cause participants to adapt to the stimulus signal, reduce their sensitivity, or form fixed expectations. By randomizing the sound pressure level, this adaptability can be broken, maintaining the participants' high sensitivity and attention to each stimulus, and improving the effectiveness of the experiment. At the same time, randomizing the sound pressure level increases the diversity of experimental conditions, making the experimental results less susceptible to the limitations of specific sound pressure level settings. This helps to improve the robustness of the experimental results and the generalization ability in different environments or conditions.

[0080] After the above experiments 1 and 2, we can get Figure 5 The average relative distance discrimination thresholds of the 10 subjects in the fixed stimulus condition and Figure 6 The average relative distance discrimination threshold of the 9 subjects under normalized stimulation is shown. After obtaining the relative distance discrimination threshold calculated by measuring the air conduction earphone and the bone conduction vibrator, the data can be analyzed by variance analysis, and a multi-factor repeated measures variance analysis can be performed to analyze the influence of factors such as direction, reference distance, and stimulation order on the relative distance discrimination threshold. Further, repeated measures variance analysis can be used to analyze the relative distance discrimination threshold of the bone conduction vibrator and the air conduction earphone to check whether there is a significant difference, thereby providing a basis and direction for improving and enhancing the near-field spatial sound reproduction performance of the bone conduction device.

[0081] In the embodiment of the present application, the step S5 uses the relative distance discrimination threshold measurement results of the subjects to calculate the relative distance discrimination threshold of the audience, and obtains the relative distance discrimination threshold of the audience at different positions and distances. It should be understood that by measuring the relative distance discrimination thresholds under different sound source directions, different sound source reference distances, different stimulation sequences, different conduction devices, and different stimulation types, the audience's distance perception ability of object movement under different circumstances can be estimated, and the minimum distance required for the audience to perceive the movement of the object can be obtained. The distance perception ability of hearing aids and consumer-grade bone conduction headphones is evaluated, and then hearing devices that enable the audience to obtain a better ability to judge the movement of objects are designed accordingly. Therefore, the relative distance discrimination threshold measurement results of the subjects are further used to calculate the relative distance discrimination threshold of the audience, and the relative distance discrimination threshold of the audience at different positions and distances is obtained.

[0082] Specifically, in step S5, the relative distance discrimination threshold is calculated as follows:

[0083]

[0084] JND r is the relative distance discrimination threshold at the reference distance r, dr is the reference distance, d′ a It is the average of the distances corresponding to the last 5 flip positions.

[0085] In the embodiment of the present application, the step S6 sets the relative distance discrimination threshold evaluation standard, and determines whether it is necessary to add the spherical harmonic domain beamforming algorithm for regulation based on the audience's distance perception ability of the object's tiny movement. Specifically, in the step S6, a relative distance discrimination threshold j is set to evaluate the individual's spatial perception ability. If the audience's relative distance discrimination threshold is greater than j, it means that their distance perception ability for the movement of the object is poor. Even if the object moves a large distance, the audience cannot perceive it. At this time, it is necessary to add the spherical harmonic domain beamforming algorithm for enhancement. On the contrary, if the audience's relative distance discrimination threshold is less than j, there is no need to add the spherical harmonic domain beamforming algorithm for enhancement.

[0086] In the embodiment of the present application, step S7 uses a beamforming algorithm based on spherical harmonics to decouple the angle and frequency in the sound field for listeners with poor distance perception of small movements of objects, so as to enhance the distance perception of small movements of objects. Specifically, step S7 uses spherical harmonic domain processing (SH-Domain) to decompose the pressure distribution in the sound field into angle-related components and radial-related components, and completes filtering through a series of weighted calculations of spherical harmonic functions and mode intensity factors.

[0087] In particular, the spherical harmonics-based beamforming algorithm is an orthogonal polynomial-based radial filter design method for separating near-field sound sources. That is, the algorithm constructs a radial filter by using a family of orthogonal polynomials, which performs better in terms of white noise gain (WNG) and directivity index (Directivity Index) than previous methods. In addition, some orthogonal polynomials are more effective in separating sound sources located near the surface of the microphone array, where the formula for calculating the radial filter coefficient d is:

[0088] d(k)=(C -1 ) T ·q

[0089] Where q is the response mode corresponding to different orders, and C is the radial coefficient matrix predetermined in .

[0090] Further filtering can be obtained:

[0091]

[0092] where r s is the distance to the target sound source, and k is the wave number. For an array with radius a, kmax Need to meet k max =N / a, where N is the order of the array. In addition, for the near-field sound source, the distance r of the sound source s Need to meet a <r s <r NF , where r NF is the boundary radius of the near field, and its value is r NF ≈N / k.

[0093] In particular, in addition to the above-mentioned spherical harmonic beamforming algorithm, other algorithms suitable for positioning in the near field can also be used for control, for example, near field S 2 H-MVDR, near field S 2 H-MUSIC.

[0094] In summary, the audio signal pickup and control method based on the relative distance discrimination threshold according to the embodiment of the present application is explained. This method uses a synthetic near-field HRTF to construct spatial audio with different directions, different distances and different stimulation orders to measure and analyze the relative distance discrimination threshold of the subject. The analysis results are used to predict the differences in the relative distance discrimination thresholds of the air conduction and bone conduction devices in the near field range in terms of the sound source direction, sound source distance and sound source stimulation order, and calculate the relative distance discrimination threshold of the listener. For listeners with a larger relative distance discrimination threshold, the beamforming method of spherical harmonics is used to pick up the audio at the corresponding position to increase their distance perception of the corresponding position. Through this method, the distance perception ability of hearing aids and consumer-grade bone conduction headphones can be evaluated, and then hearing devices that enable listeners to gain a better ability to judge the movement of objects can be designed accordingly.

[0095] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. A method for controlling audio signal pickup based on relative distance discrimination threshold, characterized in that: include: S1, synthesize near-field HRTF using sound source information and head parameters; S2, constructing virtual spatial audio according to the near-field HRTF; S3. Use equal loudness matching method to make the stimuli presented through air conduction device and bone conduction device have the same loudness level within the measurement frequency range; S4. Use air conduction devices and bone conduction devices to play virtual space audio, and measure the subjects' relative distance discrimination thresholds in different directions and distances using a two-item forced choice method and a 2-up 1-down adaptive method; S5. Calculate the relative distance discrimination threshold of the listeners using the relative distance discrimination threshold measurement results of the subjects, and obtain the relative distance discrimination thresholds of the listeners at different positions and distances; S6. Setting a relative distance discrimination threshold evaluation standard, and judging whether it is necessary to add a spherical harmonic domain beamforming algorithm for regulation according to the audience's ability to perceive the distance of small movements of objects; S7. For listeners who have poor distance perception of tiny object movements, a beamforming algorithm based on spherical harmonics is used to decouple the angle and frequency in the sound field to enhance the distance perception of tiny object movements. The S4 comprises: S41, a preliminary experiment was conducted for each subject; S42, setting the reference distance of the sound source in the formal experiment and configuring the initial adaptive virtual distance for the sound source in different directions; S43, the sound sources were played in different stimulus orders, and the subjects made judgments on two consecutively played sound stimuli using a binary forced choice method; S44, using a 2-up 1-down adaptive method, when the subject makes two correct judgments in a row, the sound source moves away from the reference distance by 1%, and when the subject makes an incorrect judgment, the sound source moves closer to the reference distance by 1%, until the adaptive virtual distance of the sound source is consistent with the reference distance; before the first incorrect judgment, the size of each step is set to 3%.

2. The audio signal pickup control method based on relative distance discrimination threshold according to claim 1 is characterized in that: In S1, the distance change function is as follows: Among them, h l (kr) is the spherical Hankel function of the first kind, with order m, h′ l (ka) is its first derivative at radius a, k = ω / c is the wave number, c is the speed of sound in air, p l is the l-th order Legendre polynomial, θ is the angle between the vector from the center of the sphere to point X on the sphere and the vector from the center of the sphere to the sound source, p n is the pressure at the ear from a sound source at a near-field distance n, p f is the pressure at the ear due to a sound source at far-field distance f.

3. The audio signal pickup control method based on relative distance discrimination threshold according to claim 2 is characterized in that: The S2 includes: convolving the synthesized near-field HRTF and Gaussian white noise to obtain spatial audio corresponding to different positions, wherein the Gaussian white noise used is generated by Audition software and is 100 Hz-8000 Hz.

4. The audio signal pickup control method based on relative distance discrimination threshold according to claim 3 is characterized in that: The S3 includes: S31, adjust the air conduction equipment to ensure that the spatial audio sound pressure level of the playback is 60dB SPL; S32, the subjects wore Sennheiser IE800 air conduction headphones and Radioear B81 bone conduction transducers, and took off the air conduction headphones when controlling the bone conduction transducers to play the spatial audio signals; At S33, the subject adjusted the gain applied to the bone conduction transducer to match the perceived loudness of the air conduction headphone stimulus.

5. The audio signal pickup control method based on relative distance discrimination threshold according to claim 4 is characterized in that: The S33 includes: controlling the gain of the air conduction headphone stimulation to remain unchanged, comparing the perceived loudness of the bone conduction vibrator stimulation with the perceived loudness of the air conduction headphone stimulation, and if the perceived loudness of the bone conduction vibrator is greater than the perceived loudness of the air conduction headphone stimulation, reducing the perceived loudness of the bone conduction vibrator; if the perceived loudness of the bone conduction vibrator is less than the perceived loudness of the air conduction headphone stimulation, increasing the perceived loudness of the bone conduction vibrator.

6. The audio signal pickup control method based on relative distance discrimination threshold according to claim 5, characterized in that: The S41 includes: Ask the subjects to listen to the surround sound in the horizontal plane through air conduction equipment or bone conduction equipment to familiarize themselves with the virtual sound field; The subjects were given stimulation signals from near to far in different azimuths in sequence, so that they could become familiar with the virtual sound sources at different distances. Each subject was given a brief training session consisting of several trials with feedback to familiarize them with the task and stimuli.

7. The audio signal pickup control method based on relative distance discrimination threshold according to claim 6, characterized in that: In the case of eliminating the loudness factor, the compensation coefficient K related to the sound source distance r is used to normalize the stimulus signals at different distances, so that the spatial audio at different distances in the same direction presents the same loudness; the calculation formula of the compensation coefficient is as follows: where d r is the distance from the sound source to the right ear, d l is the distance from the sound source to the left ear; S′(n)=K*S(n) Where S′(n) is the normalized audio signal, and S(n) is the original audio signal.

8. The audio signal pickup control method based on relative distance discrimination threshold according to claim 7 is characterized in that: In S5, the relative distance discrimination threshold is calculated as follows: JND r is the relative distance discrimination threshold at the reference distance r, d r is the reference distance, d′ a It is the average of the distances corresponding to the last 5 flip positions.

9. The audio signal pickup control method based on relative distance discrimination threshold according to claim 8, characterized in that: The S6 includes: setting a relative distance discrimination threshold j to evaluate the individual's spatial perception ability. If the relative distance discrimination threshold of the listener is greater than j, it means that the listener's perception ability of the distance of the moving object is poor. Even if the moving distance of the object is large, the listener cannot perceive it, and it is necessary to add a beamforming algorithm in the spherical harmonic domain for enhancement. On the contrary, if the relative distance discrimination threshold of the listener is less than j, it is not necessary to add a beamforming algorithm in the spherical harmonic domain for enhancement.

10. The audio signal pickup control method based on relative distance discrimination threshold according to claim 9, characterized in that: In S7, the formula for calculating the radial filtering coefficient d is: d(k)=(C -1 ) T ·q Where q is the response mode corresponding to different orders, C is the radial coefficient matrix predetermined; To filter: where k is the wave number, r s is the distance to the target sound source, r s Satisfy a <r s <r NF , where a is the radius, r NF is the boundary radius of the near field, and its value is r NF ≈N / k.

Images