Method for finding the most suitable hrtf in a hrtf database

Abstract

A method for audio personalization for a first user includes testing the first user in a calibration test, the calibration test including asking the user to match test sounds to test positions by controlling a position of a presented sound or controlling a position of a presented localization for a series of test matches, each test sound presented at a position using a default head-related transfer function (HRTF), receiving an estimate of each matched position from the first user, and calculating a respective error of each estimate to generate a series of position estimate errors for the first user; and comparing at least some of the position estimate errors for the first user to position estimate errors for a same position generated previously for a subset of a corpus of at least reference individuals; identifying a reference individual whose compared position estimate errors most closely match the compared position estimate errors for the first user; and using an HRTF previously obtained for the identified reference individual for the first user.

Description

Technical Field

[0001] This invention relates to methods and systems for audio personalization. Background Technology

[0002] Consumers of media content (including interactive content such as video games) enjoy immersion while engaging with it. For pre-recorded content, the existence of the content is a fixed, tacit understanding, but this is true for video and audio. However, for interactive content such as video games, where the content and its viewpoint typically change with user input, a similar responsiveness is expected for audio.

[0003] This invention aims to reduce or alleviate this need. Summary of the Invention

[0004] Various aspects and features of the present invention are defined in the appended claims and the text of the appended description, and include at least:

[0005] - In a first aspect, an audio personalization method for a first user is provided, comprising the following steps:

[0006] The calibration test is conducted on the first user and includes:

[0007] Users are required to match test sounds to test locations by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matching.

[0008] Each test sound is presented at its location using the default Head-Related Transfer Function (HRTF).

[0009] Receive estimates for each matched location from the first user, and

[0010] Calculate the corresponding error for each estimate to generate a series of location estimation errors for the first user; and

[0011] At least some of the location estimation errors of the first user are compared with the same location estimation errors previously generated for at least a subset of the corpus of the reference individuals;

[0012] Identify the reference individual whose location estimation error best matches the location estimation error of the first user; and

[0013] The HRTF previously obtained for the identified reference individual is used for the first user.

[0014] - On the other hand, an audio personalization method for reference individuals is provided, comprising the following steps:

[0015] Obtain the head-related transfer functions "HRTF" for each reference individual's corpus;

[0016] The calibration test involves testing each reference individual, and the calibration test for each reference individual includes:

[0017] The reference individual is required to match the test sound to the test location by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matching.

[0018] Each test sound was presented at a series of different locations using the same default head-related transfer function "HRTF".

[0019] Receive an estimate of each matched location from the reference individual, and

[0020] Calculate the corresponding error for each estimate to generate a series of localization estimation errors for the corresponding measured reference individual; and

[0021] The series of positioning estimation errors of the reference individuals are correlated with their respective obtained HRTFs.

[0022] - On the other hand, an audio personalization system for a first user is provided, comprising:

[0023] The test processor is configured to test a first user in a calibration test, which includes:

[0024] Users are required to match test sounds to test locations by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matches. Each test sound is presented at the location using the default Head Related Transfer Function "HRTF".

[0025] Receive estimates for each matched location from the first user, and

[0026] Calculate the corresponding error for each estimate to generate a series of location estimation errors for the first user; and

[0027] The comparison processor is configured to compare at least some of the localization estimation errors of the first user with the same localization estimation errors previously generated for at least a subset of a corpus of a reference individual;

[0028] The comparison processor is configured to identify a reference individual whose location estimation error best matches the location estimation error of the first user; and

[0029] The HRTF processor is configured to use the HRTF previously obtained for the identified reference individual for the first user.

[0030] - On the other hand, an audio personalization system for reference individuals is provided, including:

[0031] The memory is configured to store the head-related transfer functions "HRTF" of the corpus of the reference individuals;

[0032] A test processor is configured to test individual references in a calibration test, the calibration test for each reference reference including:

[0033] The reference individual is required to match a test sound to a test location by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matches. Each test sound is presented at a series of different locations using the same default Head Relevant Transfer Function (HRTF).

[0034] Receive an estimate of each matched location from the reference individual, and

[0035] Calculate the corresponding error for each estimate to generate a series of localization estimation errors for the corresponding measured reference individual; and

[0036] The association processor is configured to associate the series of positioning estimation errors of the reference individual with their respective obtained HRTFs. Attached Figure Description

[0037] When considered in conjunction with the accompanying drawings, the present disclosure and its many accompanying advantages become better understood by referring to the following detailed description, thereby readily obtaining a more complete understanding of the present disclosure and its many accompanying advantages, and in the accompanying drawings:

[0038] - Figure 1 This is a schematic diagram of an entertainment device according to an embodiment of this specification;

[0039] - Figure 2A and Figure 2B This is a schematic diagram of head-related audio characteristics;

[0040] - Figure 3A and Figure 3B This is a schematic diagram of ear-related audio characteristics;

[0041] - Figure 4A and Figure 4B This is a schematic diagram of an audio system used to generate data for calculating head-related transfer functions, according to embodiments of this specification.

[0042] - Figure 5 This is a schematic diagram of the impulse response of the user's left and right ears in the time and frequency domains;

[0043] - Figure 6This is a schematic diagram of the head-related transfer function spectrum of the user's left and right ears;

[0044] - Figure 7 This is a flowchart of a method for audio personalization for a first user according to an embodiment of this specification; and

[0045] - Figure 8 This is a flowchart of a method for audio personalization for a reference individual according to an embodiment of this specification. Detailed Implementation

[0046] An audio personalization method and system are disclosed. Several specific details are presented in the following description to provide a thorough understanding of embodiments of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. Instead, for clarity, specific details known to those skilled in the art have been omitted where appropriate.

[0047] In exemplary embodiments of the invention, a suitable system and / or platform for implementing the methods and techniques described herein may be an entertainment device, such as a Sony PlayStation. ® 4 or 5 video game consoles.

[0048] For illustrative purposes, the following description is based on PlayStation 4. ® However, this is to be understood as a non-restrictive example.

[0049] Referring now to the accompanying drawings, in which similar reference numerals in all the views denote the same or corresponding parts. Figure 1 The illustration shows Sony ® PlayStation 4 ® The overall system architecture of the entertainment equipment. System unit 10 is provided, through which various peripheral devices can be connected.

[0050] System unit 10 includes an acceleration processing unit (APU) 20 as a single chip, which in turn includes a central processing unit (CPU) 20A and a graphics processing unit (GPU) 20B. APU 20 can access random access memory (RAM) unit 22.

[0051] The APU 20 may optionally communicate with the bus 40 via I / O bridge 24, which may be a discrete component or part of the APU 20.

[0052] Connected to bus 40 are data storage components, such as hard disk drive 37 and Blu-ray discs operable to access data on compatible optical disc 36A. ®Driver 36. Additionally, RAM cell 22 can communicate with bus 40.

[0053] Optionally, the auxiliary processor 38 is also connected to the bus 40. The auxiliary processor 38 can be provided to run or support an operating system.

[0054] System unit 10 may use audio / video input port 31 or Ethernet as appropriate. ® Port 32, Bluetooth ® Wireless Link 33, Wi-Fi ® A wireless link 34 or one or more Universal Serial Bus (USB) ports 35 communicate with peripheral devices. Audio and video can be output via AV output 39 (such as HDMI). ® (Port) output.

[0055] Peripheral devices may include single-view or stereoscopic cameras 41 (such as PlayStation) ® Eye); Stick-type video game controller 42 (such as PlayStation) ® Move) and traditional handheld video game controllers 43 (such as DualShock) ® 4); Portable entertainment devices 44 (such as PlayStation) ® Portable and PlayStation ® Vita); keyboard 45 and / or mouse 46; media controller 47 in the form of a remote control; and headset receiver 48. Similarly, other peripheral devices, such as printers or 3D printers (not shown), may be considered.

[0056] Optionally, the GPU 20B, in conjunction with the CPU 20A, generates video images and audio for output via AV output 39. Optionally, the audio may be generated in conjunction with an audio processor (not shown) or alternatively generated by an audio processor (not shown).

[0057] Video and optional audio can be presented to a television 51. If supported by the television set, the video can be stereo. Audio can be presented to the home theater system 52 in one of several formats, such as stereo, 5.1 surround sound, or 7.1 surround sound. Video and audio can also be presented to a head-mounted display unit 53 worn by the user 60.

[0058] During operation, entertainment devices default to the operating system, such as FreeBSD. ® A variant of version 9.0. The operating system can run on a CPU 20A, a coprocessor 38, or a combination of both. The operating system provides a graphical user interface to the user, similar to that of PlayStation.® Dynamic Menu. The menu allows users to access operating system features and select games and other optional content.

[0059] When playing such games or other content, users will typically receive audio from a stereo or surround sound system 52 or headphones while viewing content on a static display 51, or similarly from a stereo surround sound system 52 or headphones while viewing content on a head-mounted display (“HMD”) 53.

[0060] In either case, while the positional relationship between objects in the game and the static screen or the user's head position (or a combination of both) can be visually displayed relatively easily, generating the corresponding audio effects is more difficult.

[0061] This is because an individual's perception of the direction of sound depends on the physical interaction with the sounds around them caused by the physical characteristics of their head; however, each person's head is different, so the physical interaction is unique.

[0062] refer to Figure 2A An example physical interaction is interaural delay or time difference (ITD), which indicates the degree to which sound is localized to the left or right of the user (resulting in a relative change in arrival time at the left and right ears), and is a function of the listener’s head size and face shape.

[0063] Similarly, refer to Figure 2B Interaural sound level difference (ILD) is related to the different loudness of the left and right ears and indicates the degree to which the sound is localized to the left and right of the user (due to the different degrees of attenuation caused by the relative obstruction of the sound source by the ears), and is also a function of head size and face shape.

[0064] In addition to this horizontal (left-right) distinction, also refer to Figure 3A The outer ear includes asymmetrical features that vary between individuals and provide additional vertical differentiation of incoming sounds; reference Figure 3B The small difference in path length between the direct and reflected sound from these features results in what is known as a spectral notch, which changes frequency depending on the height of the sound source.

[0065] Furthermore, these characteristics are not independent; horizontal factors (such as ITD and ILD) also vary with source height due to the changing facial / head contours encountered by the sound waves propagating to the ear. Similarly, vertical factors (such as spectral notch) also change with left / right positioning, as the physical shape of the ear relative to the incoming sound and the resulting reflections also change with the horizontal angle of incidence.

[0066] The result is a complex two-dimensional response for each ear, which is a function of monoauricular cues (such as spectral notch) and binaural or interauricular cues (such as ITD and ILD). The individual's brain learns to associate this response with the physical source of the object, enabling them to distinguish between left and right, up and down, and actually front and back, in order to estimate the object's location in 3D relative to the user's head.

[0067] The goal (e.g., using headphones) is to provide the user with sounds that replicate these characteristics in order to create the illusion that objects in the game (or other sound sources in other forms of consumer content) are located at specific points in space relative to the user, just as they would in the real world. This type of sound is often referred to as binaural sound.

[0068] However, it should be understood that because each user is unique and therefore requires unique feature replication, this will be difficult to do without extensive testing.

[0069] In particular, it is necessary to determine the user's intraauricular response at multiple locations, such as within a sphere surrounding them; Figure 4A A fixed speaker arrangement for this purpose is shown, while Figure 4B A simplified system is shown, in which, for example, a speaker assembly or user can rotate the sphere in fixed increments so that the speaker continuously fills the remaining sample points in the sphere.

[0070] refer to Figure 5 For each sampling location of sound (e.g., an impulse, such as a single increment or click), the recorded impulse response inside the ear is obtained (e.g., using a microphone positioned at the entrance of the ear canal), as shown in the graph above. The Fourier transform of these impulse responses produces a so-called head correlation transfer function (HRTF), which describes the effect of each ear of the user's head on the received spectrum at that point in space.

[0071] Measurements can be taken at many locations to calculate the complete HRTF, such as... Figure 6 The diagram partially illustrates both the left and right ears (showing the frequency on the y-axis and the azimuth on the x-axis). Brightness is a function of the Fourier transform value, where dark regions correspond to spectral notches.

[0072] Will understand, use systems (such as) Figure 4A and Figure 4B It is impractical to use the system shown in the diagram to obtain the HRTF of each of the potential tens of millions of users of entertainment devices, because it is providing individual users with some form of array system in order to perform self-testing.

[0073] Therefore, different techniques are disclosed in the embodiments of this specification.

[0074] In these embodiments, a system (such as Figure 4A and Figure 4B The system shown is used to obtain complete HRTFs from multiple reference individuals to generate an HRTF library. This library can initially be small, with, for example, a few individuals of different ages, ethnicities, and sexes representing those tested, or simply a random selection of volunteers, beta testers, quality assurance testers, early adopters, etc. However, over time, more and more individuals can be tested, with their generated HRTFs added to the library.

[0075] In addition to HRTF testing, each of these individuals undergoes calibration testing, for example, using the entertainment system and headphones described herein, or an HMD system (e.g., with headphones), or optionally a stereo or surround sound speaker system, and optionally two or more of these consecutively.

[0076] Calibration testing requires users to identify where in the space around them the sound appears to originate. For users wearing an HMD system, once the sound is played, they can look in the direction they perceive the sound to be coming from and can measure that direction (e.g., using head tracking and, where appropriate, gaze tracking techniques known in the art). Alternatively or additionally, they can use one or more handheld controllers to move scale lines or other indicators to the desired location. In the latter case, they can move the indicator to a position on the screen corresponding to the location from which the sound appears to originate, or if the screen displays the user's nominal position surrounded by a sphere or part of a sphere, they can use the controllers to move the indicator to the nominal location of the sound on the surface of that sphere.

[0077] Alternatively or additionally, other input methods may be considered, such as gesture input captured by a camera (e.g., pointing in the perceived direction from which the sound is coming), which can then be used to determine the direction.

[0078] Equivalently, the location can be presented to the user graphically, and the user must then control the source sound to that location; in this case, pointing or other direct controls would be inappropriate because this would not require the user to estimate the location of the sound source; instead, joystick or handle controls or motion gestures (e.g., horizontal and / or vertical translation) can be used to move the sound source. However, this method may be slower.

[0079] Therefore, more generally, users must attempt to match the presented sound to the presented location by controlling the position of the presented sound or the position of the presented location.

[0080] Individuals for which a complete HRTF is calculated and added to the library perform the test (identifying the location of the sound, or moving the sound to the identified location) using a sound transformed by the default HRTF (e.g., an HRTF calculated using a emulated head) to generate a default binaural sound signal.

[0081] Depending on how different the shape of an individual's head is from that of a emulated head, the default HRTF used to drive binaural sound in headphones or speakers will differ in a different way from their own natural HRTF. This, in turn, will affect their perception of where the sound source presented using the default HRTF actually is.

[0082] By testing the localization of multiple sound sources in this manner, the localization estimates of individuals (especially the error rate of the localization estimates) serve as a proxy describing how their individual HRTF differs from the default HRTF. This proxy can also be considered a fingerprint of the complete HRTF of the reference individual.

[0083] Subsequently, in embodiments of this specification, a user at home can perform the same calibration test. If more than one type of audio delivery component is supported, such as not only headphones (and / or HMD systems, where this is considered equivalent to headphones), then optionally, the user will indicate the type of audio system they are using (e.g., stereo or surround sound speakers, or headphones, or an HMD system with built-in headphones). This affects the subset of the default HRTF format used (headphones, surround sound, etc.) and the metric results of the reference individuals in the library that will be compared with the results of the user at home.

[0084] Then, users at home can perform the same calibration tests as the reference individual (for a set of locations, or to identify the location of the sound, or to move the sound to the identified location) to estimate the location of the sound source presented to them using the default HRTF.

[0085] Then, the closest pattern of the location estimation error in the set of indicator results is used to indicate the HRTF in the library that most closely matches the user's true HRTF.

[0086] The closest matching HRTF of this instruction can then be installed on the entertainment device as the user's HRTF, thereby providing the user with more realistic and accurate binaural sound.

[0087] Furthermore, user localization estimates for test sounds can be kept recorded; if new reference individuals are added to the library, user localization estimates can be tested against those new reference individuals to see if they match better, for example, as a background service provided by a remote server. If a better match is found, the closest matching HRTF with the better indication can be installed as the user's HRTF, thereby further improving their experience.

[0088] In this way, the HRTF of an entertainment device user can be estimated, for example, without placing the microphone in the user's ear canal or measuring any impulse response.

[0089] Advantageously, this allows tens of millions of potential users to enjoy good binaural sound, where the quality of the sound is improved as new reference individuals are added to the HRTF library.

[0090] It is also possible to wisely select the individuals chosen to expand the library; one can assume that for a representative set of reference individuals, the random distribution of users will map to each reference individual in roughly equal proportions; however, if a relatively large number of users map to reference individuals (e.g., above a threshold variance of the number of users mapped to reference individuals), this indicates at least one of the following:

[0091] i. The user group is not random (e.g., due to demographics), and therefore there are more people similar to the reference individual than the standard; and

[0092] ii. The reference set of individuals is insufficient to represent the user, and there are gaps in the metric outcome space around that particular reference individual, causing people who are not actually very similar to the individual to be mapped to them due to a lack of better matches.

[0093] In either case, it will be desirable to find other reference individuals that are morphologically similar to the current individual in the library, in order to provide more nuanced distinctions within that subgroup of the user population. Such individuals can be optionally identified, for example, by comparing photographs of candidate individuals (e.g., frontal and side views (showing the ears)) to help automatically assess head and outer ear shape. Other methods can also be used to find such individuals, such as identifying individuals with similar demographics or inviting relatives of existing individuals.

[0094] In this way, the HRTF library can optionally grow over time in response to the characteristics of the user base.

[0095] In cases where it is impossible to find a suitable new reference individual, or while waiting to add a reference individual to the library, optionally for users who are close to, but not within, the threshold matching degree of two or more reference individuals, a mixture of the HRFTs of two or more reference individuals may be generated to provide a better estimate of their own HRFT. This mixture may be a weighted average or other combination of the relative matching degree of the HRFTs of the two or more reference individuals (e.g., the proximity of the location estimation error value vectors in the location error space).

[0096] Optionally, as the database grows, and as the user base grows, the database can be pre-filtered for a given user based on demographic criteria; for example, based on one or more of age, gender, and ethnicity. The reference individual set, along with the calibration test results to be compared, can then be reduced to a subset that matches these basic demographics. Subsequently, the user is compared to the full corpus of the reference individual's metrics results only if the best match of the user's location estimate differs from the best match of the corresponding reference individual's location estimate by a threshold amount. This reduces the computational overhead of the server performing these comparisons while still allowing individuals who do not perfectly match their expected demographics (e.g., children with relatively large heads or adults with relatively small heads) to still find good matches within a wider reference individual database.

[0097] The above description assumes that a full calibration test is performed by a home user. A full calibration test may include locating sound at a large number of locations, typically on the surface of a sphere or part of a sphere, thereby capturing the impact of the interconnections between the horizontal and vertical audio characteristics of the previously discussed ITD, ILD, and spectral notch on the user's ability to estimate the localization of objects whose sound has been processed using the default HRTF.

[0098] Full calibration tests can be performed on a uniform grid of locations or on a non-linear distribution, such as one that is slightly more sound in the user’s normal field than just outside the user’s normal field, then more sound at the far left and far right, and then more sound behind the user, so that the test location density appears to spread out from the area in front of the user’s stationary line of sight to become sparsest behind the user.

[0099] Full calibration testing can also be focused on areas known to have particularly variable characteristics; one can consider, if... Figure 6Averaging multiple HRTF sets of the type shown (e.g., reference individuals of similar types (e.g., age, sex, race), or reference individuals based on other physiological measurements (or indicators such as hat size or sensed HMD fit circumference) such as head size) will reveal regions where the individual transfer function differs more than other regions, or in other words, the corresponding variance plots show areas where there are greater differences in calibration tests.

[0100] Therefore, there may be regions in space where reference individuals tend to exhibit larger estimation errors (e.g., variability above a threshold); for these reference individuals, additional tests in nearby locations can provide useful additional distinctions between them.

[0101] Similarly, when testing users, if large errors above this threshold are identified, corresponding additional tests in nearest neighbor localization can be used to improve the selection of results for corresponding reference individuals, and thus improve the selection of HRTFs. Furthermore, the locations corresponding to the large errors or errors that appear as outliers relative to candidate reference individuals can be revisited to see if the errors are consistent and repeatable. If consistent, the location can be retained and can be considered important (e.g., to suggest adding another reference individual, including possibly inviting the current user). If inconsistent, the location can be completely or partially ignored when searching for corresponding results for reference individuals.

[0102] In this way, the search space for calibration testing can be quickly improved.

[0103] Meanwhile, testing over a wide frequency range (e.g., bursts of white noise, or crackling and banging sounds) may be useful for some characteristics (e.g., some notch measurements), while testing over a narrower frequency range may be useful for others; for example, pink noise below approximately 1.5 kHz may be more useful for ITD-based estimations, while blue noise above 1.5 kHz may be more useful for ILD-based estimations. Other sounds (such as chirps or pure tones) can be used similarly, as can natural sounds (such as speech, music, or ambient noise). Therefore, a mixture of wideband and narrowband sounds can be used in calibration to better distinguish and characterize the impact of different aspects of a user's hearing on their localization estimations.

[0104] Calibration testing typically randomizes the selection of individual test locations within a predetermined set of locations to be tested, so that neither the reference individuals nor home users are aware of the progression patterns within the audio locations.

[0105] However, it should be understood that full calibration testing can be time-consuming and is undesirable or impractical for home users. It will also be understood, however, that testing can be performed incrementally, with additional test points added to the user's metric results and potentially improving the accuracy of the metric match with a reference individual.

[0106] Therefore, aspects of the test can be prioritized or performed in order of priority, and refined with more data in any successive calibration.

[0107] For example, measuring the centerline height estimate can provide a first estimate of the height of the user's ear notch (or more precisely, a pattern characterizing the positional estimation error of that notch). Similarly, measuring the horizontal position of the centerline can provide a first estimate of the user's ITD and / or ILD (or more precisely, a pattern characterizing the estimation error of the ITD and / or ILD).

[0108] These test locations can be randomized again, either only within a vertical or horizontal range, or in between, or within a test set that includes a similar number of other predetermined locations that deviate from these lines.

[0109] The user results from this initial calibration test can be compared only with the corresponding initial results of the reference individual's metrics to find the initial closest match. The corresponding HRTF may still provide the user with a better experience than the default.

[0110] Users can then revisit the calibration test at different times to continue testing, thus populating their metric results set. Test positioning can be prioritized again for certain positioning that may provide specific distinctions for a given spectrum of notch, or for certain positioning that provides ITD and / or ILD measurements across subsequent heights.

[0111] Users can have their calibration tests re-performed as they wish; for example, growing children may want to do so annually as their head shape changes with age. Similarly, if older adults suspect some hearing loss in either ear, they can have their calibration tests re-performed.

[0112] Still referencing Figure 7 and Figure 8 In the summary embodiments of this specification, the audio personalization method for reference individuals therefore includes the following steps.

[0113] In the first step s810, the head-related transfer functions "HRTF" of the corpus for reference individuals are obtained, as described elsewhere in this paper.

[0114] In the second step s820, each reference individual is tested in a calibration test. As described elsewhere herein, the calibration test typically includes: requiring the corresponding reference individual under test to match test sounds to test locations by controlling the position of the presented sound or by controlling the position of the presented location, to perform a series of test matches (e.g., by presenting a series of test sounds that may be of the same type or different according to a predetermined scheme); each test sound is presented at the location using a default head-related transfer function “HRTF”; as described elsewhere herein, receiving an estimate of each matched location from the corresponding reference individual under test (e.g., by receiving an estimate of the corresponding location of each test sound from the reference individual, or receiving the final selected location of the corresponding sound estimated to be consistent with each test location); and calculating the corresponding location error for each estimate (e.g., the difference between the estimated location and the sound location or between the location of the sound source and the location), to generate a series of location estimation errors for the corresponding reference individual under test, as described elsewhere herein.

[0115] Then, in the third step s830, a series of positioning estimation errors of the reference individuals are correlated with their respective HRTFs, as described elsewhere in this paper.

[0116] Meanwhile, in the overview embodiments of this specification, the audio personalization method for a first user includes the following steps:

[0117] The first step, s710, includes testing the first user in a calibration test, as described elsewhere in this document.

[0118] The calibration test further includes: a sub-step s712, as described elsewhere herein, requiring the user to match test sounds to test locations by controlling the position of the presented sound or the position of the presented location, to perform a series of test matches (e.g., by presenting a series of test sounds that may also be of the same type or different according to a predetermined scheme), each test sound being presented at the location using the default head-related transfer function "HRTF"; a sub-step s714, as described elsewhere herein, receiving an estimate of each matched location from the first user (e.g., by receiving an estimate of the corresponding location of each test sound from the first user, or estimating the final selected location of the corresponding sound consistent with each test location); and a sub-step s716, as described elsewhere herein, calculating the corresponding error of each estimate (e.g., the difference between the user-estimated location and the sound location or the difference between the sound source and the location located by the user), to generate a series of location estimation errors for the first user.

[0119] The second step s720 then includes comparing at least some of the localization estimation errors of the first user with the same localization estimation errors previously generated for at least a subset of the corpus of the reference individuals, as previously described herein.

[0120] The third step, s730, then includes identifying the reference individual whose location estimation error best matches the location estimation error of the first user, as previously described herein.

[0121] Then, the fourth step s740 includes using the HRTF previously obtained for the identified reference individual for the first user, as previously described herein.

[0122] It will be understood that methods relating to the reference individual are typically performed by the provider of the video game console or other content playback device, or the provider of the system software of such console or device, or the provider of the audio suite of the software developer of such console or device, while methods relating to the first user are performed for the first user using the first user's own console or other content playback device.

[0123] Therefore, although the methods associated with the first user assume that the methods associated with the reference individuals have been implemented to the extent that some HRTF and positioning estimation error sets exist for some reference individuals, these methods can be used independently.

[0124] However, it will also be understood that these two approaches can also be considered as part of a single, broader approach, such as audio configuration for general users.

[0125] It will be apparent to those skilled in the art that variations of the above-described methods, corresponding to the operation of various embodiments of the methods and / or apparatus as described and claimed herein, are considered within the scope of this disclosure, including but not limited to:

[0126] -As described elsewhere in this document, as the corpus grows, users are occasionally re-compared to the corpus; therefore, if the HRTF and a associated set of localization estimation errors for a predetermined number of reference individuals are added to the corpus, at least some of the localization estimation errors of the first user are compared with the localization estimation errors of the same localization of at least a subset of the corpus of other reference individuals; and if the localization estimation errors of the other reference individuals match the localization estimation errors of the first user better than those of the currently identified reference individuals, the HRTF obtained for the other reference user is used for the first user, as described elsewhere in this document;

[0127] -As described elsewhere in this article, a subset of the corpus is selected in response to demographic details of the first user and the reference individual;

[0128] - The corresponding location includes at least a subset of locations, which is selected because it has at least a threshold variance in the location estimation error of a subset of reference individuals, as described elsewhere in this document;

[0129] - The corresponding sounds used in the calibration tests include one or more selected from a list consisting of narrowband sounds, wideband sounds, impulse sounds, monotones, chirps, and speech, as described elsewhere in this document;

[0130] - For calibration tests, the appropriate location is selected from a predetermined set of locations within a predetermined subset, as described elsewhere in this document;

[0131] In this case, optionally, the subsets including the locations on the horizontal centerline and the subsets including the locations on the vertical centerline are included in the first N subsets of a predetermined series of subsets, where N is between 2 and 5, as described elsewhere in this document;

[0132] Similarly, in this case, optionally, after a predetermined number of subsets have been completed within a predetermined series of subsets, the comparison step S720, the identification step S730, and the use step S740 are performed, as described elsewhere herein;

[0133] In this case, optionally, if the first user subsequently performs calibration tests using a predetermined number of subsequent subsets from a predetermined set of subsets, the comparison, identification, and usage steps are performed again, as described elsewhere in this document;

[0134] - For calibration testing, the corresponding location is randomly selected from at least a predetermined subset of locations (which may include one or more subsets from a predetermined set of subsets), as described elsewhere in this document;

[0135] - If the first reference individual is identified as the best match for a user with a greater threshold amount than other reference individuals, then another reference individual with morphological similarity to the first reference individual within a predetermined tolerance is selected, as described elsewhere in this document; and

[0136] If the localization estimation error of any single reference individual's comparison does not match the localization estimation error of the first user's comparison at a predetermined matching threshold level, the method includes mixing the HRTFs of the M closest matching reference individuals, where M is a value of two or greater, and applying the mixed HRTFs to the first user, as described elsewhere in this document.

[0137] It should be understood that the above methods can be appropriately adapted to be executed via software instructions or via conventional hardware that includes or replaces dedicated hardware.

[0138] Therefore, the required adaptation to existing portions of a conventional equivalent device can be implemented in the form of a computer program product comprising processor-executable instructions stored on a non-transitory machine-readable medium (such as a floppy disk, optical disk, hard disk, solid-state drive, PROM, RAM, flash memory, or any combination of these or other storage media), or implemented in hardware as an ASIC (Application-Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array), or other configurable circuitry suitable for use in adapting to a conventional equivalent device. Separately, such a computer program can be transmitted via data signals over a network (such as Ethernet, wireless network, the Internet, or any combination of these or other networks).

[0139] Although the data required to calculate HRTF may be from specialized equipment (such as...) Figure 4A and Figure 4B The device shown is a dedicated device, but the device used to perform calibration tests and steps such as correlating positioning estimation errors with individual and / or HRTFs, comparing results, identifying best matches, and using the corresponding HRTFs can be a video game console (such as PS4). ® Or PS5 ® (or equivalent development kits, PCs, etc.)

[0140] Therefore, in the overview embodiment, the audio personalization system for the first user may be an entertainment device 10, including:

[0141] A test processor (e.g., CPU 20A) is configured (e.g., via appropriate software instructions) to test a first user in a calibration test comprising: requiring the user, as elsewhere herein, to match test sounds to test locations by controlling the position of presented sounds or controlling the position of presented locations, to perform a series of test matches (e.g., by presenting a series of test sounds that may also be of the same type or different according to a predetermined scheme), each test sound being presented at a location using a default head-related transfer function “HRTF”, receiving an estimate of each matched location from the first user as elsewhere herein (e.g., by receiving an estimate of the corresponding location of each test sound from the first user), and calculating a corresponding error for each estimate as elsewhere herein to generate a series of location estimation errors for the first user;

[0142] A comparison processor (e.g., CPU 20A) is configured (e.g., via suitable software instructions) to compare at least some of the localization estimation errors of the first user with estimation errors of the same localizations generated previously from a subset of a corpus of at least a reference individual; the comparison processor is also configured (e.g., via suitable software instructions) to identify the reference individual whose compared localization estimation errors best match the localization estimation errors of the first user, as described elsewhere herein; and

[0143] An HRTF processor (e.g., CPU 20A) is configured (e.g., via appropriate software instructions) to use the HRTF previously obtained for the identified reference individual for the first user, as described elsewhere herein.

[0144] For example, it will be understood that the role of the comparison processor can be divided between an entertainment device and a remote server, which also stores the localization estimation errors of a corpus of reference individuals. Therefore, within the entertainment device, the comparison processor is configured to induce comparisons that can be performed locally (e.g., by performing a comparison) or remotely (e.g., by sending the localization estimation error of a first user to the server and requesting a comparison).

[0145] Similarly, it will be understood that the HRTF processor can receive appropriate HRTF data from such a remote server.

[0146] Similarly, in the overviewed embodiments, the audio personalization system for a reference individual may be an entertainment device 10, or equivalently a development kit or server, including:

[0147] The memory (such as HDD 37 combined with CPU 20A) is configured (e.g., by appropriate software instructions) to store the head-related transfer functions "HRTF" of the corpus of the reference individuals;

[0148] A test processor (e.g., CPU 20A) is configured (e.g., via suitable software instructions) to test individual references in a calibration test comprising: requiring the respective reference under test to match test sounds to test locations by controlling the position of presented sounds or controlling the position of presented locations, as described elsewhere herein, to perform a series of test matches (e.g., by presenting a series of test sounds that may be of the same type or different according to a predetermined scheme), each test sound being presented at a location using a default head-related transfer function “HRTF”, receiving an estimate of each matched location from the respective reference under test (e.g., by receiving an estimate of the corresponding location of each test sound from the references, or the final selected location of the corresponding sound estimated to be consistent with each test location), and calculating the corresponding location error for each estimate (e.g., the difference between the estimated location and the sound location or between the sound source and the location of the location), to generate a series of location estimation errors for the respective reference under test, as described elsewhere herein; and

[0149] The associated processor (e.g., CPU 20A) is configured (e.g., via appropriate software instructions) to correlate a series of positioning estimation errors of the reference individuals with their respective obtained HRTFs.

[0150] It should be understood that the calibration tests for the first user and the reference individual are usually the same.

[0151] The foregoing discussion has only disclosed and described exemplary embodiments of the invention. As those skilled in the art will understand, the invention may be embodied in other specific forms without departing from its spirit or essential characteristics. Therefore, the disclosure of this invention is intended to be illustrative and not to limit the scope of the invention and the other claims. This disclosure, including any readily identifiable variations taught herein, partially defines the scope of the foregoing claims, such that no inventive subject matter is contributed to the public.

Claims

1. An audio personalization method for a first user, comprising the following steps: The calibration test is conducted on the first user and includes: Users are required to match test sounds to test locations by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matching. Each test sound is rendered at its location using the default Head-Related Transfer Function (HRTF). Receive estimates for each matched location from the first user, and Calculate the corresponding error for each estimate to generate a series of location estimation errors for the first user; and At least some of the location estimation errors of the first user are compared with the same location estimation errors previously generated for at least a subset of the corpus of the reference individuals; Identify the reference individual whose location estimation error best matches the location estimation error of the first user; and The HRTF previously obtained for the identified reference individual is used for the first user.

2. The audio personalization method according to claim 1, wherein... If the HRTF and a associated set of localization estimation errors are added to the corpus for a predetermined number of available reference individuals, then At least some of the location estimation errors of the first user are compared with the estimation errors of the same location in a subset of the corpus of at least another reference individual; and If, compared to the currently identified reference individual, the localization estimation error of another reference individual matches the localization estimation error of the first user's comparison better, then The HRTF obtained for the other reference user will be used for the first user.

3. The audio personalization method according to claim 1 or 2, wherein... The subset of the corpus is selected in response to demographic details of the first user and the reference individual.

4. The audio personalization method according to claim 1 or 2, wherein The corresponding location includes at least a subset of locations, which is selected because it has at least a threshold variance in the location estimation error of a subset of reference individuals.

5. The audio personalization method according to claim 1 or 2, wherein... The corresponding sounds used in the calibration test include one or more selected from a list consisting of: i. Narrowband sound; ii. Broadband audio; iii. Pulsating sound; iv. Monosyllabic; v. to chirp; and vi. pronunciation.

6. The audio personalization method according to claim 1 or 2, wherein for calibration testing: Select the appropriate location from a set of predetermined locations within a predetermined subset.

7. The audio personalization method according to claim 6, wherein The subsets including the locations on the horizontal centerline and the subsets including the locations on the vertical centerline are included in the first N subsets of the predetermined series of subsets, where N is between 2 and 5.

8. The audio personalization method according to claim 6, wherein After a predetermined number of subsets have been completed within the predetermined set of subsets, the following steps are performed: At least some of the location estimation errors of the first user are compared with the corresponding estimation errors of at least a subset of the corpus of the reference individuals. Identify the reference individual whose location estimation error best matches the location estimation error of the first user; and The HRTF obtained for the identified reference user will be used for the first user.

9. The audio personalization method according to claim 8, wherein If the first user subsequently performs the calibration test using a predetermined number of subsequent subsets from the predetermined set of subsets, the comparison step, identification step, and usage step are performed again.

10. The audio personalization method according to claim 1 or 2, wherein for calibration testing: The corresponding location is randomly selected from at least a subset of the predetermined locations.

11. The audio personalization method according to claim 1 or 2, wherein If the first reference individual is identified as the best match for a user with a greater threshold amount than other reference individuals, then Select another reference individual whose morphological similarity to the first reference individual is within a predetermined tolerance.

12. The audio personalization method according to claim 1 or 2, wherein If the location estimation error of any single reference individual's comparison does not match the location estimation error of the first user's comparison within a predetermined matching threshold level, then the method includes: The HRTFs of the M closest matching reference individuals are combined, where M is a value of two or greater; as well as The mixed HRTF is used for the first user.

13. An audio personalization method for a reference individual, comprising the following steps: Obtain the head-related transfer functions "HRTF" for each reference individual's corpus; The calibration test involves testing each reference individual, and the calibration test for each reference individual includes: The reference individual is required to match the test sound to the test location by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matching. Each test sound was presented at a series of different locations using the same default head-related transfer function "HRTF". Receive an estimate of each matched location from the reference individual, and Calculate the corresponding error for each estimate to generate a series of localization estimation errors for the corresponding measured reference individual; and The series of positioning estimation errors of the reference individuals are correlated with their respective HRTFs.

14. A computer program product comprising computer-executable instructions adapted to cause a computer system to perform the method of any one of claims 1 to 13.

15. An audio personalization system for a first user, comprising: The test processor is configured to test a first user in a calibration test, which includes: Users are required to match test sounds to test locations by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matches. Each test sound is presented at the location using the default Head Related Transfer Function "HRTF". Receive estimates for each matched location from the first user, and Calculate the corresponding error for each estimate to generate a series of location estimation errors for the first user; and The comparison processor is configured to compare at least some of the localization estimation errors of the first user with the same localization estimation errors previously generated for at least a subset of a corpus of a reference individual; The comparison processor is configured to identify a reference individual whose location estimation error best matches the location estimation error of the first user; and The HRTF processor is configured to use the HRTF previously obtained for the identified reference individual for the first user.

16. An audio personalization system for a reference individual, comprising: The memory is configured to store the head-related transfer functions "HRTF" of the corpus of the reference individuals; A test processor is configured to test individual references in a calibration test, the calibration test for each reference reference including: The reference individual is required to match a test sound to a test location by controlling the position of the presented sound or the position of the presented location, in order to perform a series of test matches. Each test sound is presented at a series of different locations using the same default Head Relevant Transfer Function (HRTF). Receive an estimate of each matched location from the reference individual, and Calculate the corresponding error for each estimate to generate a series of positioning estimation errors for the corresponding measured reference individual; as well as The association processor is configured to associate the series of positioning estimation errors of the reference individual with their respective obtained HRTFs.

Images