Sound direction discrimination ability training system and method

Abstract

This invention relates to a system and method for training sound directionality discrimination ability. The system may include: a speaker positioned within a predetermined left-right angular range in front of a mannequin, or positioned in front of a rotating subject, for generating multiple sound directionality training sounds; microphones worn on both ears of the mannequin or subject to acquire the multiple training sounds received from the speaker, and for acquiring the multiple training sounds at predetermined multiple set angles between the mannequin and the speaker, or between one ear of the rotating subject and the speaker; a training sound playback unit that plays at least a portion of the acquired training sounds to the subject at the set angles; a reaction information judgment unit that provides the subject with multiple visual targets at the set angles and determines whether the subject has selected a positive-response visual target from among the multiple visual targets that corresponds to the set angle of the played training sound; and a level adjustment unit that selects at least one angle from the set angles of the training sounds to be played to the subject.

Description

Technical Field

[0001] This invention relates to a sound direction discrimination training system and method for hearing rehabilitation, which utilizes virtual reality or augmented reality to train an individual's sound direction discrimination ability. Background Technology

[0002] The limitations of existing hearing rehabilitation can be categorized into a lack of training methods, spatial constraints, and the absence of training methods for asymmetric hearing loss.

[0003] Regarding the lack of training methods, training tools for sound directionality discrimination are almost non-existent in South Korea. Although studies comparing normal individuals and cochlear implant rehabilitation users have been published abroad to verify the effects of training using auditory stimulation (Firszt et al., 2015) and training using audiovisual integration function (Strelnikov et al., 2011; Isaiah et al., 2014), the number of such studies is negligible.

[0004] Regarding spatial limitations, hearing rehabilitation for hearing-impaired patients has so far mainly relied on in-home training, and portable training tools based on apps have not yet been widely used. In sound directionality training, directional information must be provided, thus requiring a spacious, soundproof space with sound sources from multiple directions. Due to this limitation, the development of examination and training methods has been difficult.

[0005] There are no training methods specifically for asymmetric hearing loss. For asymmetric hearing loss, where the ability to discern sound direction is particularly poor, different directional information is needed compared to that used when both ears are normal. Therefore, personalized stimuli are required. In cases of bilateral symmetrical hearing, such as normal hearing, the directional information is determined by the time difference between the arrival times of the sound source and the intensity difference of the arriving sound.

[0006] However, when hearing is asymmetrical, this interaural time / intensity difference information is distorted, making the spectral cue, as a third source of sound directionality information, a crucial factor. Spectral information is the characteristic of sound reflected from the auricle and external auditory canal, upper body, and head. Because each person's body is different, this spectral information must also vary from person to person. For asymmetrical hearing loss, sound directionality training can only be achieved by using sound sources that extract individualized spectral information.

[0007] There are three types of cues that humans use to determine the direction of sound: two cues generated by using both ears (binaural effect) and one cue obtained through one ear.

[0008] The cues obtained by using both ears consist of the interaural time difference (IDT) and the interaural level difference (ILD) between the two ears. People with normal hearing use these two types of information to determine the directionality of sound in the horizontal direction.

[0009] On the other hand, there are auditory cues that can be used to identify the directionality of sound for each person using only one ear, rather than comparing the sounds perceived by both ears. These are spectral cues that filter the sound source frequencies based on the different ear shapes and head and upper body shapes of individuals.

[0010] Unlike people with normal hearing, patients with asymmetrical hearing loss cannot use bilateral ear-based cues to distinguish sound direction, and therefore can only use spectral cues modulated on one side. Spectral cues are generated while filtering sound information according to the individual's anatomical features of the head, torso, and auricle, and each person's spectral cues are different.

[0011] Therefore, for patients with asymmetrical hearing loss, using the same standardized stimulus sound as normal people cannot reflect individual differences and may result in the inability to provide sound direction information. Summary of the Invention

[0012] Technical issues

[0013] The purpose of this invention is to provide a sound directionality discrimination training system and method, which can achieve personalized training by processing personally recorded sound sources, thereby improving the ability to discern the sound directionality of sound locations.

[0014] Technical solution

[0015] A first aspect of the present invention, aimed at addressing the aforementioned issues, relates to a sound directionality discrimination training system. The system may include: a loudspeaker disposed within a predetermined angular range to the left and right of a mannequin, or disposed in front of a rotating subject, generating multiple sound directionality training sounds; microphones worn on both ears of the mannequin or the subject to acquire the multiple training sounds received from the loudspeaker, wherein the microphones are positioned between the mannequin and the loudspeaker, or between one ear of the rotating subject and the loudspeaker, acquiring the multiple training sounds at predetermined multiple set angles; a training sound playback unit that plays at least a portion of the acquired training sounds to the subject at the set angles; a response information judgment unit that provides the subject with multiple visual targets at the set angles and determines whether the subject has selected a positive response visual target from among the multiple visual targets that corresponds to the set angle of the played training sound; and a level adjustment unit that selects at least one angle from the set angles of the training sounds to be played to the subject.

[0016] According to an embodiment of the present invention, the level adjustment unit can adjust the level according to the number of times the subject selects the positive reaction time target, such that the higher the level, the more set angles of the training sound played to the subject.

[0017] According to an embodiment of the present invention, the training sound playback unit can play the training sound corresponding to the angle set by the level adjustment unit multiple times to the subject, so that the subject can confirm the playback direction of the training sound in advance.

[0018] According to an embodiment of the present invention, it may further include a visual response providing unit, which provides visual stimulation to the subject when the subject selects a positive response visual target.

[0019] According to an embodiment of the present invention, the training sound playback unit can play multiple training sounds with different signal-to-noise ratios at each of the various levels, according to a set angle at which the training sounds are played to subjects exposed to the surrounding environment.

[0020] According to an embodiment of the present invention, when the frequency of the subject selecting the positive response visual target at level n is greater than or equal to a preset probability, the level adjustment unit can raise the level to level n+1.

[0021] The second aspect of this invention relates to a method for training sound directionality discrimination ability. The method, executed by a sound directionality discrimination ability training system including a loudspeaker, a microphone, a training sound playback unit, a response information judgment unit, and a level adjustment unit, may include: (a) the loudspeaker being positioned within a predetermined left-right angle range in front of a human body model, or positioned in front of a rotating subject, to generate multiple sound directionality training sounds; (b) the microphone being worn on both ears of the human body model or the subject to obtain the multiple training sounds received from the loudspeaker, wherein the microphone is positioned between the human body model and the loudspeaker, or between one ear of the rotating subject and the loudspeaker, at predetermined multiple set angles to obtain the multiple training sounds. (c) The step of playing at least a portion of the training sounds acquired by the training sound playback unit to the subject at the set angle; (d) The step of the response information determination unit providing the subject with multiple visual targets at the set angle and determining whether the subject has selected a positive response visual target from among the multiple visual targets that corresponds to the set angle of the played training sound; (e) The step of the level adjustment unit selecting at least one angle from the set angle of the training sound to be played to the subject; (f) The step of the training sound playback unit playing at least one training sound to the subject based on the angle set by the level adjustment unit; and (g) The step of repeating steps (d) to (f).

[0022] According to an embodiment of the present invention, in step (e), the level adjustment unit can adjust the level according to the number of times the subject selects the positive reaction time target, such that the higher the level, the more set angles of the training sound played to the subject.

[0023] According to an embodiment of the present invention, step (c) may include: the training sound playback unit plays the training sound corresponding to the angle set by the level adjustment unit to the subject multiple times, so that the subject can confirm the playback direction of the training sound in advance.

[0024] According to an embodiment of the present invention, the sound directionality discrimination ability training system may further include a visual response providing unit. Between step (d) and step (e), there may be a step in which the visual response providing unit provides visual stimulation to the subject when the subject selects a positive response visual target.

[0025] According to an embodiment of the present invention, in step (f), the training may be that the playback unit plays multiple training sounds with different signal-to-noise ratios at each level, according to a set angle at which the training sounds are played to the subjects exposed to the surrounding environment.

[0026] According to an embodiment of the present invention, in step (e), when the frequency at which the subject selects the positive response visual target is greater than or equal to a preset probability at level n, the level adjustment unit may raise the level to level n+1.

[0027] Invention Effects

[0028] The portable, personalized sound directionality discrimination training method based on virtual reality and augmented reality according to the present invention breaks away from the existing clinical methods, namely, the method of checking and training through multiple speakers in a sound field, and the method of recording sound through headphones. Therefore, it can overcome spatial limitations and easily evaluate and train sound directionality discrimination ability.

[0029] In addition, when needed, personalized sound directionality information obtained through each person's binaural auditory recordings can be used to perform training without spatial limitations.

[0030] In addition, it enables the subjects to perform sound discrimination training on their own in daily life. Attached Figure Description

[0031] Figure 1 This is a configuration diagram showing the various components of the sound directionality discrimination ability training system of the present invention.

[0032] Figure 2 This is a diagram illustrating an example of sound directionality training sound recording using a human body model according to the first embodiment of the present invention.

[0033] Figure 3 (a) to (c) are top views illustrating an example of personalized sound directionality training sound recording according to a second embodiment of the present invention.

[0034] *31 Figure 4 (a) and (b) are diagrams showing a stereo microphone used for binaural hearing recording and its state when worn on the outer ear.

[0035] Figure 5 Figures (a) and (b) are diagrams showing the results of experimental observation of whether binaural auditory recordings of an embodiment of the present invention can accurately reproduce directional sound sources.

[0036] Figure 6 This is a diagram illustrating the wearing configuration of the VR device and the VR training program according to an embodiment of the present invention.

[0037] Figure 7 This is a sequence diagram illustrating the steps of the sound directionality discrimination ability training method of the present invention.

[0038] Figure 8 It is shown Figure 7A sequence diagram of the training sound generation and acquisition steps in each step.

[0039] Figure 9 It is shown Figure 7 The sequence diagram of the training sound playback steps, judgment steps, level adjustment steps, and training sound playback steps for each level.

[0040] Figure 10 This is a diagram illustrating examples of sound source direction cues at various levels trained using the sound discrimination capabilities of VR / AR devices according to the present invention.

[0041] Figure 11 This is a sequence diagram illustrating the steps of training using AR in an embodiment of the present invention while exposing the subject to everyday environmental noise.

[0042] Figure 12 This is an illustrative diagram showing the sound source direction angle and volume adjustment at various levels of sound discrimination capability training using AR in an embodiment of the present invention.

[0043] Best practice

[0044] The specific details required for implementing this invention will now be described in detail with reference to the accompanying drawings. Furthermore, in describing this invention, if it is determined that a known function involved is familiar to those skilled in the art and might obscure the essence of the invention, its detailed description will be omitted.

[0045] Figure 1 This is a configuration diagram showing the various components of the sound directionality discrimination ability training system of the present invention.

[0046] Reference Figure 1 The sound directionality discrimination ability training system of the present invention includes a loudspeaker 10, a microphone 20, a training sound playback unit 30, a reaction information judgment unit 40, a level adjustment unit 50, and a visual reaction providing unit 60.

[0047] The speaker 10 is positioned within a predetermined left-right angle range in front of the human body model 1, or in front of the rotating subject 2, to generate multiple directional training sounds.

[0048] For example, the design can be configured with multiple speakers 10 within a predetermined angle range to the left and right in front of the human body model 1, or the human body model 1 can be rotated to the left and right within a predetermined angle range relative to a speaker 10. Alternatively, the design can be configured with the subject 2 rotating around a speaker 10 positioned in front of him / her.

[0049] Examples of human body models 1 include GRAS's KEMAR (Knowles Electronics Manikin for Acoustic Research) or B&K's HATS (Head and Torso Simulator). For instance, the Kemar-45BA (GRAS Sound and Vibration) can also be used.

[0050] Microphones 20 are worn on both ears of the mannequin 1 or the subject 2 to acquire multiple training sounds received from the speaker 10. The microphones 20 are positioned between the mannequin 1 and the speaker 10, or between one ear of the subject 2 and the speaker 10, at predetermined angles to acquire the multiple training sounds. As an example of the microphone 20, a binaural recording microphone (Sound Professionals SP-TFB-2) can be used. After the microphones 20 are connected to a device such as a regular PC and store the recorded training sound data, they are sent to the training sound playback unit 30.

[0051] The training sound playback unit 30 can play at least a portion of the acquired training sounds to the subject 2 at the set angle. The training sound playback unit 30 can receive training sound data from the microphone 20 and, using the received training sound data, play at least one training sound to the subject 2 through, for example, headphones connected to a PC.

[0052] The reaction information judgment unit 40 can provide the subject 2 with multiple visual targets 3 (refer to) at the set angle. Figure 6 The reaction information determination unit 40 determines whether the subject 2 has selected a positive-response visual target from among the multiple visual targets that corresponds to a set angle of the played training sound. Specifically, multiple visual targets are displayed to the subject 2 through, for example, a VR (or AR) device connected to a PC. The subject 2 listens to the training sound played by the training sound playback unit 30, determines the direction of the training sound, and selects the corresponding visual target 3. The reaction information determination unit 40 can determine whether the visual target 3 selected by the subject 2 is a positive-response visual target.

[0053] The level adjustment unit 50 can select at least one of the set angles of the training sound to be played to the subject 2.

[0054] When subject 2 selects a positive response visual target, the visual response provision unit 60 can provide visual stimulation to subject 2.

[0055] On the other hand, the detailed configuration of the level adjustment unit 50 and the visual response providing unit 60 will be described later.

[0056] Figure 2 This is a diagram illustrating an example of sound directionality training sound recording using a human body model according to a first embodiment of the present invention. Figure 3 (a) to (c) are top views illustrating an example of personalized sound directionality training sound recording according to a second embodiment of the present invention. Figure 4 (a) and (b) are diagrams showing a stereo microphone used for binaural hearing recording and its state when worn on the outer ear.

[0057] This invention performs sound source creation through standardized equipment and sound source creation through binaural hearing recording of each individual. The binaural hearing recording method of this invention has the following two embodiments.

[0058] The following describes the configuration of the first embodiment of the present invention for recording standard sound directional stimulus sounds.

[0059] Reference Figure 2 Using human model 1 (e.g., Figure 1 The Kemar-45BA (GRAS Sound and Vibration) shown uses microphones 20 attached to both ears of a mannequin 1 with an average human ear shape to record training sounds.

[0060] The speaker 10 can be configured relative to the mannequin 1 at angles of -90° (left), -45°, -30°, -15°, 0° (front), 15°, 30°, 45°, 60°, 90°, or combinations thereof. Two microphones 20 (e.g., Figure 3 The Soundprofessionals SP-TFB-2 shown is fixed to the external auditory canal entrances on both sides of the human body model 1 to record sound sources from multiple directions configured by the speaker 10.

[0061] Alternatively, to control the variables between speakers, the direction of the mannequin 1 can be changed sequentially and recording can be performed while moving it to align with the speaker angle. Therefore, a single speaker can be used to obtain stimuli from multiple directions.

[0062] The following describes the configuration of a second embodiment of the present invention for recording personalized directional sound stimuli.

[0063] The following reference Figure 3 This describes an embodiment for recording and acquiring personalized sound directionality training sounds optimized for individual training subjects.

[0064] Connect two microphones 20 (for example, Figure 4The Sound Professionals SP-TFB-2 (shown) speakers were fixed to the two external auditory canal entrances of the subject, respectively, to record training sounds in, for example, directions 11. To control for speaker variations, only one speaker 10 was used, and the subject 2 rotated to record while changing direction.

[0065] Figure 4 (a) is an example of the microphone 20 used in this invention, showing a binaural microphone (Sound Professionals SP-TFB-2) used for binaural hearing recording. Figure 4 (b) is a diagram showing the state in which the binaural microphones are fixed to match the shape of the subject's ear when worn on the outer ear. This allows for more accurate recording of training sounds from multiple sound sources.

[0066] Figure 5 Figures (a) and (b) are diagrams showing the results of experimental observation of whether binaural auditory recordings of embodiments of the present invention can accurately reproduce directional sound sources. Figure 5 (a) shows the original sound wave file. Figure 5 (b) shows the sound wave file of the auditory stimulus recorded in both ears according to the angle.

[0067] like Figure 5 As shown in (b), examining the waveforms of the actual recordings in the microphone 20 based on the angle of the speaker 10, it can be confirmed that the speaker 10, located at 60° relative to the mannequin 1 or the subject 2, reaches the right ear first, resulting in a higher recording intensity in the right channel. Conversely, the speaker 10, located at -30°, reaches the left ear first, resulting in a higher recording intensity in the left channel. Thus, a visually perceptible intensity difference and time difference can be confirmed between the left and right channels, demonstrating that the configuration of this invention conforms to the design of a directional sound source for hearing rehabilitation training.

[0068] Therefore, in this hearing training, training sounds that are directly recorded from the sound source reaching each person's ear can be used for VR / AR training. This method is particularly effective for patients with asymmetrical hearing loss.

[0069] Figure 6 This is a diagram illustrating the wearing configuration of the VR device and the VR training program according to an embodiment of the present invention.

[0070] The following reference Figure 6 The design of a portable training tool utilizing VR (or AR) devices for applying this invention was examined in detail.

[0071] *67 The following describes the design and configuration of a portable training tool utilizing a VR (or AR) device according to an embodiment of the present invention.

[0072] As a device for designing portable training tools, it can be a PC, PC monitor, VR device (e.g., VIVE pro (HTC)) or AR device, or headphones.

[0073] This study designs a sound direction discrimination rehabilitation training tool using VR (AR) devices, targeting individuals with normal hearing and all hearing-impaired patients, including those with unilateral hearing loss. As a portable training tool utilizing combined visual and auditory stimulation, it can overcome the spatial constraints of being confined outside the booth.

[0074] The listening training program used in this training can be Unity (Unity Tech, 2019) and can be driven by Steam (Valve Cooperation, 2013). The virtual reality device used in the training can be VIVE Pro Eye Tracking (HTC). However, VR and AR devices can be used without model restrictions.

[0075] In addition, as the stimulus sound source used in this training, for hearing-healthy individuals (normal hearing individuals), a binaural sound source such as a human model 1 recorded in a standardized adult ear can be used, and for patients with asymmetrical hearing impairment, training sounds made through personalized binaural auditory recordings of subject 2 can be used.

[0076] Figure 7 This is a sequence diagram illustrating the steps of the sound directionality discrimination ability training method of the present invention. Figure 8 It is shown Figure 7 A sequence diagram of the training tone generation and acquisition steps in each step. Figure 9 It is shown Figure 7 The sequence diagram of the training sound playback steps, judgment steps, level adjustment steps, and training sound playback steps for each level.

[0077] Reference Figure 7 The sound directionality discrimination training method of the present invention includes a preparation step s1, a training sound generation step s10, a training sound acquisition step s20, a training sound playback step s30, a judgment step s40, a level adjustment step s50, and a training sound playback step s60 for each level.

[0078] Reference Figure 8 Preparation step s1 includes step s1-1 of attaching microphone 20 to the external auditory canal entrance of mannequin 1 or subject 2. To obtain standard sound directionality training sounds, Kemar mannequin 1 can be used; to obtain personalized sound directionality training sounds, recording of subject 2 can be performed separately.

[0079] The training sound generation step s10 involves positioning the speaker 10 within a predetermined left-right angle range in front of the human model 1, or in front of the rotating subject 2, to generate multiple directional training sounds. The speaker 10 can generate multiple training sounds relative to the human model 1 or the subject 2 at angles of (left) -90°, -45°, -30°, -15°, (front) 0°, (right) 15°, 30°, 45°, 60°, 90°, or in other directions.

[0080] The training sound acquisition step s20 involves the microphone 20 being worn on both ears of the human model 1 or the subject 2 to acquire multiple training sounds received from the speaker 10. The microphone 20 records the directional training sounds generated by the speaker 1 s21. After the recorded training sounds undergo a sound source processing s22 using known audio software, the final directional training sounds s23 are generated.

[0081] Reference Figure 9 The sound directionality discrimination training method of the present invention allows the subject 2 to be trained according to training level levels, starting with level 1. The angle at which training sounds are played and the number of training sounds played are predetermined among the training sounds acquired at each level. The level adjustment unit 50 can be configured to initially start training from level 1 and adjust the level applied to each training session according to the progress of the training.

[0082] The training sound playback step s30 is a step of playing at least a portion of the training sounds acquired by the training sound playback unit 30 to the subject at the set angle. The training sound playback step s30 includes a training sound pre-play step s31 and a training sound main playback step s32.

[0083] In the training sound pre-play step s31, the training sound playback unit 30 plays multiple training sounds corresponding to at least one angle set by the level adjustment unit 50 to the subject 2, so that the subject 2 can confirm the playback direction of the training sound in advance.

[0084] Specifically, the training sound playback unit 30 can provide sound source prompts from all training angles to familiarize the user with sound direction in a VR / AR environment. For example, before a sound discrimination training course begins, the subject 2 listens to sounds emitted from angles corresponding to each step 3 to 5 times and confirms the direction of the target sound. At the same time, the visual response providing unit 60 provides visual feedback.

[0085] In the main training sound playback step s32, if the main training (actual training) is entered through the VR / AR device, the training sound playback unit 30 randomly plays a predetermined number of training sounds to the subject 2 from various angles.

[0086] The judgment step s40 is the step in which the reaction information judgment unit 40 provides multiple visual targets to the subject at the set angle and judges whether the subject has selected a positive reaction visual target from among the multiple visual targets that corresponds to the set angle of the training sound being played.

[0087] In the judgment step s40, if the subject 2 responds to the visual target that is believed to emit the sound for each of the training sound pairs, the response information judgment unit 40 checks the subject 2's response via a VR controller (not shown).

[0088] Then, the visual stimulus provision step s45 is performed. Visual stimulus provision step s45 is the step whereby the visual response provision unit 60 provides visual stimulation to the subject 2 when the subject 2 selects a positive response visual target. The visual response provision unit 60 provides visual stimulation to the subject 2's own positive or negative response via the display screen of the VR / AR device; for example, the color of the visual target can be different when displaying a positive response and when displaying an incorrect response.

[0089] Figure 10 This is a diagram illustrating examples of sound source direction cues at various levels trained using the sound discrimination capabilities of VR / AR devices according to the present invention.

[0090] Reference Figure 10 The level adjustment step s50 is a step in which the level adjustment unit 50 selects at least one angle from the set angles of the training sound to be played to the subject. The level adjustment unit 50 can adjust the level according to the number of times the subject 2 selects the positive reaction time target, such that the higher the level, the more set angles of the training sound will be played to the subject 2.

[0091] When the frequency of the subject selecting the positive visual target at level n is greater than or equal to a preset probability, the level adjustment unit 50 raises the level to level n+1.

[0092] Specifically, training sound discrimination capabilities using VR / AR devices can be performed in multiple steps with varying levels of difficulty. The level adjustment step s50 can, for example, consist of four levels, starting with the easiest level 1 and proceeding to the next level. Proceeding to the next level means that after training at the previous level, performance evaluation is conducted. If the positive response reaches a predetermined target level (e.g., 70%), then training proceeds to the next level. Conversely, if the performance evaluation result does not reach the target level, training at that level is repeated.

[0093] The training sound playback step s60 for each level is a step in which the training sound playback unit 30 plays at least one training sound based on an angle set by the level adjustment unit 50 to the subject. In the training sound playback step s60 for each level, training sounds based on the level adjusted in the level adjustment step s50 are played. As described above, the angle of the training sound to be played from the training sounds acquired according to each level, as well as the number of training sounds to be played, are predetermined, and the training sounds determined according to the level are played to the subject 2.

[0094] Then, the judgment step s40, the visual stimulus provision step s45, the level adjustment step 50, and the training sound playback step s60 at each level are executed repeatedly for a predetermined number of times.

[0095] Sound discrimination training using VR / AR devices can be structured in multiple time periods, cycles, and sessions. For example, three times a week, for a total of nine sessions over three weeks, with each training session lasting approximately 30 minutes to one hour. The training time and frequency can be adjusted based on the participant's performance. The results of each listening training session are stored in a database as log files to monitor progress.

[0096] Figure 11 This is a sequence diagram illustrating the steps of training using AR in an embodiment of the present invention while exposing subjects to everyday environmental noise. Figure 12 This is an illustrative diagram showing the sound source direction angle and volume adjustment at various levels of sound discrimination capability training using AR in an embodiment of the present invention.

[0097] Reference Figure 11 and Figure 12 The training sound playback step s60 for each level can be performed by the training sound playback unit 30 at each level, playing multiple training sounds with different signal-to-noise ratios (SNR) according to a set angle at which the training sounds are played to the subject exposed to ambient noise.

[0098] Therefore, preparation step s1 may include microphone installation step s1-1, training environment selection step s1-2, and signal-to-noise ratio (SNR) selection step s1-3.

[0099] In the training environment selection steps s1-2, Subject 2 can choose a training environment, such as a specific surrounding environment from various environments like a school, coffee shop, subway, or library. The VR device plays typical noise from the surrounding environment selected by Subject 2. Training using an AR device (or mobile application) exposes Subject 2 to various everyday environments (e.g., buses, stations, subways, coffee shops, restaurants, schools, etc.) to train sound discrimination skills within real-world, given noise levels.

[0100] In the signal-to-noise ratio (SNR) selection steps s1-3, the training sound playback unit 30 can receive the SNR value selected by the subject 2. The subject 2 can, for example, select a specific SNR value from different SNR values ​​such as -5dB, 0dB, and 5dB.

[0101] Then, the following steps are executed: training sound generation step s10, training sound acquisition step s20, training sound playback step s30, judgment step s40, level adjustment step s50, and training sound playback step s60 for each level.

[0102] Specifically, the training process using AR devices in this embodiment consists of a process similar to the training process using VR devices in the aforementioned embodiments (see [reference]). Figure 11 When Subject 2 was in an everyday environment where they wanted to train their sound discrimination skills, they were randomly exposed to predetermined sound sources from various directions and responded to visual targets located in the perceived direction of the sound. The responses were then checked using an AR device (or mobile phone) to assess whether the subject's response was positive or negative, using visual stimuli.

[0103] When a positive response is displayed and an incorrect response is displayed, “O” and “X” are shown on the visual target, allowing subject 2 to self-check their performance through visual feedback.

[0104] This embodiment of sound discrimination training using AR devices can be performed in multiple steps under various environments. The difficulty can be adjusted by changing the volume of the given stimulus sound and the number of directions in which the sound is heard. For example, a training sound direction exercise consisting of 3 volume adjustments (using SNR, -5dB, 0dB, 5dB SNR) and 4 levels (the level is set by the number of angles of the given sound direction) can be used, forming a total of 12 levels (see [reference]). Figure 12 The so-called starting from the easiest level 1 and proceeding to the next level means that after the previous step of training, the performance is evaluated, and if the positive response reaches the predetermined target level (e.g., 70%), then the next step of training is initiated.

[0105] The training utilizes AR to improve sound discrimination skills, ultimately allowing Subject 2 to perform sound discrimination training independently in daily life. The primary goal is to generalize the training, enabling subjects to train at their preferred time and place without spatial or temporal limitations. Training time and frequency can be adjusted based on the participant's performance. The results of each training session can be stored as a log file, allowing subjects to monitor their own progress.

[0106] The training effect verification test to evaluate the training effect of the above embodiment can be performed as follows.

[0107] The responses of participants to the training sounds in each training session can be recorded in a database in log form to evaluate the training effect. Based on the responses, the training effect can be evaluated using indicators commonly used in the assessment of sound directionality discrimination ability, such as the correctness of responses in each training session, root mean square error (RMS error), mean absolute error (MAE), and error index. In addition, the subjective improvement can be assessed using questionnaires such as the Korean version of the Speech, Spatial and Qualities of Hearing Scale (K-SSQ).

[0108] The scope of protection in this field is not limited to the embodiments described and expressed above. Furthermore, it should be reiterated that the scope of protection of this invention should not be limited by the self-evident changes or substitutions within the technical field to which this invention pertains.

[0109] [Figure Labels]

[0110] 1: Human model

[0111] 2: Subjects

[0112] 3: Visual target

[0113] 10: Speaker

[0114] 20: Microphone

[0115] 30: Training Sound Playback Department

[0116] 40: Response Information Judgment Department

[0117] 50: Level Adjustment Department

[0118] 60: Visual Response Provision Department

Claims

1. A sound directionality discrimination ability training system, characterized in that, include: A loudspeaker is positioned within a predetermined angular range to the left and right of the front of the human body model, or positioned in front of a rotating subject, to generate multiple directional training sounds. A microphone, which is worn on both ears of a mannequin or a subject to receive multiple training sounds from the speaker, wherein the microphone is positioned between the mannequin and the speaker, or between one ear of a rotating subject and the speaker, and receives the multiple training sounds at a predetermined multiple set angles. The training sound playback unit plays at least a portion of the acquired training sounds to the subject at the set angle. A response information determination unit provides multiple visual targets to the subject at a set angle, and determines whether the subject has selected a positive response visual target from among the multiple visual targets that corresponds to the set angle of the played training sound; and A level adjustment unit, wherein the level adjustment unit selects at least one angle from a set angle of the training sound to be played to the subject; The level adjustment unit adjusts the level based on the number of times the subject selects the positive response visual target. The higher the level, the more set angles of the training sound are played to the subject. When the frequency of the subject selecting the positive response visual target at level n is higher than the preset probability, the level adjustment unit raises the level to level n+1.

2. The sound directionality discrimination ability training system according to claim 1, characterized in that, The training sound playback unit plays the training sound to the subject multiple times, corresponding to the angle set by the level adjustment unit, so that the subject can confirm the playback direction of the training sound in advance.

3. The sound directionality discrimination ability training system according to claim 1, characterized in that, The sound directionality discrimination ability training system also includes a visual response provision unit, which provides visual stimulation to the subject when the subject selects a positive response visual target.

4. The sound directionality discrimination ability training system according to claim 1, characterized in that, The training sound playback unit plays multiple training sounds with different signal-to-noise ratios at each level, according to a set angle at which the training sounds are played to subjects exposed to the surrounding environment.

Images