Auditory training method and device

Abstract

According to an embodiment, a method for auditory training may include an operation of providing a test sound having a first auditory pattern for at least one characteristic of the sound. The method for auditory training may also include providing an operation of a first visual object having a first visual pattern corresponding to a user input through a user interface for receiving the visual pattern based on the user input through the user interface. A first sound may be provided that is substantially synchronized with a detected point in time of the user input, the first sound having at least one feature identified based on a location of the user input defined in the user interface. The method for auditory training may further include, based on identifying an event for providing a second visual pattern corresponding to the test sound, providing an operation of a second visual object having the second visual pattern corresponding to at least one feature of the first auditory pattern of the test sound.

Description

Technical Field

[0001] This invention relates to a method and apparatus for auditory training. Background Technology

[0002] As the population ages, the number of people with hearing impairment due to age-related hearing loss is increasing. Audiologists predict that this number will further increase as average life expectancy rises. Furthermore, hearing impairment can occur at any age due to congenital or acquired causes.

[0003] Therefore, there is increasing attention being paid to assistive devices for people with hearing impairments, such as hearing aids and cochlear implants. In particular, there is growing interest in cochlear implants for those with severe hearing loss whose hearing cannot be significantly improved even with hearing aids. However, even after cochlear implantation surgery, rehabilitation training is necessary because the ability to hear does not immediately return.

[0004] Currently known rehabilitation training methods mainly involve repeatedly listening to recorded sounds (such as words or short phrases) and solving problems based on them. These methods are monotonous and inefficient because they rely primarily on repetitive and consistent practice, which can bore users. Meanwhile, even individuals who do not experience discomfort in daily life due to hearing problems may wish to improve their hearing function for various reasons (such as improving musical abilities or developing talents in infants and toddlers).

[0005] Therefore, there is a need to develop user-friendly and effective auditory training (or auditory rehabilitation training) methods that take into account the individual characteristics of users, including those with hearing loss and / or those who wish to enhance their auditory function.

[0006] Known rehabilitation training methods may include repeatedly providing sounds corresponding to words or phrases with semantic (syntactic) meaning, and assessing whether the user has correctly perceived these sounds based on user input corresponding to these sounds. Summary of the Invention

[0007] Technical problems to be solved

[0008] Traditional auditory training methods provide sounds corresponding to words or phrases, enabling hearing-impaired individuals to listen to these sounds and check if their auditory recognition matches the correct answer. However, in some cases, the hearing-impaired individual's auditory recognition differs from the actual sound. In such situations, although the hearing-impaired individual may be aware that their recognition differs from the actual sound, this awareness alone may prolong the rehabilitation process.

[0009] People with hearing loss require extensive training and repeated listening to sounds to reduce the discrepancy between their perceived sound and the actual sound. Therefore, the auditory rehabilitation process can be lengthy. Consequently, there is a need for a technology that can provide the actual sound itself while visually showcasing its characteristics, and / or receive input from the hearing-impaired individual regarding these characteristics.

[0010] Methods for providing sounds corresponding to words or phrases have limitations in providing sound coverage across a wide frequency range. Many people with hearing loss have difficulty perceiving sounds within specific frequency bands. However, when providing conventional sounds corresponding to meaningful words or phrases, it may be impossible to provide sound coverage across a wide frequency range, and / or to provide sound within frequency bands that are difficult for people with hearing loss. Therefore, there is a need to develop a technology that not only provides sounds corresponding to words or phrases, but also provides sounds corresponding to various frequencies and / or target frequency bands. The technical problem to be solved by this invention is not limited to the above-mentioned problems, and other technical problems not explicitly mentioned will be readily understood by those skilled in the art from the following description.

[0011] Technical solutions to the problem

[0012] According to one embodiment, the auditory training method may include: providing a test sound having a first auditory pattern having at least one characteristic for the sound; providing a first visual object having a first visual pattern corresponding to user input input through the user interface via a user interface for receiving visual patterns; and providing a second visual object having a second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound based on recognizing an event for providing a second visual pattern corresponding to the test sound. A first sound may be provided, the first sound being substantially synchronized with the detection time of the user input and having at least one characteristic identified based on the position of the user input defined in the user interface.

[0013] According to one embodiment, providing the first visual object through the user interface may include: providing at least a portion of the first visual object associated with the first position, the second position, and at least one intermediate position between the first position and the second position of the user interface based on detecting at least a portion of the user input associated with a first position and a second position of the user interface.

[0014] According to one embodiment, based on at least a portion of the user input being associated with the first position, a first portion of the first sound having at least one characteristic corresponding to the first position is provided; based on at least a portion of the user input being associated with each of the at least one intermediate positions, at least one intermediate portion of the first sound having at least one characteristic corresponding to each intermediate position is provided; and based on at least a portion of the user input being associated with the second position, a second portion of the first sound having at least one characteristic corresponding to the second position is provided.

[0015] According to one embodiment, at least a portion of the user input may include input for specifying the first position and the second position, and / or input for specifying the first position, the at least one intermediate position, and the second position.

[0016] According to one embodiment, the test sound may include multiple parts provided sequentially over time, and each of the multiple parts may have at least one characteristic that varies over time or remains constant over time according to the first auditory pattern.

[0017] According to one embodiment, the method may further include providing a result comparing the first visual object with the second visual object.

[0018] According to another embodiment, providing the comparison results may include providing information on user vulnerabilities identified based on the comparison results.

[0019] According to one embodiment, events for providing a second visual mode corresponding to the test sound may include providing an availability selection of the correct visual object, completing the provision of the first visual object, and / or elapsed a predetermined time.

[0020] According to another embodiment, the method may further include performing an operation to modify at least a portion of the first visual object based on another user input for modifying the first visual object.

[0021] According to one embodiment, modifying at least a portion of the first visual object may include identifying a deletion command for a first portion of the first visual object, and deleting the first portion associated with the deletion command while maintaining the display of the remaining portion of the first visual object excluding the first portion.

[0022] According to another embodiment, at a certain time, each of at least one characteristic of at least a portion of the test sound may have a single value.

[0023] In another embodiment, at a certain time, each of at least one characteristic of at least a portion of the test sound may have multiple values.

[0024] According to one embodiment, the user interface may include a plurality of reference objects for association with the user input.

[0025] In another embodiment, the plurality of reference objects may be arranged in a grid, and the user input may include an input that connects one of a plurality of first reference objects contained in a column to one of a plurality of second reference objects contained in an adjacent column.

[0026] According to one embodiment, a visual object associated with two or more reference objects is provided as the first visual object based on two or more reference objects being associated with a first temporary user input; and if two or more reference objects are not associated with a second temporary user input, the display of the visual object temporarily provided based on the trajectory of the second temporary user input is stopped.

[0027] According to another embodiment, the number, density, and / or arrangement of the plurality of reference objects may be set based on user selection and / or testing difficulty.

[0028] In one embodiment, the lower limit, upper limit, and / or the difference between the upper and lower limits of the test sound can be set based on user selection and / or test difficulty.

[0029] According to another embodiment, sound effects based on user selection and / or test difficulty identification can be applied to at least a portion of the test sound.

[0030] In one embodiment, background sounds based on user selection and / or test difficulty identification may be provided along with at least a portion of the test sounds.

[0031] According to another embodiment, the method may further include setting at least one different characteristic of the sound other than the at least one characteristic of the first auditory mode, based on user selection and / or test difficulty.

[0032] In one embodiment, the method may further include providing at least one indicator visually while providing the test sound, for notifying the playback point of the test sound over time.

[0033] According to one embodiment, an auditory training system may include a server and an electronic device including at least one processor. The server may be configured to provide instructions to the electronic device based on a connection between the electronic device and the server and / or a request to the server. When executed based on at least a portion of the at least one processor of the electronic device, the instructions may cause the electronic device to perform at least one operation, wherein the at least one operation may include providing a test sound having a first auditory pattern having at least one characteristic for sound; providing a first visual object having a first visual pattern corresponding to user input input through the user interface via a user interface for receiving visual patterns; and providing a second visual object having a second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound. A first sound may be provided, the first sound being substantially synchronized with the detection time of the user input and having at least one characteristic identified based on the position of the user input defined in the user interface.

[0034] According to one embodiment, an auditory training method performed by a system including a server and an electronic device may include: the server providing instructions to the electronic device based on a connection between the electronic device and the server and / or a request to the server. The method may include the electronic device executing the instructions, wherein the instructions, when executed based on at least a portion of at least one processor of the electronic device, cause the electronic device to perform at least one operation, and the at least one operation may include providing a test sound having a first auditory pattern having at least one characteristic for sound; providing a first visual object having a first visual pattern corresponding to user input input through the user interface via a user interface for receiving visual patterns; and providing a second visual object having a second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound based on an event recognizing an event for providing a second visual pattern corresponding to the test sound. A first sound may be provided, the first sound being substantially synchronized with the detection time of the user input and having at least one characteristic identified based on the position of the user input defined in the user interface.

[0035] According to one embodiment, an auditory training method performed by a system including a server and an electronic device may include providing a server having at least one first processor. The server may be configured to provide instructions to the electronic device based on a connection between the electronic device and the server and / or a request to the server. The method may include the electronic device executing the instructions, wherein the instructions, when executed based on at least a portion of the at least one processor of the electronic device, cause the electronic device to perform at least one operation. The at least one operation may include providing a test sound having a first auditory pattern having at least one characteristic for the sound; providing a first visual object having a first visual pattern corresponding to user input input through the user interface via a user interface for receiving visual patterns; and providing a second visual object having a second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound based on an event recognizing an event for providing a second visual pattern corresponding to the test sound. A first sound may be provided, the first sound being substantially synchronized with the detection time of the user input and having at least one characteristic identified based on the position of the user input defined in the user interface.

[0036] According to one embodiment, a storage medium storing computer-readable instructions may include instructions that, when executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation. The at least one operation may include providing a test sound having a first auditory pattern having at least one characteristic for sound; providing a first visual object having a first visual pattern corresponding to user input input through the user interface via a user interface for receiving visual patterns; and providing a second visual object having a second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound based on an event recognizing an event for providing a second visual pattern corresponding to the test sound. A first sound may be provided, the first sound being substantially synchronized with the detection time of the user input and having at least one characteristic identified based on the position of the user input defined in the user interface.

[0037] According to one embodiment, an electronic device may include at least one processor and a memory storing instructions. When executed based on at least a portion of the at least one processor, the instructions may cause the electronic device to perform at least one operation. The at least one operation may include providing a test sound having a first auditory pattern having at least one characteristic for sound; providing a first visual object having a first visual pattern corresponding to user input input through the user interface via a user interface for receiving visual patterns; and providing a second visual object having a second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound based on an event recognizing an event for providing a second visual pattern corresponding to the test sound. A first sound may be provided, the first sound being substantially synchronized with the detection time of the user input and having at least one characteristic identified based on the position of the user input defined in the user interface.

[0038] Beneficial effects of the invention

[0039] According to various embodiments, an implementation can be provided in which the characteristics of sound are presented visually along with auditory training sounds to a person with hearing loss, and / or the sound characteristics are received as input from the person with hearing loss.

[0040] According to various embodiments, sounds can be provided that correspond not only to words or phrases, but also to various frequencies and / or target frequencies.

[0041] According to various embodiments, it is possible to provide not only language-based sounds with semantic meaning, but also non-language sounds without semantic meaning.

[0042] According to various embodiments, it is possible to identify the sound characteristics that a person with hearing loss is susceptible to, thereby enabling the provision of sound with the identified characteristics.

[0043] The beneficial effects of the present invention are not limited to the above description, and other effects not explicitly mentioned will be readily understood by those skilled in the art from the claims and / or the specific description of the invention. Attached Figure Description

[0044] Figure 1a This is a schematic diagram illustrating an electronic device, a server, and an external electronic device according to an embodiment of the present invention.

[0045] Figure 1b This is a schematic diagram illustrating a system for providing services according to one embodiment.

[0046] Figure 1c This is a schematic diagram illustrating a method for providing a service according to an embodiment.

[0047] Figure 2This is a flowchart illustrating a method for providing auditory training content according to an embodiment of the present invention.

[0048] Figures 3a to 3c These are examples of screens provided by electronic devices according to various embodiments.

[0049] Figure 4 This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0050] Figures 5a to 5i This is a schematic diagram illustrating the content provided by an electronic device according to various embodiments.

[0051] Figure 6a This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0052] Figure 6b This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0053] Figure 7 This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0054] Figure 8 This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0055] Figure 9 This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0056] Figure 10a This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0057] Figure 10b This is a schematic diagram illustrating multiple sounds according to one embodiment.

[0058] Figure 11a This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0059] Figure 11b This is a schematic diagram illustrating a user interface according to one embodiment.

[0060] Figure 12a This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0061] Figure 12b This is a schematic diagram showing the user interface for different test difficulty levels.

[0062] Figure 12c This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0063] Figure 12d This is a schematic diagram showing the user interface for different test difficulty levels.

[0064] Figure 12e This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0065] Figure 12f This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0066] Figure 12g This is a schematic diagram illustrating the sound effect according to one embodiment.

[0067] Figure 12h This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0068] Figure 13a This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0069] Figure 13b This is a schematic diagram illustrating the location of a virtual sound source in a stereo environment according to an embodiment.

[0070] Figures 13c to 13f This is a schematic diagram illustrating a screen for providing auditory training content according to various embodiments.

[0071] Figure 14a This is a schematic diagram illustrating a method for providing auditory training content according to an embodiment.

[0072] Figure 14b This is a schematic diagram illustrating a resonance peak-based analysis according to one embodiment.

[0073] Figure 15 This is a schematic diagram illustrating auditory training content according to one embodiment.

[0074] Figures 16a to 16i This is a schematic diagram illustrating auditory training content according to various embodiments.

[0075] Figure 17 This is a schematic diagram illustrating auditory training content according to one embodiment.

[0076] Figure 18a and Figure 18b This is a schematic diagram illustrating auditory training content according to various embodiments.

[0077] Figure 19 This is a schematic diagram illustrating the changes in auditory assessment resulting from performing auditory training content according to one embodiment. Detailed Implementation

[0078] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The advantages, features, and ways in which these are implemented will become apparent from the detailed description of the following embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various other forms. These embodiments are intended to fully disclose the invention and to enable those skilled in the art to understand its scope, which is defined only by the claims. The same reference numerals denote the same parts in the following description.

[0079] Although terms such as "first" and "second" are used to describe various elements, components, and / or parts, these terms are not intended to limit these elements, components, and / or parts. These terms are only used to distinguish one element, component, or part from another. Therefore, the "first" element, component, or part mentioned below may also be a "second" element, component, or part within the technical spirit of this invention.

[0080] The terminology used in this specification is for descriptive purposes only and is not intended to limit the invention. Unless otherwise expressly stated, the singular forms used herein also include the plural forms. The terms "comprising" and / or "consisting of" as used herein do not exclude the presence or addition of one or more other elements, steps, operations, and / or components.

[0081] In this specification, when one part is referred to as "connected" to another part, it can mean not only physical contact (or fastening) but also connection through an intermediate entity. Therefore, "connection" can refer to a state of direct physical contact or a state involving the intervention of another entity. Furthermore, "connection" can refer not only to physical connection but also to "logical connection," which can indicate a connection based on wireless communication.

[0082] The identification codes in the steps are for illustrative purposes only and do not indicate the order of the steps. Unless a specific order is explicitly specified in the context, the steps may be performed in a different order than described.

[0083] Unless otherwise defined, all terms used herein (including technical and scientific terms) are to be understood as having the meaning commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, terms as defined in general dictionaries should not be idealized or over-interpreted unless explicitly defined otherwise.

[0084] The various embodiments described in this specification can be implemented as software (e.g., a program) comprising one or more instructions stored in a machine-readable storage medium (e.g., internal or external memory). For example, a processor of a machine (e.g., an electronic device) can fetch at least one instruction from the storage medium and execute it, thereby causing the machine to perform at least one function according to the fetched instructions. The instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), but the term does not distinguish whether data is permanently or temporarily stored on the storage medium.

[0085] The methods according to various embodiments of this specification can be provided in the form of a computer program product. This computer program product can be traded as a commodity between a seller and a buyer. It can be distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or distributed online (e.g., downloaded or uploaded) directly between two user devices (e.g., smartphones) through an app store (e.g., the Play Store™). In the case of online distribution, at least a portion of the computer program product can be temporarily stored or generated in a machine-readable storage medium, such as the memory of a manufacturer's server, app store server, or intermediary server.

[0086] Figure 1a This is a schematic diagram illustrating an electronic device, a server, and an external electronic device according to an embodiment of the present invention.

[0087] According to one embodiment, electronic device 100 may be implemented in the form of a portable electronic device (e.g., a smartphone or tablet, but not limited thereto), a wearable electronic device (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted display device (HMD), but not limited thereto), or a computer. However, those skilled in the art will understand that electronic device 100 is not limited to the above forms as long as it has the ability to provide visual, auditory, and / or tactile content. In one example, electronic device 100 may perform at least some of the operations according to various embodiments of the present disclosure without integration with other external electronic devices (this may be referred to as standalone). For example, electronic device 100 may store applications for independent operation and perform at least some of the operations according to various embodiments of the present disclosure by executing the stored applications. In another example, electronic device 100 may perform at least some of the operations according to various embodiments of the present disclosure in conjunction with other external electronic devices. For example, electronic device 100 may provide data to or receive data from server 190 (e.g., a web server, application server, computing server, database server, file server, game server, mail server, proxy server, and / or cloud server, but not limited thereto). Based on this integration, electronic device 100 can perform at least some of the operations according to various embodiments of this disclosure. For example, electronic device 100 can temporarily or permanently store a program (or alternatively, at least one instruction or algorithm, but not limited thereto) provided by server 190. When the program is executed by electronic device 100, at least some of the operations described in this disclosure can be performed. For example, a web server can provide data representing content according to various embodiments of this disclosure based on a request from electronic device 100. Electronic device 100 can issue requests based on a web browsing application or a PWA (Progressive Web Application), but the request method is not limited. The web server can independently obtain data representing content and / or generate data based on information provided by an additional database server. However, those skilled in the art will understand that the methods for generating such data are not limited. Alternatively, electronic device 100 can download and install applications capable of providing the operations described in this disclosure from an application market. By executing the installed applications, electronic device 100 can provide the operations described in this disclosure. Furthermore, those skilled in the art will understand that applications for providing the operations described in this disclosure may also be pre-installed on electronic device 100.

[0088] Reference Figure 1aAccording to an embodiment of the present invention, the electronic device 100 can provide auditory training content to users who wish to enhance their hearing function. For example, the electronic device 100 of the present invention can provide auditory training (or rehabilitation training) content to users with hearing impairments who wear cochlear implants and users who wish to improve their hearing function for various reasons (such as the elderly, infants, or musicians).

[0089] According to one embodiment, the electronic device 100 may include a memory 110, a processor 120, a camera 130, a touch screen 140, an audio output device 150, a sensor 160, an input device 170, and / or a communication device 180.

[0090] Memory 110 may include at least one type of storage medium, such as flash memory, hard disk, SSD (Solid State Drive), SDD (Silicon Disk Drive), micro multimedia card, card-type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic storage, magnetic disk, or optical disk. Those skilled in the art will understand that memory 110 may also refer to, for example, cache memory for interfacing with processor 120 and / or cache memory and / or registers included within processor 120.

[0091] Memory 110 may permanently or temporarily store programs (or applications) that cause electronic device 100 to perform at least some of the operations disclosed herein. Those skilled in the art will understand that the program (or the instructions constituting the program) may be stored in a single entity or distributed across multiple entities. Electronic device 100 may download applications (e.g., from an app store or app source) and store them in memory 110. By executing the application, electronic device 100 may perform operations according to various embodiments of this disclosure. Alternatively, electronic device 100 may download data from server 190 that causes the electronic device to perform operations according to various embodiments of this disclosure and store that data in memory 110. For example, electronic device 100 may temporarily store data in memory 110 and utilize that data to provide content for auditory training. Subsequently, the device may delete or retain the data.

[0092] Processor 120 may include a CPU, GPU, NPU, DPU, FPGA, ASIC, and / or SoC, and its implementation is not limited to a specific form. For example, in some embodiments, operations performed by electronic device 100 and / or server may be performed by any one processor (e.g., CPU, GPU, NPU, FPGA, ASIC, and / or SoC), or by the cooperation of two or more processors. For example, multiple operations performed by electronic device 100 and / or server 190 may be performed by any one processor (e.g., CPU, GPU, NPU, FPGA, ASIC, and / or SoC), or may be allocated such that some operations are performed by one processor while others are performed by another processor. Electronic device 100 and / or server may include at least one memory for storing at least one instruction. The memory may include volatile memory and / or non-volatile memory, and is not limited to a specific type. When at least one processor 120 executes the at least one instruction, the at least one instruction may cause electronic device 100 and / or server 190 to perform at least one operation (e.g., at least a portion of the operations described herein). Instructions for causing electronic device 100 and / or server 190 to perform one or more operations can be stored in a single, physically independent memory or distributed across multiple memories. Furthermore, at least one processor can operate based on inference from at least one artificial intelligence (AI) model. The AI ​​model can be trained using training data. Such training can be performed on a device implementing the AI ​​of this disclosure or via a separate server and / or system. Examples of training algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The processor can also operate based on predetermined rule-based logic.

[0093] Camera 130 can capture an external scene and provide an image corresponding to the external scene. For example, during the execution of an application for auditory training (e.g., when displaying an interface screen), camera 130 can be activated to capture an external scene, although the activation conditions are not limited to this. Those skilled in the art will understand that camera 130 can be implemented as one or more units. Processor 120 can analyze at least one image provided by camera 130 to identify motion input based on user body parts contained in the image. For example, at least a portion of one or more sub-user inputs detected by electronic device 100 as described below can be identified based on images captured by camera 130. For example, when electronic device 100 is implemented as a wearable electronic device (e.g., HMD or glasses), user input can be identified (or recognized) by analyzing images provided by camera 130 for hand tracking.

[0094] Touchscreen 140 may include a display for performing display functions and a touch panel for performing input functions. For example, touchscreen 140 may output various screens (e.g., interface screens, visual data, etc.) for auditory training. Furthermore, touchscreen 140 may detect user touch input based on the interface. On the other hand, those skilled in the art will understand that when electronic device 100 is implemented in a form other than a smartphone, touchscreen 140 may be replaced by another entity. For example, when electronic device 100 is implemented as an HMD, it may include a display device for displaying the screen. As another example, when electronic device 100 is implemented as an eyeglass-type wearable electronic device, it may include a projector, optical waveguide, and / or lens for displaying the screen.

[0095] Audio output device 150 can output sound. According to one embodiment, processor 120 can control audio output device 150 to output sound in real time based on user input detected through an interface screen. For example, processor 120 can provide sound composed of segments corresponding to multiple sub-user inputs. The attributes of each sound segment corresponding to a sub-user input (e.g., frequency, overtone density, timbre, and / or volume, but not limited thereto) can be determined based on the input characteristics (e.g., input type or attributes, but not limited thereto) and / or input position of each sub-user input. Adjacent sound segments can have different or the same attributes. Providing sound segments corresponding to the input (or detection, recognition, or holding) of a sub-user input can overlap in time, but implementation is not limited thereto. In addition, processor 120 can also control audio output device 150 to provide test sounds for auditory training before receiving user input. At the same time, electronic device 100 can also provide data for sound output to an external audio output device (not shown) connected to electronic device 100 via a wired or wireless connection.

[0096] Sensor 160 may include at least one of the following: a proximity sensor, an illuminance sensor, a touch sensor (which may be implemented as part of touchscreen 140 or as standalone hardware, and its implementation is not limited), an accelerometer, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor), a fingerprint sensor, an ultrasonic sensor, an optical sensor (e.g., a camera), a microphone, an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a thermal detection sensor, a gas detection sensor), or a biometric sensor (e.g., a healthcare sensor, a biometric sensor). Furthermore, the device may combine and utilize information sensed by at least two or more of the aforementioned sensors.

[0097] Input device 170 can be configured to receive user input. Input device 170 can be implemented, for example, as a button, touchpad, or touch sensor as part of touchscreen 140, and its implementation is not limited. Electronic device 100 can recognize user input through input device 170. Alternatively, electronic device 100 can also recognize user input based on data received from external electronic device 195 via a wired or wireless communication connection, rather than through input device 170 included in electronic device 100. For example, external electronic device 195 may include sensors and / or input devices for sensing user input and provide electronic device 100 with the data required to recognize user input. Electronic device 100 can recognize user input based on the data provided by external electronic device 195. External electronic device 195 can be implemented, for example, as a user controller of HMD; however, this is merely an example, and implementation is not limited thereto.

[0098] The communication device 180 may include one or more components for communicating with various devices capable of communication. For example, it may include at least one of the following: a wired communication device, a cellular-based wireless communication device, an IEEE 802.11-based wireless communication device (e.g., commonly referred to as Wi-Fi), a short-range communication device (e.g., Bluetooth, Bluetooth Low Energy, UWB, Zigbee, etc., but not limited thereto), or a location information module. For example, electronic device 100 may send data to and / or receive data from server 190 via the communication device 180. Server 190 may provide electronic device 100 with data that enables the electronic device to perform at least some of the operations performed according to various embodiments of this disclosure. For example, those skilled in the art will understand that when electronic device 100 is implemented as a standalone device, data transmission and / or reception between electronic device 100 and server 190 may not be necessary. The communication device 180 may include components such as transceivers, communicators, etc.

[0099] Figure 1b This is a schematic diagram illustrating a system that provides services according to one embodiment.

[0100] According to one embodiment, service-providing terminal 102 can provide data to IaaS (Infrastructure as a Service) system 101a for creating a Virtual Private Cloud (VPC) 101b. IaaS system 101a can provide hardware resources for executing an operating system and / or programs based on client requests. For example, based on a request from service-providing terminal 102, IaaS system 101a can execute and / or create a Virtual Private Cloud 101b for providing services (e.g., content for auditory training, but not limited to this). Virtual Private Cloud 101b can provide data to the user terminal—electronic device 100—disclosed herein to enable the electronic device 100 to perform operations. Electronic device 100 can perform at least one operation to access Virtual Private Cloud 101b. Based on this access process, Virtual Private Cloud 101b can provide data to electronic device 100 to enable the user terminal to perform the operations described herein.

[0101] Figure 1c This is a schematic diagram illustrating a method of providing services according to one embodiment.

[0102] According to one embodiment, the method may include providing server operations 181. Providing server operations 181 may include, for example, creating, constructing, and / or activating a physical device such as server 101, but is not limited thereto. Alternatively, providing server operations 181 may also include, as Figure 1b The sending of a request to create a Virtual Private Cloud (VPC) 101b, and the creation, execution, and / or activation of VPC 101b within the IaaS system 101a, are not limited to this.

[0103] According to one embodiment, the method may include an operation 182 of obtaining a service provision request. The method may also include an operation 183 of providing data for service provision corresponding to the service provision request. For example, electronic device 100 may perform at least one operation to access server 101 (or virtual private cloud 101b). Server 101 (or virtual private cloud 101b) may interpret the at least one access operation as obtaining a service provision request and provide data for service provision to electronic device 100. For example, after performing at least one access operation, server 101 (or virtual private cloud 101b) may also be configured to receive additional service provision requests from electronic device 100. Based on the received service provision request, server 101 (or virtual private cloud 101b) may provide data for service provision to electronic device 100.

[0104] According to one embodiment, the method may include operation 184 of providing a service based on the data. For example, electronic device 100 may provide the services described herein based on data received from server 101 (or virtual private cloud 101b). Details regarding the services provided will be described below. Electronic device 100 may temporarily store the received data for service provision and subsequently delete it. Alternatively, electronic device 100 may also store the received data and delete it upon recognition of an additional deletion instruction. For example, electronic device 100 may provide the service via the cloud or by downloading and executing an application package. The service provision method is not limited. If an application package is downloaded, server 101 may be implemented as the source of the application package. However, this is merely an example; an application marketplace (not shown) may also be the source of the application package. Service providing terminal 102 may generate an application package associated with instructions that cause the operations described herein to be performed and / or provide it to a source (e.g., server and / or application marketplace).

[0105] Figure 2 This is a flowchart illustrating a method for providing auditory training content according to an embodiment of the present invention.

[0106] Reference Figure 2 The method according to one embodiment may include operation 201 of performing an access process based on user account information. For example, electronic device 100 may perform an access process to server 190 (e.g., accessing a webpage or executing a pre-installed application via an internet browser, and is not limited thereto). This access process may include, for example, logging into an auditory training service by entering pre-registered account information (e.g., ID and / or password, or biometric information such as fingerprints, iris scans, facial images, etc.). However, this is merely an example, and the login process may also be performed using an automatic login function. For example, the method may also include operations of verifying user information, although this is not required. User information may include, for example, auditory training details (e.g., auditory training services completed by the user, daily training progress, weekly training progress, monthly training progress, etc.), but is not limited thereto. Auditory training content may be configured based on user information. However, those skilled in the art will understand that content may also be configured independently of user information.

[0107] This method may include operation 203 of providing a user interface (UI). The user interface may, for example, be configured to receive user input for providing auditory training content and / or deliver content based on that user input. Details regarding various examples will be described later. This method may identify and provide auditory training content based on user input entered through the user interface. Auditory training content may, for example, include non-verbal training content and verbal training content, although the content classification criteria are not limited. For example, non-verbal training content may include, but is not limited to, sound-based imagery or sound-based drawing content; its detailed description will be given later. Similarly, language training services may include, but are not limited to, phoneme training content. Phoneme training content may, for example, include phoneme familiarization content, phoneme pair training content, or annotation training content, but is not limited to; its further details will be described later.

[0108] Figure 3a and Figure 3b Examples of screens provided by electronic devices according to various embodiments are shown.

[0109] According to one embodiment, the electronic device 100 can display as follows: Figure 3a The first screen 311 is shown. For example, the first screen 311 may be provided after accessing the auditory training service, although the order of provision is not limited. For example, the first screen 311 may include objects (or visual elements, availability, icons, buttons, pop-ups, etc.) 301a related to user profile information. Based on the designation of object 301a, the electronic device 100 may provide user profile information or enable the user profile information to be modified. User profile information may include, but is not limited to, personal details, training service start information, cochlear implant surgery related information, current level information, and instructions for use. The first screen 311 may also include an object 301b for providing chat services with the administrator. Based on the selection of object 301b, the electronic device 100 may provide chat services (e.g., display a chat window), and is not limited to this.

[0110] For example, the first screen 311 may include user level information 302. User level information 302 may indicate the user's current training level, which may be determined based on the frequency of repetition of the training service. Alternatively, the user level may also be determined based on an evaluation of training performance. Based on the user's training level, the difficulty of the content, the types of accessible content, and / or content configuration settings may be determined, but are not limited to these.

[0111] The first screen 311 may also include progress-related information 303. This progress-related information 303 may include a weekly calendar and a weekly progress rate (e.g., the ratio of training completed by the user to the total training required for that week). Furthermore, the first screen 311 may include training cycle information 304 representing training cycles. For example, the training cycle information 304 may be represented numerically as training cycles and / or as a bar chart showing progress relative to a set cycle, and is not limited thereto. The first screen 311 may also display phoneme status information 305 in a deactivated state. Figure 3a As shown, certain areas (305, 306, 307) on the first screen 311 can be displayed as disabled. The electronic device 100 can determine whether these areas (305, 306, 307) are activated based on the user's training level. For example, in the second screen 312, these areas (305, 306, 307) can become active depending on the training level. Based on the detected selection 305a of the phoneme map state information 305, a third screen 313 can be displayed. The third screen 313 can be configured, for example, based on the user's performance in auditory training content for at least one phoneme. Figure 3a As shown, consonants that form phonemes can be arranged vertically, and vowels can be arranged horizontally, but this arrangement is not limited. (See reference...) Figure 3a Syllables learned by the user (e.g., combinations of consonants and vowels) can be displayed at the corresponding consonant / vowel intersection. For a specific syllable, the training content performed by the user can cause that syllable to appear at the consonant / vowel intersection. Conversely, if no training content has been performed for a specific syllable, the intersection can remain blank. While displaying syllables at intersections is one example, it is also possible to display objects indicating content performed for a specific syllable, objects showing satisfactory performance, or objects showing unsatisfactory performance, etc. If a selection of a syllable displayed on the phoneme map is detected, the corresponding speech sound can be provided.

[0112] If the content presentation is satisfactory, the corresponding syllable can be displayed using the first attribute (here, attributes may include, but are not limited to, color, transparency, saturation, brightness, etc.). If the content presentation is unsatisfactory, the syllable can be displayed using an attribute different from the first attribute. For example, in Figure 3a In the third screen 313, if the user sets the syllable " " during phoneme training "Misheard as" "and select with " The corresponding answer is that electronic device 100 can pronounce the syllable. "Displayed on consonants" "with vowels" The intersection of the syllables is highlighted with a different color (e.g., red instead of black) than other syllables. In another example, electronic device 100 can display the consonant " "and vowels" "Incorrect syllables are displayed in intersecting cells" This allows users to identify how the syllable was misheard. In yet another example, it is only possible to hear the consonant " "and vowels" The intersection of “” shows the partial phonemes (such as the misheard consonant or vowel) that make up the incorrect syllable. For example, it can only be found at the intersection of “”. "and" The intersection of the symbols indicates the consonant "". In this case, the consonant " "It can be displayed with attributes that are the same as or different from the correct syllable."

[0113] For example, the third screen 313 may include a return navigation object 310a for returning to a previous state (e.g., the second screen 312). For example, the third screen 313 may include a consonant selection object 310b for selecting one or all consonants and / or a vowel selection object 310c for selecting one or all vowels. When a consonant is selected (e.g., "...") via the consonant selection object 310b and / or the vowel selection object 310c... ) and vowels (e.g. " When selecting a consonant, you can provide a syllable (e.g., ") below the consonant selection object 310b and the vowel selection object 310c, which combines the selected consonant with the vowel. ”310d and playback object 310e.

[0114] Referring again to the first screen 311 and / or the second screen 312, for example, quick access objects 306 for sound-picture appreciation training, quick access objects 307 for sound-drawing training, phoneme learning training information 308, and / or quick access objects 309 for training may be provided. Based on the designation of object 306 or 307, the electronic device 100 may provide corresponding training content, the details of which will be given later. Phoneme learning training information 308 may include phonemes (such as "...") planned for current training (e.g., today). The information includes the vowel or consonant of the word and the part to be performed (e.g., the upper part, the lower part, or the comprehensive training).

[0115] Based on the selection of training fast access object 309, it is possible to provide, for example Figure 3bThe fourth screen 380 is shown. The fourth screen 380 may include objects 381, 382, ​​383, 384, 385, and 386 associated with multiple training contents. Based on the selection of at least one of objects 381, 382, ​​383, 384, 385, and 386, training content corresponding to the selected object can be provided. Alternatively, objects 381, 382, ​​383, 384, 385, and 386 can be implemented as unselectable. For example, the electronic device 100 can be configured to provide training content in a specified order. In this case, objects corresponding to content that cannot yet be executed (e.g., objects 382, ​​383, 384, and 385) can be displayed as disabled, although this is merely an example and not a limitation. In this case, based on the designation of object 386 in the fourth screen 380 corresponding to "Proceed to the next training," the electronic device 100 can provide a specific training content based on the specified order. For example, the fourth screen 380 may also include an object 387 for transitioning the state to the initial screen (e.g., the first screen 311 or the second screen 312).

[0116] Figure 3c An example of a screen provided by an electronic device 100 according to one embodiment is shown.

[0117] According to one embodiment, the electronic device 100 can provide, for example... Figure 3c The screen 390 is shown. For example, screen 390 can be configured based on the user's performance on auditory training content for at least one English phoneme. For example, consonants that can constitute an English phoneme can be arranged vertically, and vowels can be arranged horizontally, although this is not a limitation. There are no restrictions on the consonants and / or vowels (or combinations thereof) used to constitute an English phoneme.

[0118] Figure 4 This is a schematic diagram illustrating a method for providing content for auditory training according to one embodiment. Figure 4 The embodiments will refer to Figures 5a to 5i Describe it. Figures 5a to 5i This is a schematic diagram illustrating the content provided by an electronic device according to various embodiments.

[0119] Reference Figure 4 In operation 401, electronic device 100 may provide a first sound (or test sound) comprising a plurality of first segments. Here, providing the first sound may, for example, involve outputting sound through a speaker included in electronic device 100, or, as understood by those skilled in the art, transmitting data for triggering sound output to an external speaker connected to electronic device 100 via a wired or wireless means. For example, electronic device 100 may provide (e.g., display) such as Figure 5aThe user interface shown can be used to provide the first sound, and this can be combined with the provision of the user interface. Those skilled in the art will understand that there is no limitation on when to begin providing the user interface.

[0120] Reference Figure 5a The user interface may include multiple areas: a first area 501, a second area 502, and a third area 503, although this is only an example. The third area 503 may include a training title 503a and a homepage object 503b for returning to the initial screen 311 or 312. The second area 502 may include a first playback object 502a for requesting (e.g., playing) a first sound (or test sound) for drawing training, and a mode display area 502b for displaying an object corresponding to the first sound. The first area 501 may include a visual mode input area 501a, a drawing object (e.g., a pencil) 501b, an erasing object (e.g., an eraser) 501c, a second playback object (e.g., "My Drawing") 501d, and a correct answer submission object (e.g., "Submit Answer") 501e. The visual mode input area 501a may include a plurality of first objects 11-1, 11-2, 11-3, 12-1, 12-2, 12-3, 13-1, 13-2, 13-3 for selecting from a plurality of specified values ​​related to the characteristics of a plurality of segments of a test sound. For example, each of the plurality of first objects 11-1, 11-2, 11-3, 12-1, 12-2, 12-3, 13-1, 13-2, 13-3 may correspond to one of the specified values ​​related to the characteristics of the sound. The plurality of first objects may be arranged in a grid (e.g., 3 rows × 3 columns). In this case, the first objects in each column may correspond to different values, while the first objects in each row may correspond to the same values. For example, the first objects 11-1, 12-1, 13-1 in the first row may correspond to a first value (e.g., a first amplitude frequency or a first timbre). The first objects 11-2, 12-2, and 13-2 in the second row can correspond to the second value (e.g., the frequency of the second amplitude or the second timbre). The first objects 11-3, 12-3, and 13-3 in the third row can correspond to the third value (e.g., the frequency of the third amplitude or the third timbre).

[0121] The first playback object 502a can be configured to trigger the provision of a test sound when selected. On the correct visual mode display area 502b, in response to the selection of the correct answer submission object 501e (or the request for the correct visual mode), an object corresponding to the correct visual mode associated with the attributes of the test sound can be displayed. Simultaneously, before confirming the selection of the correct answer submission object 501e (or confirming the request for the correct visual mode), the correct visual mode display area 502b can, for example, as follows: Figure 5a The image shows objects that are prevented from being previewed (e.g., question marks). It should be noted that... Figure 5a As shown, providing the correct visual mode display area 502b as a region distinct from the visual mode input area 501a is merely an example. In other implementations, the user interface may be designed without omitting the correct visual mode display area 502b, instead including only the visual mode input area 501a. In this case, objects with the correct visual mode may be displayed on the visual mode input area 501a, either side-by-side with objects displaying user-input visual modes, or sequentially relative to objects displaying user-input visual modes. Implementations are not limited to these examples.

[0122] Upon confirmation of the selection of the first playback object 502a, the electronic device 100 may provide a test sound 540. Based on the provision of the test sound 540, the pause object 502aa may be displayed in place of the first playback object 502a; however, this is not limited to this. The test sound 540 may have at least one characteristic (e.g., frequency, timbre, overtone density, and / or volume, but not limited to these). At least some characteristics of the test sound 540 may be designed for auditory training. For example, in Figure 5b In the example, the frequency of the first portion 543 of the test sound 540 can vary from f1 to f2 within a first time interval 541 between a first time point t1 and a second time point t2 (e.g., a linear variation, although the manner of variation is not limited). Similarly, the frequency of the second portion 544 of the test sound 540 can vary from f2 to f1 within a second time interval 542 between a second time point t2 and a third time point t3. It should be noted that frequency is merely an exemplary characteristic, and the type of characteristic is not limited. Furthermore, although... Figure 5b The variation of the characteristic (frequency) is shown, but those skilled in the art should understand that the characteristic may also remain constant over certain time intervals.

[0123] As test sound 540 is provided, a first indicator 51 can be depicted moving across the mode display area 502b at a speed of v1, visually indicating the playback position of test sound 540. The speed v1 can be set based on a test difficulty level, which will be described in detail later. For example, the horizontal length of the mode display area 502b can correspond to the entire duration of the time intervals 541 and 542 during which test sound 540 is provided. At the start of providing test sound 540, the first indicator 51 can be depicted moving from the left edge of the mode display area 502b. As time progresses and test sound 540 continues to play, the first indicator 51 can also be depicted moving proportionally to the cumulative time during which test sound 540 is provided, advancing to the right. At a third time point t3, the first indicator 51 can be depicted reaching the right edge of the mode display area 502b.

[0124] For example, during the provision of test sound 540, the second indicator 52 can be depicted moving at the same speed v1 as the first indicator 51. The horizontal length of the visual pattern input area 501a can correspond to the total duration of the time intervals 541 and 542 during which the test sound 540 is provided. For example, a first portion 531 of the visual pattern input area 501a can correspond to time interval 541, and a second portion 532 can correspond to time interval 542. At the start of test sound 540, the second indicator 52 can be depicted moving from the left edge of the visual pattern input area 501a. As time progresses and test sound 540 continues to play, the second indicator 52 can also be depicted moving proportionally to the cumulative time during which test sound 540 is provided, advancing to the right. At a third time point t3, the second indicator 52 can be depicted reaching the right edge of the visual pattern input area 501a. Thus, the user can identify which portion of the visual pattern input area 501a corresponds to the auditory test sound 540, and where to input a visual pattern object matching that sound. Meanwhile, the provision of indicators 51 and 52 is merely illustrative, and it should be understood that at least some representations of indicators 51 and 52 may be omitted or skipped depending on the difficulty level of the auditory training.

[0125] For example, the vertical direction of the visual mode input area 501a can correspond to the characteristics of sound. Figure 5b In the example shown, the vertical direction of the visual pattern input region 501a can correspond to the frequency range f1 to f3. For example, the top portion of the visual pattern input region 501a (or objects 11-1, 12-1, 13-1 at the highest position) can correspond to frequency f1. The middle portion (or objects 11-2, 12-2, 13-2 at the center) can correspond to frequency f2, and the bottom portion (or objects 11-3, 12-3, 13-3 at the lowest position) can correspond to frequency f3. The frequency boundaries (f1, f3) and / or the frequency range Δf can be predefined or set based on the training difficulty level; however, these settings are not limited, and their detailed descriptions will be given later. Figure 5b As shown, while listening to the test sound 540, the user can prepare to input an object with a visual pattern corresponding to the test sound 540 in the visual pattern input area 501a.

[0126] Refer again Figure 4In operation 403, electronic device 100 can identify user input associated with at least some of a plurality of first objects (e.g., objects 11-1, 11-2, 11-3, 12-1, 12-2, 12-3, 13-1, 13-2, 13-3) through a user interface configured to receive user input including multiple sub-user inputs. In operation 405, electronic device 100 can provide at least one second object (e.g., an object with a visual pattern) associated with at least some of the plurality of first objects (e.g., objects 11-1, 11-2, 11-3, 12-1, 12-2, 12-3, 13-1, 13-2, 13-3) in response to the plurality of sub-user inputs. Furthermore, electronic device 100 can provide a second sound including multiple second segments, the multiple second segments being identified based on the respective sub-user inputs. For example, as... Figure 5c As shown, a user can use finger 1 to input a visual pattern corresponding to the test sound 540 on the visual pattern input area 501a. The user can input a visual pattern when the object 501b for visual pattern input is activated. However, this is not limited to this. When the object 501b for visual pattern input is activated, the object 501c for deleting the input visual pattern can be deactivated, but this is only an example. For example, as... Figure 5c As shown, during the first time interval 551, the user can input a first user input (input1) from object 11-1 to object 12-1 (e.g., dragging, but not limited to this). During the second time interval 552, the user can release touch on object 12-1 with finger 1, move finger 1, and touch object 12-2. During the third time interval 553, the user can input a second user input (input2) from object 12-2 to object 13-2.

[0127] Electronic device 100 can provide sound 550 based on at least one user input. For example, a first segment 554 of sound 550 can have characteristics corresponding to a first user input (input1). For example, the first user input (input1) can be a rightward movement from object 11-1 to object 12-1. Therefore, the first segment 554 provided during a first time interval 551 can include: a sub-segment corresponding to object 11-1 and having a first frequency (f1), sub-segments corresponding to multiple points between objects 11-1 and 12-1 and each having a first frequency (f1), and a sub-segment corresponding to object 12-1 and having a first frequency (f1). Subsequently, during a second time interval 552, in response to the absence of user input, electronic device 100 can not output sound. For example, a second segment 555 of sound 550 can have characteristics corresponding to a second user input (input2). For example, the second user input (input2) can be a rightward movement from object 12-2 to object 13-2. Therefore, the second segment 555 provided during the third time interval 553 may include: a sub-segment corresponding to object 12-2 and having a second frequency (f2), a sub-segment corresponding to multiple points between objects 12-2 and 13-2 and each having a second frequency (f2), and a sub-segment corresponding to object 13-2 and having a second frequency (f2). As described above, while listening to the sound 550 corresponding to the point (or transition between points) that the user is inputting, the user can input a visual pattern corresponding to the test sound 540.

[0128] Reference Figure 5d It can display as follows Figure 5c The image shows objects 571 and 572 with visual patterns recognized based on user input (input1 and input2). Furthermore, even after objects 571 and 572 have been displayed, the user can continue to provide additional input. For example, as... Figure 5d As shown, if a user touches and holds object 11-3, electronic device 100 can provide sound 566 corresponding to object 11-3. Sound 566 may have a third frequency (f3) corresponding to object 11-3. On the other hand, if no event for displaying an object with a visual pattern is detected, the device can be configured not to display objects such as visual patterns 571 and 572 even if user input is recognized. For example, an event for displaying an object with a visual pattern may involve recognizing user input that designates at least two of the objects (11-1, 11-2, 11-3, 12-1, 12-2, 12-3, 13-1, 13-2, 13-3) as a start point and an end point. However, this is only an example, and there is no limitation on the type of event for displaying an object with a visual pattern. For example, electronic device 100 may temporarily display an object corresponding to a user touch point and stop displaying the object if no event is detected, but this is not a limitation.

[0129] Meanwhile, providing objects 571 and 572 with visual patterns corresponding to user inputs (input1, input2) and associated sounds 554 and 555 is merely an example. Depending on the implementation, the electronic device 100 may provide objects 571 and 572 with visual patterns corresponding to user inputs (input1, input2) without providing sounds 554 and 555 associated with the objects 571 and 572. Those skilled in the art will understand this.

[0130] For example, if a user wants to hear the sound corresponding to an object they created, they can select the second playback object 501d. Figure 5e As shown, based on the selection of the second playback object 501d, the stopped playback object 501dd can be displayed as a substitute for the second playback object 501d, although this is not limited to this. The electronic device 100 can provide sound 570 corresponding to objects 571 and 572 having visual modes, based on the selection of the second playback object 501d. For example, a first portion 573 of sound 570 can be provided during a first time interval 571. The first portion 573 of sound 570 can correspond to the first portion 571 of the object. Since the first portion 571 of the object corresponds to a first frequency f1, the first portion 573 of sound 570 can have the first frequency f1. Similarly, a second portion 574 of sound 570 can be provided during a second time interval 572. The second portion 574 of sound 570 can correspond to the second portion 572 of the object having a visual mode. Since the second portion 572 of the object corresponds to a second frequency f2, the second portion 574 of sound 570 can have the second frequency f2. The duration of the first time interval 571 can, for example, be substantially the same as the duration of the second playback object 571. Figure 5b The duration of the first time interval 541 of the first portion 543 of the test sound 540 is the same, although this is not a limitation. Similarly, the duration of the second time interval 572 can be substantially the same as that in the first time interval 541. Figure 5b The duration of the second time interval 542 of the second part 544 of the test sound 540 is the same, although this is not a limitation. Thus, the user can confirm whether the visual pattern they created corresponds to the test sound 540. If the user recognizes that the visual pattern they created does not correspond to the test sound 540, the user can activate (e.g., touch) the erase object 501c to delete at least a portion of the displayed objects 571 and 572 with visual patterns, and input a new object with a different visual pattern.

[0131] Refer again Figure 4In operation 407, electronic device 100 may provide at least one third object (e.g., an object with a correct visual pattern) corresponding to multiple degrees of a first characteristic of multiple first parts of a first sound. For example, electronic device 100 may provide, based on the selection of the correct answer submission object 501e, such as Figure 5f The display shows an object 573 with the correct visual pattern corresponding to the test sound 540. Based on the selection of the correct answer submission object 501e, an object 501f for the next test can be displayed in place of the correct answer submission object 501e, although this is not a limitation. Simultaneously, the presentation of the object 573 with the correct visual pattern can also be performed based on various events (e.g., elapsed time, or based on user input matching the object with the correct answer with a visual pattern), and is not limited to the designation of object 501e alone. Furthermore, as described above, in another implementation, the electronic device 100 can simultaneously display objects 571 and 572 with visual patterns recognized based on user input, as well as the object with the correct visual pattern, in the visual pattern input area 501a. (See also...) Figure 5f On the right side, an object 581 with the correct visual pattern corresponding to the test sound of the auditory training previously performed by the user, and an object 582 with a visual pattern recognized by the user input, can be displayed together. Furthermore, based on the selection of an object 501g associated with the termination of auditory training based on auditory and visual patterns, the electronic device 100 can end the training.

[0132] exist Figure 5g In this context, user input can be identified as corresponding to the correct object in the test sound 540. For example, such as... Figure 5g As shown, the user can use finger 1 to input the visual pattern corresponding to the test sound 540 in the visual pattern input area 501a. For example, as Figure 5g As shown, during the first time interval 561, the user can input a first user input (input1) from object 11-1 to object 12-2, such as a dragging action (but not limited to this). During the second time interval 562, the user can keep their finger (1) touching object 12-1. During the third time interval 563, the user can input a second user input (input2) from object 12-2 to object 13-1.

[0133] Electronic device 100 can provide sound 560 based on at least one user input. For example, a first portion 564 of sound 560 can have characteristics corresponding to a first user input (input1). The first user input (input1) can involve a downward and rightward movement from object 11-1 to object 12-2. Therefore, the first portion 564 provided during a first time interval 561 can include: a sub-portion corresponding to object 11-1 and having a first frequency (f1), a sub-portion corresponding to several points between object 11-1 and 12-2 and having multiple intermediate frequencies (frequency between f1 and f2), and a sub-portion corresponding to object 12-2 and having a second frequency (f2). Subsequently, during a second time interval 552, based on the user input remaining on object 12-2, electronic device 100 can provide a second portion 565 of sound 560, which corresponds to the second frequency (f2) of object 12-2. For example, a third portion 566 of sound 560 can have characteristics corresponding to a second user input (input2). The second user input (input2) may involve moving upwards and to the right from object 12-2 to object 13-1. Therefore, the third portion 566 provided during the third time interval 563 may include: a sub-portion corresponding to object 12-2 and having a second frequency (f2), a sub-portion corresponding to several points between objects 12-2 and 13-1 and having multiple intermediate frequencies (frequency between f2 and f1), and a sub-portion corresponding to object 13-1 and having a first frequency (f1). As described above, the user can listen to the sound 560 corresponding to the point (or transition between points) they are inputting and use it as a reference to input an object with a visual pattern corresponding to the test sound 540.

[0134] Figure 5h This is a schematic diagram illustrating a sound test based on characteristics other than frequency, according to one embodiment.

[0135] For example, in Figures 5b to 5e In the example, frequency can be set as one of the characteristics of the test sound as a target feature for training. Conversely, in Figure 5h In the illustrated embodiment, volume can be set as one of the characteristics of the test sound as a target feature for training. Figure 5hIn this configuration, the vertical direction of the visual pattern input area 501a can correspond to a volume range, such as V1 to V3. For example, the top of the visual pattern input area 501a (or the uppermost object, such as 11-1, 12-1, 13-1) can correspond to volume level V1. The center of the visual pattern input area 501a (or the object at the center, such as 11-2, 12-2, 13-2) can correspond to volume level V2, and the bottom of the visual pattern input area 501a (or the lowest object, such as 11-3, 12-3, 13-3) can correspond to volume level V3. The volume thresholds V1, V3, and / or the volume range ΔV can be predetermined or adjusted based on training difficulty, although the implementation method is not limited. A detailed explanation of this configuration will be given later. Figure 5h As shown, while listening to the corresponding portions 593 and 594 of the test sounds provided during time intervals 591 and 592, the user can prepare to input an object with a visual pattern corresponding to the test sounds 593 and 594 into the visual pattern input area 501a. Those skilled in the art should also understand that, apart from characteristics that can be numerically represented such as frequency or volume, all sound characteristics can be used without restriction for auditory training.

[0136] Figure 5i This is a schematic diagram illustrating auditory training based on multidimensional characteristics according to one embodiment.

[0137] For example, multiple characteristics of a test sound (e.g., a first characteristic and a second characteristic) can be designed for auditory training. The test sound at a first time point t1 may include a portion having a first characteristic of degree x1 and a second characteristic of degree y1. Between the first time point t1 and the second time point t2, the test sound may include a portion where the first characteristic changes from x1 to x2 and the second characteristic changes from y1 to y2. Between the second time point t2 and the third time point t3, the test sound may include a portion where the first characteristic changes from x2 to x3 and the second characteristic changes from y2 to y3. Between the third time point t3 and the fourth time point t4, the test sound may include a portion where the first characteristic changes from x3 to x4 and the second characteristic changes from y3 to y4. Between the fourth time point t4 and the fifth time point t5, the test sound may include a portion where the first characteristic changes from x4 to x5 and the second characteristic changes from y4 to y5.

[0138] Electronic device 100 can provide a user interface for inputting a visual pattern corresponding to a test sound. For example, the user interface may include a first sub-user interface for inputting a portion of the projected visual pattern corresponding to a first characteristic, and a second sub-user interface for inputting a portion of the projected visual pattern corresponding to a second characteristic. For example, a user can input the portion of the projected visual pattern for the first characteristic through the first sub-user interface and the portion of the projected visual pattern for the second characteristic through the second sub-user interface. In another implementation, the user interface can be configured to allow input of three-dimensional information (first characteristic, second characteristic, and time). For example, when electronic device 100 provides a VR, AR, or MR environment, electronic device 100 can identify the position of the input device (or a part of the user's body) in three-dimensional space. Therefore, based on the trajectory of the input device (or a part of the user's body) in three-dimensional space, electronic device 100 can identify the visual pattern defined in three dimensions. Meanwhile, those skilled in the art will understand that the aforementioned two-dimensional visual patterns can also be identified based on a VR, AR, or MR environment.

[0139] Figure 6a This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0140] In this embodiment, the electronic device 100 may provide a first sound having a first auditory pattern corresponding to a first characteristic of the sound in operation 601. For example, if the first characteristic is frequency, the first auditory pattern may be a sound whose frequency varies over time; however, this is merely an example, and there is no limitation on the type or number of characteristics used to form the pattern. The first sound here may serve as a test sound for assessing a user's hearing ability (and / or for improving hearing ability). For example, the electronic device 100 may provide a first sound 540 as a test sound, which has, for example... Figure 5b The first auditory pattern (e.g., a pattern in which the frequency decreases from f1 to f2 during the first time interval 541 and increases from f2 to f1 during the second time interval 542).

[0141] Electronic device 100 can, in operation 603, use a user interface (e.g., for inputting a visual pattern associated with the first characteristic) to input a visual pattern. Figure 5c The visual mode input area 501a in the middle provides a corresponding input mode for user input (e.g., visual mode input area 501a). Figure 5c The first object is a first visual pattern (the first user input 1 and the second user input 2 in the system). Furthermore, the electronic device can provide a second sound (e.g., a second auditory pattern corresponding to the user input). Figure 5c The sounds 554 and 555 in the text. For example, the second sound (e.g.) Figure 5cThe timing and / or duration of sounds 554, 555 in the input circuit can be substantially synchronized with the detection timing and / or duration of the first user input 1 and the second user input 2; however, this synchronization is not required. Alternatively or additionally, the electronic device may replace or exclude a second sound synchronized with the first and second user inputs (e.g., ...). Figure 5c In addition to the sounds 554 and 555 in the original text, a sound corresponding to the first object having the first visual mode is provided (e.g., ...). Figure 5e (Voice 570 in the text). This will refer to... Figure 6b The description should be provided. It should also be noted that providing a second sound with a second auditory mode corresponding to the user input (e.g., ...) can be omitted. Figure 5c (Sounds 554, 555). In operation 605, electronic device 100 can provide a second object (e.g., a second visual mode corresponding to the first auditory mode) having a second visual mode. Figure 5f Object 573 in the document, also known as the correct visual mode object. For example, based on the selection of the correct answer submission object (e.g., "Submit Answer" object 501e), the device can provide a second object with a second visual mode (e.g., ...). Figure 5f (Object 573 in the example), although the way to trigger providing this second object is not limited to this example.

[0142] Figure 6b This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0143] According to this embodiment, the electronic device 100 can provide a first sound with a first auditory pattern having a first characteristic of the sound in operation 621. In operation 623, the electronic device 100 can provide a first sound through a user interface for inputting a visual pattern associated with the first characteristic (e.g., ...). Figure 5c The visual mode input area 501a in the user interface provides a corresponding input (e.g., visual mode input area 501a) via the user interface. Figure 5c The first object is a first visual pattern of the first user input (input1) and the second user input (input2). For example, in operation 625, the electronic device 100 can provide a first object (e.g., based on the detected request to play a sound corresponding to the first object) with the first object (e.g., [the first object]). Figure 5d The second sound corresponds to objects 571 and 572 in the first object. For example, based on the selection of the second playback object (e.g., "My Drawing" object 501d), the electronic device 100 can provide a second sound corresponding to the first object (e.g., the second sound corresponding to the first object). Figure 5d The second sound corresponding to objects 571 and 572 in the text (e.g.) Figure 5e (Sound 570 in the example). However, the way in which the sound is triggered is not limited to this example. In operation 627, electronic device 100 can provide a second object having a second visual mode corresponding to the first auditory mode (e.g., sound 570 in the example). Figure 5f Object 573 in the document, also known as the correct visual mode object. For example, based on the selection of the correct answer submission object (e.g., "Submit Answer" object 501e), the device can provide a second object with a second visual mode (e.g., ...). Figure 5f (Object 573 in the example), although the way to trigger providing this second object is not limited to this example.

[0144] Figure 7 This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0145] According to this embodiment, the electronic device 100 can provide a test sound with a first auditory pattern having a first characteristic for the sound in operation 701. In operation 703, the electronic device 100 can provide a first sound with a first degree of a first characteristic corresponding to a first location, based on recognizing user input located at a first location on the user interface. For example, as shown in reference... Figure 5b The vertical direction of the visual mode input area 501a can correspond to the characteristics of sound. Figure 5b In the example, the vertical direction of the visual pattern input region 501a corresponds to a frequency range from f1 to f3. Therefore, based on user input at a first point in the visual pattern input region 501a, a sound with characteristic values ​​(e.g., frequency) corresponding to that first point can be provided. For example, the sound can be provided while the user input remains at that first point, although this is not limited to this. For example, as shown in the reference... Figure 5g As described, during a first time interval (561), the user can input a first user input (input1) from object 11-1 to object 12-2, such as a drag operation. Based on the user input at object 11-1, a sound with a frequency f1 corresponding to object 11-1 can be provided. In operation 705, the electronic device 100 can move from a first position to a second position based on the user input, providing a second sound with a first characteristic and a second degree corresponding to the second position of the user interface. Simultaneously, the device can provide a first object associated with the first and second positions. For example, such as... Figure 5g As shown, a first user input (input1) (e.g., a drag operation) moves the user input from object 11-1 to object 12-2. Therefore, the electronic device 100 can provide a sound with a frequency f2 corresponding to object 12-2. The device can also provide at least one intermediate sound with characteristics corresponding to the position between the first position (object 11-1) and the second position (object 12-2) during the time interval between providing the sound corresponding to the first position and providing the sound corresponding to the second position. For example, as... Figure 5gThe sound 564 may include a sound corresponding to frequency f1 at a first position, a sound corresponding to frequency f2 at a second position, and an intermediate sound between the two sub-sounds. For example, the intermediate sound may have a frequency between f1 and f2. Although the frequency of the intermediate sound is shown to vary linearly, this is merely an example and not a limitation. In operation 707, after providing the first object, the electronic device 100 may provide a second object, such as a correct object, corresponding to a first auditory pattern of a first characteristic of the test sound.

[0146] Figure 8 This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0147] According to this embodiment, in operation 801, electronic device 100 can provide a test sound of a first auditory pattern for a first characteristic of sound. In operation 803, electronic device 100 can detect user input specifying multiple locations on the user interface. Based on the specified multiple locations, the device can provide a first object representing a trajectory associated with the user input while providing a first sound including portions corresponding to the degree of the first characteristic at the multiple locations. In operation 805, electronic device 100 can determine whether to keep displaying or stop displaying the first object based on whether a confirmation event for the first object is detected. For example, if the trajectory associated with the user input (e.g., the first object) corresponds to multiple reference points on the user interface (e.g., ...) by touching, passing through, or approaching them. Figure 5a If multiple first objects 11-1, 11-2, 11-3, 12-1, 12-2, 12-3, 13-1, 13-2, and 13-3 are displayed, the device can confirm that a confirmation event for the first object has occurred. After detecting the confirmation event, the electronic device 100 can maintain the display of all or part of the first object. For example, the device can maintain the display of at least one trajectory connecting multiple reference points on the user interface (e.g., as shown in the image). Figure 5a(as described above). However, this behavior is not limited. If no confirmation event for the first object is detected, the electronic device 100 may stop displaying the trajectory. For example, if the trajectory does not correspond to multiple reference points, the device may stop displaying the trajectory after it has been displayed based on the detection of user input termination (e.g., release of touch). In operation 807, based on confirmation of a request to provide the second object, the electronic device 100 may provide a second object corresponding to the first auditory pattern of the first characteristic of the test sound. For example, when a user determines that the input object corresponds to their desired visual pattern, the user may request a second object corresponding to the first auditory pattern (e.g., by selecting a correct confirmation object such as submitting answer 501e). After confirmation of the request (e.g., detection of a selection of submitting answer 501e), the electronic device 100 may provide a second object corresponding to the first auditory pattern.

[0148] Figure 9 This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0149] According to this embodiment, in operation 901, electronic device 100 can provide a test sound of a first auditory pattern for a first characteristic of sound. In operation 903, electronic device 100 can detect user input at multiple locations on a specified user interface. Based on the specified locations, the device can provide a first object representing a trajectory associated with the user input while providing a first sound including a portion corresponding to the degree of the first characteristic at the specified location. In operation 905, based on the detection of a confirmation event for the first object, electronic device 100 can maintain the display of all or part of the first object. In operation 907, based on a modification request for at least a portion of the first object, electronic device 100 can provide a modified first object. For example, after confirming the selection to delete an object (e.g., eraser 501c), electronic device 100 can change the mode of the user interface from an object input mode to an object deletion mode (or modification mode). In object input mode, an object corresponding to user input on the user interface (e.g., visual pattern input area 501a) can be displayed. In object deletion mode, a previously displayed object (e.g., visual pattern input area 501a) on the user interface can be deleted (or hidden) based on user input. For example, an entire previously displayed object can be deleted based on user input. Alternatively, a portion of a previously displayed object can be deleted based on user input. The deleted portion may correspond to segments between reference points, but this is merely an example, and the deletion units are not limited. If the selection of a drawing object (e.g., pencil 501b) is confirmed, the electronic device 100 can switch the user interface mode from deletion mode (or modification mode) back to object input mode. In operation 909, based on confirmation of a request to provide a second object corresponding to the first auditory mode of the first characteristic of the test sound, the electronic device 100 can provide the second object. For example, after confirming the user's selection of a correct confirmation object (e.g., submitting answer 501e), the electronic device 100 can provide the second object corresponding to the first auditory mode.

[0150] Figure 10a This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment, which will combine... Figure 10b Please provide an explanation. Figure 10b This is a schematic diagram illustrating multiple sounds according to one embodiment.

[0151] In this embodiment, during operation 1001, the electronic device 100 can at least partially provide a first sound composed of multiple sounds. For example, such as Figure 10bAs shown, the frequency of the first part 543 of test sound 540A can change from f1 to f2 within a first time interval 541 between time points t1 and t2. The frequency of the second part 544 of test sound 540A can change from f2 to f1 within a second time interval 542 between time points t2 and t3. Furthermore, the frequency of the third part 546 of test sound 540A can remain constant at f3 within the same time interval 542 between time points t2 and t3. Figure 10b As shown, within the interval between t2 and t3, multiple sounds (e.g., second part 544 and third part 546) can be provided at least partially simultaneously. In operation 1003, electronic device 100 can detect user input associated with at least some first objects through a user interface for receiving input from multiple sub-users. In operation 1005, electronic device 100 can provide at least one second object associated with at least some first objects and provide a second sound including multiple second parts corresponding to the multiple sub-user inputs. For example, when listening to the multiple sounds (e.g., second part 544 and third part 546) within the interval between t2 and t3, the user can be prompted to input multiple objects corresponding to the multiple sounds in a second part 532 of the user interface corresponding to the interval between t2 and t3. These objects may include an object connecting objects 12-2 and 12-1 and another object connecting objects 12-3 and 13-3. In operation 1007, electronic device 100 can provide at least one third object, such as a correct answer object corresponding to the degree of the first characteristic of the multiple first parts of the first sound.

[0152] Figure 11a This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0153] In this embodiment, in operation 1101, electronic device 100 can provide a test sound having a first auditory pattern corresponding to a first characteristic of the sound. In operation 1103, electronic device 100 can detect user input and, based on the detection, provide a sound associated with a first object corresponding to the user input while providing the sound associated with the first object. In operation 1105, based on confirmation of a request to provide a second object, electronic device 100 can provide a second object corresponding to the first characteristic of the test sound and the first auditory pattern. In operation 1107, electronic device 100 can identify weaknesses in the user's association with the first characteristic based on a comparison of the first object and the second object. For example, electronic device 100 can identify weaknesses in the user's association with the first characteristic based on mismatches between the first object and the second object. These weaknesses can be represented as values ​​associated with the first characteristic (e.g., frequency values ​​or ranges in the case of frequency); however, the representation is not limited to this example.

[0154] According to one embodiment, electronic device 100 can provide auditory training content based on vulnerability information identified for each user. For example, such as Figure 11b As shown, electronic device 100 can determine that, based on the results of auditory training content 1131 associated with a first frequency range (f1 to f3), the weakness of a first user is located in the frequency range f4 to f5, while the weakness of a second user is located in the frequency range f6 to f7. Electronic device 100 can then provide auditory training content 1132 targeting the weakness of the first user based on a second frequency range (f4 to f5). Similarly, electronic device 100 can provide auditory training content 1133 targeting the weakness of the second user based on a third frequency range (f6 to f7). For example, electronic device 100 can dynamically adjust and provide content corresponding to the updated weakness based on changes in the user's weakness over time.

[0155] Figure 12a This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment. Figure 12a The embodiments will refer to Figure 12b To explain, Figure 12b This is a schematic diagram showing the user interface corresponding to different test difficulty levels.

[0156] According to one embodiment, electronic device 100 can determine a test difficulty level in operation 1201. For example, the test difficulty level can be determined based on settings specified by the user and / or administrator. Furthermore, the test difficulty level can also be determined at least in part based on the results of previously performed tests. For example, the test difficulty level can be adjusted based on the cumulative number of tests completed. However, there are no specific limitations on the jjs seemaaassaav method for determining the test difficulty level.

[0157] In operation 1203, electronic device 100 can determine a range of the first characteristic of the sound based on the test difficulty level. In operation 1205, electronic device 100 can provide a test sound and / or user interface configured based on the determined range. For example, refer to... Figure 12bThe first user interface 1211 can be represented based on a first frequency range of f1 to f3, and the test sound can also be designed within the first frequency range of f1 to f3. The size of the first frequency range can be represented as Δf1. Similarly, the second user interface 1212 can be represented based on a second frequency range of f5 to f6, and the test sound can be designed within the second frequency range of f5 to f6. The size of the second frequency range can be represented as Δf2. For example, a relatively high test difficulty level can correspond to a smaller feature range (e.g., a smaller frequency range) in the user interface. If the test difficulty level of the second user interface 1212 is higher than that of the first user interface 1211, the size of the second frequency range Δf2 can be smaller than the size of the first frequency range Δf1. However, those skilled in the art will understand that the opposite is also possible.

[0158] In another example, the range of this feature can be set based on the test difficulty level, rather than the size of the range. For example, the third user interface 1213 can be represented based on a third frequency range of f7 to f8, and the test sound can also be designed within the third frequency range of f7 to f8. The size of the third frequency range can be represented as Δf1. This means that the first frequency range and the third frequency range can have the same size Δf1, but their actual ranges can be different. As the test difficulty level changes, the size of the frequency range can remain constant, while the specific frequency range itself is adjusted.

[0159] Figure 12c This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment. Figure 12c The embodiments will refer to Figure 12d Please provide an explanation. Figure 12d This is a schematic diagram illustrating the user interface based on the test difficulty level.

[0160] According to one embodiment, electronic device 100 can identify a test difficulty level in operation 1221. In operation 1223, electronic device 100 can determine the unit (or density and / or quantity of reference objects) that the user can select based on the test difficulty level. In operation 1225, electronic device 100 can provide test sounds and / or user interfaces configured based on the determined unit (or density and / or quantity of reference objects). For example, referring to... Figure 12dThe first user interface 1231 may include reference objects arranged in a 3x3 grid. The auditory pattern of the test sound corresponding to the first user interface 1231 may be designed, for example, based on three levels corresponding to the vertical number of reference objects. The second user interface 1232 may include reference objects arranged in a 5x5 grid. The auditory pattern of the test sound corresponding to the second user interface 1232 may be designed, for example, based on five levels corresponding to the vertical number of reference objects. For example, as the test difficulty level increases, the units selectable by the user can be relatively smaller, the number of reference objects included in the user interface can be relatively larger, and / or the density of reference objects included in the user interface can be relatively higher.

[0161] Figure 12e This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0162] According to one embodiment, electronic device 100 can identify a test difficulty level in operation 1241. In operation 1243, electronic device 100 can determine additional sound based on the test difficulty level. In operation 1245, electronic device 100 can provide the test sound, the additional sound, and the user interface. For example, the additional sound can be noise. Listening in a noisy environment can be relatively more challenging. Therefore, whether to provide the additional sound and / or the characteristics of the additional sound (e.g., the type, frequency, timbre, and / or volume of the additional sound, etc., are not limited) can be determined based on the test difficulty level.

[0163] Figure 12f This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment. Figure 12f The embodiments will refer to Figure 12g To explain, Figure 12g This is a schematic diagram illustrating the application of sound effects according to one embodiment.

[0164] According to one embodiment, electronic device 100 can identify a test difficulty level in operation 1251. In operation 1253, electronic device 100 can determine a sound effect based on the test difficulty level. In operation 1255, electronic device 100 can provide a test sound with the determined sound effect applied, along with a user interface. For example, the sound effect may include a reverb effect, but the type or number of sound effects is not limited. When sound effects such as reverb are applied, auditory perception may be relatively more challenging. Therefore, the application, type, number, and / or intensity of the sound effect can be determined based on the test difficulty level. Figure 12g As shown, selectable objects 1261 and 1262 can be provided to the user. For example, based on the selection of object 1261, the electronic device 100 can, as Figure 12eThe device provides noise as an additional sound while providing the test sound. Based on the selection of object 1262, electronic device 100 can, as described... Figure 12f The test sound with a reverberation effect is provided. For example, when providing a test sound with a reverberation effect, the electronic device 100 can also apply visual effects such as reverberation to the object corresponding to the user input and make a visual representation.

[0165] Figure 12h This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment.

[0166] According to one embodiment, electronic device 100 identifies a test difficulty level in operation 1271. In operation 1273, electronic device 100 determines the playback speed of the test sound based on the test difficulty level. In operation 1275, electronic device 100 provides the test sound at the determined speed. When the playback speed of the test sound is relatively high, auditory perception may become more challenging. Therefore, the playback speed of the test sound can be determined based on the test difficulty level.

[0167] Figure 13a This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment. Figure 13a The illustrated embodiments will refer to Figure 13b To explain, Figure 13b This is a schematic diagram illustrating the location of a virtual sound source in a stereo environment according to an embodiment.

[0168] According to one embodiment, electronic device 100 provides a first training program based on a first setting in operation 1301. In operation 1303, electronic device 100 provides the same first training program based on a second setting. For example, as in combination Figures 12a to 12g The electronic device 100 can adjust various parameters based on the identified test difficulty level. For example, the electronic device 100 can determine parameters corresponding to a first setting and a second setting based on the identified test difficulty level. In another example, the electronic device 100 can configure the same training program with different voices (or frequencies or timbres).

[0169] In one example, electronic device 100 can provide test sounds for both the left and right ears in a stereo environment (or headphone environment). For example, electronic device 100 can provide test sounds only for the left ear, only for the right ear, or for both ears, thus providing test sounds based on multiple settings. For example, a user may have hearing loss in the left ear, right ear, or both ears. Electronic device 100 can configure the combination of test sounds in the stereo environment based on information related to the user's hearing loss (such as, but not limited to, cochlear implant surgery information).

[0170] For example, such as Figure 13b As shown, sound output devices 1304 and 1305 (e.g., headphones) can be inserted into a user's ears. By controlling the phase of the sound output by each sound output device 1304 and 1305, the perceived direction of virtual sound sources 1311, 1312, and 1313 can be adjusted. For example, electronic device 100 can manipulate the phase of the sound provided by sound output devices 1304 and 1305, causing the user to perceive a test sound source as originating from a first virtual sound source 1311, a second virtual sound source 1312, or a third virtual sound source 1313. As mentioned above, a user may have hearing loss in their left ear, right ear, or both ears. A user with hearing loss in their left ear may have difficulty perceiving sounds relative to their position originating from the left. Electronic device 100 can configure the positions of virtual sound sources 1311, 1312, and 1313 in various ways based on information about the user's hearing loss (e.g., cochlear implant surgery information, but not limited to this).

[0171] Figure 13c This is a schematic diagram illustrating an exemplary screen for providing auditory training content according to an embodiment.

[0172] According to one embodiment, the electronic device 100 can be as follows: Figure 13cA list of writing tools 1340 is provided. The triggering conditions for displaying the list of writing tools 1340 are not limited to any specific conditions. The list of writing tools 1340 may include multiple objects 1341, 1342, 1343, 1344, 1345, 1346, 1347, 1348, and 1349. Each object (e.g., 1341 to 1349) may visually represent an attribute or characteristic corresponding to a specific writing tool (e.g., text labels and graphical previews of writing tool attributes), but this representation is merely illustrative and not restrictive. The electronic device 100 can detect the selection of any object in the list of writing tools 1340. Based on the selected object, the device can configure certain attributes of the test sound. For example, if object 1346 labeled "crayon" is selected, the electronic device 100 can set the timbre of the test sound to correspond to the attributes associated with the "crayon" object. For example, the electronic device 100 can provide a test sound obtained by applying a bandpass filter to white noise. The timbre can be altered by adjusting the passband width (or Q value) of the filter. Alternatively, the device can adjust individual components of the test sound (e.g., harmonic components) to achieve specific timbre variations. Electronic device 100 can be configured to provide test sounds with various attributes corresponding to different objects, thereby enabling a wide range of test sound variations. Furthermore, the device can visually represent the trajectory of a specific attribute corresponding to a selected object based on user input. User-interactive selection of a writing tool is merely an example. Electronic device 100 can also autonomously select a writing tool based on predefined rules (e.g., difficulty level, user training level, or user weaknesses). If a writing tool is selected according to these rules, electronic device 100 can provide a test sound reflecting the attributes of the selected writing tool. Furthermore, electronic device 100 can visually represent objects reflecting the attributes of the selected writing tool. In another example, electronic device 100 can provide test sounds with predefined attributes and present a user interface requiring the user to identify these attributes. For example, a user can listen to a test sound with unknown attributes from a list of writing tools 1340 (e.g., ...). Figure 13c (As shown) Select the corresponding writing tool and input its response. Then, the electronic device 100 can represent an object with the attributes of the writing tool selected by the user and provide a sound reflecting the corresponding attribute. The user can compare the sound attributes associated with their selected writing tool with the attributes of the test sound to determine if they match. Subsequently, the user can submit their answer, modify the object, or reselect the writing tool if necessary.

[0173] For example, a single test sound can be assigned to a single writing tool; however, this is merely an example. Each sub-sound of a test sound can correspond to a different writing tool (e.g., the first part corresponds to "pencil," and the second part to "crayon"). In this case, it might be necessary to select a different writing tool for each sub-object based on user input. Therefore, the attributes of each sub-object can be represented in different ways to reflect the attributes of the corresponding writing tool.

[0174] Figure 13d This is a schematic diagram illustrating a screen for providing auditory training content according to one embodiment.

[0175] In this embodiment, the electronic device 100 can be as follows: Figure 13d A list of paper materials 1350 is provided. The triggering conditions for displaying the list of paper materials 1350 are unrestricted. The list of paper materials 1350 may include multiple objects 1351, 1352, 1353, and 1354. For example, each object 1351, 1352, 1353, and 1354 can be represented by a text name containing the paper material; however, this is merely an example and not limiting. The electronic device 100 can recognize the selection of any one of the multiple objects 1351, 1352, 1353, and 1354 in the list of paper materials 1350. Based on the selected object, the electronic device 100 can configure the attributes of the test sound. For example, if object 1351 corresponding to "Korean traditional paper" is selected, the electronic device 100 can set the timbre, one of the attributes of the test sound, to correspond to object 1351. For example, the electronic device 100 can base its configuration on... Figure 13c The selected writing instrument type shown is similar to... Figure 13d The combination of paper materials selected as shown determines the properties of the test sound. Electronic device 100 can also provide a visual representation of the trajectory of the properties of the selected object based on user input. For example, electronic device 100 can be based on... Figure 13c The selected writing instrument type shown is similar to... Figure 13dThe combination of paper materials selected is shown to determine the attributes of a visual object. The user selection of paper materials is merely an example. Electronic device 100 can also select paper materials based on predefined rules (e.g., difficulty level, user training level, and user weaknesses) without user input. For example, if paper materials are selected based on predefined rules, electronic device 100 can provide test sounds and / or background sounds corresponding to the attributes of the selected paper materials. Furthermore, electronic device 100 can visually represent objects with the attributes of the selected paper materials based on user input. Alternatively, electronic device 100 can modify the attributes of the input area. In another example, electronic device 100 can provide test sounds with predefined attributes based on specified rules and provide a user interface that prompts the user to select these attributes. For example, a user can listen to test sounds and / or background sounds with unknown attributes via a list of paper materials 1350 (e.g., ...). Figure 13d (As shown) Select the corresponding paper material and then input its response. The electronic device 100 can visually represent an object with the attributes of the paper material selected by the user and / or provide sound with the attributes of the paper material selected by the user. The user can then determine whether the sound attributes of their selected paper material match the attributes of the test sound and / or the background sound. If they do not match, the user can submit a correction, such as reselecting the paper material.

[0176] exist Figure 13c and / or Figure 13d While examples of configuring test sound properties based on writing instruments and / or paper materials are given, these are merely examples. For instance, electronic device 100 may also configure the properties of additional sounds (e.g., background sounds) provided with the test sound based on at least a portion of the writing instrument or paper material. For example, the properties of the test sound may be adjusted based on the writing instrument, while the properties of the additional sounds (e.g., background sounds) may be adjusted based on the paper material. However, this is merely an example and not restrictive.

[0177] For example, a paper material could be assigned to the test sound and / or background sound, but this is just an example. It's also possible to assign multiple sub-sounds of the test sound and / or background sound to different paper materials (e.g., the first part corresponds to sketchbook paper, and the second part to traditional Korean paper). In this case, it might be necessary to select different paper materials for each sub-object associated with the user input. Therefore, the attributes and / or corresponding background of each sub-object can be represented distinctly.

[0178] Figure 13e This is a schematic diagram illustrating a screen for providing auditory training content according to one embodiment.

[0179] According to one embodiment, the electronic device 100 can be as follows: Figure 13eA background sound list 1360 is provided, and the triggering conditions for providing this list are not limited. The background sound list 1360 may include multiple objects 1361, 1362, 1363, and 1364. For example, each object may be represented by text indicating the name of the background sound, but this is merely an example and not a limitation. The electronic device 100 can confirm the selection of any object in the background sound list 1360. Based on the selected object, the electronic device 100 can configure the attributes of the background sound. For example, if object 1361 corresponding to "street" is selected, the electronic device 100 can set the street atmosphere as the background sound. User selection of background sounds is merely an example. The electronic device 100 can also select background sounds based on predefined rules (such as difficulty level, user training level, and / or user weaknesses) without user input. When a background sound is selected based on this rule, the electronic device 100 can provide a background sound with attributes corresponding to the selected background sound. In another example, the electronic device 100 can provide a background sound with predefined attributes and prompt the user to select the attributes of the background sound through a user interface. For example, the user can listen to an unknown background sound combined with a test sound and use, for example... Figure 13e The user selects the corresponding background sound attribute from the displayed list of background sounds and then enters user input. The electronic device 100 can display the user-selected object and / or provide background sounds with the selected attributes, as well as sounds based on the user input. The user can determine whether the attributes of their selected background sound match the attributes of the background sounds provided with the test sounds. The user can submit a response or modify their selection by reselecting a background sound object. For example, a background sound can be assigned a single attribute, but this is just an example. It is also possible for multiple sub-sounds of a background sound to correspond to different attributes (e.g., the first part corresponds to "street," and the second part corresponds to "hall"). In this case, different sub-objects corresponding to the user input may need to select different background sounds. Furthermore, different areas can be represented by different background images corresponding to different background sounds (e.g., the first part is a street image, and the second part is a hall image). If the auditory training content is presented in a VR environment, visual content can be provided to represent the virtual reality corresponding to the selected background sound.

[0180] Figure 13f This is a schematic diagram illustrating a screen for providing auditory training content according to one embodiment.

[0181] According to one embodiment, the electronic device 100 can be as follows: Figure 13fA list 1370 of writing tool thicknesses is provided, and the triggering conditions for displaying this list are not limited. The writing tool thickness list 1370 may include multiple objects 1371, 1372, 1373, and 1374. For example, each object may represent text indicating the thickness of the writing tool and / or an example of an object attribute corresponding to that thickness. However, this representation is merely illustrative and not restrictive. The electronic device 100 can recognize the selection of any object in the writing tool thickness list 1370. Based on the selected object, the electronic device 100 can configure the attributes of a test sound. For example, if object 1372 corresponding to "thickness 2" is selected, the electronic device 100 can set the volume, as one of the test sound attributes, to the volume corresponding to object 1372. The electronic device 100 can also represent objects displaying the trajectory of the selected thickness based on user input. The user selection of the writing tool thickness is merely an example. The electronic device 100 can select the thickness based on predefined rules (e.g., difficulty level, user training level, and / or user weaknesses) without user input. If the thickness of the writing instrument is selected based on this rule, the electronic device 100 can provide a test sound with attributes corresponding to the selected thickness. Furthermore, the electronic device 100 can represent an object with the selected thickness based on user input. In another example, the electronic device 100 can provide a test sound with predefined attributes (e.g., volume) and prompt the user to select the corresponding attribute via a user interface. For example, the user can listen to a test sound with unknown attributes, select the corresponding thickness using the writing instrument thickness list 1370, and then provide user input. The electronic device 100 can display an object representing the selected thickness and / or provide a sound with the selected thickness attribute. The user can determine whether the sound attribute corresponding to the selected thickness matches the test sound attribute. Subsequently, the user can submit their response or modify their selection, for example, by reselecting the thickness from the writing instrument list. Setting the thickness via the writing instrument thickness list 1370 is merely an example. The electronic device 101 can also set the thickness of an object based on factors such as pressure intensity and / or touch duration (e.g., proportionality). For example, a single test sound can be assigned a thickness, but this is merely an example. It's also possible to assign different thicknesses to multiple sub-voices within a test sound (e.g., the first part corresponds to "thickness 2", the second part to "thickness 5"). In this case, it might be necessary to select different thicknesses for each sub-object corresponding to the user input. Therefore, the attributes of each sub-object (e.g., thickness) can be represented distinctly.

[0182] Figure 14a This is a schematic diagram illustrating a method for providing auditory training content according to one embodiment, which will combine... Figure 14b To explain, Figure 14b This is a schematic diagram illustrating a resonance peak-based analysis according to one embodiment.

[0183] According to one embodiment, electronic device 100 may provide at least one training procedure based on non-linguistic elements during operation 1401. For example, as shown in reference... Figures 5a to 5i As explained, electronic device 100 can provide a training program based on matching auditory and visual patterns, as at least one training program based on non-verbal elements. Furthermore, electronic device 100 can also provide a training program based on matching auditory and visual patterns, which combines... Figure 18a To be further described in detail with reference to Figure 18c, as at least one training procedure based on non-linguistic elements. In operation 1403, after providing at least one training procedure based on non-linguistic elements, electronic device 100 can provide at least one training procedure based on linguistic elements. The linguistic element-based training procedure can be associated with phonemes defined by language, which will combine Figure 15 a to Figure 17 To elaborate further. It should be noted that providing a training program based on non-linguistic elements before a training program based on linguistic elements is merely an example; the order can vary. Furthermore, at least a portion of a training program based on phonemes defined by language can be included within a training program based on non-linguistic elements. For example, as... Figure 14b As shown, the pronunciation of a language-defined phoneme can be located on a coordinate plane based on the first and second formant frequencies. Therefore, a higher number of incorrect responses in a test targeting a specific phoneme can indicate a lower level of proficiency for the user within that frequency range. Based on this information, the electronic device 100 can set up additional training programs designed to address the weaknesses identified within that frequency range.

[0184] Figure 15 The auditory training content according to one embodiment is shown.

[0185] According to one embodiment, the electronic device 100 may provide screens 1511, 1512, 1513, and 1514 related to phoneme training information. Screen 1511 may indicate the start of training for a new phoneme. Screen 1512 may include information about a target vowel (e.g., the Korean vowel "..."). As shown on screens 1512, 1513, and 1514, phoneme training can be configured to perform phoneme training in a partial manner (e.g., divided into first and second halves, but not limited to) or in an integrated manner based on a target vowel (e.g., 14 phonemes). For example, when training start object 1501 is selected within screens 1512, 1513, and 1514, electronic device 100 can provide a first self-evaluation training screen. In one example, electronic device 100 can select a target vowel and determine the training phoneme by combining the selected vowel with multiple consonants. The target vowel can be selected based on the user's training performance history, cumulative training count, or test difficulty, but the selection criteria are not limited to these. In another example, electronic device 100 can be implemented to select a target consonant and determine the training phoneme by combining it with multiple vowels.

[0186] Figure 16a and Figure 16b This is a schematic diagram illustrating auditory training content according to one embodiment.

[0187] According to one embodiment, the electronic device 100 can be based on Figure 15 The selection of training start object 1501 in the middle provides Figure 16a and Figure 16b At least a portion of screens 1621, 1622, 1623, 1631, 1632, and 1633 shown. For example, as Figure 16a and Figure 16b As shown, a third playback object 1602a can be provided for requesting playback of a question (e.g., one of the syllables composed of consonants and vowels to be trained), a first navigation object 1602b for moving to the previous question, a second navigation object 1602c for moving to the next question, and a registration object 1602d for registering questions of interest. For example, the first self-assessment training screen may include a counting display area 1602e for displaying the number of times each question has been listened to, and a syllable display area 1602f for displaying multiple selectable syllables.

[0188] For example, training can be conducted by the user listening to a provided question and selecting one of several syllables displayed in the syllable display area 1602f, but is not limited to this. For example, when a third playback object 1602a is selected, the electronic device 100 can provide the corresponding syllable through the audio output device. Referring to screen 1622, it can be confirmed that the user has selected a syllable displayed in the syllable display area 1602f. Referring to screens 1622 and 1623, the number of listens in the count display area 1602e can be increased each time the third playback object 1602a is selected.

[0189] For example, referring to screens 1631 and 1632, when an incorrect answer is entered, the error can be visually indicated (e.g., by changing the background to a first color (e.g., red) and / or displaying a specified symbol (e.g., "x") in the first color on one side). For example, referring to screen 1633, when the correct answer is selected, the correct answer can be visually indicated (e.g., by changing the background to a second color (e.g., green), displaying a specified symbol (e.g., "√") in the second color on one side, and / or replacing the third playback object 1602a with the correct syllable).

[0190] For example, after training has reached the last question, when the second moving object 1602c is selected to request the next question, a first self-diagnostic results screen can be provided. Figure 16c As shown, the first self-diagnosis result screen 1641 may include a result display area 1604a, which displays a pair of syllables (or characters) corresponding to the question and syllables selected by the user (hereinafter referred to as phoneme pairs). For example, the syllable corresponding to the question may be displayed on the left, while the syllable selected by the user may be displayed on the right, thus displaying a phoneme pair. For example, when an incorrect answer is confirmed, other syllables may be displayed with different background colors.

[0191] For example, the first self-diagnosis results screen 1641 may include phoneme pair training objects 1604b and labeled training objects 1604c for incorrect questions. In this case, labeled training objects 1604c can be displayed in a deactivated state and activated only after the phoneme pair training is complete. For example, if there were no incorrect answers in the previous training, phoneme pair training objects 1604b can be deactivated (or not displayed), and only labeled training objects 1604c are activated.

[0192] For example, when phoneme pair training object 1604b is selected, a phoneme pair training screen can be provided. Phoneme pair training involves repeatedly listening to incorrect phoneme pairs to help the user identify their differences. For example, when phoneme pair training object 1604b is selected, such as... Figure 16d As shown in screen 1651, a phoneme pair training screen can be provided. The phoneme pair training screen may include a phoneme pair display area 1605a for providing information about the phoneme pairs to be trained, a playback control area 1605b for controlling the playback speed, a sequential random object 1605c for randomly changing the order of the phoneme pairs, a hidden object 1605d, and / or a training end object 1605e.

[0193] After the specified time has elapsed, such as Figure 16dAs shown on screen 1652, the phoneme pairs to be trained can be hidden (e.g., displayed as question marks), and the order of the phoneme pairs can be randomly changed to alternately provide the sounds corresponding to each phoneme. At this time, the hidden object 1605d can be replaced by a character display object 1605f, which is used to display the hidden characters. After fully listening to the alternately provided phoneme pairs, the user can select the character display object 1605f to confirm whether they have accurately perceived the sounds. At this time, the character display object 1605f can be replaced back to the hidden object 1605d.

[0194] For example, when phoneme pair training is complete, annotation training object 1604c can be activated. An annotation training screen can be provided when annotation training object 1604c is selected. Annotation training can include articulation training and self-diagnosis training. Articulation training involves repeatedly speaking (or pronouncing) and listening (or hearing). For example, as... Figure 16e As shown on screen 1661, the syllables to be trained (e.g., the Korean word "...") The speaker icon 1606b (pronounced "Hu") can be displayed on one side, and the speaker icon 1606b and listener icon 1606c can be activated alternately. When the listener icon 1606c is activated, the corresponding syllable's sound can be provided. According to some embodiments, when the speaker icon 1606b is activated, the user's voice can also be recorded.

[0195] For example, after a specified number of repetitions (e.g., two) of articulation training, as shown on screen 1662, it is possible to provide guidance for the next syllable (e.g., the Korean word "..."). The training screen is 1606d, which is pronounced "Pu". Once all training is complete, as shown on screen 1663, the articulation training results screen can be displayed. The articulation training results screen may include a relearning object 1606e for retraining articulation and / or a next training object 1606e for ending articulation training and proceeding to the next training session.

[0196] For example, when selecting the next training object 1606e, self-assessment training for annotation training can be provided. For example, as Figure 16f As shown in 1671, a screen can be provided for selecting a training mode for self-assessment training in labeled training. Self-assessment training in labeled training can include a target time training mode 1607a (training within a target time) and a target score training mode 1607b (training until a target score is reached). Once the user selects either mode and sets the target, self-assessment training in labeled training can begin. For example, as shown in screens 672-1 and 672-2, target time training mode 1607a can be selected, and a target time (e.g., 5 minutes) 1607c can be set. In another example, as... Figure 16gAs shown on screens 1673-1 and 1673-2, a target score training mode 1607b can be selected, and a target score (e.g., 80 points) 1607d can be set. When a target time 1607c or a target score 1607d is set, the training start object 1607e can be activated.

[0197] Once you set the target time (1607c) or target score (1607d) and select the training start object (1607e), as follows: Figure 16h As shown, a self-assessment training screen 1681 for annotation training can be provided. The self-assessment training screen 1681 for annotation training may include a fourth playback object 1608a, an answer viewing object 1608b, a correct answer object 1608c, an incorrect answer object 1608d, and / or a training end object 1608e. When the fourth playback object 1608a is selected, a sound corresponding to the trained syllable can be provided. The user can estimate the syllables of the sound heard. When the answer viewing object 1608b is selected, as shown on screen 1682, the fourth playback object 1608a can be changed (or replaced) with the syllable (or character) corresponding to that sound. For example, the electronic device 100 can activate the previously deactivated correct answer object 1608c and incorrect answer object 1608d so that the user can select them.

[0198] The user can compare their estimated syllables with the correct answer and choose between the correct answer object 1608c and the incorrect answer object 1608d. For example, if the user's estimated syllables match the correct answer, they can choose the correct answer object 1608c; if they do not match, they can choose the incorrect answer object 1608d. Once either the correct answer object 1608c or the incorrect answer object 1608d is selected, the next question screen is provided. Specifically, if the correct answer object 1608c is selected, as shown in screen 1683, the corresponding sound can be provided once, and a visual notification indicating correctness (e.g., the border of the area displaying the syllable changes to a first color) is displayed before the next question screen is provided. Conversely, if the incorrect answer object 1608d is selected, the corresponding sound can be provided a specified number of times (e.g., three times), and a visual notification indicating error (e.g., the border of the area displaying the syllable flashes in a second color) is displayed before the next question screen is provided.

[0199] If the target time 1607c or target score 1607d is reached, the electronic device 100 can activate the previously deactivated training termination object 1608e (allowing the user to select this object). Even if the target time 1607c or target score 1607d is reached, the user can choose not to select the training termination object 1608e and continue with additional training. Meanwhile, when the training termination object 1608e is selected, the electronic device 100 can provide, for example... Figure 16i The results screen 1691 is shown. The results screen 1691 may include a results display area 1609a, which displays trained phoneme pairs, similar to... Figure 16c The first self-diagnosis results screen 1641. However, since both phoneme pair training and annotation training have been completed, the results screen 1691 may include the next training object 1609b (e.g., for second self-diagnosis training) instead of phoneme pair training object 1604b and annotation training object 1604c.

[0200] When the next training object 1609b is selected, the electronic device 100 can provide a second self-diagnostic training. According to some embodiments, when the next training object 1609b is selected, the electronic device 100 can display the main screen. The electronic device 100 can display the object corresponding to the completed training as active.

[0201] Reference Figure 17 When the second self-diagnostic training is complete, the electronic device 100 can provide (or return to) the sub-homepage screen 1711. Figure 17 On the sub-homepage screen 1711, all objects corresponding to those that have completed training can be displayed in an active state. Furthermore, Figure 17 The sub-home screen 1711 may also include a results viewing object 1701. The results viewing object 1701 may be displayed based on the completion of all training as indicated on the sub-home screen 1711.

[0202] When the result viewing object 1701 is selected, the electronic device 100 can provide a first result screen 1712. The first result screen 1712 may include the phoneme information being trained 1702, a first self-diagnostic training result 1703, a second self-diagnostic training result 1704, and a next object 1705 for requesting the second result screen. The first result screen may, for example, use a bar chart to display the first self-diagnostic training result 1703 and the second self-diagnostic training result 1704, but is not limited to this.

[0203] When the next object 1705 is selected, the electronic device 100 can provide a second results screen 1713. The second results screen 1713 may include the accuracy of the first self-diagnostic training 1706a, the accuracy of the second self-diagnostic training 1706b, the phoneme pairs trained in the first self-diagnostic training 1707a, the phoneme pairs trained in the second self-diagnostic training 1707b, and a homepage object 1708 for returning to the main screen.

[0204] Figure 18a This is a schematic diagram illustrating auditory training content according to one embodiment.

[0205] Reference Figure 18a The electronic device 100 may provide screens 1811, 1812, and 1813 for audio-visual appreciation training. The content provided for audio-visual appreciation training may include: providing sounds with auditory patterns, wherein the characteristics corresponding to the auditory patterns (e.g., frequency, timbre, etc., but not limited thereto) change or remain over time; and providing objects with visual patterns, wherein the visual patterns visualize the auditory patterns of the characteristics of the output sounds.

[0206] For example, when requesting audio-visual appreciation training, electronic device 100 can provide an audio-visual appreciation training screen 1811. The audio-visual appreciation training screen 1811 may include a visual area 1801 for displaying visual objects, a fifth playback object 1802 for playing a specified training sound, a repeating object 1803 for replaying the training sound, and an appreciation end object 1804 for ending the audio-visual appreciation training.

[0207] When the fifth playback object 1802 is selected, the electronic device 100 can provide (e.g., play or output) the specified training sound. The screen 1812 may include a visual object 1805 corresponding to the specified training sound, displayed in the visual area 1801. At this time, the fifth playback object 1802 can become a stop object 1806. The presentation of the visual object 1805 can be substantially synchronized with the provision of the training sound. For example, as the training sound is provided over time, an animation effect can be applied to make the visual object 1805 gradually appear. However, those skilled in the art will understand that the synchronization method is not limited to this.

[0208] When the training audio playback is complete, the electronic device 100 can activate the end-of-play object 1804. At this time, the stop object 1806 can be changed (or restored) to the fifth playback object 1802. Subsequently, the user can choose the end-of-play object 1804 to end the audio-visual appreciation training, choose the fifth playback object 1802 to replay the audio-visual appreciation training, or choose the repeat object 1803 to repeat the training a specified number of times (e.g., 5 times) or until the stop object 1806 is selected.

[0209] While listening to sounds with auditory patterns, users can also view objects with visual patterns, enabling them to visually understand auditory patterns and map them to their auditory results. This allows users' hearing to be effectively trained.

[0210] Figure 18b The auditory training content according to one embodiment is shown.

[0211] According to one embodiment, electronic device 100 can provide specified effects and / or background sounds simultaneously with the training sounds. For example, electronic device 100 can display an object 1851 having a visual pattern corresponding to the training sounds, substantially synchronized with the training sounds. Electronic device 100 can display an object 1852 corresponding to a specified effect (e.g., a diffusion effect). Although not shown, for intermittent pauses during playback, electronic device 100 can display an object corresponding to the background sounds. By examining not only the training sounds but also the visual objects corresponding to the applied effects and / or background sounds, users can perform auditory training of increasing difficulty.

[0212] Figure 19 The illustration shows the changes in auditory assessment resulting from the execution of auditory training content, according to one embodiment.

[0213] Reference Figure 19 The effectiveness of implementing the auditory training content according to this embodiment can be examined.

[0214] 1. Experimental Design

[0215] Eight adult participants who underwent cochlear implantation surgery were selected. The participants were instructed to 1) undergo an auditory perception assessment before undertaking the auditory training content according to this embodiment, and 2) undergo an auditory perception assessment again after completing the auditory training content, and the results were compared accordingly.

[0216] Specifically, the procedure proceeds in the following order: 1) pre-evaluation, 2) execution of the content based on this embodiment, and 3) post-evaluation. The execution of the content based on this embodiment lasts for 30 to 50 minutes.

[0217] 2. Evaluation Methods

[0218] A test was conducted to distinguish Korean consonants and vowels. Participants listened to consonants such as ga, na, da, ra, ..., ka, and kka, and the accuracy of their recognition results was evaluated.

[0219] 3. Results

[0220] Reference Figure 19It can be observed that the average accuracy rate was 48% before using the content, while the average accuracy rate improved to 73% after using the content. In addition, significant improvements in accuracy were observed in 7 out of 8 participants, except for one.

[0221] The embodiments described above in conjunction with the present invention are merely examples; various modifications, alterations, and equivalent embodiments can be made by those skilled in the art to which this invention pertains without departing from the spirit and scope of the invention. Therefore, the true scope of protection of this invention should be determined by the technical spirit of the appended claims.

Claims

1. A method of auditory training, comprising: providing a test sound having at least one characteristic of a first auditory pattern for a sound; based on a user input inputted through a user interface for receiving a visual pattern, providing, through the user interface, a first visual object having a first visual pattern corresponding to the user input, wherein a first sound is provided, the first sound being substantially synchronized with a detection time point of the user input and having at least one characteristic identified based on a position of the user input defined in the user interface; and based on an event identified for providing a second visual pattern corresponding to the test sound, providing a second visual object having the second visual pattern, the second visual pattern corresponding to the at least one characteristic of the first auditory pattern.

2. The method of claim 1, characterized in that providing the first visual object through the user interface comprises: based on detecting at least a portion of the user input associated with a first position and a second position of the user interface, providing at least a portion of the first visual object associated with the first position, the second position and at least one intermediate position between the first position and the second position in the user interface.

3. The method of claim 2, characterized in that based on at least a portion of the user input being associated with the first position, providing a first portion of the first sound having at least one characteristic corresponding to the first position; based on at least a portion of the user input being associated with each of the at least one intermediate position, providing at least one intermediate portion of the first sound having at least one characteristic corresponding to each of the intermediate positions; and based on at least a portion of the user input being associated with the second position, providing a second portion of the first sound having at least one characteristic corresponding to the second position.

4. The method of claim 2, characterized in that at least a portion of the user input comprises an input for specifying the first position and the second position, and / or an input for specifying the first position, the at least one intermediate position and the second position.

5. The method of claim 1, characterized in that the test sound comprises a plurality of portions provided sequentially in time, and each of the plurality of portions has at least one characteristic varying over time or remaining constant over time according to the first auditory pattern.

6. The method of claim 1, further comprising: providing a result of comparing the first visual object and the second visual object.

7. The method of claim 6, wherein providing the comparison result comprises: providing information about a weakness of a user identified based on the comparison result.

8. The method of claim 1, characterized in that the event for providing the second visual pattern corresponding to the test sound comprises: an availability of a correct visual object, a completion of providing the first visual object, and / or an elapse of a specified time.

9. The method of claim 1, further comprising: After providing the first visual object and before providing the second visual object, performing a modification of at least a portion of the first visual object based on another user input for modifying the first visual object.

10. The method of claim 9, wherein performing the modification of at least a portion of the first visual object comprises: identifying a deletion command for a first portion of the first visual object; and while maintaining display of a remaining portion of the first visual object other than the first portion, deleting the first portion associated with the deletion command.

11. The method of claim 1, each of the at least one characteristic of at least a portion of the test sound has a single value at a particular point in time. characterized in that 12. The method of claim 1, each of the at least one characteristic of at least a portion of the test sound has a plurality of values at a particular point in time. characterized in that 13. The method of claim 1, the user interface comprises a plurality of reference objects for association with the user input. characterized in that 14. The method of claim 13, the plurality of reference objects are arranged in a grid pattern, characterized in that and the user input comprises an input connecting one of a plurality of first reference objects contained in a column to one of a plurality of second reference objects contained in an adjacent column.

15. The method of claim 13, based on two or more of the reference objects being associated with a first temporary user input, providing as the first visual object a visual object associated with the two or more reference objects; characterized in that and based on two or more of the reference objects not being associated with a second temporary user input, ceasing display of a visual object temporarily provided based on a trajectory of the second temporary user input.

16. The method of claim 13, a number, density, and / or arrangement of the plurality of reference objects is set based on user selection and / or test difficulty. characterized in that 17. The method of claim 1, the first auditory pattern of the test sound is represented based on at least one quantifiable characteristic, characterized in that and a lower limit value of the test sound, an upper limit value of the test sound, and / or a difference between the upper limit value and the lower limit value is set based on user selection and / or test difficulty.

18. The method of claim 1, applying an audio effect identified based on user selection and / or test difficulty to at least a portion of the test sound. characterized in that 19. The method of claim 1, further comprising: during provision of the test sound, providing a background sound identified based on user selection and / or test difficulty along with at least a portion of the test sound.

20. The method of claim 1, further comprising, prior to providing the test sound: setting at least one different characteristic of a sound other than the at least one characteristic of the first auditory pattern based on user selection and / or test difficulty.

21. The method of claim 1, further comprising: while providing the test sound, providing at least one indicator for visually cueing a playback point of the test sound over time. ​ 22. An auditory training system, comprising: a server; and an electronic device comprising at least one processor, wherein the server is configured to provide instructions to the electronic device based on a connection of the electronic device to the server and / or a request to the server, and wherein the instructions, when executed based on at least a portion of the at least one processor of the electronic device, cause the electronic device to perform at least one operation, and the at least one operation comprises: providing a test sound having at least one characteristic for a first auditory pattern of sound; based on a user input entered through a user interface for receiving a visual pattern, providing, through the user interface, a first visual object having a first visual pattern corresponding to the user input, wherein the first sound is provided substantially synchronously with a detected time point of the user input and has at least one characteristic identified based on a position of the user input defined in the user interface; and based on identifying an event for providing a second visual pattern corresponding to the test sound, providing a second visual object having the second visual pattern corresponding to the at least one characteristic of the first auditory pattern of the test sound.

23. The system of claim 22, characterized in that the operation of providing the first visual object through the user interface comprises: based on detecting at least a portion of the user input associated with a first position and a second position of the user interface, providing at least a portion of the first visual object associated with the first position, the second position, and at least one intermediate position between the first position and the second position in the user interface.

24. The system of claim 23, characterized in that based on at least a portion of the user input being associated with the first position, providing a first portion of the first sound having at least one characteristic corresponding to the first position; based on at least a portion of the user input being associated with each of the at least one intermediate position, providing at least one intermediate portion of the first sound having at least one characteristic corresponding to each of the intermediate positions; and based on at least a portion of the user input being associated with the second position, providing a second portion of the first sound having at least one characteristic corresponding to the second position.

25. The system of claim 23, characterized in that at least a portion of the user input comprises an input for specifying the first position and the second position, and / or an input for specifying the first position, the at least one intermediate position, and the second position.

26. An auditory training method performed by a system comprising a server and an electronic device, comprising: by the server, providing instructions to the electronic device based on a connection of the electronic device to the server and / or a request to the server; and by the electronic device, executing the instructions, wherein the instructions, when executed based on at least a portion of at least one processor of the electronic device, cause the electronic device to perform at least one operation, and the at least one operation comprises: providing a test sound having a first auditory pattern of at least one characteristic for a sound; based on a user input entered through a user interface for receiving a visual pattern, providing, through the user interface, a first visual object having a first visual pattern corresponding to the user input, wherein a first sound is provided, the first sound being substantially synchronized with a detection time point of the user input and having at least one characteristic identified based on a position of the user input defined in the user interface; and based on identifying an event for providing a second visual pattern corresponding to the test sound, providing a second visual object having the second visual pattern, the second visual pattern corresponding to the at least one characteristic of the first auditory pattern of the test sound.

27. The method of claim 26, characterized in that providing the first visual object through the user interface comprises: based on detecting at least a portion of the user input associated with a first position and a second position of the user interface, providing at least a portion of the first visual object associated with the first position, the second position, and at least one intermediate position between the first position and the second position in the user interface.

28. The method of claim 27, characterized in that based on at least a portion of the user input being associated with the first position, providing a first portion of the first sound having at least one characteristic corresponding to the first position; based on at least a portion of the user input being associated with each of the at least one intermediate position, providing at least one intermediate portion of the first sound having at least one characteristic corresponding to each of the intermediate positions; and based on at least a portion of the user input being associated with the second position, providing a second portion of the first sound having at least one characteristic corresponding to the second position.

29. The method of claim 27, characterized in that at least a portion of the user input comprises an input for specifying the first position and the second position, and / or an input for specifying the first position, the at least one intermediate position, and the second position.

30. An auditory training method performed by a system comprising a server and an electronic device, comprising: providing, by a server comprising at least one first processor, an instruction to an electronic device based on a connection of the electronic device to the server and / or a request to the server, wherein the server is configured to provide the instruction; and executing, by the electronic device, the instruction, wherein the instruction, when executed based on at least a portion of at least one processor of the electronic device, causes the electronic device to perform at least one operation, and the at least one operation comprises: providing a test sound having a first auditory pattern of at least one characteristic for a sound; Based on user input via a user interface for receiving visual patterns, a first visual object having a first visual pattern corresponding to the user input is provided through the user interface, wherein a first sound is provided, the first sound being substantially synchronized with the detection time point of the user input, and having at least one characteristic identified based on the position of the user input defined in the user interface; and Based on the identification of an event for providing a second visual pattern corresponding to the test sound, a second visual object is provided having the second visual pattern, the second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound.

31. The method of claim 30, characterized in that Providing the first visual object through the user interface includes: Based on detecting at least a portion of the user input associated with a first position and a second position of the user interface, at least a portion of the first visual object associated with the first position, the second position, and at least one intermediate position between the first position and the second position in the user interface is provided.

32. The method as described in claim 31, characterized in that Based on the association of at least a portion of the user input with the first location, a first portion of the first sound is provided having at least one characteristic corresponding to the first location; Based on the association of at least a portion of the user input with each of the at least one intermediate position, at least one intermediate portion of the first sound is provided having at least one characteristic corresponding to each of the intermediate positions; Furthermore, based on at least a portion of the user input being associated with the second location, a second portion of the first sound is provided having at least one characteristic corresponding to the second location.

33. The method as described in claim 31, characterized in that At least a portion of the user input includes input for specifying the first position and the second position, and / or input for specifying the first position, the at least one intermediate position, and the second position.

34. A storage medium for storing computer-readable instructions, wherein, When the instruction is executed by at least one processor of the electronic device, it causes the electronic device to perform at least one operation. And the at least one operation includes: Provide a test sound with a first auditory pattern having at least one characteristic of sound; Based on user input via a user interface for receiving visual patterns, a first visual object having a first visual pattern corresponding to the user input is provided through the user interface, wherein a first sound is provided, the first sound being substantially synchronized with the detection time point of the user input, and having at least one characteristic identified based on the position of the user input defined in the user interface; and Based on the identification of an event for providing a second visual pattern corresponding to the test sound, a second visual object is provided having the second visual pattern, the second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound.

35. The storage medium as described in claim 32, characterized in that Providing the first visual object through the user interface includes: Based on detecting at least a portion of the user input associated with a first position and a second position of the user interface, at least a portion of the first visual object associated with the first position, the second position, and at least one intermediate position between the first position and the second position in the user interface is provided.

36. The storage medium as described in claim 35, characterized in that Based on the association of at least a portion of the user input with the first location, a first portion of the first sound is provided having at least one characteristic corresponding to the first location; Based on the association of at least a portion of the user input with each of the at least one intermediate position, at least one intermediate portion of the first sound is provided having at least one characteristic corresponding to each of the intermediate positions; Furthermore, based on at least a portion of the user input being associated with the second location, a second portion of the first sound is provided having at least one characteristic corresponding to the second location.

37. The storage medium as described in claim 35, characterized in that At least a portion of the user input includes input for specifying the first position and the second position, and / or input for specifying the first position, the at least one intermediate position, and the second position.

38. An electronic device comprising: At least one processor; as well as Memory for storing instructions Wherein, when the instruction is executed based on at least a portion of the at least one processor, the electronic device performs at least one operation. And the at least one operation includes: Provide a test sound with a first auditory pattern having at least one characteristic of sound; Based on user input via a user interface for receiving visual patterns, a first visual object having a first visual pattern corresponding to the user input is provided through the user interface, wherein a first sound is provided, the first sound being substantially synchronized with the detection time point of the user input, and having at least one characteristic identified based on the position of the user input defined in the user interface; and Based on the identification of an event for providing a second visual pattern corresponding to the test sound, a second visual object is provided having the second visual pattern, the second visual pattern corresponding to at least one characteristic of the first auditory pattern of the test sound.

39. The electronic device as claimed in claim 38, characterized in that Providing the first visual object through the user interface includes: Based on detecting at least a portion of the user input associated with a first position and a second position of the user interface, at least a portion of the first visual object associated with the first position, the second position, and at least one intermediate position between the first position and the second position in the user interface is provided.

40. The electronic device as claimed in claim 39, characterized in that Based on the association of at least a portion of the user input with the first location, a first portion of the first sound is provided having at least one characteristic corresponding to the first location; Based on the association of at least a portion of the user input with each of the at least one intermediate position, at least one intermediate portion of the first sound is provided having at least one characteristic corresponding to each of the intermediate positions; Furthermore, based on at least a portion of the user input being associated with the second location, a second portion of the first sound is provided having at least one characteristic corresponding to the second location.

41. The electronic device as claimed in claim 39, characterized in that At least a portion of the user input includes input for specifying the first position and the second position, and / or input for specifying the first position, the at least one intermediate position, and the second position.

Images