Information processing method, information processing system, and information processing program

Abstract

An information processing method according to the present disclosure is implemented by one or more processors, the information processing method including: acquiring action recognition information relating to actions by a user to be subjected to an auditory test; and controlling, on the basis of the acquired action recognition information, a function of the auditory test performed on the user.

Description

Information processing methods, information processing systems, and information processing programs

[0001] This disclosure relates to information processing methods, information processing systems, and information processing programs.

[0002] Technologies for performing auditory function tests such as hearing tests (hereinafter also referred to as "auditory tests") have been disclosed. For example, a system has been disclosed that measures the user's brainwaves using a hearing aid that comes into contact with the user's skin, and performs an auditory test based on the measured brainwaves (for example, Patent Document 1).

[0003] European Patent Application Publication No. 2581038

[0004] However, there is room for improvement in conventional technology. For example, while conventional technology allows users to perform hearing tests in their daily lives, the user's daily life is not always suitable for performing hearing tests, and there is room for improvement in controlling the hearing test function. Therefore, it is desirable to appropriately control the hearing test function according to the target user.

[0005] Therefore, this disclosure proposes an information processing method, an information processing system, and an information processing program that can appropriately control the functions of hearing tests according to the target user.

[0006] To solve the above problems, one form of information processing method relating to this disclosure is an information processing method implemented by one or more processors, which includes acquiring behavioral recognition information relating to the behavior of a user who is the subject of a hearing test, and controlling the function of the hearing test for the user based on the acquired behavioral recognition information.

[0007] This is a diagram showing the schematic configuration of an information processing system according to an embodiment. This is a diagram showing an example configuration of a hearing aid and charger according to an embodiment. This is a diagram showing an example configuration of a terminal device according to an embodiment. This is a flowchart showing the processing procedure executed by the information processing system according to the first embodiment. This is a diagram showing an example of a behavior-sound stimulus mapping dictionary. This is a diagram showing an example of audiogram data. This is a diagram showing an example configuration of an information processing system according to the first embodiment. This is a diagram showing an example configuration of an information processing system according to the second embodiment. This is a diagram showing an example configuration of an information processing system according to the third embodiment. This is a diagram showing an example of a user environment determination dictionary. This is a flowchart showing the processing procedure executed by the information processing system according to the third embodiment. This is a diagram showing an example of a first notification. This is a diagram showing an example of a second notification. This is a diagram showing an example of a third notification. This is a diagram showing another example of a behavior-sound stimulus mapping dictionary. This is a hardware configuration diagram showing an example of a computer that realizes the functions of an information processing device. This is a diagram showing an example configuration of an information processing system according to the fourth embodiment. This is a flowchart showing the processing procedure executed by the information processing system according to the fourth embodiment. This is a diagram showing an example of acquiring unpleasant sound pressure. This is a diagram showing an example of acquiring unpleasant sound pressure. This is a diagram showing an example of processing to counter unpleasant sound pressure. This is a diagram showing an example configuration of an information processing system according to the fifth embodiment. This is a flowchart showing the processing procedure executed by the information processing system according to the fifth embodiment. This figure shows an example of headphones equipped with an electroencephalogram (EEG) sensor. This figure shows an example of parameter control. This figure shows an example of headphones equipped with an EEG sensor. This figure shows an example of parameter control. This figure shows an example of behavior measurement-compatible information. This flowchart shows the information processing procedure according to the sixth embodiment. This figure shows an example of noise reduction processing.

[0008] Embodiments of this disclosure will be described in detail below with reference to the drawings. Note that these embodiments do not limit the information processing method, information processing system, and information processing program described herein. Furthermore, in each of the following embodiments, the same parts are denoted by the same reference numerals to avoid redundant explanations.

[0009] This disclosure will be described in the following order of items. 1. Embodiments 1-1. First Embodiment 1-1-1. Overview of the Information Processing System 1-1-2. Overview of Information Processing 1-1-3. Example of Information Processing System Configuration 1-2. Second Embodiment (Function Distribution) 1-2-1. Example of Information Processing System Configuration 1-3. Third Embodiment (User Environment Dictionary) 1-3-1. Example of Information Processing System Configuration 1-3-2. Overview of Information Processing 1-4. Notification Examples 1-4-1. First Notification Example (Sound Output Notification) 1-4-2. Second Notification Example (User Prompt Notification) 1-4-3. Third Notification Example (Progress Notification) 2. Others 2-1. Other Behavior-Sound Stimulus Correspondence Dictionary Examples 2-2. Summary 3. Hardware Configuration 4. Other Embodiments 4-1. Fourth Embodiment (Measures against Unpleasant Sound Pressure) 4-1-1. 4-1-2. Example of acquiring unpleasant sound pressure. 4-2. Example of processing when unpleasant sound pressure is exceeded. 5th embodiment (measurement parameter control). 4-2-1. Configuration and processing example of parameter control. 4-3. 6th embodiment (noise reduction processing).

[0010] <1. Embodiments> The information processing system related to this disclosure and the information processing performed by the information processing system will be described below in each embodiment, such as the first to third embodiments. In the following, ASSR (Auditory Steady-State Response) will be described as an example of an auditory test, but the auditory test may be any type of auditory test, not limited to ASSR. The information processing system related to this disclosure may be applied to auditory tests other than ASSR, such as ABR (Auditory Brainstem Response).

[0011] <1-1. First Embodiment> <1-1-1. Overview of the Information Processing System> Referring to Figures 1 to 3, an overview of the information processing system 1, which is an example of the information processing system according to the embodiment, will be described. Figure 1 is a diagram showing the schematic configuration of the information processing system according to the embodiment. Figure 2 is a diagram showing an example configuration of a hearing aid and charger according to the embodiment. Figure 3 is a diagram showing an example configuration of a terminal device according to the embodiment.

[0012] The following description of the information processing system 1 will use the example of a binaural hearing aid 2, which is worn on both the left and right ears. However, the information processing device is not limited to this example; it may be a uni-ear type worn on either the left or right ear, or it may be an earphone or headphones. Furthermore, the information processing device may be any device that can determine whether or not to perform a hearing test. In addition, the configuration of the information processing system is not limited to the configuration shown in information processing system 1, but may be any configuration. For example, in the configuration of the information processing system, the device that outputs sound for the hearing test and the device that determines whether or not to perform a hearing test may be separate devices, but details on this point will be described later.

[0013] In Figure 1, the information processing system 1 according to the embodiment includes a pair of hearing aids 2 (left and right), a charger 3 that houses and charges the hearing aids 2, and a terminal device 40 such as a smartphone that can communicate with at least one of the hearing aids 2 and the charger 3.

[0014] The hearing aid 2 is worn in the user's ear. In Figure 2, the hearing aid 2 comprises a sound collection unit 20, a signal processing unit 21, an output unit 22, a battery 23, a connection unit 24, a communication unit 25, a storage unit 26, a control unit 27, a communication unit 28, and a sensor unit 29.

[0015] The sound collection unit 20 includes an external sound collection unit 20f that acquires external acoustic signals, which are acoustic signals originating from sounds propagating outside the user's ear canal, and an internal sound collection unit 20b that acquires internal acoustic signals.

[0016] The outer sound collection unit 20f and the inner sound collection unit 20b each include a microphone 201 and an A / D (analog / digital) conversion unit 202. The microphone 201 collects sound from the outside or inside of the ear canal, generates an analog sound signal from the sound, and outputs it to the A / D conversion unit 202. The A / D conversion unit 202 performs a digital conversion process on the analog sound signal input from the microphone 201 to obtain a digitized sound signal, which is the outer sound signal or the inner sound signal, and outputs the outer sound signal or the inner sound signal to the signal processing unit 21.

[0017] The "outer acoustic signal" is not limited to the acoustic signal input to the microphone 201f outside the user's external auditory canal, and includes, for example, an acoustic signal obtained by performing predetermined processing on the outer acoustic signal. The "inner acoustic signal" is not limited to the acoustic signal input to the microphone 201b inside the user's external auditory canal, and includes, for example, an acoustic signal obtained by performing predetermined processing on the inner acoustic signal.

[0018] The signal processing unit 21 is composed of a memory and a processor having hardware such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor). The signal processing unit 21 performs signal processing on the digital acoustic signal input from the sound collection unit 20, and outputs the processed signal to the output unit 22.

[0019] The signal processing includes filtering processing for separating acoustic signals into frequencies, howling cancel processing for suppressing howling, voice enhancement processing for emphasizing the acoustic signal that the user wants to hear, and hearing correction processing for adjusting the volume of the outer acoustic signal based on pre-measured hearing characteristics of the user, etc.

[0020] The "per frequency" mentioned above and below is not limited to meaning every 1 Hz, but includes the meaning of every predetermined spaced frequency such as 500 Hz, 1000 Hz, 2000 Hz, or every predetermined frequency band such as 500 Hz to 1000 Hz, 1000 Hz to 2000 Hz.

[0021] The output unit 22 includes a D / A (digital / analog) conversion unit 221 and a receiver 222. The D / A conversion unit 221 performs analog conversion processing on the digital acoustic signal input from the signal processing unit 21, and outputs the converted signal to the receiver 222. The receiver 222 is a speaker. The receiver 222 generates an output sound from the analog acoustic signal input from the D / A conversion unit 221, and outputs the output sound.

[0022] The battery 23 is a rechargeable secondary battery such as a lithium-ion battery. The battery 23 supplies power to each part of the hearing aid 2. The battery 23 is charged by power supplied from the charger 3 via the connection unit 24.

[0023] The connection part 24 consists of one or more pins. When the hearing aid 2 is placed in the charger 3, the connection part 24 connects to the connection part 331 of the charger 3, receives power and various information from the charger 3, and outputs various information to the charger 3.

[0024] The communication unit 25 is a communication module. The communication unit 25 communicates with the charger 3 and terminal device 40 via a communication network in accordance with communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0025] The memory unit 26 is a RAM (Random Access Memory), ROM (Read Only Memory), or memory card. The memory unit 26 stores various information related to the hearing aid 2. The memory unit 26 stores the program 261 executed by the hearing aid 2 and data 262 containing various information related to hearing tests. The data 262 includes a behavior-sound stimulus mapping dictionary, audiogram data, etc. In addition, the data 262 may include various other types of information. For example, the data 262 may include hearing test history data. For example, the data 262 may include electroencephalogram-sound stimulus / behavior data. For example, the data 262 may include a user environment determination dictionary. Furthermore, the data 262 may include the user's age, whether or not they have experience using the hearing aid 2, gender, and the duration of hearing aid use.

[0026] The control unit 27 consists of memory and a processor having hardware such as a CPU and DSP. The control unit 27 controls each part of the hearing aid 2. The control unit 27 reads program 261 into the working area of ​​memory and executes it, and controls each part through the execution of program 261 by the processor. Note that the processor referred to in this application does not include a functional representation of an information processing device, but rather a processing circuit as hardware.

[0027] The control unit 27 includes an acquisition unit 271, a determination unit 272, a selection unit 273, and a notification unit 274. With this configuration, the control unit 27 controls the function of the hearing test targeting the user based on behavioral recognition information regarding the user's actions. This allows the information processing system 1 to appropriately control the function of the hearing test according to the target user. Note that the configuration of the control unit 27 shown in Figure 2 is merely an example, and the control unit 27 may be configured in any way other than the configuration shown in Figure 2, as long as the information processing system 1 can perform the desired processing. For example, the information processing system 1 may select a sound according to behavioral recognition without determining whether to conduct a hearing test. In this case, the control unit 27 does not need to have a determination unit 272. The selection unit 273 of the control unit 27 selects a sound based on behavioral recognition information regarding the user's actions. For example, the selection unit 273 of the control unit 27 selects an output sound stimulus based on behavioral recognition information regarding the user's actions. Also, if the information processing system 1 does not determine whether to conduct a hearing test, it may conduct a hearing test in response to instructions from the user, etc.

[0028] The acquisition unit 271 acquires various types of information. The acquisition unit 271 acquires information from the storage unit 26. The acquisition unit 271 acquires information from the storage unit 26 to be used for processing. The acquisition unit 271 acquires information from the storage unit 26 to determine whether or not to perform a hearing test. The acquisition unit 271 acquires information from the storage unit 26 to perform a hearing test. The acquisition unit 271 acquires information from the sensor unit 29.

[0029] The acquisition unit 271 acquires behavioral recognition information relating to the actions of the user being subjected to the hearing test. The acquisition unit 271 acquires behavioral recognition information of the user being subjected to the ASSR hearing test. The acquisition unit 271 acquires sensor data. The acquisition unit 271 acquires sensor data features. The acquisition unit 271 acquires behavioral recognition information indicating the recognition result of the user's actions based on the sensor data detected by the sensor.

[0030] The acquisition unit 271 may perform behavior recognition processing using sensor data detected by sensors in the sensor unit 29. The acquisition unit 271 may generate information indicating the type of user behavior (also referred to as "behavior type") through behavior recognition processing. For example, the acquisition unit 271 may perform behavior recognition processing using a machine learning model (hereinafter also referred to as "behavior recognition model") that takes input data based on sensor data as input and outputs information indicating the behavior type corresponding to the input data. Any type of behavior can be adopted as long as it indicates the user's behavior, but an example of this will be described later. In addition, the acquisition unit 271 may perform noise environment recognition processing using sound information detected by the sound sensor of the sensor unit 29 and generate information indicating the noise type, such as volume.

[0031] The determination unit 272 performs determination processing to determine various types of information. For example, the determination unit 272 performs determination processing based on information acquired by the acquisition unit 271. For example, the determination unit 272 performs determination processing based on information stored in the storage unit 26. The determination unit 272 performs determination processing based on information acquired by the acquisition unit 271.

[0032] The determination unit 272 determines, based on the behavior recognition information, whether or not to perform a hearing test on the user. The determination unit 272 determines, based on the behavior recognition information, whether or not to perform ASSR as a hearing test on the user. The determination unit 272 determines, based on the behavior recognition information, whether or not to output a sound for the hearing test.

[0033] The determination unit 272 determines whether or not to output a sound for a hearing test based on environmental information about the user. The determination unit 272 determines whether or not to output a sound for a hearing test based on environmental information about noise. The determination unit 272 determines whether or not to perform a hearing test based on the recognition result of the user's behavior indicated by the behavior recognition information. Note that, as described above, if the information processing system 1 selects a sound according to behavior recognition without determining whether or not to perform a hearing test, the control unit 27 does not need to have the determination unit 272.

[0034] The selection unit 273 selects various types of information. The selection unit 273 selects information based on the information stored in the memory unit 26. For example, the selection unit 273 selects information based on the information acquired by the acquisition unit 271. For example, the selection unit 273 selects the mode of the hearing test based on the behavior recognition information.

[0035] The notification unit 274 provides notifications of various types of information. The notification unit 274 also notifies other computers, such as the terminal device 40, of the information. For example, the notification unit 274 provides notifications by transmitting information to other computers, such as the terminal device 40.

[0036] The notification unit 274 notifies the user of information regarding the hearing test. For example, the notification unit 274 notifies the user of information regarding the hearing test by transmitting the information regarding the hearing test to the terminal device 40 used by the user. If the hearing test outputs a sound above a predetermined standard, the notification unit 274 notifies the user of information indicating that sound output will be performed as part of the hearing test. The notification unit 274 notifies the user of information prompting the user to prepare for the hearing test. The notification unit 274 notifies the user of information regarding the progress of the hearing test.

[0037] The communication unit 28 is a communication module. The communication unit 28 communicates with the other hearing aid 2 using short-range communication such as NFMI (Near Field Magnetic Induction).

[0038] The sensor unit 29 includes an electroencephalogram (EEG) sensor 291 and an acceleration sensor 292. The EEG sensor 291 is a sensor that detects the user's brainwaves. For example, the EEG sensor 291 is placed in a location on the hearing aid 2 that comes into contact with the user's skin. Any sensor capable of detecting the user's brainwaves can be used as the EEG sensor 291. For example, the EEG sensor 291 is not limited to a contact-type sensor; it may also be a non-contact-type sensor. Furthermore, the EEG sensor 291 may be provided separately from the hearing aid 2.

[0039] The acceleration sensor 292 is a sensor that detects acceleration. For example, the acceleration sensor 292 detects acceleration related to the hearing aid 2. The acceleration sensor 292 detects acceleration related to the user wearing the hearing aid 2.

[0040] Furthermore, the sensor unit 29 may have various sensors in addition to the electroencephalogram sensor 291 and the acceleration sensor 292. For example, the sensor unit 29 may have various sensors that detect information used for estimating user behavior and the user's environment. For example, the sensor unit 29 may have a gyro sensor (angular velocity sensor), a geomagnetic sensor, and a barometric pressure sensor. The sensor unit 29 may also have various sensors such as a position sensor using GNSS (Global Navigation Satellite System), GPS (Global Positioning System), etc., a pulse sensor, a vibration sensor, a heart rate sensor, a blood pressure sensor, a sweat sensor, a body temperature sensor, and an image sensor.

[0041] Furthermore, the sensor unit 29 has a sensor that detects sound (noise) in the environment in which the hearing test is performed. For example, the sensor unit 29 may have a sound sensor such as a microphone that detects sound. For example, the sensor unit 29 may have a microphone that detects sound inside the user's ear (also called an "in-ear microphone") as a sound sensor. For example, the sensor unit 29 may have a microphone that detects sound around the user (also called an "external microphone") as a sound sensor. The sound sensor can be configured and arranged in any way as long as it can detect sound in the desired environment, and may be, for example, a sound collection unit 20.

[0042] The hearing aid 2 may also be further equipped with an operating unit (not shown). The operating unit may be a push-type switch, button, or touch panel. The operating unit receives signals for operating the hearing aid 2 and outputs the signals to the control unit 27.

[0043] In Figure 2, the charger 3 comprises a display unit 31, a battery 32, a storage unit 33, a communication unit 34, a memory unit 35, and a control unit 36.

[0044] The display unit 31 is a light-emitting LED (Light Emitting Diode). The display unit 31 displays various information. The display unit 31 displays information indicating that the hearing aid 2 is charging, and information indicating that it is receiving various information from the terminal device 40.

[0045] The battery 32 is a rechargeable battery, such as a lithium-ion battery. The battery 32 supplies power to the hearing aid 2 and charger 3 housed in the storage compartment 33 via a connection part 331 provided in the storage compartment 33.

[0046] The storage unit 33 stores each of the two left and right channels of the hearing aid 2 individually. The storage unit 33 is provided with a connection part 331 that can be connected to the connection part 24 of the hearing aid 2. The connection part 331 consists of one or more pins. When the hearing aid 2 is stored in the storage unit 33, the connection part 331 connects to the connection part 24 of the hearing aid 2, transmits power from the battery 32 and various information from the control unit 36, and receives various information from the hearing aid 2 and outputs it to the control unit 36.

[0047] The communication unit 34 is a communication module. The communication unit 34 communicates with the terminal device 40 via a communication network in accordance with communication standards such as Wi-Fi and Bluetooth.

[0048] The memory unit 35 is a RAM, ROM, flash memory, or memory card. The memory unit 35 stores the program 351 that the charger 3 executes.

[0049] The control unit 36 ​​consists of memory and a processor having hardware such as a CPU and DSP. The control unit 36 ​​controls each part of the charger 3. When the hearing aid 2 is placed in the storage unit 33, the control unit 36 ​​supplies power from the battery 32 via the connection unit 331. The control unit 36 ​​reads the program 351 into the working area of ​​the memory and executes it, and controls each part through the execution of the program 351 by the processor.

[0050] In Figure 3, the terminal device 40 includes an input unit 41, a communication unit 42, an output unit 43, a display unit 44, a storage unit 45, and a control unit 46.

[0051] The input unit 41 is a switch or touch panel. The input unit 41 receives various operation inputs from the user and outputs a signal corresponding to the received operation to the control unit 46.

[0052] The communication unit 42 is a communication module. The communication unit 42 communicates with the charger 3 or the hearing aid 2 via a communication network.

[0053] The output unit 43 is a speaker. The output unit 43 outputs sound at a volume set for each frequency.

[0054] The display unit 44 is a liquid crystal display or an organic electroluminescent (EL) display. The display unit 44 displays various information related to the terminal device 40 and the hearing aid 2.

[0055] The storage unit 45 is a recording medium such as RAM, ROM, flash memory, or memory card. The storage unit 45 stores various information related to the terminal device 40, programs 451 executed by the terminal device 40, and so on.

[0056] The control unit 46 consists of memory and a processor having hardware such as a CPU and DSP. The control unit 46 controls each part of the terminal device 40. The control unit 46 reads the program 451 stored in the storage unit 45 into the working area of ​​the memory and executes it, and controls each part through the execution of the program 451 by the processor.

[0057] <1-1-2. Overview of Information Processing> Next, an overview of the information processing performed by the information processing system 1 will be described, with reference to Figure 4 and other figures as appropriate. Figure 4 is a flowchart showing the processing procedure performed by the information processing system according to the first embodiment. Note that the processing described with the information processing system 1 as the processing entity may be performed by any device capable of executing that processing, depending on the device configuration included in the information processing system 1.

[0058] First, the information processing system 1 starts acquiring sensor data (step S101). For example, the hearing aid 2 acquires sensor data detected by sensors in the sensor unit 29, sound collection unit 20, etc. For example, the hearing aid 2 acquires sensor data detected by the acceleration sensor 292 of the sensor unit 29 as data to be used for user behavior recognition.

[0059] Sensor data can be various types of data, but examples include the following: For example, sensor data includes feature quantities detected by the sensor (also called "sensor data feature quantities"). Sensor data includes sensor data feature quantities of acceleration detected by the accelerometer 292. For example, sensor data includes data with DC (Direct Current) offset removed. Also, for example, sensor data includes data from which vector power has been calculated.

[0060] For example, the sensor data includes data on peak detection of vector power. For example, the sensor data includes data on threshold determination of the moving average value of vector power. For example, the sensor data includes data on the number of changes in the vector direction.

[0061] The above is merely an example, and sensor data may include a variety of data beyond those listed above. For example, sensor data may include data detected by multiple sensors. For example, sensor data may include not only acceleration but also sensor data features such as gyroscope, geomagnetic field, and atmospheric pressure. Furthermore, sensor data may include sensor data features such as outdoor positioning data from GNSS and user pulse rate data.

[0062] Furthermore, the information processing system 1 performs behavioral type recognition (step S102). For example, the hearing aid 2 uses a behavioral recognition model that takes sensor data features as input and outputs information indicating the behavioral type corresponding to the input sensor data features to perform behavioral recognition processing. Any machine learning model can be used as the behavioral recognition model, as long as it can produce the desired output according to the input.

[0063] For example, the behavior recognition model may be a neural network. For example, the behavior recognition model may be a neural network including CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network). The behavior recognition model is not limited to a neural network and may be any machine learning model. For example, the behavior recognition model may be a machine learning model that takes the low-pass filter results of each sensor data as input. For example, the behavior recognition model may be a machine learning model that performs discriminant analysis such as SVM (Support Vector Machine) and LDA (Linear Discriminant Analysis).

[0064] For example, hearing aid 2 inputs sensor data features into an action recognition model, and performs an action recognition process that determines the type of action indicated by the information output by the action recognition model as the user's action type. For example, the action type may be resting (awake), sleeping, riding (in a vehicle), etc. Here, resting (awake) may mean, for example, that the user is awake and there are no changes in the user's position or body movements that would affect the hearing test. Note that the above is just an example, and there may be various types of action. For example, the action type may be a further subdivision of the types described above.

[0065] Furthermore, for example, the types of behavior may be categorized by sleep stage. For example, the types of behavior may be categorized by sleep stage (sleep depth) estimated using acceleration and pulse rate. For example, the types of behavior may be categorized by sleep location. For example, the types of behavior may be categorized by sleep location determined by indoor positioning and seated position determination. For example, the types of behavior may be categorized by type of vehicle. For example, the types of behavior may be categorized by type of vehicle such as airplane, train, or car.

[0066] The information processing system 1 then determines whether the recognized type of behavior exists in the behavior-sound stimulus mapping dictionary (step S103). For example, the hearing aid 2 uses the behavior-sound stimulus mapping dictionary DT1 shown in Figure 5 to determine whether the type of user behavior recognized in step S102 is included in the behavior-sound stimulus mapping dictionary DT1. Figure 5 is a diagram showing an example of a behavior-sound stimulus mapping dictionary.

[0067] The behavior-sound stimulus mapping dictionary DT1 shown in Figure 5 is behavior-sound stimulus mapping information in which sound stimuli to be played are associated with each of several behavior types. The behavior-sound stimulus mapping dictionary DT1 includes the type of sound to be output and the modulation method as information about the sound stimuli to be played. In addition, as shown in the row for the behavior type "Sleep (Deep Sleep)", it may also include a specification for changing the volume of the sound corresponding to the behavior type. The behavior-sound stimulus mapping dictionary DT1 shows the cases in which "Stillness", "Sleep (Light Sleep)", "Sleep (Deep Sleep)", and "Vehicle (Airplane, Train, Car)" are used as behavior types.

[0068] The behavior-sound stimulus mapping dictionary DT1 shows the case where a chirp with 40Hz amplitude modulation is mapped to the sound stimulus to be played for the behavior type "stillness". Furthermore, the behavior-sound stimulus mapping dictionary DT1 also shows the case where a tone-pip with 90Hz amplitude modulation is mapped to the sound stimulus to be played for the behavior type "sleep (light sleep)". Note that the sound types and modulation methods shown in Figure 5 are merely examples, and the information on sound stimuli to be played that are mapped to each behavior type is not limited to the examples shown in Figure 5. For example, the sound type is not limited to chirp and tone-pip; it could be any audio signal such as an AM tone.

[0069] The behavior-sound stimulus mapping dictionary DT1 may be generated by any method. For example, the behavior-sound stimulus mapping dictionary DT1 may be generated by performing measurements in advance. Alternatively, the behavior-sound stimulus mapping dictionary DT1 may be generated by adjusting it so that the user's electroencephalogram response (strength of the 40 Hz component peak) to sound stimuli is increased.

[0070] If the information processing system 1 determines that the recognized type of action exists in the action-sound stimulus mapping dictionary (step S103: Yes), it selects a sound stimulus (step S104). For example, if the user's action type corresponds to "stillness," the information processing system 1 selects Chirp and 40Hz amplitude modulation as the sound stimulus to be played. Also, for example, if the user's action type corresponds to "sleep (light sleep)," the information processing system 1 selects Tone-pip and 90Hz amplitude modulation as the sound stimulus to be played.

[0071] Then, the information processing system 1 determines whether or not it has recognized noise (step S105). For example, the hearing aid 2 determines whether or not it has recognized noise based on the detection by the sound sensor of the sensor unit 29. The hearing aid 2 uses the sound information detected by the sound sensor of the sensor unit 29 as environmental information to determine whether or not it has recognized noise.

[0072] For example, the hearing aid 2 determines that noise has been recognized when the sensor unit 29 detects a sound above a predetermined threshold. For example, the criteria (guideline) for determining whether something is noise may be set based on the guidelines for hearing tests. For example, the hearing aid 2 may use an ambient noise level of 50 dB (A-weighted) as a reference. For example, the hearing aid 2 may determine that noise has been recognized when it detects a sound of 50 dB or higher.

[0073] The above is merely an example, and hearing aid 2 may determine whether or not it has detected noise based on various information. For example, the noise recognition process may be a recognition process based on sound pressure measurement using an in-ear microphone. Alternatively, for example, the noise recognition process may be a noise recognition process based on a prediction of in-ear noise after noise cancellation, which is obtained by multiplying the sound measured by an external microphone with the noise cancellation characteristics. For example, hearing aid 2 may perform noise recognition processing based on measurements taken with an in-ear microphone when noise cancellation is applied. Alternatively, for example, hearing aid 2 may perform noise recognition processing based on measurements taken with an external microphone when noise cancellation is not applied.

[0074] If the information processing system 1 does not recognize noise (step S105: No), it starts measuring brain waves (step S106). For example, if the hearing aid 2 determines that it does not recognize noise, it starts measuring brain waves using the brain wave sensor 291 of the sensor unit 29. Note that the start of brain wave measurement by the brain wave sensor 291, as referred to here, may also be defined as acquiring sensor data detected by the brain wave sensor 291, in which case the brain wave sensor 291 may perform continuous detection (measurement).

[0075] Then, the information processing system 1 reads the audiogram data (step S107). For example, the hearing aid 2 obtains an audiogram from the memory unit 26 that shows the results of the user's hearing test at that time, such as the audiogram data DT2 shown in Figure 6. Figure 6 is a diagram showing an example of audiogram data.

[0076] The audiogram data DT2 shown in Figure 6 is a matrix in which columns correspond to frequencies and rows to sound pressure, indicating the status of the hearing test for each combination of frequency and sound pressure. For example, among the elements of the audiogram data DT2, blank elements indicate frequency and sound pressure combinations for which a hearing test was not performed. Among the elements of the audiogram data DT2, elements associated with "N" indicate frequency and sound pressure combinations for which there was no user response to the sound output. Among the elements of the audiogram data DT2, elements associated with "Y" indicate frequency and sound pressure combinations for which there was a user response to the sound output.

[0077] Thus, in the audiogram data DT2, "N" indicates a combination of frequency and sound pressure corresponding to no user response (not hearing). For example, the combination of frequency "2000 Hz" and sound pressure "0 dBHL" indicates a frequency and sound pressure combination in which there was no user response. Also, in the audiogram data DT2, "Y" indicates a combination of frequency and sound pressure corresponding to a user response (hearing). For example, the combination of frequency "1000 Hz" and sound pressure "0 dBHL" indicates a frequency and sound pressure combination in which there was a user response. For example, in Figure 6, the audiogram data is still in progress, and the hearing threshold is determined only for 1000 Hz.

[0078] Then, the information processing system 1 starts playing the sound stimulus (step S108). The hearing aid 2 outputs the sound stimulus from the output unit 22. For example, the hearing aid 2 selects one of the frequency and sound pressure combinations in the audiogram data DT2 and plays the sound stimulus with the selected frequency and sound pressure. In this way, the hearing aid 2 selects the sound stimulus to be played from the behavior-sound stimulus mapping dictionary DT1 and also selects the frequency and sound pressure of the sound stimulus from the audiogram data DT2. For example, the hearing aid 2 selects a frequency and sound pressure combination that is "N" or blank from the frequency and sound pressure combinations in the audiogram data DT2 and plays the sound stimulus with the selected frequency and sound pressure.

[0079] The above is merely an example, and the selection of frequency and sound pressure may be performed in any manner. For example, hearing aid 2 may select frequencies and sound pressure from the audiogram data DT2 in the following order of priority #1 to #3. For example, hearing aid 2 selects a frequency for which no response was heard as priority #1. For example, hearing aid 2 selects a low sound pressure from the audiogram data DT2 as priority #2. For example, hearing aid 2 selects a frequency and sound pressure for which the total length of the data corresponding to the brainwave-sound stimulus / behavior DB described later is short as priority #3. Then, after the sound stimulus is reproduced in step S108, the information processing system 1 returns to step S102 and repeats the processing.

[0080] Furthermore, if the information processing system 1 determines that the recognized type of behavior does not exist in the behavior-sound stimulus correspondence dictionary (step S103: No), it executes the process in step S109. Also, if the information processing system 1 recognizes noise (step S105: Yes), it executes the process in step S109.

[0081] The information processing system 1 stops measuring brain waves (step S109). For example, the hearing aid 2 stops measuring brain waves using the brain wave sensor 291 of the sensor unit 29. Note that stopping brain wave measurement using the brain wave sensor 291 as referred to here may also mean stopping the acquisition of sensor data detected by the brain wave sensor 291, in which case the brain wave sensor 291 may continue to detect (measure).

[0082] The information processing system 1 stores electroencephalogram (EEG) data, reproduced sound stimulus data, and recognized behavior type in the EEG-sound stimulus / behavior DB (step S110). The data stored in the EEG-sound stimulus / behavior DB includes EEG data, sound stimulus data, behavior information, and noise information. For example, the EEG data includes signal measurement location, time-series potential difference, timestamp, etc. The sound stimulus data includes sound stimulus type, frequency [Hz], sound pressure [dBHL], specifications, etc. The behavior information includes behavior type, timestamp, etc. The noise information includes external noise level [dBSPL], intra-ear noise level [dBSPL], etc.

[0083] The EEG-sound stimulus / behavior DB may also be a memory unit 26. For example, the hearing aid 2 stores EEG data, reproduced sound stimulus data, and EEG-sound stimulus / behavior data including recognized behavior types in the memory unit 26. The information processing system 1 also performs a hearing level estimation process to estimate the hearing level from the EEG-sound stimulus / behavior DB. For example, the information processing system 1 uses the EEG data and sound stimulus data from the EEG-sound stimulus / behavior DB to perform a hearing level estimation process using the conventional method disclosed in Non-Patent Literature 1. The above is merely an example, and the information processing system 1 may perform a hearing level estimation process using the EEG-sound stimulus / behavior DB using any method, not limited to the conventional method disclosed in Non-Patent Literature 1. The information processing system 1 performs a hearing level estimation process that analyzes (estimates) and records whether there was a "response (heard)" or "no response (did not hear)" for each frequency and sound pressure level.

[0084] The information processing system 1 stops playing the sound stimulus (step S111) and returns to step S102 to repeat the process. The hearing aid 2 stops outputting the sound stimulus from the output unit 22 and returns to step S102 to repeat the process.

[0085] <1-1-3. Example of Information Processing System Configuration> Here, an example of the configuration of the information processing system 1 according to the first embodiment will be explained using Figure 7. Figure 7 is a diagram showing an example of the configuration of the information processing system according to the first embodiment.

[0086] In the example shown in Figure 7, the hearing aid 2 has an acceleration / gyro sensor 101. The acceleration / gyro sensor 101 detects acceleration and angular velocity. The acceleration / gyro sensor 101 corresponds to the acceleration sensor 292 and the gyro sensor of the sensor unit 29. The hearing aid 2 also has a position sensor 102 (hereinafter referred to as "GNSS position sensor 102") that utilizes GNSS, which is written as "GNSS" in the figure. The GNSS position sensor 102 detects position information. The GNSS position sensor 102 corresponds to the position sensor of the sensor unit 29.

[0087] The hearing aid 2 has a sensor feature extraction unit 103. The sensor feature extraction unit 103 extracts sensor data features. The sensor feature extraction unit 103 corresponds to the acquisition unit 271.

[0088] The hearing aid 2 has an action recognition unit 104. The action recognition unit 104 performs action recognition processing using sensor data features and generates information indicating the type of action. The action recognition unit 104 corresponds to the acquisition unit 271.

[0089] The hearing aid 2 has an in-ear microphone 105. The in-ear microphone 105 detects sounds inside the user's ear. The in-ear microphone 105 corresponds to the sound sensor of the sensor unit 29.

[0090] The hearing aid 2 has a noise environment recognition unit 106. The noise environment recognition unit 106 performs noise environment recognition processing using sound information detected by the in-ear microphone 105 and generates information indicating the type of noise, such as volume. The noise environment recognition unit 106 corresponds to the acquisition unit 271. Note that the noise environment recognition unit 106 may perform noise environment recognition using an external microphone, not just the in-ear microphone 105. In the case of the in-ear microphone 105, the noise environment recognition unit 106 may be capable of determining whether it is quiet or not, including the noise-canceling effect.

[0091] The hearing aid 2 has a behavior-sound stimulus mapping dictionary 107. The behavior-sound stimulus mapping dictionary 107 contains information related to the behavior-sound stimulus mapping dictionary. The behavior-sound stimulus mapping dictionary 107 corresponds to the behavior-sound stimulus mapping dictionary DT1 and is stored in the memory unit 26.

[0092] The hearing aid 2 has audiogram data 108. The audiogram data 108 includes information about the audiogram. The audiogram data 108 corresponds to the audiogram data DT2 and is stored in the storage unit 26.

[0093] The hearing aid 2 has an ASSR measurement control unit 109. The ASSR measurement control unit 109 controls the hearing test using a behavior-sound stimulus correspondence dictionary 107, audiogram data 108, behavior type, noise type, etc. The ASSR measurement control unit 109 corresponds to the control units 27, such as the determination unit 272 and the selection unit 273.

[0094] The hearing aid 2 has an audio output control unit 110. The audio output control unit 110 controls the audio output based on the audio signal from the ASSR measurement control unit 109. The audio output control unit 110 corresponds to the D / A conversion unit 221.

[0095] The hearing aid 2 has a single-ear / double-ear receiver 111. The single-ear / double-ear receiver 111 outputs sound in accordance with the control of the audio output control unit 110. The single-ear / double-ear receiver 111 corresponds to the receiver 222.

[0096] The hearing aid 2 has an electroencephalogram (EEG) sensor control unit 112. The EEG sensor control unit 112 controls the EEG sensor 113 in response to measurement start / stop instructions from the ASSR measurement control unit 109. The EEG sensor control unit 112 corresponds to the control units 27, such as the determination unit 272 and the selection unit 273. For example, the EEG sensor control unit 112 controls the start and stop of detection (measurement) by the EEG sensor 113. In addition, if the EEG sensor 113 is constantly detecting (measuring), the control of the EEG sensor 113 by the EEG sensor control unit 112 may also involve acquiring the EEG signal obtained from that detection.

[0097] The hearing aid 2 has an electroencephalogram (EEG) sensor 113. The EEG sensor 113 detects the user's brainwaves. The EEG sensor 113 corresponds to the EEG sensor 291.

[0098] The hearing aid 2 has an EEG-sound stimulus / behavior DB 114. The EEG-sound stimulus / behavior DB 114 stores information (data) obtained from hearing tests. For example, the data stored in the EEG-sound stimulus / behavior DB 114 includes various data such as EEG data, reproduced sound stimulus data, and recognized behavior types. The EEG-sound stimulus / behavior DB 114 corresponds to the memory unit 26.

[0099] As described above, the hearing aid 2 determines whether or not to perform a hearing test on the user based on behavioral recognition information. The hearing aid 2 determines whether or not to perform ASSR as a hearing test on the user based on behavioral recognition information. This allows the information processing system 1 to appropriately determine the timing for performing the user's hearing test. Therefore, the information processing system 1 can appropriately control the functions of the hearing test according to the target user.

[0100] Furthermore, the hearing aid 2 determines whether or not to perform a hearing test based on the recognition results of the user's actions indicated by the action recognition information. The hearing aid 2 also determines whether or not to output a sound for the hearing test based on the action recognition information. This allows the information processing system 1 to appropriately determine the timing for outputting the sound for the hearing test.

[0101] Furthermore, the hearing aid 2 determines whether or not to output a sound for hearing tests based on environmental information about the user. The hearing aid 2 also determines whether or not to output a sound for hearing tests based on environmental information about noise. This allows the information processing system 1 to appropriately determine the timing for outputting the sound for hearing tests.

[0102] Furthermore, if the hearing aid 2 determines that a hearing test should be performed, it selects a hearing test mode. This allows the information processing system 1 to appropriately select the hearing test mode. The hearing aid 2 then selects an output sound stimulus, which is a sound stimulus to be output for the hearing test. The hearing aid 2 selects an output sound stimulus using behavioral sound stimulus correspondence information, in which sound stimuli to be played are associated with each of several behavioral types. The hearing aid 2 selects as the output sound stimulus a sound stimulus that corresponds to the type of behavior of the user indicated by the behavioral recognition information, from among the multiple behavioral types included in the behavioral sound stimulus correspondence information. This allows the information processing system 1 to appropriately select the sound stimulus to be output for the hearing test.

[0103] Hearing aid 2 selects the output frequency, which is the frequency used for hearing tests, and the output sound pressure, which is the sound pressure. Hearing aid 2 selects the output frequency and output sound pressure using the user's audiogram. From the frequencies included in the user's audiogram, hearing aid 2 selects the frequency at which there is no audible response as the output frequency. From the sound pressures included in the user's audiogram, hearing aid 2 selects the sound pressure at which there is no audible response and the sound pressure is low as the output sound pressure. As a result, information processing system 1 can appropriately select the frequency and sound pressure for hearing tests.

[0104] <1-2. Second Embodiment (Distribution of Functions)> The information processing system 1 described above is merely an example, and the information processing system may have any configuration as long as the desired processing can be executed. For example, in the information processing system 1, the hearing aid 2 is shown to have a function related to hearing tests, but the function related to hearing tests may be distributed among multiple devices. For example, the function related to hearing tests may be distributed between the hearing aid 2 and the terminal device 40.

[0105] An example of this point will be explained using Figure 8 as a second embodiment. Figure 8 is a diagram showing an example of the configuration of an information processing system according to the second embodiment. Hereinafter, the information processing system 1 according to the second embodiment will be referred to as information processing system 1A, the hearing aid 2 according to the second embodiment as hearing aid 2A, and the terminal device 40 according to the second embodiment as terminal device 40A. Note that the same points as in the first embodiment will be omitted from explanation as appropriate by using the same reference numerals.

[0106] In the example shown in Figure 8, the information processing system 1A includes a hearing aid 2A and a terminal device 40A. For example, the hearing aid 2A and the terminal device 40A are connected via Bluetooth for communication.

[0107] The hearing aid 2A includes an in-ear microphone 105, a noise environment recognition unit 106, an audio output control unit 110, a single-ear / double-ear receiver 111, an electroencephalogram (EEG) sensor control unit 112, and an EEG sensor 113. In other words, in the information processing system 1A according to the second embodiment, the hearing aid 2A has the configurations of the hearing aid 2 in the information processing system 1 according to the first embodiment, which correspond to the in-ear microphone 105, noise environment recognition unit 106, audio output control unit 110, single-ear / double-ear receiver 111, EEG sensor control unit 112, and EEG sensor 113.

[0108] Furthermore, the terminal device 40A includes an acceleration / gyro sensor 101, a GNSS position sensor 102, a sensor feature extraction unit 103, an action recognition unit 104, an action-sound stimulus mapping dictionary 107, audiogram data 108, an ASSR measurement control unit 109, and an electroencephalogram-sound stimulus / action DB 114. In other words, in the information processing system 1A according to the second embodiment, the terminal device 40A has the configurations of the hearing aid 2 in the information processing system 1 according to the first embodiment, which correspond to the acceleration / gyro sensor 101, the GNSS position sensor 102, the sensor feature extraction unit 103, the action recognition unit 104, the action-sound stimulus mapping dictionary 107, the audiogram data 108, the ASSR measurement control unit 109, and the electroencephalogram-sound stimulus / action DB 114.

[0109] Furthermore, since information processing system 1A is the same as information processing system 1 except that the functions related to hearing tests are distributed between the hearing aid 2 and the terminal device 40, a detailed explanation of the information processing will be omitted.

[0110] As shown in Figure 8, in the information processing system 1A, the terminal device 40A determines whether or not to perform a hearing test on the user based on the behavior recognition information. The terminal device 40A determines whether or not to perform ASSR as a hearing test on the user based on the behavior recognition information. This allows the information processing system 1A to appropriately determine the timing for performing the hearing test on the user. Therefore, the information processing system 1A can appropriately control the function of the hearing test according to the target user.

[0111] Furthermore, the terminal device 40A determines whether or not to perform a hearing test based on the recognition result of the user's actions indicated by the action recognition information. The terminal device 40A also determines whether or not to output a sound for the hearing test based on the action recognition information. This allows the information processing system 1A to appropriately determine the timing for outputting the sound for the hearing test.

[0112] Furthermore, the terminal device 40A determines whether or not to output a sound for hearing tests based on environmental information about the user. The terminal device 40A also determines whether or not to output a sound for hearing tests based on environmental information about noise. This allows the information processing system 1A to appropriately determine the timing for outputting the sound for hearing tests.

[0113] Furthermore, if the terminal device 40A determines that a hearing test should be performed, it selects the mode for the hearing test. This allows the information processing system 1A to appropriately select the mode for the hearing test. The terminal device 40A selects the output sound stimulus, which is the sound stimulus to be output for the hearing test. The terminal device 40A selects the output sound stimulus using behavioral sound stimulus correspondence information, which associates sound stimuli to be played with each of a plurality of behavioral types. The terminal device 40A selects the sound stimulus to be output that corresponds to the behavioral type of the user's behavior indicated by the behavioral recognition information, from among the plurality of behavioral types included in the behavioral sound stimulus correspondence information. This allows the information processing system 1A to appropriately select the sound stimulus to be output for the hearing test.

[0114] Terminal device 40A selects the output frequency, which is the frequency for the hearing test, and the output sound pressure, which is the sound pressure. Terminal device 40A selects the output frequency and output sound pressure using the user's audiogram. Terminal device 40A selects the frequency in the user's audiogram that does not elicit a response as the output frequency. Terminal device 40A selects the sound pressure in the user's audiogram that is low and does not elicit a response as the output sound pressure. As a result, the information processing system 1A can appropriately select the frequency and sound pressure for the hearing test.

[0115] <1-3. Third Embodiment (User Environment Dictionary)> The processing related to hearing tests may be performed using various information other than the information described above. An example of this will be described as a third embodiment. In the following, we will describe as an example a case in which a new configuration is added to the configuration of the information processing system 1A according to the second embodiment shown in Figure 8, but the configuration described below may also be added to the information processing system 1 according to the first embodiment.

[0116] <1-3-1. Example of Information Processing System Configuration> First, an example of the configuration of an information processing system will be explained using Figure 9. Figure 9 is a diagram showing an example of the configuration of an information processing system according to the third embodiment. Hereinafter, the information processing system 1 according to the third embodiment will be referred to as information processing system 1B, the hearing aid 2 according to the third embodiment as hearing aid 2B, and the terminal device 40 according to the third embodiment as terminal device 40B. Note that the same points as in the first or second embodiment will be omitted from explanation as appropriate by using the same reference numerals.

[0117] In the example shown in Figure 9, the information processing system 1B includes a hearing aid 2B and a terminal device 40B. In Figure 9, the case where the terminal device 40B is a smartphone is explained as an example. The hearing aid 2B includes an in-ear microphone 105, a noise environment recognition unit 106, an audio output control unit 110, a single-ear / double-ear receiver 111, an electroencephalogram (EEG) sensor control unit 112, and an EEG sensor 113. The terminal device 40B includes an acceleration / gyro sensor 101, a GNSS position sensor 102, a sensor feature extraction unit 103, an action recognition unit 104, an action-sound stimulus mapping dictionary 107, audiogram data 108, an ASSR measurement control unit 109, and an EEG-sound stimulus / action DB 114.

[0118] Furthermore, in the example shown in Figure 9, the terminal device 40B has smartphone operation information 115. The smartphone operation information 115 includes information related to the operation of the terminal device 40B, which is a smartphone. The smartphone operation information 115 is information used for environmental recognition regarding the user and is stored in the storage unit 45.

[0119] The terminal device 40B has a user environment recognition unit 116. The user environment recognition unit 116 uses smartphone operation information 115 to perform environment recognition processing related to the user and generates user environment information that indicates the environment related to the user. For example, the user environment recognition unit 116 corresponds to the control unit 46 (or its acquisition unit, etc.). For example, the user environment recognition unit 116 generates information indicating the music playback status, the user's smartphone operation history, the user's current schedule information, etc., as user environment information through environment recognition processing based on information regarding the user's operation of the terminal device 40B.

[0120] The terminal device 40B has a user environment determination dictionary 117. The user environment determination dictionary 117 contains information related to determining the user environment. The user environment determination dictionary 117 is information used to determine the user environment and is stored in the storage unit 45. For example, the user environment determination dictionary 117 contains information such as that shown in the user environment determination dictionary DT3 in Figure 10. Figure 10 is a diagram showing an example of the user environment determination dictionary.

[0121] The user environment determination dictionary DT3 shown in Figure 10 is user environment determination information in which conditions for a "Yes" determination are associated with each of several environment types. The user environment determination dictionary DT3 shows the case where "music playback status," "user's smartphone operation history," and "user's current schedule information" are used as environment types. The conditions for a "Yes" determination in the user environment determination dictionary DT3 indicate the conditions under which the determination for the corresponding environment type becomes "Yes." Note that the environment types and "Yes" determination conditions shown in Figure 10 are merely examples, and the user environment determination dictionary DT3 may contain various combinations of environment types and "Yes" determination conditions.

[0122] In the user environment determination dictionary DT3, the condition for determining "Yes" is associated with the environment type "music playback status". In other words, the information processing system 1B determines "Yes" for the environment type "music playback status" if music is not being played.

[0123] In the user environment determination dictionary DT3, the environment type "User's smartphone operation history" is associated with the condition "No operation within 1 minute" for a Yes determination. In other words, for the environment type "User's smartphone operation history," the information processing system 1B determines Yes if the terminal device 40B has not been operated within 1 minute.

[0124] In the user environment determination dictionary DT3, the environment type "User's current schedule information" is associated with the condition "No meetings or exercises scheduled." In other words, for the environment type "User's current schedule information," the information processing system 1B determines it to Yes if the user does not have any meetings or exercises scheduled at that time.

[0125] <1-3-2. Overview of Information Processing> Next, an overview of the information processing performed by the information processing system 1B will be described, with reference to Figure 11 and other figures as appropriate. Figure 11 is a flowchart of the processing procedure performed by the information processing system according to the third embodiment. Note that explanations of points similar to those explained in Figure 4 and other figures will be omitted as appropriate.

[0126] First, the information processing system 1B starts acquiring sensor data (step S201). For example, the hearing aid 2B and the terminal device 40B acquire the sensor data as data to be used for recognizing the user's actions.

[0127] Furthermore, the information processing system 1B performs behavioral type recognition (step S202). For example, the terminal device 40B takes sensor data features as input and performs behavioral recognition processing using a behavioral recognition model that outputs information indicating the behavioral type corresponding to the input sensor data features.

[0128] Furthermore, the information processing system 1B recognizes the user environment corresponding to the environment type in the user environment determination dictionary (step S203). For example, the terminal device 40B uses information regarding the user's operation of the terminal device 40B to recognize the user environment corresponding to the environment type in the user environment determination dictionary DT3 shown in Figure 10.

[0129] Then, the information processing system 1B determines whether the recognized user environment meets all the "Yes judgment conditions" for all environment types in the user environment determination dictionary (step S204). For example, the terminal device 40B uses the user environment determination dictionary DT3 shown in Figure 10 to determine whether the user environment recognized in step S203 meets all the "Yes judgment conditions" for all environment types in the user environment determination dictionary DT3.

[0130] If the information processing system 1B determines that the recognized user environment meets all the "Yes determination conditions" for environment types in the user environment determination dictionary (step S204: Yes), it determines whether the recognized type of action exists in the action-sound stimulus mapping dictionary (step S205). For example, the terminal device 40B uses the action-sound stimulus mapping dictionary DT1 shown in Figure 5 to determine whether the type of action of the user recognized in step S202 is included in the action-sound stimulus mapping dictionary DT1.

[0131] If the information processing system 1B determines that the recognized type of action exists in the action-sound stimulus mapping dictionary (step S205: Yes), it selects a sound stimulus (step S206).

[0132] Then, the information processing system 1B determines whether or not it has recognized the noise (step S207). For example, terminal device 40B determines whether or not it has recognized the noise based on detection by the sound sensor. Terminal device 40B uses the sound information detected by the sound sensor as environmental information to determine whether or not it has recognized the noise.

[0133] If the information processing system 1B does not detect noise (step S207: No), it starts measuring brain waves (step S208). For example, if the terminal device 40B determines that it has detected noise, it instructs the hearing aid 2B to start measuring brain waves using the brain wave sensor 291 of the sensor unit 29.

[0134] Then, the information processing system 1B reads the audiogram data (step S209). For example, the terminal device 40B obtains an audiogram from the storage unit 26, such as the audiogram data DT2 shown in Figure 6, which shows the results of the user's hearing test at that time.

[0135] Then, the information processing system 1B starts playing the sound stimulus (step S210). The terminal device 40B instructs the hearing aid 2B to output the sound stimulus, and the hearing aid 2B outputs the sound stimulus. After the sound stimulus is played back in step S210, the information processing system 1B returns to step S202 and repeats the process.

[0136] Furthermore, if the information processing system 1B determines that the recognized user environment does not meet at least one of the "Yes judgment conditions" among the environment types in the user environment determination dictionary (step S204: No), it executes the process in step S211. Also, if the information processing system 1B determines that the recognized behavior type does not exist in the behavior-sound stimulus correspondence dictionary (step S205: No), it executes the process in step S211. Also, if the information processing system 1B recognizes noise (step S207: Yes), it executes the process in step S211.

[0137] The information processing system 1B stops measuring brain waves (step S211). For example, the terminal device 40B instructs the hearing aid 2B to stop measuring brain waves using the brain wave sensor 291 of the sensor unit 29.

[0138] The information processing system 1B stores brainwave data, played sound stimulus data, and recognized behavior types in the brainwave-sound stimulus / behavior DB (step S212). For example, terminal device 40B stores brainwave-sound stimulus / behavior data, including brainwave data received from hearing aid 2B, played sound stimulus data, and recognized behavior types, in storage unit 45.

[0139] The information processing system 1B stops the playback of the sound stimulus (step S213) and returns to step S202 to repeat the process. The terminal device 40B instructs the hearing aid 2B to stop outputting the sound stimulus, and the hearing aid 2B stops outputting the sound stimulus and returns to step S202 to repeat the process.

[0140] As described above, in the information processing system 1B, the terminal device 40B determines whether or not to perform a hearing test on the user based on the user environment. The terminal device 40B determines whether or not to perform ASSR as a hearing test on the user based on the behavior recognition information and the user environment. This allows the information processing system 1B to appropriately determine the timing for performing the hearing test on the user. Therefore, the information processing system 1B can appropriately control the function of the hearing test according to the target user.

[0141] <1-4. Examples of Notifications> From here, examples of information notifications in the first to third embodiments described above will be described below, using Information Processing System 1 as an example. Note that explanations of points similar to those described above will be omitted as appropriate.

[0142] <1-4-1. First Notification Example (Audio Output Notification)> The information processing system 1 notifies the user of content CT 11, as shown in Figure 12. Figure 12 is a diagram showing a first notification example. For example, Figure 12 is a diagram showing an example of an audio output notification. Content CT 11 includes information that is notified when the auditory stimulus is a sound loud enough for the user to notice. Content CT 11 is a display screen (content) that includes information such as "Hearing level measurement is starting. You may hear a signal sound."

[0143] The hearing aid 2 generates content CT11 containing information related to the hearing test. In this case, the hearing aid 2 (for example, the control unit 27) has a generation unit that generates information to be notified regarding the hearing test, such as content CT11 (also called "notification information"). The generation unit of the hearing aid 2 generates notification information such as content CT11 to CT13. For example, the generation unit may use various technologies such as Java® as appropriate to generate notification information such as content to be transmitted (notified) to the terminal device 40. The generation unit may also generate notification information such as content to be transmitted (notified) to the terminal device 40 based on the format of CSS (Cascading Style Sheets), JavaScript®, or HTML.

[0144] Furthermore, for example, the generation unit of the hearing aid 2 may generate notification information such as content in various formats such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), and PNG (Portable Network Graphics). Note that if the terminal device 40 generates the above-mentioned content CT11 to CT13, the hearing aid 2 does not need to have a generation unit. In this case, the hearing aid 2 may transmit information used by the terminal device 40 to generate the content CT11 to CT13 to the terminal device 40, and the terminal device 40 may generate notification information such as content based on the information received from the hearing aid 2.

[0145] The hearing aid 2 notifies the user of the generated content CT 11 by transmitting it to the terminal device 40. The terminal device 40 displays the content CT 11, which includes information about the hearing test. In this way, the information processing system 1 notifies the user of information about the hearing test. This allows the information processing system 1 to appropriately notify the user of information about the hearing test.

[0146] In Figure 12, if the terminal device 40 outputs a sound above a predetermined standard during the hearing test, it notifies the user of content CT 11 containing information indicating that sound output will be generated due to the hearing test. This allows the information processing system 1 to appropriately notify the user that sound output will be generated due to the hearing test.

[0147] <1-4-2. Second Notification Example (Notification Prompting the User)> The information processing system 1 notifies the user of content CT 12, as shown in Figure 13. Figure 13 is a diagram illustrating a second notification example. For example, Figure 13 is a diagram illustrating an example of a notification prompting the user. Content CT 12 includes information that is notified when the noise environment is good but there is a lot of activity and the user is encouraged to remain still. Content CT 12 is a display screen (content) that includes information such as "This is a suitable environment for hearing level measurement, so please remain still." For example, content CT 12 is notification information that is notified when step S205 in Figure 11 is determined to be No. For example, the information processing system 1 notifies the user of content CT 12 when step S205 is determined to be No.

[0148] The hearing aid 2 generates content CT 12 containing information related to the hearing test. The hearing aid 2 notifies the user of the generated content CT 12 by transmitting it to the terminal device 40. The terminal device 40 notifies the user of the content CT 12 containing information prompting the user to prepare for the hearing test. This allows the information processing system 1 to appropriately notify the user of information prompting the user to perform the hearing test properly.

[0149] <1-4-3. Third Notification Example (Progress Notification)> The information processing system 1 notifies the user of content CT 13 as shown in Figure 14. Figure 14 is a diagram showing a third notification example. For example, Figure 14 is a diagram showing an example of a progress notification. Content CT 13 includes information to confirm the progress until the completion of audiogram measurement and to present the path for additional measurements. Content CT 13 is a display screen (content) that includes a diagram that graphically shows the current progress (in Figure 14, a meter showing 55% progress).

[0150] Content CT13 is a display screen (content) that includes information such as, "Hearing level measurement progress: If you do mindfulness for 10 minutes now, it will advance by 20%." Content CT13 is also a display screen (content) that includes a button labeled "Start Mindfulness Session" and, if selected, starts a mindfulness session.

[0151] The hearing aid 2 generates content CT 13 containing information about the hearing test. The hearing aid 2 notifies the user of the generated content CT 13 by transmitting it to the terminal device 40. The terminal device 40 notifies the user of the content CT 13 containing information about the progress of the hearing test. This allows the information processing system 1 to appropriately notify the user of the progress of the hearing test.

[0152] <2. Others> The processes according to each embodiment described above may be carried out in various different forms (modifications) other than the embodiments and modifications described above. The information processing system 1 is described below as an example. Points that are the same as those described above will be omitted from explanation as appropriate.

[0153] <2-1. Examples of Other Behavior-Auditory Stimulus Correspondence Dictionaries> In the example described above, the behavior-audio stimulus correspondence dictionary DT1 shown in Figure 5 was used as an example, but the behavior-audio stimulus correspondence dictionary is not limited to the behavior-audio stimulus correspondence dictionary DT1 shown in Figure 5. For example, the sound stimulus to be played may be any sound stimulus, such as the sound stimulus shown in Figure 15. Figure 15 is a diagram showing another example of a behavior-audio stimulus correspondence dictionary.

[0154] The behavior-sound stimulus mapping dictionary DT4 shown in Figure 15 is behavior-sound stimulus mapping information in which sound stimuli to be played are associated with each of several types of behavior. The behavior-sound stimulus mapping dictionary DT4 includes the modulation of conversational speech that reached the user's ears as information about the sound stimuli to be played.

[0155] As shown in Figure 15, the information processing system 1 outputs a sound for hearing tests by modulating it according to the conversational speech that reaches the user's ears. For example, the information processing system 1 outputs a sound for hearing tests by modulating it according to the conversational speech that reaches the user's ears using a conventional method disclosed in European Patent Application Publication No. 4065204. However, the above is merely an example, and the information processing system 1 may output a sound for hearing tests by modulating it according to the conversational speech that reaches the user's ears using any method, not limited to the conventional method disclosed in European Patent Application Publication No. 4065204.

[0156] As a result, even when the behavior type is static, sleeping, or in a vehicle, or in a situation with little body movement, but noise is detected, if the noise is conversation, the information processing system 1 can add brainwave data and sound stimulus data to the brainwave-sound stimulus / behavior database.

[0157] <2-2. Summary> As described above, the information processing systems 1, 1A, and 1B can appropriately perform hearing tests by determining whether or not to conduct a hearing test on a user based on behavioral recognition information regarding the user's actions.

[0158] For example, when performing hearing tests such as ASSR on a daily basis, the equipment may be noisy due to its characteristics. Therefore, in conventional hearing tests, the measurement time becomes long, and if a sufficient period of quiet measurement cannot be secured, the measurement cannot be completed. Consequently, automatic measurement is not possible, and ultimately, the user must perform the test intentionally, which is not very convenient for the user. Thus, in conventional hearing tests, when performing automatic measurements, it is impossible to control the environment in which the user is placed, such as noise, electromagnetic noise, and motion noise.

[0159] Thus, in conventional hearing tests, when hearing tests such as ASSR are automatically performed in the user's daily life, there were problems such as the data not being useful even if measurements were taken due to loud noise or large body movements, resulting in power consumption and the need to add extra data.

[0160] Therefore, information processing systems 1, 1A, and 1B detect favorable conditions for hearing tests such as ASSR in daily life through behavior recognition and noise measurement, and control the start of measurement. For example, when information processing systems 1, 1A, and 1B detect a quiet environment with little movement through behavior recognition and noise measurement, they select and present sound stimuli and start electroencephalogram (EEG) measurement. As a result, information processing systems 1, 1A, and 1B can reduce power consumption and shorten the time required to complete hearing tests such as ASSR, which can be performed automatically without the user's awareness.

[0161] Therefore, information processing systems 1, 1A, and 1B can perform hearing tests for adjusting hearing aids without the user being aware of it. In particular, information processing systems 1, 1A, and 1B can perform hearing tests with high accuracy because the signal quality of the measurement is high. In addition, less data is required to output the test results. As a result, information processing systems 1, 1A, and 1B can enable the early detection of changes in hearing.

[0162] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0163] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0164] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0165] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0166] <3. Hardware Configuration> The information processing devices (information equipment) such as the hearing aid 2 and terminal device 40 according to the embodiments described above are realized by a computer 1000 with the configuration shown in Figure 16. Figure 16 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device according to the embodiment. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD (Hard Disk Drive) 1400, communication interface 1500, and input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050.

[0167] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each part. The CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0168] ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that are executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0169] The HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by such programs. Specifically, the HDD 1400 is a recording medium that records an acoustic processing program according to this disclosure, which is an example of program data 1450.

[0170] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (such as the Internet). The CPU 1100 receives data from other devices and transmits data it generates to other devices via the communication interface 1500.

[0171] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. The CPU 1100 receives data from input devices such as a keyboard and mouse via the input / output interface 1600. The CPU 1100 transmits data to output devices such as a display, speaker, and printer via the input / output interface 1600. The input / output interface 1600 can also function as a media interface for reading programs recorded on a predetermined recording medium (media).

[0172] Media refers to optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase-change rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical disks), tape media, magnetic recording media, or semiconductor memory.

[0173] When the computer 1000 functions as a hearing aid 2 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control units 27 and 46 by executing an information processing program loaded onto the RAM 1200. The HDD 1400 stores the information processing program according to this disclosure and the data in the storage units 26 and 45.

[0174] The CPU 1100 reads program data 1450 from the HDD 1400 and executes it. However, as an alternative, the CPU 1100 can also obtain these programs from other devices via an external network 1550.

[0175] <4. Other Embodiments> The configurations of the hearing aids and other components included in the information processing systems according to the first to third embodiments described above are merely examples, and the information processing system is not limited to the configurations shown in the first to third embodiments, but can employ any configuration. For example, the hearing aids are not limited to the configurations of the hearing aids 2, 2A, and 2B described above, but may have configurations with various functions. Examples in this regard are described below.

[0176] <4-1. Fourth Embodiment (Measures against Unpleasant Sound Pressure)> First, an example of the configuration of the information processing system 1 according to the fourth embodiment (hereinafter sometimes referred to as "information processing system 1C") will be described using Figure 17. Figure 17 is a diagram showing an example of the configuration of the information processing system according to the fourth embodiment. Note that for configurations similar to those described in Figure 7, etc., the same reference numerals will be used, and explanations will be omitted as appropriate.

[0177] In the example shown in Figure 17, the hearing aid 2C has unpleasant sound pressure data 118. The unpleasant sound pressure data 118 includes information (unpleasant sound condition information) about the sound pressure that the subject (user, etc.) finds unpleasant (also called "unpleasant sound pressure"). The unpleasant sound pressure data 118 is stored in the memory unit 26. The method for acquiring unpleasant sound pressure will be described later.

[0178] Next, using Figure 18, an overview of the information processing performed by the information processing system 1 (information processing system 1C) including the hearing aid 2C will be described. Figure 18 is a flowchart showing the processing procedure performed by the information processing system according to the fourth embodiment. Note that detailed explanations of points similar to those described in Figure 4, etc., will be omitted as appropriate.

[0179] First, the information processing system 1C starts acquiring sensor data (step S301). Step S301 is the same as step S101 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2C acquires sensor data detected by sensors in the sensor unit 29, sound collection unit 20, etc.

[0180] Furthermore, the information processing system 1C performs behavioral type recognition (step S302). Although a detailed explanation of step S302 is omitted as it is the same as step S102 in Figure 4, for example, the hearing aid 2C performs behavioral recognition processing using a behavioral recognition model that takes sensor data features as input and outputs information indicating the behavioral type corresponding to the input sensor data features.

[0181] Then, the information processing system 1C determines whether the recognized type of behavior exists in the behavior-sound stimulus mapping dictionary (step S303). Step S303 is the same as step S103 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2C uses the behavior-sound stimulus mapping dictionary DT1 shown in Figure 5 to determine whether the type of user behavior recognized in step S302 is included in the behavior-sound stimulus mapping dictionary DT1.

[0182] If the information processing system 1C determines that the recognized type of action exists in the action-sound stimulus mapping dictionary (step S303: Yes), it determines whether or not noise has been recognized (step S304). Step S304 is the same as step S105 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2C determines whether or not noise has been recognized based on detection by the sound sensor of the sensor unit 29.

[0183] If the information processing system 1C does not recognize noise (step S304: No), it starts measuring electroencephalograms (step S305). Step S305 is the same as step S106 in Figure 4, so a detailed explanation is omitted, but for example, if the hearing aid 2C determines that it does not recognize noise, it starts measuring electroencephalograms using the electroencephalogram sensor 291 of the sensor unit 29.

[0184] The information processing system 1C then reads the audiogram data and the unpleasant sound pressure data (step S306). For example, the hearing aid 2C obtains an audiogram from the memory unit 26, such as the audiogram data DT2 shown in Figure 6, which shows the results of the user's hearing test at that time. Also, for example, the hearing aid 2C obtains data from the memory unit 26 that shows the unpleasant sound pressure corresponding to the user who is the subject.

[0185] The information processing system 1C then selects an audible stimulus and starts playing it (step S307). Step S307 is the same as steps S104 and S108 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2C selects an audible stimulus corresponding to the user's type of action and outputs the selected audible stimulus from the output unit 22. After playing the audible stimulus in step S307, the information processing system 1C returns to step S302 and repeats the process.

[0186] Furthermore, if the information processing system 1C determines that the recognized type of action does not exist in the action-sound stimulus correspondence dictionary (step S303: No), it executes the process in step S308. Also, if the information processing system 1C recognizes noise (step S304: Yes), it executes the process in step S308.

[0187] The information processing system 1C stops measuring brain waves (step S308). Step S308 is the same as step S109 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2C stops measuring brain waves using the brain wave sensor 291 of the sensor unit 29.

[0188] The information processing system 1C stores the electroencephalogram (EEG) data, the played-back sound stimulus data, and the recognized behavior type in the EEG-sound stimulus / behavior DB (step S309). Since step S309 is the same as step S110 in Figure 4, a detailed explanation is omitted.

[0189] The information processing system 1C stops the playback of the sound stimulus (step S310) and returns to step S302 to repeat the process. The hearing aid 2C stops the output of the sound stimulus from the output unit 22 and returns to step S302 to repeat the process.

[0190] <4-1-1. Examples of acquiring unpleasant sound pressure> Based on the configuration and processing described above, several examples of methods for acquiring unpleasant sound pressure in the information processing system 1C are described below.

[0191] For example, the information processing system 1C may acquire unpleasant sound pressure through a subjective evaluation test in the application. This example will be explained as a first example of acquiring unpleasant sound pressure. In this case, the information processing system 1C acquires information indicating the user's unpleasant sound pressure from the content CT21, as shown in Figure 19. Figure 19 is a diagram showing an example of acquiring unpleasant sound pressure. For example, Figure 19 is a diagram showing an example of acquiring unpleasant sound pressure through a subjective evaluation test in the application.

[0192] Content CT21 is a display screen (content) that includes information such as, "Please tell us the volume level you do not want to hear during the automated hearing test." Content CT21 also includes elements for the user to input unpleasant sound pressure levels. Figure 19 shows how the user can input unpleasant sound pressure levels using a seek bar.

[0193] The hearing aid 2C generates content CT21 containing information related to hearing tests. In this case, the hearing aid 2C (e.g., the control unit 27) has a generation unit that generates content such as content CT21 for acquiring the user's uncomfortable sound pressure. The generation unit of the hearing aid 2C generates notification information such as content CT21 to CT23.

[0194] Furthermore, if the terminal device 40 generates the above-mentioned content CT21 to CT23, the hearing aid 2C does not need to have a generation unit. In this case, the hearing aid 2C may transmit information used by the terminal device 40 to generate the content CT21 to CT23 to the terminal device 40, and the terminal device 40 may generate notification information such as content based on the information received from the hearing aid 2C.

[0195] The hearing aid 2C transmits the generated content CT21 to the terminal device 40. The terminal device 40 displays the content CT21 for acquiring the user's unpleasant sound pressure and accepts the user's input of the unpleasant sound pressure for the content CT21. The terminal device 40 then transmits the information indicating the user's unpleasant sound pressure received from the content CT21 to the hearing aid 2C. As a result, the hearing aid 2C acquires the user's unpleasant sound pressure. The hearing aid 2C registers the acquired user's unpleasant sound pressure in the storage unit 26 as unpleasant sound condition information indicating the conditions of sound pressure that the user finds unpleasant.

[0196] Furthermore, for example, the information processing system 1C may acquire the sound pressure at which the user presses the stop button in response to a notification during inspection as an unpleasant sound pressure. This example will be explained as a second example of acquiring unpleasant sound pressure. In this case, as shown in Figure 20, the information processing system 1C acquires information indicating the user's unpleasant sound pressure from the content CT 22. Figure 20 is a diagram showing an example of acquiring unpleasant sound pressure. Note that explanations of points similar to those in the first acquisition example will be omitted as appropriate.

[0197] Content CT22 is a display screen (content) that includes information such as "Tap here if you want to interrupt the measurement and mute the sound." The hearing aid 2C transmits the generated content CT22 to the terminal device 40. The terminal device 40 displays content CT22 to acquire the user's uncomfortable sound pressure and performs the test. Then, when the user presses stop in response to a notification during the test, the terminal device 40 transmits information indicating the user's uncomfortable sound pressure, received by content CT22, to the hearing aid 2C. As a result, the hearing aid 2C acquires the user's uncomfortable sound pressure.

[0198] Furthermore, for example, the information processing system 1C may estimate and acquire the unpleasant sound pressure based on the user's age. This example will be explained as a third example of acquiring unpleasant sound pressure. For example, the information processing system 1C estimates and acquires the unpleasant sound pressure based on the user's age, using information that associates age with unpleasant sound pressure. For example, if the user is 60 years old, the information processing system 1C estimates that 4 kHz at 30 dB HL is unpleasant and acquires it as the unpleasant sound pressure. Note that this value may be determined based on statistical data of average hearing for each age and statistical data of sound pressure perceived as unpleasant.

[0199] Furthermore, for example, the information processing system 1C may estimate and acquire the unpleasant sound pressure based on the user's previously measured hearing level. This example will be explained as a fourth example of acquiring unpleasant sound pressure. For example, if the user's previously measured hearing level was 10 dB HL, the information processing system 1C estimates +20 dB HL as unpleasant and acquires it as the unpleasant sound pressure. This value may be determined empirically based on data from multiple users.

[0200] Furthermore, for example, the information processing system 1C may acquire unpleasant sound pressure by facial expression recognition. This example will be explained as a fifth example of acquiring unpleasant sound pressure. For example, the information processing system 1C acquires unpleasant sound pressure by recognizing the user's facial expression using information in which the sound pressure of the inspection sound is set for different facial expressions. In this case, as shown in Figure 21, the information processing system 1C acquires information indicating the user's unpleasant sound pressure from the content CT 23. Figure 21 is a diagram showing an example of acquiring unpleasant sound pressure. Note that explanations of points similar to those in the first acquisition example described above will be omitted as appropriate.

[0201] The information processing system 1C starts facial expression recognition using the front camera 293 of the terminal device 40, such as a smartphone, immediately before playing a sound. If the information processing system 1C detects a negative change in the user's facial expression before or after the playback of the sound stimulus, it acquires the sound pressure at that time as an unpleasant sound pressure.

[0202] Content CT23 shows an example of how unpleasant sound pressure is displayed when it is detected through the user's facial expression recognition. Content CT23 is a display screen (content) that includes information such as, "A sound was played for hearing measurement, but playback will be interrupted because a negative facial expression was detected at the start of sound playback."

[0203] It should be noted that the methods for acquiring unpleasant sound pressure described in the first to fifth acquisition examples above are merely examples, and the information processing system 1C may acquire unpleasant sound pressure by any method, not limited to the methods described in the first to fifth acquisition examples.

[0204] <4-1-2. Examples of processing when the unpleasant sound pressure is exceeded> From here, we will explain examples of processing when the unpleasant sound pressure is exceeded. For example, the information processing system 1C selects a sound stimulus when the sound pressure of the sound stimulus in the hearing test exceeds the unpleasant sound pressure. For example, when it becomes necessary to perform a test that exceeds this sound pressure during the ASSR test, the information processing system 1C can perform the test by processing as follows.

[0205] For example, if the sound pressure of the sound stimulus in the hearing test exceeds the user's uncomfortable sound pressure, the information processing system 1C executes a first corresponding process as a corresponding process for uncomfortable sounds, which changes the sound stimulus in the hearing test to a first sound stimulus that includes a natural sound or a second sound stimulus that is a composite sound containing multiple frequency components.

[0206] In the following explanation, the first corresponding process, which involves changing the auditory stimulus in the hearing test to a first sound stimulus containing natural sounds, will be referred to as the first corresponding process #1, and the first corresponding process, which involves changing the auditory stimulus in the hearing test to a second sound stimulus that is a composite sound containing multiple frequency components, will be referred to as the first corresponding process #2.

[0207] For example, if the sound pressure of the auditory stimulus in the hearing test exceeds the user's uncomfortable sound pressure, the information processing system 1C executes the first corresponding process #1 as a corresponding process related to the uncomfortable sound. In this case, the information processing system 1C executes the first corresponding process #1 using the method disclosed in European Patent Application Publication No. 4065204.

[0208] For example, if the sound pressure of the auditory stimulus in the hearing test exceeds the user's uncomfortable sound pressure, the information processing system 1C executes the first response process #2 as a response process related to the uncomfortable sound. In this case, the information processing system 1C executes the first response process #2 by the process shown in Figure 22. Figure 22 is a diagram showing an example of a process for countermeasures against uncomfortable sound pressure. For example, Figure 22 is a conceptual diagram of the generation of a musical piece in which the test frequency and modulation frequency are determined, and a partial extraction used for ASSR analysis.

[0209] The information processing system 1C performs the following processes using the processing PS1 shown in Figure 22: generating a musical piece in which the test frequency and modulation frequency are determined, and extracting the portion to be used for ASSR analysis. For example, the information processing system 1C generates music data DT12 based on MIDI (Musical Instruments Digital Interface) data DT11. Data DT12 is data that includes sound modulated with pitch as the test frequency based on the MIDI data DT11.

[0210] Furthermore, data DT13 is data showing the brainwaves of the subject (user) when they listened to the sound of data DT12, which is music data. The information processing system 1C obtains data DT14 to be used for ASSR analysis by extracting the brainwave data to be used for ASSR analysis from data DT13, which shows the brainwaves of the subject (user) when they listened to the sound of data DT12.

[0211] In Figure 22, the information processing system 1C extracts electroencephalogram (EEG) data corresponding to 1000 Hz, which is an example of a test frequency (the two sections separated by vertical lines corresponding to 1000 Hz in Figure 22), thereby obtaining data DT14 to be used for ASSR analysis. The information processing system 1C then performs ASSR analysis using the acquired data DT14.

[0212] Conventional ASSR test sounds are designed to elicit brainwave responses and are therefore unpleasant. However, the information processing system 1C uses test sounds generated based on musical data, making them enjoyable as music and reducing the degree of unpleasantness. Furthermore, if, for example, there is MIDI data of a song composed of a single instrument (corresponding to data DT11), the information processing system 1C generates data (corresponding to data DT12) that maps pitch to the frequency to be tested.

[0213] For example, the information processing system 1C can set the frequency to be tested for the note "Do" to 1000 Hz, and the note "Si" to a relative frequency of 943.9 Hz. For example, it is desirable that the timbre be a simple sound waveform such as a music box sound or a sine wave. The information processing system 1C also applies modulation such as 40 Hz amplitude modulation. The information processing system 1C then records the brainwave data (corresponding to data DT13) when the sound is heard. Since the information processing system 1C can determine whether the brainwave data is at the test frequency of 1000 Hz, it performs ASSR analysis using the extracted data (corresponding to data DT14).

[0214] Furthermore, for example, if the sound pressure of the sound stimulus in the hearing test exceeds the user's uncomfortable sound pressure, the information processing system 1C executes a second response process as a response process related to uncomfortable sounds, which changes the hearing test to a standard pure-tone audiometry test. In this case, the information processing system 1C notifies the user that a pure-tone audiometry test will be performed.

[0215] The information processing system 1C can reduce the possibility of users experiencing discomfort from the test sounds they hear when performing ASSR measurements on a daily basis by implementing processing related to countermeasures against unpleasant sound pressure, such as the first or second response processing described above.

[0216] <4-2. Fifth Embodiment (Measurement Parameter Control)> Next, an example of the configuration of the information processing system 1 according to the fifth embodiment (hereinafter sometimes referred to as "information processing system 1D") will be described using Figure 23. Figure 23 is a diagram showing an example of the configuration of the information processing system according to the fifth embodiment. Note that for configurations similar to those described in Figure 7, etc., the same reference numerals will be used, and the explanation will be omitted as appropriate.

[0217] In the example shown in Figure 23, the hearing aid 2D has an action / noise-measurement parameter dictionary 119. The action / noise-measurement parameter dictionary 119 is action / noise-measurement parameter correspondence information in which measurement parameters are associated with multiple types of actions and noises. The action / noise-measurement parameter dictionary 119 is stored in the memory unit 26.

[0218] Next, using Figure 24, an overview of the information processing performed by the information processing system 1 (information processing system 1D) including the hearing aid 2D will be described. Figure 24 is a flowchart of the processing procedure performed by the information processing system according to the fifth embodiment. Note that detailed explanations of points similar to those described in Figure 4, etc., will be omitted as appropriate.

[0219] First, the information processing system 1D starts acquiring sensor data (step S401). Step S401 is the same as step S101 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2D acquires sensor data detected by sensors in the sensor unit 29, sound collection unit 20, etc.

[0220] Furthermore, the information processing system 1D performs behavioral type recognition (step S402). Although a detailed explanation of step S402 is omitted as it is the same as step S102 in Figure 4, for example, the hearing aid 2D performs behavioral recognition processing using a behavioral recognition model that takes sensor data features as input and outputs information indicating the behavioral type corresponding to the input sensor data features.

[0221] Then, the information processing system 1D determines whether the recognized type of behavior exists in the behavior-sound stimulus mapping dictionary (step S403). Step S403 is the same as step S103 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2D uses the behavior-sound stimulus mapping dictionary DT1 shown in Figure 5 to determine whether the type of user behavior recognized in step S402 is included in the behavior-sound stimulus mapping dictionary DT1.

[0222] If the information processing system 1D determines that the recognized type of action exists in the action-sound stimulus mapping dictionary (step S403: Yes), it determines whether or not noise has been recognized (step S404). Step S404 is the same as step S105 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2D determines whether or not noise has been recognized based on detection by the sound sensor of the sensor unit 29.

[0223] If the information processing system 1D does not recognize noise (step S404: No), it determines the electroencephalogram (EEG) measurement parameters based on the recognized type of behavior. For example, if the hearing aid 2D determines that it does not recognize noise, the EEG sensor control unit 112 determines the EEG measurement parameters based on the recognized type of behavior (step S405). Then, the information processing system 1D starts EEG measurement (step S406). Although a detailed explanation of step S406 is omitted as it is the same as step S106 in Figure 4, for example, the hearing aid 2D starts EEG measurement using the EEG sensor 291 of the sensor unit 29.

[0224] Then, the information processing system 1D reads the audiogram data (step S407). Step S407 is the same as step S107 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2D obtains an audiogram from the memory unit 26, such as the audiogram data DT2 shown in Figure 6, which shows the results of the user's hearing test at that time.

[0225] The information processing system 1D then selects an auditory stimulus and starts playing it (step S408). Step S408 is the same as steps S104 and S108 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2D selects an auditory stimulus corresponding to the user's type of action and outputs the selected auditory stimulus from the output unit 22. After playing the auditory stimulus in step S408, the information processing system 1D returns to step S402 and repeats the process.

[0226] Furthermore, if the information processing system 1D determines that the recognized type of action does not exist in the action-sound stimulus correspondence dictionary (step S403: No), it executes the process in step S409. Also, if the information processing system 1D recognizes noise (step S404: Yes), it executes the process in step S409.

[0227] The information processing system 1D stops measuring brain waves (step S409). Step S409 is the same as step S109 in Figure 4, so a detailed explanation is omitted, but for example, the hearing aid 2D stops measuring brain waves using the brain wave sensor 291 of the sensor unit 29.

[0228] The information processing system 1D stores the electroencephalogram (EEG) data, the played-back sound stimulus data, and the recognized behavior type in the EEG-sound stimulus / behavior DB (step S410). Since step S410 is the same as step S110 in Figure 4, a detailed explanation is omitted.

[0229] The information processing system 1D stops playing the sound stimulus (step S411) and returns to step S402 to repeat the process. The hearing aid 2D stops outputting the sound stimulus from the output unit 22 and returns to step S402 to repeat the process.

[0230] <4-2-1. Configuration and Processing Example of Parameter Control> Starting from here, based on the above-described configuration and processing, some examples of the configuration and processing of parameter control in the information processing system 1D will be described.

[0231] For example, in the information processing system 1D, headphones 2E as shown in FIG. 25 may be used as the hearing aid 2D. FIG. 25 is a diagram showing an example of headphones equipped with an electroencephalogram sensor. The headphones 2E are equipped with an electroencephalogram sensor 291, and a plurality of electroencephalogram electrodes 294 are provided at the ear pad portions.

[0232] In the example of FIG. 25, among the two ear pads of the headphones 2E, the ear pad worn on the left ear side is provided with electroencephalogram electrode 294 １ , electroencephalogram electrode 294 ２ , electroencephalogram electrode 294 ３ , electroencephalogram electrode 294 ４ , electroencephalogram electrode 294 ５ , electroencephalogram electrode 294 ６ , electroencephalogram electrode 294 ７ , electroencephalogram electrode 294 ８ , that is, eight electroencephalogram electrodes 294 in total. Note that when each electroencephalogram electrode 294 is described without particular distinction, it is referred to as electroencephalogram electrode 294.

[0233] Similarly, in the example of FIG. 25, among the two ear pads of the headphones 2E, the ear pad worn on the right ear side is also provided with eight electroencephalogram electrodes 294. Furthermore, each electroencephalogram electrode 294 of the headphones 2E is attached with an amplifier, and the gain or the like can be adjusted for each electroencephalogram electrode 294.

[0234] Here, an example of parameter control in the case of the headphones 2E will be described with reference to FIG. 26. FIG. 26 is a diagram showing an example of parameter control. In FIG. 26, on the face of the subject (user), when the user wears the headphones 2E, circles are drawn at positions facing electroencephalogram electrode 294 １ to 294 ８ , and only some of electroencephalogram electrode 294 １ to 294 ８ are shown with reference signs.

[0235] The normal-time control MD11 in FIG. 26 corresponds to electroencephalogram electrodes 294 of the headphones 2E in normal time１ ~294 ８ An example of control is shown. In Figure 26, the hatched electroencephalogram (EEG) electrode 294 indicates that the EEG electrode 294 is used for detection. That is, in normal control MD11, the information processing system 1D controls the EEG electrode 294 of the headphones 2E. １ ~294 ８ It was decided to use all of them for detection, and electroencephalogram electrodes 294 １ ~294 ８ The detection is performed using all of the following.

[0236] In Figure 26, the sleep control MD12 is the electroencephalogram electrode 294 of the headphones 2E during sleep. １ ~294 ８ An example of control is shown. In this way, in sleep control MD12, the electroencephalogram electrodes 294 of headphones 2E １ ~294 ８ Of these, 294 electroencephalogram electrodes １ This indicates that only is used for detection. In other words, in sleep control MD12, the information processing system 1D uses the electroencephalogram electrode 294, which is the posterior-upper electrode of the headphones 2E. １ It was decided to use only the electroencephalogram electrodes 294 for detection. １ Detection is performed using only the electroencephalogram electrodes 294. In this case, headphones 2E are connected to the electroencephalogram electrodes 294. １ Only enable the EEG electrodes 294 １ Increase the amplifier gain.

[0237] For example, the information processing system 1D may use headphones 2F as a hearing aid 2D, as shown in Figure 27. Figure 27 shows an example of headphones equipped with an electroencephalogram (EEG) sensor. The headphones 2F are equipped with an EEG sensor 294, and multiple EEG electrodes 294 are provided in a total of three locations: one on the part located on the top of the head when worn (also called the "headband") and two on the parts worn on the ears (also called "earpieces").

[0238] In the example shown in Figure 27, the headband of the headphones 2F has electroencephalogram electrodes 294. ９ A feature is provided, and the earpiece worn on the left ear has an electroencephalogram electrode 294. １０ A feature is provided, and the earpiece worn on the right ear has an electroencephalogram electrode 294.１１ A feature is provided. When describing each electroencephalogram electrode 294 without making any particular distinction, it will be referred to simply as "electroencephalogram electrode 294". Each electroencephalogram electrode 294 of the headphones 2F is equipped with an amplifier, and the gain and other settings can be adjusted for each electroencephalogram electrode 294.

[0239] Here, an example of parameter control for headphones 2F will be explained using Figure 28. Figure 28 is a diagram showing an example of parameter control. In Figure 28, 294 ９ ~294 １１ Only a portion of it is shown with a symbol.

[0240] In Figure 28, the awake control MD21 is the electroencephalogram electrode 294 of the headphones 2F during wakefulness. ９ ~294 １１ An example of control is shown. In Figure 28, the hatched electroencephalogram (EEG) electrode 294 indicates that the EEG electrode 294 is used for detection. That is, in the wakefulness control MD21, the information processing system 1D uses the EEG electrode 294 of the headphones 2F. ９ It was decided to use only the electroencephalogram electrodes 294 for detection. ９ Detection is performed using only the electroencephalogram electrode 294 located at the top of the head, as the top of the head responds well when the person is awake. ９ It is decided to enable it. In this case, headphones 2F are connected to the electroencephalogram electrodes 294. ９ Only enable the EEG electrodes 294 ９ Increase the amplifier gain.

[0241] In Figure 28, the sleep control MD22 refers to the electroencephalogram electrodes 294 of the headphones 2F during sleep. ９ ~294 １１ An example of control is shown. In this way, in the sleep control MD22, the electroencephalogram electrodes 294 of headphones 2F ９ ~294 １１ Of these, 294 electroencephalogram electrodes １０ ,294 １１ This indicates that it is used for detection. In other words, in the sleep control MD22, the information processing system 1D uses the electroencephalogram electrodes 294 on the top of the headphones 2F. ９ Turn it off, and use EEG electrodes 294 １０ ,294 １１It was decided to use the electroencephalogram electrode 294 for detection. １０ ,294 １１ Detection is performed using this. In this case, headphones 2F are the electroencephalogram electrodes 294 which are intra-ear electrodes. １０ ,294 １１ Enable and EEG electrodes 294 １０ ,294 １１ Increase the amplifier gain.

[0242] The information processing system 1D controls parameters by changing the measurement parameters based on the behavior recognition results. For example, the information processing system 1D may control parameters by changing the measurement parameters using behavior measurement-corresponding information TB1 as shown in Figure 29. Figure 29 is a diagram showing an example of behavior measurement-corresponding information. The behavior measurement-corresponding information TB1 is information to which the control modes of electrodes, amplifier gain and sample rate are associated with the behavior and noise environment recognition results.

[0243] For example, when sleep is detected in the behavior recognition result, 90 Hz modulation is suitable, so the information processing system 1D controls at least one of the electrodes and amplifier gain when sleep is detected in the behavior recognition result. For example, in the case of a headphone type, the information processing system 1D may perform detection using two electroencephalogram electrodes located above and behind the head, or it may perform detection using an intra-ear sensor. Also, for example, when sleep is detected in the behavior recognition result, the information processing system 1D controls the sample rate to 200 Hz, which can detect 90 Hz.

[0244] Furthermore, the information processing system 1D controls at least one of the electrodes and amplifier gain to activate all electrodes when charging time is approaching and increased battery consumption poses little problem. For example, the information processing system 1D controls at least one of the electrodes and amplifier gain to activate all electrodes when the user is at rest, at home, and at night. Also, for example, the information processing system 1D controls the sample rate to 200 Hz, which is capable of detecting 90 Hz, when the user is at rest, at home, and at night.

[0245] Furthermore, the information processing system 1D controls at least one of the electrode and amplifier gain settings to activate all electrodes when the user is at rest, at home, and in the morning. For example, the information processing system 1D controls the sample rate to 100 Hz when the user is at rest, at home, and in the morning. The information processing system 1D also determines that the measurement is unsuitable if there is movement such as walking or if there is a lot of noise, and controls at least one of the electrode and amplifier gain settings to deactivate all electrodes.

[0246] In this way, the information processing system 1D controls at least one of the following based on the behavior recognition information: the selection of electrodes for electroencephalogram measurement in the auditory examination, the adjustment of the amplifier gain, and the setting of the sampling rate. The information processing system 1D performs control using behavior measurement correspondence information, in which parameters for controlling at least one of the following—electrode selection, amplifier gain adjustment, and sampling rate setting—are associated with each of the multiple behavior types.

[0247] When performing ASSR measurements with electroencephalogram (EEG) sensor-equipped devices for everyday use, the EEG electrodes used and the sample rate settings are important for power saving. Therefore, the information processing system 1D controls the measurement parameters of the EEG sensor based on the behavior and noise recognition results.

[0248] The information processing system 1D allows the electroencephalogram (EEG) sensor control unit 112 to receive the behavior / noise recognition results. By using the behavior / noise-measurement parameter dictionary 119, the system can refer to this dictionary based on the recognized behavior / noise recognition results and determine appropriate EEG measurement parameters. This process allows the information processing system 1D to reduce power consumption due to ASSR measurement and extend the battery life of the EEG data measurement device.

[0249] <4-3. Sixth Embodiment (Noise Reduction Processing)> In addition, noise reduction processing may be performed in the embodiments described above. An example of this processing will be described as information processing according to the sixth embodiment. In the following description, the case in which information processing system 1 performs the processing will be described as an example, but the noise reduction processing may be performed not only by information processing system 1, but also by information processing systems 1C, 1D, etc. Note that the same points as in the first to fifth embodiments described above will be omitted as appropriate.

[0250] An example of noise reduction processing performed by the information processing system 1 will be explained using Figure 30. Figure 30 is a flowchart showing the information processing procedure according to the sixth embodiment.

[0251] First, the information processing system 1 determines whether there are multiple data points with the same sound pressure, test frequency, and modulation frequency in the EEG-sound stimulus / behavior DB (step S501). If the information processing system 1 determines that there are no multiple data points with the same sound pressure, test frequency, and modulation frequency in the EEG-sound stimulus / behavior DB (step S501: No), it terminates the process.

[0252] Furthermore, if the information processing system 1 determines that there are multiple data points with the same sound pressure, test frequency, and modulation frequency in the EEG-sound stimulus / behavior DB (step S501: Yes), it executes a process to extract EEG data at the timing when the modulation phase of the sound stimulus becomes 0 (step S502).

[0253] Then, the information processing system 1 performs a process to combine the extracted electroencephalogram data (step S503). Then, the information processing system 1 performs an ASSR detection process based on spectral analysis (step S504). For example, the information processing system 1 performs the ASSR detection process based on spectral analysis using an ASSR detection method based on spectral analysis disclosed in the following literature.

[0254] (Reference #1) MS John., “Recording auditory steady-state responses in young infants,” Ear ​​and Hearing, 25(6), pp. 539-553.

[0255] Then, the information processing system 1 determines whether ASSR has been detected (step S505). If the information processing system 1 determines that ASSR has been detected (step S505: Yes), it updates the audiogram data, indicating that the sound pressure and test frequency of interest are "audible" (step S506). For example, if the hearing aid 2 determines that ASSR has been detected, it updates the audiogram data stored in the memory unit 26, indicating that the sound pressure and test frequency of interest are "audible".

[0256] Furthermore, if the information processing system 1 determines that ASSR has not been detected (step S505: No), it updates the audiogram data, indicating that the sound pressure and test frequency of interest are "not audible" (step S507). For example, if the hearing aid 2 determines that ASSR has not been detected, it updates the audiogram data stored in the memory unit 26, indicating that the sound pressure and test frequency of interest are "not audible".

[0257] For example, if the information processing system 1 has multiple data points with the same sound pressure, test frequency, and modulation frequency in the EEG-sound stimulus / behavior DB, it performs noise reduction processing as shown in Figure 31. Figure 31 is a diagram showing an example of noise reduction processing. For example, Figure 31 is a conceptual diagram of a process that improves the SNR (Signal-to-Noise Ratio) by accumulating EEG data under the same stimulus sound conditions.

[0258] The information processing system 1 performs a process to combine electroencephalogram (EEG) data using the process PS2 shown in Figure 31. For example, the information processing system 1 acquires data DT31, which is EEG data at the time of auditory stimulation, using auditory stimulation data from 10:00 on December 24, as shown in data DT21. Also, for example, the information processing system 1 acquires data DT32, which is EEG data at the time of auditory stimulation, using auditory stimulation data from 16:00 on December 25, as shown in data DT22.

[0259] Information processing system 1 generates data DT41, which is brainwave data extracted from data DT31, which is brainwave data at the time of sound stimulus listening, using sound stimulus data from 10:00 on December 24th, at the timing when the modulation phase of the sound stimulus becomes 0.

[0260] Furthermore, the information processing system 1 generates data DT42, which is brainwave data extracted from data DT32, which is brainwave data at the time of sound stimulus listening, using sound stimulus data from 16:00 on December 25th, at the timing when the modulation phase of the sound stimulus becomes 0.

[0261] The information processing system 1 then generates data DT43, which is combined electroencephalogram data for ASSR analysis. For example, the information processing system 1 generates data DT43, which is electroencephalogram data that combines data DT41 and data DT42.

[0262] Thus, when there are multiple data points with the same sound pressure, test frequency, and modulation frequency in the EEG-sound stimulus / behavior DB, the information processing system 1 combines the data in a time series so that the modulation phase is continuous. The information processing system 1 can also calculate relative weights for each measurement. For example, the information processing system 1 may assign higher weights to less noise or to behaviors that are closer to rest.

[0263] Furthermore, the information processing system 1 may determine the weights based on an electroencephalogram (EEG) signal quality index. The signal quality index may be, for example, the impedance of the EEG electrodes or the magnitude of artifacts such as electromyography (EMG). The information processing system 1 may also calculate the SNR of the ASSR.

[0264] As described above, the information processing system 1 performs noise reduction processing based on electroencephalogram (EEG) data obtained from multiple hearing tests using stimulus sounds of the same frequency and sound pressure. For example, the information processing system 1 performs noise reduction processing by averaging the EEG data. For example, the information processing system 1 performs noise reduction processing by dividing the sum of the values ​​in multiple EEG data by the number of EEG data. For example, when the information processing system 1 synthesizes multiple EEG data, it synthesizes the multiple EEG data by dividing the sum of the values ​​in the part to be synthesized by the number of EEG data.

[0265] In daily life, the ideal conditions for ASSR measurement are those where the user is at rest with minimal movement and low noise levels. However, it is unlikely that such conditions can be maintained continuously. Therefore, the information processing system 1 accumulates and combines electroencephalogram data under the same stimulus sound conditions to simulate long-term measurement, thereby suppressing random noise through additive averaging in ASSR analysis.

[0266] In the execution of noise reduction processing in the sixth embodiment, it is assumed that there are multiple data points with the same sound pressure, test frequency, and modulation frequency recorded in the EEG-sound stimulus / behavior DB. If there are multiple data points with the same sound pressure, test frequency, and modulation frequency, the information processing system 1 generates multiple EEG data points (corresponding to data DT41 and DT42 in Figure 31) by executing a process to extract EEG data, using the time when the modulation phase of the sound stimulus presentation is constant (for example, phase 0) as the starting and ending points.

[0267] The information processing system 1 can generate electroencephalogram (EEG) data (corresponding to data DT43 in Figure 31) that can be treated as a pseudo-long-term measurement by combining multiple extracted EEG data in a time series. The information processing system 1 can suppress random noise by averaging by performing ASSR detection processing using the combined EEG data (corresponding to data DT43 in Figure 31).

[0268] As a result, the information processing system 1 can theoretically improve the SNR (Signal-to-Noise Ratio) by a factor of √N (square root N) when the number of measurements is N. Through the above processing, the information processing system 1 can improve the accuracy of ASSR detection even for electroencephalograms measured around the ear, which are easy to use in everyday life but have a poor SNR, such as headphones 2E in Figure 25. As a result, the information processing system 1 can improve the accuracy of audiogram data.

[0269] In the various embodiments described above, a hearing aid was used as an example for illustrative purposes. However, any device capable of outputting sound for hearing tests in an information processing system is applicable, not limited to a hearing aid. For example, the device that outputs sound for hearing tests in an information processing system is not limited to a hearing aid; it could be any audio device worn by the user, such as headphones or earphones.

[0270] Furthermore, this technology can also take the following configurations: (1) An information processing method implemented by one or more processors, comprising: acquiring behavioral recognition information relating to the behavior of a user who is the subject of a hearing test; and controlling the function of the hearing test for the user based on the acquired behavioral recognition information. (2) The information processing method according to (1) for determining whether or not to perform a hearing test for the user based on the behavioral recognition information. (3) The information processing method according to (1) or (2) for selecting the mode of the hearing test based on the behavioral recognition information. (4) The information processing method according to (2) for acquiring the behavioral recognition information of the user who is the subject of the hearing test which is an ASSR (Auditory Steady-State Response), and determining whether or not to perform the ASSR as the hearing test for the user based on the behavioral recognition information. (5) The information processing method according to (2) or (4) for determining whether or not to output a sound for the hearing test based on the behavioral recognition information. (6) The information processing method according to (2), (4), or (5) which determines whether or not to output the sound for the hearing test based on environmental information relating to the user. (7) The information processing method according to (6) which determines whether or not to output the sound for the hearing test based on environmental information relating to noise. (8) The information processing method according to any one of (2), (4) to (7) which, if the control unit determines to perform the hearing test, selects the mode of the hearing test. (9) The information processing method according to (8) which selects an output sound stimulus which is a sound stimulus to be output for the hearing test. (10) The information processing method according to (9) which selects the output sound stimulus using behavior sound stimulus correspondence information which associates a sound stimulus to be played with each of a plurality of behavior types. (11) The information processing method according to (10) which selects the output sound stimulus from among the plurality of behavior types included in the behavior sound stimulus correspondence information which corresponds to the behavior type of the user indicated by the behavior recognition information.(12) The information processing method according to any one of (9) to (11), wherein the output frequency is the frequency for the hearing test and the output sound pressure is the sound pressure. (13) The information processing method according to (12), wherein the output frequency and the output sound pressure are selected using the user's audiogram. (14) The information processing method according to (13), wherein the output frequency is selected from among the frequencies included in the user's audiogram that did not elicit a response. (15) The information processing method according to (13) or (14), wherein the output sound pressure is selected from among the sound pressures included in the user's audiogram that did not elicit a response and were low in sound pressure. (16) The information processing method according to any one of (2), (4) to (15), wherein the (17) An information processing method according to any one of (1) to (16) wherein the information processing method according to (17) wherein, if the hearing test outputs a sound above a predetermined standard, the information processing method according to (17) wherein the information processing method according to (17) wherein the information processing method according to (19) wherein the information processing method according to (17) or (18) wherein the information processing method according to (17) or (18) wherein the information processing method according to any one of (17) to (19) wherein the information processing method according to is wherein the information processing method according to (17) to (19) is wherein the information processing method according to (17) to (19) is wherein the information processing method according to (17)(23) An information processing method according to any one of (1) to (22), wherein an information processing method according to any one of (1) to (22) is obtained, wherein an information processing method according to any one of (1) to (22) is obtained, wherein an information processing method according to any one of (1) to (22) is obtained, wherein an information processing method according to any one of (1) to (22) is obtained, wherein an information processing method according to any one of (23) is obtained, wherein an information processing method according to any one of (1) to (24 (26) The information processing method according to (25), wherein at least one of the following is controlled by using behavior measurement corresponding information, in which a parameter for controlling at least one of the following is controlled for each of a plurality of behavior types: electrode selection, amplifier gain adjustment, and sampling rate setting. (27) The information processing method according to any one of (1) to (26), wherein noise reduction processing is performed based on electroencephalogram data obtained from multiple hearing tests using stimulus sounds of the same frequency and sound pressure. (28) The information processing method according to (27), wherein noise reduction processing is performed by averaging the electroencephalogram data.

[0271] 1 Information processing system 2 Hearing aid 20 Sound collection unit 21 Signal processing unit 22 Output unit 23 Battery 24 Connection unit 25 Communication unit 26 Memory unit 27 Control unit 271 Acquisition unit 272 Judgment unit 273 Selection unit 274 Notification unit 28 Communication unit 29 Sensor unit 291 Electroencephalogram sensor 292 Acceleration sensor 40 Terminal device 41 Input unit 42 Communication unit 43 Output unit 44 Display unit 45 Memory unit 46 Control unit

Claims

1. An information processing method implemented by one or more processors, comprising: acquiring behavioral recognition information relating to the behavior of a user subject to a hearing test; and controlling the functions of the hearing test applied to the user based on the acquired behavioral recognition information.

2. The information processing method according to claim 1, which determines whether or not to perform a hearing test on the user based on the behavior recognition information.

3. The information processing method according to claim 1, wherein the mode of the auditory test is selected based on the behavior recognition information.

4. The information processing method according to claim 2, which acquires the behavioral recognition information of the user who is the subject of the auditory examination which is an ASSR (Auditory Steady-State Response), and determines whether or not to perform the ASSR as the auditory examination for the user based on the behavioral recognition information.

5. The information processing method according to claim 2, wherein when it is determined that the hearing test should be performed, the mode of the hearing test is selected.

6. The information processing method according to claim 5, which selects an output sound stimulus, which is a sound stimulus to be output for the aforementioned hearing test.

7. The information processing method according to claim 6, which selects the output sound stimulus using behavioral sound stimulus correspondence information, in which sound stimuli to be played are associated with each of a plurality of behavioral types.

8. The information processing method according to claim 7, wherein, from among the plurality of behavior types included in the behavior sound stimulus corresponding information, a sound stimulus corresponding to a behavior type that corresponds to the type of user behavior indicated by the behavior recognition information is selected as the output sound stimulus.

9. The information processing method according to claim 5, which selects an output frequency, which is the frequency for the hearing test, and an output sound pressure, which is the sound pressure.

10. The information processing method according to claim 1, wherein if the hearing test produces a sound above a predetermined standard, the user is notified of information indicating that the hearing test will produce a sound.

11. The information processing method according to claim 1, wherein the user is notified of information that prompts the user to be in a state suitable for the hearing test.

12. The information processing method according to claim 1, which notifies the user of information regarding the progress of the hearing test.

13. The information processing method according to claim 1, which acquires unpleasant sound condition information indicating the conditions of sounds that the user finds unpleasant, and performs corresponding processing related to unpleasant sounds if the sound stimulus of the hearing test matches the conditions of the unpleasant sound condition information.

14. The information processing method according to claim 13, wherein the method obtains unpleasant sound condition information indicating the conditions of sound pressure that the user finds unpleasant, and if the sound pressure of the sound stimulus of the hearing test matches the sound pressure conditions of the unpleasant sound condition information, the method performs either a first corresponding process, which is changing the sound stimulus of the hearing test to a first sound stimulus including a natural sound or a second sound stimulus which is a composite sound including multiple frequency components, or a second corresponding process, which is changing the hearing test to a standard pure-tone audiometry test, as the corresponding process for the unpleasant sound.

15. The information processing method according to claim 1, which controls at least one of the following based on the behavior recognition information: selecting electrodes for electroencephalogram measurement in the auditory examination, adjusting the gain of the amplifier, and setting the sampling rate.

16. The information processing method according to claim 15, which controls at least one of the following: electrode selection, amplifier gain adjustment, and sampling rate setting, using behavior measurement corresponding information to which a parameter for controlling at least one of the following: electrode selection, amplifier gain adjustment, and sampling rate setting is associated with each of a plurality of behavior types.

17. The information processing method according to claim 1, which performs noise reduction processing based on electroencephalogram data obtained from multiple hearing tests using stimulus sounds of the same frequency and sound pressure.

18. The information processing method according to claim 17, wherein the noise reduction process is performed by averaging the electroencephalogram data.

19. An information processing system implemented by one or more processors, which acquires behavioral recognition information relating to the behavior of a user subject to a hearing test, and controls the function of the hearing test for the user based on the acquired behavioral recognition information.

20. An information processing program that causes a computer to perform the following actions: acquire behavioral recognition information regarding the actions of a user who is the subject of a hearing test; and control the functions of the hearing test for the user based on the acquired behavioral recognition information.

Images