Information Processing Method

Abstract

To execute processing depending on whether a user has hearing loss when a user is wearing a device that emits sound.SOLUTION: An information processing method causes a computer to execute outputting a result of determination as to whether a user has hearing loss according to an operation history indicating an operating time of a device that is worn by the user and that emits sound, and biometric information of the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing method. [Background technology]

[0002] Patent Document 1 describes a reporting device that normally collects ambient sounds with a built-in microphone and outputs the collected sounds to earphones.

[0003] Patent document 2 describes a hearing aid that amplifies sounds collected by a hearing aid microphone placed in the external world, emits the amplified sounds and test sounds, and stores test results based on the test sounds collected by a measurement microphone placed in the ear canal.

[0004] Patent Document 3 describes a hearing aid that amplifies a hearing aid sound by a first amplification factor and amplifies a search sound by a second amplification factor that is greater than the first amplification factor.

[0005] Patent Document 4 describes a headphone device that allows a user to listen to an audio signal source while also listening to ambient sounds.

[0006] Patent document 5 describes a hearing aid having a hearing aid processing unit that applies hearing aid processing to ambient sounds collected by a sound collection unit, a specific sound generation unit that generates a predetermined signal, and a selection unit that selects either the sound that has been processed by the hearing aid processing unit or the sound generated by the specific sound generation unit based on the result of determining whether the main body of the hearing aid is worn on the ear, and outputs the selected sound to a sound output unit. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2004-185563 A [Patent Document 2] JP 2010-200258 A [Patent Document 3] International Publication No. 2009 / 125579 [Patent Document 4] JP 2005-295175 A [Patent Document 5] International Publication No. 2010 / 061539 Summary of the Invention [Problem to be solved by the invention]

[0008] An object of the present invention is to execute processing according to a determination of whether or not a user has hearing loss when the user is wearing a device that emits sound. [Means for solving the problem]

[0010] Claim 1 The invention relates to an information processing method that causes a computer to execute processing including a step of outputting a result of a judgment as to whether or not the user has hearing loss based on a behavioral history related to the user's lifestyle habits and biometric information of the user. Effect of the Invention

[0013] Claim 1 To According to this invention, when a user is wearing a device that emits sound, it is possible to determine whether or not the user has hearing loss, and to execute processing according to the determination of whether or not the user has hearing loss.

[0014] Claim 1 To According to this invention, it is possible to determine with higher accuracy whether a user has hearing loss, compared to a case in which it is determined whether a user has hearing loss based only on the user's biological information. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a configuration of an information processing device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing the configuration of the device. [Figure 4]FIG. 13 illustrates an operation history management table. [Diagram 5] This is information showing a user attribute management table. [Figure 6] FIG. 13 is a diagram illustrating a user action history management table. [Figure 7] FIG. 1 is a diagram showing levels of hearing loss. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] An information processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 shows an example of the configuration of the information processing system according to this embodiment.

[0017] The information processing system according to this embodiment includes an information processing device 10, one or more biological information measuring devices 12, and one or more devices 14. The number of devices shown in Fig. 1 is merely an example. The information processing system may include devices (e.g., terminal devices, servers, etc.) other than the devices shown in Fig. 1.

[0018] Each device included in the information processing system is configured to communicate with other devices. The communication may be wired communication using a cable, or wireless communication. That is, each device may be physically connected to other devices by a cable and transmit and receive information to and from each other, or may transmit and receive information to and from each other by wireless communication. For example, short-range wireless communication or Wi-Fi (registered trademark) is used as wireless communication. Wireless communication of other standards may also be used. For example, short-range wireless communication is Bluetooth (registered trademark), RFID (Radio Frequency Identifier), NFC, etc. Each device may communicate with other devices via a communication path such as a LAN (Local Area Network) or the Internet.

[0019] The information processing device 10 is, for example, a personal computer (hereinafter referred to as "PC"), a tablet PC, a smartphone, a mobile phone, or other devices (for example, a server, a robot, or other equipment). The information processing device 10 may be a terminal device that can be carried by a person (for example, a tablet PC, a smartphone, a mobile phone, etc.), a device that is installed on a table or the like and used, a mobile device (for example, a self-propelled device), a device that performs work, or a device that can interact with a person. For example, the information processing device 10 may be a smart speaker having a communication function, a microphone, and a speaker, or a robot that interacts with a person and works. The information processing device 10 may be a device equipped with artificial intelligence (AI).

[0020] The biological information measuring device 12 is a device configured to measure biological information of a user. For example, the biological information measuring device 12 has various sensors, electrodes, etc., and measures biological information of a user. When multiple biological information measuring devices 12 are used, each biological information measuring device 12 may measure different types of biological information. Some or all of the multiple biological information measuring devices 12 may measure the same type of biological information. Furthermore, the biological information measuring device 12 may be configured to measure one type of biological information, or may be configured to measure multiple types of biological information.

[0021] The vital information measuring device 12 transmits the measured vital information to the information processing device 10. The vital information measuring device 12 may transmit the vital information to the information processing device 10 every time the vital information is measured, may store the vital information and transmit the vital information to the information processing device 10 at predetermined time intervals, or may transmit the vital information to the information processing device 10 at a timing designated by a user. In addition, the vital information measuring device 12 may receive vital information measured by another vital information measuring device 12 from the other vital information measuring device 12, and transmit the vital information measured by the vital information measuring device 12 and the vital information measured by the other vital information measuring device 12 to the information processing device 10.

[0022] The biological information measuring device 12 may analyze biological information measured by itself or another biological information measuring device 12, and transmit information indicating the result of the analysis to the information processing device 10. For example, the biological information measuring device 12 may include a processor, and the processor may analyze the biological information. Of course, the analysis may be performed by the information processing device 10 or another device.

[0023] The biological information measuring device 12 may include a battery and be driven by power supplied from the battery, or may be driven by power supplied from another device (such as the information processing device 10).

[0024] The biological information measuring device 12 may be installed on a user. For example, the biological information measuring device 12 may be a wearable device that is worn by a user to measure biological information. To give a specific example, the biological information measuring device 12 may be a device worn on the user's head (for example, the forehead, etc.), a hearable device worn on the user's ear (for example, earphones, headphones, hearing aids, etc.), a device worn on the user's arm, hand, wrist, finger, etc. (for example, a watch-type device such as a smart watch, etc.), a device worn around the user's neck, a device worn on the user's torso (for example, the abdomen, chest, etc.), or a device worn on the user's lower limbs (for example, the thighs, lower legs, knees, feet, ankles, etc.). The biological information measuring device 12 may be a health tool worn on the user's arm, hand, torso, or lower limbs. The biological information measuring device 12 may be worn on other parts of the body. In addition, the biological information measuring device 12 may be worn on each of a plurality of parts of the body.

[0025] Biometric information is various physiological information and anatomical information emitted from a human being, which is a living body. The concept of biological information includes, for example, information indicating brain activity (for example, brain waves, cerebral blood flow, brain magnetic field signals, etc.), information indicating pulse rate, myoelectric information such as myoelectric waveform, information on saliva (for example, information indicating saliva volume), information indicating pulse wave, information indicating blood pressure, information indicating blood flow, information indicating pulse, information indicating heart rate, information indicating electrocardiogram waveform, information indicating eye movement, information indicating body temperature, information indicating sweat amount, information indicating gaze, voice information, and information indicating human movement. Information specified by a biomarker may also be used as biological information. These are merely examples of biological information, and other physiological information and anatomical information may also be used as biological information. The biological information measuring device 12 may measure one of these biological information, or may measure multiple biological information.

[0026] The concept of bioinformation also includes bioelectric potential information that indicates an electric potential measured from a living organism. The concept of bioelectric potential information includes, for example, an electroencephalogram that is a measurement result of a microcurrent generated with brain activity, an electrocardiogram that is a measurement result of a microcurrent generated with heartbeat, an electromyogram that is a measurement result of a microcurrent generated with muscle activity, and a skin potential that is a measurement result of a microcurrent generated on the skin. These are merely examples of bioelectric potential information, and other bioelectric potential information may also be used.

[0027] The information processing device 10 receives bioinformation from the bioinformation measuring device 12, and performs the following operations: analyzing the bioinformation, storing the bioinformation, outputting the bioinformation, storing information indicating the results of the analysis of the bioinformation, and outputting information indicating the results of the analysis of the bioinformation. Of course, the analysis of the bioinformation may be performed by the bioinformation measuring device 12 or another device. Outputting the bioinformation means, for example, displaying the bioinformation or outputting the bioinformation as audio information. Outputting information indicating the results of the analysis of the bioinformation means, for example, displaying information indicating the results of the analysis or outputting the results of the analysis as audio information. The information processing device 10 may transmit the bioinformation or information indicating the results of the analysis to another device.

[0028] The information processing device 10 may include one or more biological information measuring devices 12. That is, one or more biological information measuring devices 12 may be incorporated into the information processing device 10 to configure one device. The entire information processing device 10 including one or more biological information measuring devices 12 may be worn by a user to measure biological information. That is, the information processing device 10 may be a wearable device. For example, the information processing device 10 may be a device worn on the user's head (e.g., forehead, etc.), a hearable device worn on the user's ear (e.g., earphones, headphones, hearing aids, etc.), a device worn on the user's arm, hand, wrist, finger, etc. (e.g., a watch-type device such as a smart watch), a device hung around the user's neck, a device worn on the user's torso (e.g., abdomen, chest, etc.), or a device worn on the user's lower limbs (e.g., thighs, lower legs, knees, feet, ankles, etc.). The information processing device 10 may be a health tool worn on the user's arm, hand, torso, or lower limbs. The information processing device 10 may be attached to a part other than these.

[0029] The information processing device 10 and the biological information measuring device 12 may be separate devices. For example, the information processing device 10 may be a device such as a robot, a smart speaker, or a server, and the biological information measuring device 12 may be a wearable device attached to a user.

[0030] The device 14 has a speaker and has a function of emitting sound. The device 14 may further have a microphone and have a sound collection function. For example, the device 14 is worn by a user when used. One or more devices 14 are worn by a user. For example, the device 14 is a hearable device (e.g., earphones, headphones, hearing aids, etc.) worn in the user's ear. The hearing aid may be a digital hearing aid or an analog hearing aid. The device 14 may be worn in a part of the user other than the ear. The device 14 may be a speaker or the like that is used without being worn by a user.

[0031] One or more biological information measuring devices 12 may be incorporated into the device 14 to configure a single device. The entire device 14 including the one or more biological information measuring devices 12 may be worn by a user. For example, the entire device 14 including the one or more biological information measuring devices 12 may be a hearable device.

[0032] Furthermore, one device may be configured by the information processing device 10, one or more biological information measuring devices 12, and the device 14. The entire device configured by the information processing device 10, one or more biological information measuring devices 12, and the device 14 may be worn by a user. For example, the entire device configured by the information processing device 10, one or more biological information measuring devices 12, and the device 14 may be a hearable device.

[0033] The configuration of the information processing device 10 will be described in detail below with reference to Fig. 2. Fig. 2 shows an example of the configuration of the information processing device 10.

[0034] The information processing device 10 includes, for example, a communication device 16, a UI 18, a storage device 20, and a processor 22. The information processing device 10 may include components other than these.

[0035] The communication device 16 is a communication interface and has a function of transmitting data to other devices and a function of receiving data transmitted from other devices. The communication device 16 may have a wireless communication function or a wired communication function. The communication device 16 may communicate with other devices by using, for example, short-distance wireless communication, or may communicate with other devices via a communication path such as a LAN or the Internet. For example, the communication device 16 receives biological information transmitted from the biological information measuring device 12. The communication device 16 may transmit control information for controlling the operation of the biological information measuring device 12 to the biological information measuring device 12. In addition, the communication device 16 may transmit control information for controlling the operation of the device 14 to the device 14, or may receive information transmitted from the device 14.

[0036] The UI 18 is a user interface and includes at least one of a display device and an operation device. The display device is a liquid crystal display, an EL display, etc. The operation device is a keyboard, an input key, an operation panel, etc. The UI 18 may be a UI such as a touch panel that combines a display device and an operation device.

[0037] The storage device 20 is a device that configures one or more storage areas for storing data. The storage device 20 is, for example, a hard disk drive, various types of memory (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more storage devices 20 are included in the information processing device 10.

[0038] The storage device 20 stores operation history management information, user attribute management information, and user action history management information.

[0039] The operation history management information is information for managing the operation history of the device 14. For example, the operation history of the device 14 is a history regarding the generation of sound from the speaker of the device 14. The operation history of the device 14 is, for example, at least one of the total operation time of the device 14, the operation time for each volume, the volume volume history, the sound range setting history (for example, the degree of amplification when the treble or bass range is amplified), and the reproduction history of sound data (for example, music data, etc.) reproduced by the device 14.

[0040] For example, device identification information, which is information for identifying the device 14, and operation history information, which is information indicating the operation history of the device 14, are associated with each other and registered in the operation history management information for each device 14. The device identification information is, for example, the ID, name, model number, or address (e.g., MAC address, IP address, etc.) of the device 14.

[0041] For example, when the device 14 operates, operation history information indicating the operation history is transmitted from the device 14 to the information processing device 10 and registered in the operation history management information. The operation history information may be stored in the device 14, and the operation history information may be transmitted from the device 14 to the information processing device 10 and registered in the operation history management information. For example, when the information processing device 10 requests the device 14 to obtain operation history information, the device 14 may transmit operation history information to the information processing device 10 in response to the request. Furthermore, the operation history information of each device 14 may be managed by a device such as a server, and the information processing device 10 may obtain the operation history information of each device 14 from the device such as the server.

[0042] The operation history of the device 14 may be managed for each user. For example, for each user, user identification information, which is information for identifying the user, device identification information of the device 14 used by the user, and operation history information, which is the operation history of the device 14 and indicates the operation history due to the use by the user, are linked and registered in the operation history management information. The user identification information is, for example, the user's name, the user's ID, an email address, or a user account.

[0043] The operation history management information may not be stored in the storage device 20, or may be stored in the storage device 20 and also in a device other than the information processing device 10 (such as the device 14 or a server).

[0044] The user attribute management information is information for managing the attributes of a user, which may be, for example, at least one of gender, age, medical history, and occupational history.

[0045] For example, for each user, user identification information and user attribute information, which is information indicating the attributes of the user, are associated with each other and registered in the user attribute management information.

[0046] The user may manually input the user identification information and the user attribute information into the information processing device 10, or the information processing device 10 may acquire the user identification information and the user attribute information from a device (e.g., a server) that manages the user identification information and the user attribute information.

[0047] The user attribute management information may not be stored in the storage device 20, or may be stored in the storage device 20 and also in a device other than the information processing device 10 (such as the device 14 or a server).

[0048] The user behavior history management information is information for managing the user's behavior history. The user's behavior history is, for example, at least one of drinking history, smoking history, eating and drinking history, exercise history, medication history, medicine taking history, sleep history, and other lifestyle habit history.

[0049] For example, for each user, user identification information and behavior history information, which is information indicating the history of the user's behavior, are associated with each other and registered in user behavior history management information.

[0050] The user may manually input the user identification information and the behavior history information into the information processing device 10, or the information processing device 10 may acquire the user identification information and the behavior history information from a device (e.g., a server) that manages the user identification information and the behavior history information.

[0051] In addition, the user behavior history management information may not be stored in the storage device 20, or may be stored in the storage device 20 and also in another device other than the information processing device 10 (for example, a device such as the device 14 or a server).

[0052] The processor 22 is configured to control the operation of each unit of the information processing device 10. The processor 22 may include a memory.

[0053] The processor 22 is also configured to execute processing according to the operation history of the device 14 and the result of the determination as to whether or not the user's biological information satisfies the condition of hearing loss.

[0054] When the biological information of the user is measured by the biological information measuring device 12, the biological information is transmitted from the biological information measuring device 12 to the information processing device 10. The processor 22 receives the biological information and determines whether or not the biological information satisfies the condition of hearing loss.

[0055] The processor 22 determines whether the user has hearing loss based on the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss, and executes processing according to the result of the determination. The hearing loss is, for example, conductive hearing loss, sensorineural hearing loss, or mixed hearing loss.

[0056] The hearing loss condition is a condition for determining whether or not the user has hearing loss based on biological information.

[0057] For example, the condition of hearing loss is a condition for determining whether or not the user has hearing loss based on an electroencephalogram. As a specific example, the processor 22 determines whether or not the user's electroencephalogram satisfies the condition of hearing loss by an Auditory Brainstem Response (ABR) or an Auditory Steady-State Response (ASSR). The processor 22 may notify the user of the result of the determination as to whether or not the user's biological information satisfies the condition of hearing loss. For example, the processor 22 may display information indicating the result of the determination on a display device, or may generate a sound representing the information from a speaker.

[0058] The auditory brainstem response test is a test method that can obtain responses originating from the cochlear nerve and the brainstem auditory pathway. When a person hears a sound, a response with peaks occurs from the cochlear nerve to the brainstem, pons, and inferior colliculus. Under normal conditions, approximately five peaks appear. The peaks correspond to the cochlear nerve (first wave), the cochlear nucleus (second wave), the olivary nucleus (third wave), the lateral lemniscus (fourth wave), and the inferior colliculus (fifth wave). If the peaks of these waves appear late or if there is a disturbance in the waveform (for example, no peaks appear), it is determined that the user may have hearing loss.

[0059] The auditory steady-state response test is a test method that uses steady-state evoked responses to record electrical potentials from the brain in response to sound stimuli. The auditory steady-state response test can be used to test a wide range of frequencies, from low to high.

[0060] In addition, the processor 22 may analyze the brain waves measured from the user while the device 14 is emitting sound, and determine whether or not the user's brain waves meet the conditions for hearing loss based on the results of the analysis. For example, if the user's brain waves indicate that it is difficult to hear sounds, the processor 22 determines that the user's brain waves meet the conditions for hearing loss.

[0061] The processor 22 may determine whether or not biological information other than the brainwave satisfies the condition of hearing loss. For example, the processor 22 may determine that the user's biological information satisfies the condition of hearing loss when the blood flow in the ear is poor or the blood pressure is not normal.

[0062] Even if the user's biometric information meets the conditions for hearing loss (i.e., even if the user is determined to be hearing-impaired based on the user's biometric information), it does not necessarily mean that the user is actually hearing-impaired. For example, even if the user is determined to be hearing-impaired by an auditory brainstem response test or an auditory steady-state response test, it does not necessarily mean that the user is actually hearing-impaired. Also, even if the user's biometric information does not meet the conditions for hearing loss (i.e., even if the user is determined not to be hearing-impaired based on the user's biometric information), the user may actually be hearing-impaired.

[0063] In this embodiment, the processor 22 determines whether the user has hearing loss based on the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss. In this way, it is possible to determine whether the user has hearing loss with higher accuracy than when determining whether the user has hearing loss using only the user's biological information. For example, the device 14 is a hearable device, and whether the user has hearing loss is determined based on the operation history of the device 14 and the user's biological information.

[0064] The processor 22 may determine the degree to which the user's biometric information satisfies the condition of hearing loss. That is, the processor 22 may determine the degree of hearing loss (e.g., the level of hearing loss) based on the user's biometric information. Hereinafter, the level of hearing loss determined based on the user's biometric information will be referred to as the "first level." For example, the processor 22 determines the first level of hearing loss by an auditory brainstem response test or an auditory steady-state response test. The processor 22 may notify the user of the first level. For example, the processor 22 may display information indicating the first level on a display device, or may generate a sound representing the information from a speaker.

[0065] The processor 22 may also determine the degree of hearing loss (e.g., the level of hearing loss) of the user based on the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss. For example, the processor 22 may determine the degree of hearing loss (e.g., the level of hearing loss) of the user based on the operation history of the device 14 and the degree of hearing loss (e.g., the first level) determined based on the user's biological information. Hereinafter, the level of hearing loss determined based on the operation history of the device 14 and the first level of hearing loss determined based on the user's biological information will be referred to as the "second level". The processor 22 may notify the user of the second level. For example, the processor 22 may display information indicating the second level on a display device, or may generate a sound representing the information from a speaker.

[0066] A specific example of processing according to the result of the hearing loss determination will be described below.

[0067] For example, the process according to the result of the hearing loss determination is a process of outputting a warning. That is, the processor 22 may be configured to output a warning according to the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss.

[0068] Outputting the warning is, for example, at least one of displaying information indicating the warning on a display device (e.g., the display device of the UI 18 or the display device of the user's terminal device) and issuing a warning sound from a speaker (e.g., the speaker of the device 14 or the speaker of the information processing device 10, etc.). For example, the processor 22 may cause a message such as "The volume is too high, the volume should be turned down, the user may be hard of hearing, or the user is hard of hearing" to be displayed on the display device, or may cause a sound representing the message to be issued from the speaker.

[0069] For example, when it is determined that the user has hearing loss based on the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss, the processor 22 outputs a warning indicating that the user has hearing loss. When it is determined that the user may have hearing loss, the processor 22 may output a warning indicating that the user may have hearing loss. The processor 22 may also output a warning indicating the level of hearing loss (e.g., first level or second level).

[0070] Furthermore, if the volume of device 14 is constantly set to above the threshold (for example, if the volume is set to above the threshold for a predetermined period of time or more, or if the volume is set to above the threshold a predetermined number of times or more), processor 22 may output a warning indicating that the volume is high or that the volume should be turned down. If the volume of device 14 is set to above the threshold only once, processor 22 may not output a warning.

[0071] As another example, the process according to the result of the hearing loss determination is a process of controlling the volume of device 14. That is, processor 22 may be configured to control the volume of device 14 based on the operation history of device 14 and the result of the determination of whether the user's biometric information satisfies the condition of hearing loss.

[0072] Controlling the volume of device 14 may include, for example, increasing the volume, decreasing the volume, controlling the output of a particular range of sounds, or muting the device.

[0073] For example, when it is determined that the user has hearing loss based on the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss, the processor 22 reduces the volume of the device 14 or sets the volume of the device 14 to mute. When it is determined that the user may have hearing loss, the processor 22 may reduce the volume of the device 14. The processor 22 may also control the volume of the device 14 according to the level of hearing loss (e.g., the first level or the second level). For example, the processor 22 reduces the volume of the device 14 as the level of hearing loss increases (i.e., the degree of hearing loss increases).

[0074] Also, if the volume of device 14 is constantly set to above the threshold, processor 22 may lower the volume of device 14 or mute the volume of device 14. If the volume of device 14 is set to above the threshold only once, processor 22 may not lower the volume.

[0075] As yet another example, the process according to the result of the hearing loss determination is a process of controlling the sound collection function of device 14. That is, processor 22 may be configured to control the sound collection function of device 14 based on the operation history of device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss.

[0076] Controlling the sound collection function of device 14 includes, for example, turning on the sound collection function, turning off the sound collection function, changing the level of the sound collection function, turning on the noise cancellation function, or turning off the noise cancellation function.

[0077] For example, when it is determined that the user has hearing loss based on the operation history of the device 14 and the result of the determination of whether the user's biological information satisfies the condition of hearing loss, the processor 22 turns on the sound collection function or increases the level of the sound collection function. The processor 22 may also control the sound collection function of the device 14 according to the level of hearing loss (e.g., the first level or the second level). For example, the higher the level of hearing loss, the higher the level of the sound collection function of the device 14.

[0078] The processor 22 may execute at least one of outputting a warning, controlling the volume of the device 14, and controlling the sound collection function of the device 14. That is, the processor 22 may execute one of these processes, or may execute a plurality of processes. For example, the processor 22 may output a warning, control the volume of the device 14, and control the sound collection function of the device 14.

[0079] In addition, even if the user is determined to have hearing loss based on the operation history of the device 14 and the result of the determination of whether the user's biological information meets the condition of hearing loss, and the volume and sound collection function of the device 14 are controlled for a predetermined time or more (for example, even if the volume is increased or the level of the sound collection function is increased), the user's ease of hearing may not be improved. In this case, the processor 22 may output a warning to encourage the user to use the device 14 that transmits sound by an electric signal. The processor 22 determines the ease of hearing of the user based on the user's biological information. The device 14 that transmits sound by an electric signal is, for example, a device 14 (for example, an implant-type hearing aid, etc.) that transmits sound by an electric signal using an electrode inserted in the inner ear (for example, the cochlea, etc.). In addition, a device that transmits sound to the user by combining electric stimulation and acoustic stimulation may be used as the device 14. For example, a residual hearing utilization type cochlear implant (EAS: Electric Acoustic Stimulation) or the like may be used as the device 14. Some people cannot hear sounds properly even if they turn up the volume or increase the level of the sound collection function because the sound is distorted. For such people, the processor 22 may output a warning to encourage the user to use the device 14 that transmits sound by electrical signals using electrodes.

[0080] In addition, if device 14 has the function of transmitting sound via electrical signals using electrodes, and the user's ability to hear sounds is not improved, processor 22 may switch the function of transmitting sound to the user to a function of transmitting sound via electrical signals using electrodes.

[0081] The processing by the processor 22 may be executed by a device other than the information processing device 10 (for example, a terminal device, a server, the biological information measuring device 12, the device 14, or other devices). Also, a part of the processing may be executed by the information processing device 10, and another part of the processing may be executed by a device other than the information processing device 10. For example, the analysis of the biological information may be executed by a device other than the information processing device 10, and the control of the device 14 may be executed by the information processing device 10.

[0082] The configuration of the device 14 will be described in detail below with reference to Fig. 3. Fig. 3 shows an example of the configuration of the device 14.

[0083] The device 14 includes, for example, a speaker 24, a microphone 26, a communication device 28, a UI 30, a storage device 32, and a processor 34. The device 14 may include components other than these.

[0084] The speaker 24 may emit a sound based on sound data, or may emit a sound collected by the microphone 26. The sound data may be data stored in the storage device 32 of the device 14, or may be data transmitted to the device 14 from the information processing device 10 or another device. The microphone 26 has a sound collection function and is a device that collects surrounding sounds.

[0085] For example, device 14 may be a hearing aid, earphones, headphones, etc. that includes speaker 24 and microphone 26. Device 14 may be a hearing aid that amplifies the volume of sound collected by microphone 26 and emits the sound from speaker 24. Device 14 may also be an earphone, headphones, etc. that does not include microphone 26.

[0086] The communication device 28 is a communication interface and has a function of transmitting data to other devices and a function of receiving data transmitted from other devices. The communication device 28 may have a wireless communication function or a wired communication function. The communication device 28 may communicate with other devices by using, for example, short-range wireless communication, or may communicate with other devices via a communication path such as a LAN or the Internet.

[0087] The UI 30 is a user interface and includes at least one of a display device and an operation device. The display device is a liquid crystal display, an EL display, etc. The operation device is an input key, an operation panel, etc. The UI 30 may be a UI such as a touch panel that combines a display device and an operation device.

[0088] The storage device 32 is a device that configures one or more storage areas for storing data. The storage device 32 is, for example, a hard disk drive, various types of memory (e.g., RAM, DRAM, ROM, etc.), other storage devices (e.g., optical disks, etc.), or a combination thereof. One or more storage devices 32 are included in the device 14. For example, sound data, etc. may be stored in the storage device 32.

[0089] The processor 34 is configured to control the operation of each part of the device 14. The processor 34 may include a memory. For example, the processor 34 causes the speaker 24 to generate a sound based on the sound data, increases the volume of the sound collected by the microphone 26 and generates the sound from the speaker 24, controls the volume of the sound emitted from the speaker 24, and controls the sound collection function of the microphone 26. Note that the control of the volume and the sound collection function may be performed by the processor 22 of the information processing device 10.

[0090] The information processing system according to this embodiment will be described in further detail below.

[0091] An example of operation history management information will be described with reference to Fig. 4. Fig. 4 shows an example of an operation history management table, which is an example of operation history management information. For example, data of the operation history management table is stored in the storage device 20. Data of the operation history management table may not be stored in the storage device 20, or may be stored in the storage device 20 and in a device other than the information processing device 10 (for example, a device 14 or a server, etc.).

[0092] In the operation history management table, for example, an ID, device identification information for identifying the device 14, and operation history information indicating the operation history of the device 14 are associated with each other.

[0093] For example, each time a device 14 operates, operation history information indicating the operation history of the device 14 may be registered in the operation history management table, or the operation history information of each device 14 may be registered in the operation history management table at predetermined time intervals.

[0094] In the example shown in Fig. 4, the operation history information of device A is registered in the operation history management table as the operation history information of ID "1". For example, information indicating the total operation time of device A, information indicating the operation time for each volume of the sound emitted from the speaker 24, information indicating the history of the range setting, and information indicating the playback content (e.g., the music played), etc. are registered in the operation history management table as the operation history information of device A. Note that operation history information for each user may be registered in the operation history management table. Operation history information for devices 14 other than device A is also registered in the operation history management table.

[0095] An example of user attribute management information will be described with reference to Fig. 5. Fig. 5 shows an example of a user attribute management table, which is an example of user attribute management information. For example, data of the user attribute management table is stored in the storage device 20. Data of the user attribute management table may not be stored in the storage device 20, or may be stored in the storage device 20 and in a device other than the information processing device 10 (for example, a server or a user's terminal device, etc.).

[0096] In the user attribute management table, for example, an ID, user identification information for identifying a user, and user attribute information indicating the attributes of the user are associated with each other.

[0097] 5, the user attribute information of user α is registered in the user attribute management table as the user attribute information of ID "1". For example, information indicating the gender, age, medical history, etc. of user α is registered in the user attribute management table as the user attribute information of user α. User attribute information of users other than user α is also registered in the user attribute management table.

[0098] An example of user behavior history management information will be described with reference to Fig. 6. Fig. 6 shows an example of a user behavior history management table, which is an example of user behavior history management information. For example, data of the user behavior history management table is stored in the storage device 20. Data of the user behavior history management table may not be stored in the storage device 20, or may be stored in the storage device 20 and in a device other than the information processing device 10 (for example, a server, a user's terminal device, etc.).

[0099] In the user behavior history management table, for example, an ID, user identification information for identifying a user, and behavior history information indicating the behavior history of the user are associated with each other.

[0100] In the example shown in Fig. 6, the behavior history information of user α is registered in the user behavior history management table as the behavior history information of ID "1". For example, information indicating the drinking history, smoking history, eating and drinking history, and medication history of user α is registered in the user behavior history management table as the behavior history information of user α. Behavior history information of users other than user α is also registered in the user behavior history management table.

[0101] Hereinafter, the level of hearing loss will be described with reference to Fig. 7. Fig. 7 shows the level of hearing loss. The horizontal axis shows the total operating time, which is an example of the operation history of the device 14. The vertical axis shows the first level of hearing loss determined based on the user's biological information. For example, the first level is a level determined by an auditory brainstem response test or an auditory steady-state response test.

[0102] Each value in FIG. 7 indicates a second level of hearing loss that is determined based on the total working time and the first level of hearing loss. A higher number indicates a higher degree of hearing loss. For example, a second level of "0" indicates the lowest degree of hearing loss. A second level of "1" indicates a higher degree of hearing loss than second level "0" and lower than second level "2."

[0103] For example, the longer the total operating time, the higher the second level of hearing loss. That is, the longer the total time that the device 14 has been operating, the higher the second level of hearing loss. Also, the higher the first level of hearing loss, the higher the second level of hearing loss. That is, the higher the level of hearing loss determined based on the user's biometric information (i.e., the first level), the higher the second level of hearing loss.

[0104] The processor 22 acquires operation history information of the device 14 from the operation history management table, determines a first level of hearing loss based on the biological information of the user, and determines a second level of hearing loss based on the operation history information and the first level by referring to the correspondence shown in FIG. 7. For example, if the second level is less than a first threshold, the processor 22 determines that the user is not hearing-impaired. If the second level is equal to or greater than the first threshold and less than a second threshold that is greater than the first threshold, the processor 22 determines that the user may be hearing-impaired. If the second level is equal to or greater than the second threshold, the processor 22 determines that the user is hearing-impaired. These determinations are merely examples, and whether or not the user is hearing-impaired may be determined by another method. For example, if the second level is "0", the processor 22 determines that the user is not hearing-impaired. If the second level is "1", "2", or "3", the processor 22 determines that the user may be hearing-impaired. If the second level is "4" or higher, the processor 22 determines that the user has hearing loss.

[0105] Also, instead of or together with the total operation time, another operation history may be used. In other words, the parameter indicated by the horizontal axis in Fig. 7 may be at least one of the total operation time, operation time for each volume, volume level, range setting, and content of the reproduced sound data.

[0106] For example, the loudness of the sound may be used as a history of operation: the louder the sound, the higher the second level of hearing loss.

[0107] Also, the history of the setting of the range may be used as the operation history. For example, when the treble range is amplified, the processor 22 may determine that the user has difficulty hearing the treble range and determine the second level of hearing loss for the treble range. For example, the greater the degree of amplification of the treble range, the higher the second level of hearing loss for the treble range. The same applies to the low range.

[0108] Also, multiple histories may be used. For example, the total operation time and the volume may be used as the operation history. For example, if the device 14 generates a sound for a predetermined time or more at a volume equal to or greater than a predetermined volume, it is presumed that the manner of use of the device 14 is likely to cause hearing loss. In this case, the second level of hearing loss is higher than a manner of use that does not meet the condition.

[0109] The processor 22 may determine whether the user has hearing loss based on the user's attributes, the operation history of the device 14, and the result of determining whether the user's biological information satisfies the condition of hearing loss. The processor 22 acquires the user's attribute information from the user attribute management table, and acquires the operation history information of the device 14 from the operation history management table.

[0110] For example, the attribute of the user is the user's age. For example, the higher the age, the higher the second level of hearing loss. The processor 22 may also determine whether the hearing loss is age-related hearing loss based on the user's age. For example, if the user's age is equal to or greater than an age threshold and the second level of hearing loss is equal to or greater than a level threshold, the processor 22 may determine that the hearing loss is age-related hearing loss.

[0111] The user attribute may be the user's age, and the operation history of the device 14 may be the history of the user's setting of the range. For example, if the user's age is equal to or greater than the age threshold and the treble range is amplified, the processor 22 may determine that the user has hearing loss in the treble range. For example, when the user is 50 years old or older, it may become difficult to hear the treble range. Therefore, if the user is 50 years old or older and the treble range is amplified, the processor 22 determines that the user has hearing loss in the treble range. The frequency of the treble range is determined in advance.

[0112] As another example, the attribute of the user is the user's history of illness. The influence of illness may lead to hearing loss (e.g., sensorineural hearing loss). For example, acoustic neuroma, cholesteatoma, Meniere's disease, meningitis, otosclerosis, Down's syndrome, and Treacher Collins syndrome are illnesses that may lead to hearing loss. If the user has a history of illness that may lead to hearing loss, the second level is higher than if the user does not have a history of illness that may lead to hearing loss. For example, even if the operation history of the device 14 in a case where the user has a history of illness that may lead to hearing loss is the same as the operation history of the device 14 in a case where the user does not have a history of illness that may lead to hearing loss, the second level for the user who has a history of illness that may lead to hearing loss is higher than the second level for the user who does not have a history of illness that may lead to hearing loss. Similarly, even if the first level is the same when the user has a history of a disease that may cause hearing loss as the first level when the user does not have a history of a disease that may cause hearing loss, the second level will be higher for a user who has a history of a disease that may cause hearing loss compared to the second level for a user who does not have a history of a disease that may cause hearing loss.

[0113] The processor 22 may determine whether the user has hearing loss based on the user's behavior history, the operation history of the device 14, and the result of determining whether the user's biological information satisfies the condition of hearing loss. The processor 22 acquires the user's behavior history information from the user behavior history management table, and acquires the operation history information of the device 14 from the operation history management table.

[0114] For example, alcohol consumption and smoking can impair blood circulation and cause sudden hearing loss. For example, the higher the alcohol intake, the higher the second level of hearing loss, and the higher the smoking, the higher the second level of hearing loss.

[0115] Also, lack of sleep may cause hearing loss. For example, when the sleep time is not within a predetermined time range, the second level is higher than when the sleep time is within the predetermined time range. The predetermined time range is a time range in which the possibility of hearing loss is low. Also, when the quality of sleep is poor, the second level is higher than when the quality of sleep is good.

[0116] In addition, the contents of the food may cause hearing loss. For example, the contents, calories, ingredients, etc. of food that may cause hearing loss are predetermined, and the more such food is consumed, the higher the second level becomes.

[0117] In addition, lifestyle habits may cause hearing loss. For example, lifestyle habits that may cause hearing loss are determined in advance, and the second level for a user who has such lifestyle habits is higher than the second level for a user who does not have such lifestyle habits.

[0118] Also, medication may cause hearing loss. For example, the second level for a user who is given medication or who takes medication is higher than the second level for a user who is not given medication or who does not take medication. Also, medications that may cause hearing loss are predefined, and the second level for a user who is given such medication or who takes such medication is higher than the second level for a user who is not given such medication or who does not take such medication.

[0119] Note that the second level shown in Fig. 7 is merely an example, and the second level may be determined by a method other than the method using the correspondence shown in Fig. 7. For example, the second level may be determined by using a function. For example, a function that determines the second level using the operation history of the device 14 and the first level as parameters may be used. Also, a function that uses the user's attributes or the user's behavior history as parameters may be used.

[0120] The processor 22 may execute a function to restore the hearing of the user. For example, if it is determined that the user has hearing loss (for example, if the second level is equal to or greater than the threshold), the processor 22 executes a function to restore the hearing of the user. Of course, if it is determined that the user may have hearing loss (for example, if the second level is less than the threshold but a tendency to have hearing loss is detected), the processor 22 may execute a function to restore the hearing of the user.

[0121] The function for recovering hearing is, for example, a function for removing stress and fatigue from the user, a function for improving the user's blood circulation, a function for encouraging the user to review the user's behavior, and the like. The function for removing stress and fatigue from the user is, for example, a function for playing music that is expected to relax the user, a function for suggesting to the user a meal for recovering from fatigue, and the like. The function for improving the user's blood circulation is, for example, a function for warming the user's ears, a function for playing music that promotes the user's blood circulation, a function for suggesting to the user a meal that promotes the user's blood circulation, and the like. For example, the device 14 is a hearable device (for example, earphones, headphones, or hearing aids) having a heating function, and the ears are warmed by wearing the device 14 on the ears. Warming the ears promotes blood circulation in the ears. In addition, since B vitamins (for example, B12) are thought to be useful for the recovery of peripheral nerves, the processor 22 may suggest to the user a meal that is rich in B vitamins. A function that prompts a user to review their behavior is, for example, a function that prompts the user to stop lifestyle habits that may lead to hearing loss, or a function that guides the user to lifestyle habits that will restore hearing.

[0122] The function for restoring the user's hearing may be a noise cancellation function. For example, the device 14 is a hearable device (e.g., earphones, headphones, or hearing aids) with a noise cancellation function, and the device 14 is worn on the ear when used. By executing the noise cancellation function, ambient noise can be removed, so that even if the volume of the hearable device (e.g., the volume of earphones, headphones, or hearing aids) is lowered, the sound emitted from the hearable device becomes easier to hear compared to when the noise cancellation function is not executed.

[0123] The function for restoring the user's hearing is a function for lowering the volume of a hearable device (for example, earphones, headphones, or hearing aids). The processor 22 may lower the volume while executing the noise cancellation function.

[0124] In addition, by keeping the balance of the autonomic nerves (for example, the sympathetic nerves and the parasympathetic nerves) normal, it may be possible to maintain or recover hearing. The processor 22 may suggest to the user a method for keeping the balance of the autonomic nerves normal.

[0125] The processor 22 may simultaneously perform multiple hearing restoration functions, or may perform one function for a predetermined period of time and then perform another function for a predetermined period of time after the previous function has finished executing.

[0126] The processor 22 may execute a function for recovering hearing according to an instruction from the user. For example, the processor 22 displays information indicating a plurality of functions for recovering hearing on a display device of the UI 18, a display device of the user's terminal device, or the like. The processor 22 executes a function selected by the user from the plurality of functions. Of course, the processor 22 may execute a function for recovering hearing without receiving an instruction from the user.

[0127] The processor 22 may estimate the degree of recovery of the user's hearing based on the user's biological information. For example, the processor 22 may estimate the degree of recovery of hearing (in other words, the degree of hearing loss) by an auditory brainstem response test or an auditory steady-state response test, or may estimate the degree of recovery of hearing based on whether or not brain waves, which are an example of biological information, indicate difficulty in hearing. The processor 22 may change the length of time for executing a function for recovering hearing, may repeatedly execute the same function for recovering hearing, may change the function for recovering hearing, or may additionally execute a function for recovering hearing, depending on the degree of recovery.

[0128] The processor 22 may estimate the cause of the user's hearing loss based on the user's attributes, the user's behavior history, and the operation history of the device 14, and execute a function for restoring the user's hearing according to the cause. For example, if it is estimated that the hearing loss is caused by diet, the processor 22 suggests to the user a meal for restoring the user's hearing. If it is estimated that the hearing loss is caused by lack of sleep, the processor 22 encourages the user to get some sleep. If it is estimated that the hearing loss is caused by drinking alcohol or smoking, the processor 22 suggests to the user to reduce the amount of alcohol or smoking.

[0129] Furthermore, when it is determined that the user has hearing loss, and the type of hearing loss (for example, sensorineural hearing loss, conductive hearing loss, or mixed hearing loss) is estimated, the processor 22 may display information indicating the type of hearing loss on a display device, or may output the information as sound. The processor 22 may execute a function for restoring hearing according to the type of hearing loss. For example, since sensorineural hearing loss may occur when there is a problem in the inner ear, the processor 22 may execute a function for restoring hearing that acts on the inner ear when the user has sensorineural hearing loss. Furthermore, since conductive hearing loss may occur when there is a problem in the area from the outer ear to the middle ear, the processor 22 may execute a function for restoring hearing that acts on the area from the outer ear to the middle ear when the user has conductive hearing loss. The processor 22 may present to the user a function for restoring hearing according to the type of hearing loss.

[0130] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operation of the processor in each of the above embodiments may not only be performed by one processor, but may also be performed by multiple processors located at physically separate locations working together. Furthermore, the order of each operation of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0131] 10 Information processing devices, 12 Vital information measuring devices, 14 Equipment, 22 Processors.

Claims

[Claim 1] On the computer, outputting a result of a judgment as to whether or not the user has hearing loss based on a behavior history related to a lifestyle habit of the user and biological information of the user; An information processing method for executing a process including the steps of:

Images