Information processing device, wearable device, information processing method and program

Abstract

To determine wearing of a wearable device in a wider range of environment.SOLUTION: An information processing device includes: an acoustic information acquisition section for acquiring acoustic information related to resonance in a body of a user wearing a wearable device; and a wearing determination section for determining whether the user is wearing the wearable device based on the acoustic information. When the wearing determination section determines that the user is not wearing the wearable device, the wearable device stops at least a partial function for a predetermined period of time.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing device, a wearable device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a headphone device equipped with an outer microphone and an inner microphone, which is capable of detecting whether the headphone device is being worn or not by comparing an audio signal of an external sound obtained by the outer microphone with an audio signal of an external sound obtained by the inner microphone.

[0003] Patent Document 2 discloses a headset equipped with a detection microphone and a speaker. The headset compares an audio signal, such as music, input to the headset with an audio detection signal detected by the detection microphone, and if there is a mismatch, determines that the headset is not being worn. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-33303 A [Patent Document 2] JP 2007-165940 A Summary of the Invention [Problem to be solved by the invention]

[0005] The headphone device of Patent Document 1 detects the wearing state using external sounds. Since external sounds can change depending on the external environment, the accuracy of the wearing determination may not be sufficient depending on the external environment. The headset of Patent Document 2 detects the wearing state based on whether the input sound signal and the detected sound detection signal match. Therefore, for example, when the headset is sealed, such as when the headset is in a case, the sound signal and the sound detection signal may match even when the headset is not worn. Thus, the accuracy of the wearing determination may not be sufficient depending on the environment in which the headset is placed.

[0006] An object of the present invention is to provide an information processing device, a wearable device, an information processing method, and a storage medium that are capable of determining whether a wearable device is worn in a wider range of environments. [Means for solving the problem]

[0007] According to one aspect of the present invention, there is provided an information processing device comprising an acoustic information acquisition unit that acquires acoustic information relating to resonance within the body of a user wearing a wearable device, and a wearing determination unit that determines whether or not the user is wearing the wearable device based on the acoustic information, wherein if the wearing determination unit determines that the user is not wearing the wearable device, the wearable device stops at least some of its functions for a predetermined period of time.

[0008] According to another aspect of the present invention, there is provided a wearable device comprising: an acoustic information acquisition unit that acquires acoustic information relating to resonance within the body of a user wearing the wearable device; a wearing determination unit that determines whether or not the user is wearing the wearable device based on the acoustic information; and a stop unit that stops at least a portion of the functions of the wearable device for a predetermined period of time when the wearing determination unit determines that the user is not wearing the wearable device.

[0009] According to another aspect of the present invention, there is provided an information processing method comprising the steps of acquiring acoustic information relating to resonance within the body of a user wearing a wearable device, determining whether or not the user is wearing the wearable device based on the acoustic information, and if it is determined that the user is not wearing the wearable device, stopping at least some functions of the wearable device for a predetermined period of time.

[0010] According to another aspect of the present invention, a program is provided for causing a computer to execute the steps of acquiring acoustic information relating to resonance within the body of a user wearing a wearable device, determining whether or not the user is wearing the wearable device based on the acoustic information, and stopping at least some functions of the wearable device for a predetermined period of time if it is determined that the user is not wearing the wearable device. Effect of the Invention

[0011] According to the present invention, it is possible to provide an information processing device, a wearable device, an information processing method, and a storage medium that are capable of determining whether a wearable device is worn in a wider range of environments. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing an overall configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration of the earphone according to the first embodiment. FIG. [Diagram 3] 1 is a block diagram showing a hardware configuration of an information communication device according to a first embodiment. [Figure 4] 1 is a functional block diagram of an earphone control device according to a first embodiment. FIG. [Diagram 5] 4 is a flowchart showing a wearing determination process performed by the earphone control device according to the first embodiment. [Figure 6] 1 is a graph showing characteristics of a chirp signal. [Figure 7]1 is a graph showing characteristics of an M-sequence signal or white noise. [Figure 8] 1 is a graph showing an example of characteristics of reverberation. [Figure 9] FIG. 1 is a structural diagram of an air column with one open end and the other closed end. [Figure 10] This is a structural diagram of an air column with both ends closed. [Figure 11] 11 is a table showing types of acoustic signals used for attachment determination and determination criteria. [Figure 12] FIG. 11 is a schematic diagram showing the overall configuration of an information processing system according to a second embodiment. [Figure 13] 13 is a graph showing the change over time in the mounting condition score according to the third embodiment. [Figure 14] 13 is a graph showing an example of determining the wearing state using two threshold values. [Figure 15] FIG. 13 is a functional block diagram of an information processing device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, exemplary embodiments of the present invention will be described with reference to the drawings. In the drawings, similar or corresponding elements are designated by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0014] [First embodiment] An information processing system according to the present embodiment will be described below. The information processing system according to the present embodiment is a system for detecting whether a wearable device such as an earphone is being worn.

[0015] 1 is a schematic diagram showing the overall configuration of an information processing system according to this embodiment. The information processing system includes an information communication device 1 and earphones 2 that can be wirelessly connected to each other.

[0016] The earphones 2 include an earphone control device 20, a speaker 26, and a microphone 27. The earphones 2 are acoustic devices that can be worn in the ears of a user 3, and are typically wireless earphones, wireless headsets, or the like. The speaker 26 functions as a sound wave generating unit that emits sound waves toward the ear canal of the user 3 when worn, and is disposed on the wearing surface side of the earphones 2. The microphone 27 is also disposed on the wearing surface side of the earphones 2 so that it can receive sound waves reverberated in the ear canal of the user 3 when worn. The earphone control device 20 controls the speaker 26 and the microphone 27, and communicates with the information and communication device 1.

[0017] In this specification, "sound" such as sound waves and voice includes inaudible sounds whose frequencies or sound pressure levels are outside the audible range.

[0018] The information and communication device 1 is, for example, a computer, and controls the operation of the earphones 2, transmits audio data for generating sound waves emitted from the earphones 2, and receives audio data obtained from sound waves received by the earphones 2. As a specific example, when the user 3 listens to music using the earphones 2, the information and communication device 1 transmits compressed data of the music to the earphones 2. Furthermore, when the earphones 2 are telephone devices for issuing operational instructions at an event venue, hospital, or the like, the information and communication device 1 transmits audio data of operational instructions to the earphones 2. In this case, audio data of the speech of the user 3 may also be transmitted from the earphones 2 to the information and communication device 1. Furthermore, the information and communication device 1 or the earphones 2 may have a function of ear acoustic authentication using sound waves received by the earphones 2.

[0019] This overall configuration is just an example, and for example, the information communication device 1 and the earphones 2 may be connected by wire. Also, the information communication device 1 and the earphones 2 may be configured as an integrated device, or another device may be included in the information processing system.

[0020] 2 is a block diagram showing an example hardware configuration of earphone control device 20. Earphone control device 20 includes a central processing unit (CPU) 201, random access memory (RAM) 202, read only memory (ROM) 203, and flash memory 204. Earphone control device 20 also includes a speaker interface (I / F) 205, a microphone I / F 206, a communication I / F 207, and a battery 208. The various components of earphone control device 20 are connected to each other via buses, wiring, drivers, etc. (not shown).

[0021] CPU 201 is a processor that performs predetermined calculations according to programs stored in ROM 203, flash memory 204, etc., and also has the function of controlling each part of earphone control device 20. RAM 202 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of CPU 201. ROM 203 is composed of a non-volatile storage medium, and stores necessary information such as programs used in the operation of earphone control device 20. Flash memory 204 is composed of a non-volatile storage medium, and is a storage device that temporarily stores data, stores programs for operating earphone control device 20, etc.

[0022] The communication I / F 207 is a communication interface based on standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), and is a module for communicating with the information communication device 1.

[0023] The speaker I / F 205 is an interface for driving the speaker 26. The speaker I / F 205 includes a digital-to-analog conversion circuit, an amplifier, etc. The speaker I / F 205 converts the audio data into an analog signal and supplies it to the speaker 26. As a result, the speaker 26 emits sound waves based on the audio data.

[0024] The microphone I / F 206 is an interface for acquiring a signal from the microphone 27. The microphone I / F 206 includes an analog-to-digital conversion circuit, an amplifier, etc. The microphone I / F 206 converts an analog signal generated by a sound wave received by the microphone 27 into a digital signal. In this way, the earphone control device 20 acquires audio data based on the received sound waves.

[0025] The battery 208 is, for example, a secondary battery, and supplies the power necessary for the operation of the earphones 2. This allows the earphones 2 to operate wirelessly without being wired to an external power source.

[0026] Note that the hardware configuration shown in FIG. 2 is an example, and other devices may be added, or some devices may not be provided. Also, some devices may be replaced with other devices having similar functions. For example, the earphone 2 may further include an input device such as a button to accept an operation by the user 3, and may further include a display device such as a display or an indicator light to provide information to the user 3. In this way, the hardware configuration shown in FIG. 2 can be changed as appropriate.

[0027] 3 is a block diagram showing an example of a hardware configuration of the information communication device 1. The information communication device 1 includes a CPU 101, a RAM 102, a ROM 103, and an HDD (Hard Disk Drive) 104. The information communication device 1 also includes a communication I / F 105, an input device 106, and an output device 107. The components of the information communication device 1 are connected to each other via a bus, wiring, a drive device, etc. (not shown).

[0028] 3, the components constituting the information communication device 1 are illustrated as an integrated device, but some of these functions may be provided by external devices. For example, the input device 106 and the output device 107 may be external devices separate from the components constituting the functions of the computer, including the CPU 101, etc.

[0029] The CPU 101 is a processor that performs predetermined calculations according to programs stored in the ROM 103, HDD 104, etc., and also has the function of controlling each part of the information communication device 1. The RAM 102 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of the CPU 101. The ROM 103 is composed of a non-volatile storage medium, and stores necessary information such as programs used in the operation of the information communication device 1. The HDD 104 is composed of a non-volatile storage medium, and is a storage device that temporarily stores data to be transmitted to and received from the earphones 2, stores programs for operating the information communication device 1, etc.

[0030] The communication I / F 105 is a communication interface based on standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), and is a module for communicating with other devices such as the earphones 2.

[0031] The input device 106 is a keyboard, a pointing device, etc., and is used by the user 3 to operate the information communication device 1. Examples of the pointing device include a mouse, a trackball, a touch panel, and a pen tablet.

[0032] The output device 107 is, for example, a display device. The display device is a liquid crystal display, an OLED (Organic Light Emitting Diode) display, etc., and is used to display information, a GUI (Graphical User Interface) for operation input, etc. The input device 106 and the output device 107 may be integrally formed as a touch panel.

[0033] The hardware configuration shown in FIG. 3 is an example, and other devices may be added, or some devices may not be provided. Some devices may be replaced with other devices having similar functions. Furthermore, some functions of this embodiment may be provided by other devices via a network, or the functions of this embodiment may be distributed and realized among a plurality of devices. For example, the HDD 104 may be replaced with an SSD (Solid State Drive) using a semiconductor memory, or may be replaced with cloud storage. In this way, the hardware configuration shown in FIG. 3 can be changed as appropriate.

[0034] 4 is a functional block diagram of the earphone control device 20 according to the present embodiment. The earphone control device 20 includes an acoustic information acquisition unit 211, an wearing determination unit 212, a sound generation control unit 213, a notification information generation unit 214, and a storage unit 215.

[0035] The CPU 201 loads a program stored in the ROM 203, the flash memory 204, etc. into the RAM 202 and executes it. In this way, the CPU 201 realizes the functions of an acoustic information acquisition unit 211, a wearing determination unit 212, a sound generation control unit 213, and a notification information generation unit 214. The CPU 201 also realizes the function of a storage unit 215 by controlling the flash memory 204 based on the program. Specific processes performed by each of these units will be described later.

[0036] 4 may be provided in the information and communication device 1 rather than in the earphone control device 20. In other words, the functions described above may be realized by the earphone control device 20, may be realized by the information and communication device 1, or may be realized by cooperation between the information and communication device 1 and the earphone control device 20. The information and communication device 1 and the earphone control device 20 are sometimes more generally referred to as information processing devices.

[0037] However, it is desirable that the wearing determination process of this embodiment be performed by the earphone control device 20 provided in the earphone 2. In this case, communication between the information communication device 1 and the earphone 2 during the wearing determination can be eliminated, and the power consumption of the earphone 2 can be reduced. Since the earphone 2 is a wearable device, it is required to be small. Therefore, there is a limit to the size of the battery 208, and it is difficult to use one with a large discharge capacity. For these reasons, it is effective to reduce power consumption by completing the wearing determination within the earphone 2. In the following description, unless otherwise specified, it is assumed that each function of the functional blocks in FIG. 4 is provided in the earphone 2.

[0038] 5 is a flowchart showing the wearing determination process performed by the earphone control device 20 according to this embodiment. The operation of the earphone control device 20 will be described with reference to FIG.

[0039] The wearing determination process in Fig. 5 is executed, for example, every time a predetermined time elapses while the power of the earphones 2 is on. Alternatively, the wearing determination process in Fig. 5 may be executed when the user 3 starts using the earphones 2 by operating the earphones 2.

[0040] In step S101, the sound generation control unit 213 generates a test signal and transmits the test signal to the speaker 26 via the speaker I / F 205. As a result, the speaker 26 emits a test sound for fitting determination toward the ear canal of the user 3.

[0041] In step S101, instead of using the test sound from the speaker 26, sounds generated within the body of the user 3 may be used. Specific examples of sounds generated within the body include biological sounds generated by the breathing, heartbeat, muscle movements, etc. of the user 3. As another example, the voice of the user 3 emitted from the vocal cords of the user 3 by encouraging the user 3 to speak may be used.

[0042] An example of the process of encouraging the user 3 to speak will be described. The notification information generating unit 214 generates notification information for encouraging the user 3 to speak. This notification information may be, for example, audio information, and may encourage the user 3 to speak by issuing a message such as "Please speak" from the speaker 26. If the information communication device 1 or the earphones 2 have a display device that the user 3 can see, the above-mentioned message may be displayed on the display device.

[0043] The process of emitting the test sound or the process of encouraging the user to speak may be performed at all times during the wearing judgment, but may be performed only when a predetermined condition is satisfied or when a predetermined condition is not satisfied. An example of the predetermined condition is when the sound pressure level included in the acquired acoustic information is not at a sufficient level for making the judgment. When this condition is satisfied, the user is prompted to speak to acquire acoustic information with a high sound pressure level. This can improve the accuracy of the wearing judgment.

[0044] In step S102, the acoustic information acquisition unit 211 acquires acoustic information based on the sound waves received by the microphone 27. This acoustic information is stored in the storage unit 215 as acoustic information related to resonance in the body of the user 3. Note that, when acquiring the acoustic information, the acoustic information acquisition unit 211 may appropriately perform signal processing such as Fourier transform, correlation calculation, noise removal, and level correction.

[0045] In step S103, the wearing determination unit 212 determines, based on the acoustic information, whether or not the user 3 is wearing the earphones 2. If it is determined that the user 3 is wearing the earphones 2 (YES in step S103), the process proceeds to step S104. If it is determined that the user 3 is not wearing the earphones 2 (NO in step S103), the process proceeds to step S105.

[0046] In step S104, the earphone 2 continues operations such as communication with the information communication device 1 and generation of sound waves based on information acquired from the information communication device 1. After a predetermined time has elapsed, the process returns to step S101, and the wearing determination is performed again.

[0047] In step S105, the earphone 2 stops operations such as communication with the information communication device 1 and generation of sound waves based on information obtained from the information communication device 1, and ends this process.

[0048] This allows the earphones 2 to continue operating when the user 3 is wearing them, and to stop operating the earphones 2 when the user 3 is not wearing them. This reduces the waste of power caused by the earphones 2 operating when not being worn.

[0049] 5, the process ends after step S105 and the earphones 2 stop operating, but this is just an example. For example, the process may return to step S101 after a predetermined time has elapsed, and the earphones may be worn again. If it is determined that the user 3 is wearing the earphones 2 after that, the operation of the earphones 2 may be resumed.

[0050] A specific example of the test sound emitted by the speaker 26 in step S101 will be described. Examples of signals used to generate the test sound include a chirp signal, a maximum length sequence (M) signal, white noise, or other signal containing frequency components in a predetermined range. This allows the frequency range of the test sound to be used for fitting determination.

[0051] Figure 6 is a graph showing the characteristics of a chirp signal. Figure 6 shows the relationship between intensity and time, the relationship between frequency and time, and the relationship between intensity and frequency. A chirp signal is a signal whose frequency changes continuously with time. Figure 6 shows an example of a chirp signal whose frequency increases linearly with time.

[0052] Fig. 7 is a graph showing the characteristics of an M-sequence signal or white noise. Since an M-sequence signal is a signal that generates pseudo-noise close to white noise, the characteristics of the M-sequence signal and white noise are almost similar. Like Fig. 6, Fig. 7 also shows the relationship between intensity and time, the relationship between frequency and time, and the relationship between intensity and frequency. As shown in Fig. 7, an M-sequence signal or white noise is a signal that evenly contains signals in a wide range of frequencies.

[0053] The chirp signal, the M-sequence signal, or the white noise has a frequency characteristic in which the frequency varies over a wide range, so that by using these signals as the test sound, it is possible to obtain reverberation sounds over a wide range of frequencies in step S102.

[0054] A specific example of the reverberation sound acquired in step S102 will be described below. Fig. 8 is a graph showing an example of the characteristics of the reverberation sound.

[0055] The horizontal axis of Fig. 8 indicates frequency, and the vertical axis indicates the sound pressure level of the acquired sound waves. In Fig. 8, the acquired sound waves are divided into three categories according to their source: "noise," "speech," and "echo."

[0056] "Noise" refers to biological noise, specifically, biological sounds generated by breathing, heartbeat, muscle movement, etc. of the user 3. As shown in Fig. 8, "noise" is concentrated in the range of 1 kHz or less.

[0057] "Speech" indicates the sound produced by the speech of user 3. As shown in Fig. 8, "speech" is concentrated in the range of 3 kHz or less. In addition, there is a small peak around 6 kHz. This peak is caused by the reverberation in the ear canal.

[0058] "Echo" indicates a sound generated by the test sound being reflected in the body of the user 3, such as the external ear canal and vocal tract. As shown in FIG. 8, "echo" has a characteristic having multiple peaks. There are multiple peaks due to vocal tract resonance around 2 kHz. In addition, there are first, second, and third peaks of external ear canal resonance around 6 kHz, 12 kHz, and 14 kHz. These peaks generated by resonance can be used for fitting judgment. Note that the peak around 20 kHz is a resonance sound in the housing of the earphone 2, and therefore the peak is not a reverberation sound within the body of the user 3. However, since the absorption rate of the resonance sound differs between when the earphone is worn and when it is not worn, the level of the peak changes depending on whether the earphone is worn or not. Therefore, the peak around 20 kHz may be used for fitting judgment.

[0059] Here, we will explain resonance in more detail. Resonance is generally a phenomenon in which a physical system exhibits characteristic behavior when it is acted upon at a specific period. An example of resonance in the case of an acoustic phenomenon is the phenomenon in which a loud reverberation occurs at a specific frequency when sound waves with wavelengths of various frequencies are sent to an acoustic system. Such a reverberation is called a resonance.

[0060] The air column resonance model is known as a simple model to explain resonant sound. Figure 9 is a structural diagram of an air column with one open end and the other closed end. In the example of Figure 9, if the length of the air column is L, the speed of sound is V, and the order of resonance is n (n = 1, 2, ...), the resonance frequency f is given by the following equation (1). Note that open end correction is ignored in equation (1).

number

[0061] In addition, Fig. 10 is a structural diagram of an air column tube with both ends closed. In the example of Fig. 10, the resonance frequency f is given by the following formula (2).

number

[0062] As can be seen from equations (1) and (2), the higher the observed resonant frequency, the shorter the air column where resonance occurs, and the lower the observed resonant frequency, the longer the air column where resonance occurs. In other words, the resonant frequency and the length of the part where resonance occurs are inversely proportional to each other, and can be related to each other.

[0063] As a specific example, consider the first-order peak seen near 6 kHz in FIG. 8. When the user 3 is wearing the earphones 2, the structure of the ear canal corresponds to an air column tube with both ends closed. Therefore, the length of the air column tube can be calculated using formula (2). The sound speed V is about 340 m / s, the resonance frequency f is about 6 kHz, and the order n is 1. Substituting these into formula (2), the value of L is calculated to be about 2.8 cm. Since this length roughly matches the length of the human ear canal, it can be said that the peak seen near 6 kHz in FIG. 8 is certainly due to ear canal resonance. Since cavities in the human body other than the ear canal (vocal tract, respiratory system, etc.) can also be explained by the air column model, it is possible to similarly associate the resonance frequency with the length of the cavity. This makes it possible to identify the length of the part where resonance occurs from the peak included in the characteristics of the reverberation, and also to identify the resonance site.

[0064] Next, a specific example of the wearing determination in step S103 will be described. Fig. 11 is a table showing the types of acoustic signals and the determination criteria used in the wearing determination. Since biological sounds ("noise" in Fig. 8) are generated inside the body of the user 3, when the earphones 2 are not worn, they are not detected, or even if they are detected, the sound pressure is very low. Therefore, wearing determination is possible using an algorithm that determines that the earphones are not worn when the sound pressure level of an acoustic signal at a predetermined detection frequency of 1 kHz or less is below a predetermined threshold, and determines that the earphones are worn when the sound pressure level is equal to or greater than the threshold.

[0065] Vocal tract reverberation (around 2 kHz for "echo" in FIG. 8) also occurs inside the body of the user 3, so if the earphones 2 are not being worn, they will not be detected, or even if they are detected, the sound pressure will be very low. Therefore, wearing determination is possible using an algorithm that determines that the earphones are not being worn when there is no peak or the sound pressure level of the acoustic signal around 2 kHz is sufficiently low, and determines that the earphones are being worn when there is a peak.

[0066] Since ear canal reverberation (around 5-20 kHz of "echo" in FIG. 8) is also generated inside the body of the user 3, it will not be detected when the earphones 2 are not being worn, or even if it is detected, it will be at a very low sound pressure. Therefore, wearing determination is possible using an algorithm that determines that the earphones are not being worn when there is no peak or the sound pressure level of the acoustic signal around 5-20 kHz is sufficiently low, and determines that the earphones are being worn when there is a peak.

[0067] In addition, since a peak due to vocal tract reverberation or ear canal reverberation may occur due to biological sound, the peak generated by biological sound may be used for fitting judgment, but the peak is often weak. Therefore, when using the peak of vocal tract reverberation or ear canal reverberation for fitting judgment, it is desirable to use a test sound or to perform processing to encourage speaking. Since the peak of vocal tract reverberation is larger when speaking than when a test sound is emitted to the ear canal, it is desirable to perform processing to encourage speaking when using it for vocal tract reverberation fitting judgment. Since the peak of ear canal reverberation is larger when a test sound is emitted to the ear canal than when speaking, it is desirable to perform processing using a test sound when using it for vocal tract reverberation fitting judgment.

[0068] The wearing determination may be performed using any one of those shown in FIG. 11, or may be performed by parameterizing one or more criteria, calculating a wearing condition score, and determining whether the wearing condition score is equal to or greater than a threshold value.

[0069] According to this embodiment, acoustic information related to resonance in the body of a user 3 wearing a wearable device such as earphones 2 can be acquired, and based on this, it can be determined whether the user 3 is wearing the wearable device. This allows the determination of whether the wearable device is being worn not only in an environment with external sounds, but also in a quiet environment without external sounds. In addition, since the determination is based on internal resonance, erroneous determination is unlikely to occur in a closed environment. Therefore, it is possible to provide an information processing device that can determine whether the wearable device is being worn in a wider range of environments.

[0070] In this embodiment, when the wearing determination is performed using the test sound, it may be determined whether or not the user 3 is wearing the earphones 2 based on the echo time from when the sound waves are emitted from the speaker 26 to when the sound waves are acquired by the microphone 27. The time from when the test sound is emitted toward the ear canal to when the echo sound is acquired is the round trip time of the sound waves in the ear canal of the user 3, and is therefore determined by the length of the ear canal. If this echo time deviates significantly from the time determined by the length of the ear canal, it is highly likely that the earphones 2 are not being worn. Therefore, by using the echo time as a factor in the wearing determination, it is possible to perform the wearing determination with higher accuracy.

[0071] [Second embodiment] The information processing system of this embodiment differs from that of the first embodiment in the structure of the earphones 2 and the wearing determination process. The following description will mainly focus on the differences from the first embodiment, and descriptions of common parts will be omitted or simplified.

[0072] 12 is a schematic diagram showing the overall configuration of an information processing system according to this embodiment. In this embodiment, the earphones 2 are equipped with multiple microphones 27, 28 arranged at different positions. The microphones 28 are controlled by the earphone control device 20. The microphones 28 are arranged on the back side opposite to the wearing surface of the earphones 2 so that they can receive sound waves from the outside when worn.

[0073] The earphone 2 of the present embodiment is more effective when performing a fitting determination using body sound. Body sound is due to breathing sounds, heart sounds, muscle movements, etc., and therefore has a weak sound pressure, and fitting determination using body sound may be insufficient in accuracy due to external noise.

[0074] Since body sounds are generated within the body, many components are propagated through the body. Therefore, when the earphones 2 are worn, the body sounds acquired by the microphone 27 are louder than the body sounds acquired by the microphone 28. Therefore, it is possible to determine that the earphones are worn when the body sounds acquired by the microphone 27 are louder than the body sounds acquired by the microphone 28. This method cancels the influence of external noise, and therefore allows for more accurate wearing determination than a method that compares the magnitude relationship with a threshold value. Therefore, according to this embodiment, in addition to obtaining the same effects as the first embodiment, it is possible to realize a more accurate wearing determination.

[0075] [Third embodiment] The information processing system of this embodiment differs from that of the first embodiment in the algorithm of the mounting determination process in step S103 in Fig. 5. The following mainly describes the differences from the first embodiment, and descriptions of common parts will be omitted or simplified.

[0076] In this embodiment, one or more criteria are parameterized to calculate a wearing state score, and wearing determination is performed based on whether the wearing state score is equal to or greater than a threshold. In addition, in the process of FIG. 5, even after the operation is stopped in step S105, the process returns to step S101, and wearing determination is repeated at a certain period. FIG. 13 is a graph showing an example of a change over time in the wearing state score according to this embodiment. The wearing state score S1 in the figure is a threshold (first threshold) between the wearing state and the non-wearing state.

[0077] In the method of the first embodiment, when the wearing state score is equal to or greater than the first threshold, it is determined that the device is in the wearing state, and when the wearing state score is less than the first threshold, it is determined that the device is in the non-wearing state. Therefore, the period before time t1, the period between time t2 and time t3, and the period after time t4 are determined to be the non-wearing state, and the period between time t1 and time t2 and the period between time t3 and time t4 are determined to be the wearing state.

[0078] In this case, the state also switches if the wearing state score changes in a short time, such as from time t2 to time t3. Because the user 3 does not often repeatedly put on and take off the earphones 2 in a short time, such short-term changes often do not appropriately indicate the wearing state. In particular, if it is determined that the earphones 2 are not being worn despite being worn, some of the functions of the earphones 2 are stopped, which impairs the convenience of the user 3. Therefore, the information processing system of this embodiment performs a wearing determination process so as to make it difficult for the state to switch when, for example, the wearing state score changes in a short time. Note that an example of such a short-term change is when the user 3 touches the earphones 2. Below, four examples of the wearing determination process that can be applied in this embodiment will be described.

[0079] (First example of wearing determination process) In a first example of the wearing determination process according to the present embodiment, when the wearing state score changes from a state equal to or greater than the first threshold to a state smaller than the first threshold, the wearing state is maintained for a predetermined period of time. When the wearing state score returns to the first threshold or greater during the period during which the wearing state is maintained, it is treated as not being in a non-wearing state. As a result, when the wearing state score drops for only a short period of time, such as from time t2 to time t3 in FIG. 13, the wearing state is maintained.

[0080] (Second example of wearing determination process) A second example of the wearing determination process according to this embodiment provides two thresholds for use in the wearing determination. Fig. 14 is a graph showing an example of determining the wearing state using two thresholds. The wearing state score S1 in Fig. 14 is a first threshold for determining switching from a non-wearing state to a wearing state, and the wearing state score S2 is a second threshold for determining switching from a wearing state to a non-wearing state.

[0081] In this example, the wearing state score falls below the first threshold value between time t2 and time t3, but does not fall below the second threshold value, so the wearing state is maintained. The wearing state is also maintained in the period from time t4 to time t5. After time t5, when the wearing state score falls below the second threshold value, it is determined that the device is not wearing the device. In this way, in this example, by providing two threshold values, hysteresis can be applied to the switching from the wearing state to the non-wearing state and the switching from the non-wearing state to the wearing state. Therefore, the switching between the wearing state and the non-wearing state due to the small fluctuations in the wearing state score that occur in a short period of time is suppressed.

[0082] (Third example of wearing determination process) A third example of the wearing determination process according to the present embodiment is to vary the interval of the wearing determination depending on the wearing state score. More specifically, when the wearing state score is larger than a predetermined value, the interval of the wearing determination is set to a long time, and when the wearing state score is smaller than the predetermined value, the interval of the wearing determination is set to a short time. This predetermined value is set to a value higher than the first threshold value used for the wearing determination. As a result, when the wearing state score becomes low, such as around times t2 and t4 in FIG. 13, the interval of the wearing determination becomes long, so that a change in state due to a short-term change in the wearing state score is unlikely to occur. Therefore, when the wearing state score decreases for only a short time, such as from time t2 to time t3 in FIG. 13, the wearing state is likely to be maintained.

[0083] (Fourth example of wearing determination process) A fourth example of the wearing determination process according to the present embodiment is to vary the interval of the wearing determination depending on the difference between the wearing state score and the first threshold. More specifically, when the difference between the wearing state score and the threshold is larger than a predetermined value, the interval of the wearing determination is set to a long time, and when the difference between the wearing state score and the first threshold is smaller than a predetermined value, the interval of the wearing determination is set to a short time. As a result, when the wearing state score is close to the threshold, such as around times t1, t2, t3, and t4 in FIG. 13, the interval of the wearing determination is long, so that a change in state due to a short-term change in the wearing state score is unlikely to occur. Therefore, when the wearing state score decreases for only a short time, such as from time t2 to time t3 in FIG. 13, the wearing state is likely to be maintained.

[0084] As described above, in the present embodiment, in cases such as when the wearing state score fluctuates in a short period of time, a wearing determination process is realized that makes it difficult for the state to switch. This reduces the likelihood of a situation occurring in which the convenience of the user 3 is impaired, such as when the earphones 2 are actually worn but are determined to be not worn, causing the earphones 2 to become unusable. Therefore, according to the present embodiment, in addition to obtaining the same effects as the first embodiment, the convenience of the user can be improved.

[0085] The system described in the above-mentioned embodiments can also be configured as in the following fourth embodiment.

[0086] [Fourth embodiment] 15 is a functional block diagram of an information processing device 40 according to the fourth embodiment. The information processing device 40 includes an acoustic information acquisition unit 411 and a wearing determination unit 412. The acoustic information acquisition unit 411 acquires acoustic information related to resonance in the body of a user wearing a wearable device. The wearing determination unit 412 determines whether or not the user is wearing the wearable device based on the acoustic information.

[0087] According to this embodiment, an information processing device 40 is provided that can determine whether a wearable device is worn in a wider range of environments.

[0088] [Modified embodiment] The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of the present invention.

[0089] In the above embodiment, the earphone 2 is used as an example of a wearable device, but as long as it is possible to obtain the acoustic information required for processing, the wearable device is not limited to being worn in the ear. For example, the wearable device may be a bone conduction acoustic device.

[0090] In the above embodiment, the frequency range of the sound used for the fitting determination is within the audible range of 20 kHz or less, as shown in Fig. 8, but is not limited thereto, and the test sound may be an inaudible sound. For example, if the frequency characteristics of the speaker 26 and the microphone 27 are compatible with the ultrasonic band, the test sound may be an ultrasonic sound. In this case, the discomfort caused by hearing the test sound during the fitting determination is reduced.

[0091] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-mentioned embodiment is recorded on a storage medium, the program recorded on the storage medium is read as code, and executed on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. In addition, not only the storage medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment. Furthermore, one or more components included in the above-mentioned embodiments may be circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays) configured to realize the functions of each component.

[0092] Examples of the storage medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disk)-ROM, a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the storage medium, but also those that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0093] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0094] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0095] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0096] (Appendix 1) an acoustic information acquisition unit that acquires acoustic information related to resonance in the body of a user who wears the wearable device; a wearing determination unit that determines whether the user is wearing the wearable device based on the acoustic information; An information processing device comprising:

[0097] (Appendix 2) the acoustic information includes information regarding resonances in the user's vocal tract; 2. An information processing device according to claim 1.

[0098] (Appendix 3) The wearing determination unit determines whether or not the user is wearing the wearable device based on a peak of a signal at a frequency corresponding to resonance in the vocal tract. 3. An information processing device according to claim 2.

[0099] (Appendix 4) The acoustic information includes information regarding resonance in the user's ear canal. 4. The information processing device according to claim 1 .

[0100] (Appendix 5) The wearing determination unit determines whether or not the user is wearing the wearable device based on a peak of a signal at a frequency corresponding to resonance in the ear canal. 5. The information processing device according to claim 4.

[0101] (Appendix 6) The wearable device includes a sound wave generating unit that generates sound waves toward the ear canal of the user. 6. An information processing device according to any one of claims 1 to 5.

[0102] (Appendix 7) a sound generation control unit that controls the sound wave generation unit to emit sound waves when a sound pressure level included in the acoustic information is not sufficient for the determination by the wearing determination unit, 7. The information processing device according to claim 6.

[0103] (Appendix 8) The wearing determination unit determines whether or not the user is wearing the wearable device based on an echo time from when the sound wave is emitted from the sound wave generation unit to when an echo sound is acquired by the wearable device. 8. The information processing device according to claim 6 or 7.

[0104] (Appendix 9) the reverberation time being based on the round trip time of sound waves in the user's ear canal; 9. The information processing device according to claim 8.

[0105] (Appendix 10) The sound wave generated by the sound wave generating unit has a frequency characteristic based on a chirp signal, an M-sequence signal, or white noise. 10. The information processing device according to claim 6,

[0106] (Appendix 11) a notification information generating unit configured to generate notification information for prompting the user to speak when a sound pressure level included in the acoustic information is not sufficient for the judgment by the wearing judgment unit, 11. An information processing device according to any one of claims 1 to 10.

[0107] (Appendix 12) the wearing determination unit determines whether the user is wearing the wearable device based on a magnitude relationship between the score based on the acoustic information and a first threshold value. 12. An information processing device according to any one of claims 1 to 11.

[0108] (Appendix 13) After the score changes from a state in which the score is equal to or greater than the first threshold to a state in which the score is less than the first threshold, the wearable device stops at least a part of its functions. 13. The information processing device according to claim 12.

[0109] (Appendix 14) When the score changes to a state where it is smaller than the first threshold value and then the score changes to equal to or greater than the first threshold value again within a predetermined period of time, the wearable device does not stop the at least some of the functions. 14. The information processing device according to claim 13.

[0110] (Appendix 15) The wearing determination unit determines whether or not the user is wearing the wearable device based on a second threshold value that is smaller than the first threshold value, When the score does not change to a state smaller than the second threshold after the score changes from a state equal to or larger than the first threshold to a state smaller than the first threshold, the wearable device does not stop the at least some of the functions. 14. The information processing device according to claim 13.

[0111] (Appendix 16) The wearable device is an acoustic device that is worn on the user's ear. 16. An information processing device according to any one of claims 1 to 15.

[0112] (Appendix 17) The acoustic information includes information regarding sounds generated within the user's body. 17. An information processing device according to any one of claims 1 to 16.

[0113] (Appendix 18) The wearing determination unit determines whether or not the user is wearing the wearable device based on a sound pressure level of a frequency corresponding to a sound generated inside the user's body. 18. The information processing device according to claim 17.

[0114] (Appendix 19) The wearing determination unit determines whether or not the user is wearing the wearable device based on the acoustic information acquired by a plurality of microphones arranged at different positions. 19. An information processing device according to any one of claims 1 to 18.

[0115] (Appendix 20) A wearable device, an acoustic information acquisition unit that acquires acoustic information related to resonance in the body of a user who wears the wearable device; a wearing determination unit that determines whether the user is wearing the wearable device based on the acoustic information; A wearable device comprising:

[0116] (Appendix 21) Obtaining acoustic information related to resonance in the body of a user wearing the wearable device; determining whether the user is wearing the wearable device based on the acoustic information; An information processing method comprising:

[0117] (Appendix 22) On the computer, Obtaining acoustic information related to resonance in the body of a user wearing the wearable device; determining whether the user is wearing the wearable device based on the acoustic information; A storage medium on which a program for executing the above is stored. [Explanation of symbols]

[0118] 1 Information and communication devices 2. Earphones 3 Users 20 Earphone control device 26 Speaker 27, 28 Microphone 40 Information processing device 101, 201 CPU 102, 202 RAM 103, 203 ROM 104 HDD 105, 207 Communication I / F 106 Input Devices 107 Output Device 204 Flash Memory 205 Speaker I / F 206 Microphone I / F 208 Battery 211, 411 Acoustic information acquisition section 212, 412 Wearing judgment section 213 Sound control section 214 Notification information generation unit 215 Storage section

Claims

1. an acoustic information acquisition unit that acquires acoustic information related to resonance in the body of a user who wears the wearable device; a wearing determination unit that determines whether the user is wearing the wearable device based on a magnitude relationship between the score based on the acoustic information and a first threshold; Equipped with the wearing determination unit repeatedly determines whether the user is wearing the wearable device at intervals corresponding to the score; the interval is set to a long time when the score is greater than a predetermined value, and set to a short time when the score is less than the predetermined value; When the wearing determination unit determines that the user is not wearing the wearable device, the wearable device stops at least a portion of its functions for a predetermined period of time.

2. the information processing device further includes an authentication unit that performs authentication of the user based on the acoustic signal; The information processing device according to claim 1 , wherein the wearable device resumes at least a portion of the function after the predetermined time has elapsed.

3. The acoustic information includes information regarding resonance in the user's ear canal.

3. The information processing device according to claim 1 or 2.

4. The wearable device includes a sound wave generating unit that generates sound waves toward the ear canal of the user. The information processing device according to claim 1 .

5. a sound generation control unit that controls the sound wave generation unit to emit sound waves when a sound pressure level included in the acoustic information is not sufficient for the determination by the wearing determination unit, The information processing device according to claim 4.

6. The wearing determination unit determines whether or not the user is wearing the wearable device based on an echo time from when the sound wave is emitted from the sound wave generation unit to when an echo sound is acquired by the wearable device.

6. The information processing device according to claim 4 or 5.

7. A wearable device, an acoustic information acquisition unit that acquires acoustic information related to resonance in the body of a user who wears the wearable device; a wearing determination unit that determines whether the user is wearing the wearable device based on a magnitude relationship between the score based on the acoustic information and a first threshold; a stop unit that stops at least a part of a function of the wearable device for a predetermined time when the wearing determination unit determines that the user is not wearing the wearable device; Equipped with the wearing determination unit repeatedly determines whether the user is wearing the wearable device at intervals corresponding to the score; A wearable device, wherein the interval is set to a long time when the score is greater than a predetermined value, and set to a short time when the score is less than the predetermined value.

8. Obtaining acoustic information related to resonance in the body of a user wearing the wearable device; determining whether the user is wearing the wearable device based on a magnitude relationship between the score based on the acoustic information and a first threshold; When it is determined that the user is not wearing the wearable device, stopping at least a part of the function of the wearable device for a predetermined time; Equipped with the determining step repeatedly determines whether the user is wearing the wearable device at intervals corresponding to the score; An information processing method, wherein the interval is set to a long time when the score is greater than a predetermined value, and is set to a short time when the score is less than the predetermined value.

9. On the computer, Obtaining acoustic information related to resonance in the body of a user wearing the wearable device; determining whether the user is wearing the wearable device based on a magnitude relationship between the score based on the acoustic information and a first threshold; When it is determined that the user is not wearing the wearable device, stopping at least a part of the function of the wearable device for a predetermined time; A program for executing the determining step repeatedly determines whether the user is wearing the wearable device at intervals corresponding to the score; The interval is set to a long time when the score is greater than a predetermined value, and is set to a short time when the score is less than the predetermined value.

Images