Information processing device, wearable device, information processing method, and storage medium

By switching operation modes based on wear detection, the power consumption of wearable devices performing biometric authentication is reduced, addressing the high power demands of such technologies.

JP7582390B2Active Publication Date: 2024-11-13NEC CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023114825
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-07-13
Publication Date
2024-11-13
Estimated Expiration
2038-12-19

AI Technical Summary

Technical Problem

Biometric authentication using acoustic characteristics in wearable devices requires significant power consumption, which is a challenge for devices like wireless earphones that rely on battery power.

Method used

Implementing a wearing determination unit to switch between operation modes based on whether the device is being worn, including a mode where biometric authentication is disabled to conserve power.

Benefits of technology

This approach reduces power consumption in wearable devices by limiting biometric authentication processes when not in use, extending battery life and improving device usability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582390000001
    Figure 0007582390000001
  • Figure 0007582390000002
    Figure 0007582390000002
  • Figure 0007582390000003
    Figure 0007582390000003
Patent Text Reader

Abstract

To reduce power consumption in a mounting-type apparatus for performing biometric authentication based on acoustic characteristics.SOLUTION: An information processing device according to one embodiment of the present invention comprises: a speaker for emitting a sound wave to perform biometric authentication based on acoustic characteristics to a part of a head of a user; a microphone for receiving echo sound obtained by echoing of the sound wave in a part of the head of the user; a mounting determination section for determining whether the information processing device is mounted on the user; and an operation mode control section for switching operation modes of a mounting-type apparatus between a plurality of operation modes including a first operation mode not capable of executing processing for biometric authentication and a second operation mode capable of executing the processing for the biometric authentication, based on a determination result of the mounting determination section. The power consumption of the second operation mode is larger than the power consumption of the first operation mode.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device, a wearable device, an information processing method, and a storage medium. [Background technology]

[0002] Patent Document 1 discloses an earphone that performs personal authentication based on the acoustic characteristics of the user's ear canal and the like. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 034178 Summary of the Invention [Problem to be solved by the invention]

[0004] A large amount of power is required for processing biometric authentication using acoustic characteristics as shown in Patent Document 1. Therefore, there are cases where a reduction in power consumption is required in a wearable device that performs biometric authentication based on acoustic characteristics.

[0005] An object of the present invention is to provide an information processing device, a wearable device, an information processing method, and a storage medium that can reduce power consumption in a wearable device that performs biometric authentication based on acoustic characteristics. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information processing device comprising: a wearability determination unit that determines whether a user is wearing a wearable device that emits sound waves to a part of the user's head for biometric authentication based on acoustic characteristics; and an operation mode control unit that switches the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which processing for the biometric authentication cannot be performed and a second operation mode in which processing for the biometric authentication can be performed based on a result of the determination by the wearability determination unit.

[0007] According to another aspect of the present invention, there is provided a wearable device that emits sound waves to a part of a user's head for performing biometric authentication based on acoustic characteristics, the wearable device comprising: a wearing determination unit that determines whether the user is wearing the wearable device; and an operation mode control unit that switches the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which processing for the biometric authentication cannot be performed and a second operation mode in which processing for the biometric authentication can be performed, based on a result of the determination by the wearing determination unit.

[0008] According to another aspect of the present invention, there is provided an information processing method comprising the steps of: determining whether a user is wearing a wearable device that emits sound waves to a part of the user's head for biometric authentication based on acoustic characteristics; and, based on the result of the determination, switching the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which processing for the biometric authentication cannot be performed and a second operation mode in which processing for the biometric authentication can be performed.

[0009] According to another aspect of the present invention, there is provided a storage medium having stored therein a program for causing a computer to execute the steps of: determining whether a user is wearing a wearable device that emits sound waves to a part of the user's head for biometric authentication based on acoustic characteristics; and, based on the result of the determination, switching the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which processing for the biometric authentication cannot be performed and a second operation mode in which processing for the biometric authentication can be performed. Effect of the Invention

[0010] According to the present invention, it is possible to provide an information processing device, a wearable device, an information processing method, and a storage medium that can reduce power consumption in a wearable device that performs biometric authentication based on acoustic characteristics. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram showing an overall configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration of the earphone according to the first embodiment. FIG. [Diagram 3] 1 is a block diagram showing a hardware configuration of an information communication device according to a first embodiment. [Figure 4] 1 is a functional block diagram of an earphone control device according to a first embodiment. FIG. [Diagram 5] FIG. 4 is a state transition diagram of the earphone according to the first embodiment. [Figure 6] 4 is a flowchart showing an example of an operational mode transition performed by the earphone control device according to the first embodiment. [Figure 7] 5 is a graph showing an example of changes in power consumption of the earphone according to the first embodiment. [Figure 8] 13 is a functional block diagram of an earphone control device and an information communication device according to a second embodiment. FIG. [Figure 9] 13 is a flowchart showing a process of ear acoustic authentication according to the second embodiment. [Figure 10] FIG. 13 is a functional block diagram of an earphone control device and an information communication device according to a third embodiment. [Figure 11] FIG. 11 is a state transition diagram of the earphone according to the third embodiment. [Figure 12] 13 is a flowchart showing an example of operational mode transitions performed by an earphone control device according to a third embodiment. [Figure 13] 13 is a graph showing an example of a change in power consumption of the earphone according to the third embodiment. [Figure 14] FIG. 13 is a functional block diagram of an information processing device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, exemplary embodiments of the present invention will be described with reference to the drawings. In the drawings, similar or corresponding elements are designated by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0013] [First embodiment] An information processing system according to the present embodiment will be described. The information processing system according to the present embodiment is a system for detecting the wearing of a wearable device such as an earphone and for performing biometric authentication.

[0014] 1 is a schematic diagram showing the overall configuration of an information processing system according to this embodiment. The information processing system includes an information communication device 1 and earphones 2 that can be wirelessly connected to each other.

[0015] The earphone 2 includes an earphone control device 20, a speaker 26, a microphone 27, and an infrared sensor 29. The earphone 2 is an acoustic device that can be worn in the ear of a user 3, and is typically a wireless device such as a wireless earphone or a wireless headset. The speaker 26 functions as a sound wave generating unit that emits sound waves toward the ear canal of the user 3 when worn, and is arranged on the wearing surface side of the earphone 2. The microphone 27 is also arranged on the wearing surface side of the earphone 2 so that it can receive sound waves reverberated in the ear canal of the user 3 when worn. The infrared sensor 29 functions as a wearing detection unit that detects infrared rays emitted from the user 3 when worn, and is composed of a photodiode or the like. The infrared sensor 29 is arranged on the wearing surface side of the earphone 2. The earphone control device 20 controls the speaker 26, the microphone 27, and the infrared sensor 29, and communicates with the information and communication device 1.

[0016] In this specification, "sound" such as sound waves and voice includes inaudible sounds whose frequencies or sound pressure levels are outside the audible range.

[0017] The information and communication device 1 is, for example, a computer communicatively connected to the earphones 2, and controls the operation of the earphones 2, transmits audio data for generating sound waves emitted from the earphones 2, and receives audio data obtained from sound waves received by the earphones 2. As a specific example, when a user 3 listens to music using the earphones 2, the information and communication device 1 transmits compressed data of the music to the earphones 2. Furthermore, when the earphones 2 are telephone devices for issuing operational instructions at an event venue, hospital, or the like, the information and communication device 1 transmits audio data of operational instructions to the earphones 2. In this case, audio data of the speech of the user 3 may also be transmitted from the earphones 2 to the information and communication device 1. Furthermore, the information and communication device 1 or the earphones 2 may have a function of ear acoustic authentication using sound waves received by the earphones 2.

[0018] This overall configuration is just an example, and for example, the information communication device 1 and the earphones 2 may be connected by wire. Also, the information communication device 1 and the earphones 2 may be configured as an integrated device, or another device may be included in the information processing system.

[0019] 2 is a block diagram showing an example of the hardware configuration of earphone control device 20. Earphone control device 20 includes a central processing unit (CPU) 201, random access memory (RAM) 202, read only memory (ROM) 203, and flash memory 204. Earphone control device 20 also includes a speaker interface (I / F) 205, a microphone I / F 206, a communication I / F 207, a battery 208, and an infrared sensor I / F 210. The various components of earphone control device 20 are connected to each other via buses, wiring, drivers, etc. (not shown).

[0020] CPU 201 is a processor that performs predetermined calculations according to programs stored in ROM 203, flash memory 204, etc., and also has the function of controlling each part of earphone control device 20. RAM 202 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of CPU 201. ROM 203 is composed of a non-volatile storage medium, and stores necessary information such as programs used in the operation of earphone control device 20. Flash memory 204 is composed of a non-volatile storage medium, and is a storage device that temporarily stores data, stores programs for operating earphone control device 20, etc.

[0021] The communication I / F 207 is a communication interface based on standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), and is a module for communicating with the information communication device 1.

[0022] The speaker I / F 205 is an interface for driving the speaker 26. The speaker I / F 205 includes a digital-to-analog conversion circuit, an amplifier, etc. The speaker I / F 205 converts the audio data into an analog signal and supplies it to the speaker 26. As a result, the speaker 26 emits sound waves based on the audio data.

[0023] The microphone I / F 206 is an interface for acquiring a signal from the microphone 27. The microphone I / F 206 includes an analog-to-digital conversion circuit, an amplifier, etc. The microphone I / F 206 converts an analog signal generated by a sound wave received by the microphone 27 into a digital signal. In this way, the earphone control device 20 acquires audio data based on the received sound waves.

[0024] The battery 208 is, for example, a secondary battery, and supplies the power necessary for the operation of the earphone 2. Since the battery 208 is built into the earphone 2, the earphone 2 can operate wirelessly without being connected to an external power source by wire.

[0025] The infrared sensor I / F 210 is an interface for acquiring a signal from the infrared sensor 29. The infrared sensor I / F 210 includes an analog-to-digital conversion circuit, an amplifier, etc. The infrared sensor I / F 210 converts an analog signal generated by the infrared light received by the infrared sensor 29 into a digital signal. In this way, the earphone control device 20 acquires detection data of the user 3 based on the received infrared light.

[0026] Note that the hardware configuration shown in FIG. 2 is an example, and other devices may be added, or some devices may not be provided. Also, some devices may be replaced with other devices having similar functions. For example, the earphone 2 may further include an input device such as a button to accept an operation by the user 3, and may further include a display device such as a display or an indicator light to provide information to the user 3. In this way, the hardware configuration shown in FIG. 2 can be changed as appropriate.

[0027] 3 is a block diagram showing an example of a hardware configuration of the information communication device 1. The information communication device 1 includes a CPU 101, a RAM 102, a ROM 103, and an HDD (Hard Disk Drive) 104. The information communication device 1 also includes a communication I / F 105, an input device 106, and an output device 107. The components of the information communication device 1 are connected to each other via a bus, wiring, a drive device, etc. (not shown).

[0028] 3, the components constituting the information communication device 1 are illustrated as an integrated device, but some of these functions may be provided by external devices. For example, the input device 106 and the output device 107 may be external devices separate from the components constituting the functions of the computer, including the CPU 101, etc.

[0029] The CPU 101 is a processor that performs predetermined calculations according to programs stored in the ROM 103, HDD 104, etc., and also has the function of controlling each part of the information communication device 1. The RAM 102 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of the CPU 101. The ROM 103 is composed of a non-volatile storage medium, and stores necessary information such as programs used in the operation of the information communication device 1. The HDD 104 is composed of a non-volatile storage medium, and is a storage device that temporarily stores data to be transmitted to and received from the earphones 2, stores programs for operating the information communication device 1, etc.

[0030] The communication I / F 105 is a communication interface based on standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), and is a module for communicating with other devices such as the earphones 2.

[0031] The input device 106 is a keyboard, a pointing device, etc., and is used by the user 3 to operate the information communication device 1. Examples of the pointing device include a mouse, a trackball, a touch panel, and a pen tablet.

[0032] The output device 107 is, for example, a display device. The display device is a liquid crystal display, an OLED (Organic Light Emitting Diode) display, etc., and is used to display information, a GUI (Graphical User Interface) for operation input, etc. The input device 106 and the output device 107 may be integrally formed as a touch panel.

[0033] The hardware configuration shown in FIG. 3 is an example, and other devices may be added, or some devices may not be provided. Some devices may be replaced with other devices having similar functions. Furthermore, some functions of this embodiment may be provided by other devices via a network, or the functions of this embodiment may be distributed and realized among a plurality of devices. For example, the HDD 104 may be replaced with an SSD (Solid State Drive) using a semiconductor memory, or may be replaced with cloud storage. In this way, the hardware configuration shown in FIG. 3 can be changed as appropriate.

[0034] 4 is a functional block diagram of the earphone control device 20 according to this embodiment. The earphone control device 20 includes a wearing determination unit 211, an ear acoustic authentication unit 212, an operation mode control unit 213, and a storage unit 214.

[0035] The CPU 201 loads a program stored in the ROM 203, the flash memory 204, etc., into the RAM 202 and executes it. In this way, the CPU 201 realizes the functions of the wearing determination unit 211, the ear acoustic authentication unit 212, and the operation mode control unit 213. The CPU 201 also realizes the function of the storage unit 214 by controlling the flash memory 204 based on the program. Specific processing performed by each of these units will be described later.

[0036] In this embodiment, the functions of the functional blocks in Fig. 4 are provided in the earphone control device 20, but some of the functions of the functional blocks in Fig. 4 may also be provided in the information and communication device 1. That is, the functions described above may be realized by the earphone control device 20, may be realized by the information and communication device 1, or may be realized by cooperation between the information and communication device 1 and the earphone control device 20. The information and communication device 1 and the earphone control device 20 are sometimes more generally referred to as information processing devices.

[0037] 5 is a state transition diagram of the earphone 2 according to this embodiment. First, an overview of the operation of the earphone control device 20 will be described with reference to FIG.

[0038] The earphone 2 can operate in three modes: a first operation mode, a second operation mode, and a third operation mode, and transitions between the modes are controlled by an operation mode control unit 213 of the earphone control device 20. The operation modes differ in whether or not there are restrictions on the functions that the earphone 2 can execute. The first operation mode and the second operation mode are operation modes for determining whether the earphone 2 is being worn and for biometric authentication, respectively. In these modes, the original functions of the earphone 2 (listening to sound, etc.) are restricted.

[0039] In the first operation mode, the wearing determination unit 211 can determine whether or not the user 3 is wearing the earphones 2. When the user 3 is wearing the earphones 2, infrared rays emitted from the user 3 are incident on the infrared sensor 29, and the intensity of the infrared rays received by the infrared sensor 29 becomes large. Therefore, the wearing determination unit 211 can determine whether or not the earphones are being worn based on a digital signal indicating the intensity of the infrared rays received by the infrared sensor 29. The determination criterion by the wearing determination unit 211 can be, for example, that the user 3 is wearing the earphones 2 when the intensity of the infrared rays is equal to or greater than a threshold value. Alternatively, the determination criterion can be that a wearing score is calculated based on the intensity of the infrared rays or a change therein, and that the user 3 is wearing the earphones 2 is determined to be worn when the wearing score is equal to or greater than a threshold value.

[0040] In the first operation mode, when the wearing determination unit 211 determines that the user 3 is wearing the earphones 2, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the second operation mode. In the first operation mode, when the wearing determination unit 211 determines that the user 3 is not wearing the earphones 2, the operation mode control unit 213 keeps the operation mode of the earphones 2 in the first operation mode.

[0041] However, in the first operation mode, the earacoustic authentication function by the earacoustic authentication unit 212 is limited, and earacoustic authentication cannot be performed. The second operation mode is an operation mode in which earacoustic authentication can be performed. In other words, the operation mode control unit 213 releases the functional restriction on earacoustic authentication by transitioning the operation mode of the earphones 2 from the first operation mode to the second operation mode.

[0042] In the second operation mode, the ear acoustic authentication unit 212 can determine whether the user 3 is a registered person by ear acoustic authentication. The ear acoustic authentication is a biometric authentication that checks the acoustic characteristics of the ear canal of the user 3. The speaker 26 emits a test sound wave to the ear canal of the user 3, and the microphone 27 receives the sound wave reflected in the ear canal of the user 3, so that the earphone 2 can acquire the acoustic characteristics of the ear canal of the user 3. The ear acoustic authentication unit 212 can determine whether the user 3 is a registered person by checking the feature amount extracted from the acquired acoustic characteristics of the ear canal with the feature amount extracted from the acoustic characteristics of the ear canal of the registered person. The registered person is a person whose acoustic characteristics or feature amount of the ear canal are registered as a regular user of the earphone 2. By performing this authentication, the functions of the earphone 2 can be restricted to unauthorized persons, and the earphone 2 can be prevented from being used by unauthorized persons.

[0043] In this embodiment, the acoustic characteristics of the ear canal of the registrant are stored in advance in the memory unit 214, but the acoustic characteristics of the ear canal of the registrant may also be obtained from another device such as the information communication device 1 at the time of authentication.

[0044] Note that the acoustic characteristics acquired in ear acoustic authentication are typically acoustic characteristics caused by resonance in the ear canal, but may also include characteristics of acoustic characteristics caused by reverberation in tissues surrounding the ear canal, such as the skull. Also, the test sound wave does not have to be emitted directly toward the ear canal, and may be emitted to a part of the head of the user 3 by, for example, a bone conduction speaker.

[0045] In the second operation mode, if the ear acoustic authentication unit 212 determines that the user 3 is a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the third operation mode. In the second operation mode, if the ear acoustic authentication unit 212 determines that the user 3 is not a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the first operation mode.

[0046] The third operation mode is an operation mode in which the user 3 can use the earphone 2. In the third operation mode, the function restrictions of the earphone 2 are released, so that the user 3 can enjoy the original functions of the earphone 2. For example, if the earphone 2 is a telephone device for business commands, in the third operation mode, the earphone 2 can receive voice data of business commands and emit the voice from the speaker 26. Note that the third operation mode may be a mode in which all function restrictions of the earphone 2 are released. Furthermore, if the functions available to each user are different, the third operation mode may be a mode in which the part of the function restrictions of the earphone 2 that is permitted to an authenticated user is released.

[0047] Furthermore, in the third operation mode, wearing detection may be performed by the wearing determination unit 211. In the third operation mode, if the wearing determination unit 211 determines that the user 3 is not wearing the earphones 2, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the first operation mode.

[0048] Next, transitions in the operation mode of the earphone 2 and the associated changes in power consumption will be described with reference to Figs. 6 and 7. Fig. 6 is a flowchart showing an example of transitions in the operation mode performed by the earphone control device 20 according to the present embodiment. Fig. 7 is a graph showing an example of changes in power consumption of the earphone 2 according to the present embodiment. Figs. 6 and 7 show typical processing when the user 3 puts on and uses the earphone 2, but depending on the circumstances of the transition of the operation mode, the flow may differ from that shown in the drawings. Note that the scale of this graph is not necessarily accurate, and this graph merely shows a schematic diagram of changes in power consumption accompanying processing.

[0049] The process in Fig. 6 is executed, for example, every time a predetermined time elapses while the power of the earphones 2 is on. Alternatively, the process in Fig. 6 may be executed when the user 3 operates the earphones 2 to start using the earphones 2.

[0050] In step S101, the operation mode control unit 213 sets the operation mode of the earphone 2 to the first operation mode. The processing time of step S101 corresponds to time t1 in Fig. 7. As shown in Fig. 7, the power consumption of the earphone 2 at time t1 is P1.

[0051] In step S102, the wearing determination unit 211 performs a process for wearing determination. This process for wearing determination may include obtaining data indicating the intensity of the infrared rays as described above.

[0052] In step S103, the wearing determination unit 211 determines whether or not the user 3 is wearing the earphones 2. If it is determined that the user 3 is wearing the earphones 2 (YES in step S103), the process proceeds to step S104. If it is determined that the user 3 is not wearing the earphones 2 (NO in step S103), the process returns to step S101, and the operation mode is maintained in the first operation mode.

[0053] In step S104, the operation mode control unit 213 transitions the operation mode of the earphone 2 to the second operation mode. The processing time of step S104 corresponds to time t2 in Fig. 7. As shown in Fig. 7, at time t2, the power consumption of the earphone 2 changes from P1 to P2, which is greater than P1.

[0054] The difference in power consumption between the first operation mode and the second operation mode will be described. As described above, the second operation mode differs from the first operation mode in that ear acoustic authentication can be performed. Ear acoustic authentication includes, for example, computational processing such as a process of extracting features from acoustic characteristic data and a process of matching features. In this case, the power consumption required for ear acoustic authentication is {(amount of computation for feature extraction + amount of computation for matching) x power consumption required per unit amount of computation}. This power consumption is too large to be ignored considering that the earphone 2 is a wireless device and receives power from the built-in battery 208, and that the earphone 2 is a small device worn on the ear and therefore the power capacity of the battery 208 is not very large. Therefore, the power consumption of the earphone 2 is greater in the second operation mode than in the first operation mode.

[0055] In step S105, the earacoustic authentication unit 212 performs processing for earacoustic authentication. This processing for earacoustic authentication may include obtaining the acoustic characteristics of the ear canal, extracting features, and comparing the features, as described above.

[0056] In step S106, the ear acoustic authentication unit 212 determines whether the user 3 is a registrant. If it is determined that the user 3 is a registrant (YES in step S106), the process proceeds to step S107. If it is determined that the user 3 is not a registrant (NO in step S106), the process returns to step S101, and the operation mode transitions to the first operation mode.

[0057] In step S107, the operation mode control unit 213 transitions the operation mode of the earphone 2 to the third operation mode. The processing time of step S107 corresponds to time t3 in Fig. 7. As shown in Fig. 7, at time t3, the power consumption of the earphone 2 changes from P2 to P3, which is greater than P2. In the third operation mode, processing such as communication of audio data between the information communication device 1 and the earphone 2 and transmission of sound waves by the speaker 26 is performed, so that power consumption is even greater than in the second operation mode.

[0058] In steps S108 and S109, the wearing detection process for the earphones 2 is performed in the same manner as in steps S102 and S103. If it is determined that the user 3 is wearing the earphones 2 (YES in step S109), the operation mode is maintained in the third operation mode, and the wearing detection process is performed at predetermined intervals thereafter. If it is determined that the user 3 is not wearing the earphones 2 (NO in step S109), the process returns to step S101, and the operation mode transitions to the first operation mode. Assumed cases in which it may be determined that the earphones 2 are not being worn after the authentication is completed include, for example, a case in which the user 3 suspends or ends use of the earphones 2 and removes the earphones 2. In this case, the operation mode transitions to the first operation mode, and the functions of the earphones 2 are restricted so that the earphones 2 cannot be used by others.

[0059] The time at which it is determined in step S109 that the user 3 is not wearing the earphone 2 corresponds to time t4 in Fig. 7. As shown in Fig. 7, at time t4, the power consumption of the earphone 2 changes from P3 to P1, which is smaller than P3.

[0060] The earphone control device 20 of this embodiment can switch the operation mode of the earphone 2 between a first operation mode in which processing for earacoustic authentication cannot be performed, and a second operation mode in which processing for earacoustic authentication can be performed. In this embodiment, the operation mode is switched based on the result of wearing determination, so that control is realized such that earacoustic authentication is not performed when a person is not wearing the earphone 2. This reduces power consumption caused by earacoustic authentication of the earphone 2. Therefore, according to this embodiment, an information processing device is provided that can reduce power consumption in a wearable device that performs biometric authentication based on acoustic characteristics.

[0061] Because the earphones 2 are typically wireless wearable devices that are worn in the ears, the battery 208 that drives the earphones 2 has a small power capacity. Therefore, it is effective to reduce power consumption through the above-mentioned control, and the usable time of the earphones 2 can be extended.

[0062] [Second embodiment] The information processing system of this embodiment differs from the first embodiment in that part of the ear acoustic authentication process is performed outside the earphone 2. The following mainly describes the differences from the first embodiment, and descriptions of common parts will be omitted or simplified.

[0063] 8 is a functional block diagram of the earphone control device 20 and the information communication device 1 according to this embodiment. The earphone control device 20 has a feature acquisition unit 215 instead of the ear acoustic authentication unit 212 of the first embodiment. The information communication device 1 has a feature matching unit 111 and a storage unit 112.

[0064] The CPU 201 of the earphone control device 20 executes a program stored in the ROM 203, the flash memory 204, etc., to realize the function of the feature acquisition unit 215. The CPU 101 of the information communication device 1 executes a program stored in the ROM 103, the HDD 104, etc., to realize the function of the feature matching unit 111. The CPU 101 also executes the function of the storage unit 112 by controlling the HDD 104 based on the program. The specific processing performed by each of these units will be described later.

[0065] Fig. 9 is a flowchart showing the ear acoustic authentication process according to this embodiment. The process in Fig. 9 shows the process of step S105 in Fig. 6 of the first embodiment in more detail. The rest of the process is the same as in Fig. 6, so the description will be omitted.

[0066] In step S110, the feature amount acquiring unit 215 of the earphone control device 20 acquires acoustic characteristics of the ear canal of the user 3 and extracts features from the acoustic characteristics. In this way, the feature amount acquiring unit 215 acquires features based on the acoustic characteristics of the ear canal of the user 3.

[0067] In step S111, the earphones 2 transmit the feature amount to the information communication device 1. The information communication device 1 stores the acquired feature amount in the storage unit 112.

[0068] In step S112, the feature matching unit 111 of the information communication device 1 matches the feature of the user 3 with the feature of a registered person stored in the storage unit 112 in advance.

[0069] In step S113, the information communication device 1 transmits the matching result to the earphone 2. The CPU 201 of the earphone 2 uses the matching result to perform the branching process of step S106 in FIG.

[0070] In this embodiment, the matching process of the ear acoustic authentication process is performed by the information communication device 1 outside the earphones 2. Even with this configuration, ear acoustic authentication can be performed as in the first embodiment. Therefore, an information processing device is provided that can reduce power consumption in a wearable device that performs biometric authentication based on acoustic characteristics as in the first embodiment.

[0071] There are advantages and disadvantages to the configuration in which ear acoustic authentication is performed by the earphones 2 as in the first embodiment, and the configuration in which the matching process, which is part of the ear acoustic authentication process, is performed by a device external to the earphones 2 as in the second embodiment. Therefore, it is desirable to appropriately select one of the above two configurations depending on the required specifications. The advantages and disadvantages of both are explained below.

[0072] In the configuration of the first embodiment, the ear acoustic authentication process is completed inside the earphone 2, so there is no need to communicate with other devices during ear acoustic authentication, which may have the advantage of reducing power consumption due to communication. In contrast, the configuration of the first embodiment has the disadvantages that it requires the features of the registered person to be stored in advance inside the earphone 2, and that if a matching algorithm with a large amount of calculation is used, there is a risk of power consumption increasing.

[0073] In the configuration of the second embodiment, the power consumption required for ear acoustic authentication is (computation amount of feature extraction × power consumption required per unit calculation amount + communication amount of feature transmission × power consumption required per unit communication amount). Therefore, when the power consumption required for communication is less than the power consumption required for matching, the configuration of the second embodiment may be able to reduce power consumption. In addition, the configuration of the second embodiment may have the advantage that it is not necessary to store the features of the registered person in advance inside the earphone 2, and that since the registered person is managed by the upper device, it is easy to deal with cases where the user is unspecified or when the user is updated. On the other hand, it may have the disadvantage that when the communication amount is large, the power consumption is larger than that of the configuration of the first embodiment, and since communication is essential, the earphone 2 cannot be used in the event of a communication failure.

[0074] [Third embodiment] The information processing system of this embodiment differs from the first or second embodiment in that the ear acoustic authentication process can be performed both inside and outside the earphone 2. The following mainly describes the differences with the first or second embodiment, and descriptions of common parts will be omitted or simplified.

[0075] Fig. 10 is a functional block diagram of earphone control device 20 and information communication device 1 according to this embodiment. Earphone control device 20 has a first feature matching unit 216 in addition to the configuration shown in Fig. 8. Information communication device 1 has a second feature matching unit 113 instead of feature matching unit 111 shown in Fig. 8.

[0076] The CPU 201 of the earphone control device 20 executes a program stored in the ROM 203, the flash memory 204, etc., to implement the function of the first feature matching unit 216. The CPU 101 of the information communication device 1 executes a program stored in the ROM 103, the HDD 104, etc., to implement the function of the second feature matching unit 113. The specific processing performed by each of these units will be described later.

[0077] Fig. 11 is a state transition diagram of the earphone 2 according to this embodiment. First, an overview of the operation of the earphone control device 20 will be described with reference to Fig. 11. The difference from the state transition diagram shown in Fig. 5 is that a fourth operation mode has been further added.

[0078] In this embodiment, the second operation mode is an operation mode in which matching processing for ear acoustic authentication (first biometric authentication) is possible in the earphones 2, and the fourth operation mode is an operation mode in which matching processing for ear acoustic authentication (second biometric authentication) is possible in the information communication device 1. The algorithm for the second biometric authentication in the information communication device 1 is more accurate but consumes more power than the algorithm for the first biometric authentication in the earphones 2.

[0079] In the second operation mode, the feature acquisition unit 215 acquires features by extracting them from the acoustic characteristics of the ear canal of the user 3. The first feature matching unit 216 determines whether the user 3 is a registered person by matching the features of the user 3 acquired by the feature acquisition unit 215 with the features of a registered person.

[0080] In the second operation mode, if the first feature matching unit 216 determines that the user 3 is a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the third operation mode. In the second operation mode, if the first feature matching unit 216 determines that the user 3 is not a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the first operation mode. In contrast, in the second operation mode, if it is difficult to determine whether the user 3 is a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the fourth operation mode. Note that a case where it is difficult to determine is, for example, a case where the score indicating the similarity between two feature amounts is close to a threshold value for determining whether or not the user is a registered person. In such a case, since it is desirable to perform a more accurate matching, the operation mode of the earphones 2 transitions to the fourth operation mode, which allows for a more accurate matching.

[0081] In the fourth operating mode, the second feature matching unit 113 acquires the features of the user 3 from the earphones 2 and determines whether the user 3 is a registered person by matching the features of the user 3 with the features of a registered person.

[0082] In the fourth operation mode, if the second feature matching unit 113 determines that the user 3 is a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the third operation mode. In the fourth operation mode, if the second feature matching unit 113 determines that the user 3 is not a registered person, the operation mode control unit 213 transitions the operation mode of the earphones 2 to the first operation mode.

[0083] Next, transitions of the operation mode of the earphone 2 and the associated changes in power consumption will be described with reference to Fig. 12 and Fig. 13. Fig. 12 is a flowchart showing an example of transitions of the operation mode performed by the earphone control device 20 according to the present embodiment. Fig. 13 is a graph showing an example of changes in power consumption of the earphone 2 according to the present embodiment. Figs. 12 and 13 show typical processing when the user 3 puts on and uses the earphone 2, but depending on the circumstances of the transition of the operation mode, the flow may differ from that shown in the figure.

[0084] In step S114, the feature acquisition unit 215 of the earphone control device 20 acquires acoustic characteristics of the ear canal of the user 3 and extracts features from the acoustic characteristics. In this way, the feature acquisition unit 215 acquires features based on the acoustic characteristics of the ear canal of the user 3. Thereafter, the first feature matching unit 216 matches the features of the user 3 with the features of registered individuals stored in advance in the storage unit 214.

[0085] In step S115, the first feature matching unit 216 judges whether or not the user 3 is a registered person. If it is judged that the user 3 is a registered person (YES in step S115), the process proceeds to step S107. If it is judged that the user 3 is not a registered person (NO in step S115), the process returns to step S101, and the operation mode transitions to the first operation mode. If it is difficult to judge whether or not the user 3 is a registered person ("difficult to judge" in step S115), the process proceeds to step S116.

[0086] In step S116, the operation mode control unit 213 transitions the operation mode of the earphone 2 to the fourth operation mode. The processing time of step S116 corresponds to time t5 in Fig. 13. As shown in Fig. 13, at time t5, the power consumption of the earphone 2 changes from P2 to P4, which is greater than P2. The reason that the power consumption increases is that the second ear acoustic authentication is a process that requires communication and therefore requires more power than the first ear acoustic authentication.

[0087] In step S117, the earphone 2 and the information communication device 1 perform second ear acoustic authentication. This process is similar to the processes from steps S111 to S113 in Fig. 9. That is, in the second ear acoustic authentication, the second feature matching unit 113 acquires features from the earphone 2 and matches the features of the user 3 with the features of the registrant. This matching result is transmitted from the information communication device 1 to the earphone 2. In this way, the second ear acoustic authentication differs from the first ear acoustic authentication in that the matching process is performed on the information communication device 1 side.

[0088] The CPU 201 of the earphone 2 uses the comparison result to perform branching processing of step S118. If it is determined that the user 3 is a registered person (YES in step S118), the processing proceeds to step S107. If it is determined that the user 3 is not a registered person (NO in step S118), the processing returns to step S101, and the operation mode transitions to the first operation mode.

[0089] In this embodiment, the matching process of the first ear acoustic authentication is performed by the earphone 2, and the matching process of the second ear acoustic authentication is performed by the information communication device 1. Even with this configuration, ear acoustic authentication can be performed as in the first embodiment. Therefore, an information processing device is provided that can reduce power consumption in a wearable device that performs biometric authentication based on acoustic characteristics as in the first embodiment. In addition, in this embodiment, a configuration in which matching is performed by the earphone 2 as in the first embodiment and a configuration in which matching is performed by the information communication device 1 as in the second embodiment are combined. Then, by performing high-precision matching by the information communication device 1 only when it is difficult to determine the identity by low-power and simple matching by the earphone 2, it is possible to achieve both reduced power consumption and ensure authentication accuracy.

[0090] The system described in the above-mentioned embodiments can also be configured as in the following fourth embodiment.

[0091] [Fourth embodiment] 14 is a functional block diagram of an information processing device 30 according to a fourth embodiment. The information processing device 30 includes a wearing determination unit 311 and an operation mode control unit 312. The wearing determination unit 311 determines whether or not a user is wearing a wearable device that emits sound waves to a part of the user's head for performing biometric authentication based on acoustic characteristics. The operation mode control unit 312 switches the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which processing for biometric authentication cannot be performed and a second operation mode in which processing for biometric authentication can be performed, based on the result of the determination by the wearing determination unit 311.

[0092] According to this embodiment, an information processing device 30 is provided that can reduce power consumption in a wearable device that performs biometric authentication based on acoustic characteristics.

[0093] [Modified embodiment] The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of the present invention.

[0094] In the above embodiment, the earphone 2 is used as an example of a wearable device, but as long as it is possible to obtain the acoustic information required for processing, the wearable device is not limited to being worn in the ear. For example, the wearable device may be a bone conduction acoustic device.

[0095] In the above embodiment, the infrared sensor 29 is exemplified as a means for determining whether the earphone is worn, but the present invention is not limited to this as long as the means is capable of determining whether the earphone is worn. For example, sound waves may be emitted to the ear canal of the user 3, and acoustic characteristics of the reflected sound, such as the intensity of the reflected sound and the time of the reflected sound, may be acquired, and the wearing determination may be performed based on the acoustic characteristics of the reflected sound. In this case, the speaker 26 and the microphone 27 function as a device for not only ear acoustic authentication but also for determining whether the earphone is worn. Therefore, the device configuration can be simplified.

[0096] In the second and third embodiments described above, the features are transmitted from the earphones 2 to the information communication device 1, but data on the acoustic characteristics of the ear canal may be sent instead. In this case, the feature extraction process may be performed by the information communication device 1. In this example, the amount of communication increases, but the amount of calculations in the earphones 2 can be reduced.

[0097] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-mentioned embodiment is recorded on a storage medium, the program recorded on the storage medium is read as code, and executed on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. In addition, not only the storage medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment. Furthermore, one or more components included in the above-mentioned embodiments may be circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays) configured to realize the functions of each component.

[0098] Examples of the storage medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disk)-ROM, a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the storage medium, but also those that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0099] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0100] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0101] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0102] (Appendix 1) a wearability determination unit that determines whether or not a user is wearing a wearable device that emits sound waves to a part of the user's head for biometric authentication based on acoustic characteristics; an operation mode control unit that switches the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which the process for the biometric authentication cannot be executed and a second operation mode in which the process for the biometric authentication can be executed based on a result of the determination by the wearing determination unit; An information processing device comprising:

[0103] (Appendix 2) The power consumption in the second operation mode is greater than the power consumption in the first operation mode. 2. An information processing device according to claim 1.

[0104] (Appendix 3) the operation mode control unit switches the operation mode of the wearable device from the first operation mode to the second operation mode when the wearing determination unit determines that the user is wearing the wearable device. 3. The information processing device according to claim 1 or 2.

[0105] (Appendix 4) the plurality of operational modes further includes a third operational mode in which the user can utilize the wearable device; The power consumption in the third operation mode is greater than the power consumption in the second operation mode. 4. The information processing device according to claim 1 .

[0106] (Appendix 5) the operation mode control unit switches the operation mode of the wearable device from the second operation mode to the third operation mode when it is determined by the biometric authentication that the user is permitted to use the wearable device. 5. The information processing device according to claim 4.

[0107] (Appendix 6) The matching of the acoustic characteristics for the biometric authentication is performed by the wearable device. 6. An information processing device according to any one of claims 1 to 5.

[0108] (Appendix 7) The matching of the acoustic characteristics for the biometric authentication is performed by a device capable of communicating with the wearable device. 6. An information processing device according to any one of claims 1 to 5.

[0109] (Appendix 8) The biometric authentication includes a first biometric authentication and a second biometric authentication having higher accuracy than the first biometric authentication, the second operation mode is capable of executing a process for the first biometric authentication; the plurality of operation modes further includes a fourth operation mode in which a process for the second biometric authentication can be executed; 4. The information processing device according to claim 1 .

[0110] (Appendix 9) The power consumption in the fourth operation mode is greater than the power consumption in the second operation mode. 9. The information processing device according to claim 8.

[0111] (Appendix 10) the operation mode control unit switches the operation mode of the wearable device from the second operation mode to the fourth operation mode when it is difficult to determine whether or not the user is able to use the wearable device through the first biometric authentication. 10. The information processing device according to claim 8 or 9.

[0112] (Appendix 11) The matching of the acoustic characteristic for the first biometric authentication is performed by the wearable device; The matching of the acoustic characteristics for the second biometric authentication is performed by a device capable of communicating with the wearable device. 11. The information processing device according to any one of claims 8 to 10.

[0113] (Appendix 12) the plurality of operational modes further includes a third operational mode in which the user can utilize a function of the wearable device; The power consumption in the third operation mode is greater than the power consumption in the fourth operation mode. 12. The information processing device according to claim 8.

[0114] (Appendix 13) the operation mode control unit switches the operation mode of the wearable device from the second operation mode or the fourth operation mode to the third operation mode when it is determined that the user is allowed to use the wearable device by the first biometric authentication or the second biometric authentication. 13. The information processing device according to claim 12.

[0115] (Appendix 14) The biometric authentication is performed by matching acoustic characteristics of the ear canal. 14. An information processing device according to any one of claims 1 to 13.

[0116] (Appendix 15) The wearable device is an acoustic device that is worn on the user's ear. 15. An information processing device according to any one of claims 1 to 14.

[0117] (Appendix 16) The wearable device is a wireless device that operates by receiving power from a battery built into the wearable device. 16. An information processing device according to any one of claims 1 to 15.

[0118] (Appendix 17) A wearable device that emits sound waves to a part of a user's head for biometric authentication based on acoustic characteristics, a wearing determination unit that determines whether the wearable device is worn by the user; an operation mode control unit that switches the operation mode of the wearable device between a plurality of operation modes including a first operation mode in which the process for the biometric authentication cannot be executed and a second operation mode in which the process for the biometric authentication can be executed based on a result of the determination by the wearing determination unit; A wearable device comprising:

[0119] (Appendix 18) determining whether or not a user is wearing a wearable device that emits sound waves to a part of the user's head for performing biometric authentication based on acoustic characteristics; switching an operation mode of the wearable device between a plurality of operation modes including a first operation mode in which the process for the biometric authentication cannot be executed and a second operation mode in which the process for the biometric authentication can be executed based on a result of the determination; An information processing method comprising:

[0120] (Appendix 19) On the computer, determining whether or not a user is wearing a wearable device that emits sound waves to a part of the user's head for performing biometric authentication based on acoustic characteristics; switching an operation mode of the wearable device between a plurality of operation modes including a first operation mode in which the process for the biometric authentication cannot be executed and a second operation mode in which the process for the biometric authentication can be executed based on a result of the determination; A storage medium on which a program for executing the above is stored. [Explanation of symbols]

[0121] 1 Information and communication devices 2. Earphones 3 Users 20 Earphone control device 26 Speaker 27 Microphone 29 Infrared Sensor 30 Information processing device 101, 201 CPU 102, 202 RAM 103, 203 ROM 104 HDD 105, 207 Communication I / F 106 Input Devices 107 Output Device 111 Feature Matching Unit 112, 214 Storage section 113 Second feature matching unit 204 Flash Memory 205 Speaker I / F 206 Microphone I / F 208 Battery 210 Infrared sensor I / F 211, 311 Wearing Judgment Section 212 Ear Acoustic Authentication Department 213, 312 Operation mode control unit 215 Feature Acquisition Unit 216 First feature matching unit

Claims

1. a speaker that emits sound waves to a part of a user's head for performing biometric authentication based on acoustic characteristics; a microphone for receiving echoes of the sound waves echoed by a part of the user's head; a wearing determination unit that determines whether the user is wearing the information processing device; an operation mode control unit that switches the operation mode of the information processing device between a plurality of operation modes including a first operation mode in which the process for the biometric authentication cannot be executed and a second operation mode in which the process for the biometric authentication can be executed based on a result of the determination by the wearing determination unit; Equipped with The power consumption in the second operation mode is greater than the power consumption in the first operation mode. Information processing device.

2. the operation mode control unit switches the operation mode of the information processing device from the first operation mode to the second operation mode when the wearing determination unit determines that the user is wearing the information processing device. The information processing device according to claim 1 .

3. A matching unit that performs matching of the acoustic characteristics for the biometric authentication is further provided. The information processing device according to claim 1 .

4. The matching of the acoustic characteristics for the biometric authentication is performed by a device capable of communicating with the information processing device. The information processing device according to claim 1 .

5. The biometric authentication is performed by matching acoustic characteristics of the ear canal. The information processing device according to claim 1 .

6. the information processing device is a wireless device that operates by receiving power from a battery built into the information processing device; The information processing device according to claim 1 .

Citation Information

Patent Citations

  • Individual authentication device and method, and signal transmitter

    JP2004065363A

  • Personal authentication device and personal authentication method by biological information

    JP2017038766A

  • Personal authentication device, personal authentication method, and personal authentication program

    WO2017069118A1

  • Personal authentication system, personal authentication device, personal authentication method, and recording medium

    WO2018034178A1