Earphone device and method for customizing the operation of the earphone device

Abstract

It helps to efficiently and arbitrarily customize the gestures used by the user to instruct operations. [Solution] The earphone device comprises an earphone device that is worn on at least one of the user's left or right ears, the earphone device having a processor, a communication unit that communicates with an operating terminal, and a sensor that detects gestures made by the user when wearing the earphone device for operating the operating terminal, the processor instructs the operating terminal to display the content of the gesture based on the sensor's detection value when the user makes an arbitrary gesture, and in response to the user's instruction operation to the operating terminal for gesture identification information based on the content of the gesture, the processor associates the sensor's detection value based on the gesture with the gesture identification information sent from the operating terminal via the communication unit and registers it in the storage unit.

Description

Technical Field

[0001] The present disclosure relates to an earphone device and a method for customizing operations for the earphone device.

Background Art

[0002] It is known that, for earphones worn on each of both ears of a user, operations can be performed on the earphones through user gestures. For example, Patent Document 1 discloses a wearable display device that incorporates a sensor for acquiring information about the posture of the wearing user and receives operation input via user gestures. This wearable display device accepts operation input via a gesture when the amount of change in posture exceeds a predetermined threshold. Further, an earphone is disclosed as an example of the wearable display device.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] The present disclosure has been devised in view of the above-mentioned conventional situation, and an object thereof is to provide an earphone device and a method for customizing operations for the earphone device that support efficient and arbitrary customization of user gestures for instructing operations.

Means for Solving the Problem

[0005] This disclosure provides an earphone device that is worn on at least one of the left or right ears of a user, wherein the earphone device comprises a processor, a communication unit that communicates with an operating terminal, and a sensor that detects gestures made by the user when wearing the earphone device for operating the operating terminal, wherein when the user makes an arbitrary gesture, the processor instructs the operating terminal to display the content of the gesture based on the value detected by the sensor, and in response to the user's instruction operation to the operating terminal for gesture identification information based on the content of the gesture, the processor associates the value detected by the sensor based on the gesture with the gesture identification information sent from the operating terminal via the communication unit and registers it in a storage unit.

[0006] Furthermore, this disclosure provides an earphone device that is worn on at least one of the user's left or right ears, the earphone device comprising: a processor; a storage unit that stores a gesture list for managing a plurality of gesture candidates; and a sensor that detects gestures made by the user when wearing the earphone device for operating an operating terminal, the processor adding the specific gesture to a habit list that manages a plurality of habitual gestures that the user makes on a daily basis, based on the detection of a specific gesture made by the user more than a certain number of times, and registering gestures that do not match the plurality of specific gestures registered in the habit list among the plurality of gesture candidates included in the gesture list to a recommended gesture list.

[0007] Furthermore, this disclosure provides an earphone device that is performed by an earphone device worn on at least one of the user's left or right ears, wherein the earphone device has at least a communication unit that communicates with an operating terminal and a sensor that detects gestures made by the user when wearing the earphone device for operating the operating terminal, and when the user makes an arbitrary gesture, the device instructs the operating terminal to display the content of the gesture based on the value detected by the sensor, and in response to the user's instruction operation of gesture identification information to the operating terminal based on the content of the gesture, the device associates the value detected by the sensor based on the gesture with the gesture identification information sent from the operating terminal via the communication unit and registers it in the storage unit.

[0008] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0009] According to this disclosure, it is possible to efficiently and arbitrarily customize the gestures of the user who instructs the operation. [Brief explanation of the drawing]

[0010] [Figure 1] Block diagram showing an example of the internal configuration of the earphone according to this embodiment. [Figure 2] Block diagram showing an example of the internal configuration of a smartphone according to this embodiment. [Figure 3] Sequence for registering user gestures using earphones according to this embodiment [Figure 4] Sequence for registering user gestures using earphones according to this embodiment, and sequence for assigning user-registered gestures to earphone operations. [Figure 5]An example of the display on the operating terminal when assigning user-registered gestures to earphone operations using earphones according to this embodiment. [Figure 6] A sequence in which a user tests the reproduction of a gesture registered by the user using the earphones according to this embodiment. [Figure 7] Flowchart for registering habitual gestures that a user performs daily using earphones according to this embodiment into a habit list. [Figure 8] Flowchart for creating a recommended gesture list using earphones according to this embodiment [Modes for carrying out the invention]

[0011] When a user wears earphones, they may listen to music transmitted from their smartphone via wireless communication between the earphones and their smartphone. In such cases, earphone operations corresponding to specific gestures (for example, shaking the head from side to side) are pre-registered in the earphones. When the user performs a specific gesture, the gyro sensor in the earphones detects it, and the earphones can perform that operation. However, there was a problem in that some user gestures could result in unintended earphone operations. Therefore, there is a need to be able to arbitrarily customize the gestures that users register for earphone operation.

[0012] The purpose of this disclosure is to provide an earphone device and a method for customizing the operation of an earphone device, which assist in efficiently and arbitrarily customizing the gestures of a user that instructs the operation.

[0013] The following description will detail embodiments specifically disclosing the earphone device and the method for customizing the operation of the earphone device according to this disclosure, with appropriate reference to the drawings. However, unnecessarily detailed explanations may be omitted. For example, detailed explanations of already well-known matters and redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter of the claims.

[0014] The earphone device according to the following embodiment provides wireless communication (e.g., Bluetooth®) compliant with, for example, True Wireless Stereo (TWS) between a pair of earphone devices worn on each of the user's left and right ears and a wireless terminal carried by the user (e.g., a smartphone). The wireless communication method is not limited to Bluetooth®; for example, a wireless Local Area Network (LAN) such as Wi-Fi® may also be used. Furthermore, the earphone devices may be worn on either the left or right ear.

[0015] In this embodiment, the earphone device is described as TWS (True Wireless Stereo), but it may also be a left and right integrated type or a single-ear wireless earphone. Wired earphones may also be used, in which case the earphone may communicate with a wireless terminal (smartphone) via a wire. Furthermore, in this embodiment, the earphone is described as an in-ear type that is worn (inserted) into the ear, but the type of earphone is not limited to over-ear, behind-the-ear, or bone conduction types.

[0016] (Embodiment 1) Hereinafter, the system configuration of the TWS system 100 according to the present disclosure will be described with reference to FIG. 1. FIG. 1 is a diagram showing an example of an internal configuration of an earphone according to the present embodiment. The earphone mentioned herein includes a left earphone 10L worn inside the external auditory canal of a user's left ear and a right earphone 10R worn inside the external auditory canal of a user's right ear. The TWS system 100 is configured to include at least a pair of earphone devices, namely the left earphone 10L and the right earphone 10R, and a smartphone 50.

[0017] The left earphone 10L is an in-ear acoustic device that is used by being worn on a user's left ear, and is held in a state of being worn (inserted) into the left ear when in use. Similarly, the right earphone 10R is an in-ear acoustic device that is used by being worn on a user's right ear, and is held in a state of being worn (inserted) into the right ear when in use.

[0018] Since the internal configurations of the left earphone 10L and the right earphone 10R are the same, the internal configuration will be described below by taking the left earphone 10L as an example, and the description of the right earphone 10R can be made by replacing "L" at the end of the reference numeral of the left earphone 10L with "R". Further, in FIG. 1, reference numerals that have a correspondence relationship with the internal configuration corresponding to the internal configuration of the left earphone 10L are assigned to the right earphone 10R. Specifically, "L" is added to the end of the reference numeral for components related to the internal configuration of the left earphone 10L, and "R" is added for components related to the internal configuration of the right earphone 10R.

[0019] The left earphone 10L has a housing where an earpiece (not shown) is detachably provided on one end side where the speaker SPKL is arranged. The left earphone 10L includes an earphone control unit 11L, a gyro sensor 12L, a Random Access Memory (RAM) 15L, a Read Only Memory (ROM) 16L, an audio input / output control unit 17L, a wireless communication unit 19L, a power monitoring unit 20L having a battery BTL, and a microphone MCL.

[0020] The right earphone 10R has a housing on one end where the speaker SPKR is located, and an earpiece (not shown) is detachably provided therein. The right earphone 10R includes an earphone control unit 11R, a gyro sensor 12R, RAM 15R, ROM 16R, an audio input / output control unit 17R, a wireless communication unit 19R, a power monitoring unit 20R with a battery BTR, and a microphone MCR.

[0021] The earphone control unit 11L is configured using a processor such as a Central Processing Unit (CPU), Micro Processing Unit (MPU), or Digital Signal Processor (DSP). The earphone control unit 11L functions as a controller that oversees the overall operation of the left earphone 10L, performing control processing to coordinate the operation of each part of the left earphone 10L, data input / output processing between each part of the left earphone 10L, data calculation processing, and data storage processing. The earphone control unit 11L operates according to the programs and data stored in the ROM 16L, and during operation, it uses the RAM 15L to temporarily store data or information generated or acquired by the earphone control unit 11L in the RAM 15L or send it to the wireless communication unit 19L.

[0022] The gyro sensor 12L is a sensor device that detects the rotational angular velocity of the user's head in each of the three orthogonal axes when the left earphone 10L is worn on the user's left ear. Hereafter, each of the rotational angular velocities in the three orthogonal axes will be simply referred to as "rotational angular velocity". Although the gyro sensor 12L is exemplified here as detecting each of the rotational angular velocities in the three orthogonal axes, it may also detect the rotational angular velocity of multiple axes other than the three orthogonal axes. The gyro sensor 12L detects the rotational angular velocity at predetermined intervals and outputs this detected value to the earphone control unit 11L. In other words, the earphone control unit 11L can periodically acquire the output value of the gyro sensor 12L (i.e., the detected value of rotational angular velocity). The left earphone 10L can detect the user's gestures based on the detected value of the gyro sensor. Note that the gyro sensors (12L, 12R) mounted on the left earphone 10L and the right earphone 10R may be mounted on only one of the earphones. In other words, the left earphone 10L may be equipped with a gyro sensor 12L, but the right earphone 10R may not be equipped with a gyro sensor 12R, or the right earphone 10R may be equipped with a gyro sensor 12R, but the left earphone 10L may not be equipped with a gyro sensor 12L. Furthermore, instead of a gyro sensor, an accelerometer that detects acceleration by measuring changes in the velocity of an object, thereby detecting the tilt and vibration of the object, may be placed in each of the left earphone 10L and the right earphone 10R. The gyro sensors (12L, 12R) and the accelerometer are examples of inertial measurement units (IMUs).

[0023] RAM15L is a work memory used when executing each process (operation) of the earphone control unit 11L, and temporarily stores data or information generated or acquired by the earphone control unit 11L.

[0024] ROM 16L stores the program that defines each process (operation) of the earphone control unit 11L, and the data used when that program is executed. ROMs 16L and 16R may be, for example, rewritable flash memory.

[0025] The audio input / output control unit 17L is connected to the microphone MCL and speaker SPKL respectively to enable input and output of data signals (e.g., audio signals), and is configured using a communication circuit capable of performing various processes (operations) related to the input and output of audio signals. The audio input / output control unit 17L converts the digital audio signal sent from the earphone control unit 11L into an analog format and outputs it from the speaker SPKL. The audio input / output control unit 17L also receives an analog sound signal (including an audio signal; the same applies hereinafter) picked up by the microphone MCL, converts it into a digital sound signal, and sends it to the earphone control unit 11L.

[0026] Microphone MCLs include speech microphones, feedforward (FF) microphones, and feedback (FB) microphones.

[0027] The speech microphone consists of a microphone device capable of picking up sound (i.e., detecting an audio signal) generated based on the user's speech. The speech microphone picks up sound generated based on the user's speech, converts it into an electrical signal, and sends it to the audio input / output control unit 17L. The sound uttered by the user is picked up by the speech microphone and converted into an electrical signal, and the presence or absence of user speech can be detected by the speech microphone based on the magnitude of this electrical signal.

[0028] The FF microphone consists of a microphone device capable of picking up (detecting) ambient sounds from the left earphone 10L as sound signals. The FF microphone converts the ambient sounds from the outside into electrical signals (sound signals) and sends them to the audio input / output control unit 17L.

[0029] The FB microphone is positioned as close as possible to the external auditory canal of the user's left ear and consists of a microphone device capable of picking up (detecting) sound near the external auditory canal. The FB microphone converts some of the sound that has bent around the external auditory canal (for example, sound output from speaker SPKL) into an electrical signal (sound signal) and sends it to the audio input / output control unit 17L.

[0030] The speaker SPKL acoustically outputs the sound signal sent from the audio input / output control unit 17L based on instructions from the earphone control unit 11L. The speaker SPKL may also acoustically output the sound signal based on notifications from, for example, the left earphone 10L and the right earphone 10R. This allows the user to easily understand what operation is being performed.

[0031] The wireless communication unit 19L has an antenna ATL and is configured using a communication circuit that performs wireless communication with the smartphone 50. The wireless communication unit 19L receives data signals transmitted from the smartphone 50 or the right earphone 10R via the antenna ATL, and transmits data signals sent from the earphone control unit 11L to the smartphone 50 or the right earphone 10R via the antenna ATL.

[0032] The smartphone 50 transmits and receives data signals to and from the left earphone 10L and the right earphone 10R, respectively, according to a predetermined wireless communication method (e.g., Bluetooth®). The smartphone 50 is carried (possessed) by the user who is wearing the left earphone 10L and the right earphone 10R. The smartphone 50 is an example of a wireless terminal that performs wireless communication such as Bluetooth® with the left earphone 10L and the right earphone 10R, respectively, and may be a mobile phone or any wireless terminal that does not have telephone functionality. The smartphone 50 and the left earphone 10L and the right earphone 10R may also be connected by a wire and communicate via wired connection.

[0033] Next, the internal configuration of the smartphone 50 that communicates wirelessly with the left earphone 10L and the right earphone 10R will be explained with reference to Figure 2. Figure 2 is a block diagram showing an example of the internal configuration of a smartphone according to an embodiment.

[0034] The smartphone 50 includes a control unit 51, a display / operation unit 52, a ROM 53, a RAM 54, an audio input / output control unit 55, a microphone MC1, a speaker SPK1, a short-range wireless control unit 56, an earphone communication I / F unit 57, an audio bus 59, and a battery BT2.

[0035] The control unit 51 is configured using, for example, a CPU, DSP, or FPGA (Field Programmable Gate Array). The control unit 51 functionally includes a smartphone OS (Operating System) processing unit 51A and a smartphone application processing unit 51B, and performs various processes and controls through the cooperation of the smartphone OS processing unit 51A and the smartphone application processing unit 51B, respectively, with the ROM 53 and RAM 54, respectively.

[0036] The smartphone OS processing unit 51A performs basic operations and controls related to various processes performed by the smartphone 50.

[0037] The smartphone application processing unit 51B performs processing of various applications, including applications suitable for the smartphone 50 that can be run between the left earphone 10L and the right earphone 10R, respectively.

[0038] The display / operation unit 52 is configured using a touch panel that accepts user input and displays data generated by the control unit 51, and forms a so-called user interface. The display / operation unit 52 may, for example, display a message based on a notification from the left earphone 10L and the right earphone 10R, under the control of the control unit 51.

[0039] ROM 53 stores a program that defines each process (operation) of the control unit 51, as well as data to be used when that program is executed.

[0040] RAM 54 is a work memory used when executing each process (operation) of the control unit 51, and temporarily stores data generated or acquired by the control unit 51.

[0041] The audio input / output control unit 55 is connected to the microphone MC1 and speaker SPK1 respectively to enable input and output of data signals (e.g., audio signals), and is configured using a communication circuit capable of performing various processes (operations) related to the input and output of audio signals. The audio input / output control unit 55 converts the digital audio signal sent from the control unit 51 via the audio bus 59 into an analog format and outputs it from the speaker SPK1. The audio input / output control unit 55 also receives an analog audio signal (including an audio signal) picked up by the microphone MC1, converts it into a digital audio signal, and sends it to the control unit 51.

[0042] Microphone MC1 consists of a microphone device capable of capturing sound (i.e., detecting an audio signal) generated based on the user's speech. Microphone MC1 converts the captured sound into an electrical signal and sends it to the audio input / output control unit 55.

[0043] Speaker SPK1 acoustically outputs sound signals sent from the audio input / output control unit 55 via the control unit 51 and the audio bus 59, based on instructions from the control unit 51. Speaker SPK1 acoustically outputs, for example, the music signal being played by the smartphone application processing unit 51B when it is not being output from the left earphone 10L and the right earphone 10R, respectively.

[0044] The short-range wireless control unit 56 is configured using a circuit capable of performing control for wirelessly transmitting audio signals such as voice signals that are input, and is connected to the earphone communication I / F unit 57 so that data signals can be input and output. The short-range wireless control unit 56 receives the audio signal sent from the earphone communication I / F unit 57 and sends it to the control unit 51 via the audio bus 59. The short-range wireless control unit 56 sends the audio signal such as voice signals that are input via the audio bus 59 to the earphone communication I / F unit 57.

[0045] The earphone communication interface unit 57 has an antenna AT1 and is configured using a communication circuit that performs wireless communication (e.g., Bluetooth®) with the left earphone 10L and the right earphone 10R, respectively. The earphone communication interface unit 57 receives data signals transmitted from the left earphone 10L and the right earphone 10R via the antenna AT1, and transmits data signals sent from the short-range wireless control unit 56 to the left earphone 10L and the right earphone 10R, respectively, via the antenna AT1.

[0046] The audio bus 59 is a data transmission path for sound signals such as voice signals, and transmits sound signals between the control unit 51 and the audio input / output control unit 55, between the control unit 51 and the short-range wireless control unit 56, and between the control unit 51, the audio input / output control unit 55, or the short-range wireless control unit 56.

[0047] Battery BT2 is constructed using a secondary battery capable of storing power supplied from an external commercial power source (not shown), and supplies the necessary power to each component of the smartphone 50.

[0048] A dedicated application (smartphone APP) is installed on the smartphone 50. The user can operate and configure the device via the touch panel used in the display / operation unit 52 using the smartphone APP. The system can then transmit signals corresponding to these operations and settings to the earphone device for further control. The smartphone 50 can also receive signals from the earphone device and display them on the display / operation unit 52 of the smartphone 50, allowing the user to be aware of the operation status.

[0049] Next, with reference to Figures 3 and 4, the process for registering user gestures using the earphones according to this embodiment will be explained. Figures 3 and 4 show the sequence for registering user gestures using the earphones according to this embodiment. As described below, user gesture registration is performed through the cooperation of the earphones worn by the user (specifically, the left earphone 10L and the right earphone 10R) and the smartphone 50 that the user possesses.

[0050] The series of processes shown in Figures 3 and 4 are mainly performed by the earphone control unit 11L if the left earphone 10L is used, and by the earphone control unit 11R if the right earphone 10R is used. For the sake of simplicity, the explanation in Figures 3 and 4 will focus on the earphone control unit 11L of the left earphone 10L, but the following explanation can be similarly applied even if the earphone control unit 11L of the left earphone 10L is replaced with the earphone control unit 11R of the right earphone 10R.

[0051] When the user performs an operation to start gesture memory on the smartphone app (i.e., a dedicated application for the smartphone 50) (St1), the smartphone 50 reflects the result of the operation, that is, displays the result on the display / operation unit 52 (St2). The earphone control unit 11L receives an instruction signal via the wireless communication unit 19L based on the user's operation to start gesture memory on the smartphone 50 performed in step St1. Next, the earphone control unit 11L makes the user auditorily aware that memory has started by outputting sound from the speaker SPKL (St3). Alternatively, instead of sound, the user may be made to physically recognize the start of memory by vibrating a vibration motor (not shown) built into the left earphone 10L. Simultaneously with step St3, the earphone control unit 11L sends a notification signal to the smartphone 50 via the wireless communication unit 19L that memory has started. The smartphone 50 displays a screen on the display / operation unit 52 that indicates that gesture memory is being stored, and at the same time, a gesture memory end button is also displayed on this screen (St4). The gesture memory end button is an icon that the user presses when they want to end the process of saving (registering) gestures.

[0052] Next, with the left earphone 10L in their left ear, the user performs a gesture of their choice that they want to save for earphone operation (St5). The gyro sensor 12L built into the left earphone 10L detects this gesture and temporarily stores the detected value in the RAM 15L (St6). Next, the earphone control unit 11L transmits the gesture information based on the detected value of the gyro sensor 12L to the smartphone 50 via the wireless communication unit 19L. The smartphone 50 displays the content of the gesture on the display / operation unit 52 (St7). The series of processes from step St5 to step St7 may be repeated.

[0053] When the user performs a gesture memory termination operation on the smartphone app (St8), the smartphone 50 reflects the result of that operation, that is, displays the result on the display / operation unit 52 (St9). The gesture memory termination operation is, for example, pressing the gesture memory termination button displayed in step St4, but is not limited to this. The earphone control unit 11L receives an instruction signal based on the gesture memory termination operation performed by the user on the smartphone 50 in step St8 via the wireless communication unit 19L. Next, the earphone control unit 11L makes the user auditorily aware that the memory has ended by outputting sound from the speaker SPKL (St10). Alternatively, instead of sound, the user may be made to physically recognize the termination by vibrating a vibration motor (not shown) built into the left earphone 10L. Simultaneously with step St10, the earphone control unit 11L sends a memory termination notification signal to the smartphone 50 via the wireless communication unit 19L. The smartphone 50 displays the gesture identification information assignment screen on the display / operation unit 52 (St11). The gesture identification information assignment screen is a screen that prompts the user to enter gesture identification information in order to identify the gestures of the user to be remembered (registered).

[0054] When the user performs an operation to assign gesture identification information to the smartphone app (St12), the smartphone 50 reflects the content of that gesture identification information on the display / operation unit 52 of the smartphone 50, that is, displays the result of the operation on the display / operation unit 52 (St13). Here, the gesture identification information may be any string assigned by the user. Alternatively, the gesture identification information may be an icon automatically generated by AI (Artificial Intelligence) that is executable by either the earphone control unit 11L mounted on the left earphone 10L or the control unit 51 mounted on the smartphone 50, based on the detected value of the gyro sensor. The earphone control unit 11L or the control unit 51 may present the automatically generated icon to the user from the smartphone 50.

[0055] When the earphone control unit 11L receives from the smartphone 50 via the wireless communication unit 19L that gesture identification information has been added, it links the detected value of the gyro sensor based on the gesture with the gesture identification information and stores it in the memory unit (St14). Examples of the memory unit include, but are not limited to, the memory inside the earphone (e.g., ROM 16L, 16R), the smartphone 50, and a cloud (cloud computer) wirelessly connected to the smartphone 50. Furthermore, the gesture identification information can be changed at any time in response to changes made by the user on the smartphone 50.

[0056] Next, referring to Figures 4 and 5, the process of assigning the detected values ​​of the gyro sensor based on the gestures stored by the earphones according to this embodiment to earphone operations will be explained. Steps St15 onwards in Figure 4 are the sequence for assigning the user-registered gestures to earphone operations. Figure 5 is an example of the display on the operation terminal when assigning the user-registered gestures to earphone operations using the earphones according to this embodiment.

[0057] When the user selects one gesture identification information and information on the control content of an arbitrary earphone device (left earphone 10L) from the memory unit on the smartphone 50 (St15), the smartphone 50 transmits these two pieces of information to the earphone control unit 11L (St16). Next, the earphone control unit 11L receives these two pieces of information via the wireless communication unit 19L and assigns the control content of the selected earphone device operation to the selected gesture identification information (St17). As a result, when the earphone device (e.g., left earphone 10L, right earphone 10R) performs the gesture corresponding to the gesture identification information selected by the user in step St15, it can obtain a sensor detection value equivalent to the sensor detection value associated with that gesture identification information (see step St14). Consequently, the earphone device (e.g., left earphone 10L, right earphone 10R) can perform the earphone operation (e.g., pausing music wirelessly transmitted from the smartphone 50) which is the control content assigned in step St17.

[0058] In the example shown in Figure 5, a screen for assigning specific earphone operations to gesture identification information of a series of gestures in which the user rotates their head 90 degrees to the left from the front and then returns their head to the front is displayed on the display / operation unit 52 of the smartphone 50. The specific earphone operations referred to here are, for example, playing or pausing music wirelessly transmitted from the smartphone 50, but are not limited to these.

[0059] Next, referring to Figure 6, the process by which a user reproduces a registered gesture using the earphones according to this embodiment will be explained. Figure 6 shows the sequence in which a user reproduces a registered gesture using the earphones according to this embodiment.

[0060] The series of processes shown in Figure 6 are mainly performed by the earphone control unit 11L if the left earphone 10L is used, and by the earphone control unit 11R if the right earphone 10R is used. For the sake of simplicity, the explanation in Figure 6 will focus on the earphone control unit 11L of the left earphone 10L, but the following explanation can be similarly applied even if the earphone control unit 11L of the left earphone 10L is replaced with the earphone control unit 11R of the right earphone 10R.

[0061] When the user initiates a gesture test using the smartphone app (i.e., a dedicated application for the smartphone 50) (St20), the smartphone 50 generates a gesture information request based on that operation and transmits it to the earphone device. The earphone control unit 11 receives the gesture information request signal from the smartphone 50 via the wireless communication unit 19L. The earphone control unit 11L generates a gesture information response signal indicating that it has received the gesture information request signal and transmits it to the smartphone 50 via the wireless communication unit 19L. When the smartphone 50 receives the gesture information response signal from the earphone control unit 11L, it displays a list of gestures currently registered in the RAM 54, etc., on the display / operation unit 52 (St21). This gesture list is a list of gesture identification information that identifies the user's gestures as described with reference to Figures 3 and 4, and may include the name of the gesture corresponding to each gesture identification information or an image showing the gesture in addition to the gesture identification information.

[0062] In step St21, the user selects a specific gesture from the list of gestures displayed on the display / operation unit 52 (St22). The smartphone 50 then determines information identifying the selected gesture (e.g., gesture identification information), generates a start command signal indicating that the user is about to perform this gesture, and transmits it to the earphone device. The earphone control unit 11L receives the start command signal via the wireless communication unit 19L. Next, the earphone control unit 11L outputs sound from the speaker SPKL to audibly notify the user that the gesture test has started (St23). Alternatively, instead of sound, the vibration motor (not shown) built into the left earphone 10L may be vibrated to physically notify the user. Simultaneously with step St23, the earphone control unit 11L generates a start notification signal via the wireless communication unit 19L and transmits it to the smartphone 50. When the smartphone 50 receives the start notification signal from the earphone control unit 11L, it generates a screen indicating that a gesture test is in progress and displays it on the display / operation unit 52 (St24).

[0063] When the user performs a specific gesture corresponding to the gesture identification information selected in step St22 (St25), the smartphone 50 transfers the information of the specific gesture (e.g., gesture identification information) displayed on the smartphone's display / operation unit 52 in step 24 to the earphone device. The earphone control unit 11L compares the detected value of the gyro sensor based on the specific gesture performed by the user with the detected value of the gyro sensor registered in the memory unit corresponding to the gesture identification information of the specific gesture sent from the smartphone 50 (St26).

[0064] The earphone control unit 11L transmits the comparison results to the smartphone 50 via the wireless communication unit 19L, and the smartphone 50 displays the comparison results on the display / operation unit 52 (St27). The comparison results may also include an index indicating the reproducibility of a specific gesture.

[0065] Next, I will explain how to delete a stored gesture.

[0066] When a user selects one of the gesture identification pieces registered in the memory unit via the smartphone app on the smartphone 50 and performs a deletion operation, the smartphone 50 generates a signal based on the selected gesture identification piece and transmits it to the earphone device. Upon receiving this signal, the earphone control unit 11L deletes the gesture identification piece and the gyro sensor detection value associated with that gesture identification piece from the memory unit.

[0067] Next, referring to Figures 7 and 8, we will explain the process of suggesting gestures that are less likely to be falsely detected in the user's everyday use. Figure 7 is a flowchart for registering the user's habitual gestures, which are performed daily, in a habit list using the earphones according to this embodiment. Figure 8 is a flowchart for creating a recommended gesture list using the earphones according to this embodiment.

[0068] The series of processes shown in Figures 7 and 8 are mainly performed by the earphone control unit 11L if it is the left earphone 10L, and by the earphone control unit 11R if it is the right earphone 10R. For the sake of simplicity, the explanation in Figures 7 and 8 will focus on the earphone control unit 11L of the left earphone 10L, but the following explanation can be similarly applied even if the earphone control unit 11L of the left earphone 10L is replaced with the earphone control unit 11R of the right earphone 10R. Furthermore, the following explanation exemplifies a case where the entity performing the processes listed in Figures 7 and 8 is an earphone device (e.g., the left earphone 10L) that communicates wirelessly with a smartphone 50, but it may also be a smartphone 50 that communicates wirelessly with the earphone device (e.g., the left earphone 10L).

[0069] First, referring to Figure 7, the process of creating a habit list will be explained. Here, a habit list is a list that manages information about gestures (habitual gestures) that a user frequently performs on a daily basis while wearing an earphone device (e.g., left earphone 10L). This habit list can be stored in, but is not limited to, the memory inside the earphone (e.g., ROM 16L, 16R), a smartphone 50, or a cloud (cloud computer) that can communicate wirelessly with the smartphone 50. The earphone control unit 11L determines whether the habit list creation mode is enabled or disabled via the smartphone 50 (St31). Note that whether the habit list creation mode is enabled or disabled is switched as needed in response to the user's operation on the smartphone 50, and this switched information is transmitted from the smartphone 50 to the earphone device (e.g., left earphone 10L) and shared each time.

[0070] If the earphone control unit 11 determines that the habit list creation mode is enabled (St31, Yes), it starts acquiring user gestures (St32). Acquiring gestures here means that, for example, when the user makes some kind of gesture (action), the gyro sensor 12L of the earphone device worn by the user (e.g., the left earphone 10L) acquires the sensor detection value. If the habit list creation mode is disabled (St31, No), the process shown in Figure 7 proceeds to termination.

[0071] Next, when the gyro sensor 12L detects the user's action A (for example, shaking the head from side to side) (St33), the gyro sensor 12L acquires a sensor detection value corresponding to action A and notifies the earphone control unit 11L that it has detected the action. The earphone control unit 11L determines that the user has performed action A based on a comparison between the sensor detection value corresponding to action A, which is stored in advance, for example, in the earphone's internal memory (for example, ROM 16L, 16R), the smartphone 50, or a cloud (cloud computer) that can communicate wirelessly with the smartphone 50, and the sensor detection value detected in step St33. After this determination, the earphone control unit 11L increments the count of action A by +1 based on the notification from the gyro sensor 12L (St34).

[0072] Next, the earphone control unit 11L determines whether the count of operation A has exceeded a certain number of times X (St35). If the count of operation A has exceeded a certain number of times X (St35, Yes), the earphone control unit 11L registers information about operation A (for example, gesture identification information for operation A) in the memory inside the earphone (for example, ROM 16L, 16R), the smartphone 50, or a cloud (cloud computer) that can communicate wirelessly with the smartphone 50, in a habit list (St36). After that, gesture acquisition is terminated (St37). If the count of operation A has not exceeded a certain number of times X (St35, No), the process proceeds to the termination of gesture acquisition (St37).

[0073] Next, referring to Figure 8, we will explain the process of creating a recommended gesture list. Here, the recommended gesture list is a list of gesture candidates included in the gesture list, which is stored in advance in the earphone's internal memory (e.g., ROM16L, 16L), the smartphone 50, or a cloud (cloud computer) that can wirelessly communicate with the smartphone 50, but does not match any of the gestures registered in the habit list. In other words, the recommended gesture list is a list that manages a list of gestures that are recommended as suitable for assignment because they do not match any of the habit actions that the user performs on a daily basis, when there are some gestures that the user has not yet decided on to assign to control the earphone device.

[0074] The earphone control unit 11L initiates the creation of a recommended gesture list via the smartphone 50 in response to user input (St41). The earphone control unit 11L obtains the first habit gesture (n=1) from the N habit gestures registered in the habit list (St42) and initiates gesture comparison (St43).

[0075] The earphone control unit 11L obtains the first gesture (m=1) from the M gesture list pre-stored in the ROM 16L (St44). Here, each gesture registered in the gesture list is associated with a sensor detection value from the gyro sensor 12L.

[0076] The earphone control unit 11L compares the detected values ​​of the gyro sensor for the habitual gesture (n=1) and the gesture (m=1). If they do not match (St45, No), it adds the gesture (m=1) to the recommended gesture list (St46).

[0077] On the other hand, the earphone control unit 11L compares the detected values ​​of the gyro sensor for the habitual gesture (n=1) and the gesture (m=1). If they match (St45, Yes), it does not add them to the recommended gesture list. The earphone control unit 11L adds 1 to the value of m (St47). For example, if the value of m is currently set to 1, step St47 makes the value of m 2.

[0078] The earphone control unit 11L compares the magnitudes of the values ​​of m and M. If m is less than or equal to M (St48, No), it returns to step St45. On the other hand, if m is greater than M (St48, Yes), the gesture comparison ends (St49).

[0079] Next, the earphone control unit 11L adds 1 to the value of n (St50). For example, if the value of n is currently set to 1, step St50 makes the value of n 2.

[0080] The earphone control unit 11L compares the magnitudes of the values ​​of n and N. If n is less than or equal to N (St51, No), it returns to step St43. If m is greater than M (St51, Yes), the earphone control unit 11L finishes creating the recommended gesture list (St52).

[0081] The above flow allows for the creation of a recommended gesture list. This recommended gesture list is updated and saved in the earphone's internal memory (e.g., ROM 16L, 16R), the smartphone 50, or a cloud (cloud computer) that can wirelessly communicate with the smartphone 50. When a user wishes to view the recommended gesture list, it is displayed on the display / operation unit 52 in response to the user's operation on the smartphone 50, allowing the user to confirm it.

[0082] Next, we will explain the gesture ambiguity setting. This setting is primarily performed by the earphone control unit 11L for the left earphone 10L and by the earphone control unit 11R for the right earphone 10R. To simplify the explanation, the explanation in Figures 7 and 8 will focus on the earphone control unit 11L of the left earphone 10L, but the following explanation can be similarly applied even if the earphone control unit 11L of the left earphone 10L is replaced with the earphone control unit 11R of the right earphone 10R.

[0083] The earphone control unit 11L can adjust and set the detection sensitivity of the gyro sensor in response to the user's operation to adjust the detection sensitivity of the gyro sensor on the smartphone 50. The sensitivity setting may be adjusted, for example, by providing a sensitivity setting menu on the smartphone app displayed on the display / operation unit 52 of the smartphone 50, through user operation, with settings such as high, medium, and low. Alternatively, the sensitivity setting may be adjusted by the user operating the touch panel, with a detection sensitivity scale bar displayed on the smartphone app. Thus, the processing of sensitivity settings is not particularly limited, as long as the user can adjust it arbitrarily in response to operations on the smartphone 50.

[0084] (Summary of this disclosure) The above description of embodiments discloses the technical concepts corresponding to the following items.

[0085] (Item 1) The earphone device related to this disclosure is The system includes an earphone device (left earphone 10L, right earphone 10R) that is worn in at least one of the user's left or right ears, The aforementioned earphone device Processor (earphone control unit 11L, 11R) A communication unit (wireless communication unit 19L, 19R) that communicates with the operating terminal (smartphone 50), The device includes sensors (gyro sensors 12L, 12R) that detect gestures made by the user to operate the aforementioned operating terminal, The aforementioned processor, When the user makes any gesture, the operating terminal is instructed to display the content of the gesture based on the sensor's detected value. In response to the user's instruction operation to the operating terminal for gesture identification information based on the content of the gesture, the detected value of the sensor based on the gesture and the gesture identification information sent from the operating terminal via the communication unit are linked and registered in the storage unit. This allows the earphone device to register gestures that the user wants to remember in its memory unit, thereby helping to improve the user experience.

[0086] (Item 2) In the earphone device described in item 1, The aforementioned processor, In response to the user's selection operation on the operating terminal between the gesture identification information registered in the storage unit and the control content for any earphone device, the selected control content for the earphone device is assigned to the selected gesture identification information and registered in the storage unit. This allows the earphone device to assign control functions to any gesture selected by the user, thus enabling the user to customize the gestures used to operate the operating terminal (smartphone).

[0087] (Item 3) In the earphone device described in item 1, The aforementioned processor, When the user performs a specific gesture corresponding to one of the gesture identification pieces registered in the storage unit, the operation terminal is instructed to display the result of comparing the sensor's detection value based on the specific gesture performed by the user with the sensor's detection value registered in the storage unit corresponding to the gesture identification piece of the specific gesture. The comparison results include an indicator showing the user's ability to reproduce the specific gesture. This allows the earphone device to perform reproduction tests of gestures selected by the user and recognize indicators of their reproducibility, thereby helping to suppress false detections of user gestures.

[0088] (Item 4) In the earphone device described in item 1, The aforementioned processor, When the user performs a deletion operation on the operation terminal for any one of the gesture identification information registered in the storage unit, the gesture identification information corresponding to the deletion operation and the detection value of the sensor associated with that gesture identification information are deleted from the storage unit. This allows the earphone device to delete user-registered gestures, thereby improving the user experience.

[0089] (Item 5) The earphone device related to this disclosure is The device comprises an earphone device that is worn in at least one of the user's left or right ears, The aforementioned earphone device Processor and It has a sensor that detects the gestures made by the user to operate the operating terminal, The aforementioned processor, It is possible to access a memory unit that stores a gesture list that manages recommended candidates for multiple gestures, a habit list that manages gestures that the user has made a habit of doing on a daily basis, and a recommended gesture list that is recommended for use by the user. Based on the detection of a specific gesture performed by the user more than a certain number of times, the specific gesture is added to the habit list. Among the recommended candidates for the multiple gestures included in the gesture list, gestures that do not match the multiple specific gestures registered in the habit list are registered in the recommended gestures list. As a result, the earphone device can suggest gestures that are less likely to be misdetected during normal use, allowing users to customize the gestures they use to operate their operating device (smartphone) to be less prone to misdetection.

[0090] (Item 6) In the earphone device described in item 5, The processor does not register in the recommended gestures list any gesture among the multiple gesture candidates included in the gesture list that matches a specific gesture registered in the habit list. This means that the earphone device does not register the user's frequently used gestures (habitual gestures) in its recommended gesture list, thus helping to prevent unintended accidental operations.

[0091] (Item 7) In the earphone device described in item 5, The earphone device further comprises a communication unit that communicates with the operating terminal, The processor instructs the operating terminal to display the recommended gesture list via the communication unit. This allows the user to easily check the recommended gesture list, according to the earphone device.

[0092] (Item 8) In the earphone device described in item 1, The processor adjusts and sets the detection sensitivity of the sensor in response to the user's operation on the operating terminal to adjust the detection sensitivity of the sensor. This allows the earphone device to set the detection sensitivity of the gyro sensor to a user-defined setting, thereby helping to prevent user errors.

[0093] (Item 9) The method for customizing the operation of the earphone device relating to this disclosure is: This is performed by an earphone device that includes an earphone device worn in at least one of the user's left or right ears, The earphone device comprises at least a communication unit for communicating with an operating terminal, and a sensor for detecting gestures made by the user to operate the operating terminal. When the user makes any gesture, the operating terminal is instructed to display the content of the gesture based on the sensor's detected value. In response to the user's instruction operation to the operating terminal for gesture identification information based on the content of the gesture, the detected value of the sensor based on the gesture and the gesture identification information sent from the operating terminal via the communication unit are linked and registered in the storage unit. This allows users to register gestures they wish to remember in the memory unit, thus improving the user experience.

[0094] While embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these are also understood to fall within the technical scope of this disclosure. Furthermore, the components of the embodiments described above can be combined in any way without departing from the spirit of the invention. [Industrial applicability]

[0095] The technology disclosed herein can provide an earphone device and a method for customizing the operation of an earphone device, which helps to efficiently and arbitrarily customize the gestures of a user that instructs the operation. [Explanation of symbols]

[0096] 10L Left Earphone 10R Right Earphone 11L, 11R Earphone Control Unit 12L, 12R Gyro Sensor 15L, 15R, 54 RAM 16L, 16R, 53 ROM 17L, 17R Audio Input / Output Control Unit 19L, 19R Wireless communication department 20L, 20R power monitoring section 50 Smartphones 51 Control Unit 52 Display / operation section 56 Short-range wireless control unit 57 Earphone Communication Interface Section 59 Audio Bus 100 TWS System SPKL, SPLR Speakers MCL, MCR microphone ATR, ATL, AT1 antenna BTL, BTR, BT2 batteries

Claims

1. The device includes an earphone device that is worn in at least one of the user's left or right ears, The aforementioned earphone device Processor and A communication unit that communicates with the operating terminal, The device includes a sensor that detects gestures made by the user when wearing the earphone device for operating the aforementioned operating terminal, The aforementioned processor, When the user makes any gesture, the operating terminal is instructed to display the content of the gesture based on the sensor's detected value. In response to the user's instruction operation to the operating terminal for gesture identification information based on the content of the gesture, the detected value of the sensor based on the gesture and the gesture identification information sent from the operating terminal via the communication unit are linked and registered in the storage unit. Earphone device.

2. The aforementioned processor, In response to the user's selection operation on the operating terminal between the gesture identification information registered in the storage unit and the control content for any earphone device, the selected control content for the selected earphone device is assigned to the selected gesture identification information and registered in the storage unit. The earphone device according to claim 1.

3. The aforementioned processor, When the user performs a specific gesture corresponding to one of the gesture identification pieces registered in the storage unit, the operation terminal is instructed to display the result of comparing the sensor's detection value based on the specific gesture performed by the user with the sensor's detection value registered in the storage unit corresponding to the gesture identification piece of the specific gesture. The comparison results include an indicator showing the user's ability to reproduce the specific gesture, The earphone device according to claim 1.

4. The aforementioned processor, When the user performs a deletion operation on the operation terminal for any one of the gesture identification information registered in the storage unit, the gesture identification information corresponding to the deletion operation and the sensor detection value associated with that gesture identification information are deleted from the storage unit. The earphone device according to claim 1.

5. The device includes an earphone device that is worn in at least one of the user's left or right ears, The aforementioned earphone device Processor and It includes a sensor that detects the gestures made by the user when wearing the earphone device for operating the control terminal, The aforementioned processor, It is possible to access a memory unit that stores a gesture list that manages recommended candidates for multiple gestures, a habit list that manages gestures that the user has made a habit of doing on a daily basis, and a recommended gesture list that is recommended for use by the user. Based on the detection of a specific gesture performed by the user more than a certain number of times, the specific gesture is added to the habit list. Among the multiple recommended gestures included in the gesture list, gestures that do not match the specific gesture registered in the habit list are registered in the recommended gestures list. Earphone device.

6. The processor does not register in the recommended gesture list any gesture among the multiple gesture candidates included in the gesture list that matches a specific gesture registered in the habit list. The earphone device according to claim 5.

7. The earphone device further comprises a communication unit that communicates with the operating terminal, The processor, via the communication unit, instructs the operating terminal to display the recommended gesture list. The earphone device according to claim 5.

8. The processor adjusts and sets the detection sensitivity of the sensor in response to the user's operation to adjust the detection sensitivity of the sensor on the operating terminal. The earphone device according to claim 1 or claim 5.

9. This is performed by an earphone device that includes an earphone device worn in at least one of the user's left or right ears, The earphone device comprises at least a communication unit for communicating with an operating terminal, and a sensor for detecting gestures made by the user when wearing the earphone device for operating the operating terminal. When the user makes any gesture, the operating terminal is instructed to display the content of the gesture based on the sensor's detected value. In response to the user's instruction operation to the operating terminal for gesture identification information based on the content of the gesture, the detected value of the sensor based on the gesture and the gesture identification information sent from the operating terminal via the communication unit are linked and registered in the storage unit. How to customize the operation of the earphone device.

Images