Information processing device and information processing method

Abstract

This information processing device comprises: a gesture detection unit that detects a user gesture performed in a detection region of a detection unit that detects objects; and a correction unit that, on the basis of information relating to the gesture and / or information relating to the state of a prescribed device comprising the detection unit, corrects a detectable range of the gesture in the detection region and / or a reference direction for detecting the gesture.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] 2. Description of the Related Art Various technologies relating to device operation have been developed. For example, in recent years, active development has been made of technologies that enable device operation using gestures. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-109873 Summary of the Invention [Problem to be solved by the invention]

[0004] While gesture-based device operation has the advantage of allowing users to easily operate devices, it also has the problem of prone to false detection. For example, when using gesture-based device operation, problems can arise such as the user's unconscious actions being erroneously detected as gestures, or the user's gesture being detected as a gesture different from the gesture intended by the user.

[0005] Therefore, the present disclosure proposes an information processing device and an information processing method that are capable of realizing highly accurate gesture detection. [Means for solving the problem]

[0006] In order to solve the above problems, an information processing device according to one embodiment of the present disclosure includes a gesture detection unit that detects a user gesture performed in a detection area of a detection unit that detects an object, and a correction unit that corrects at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture based on at least one of information regarding the gesture and information regarding the state of a specified device that includes the detection unit. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 10 is a diagram illustrating how a user performs a gesture. [Figure 2] FIG. 10 is a diagram illustrating how a user performs a gesture. [Figure 3A] 10A and 10B are diagrams illustrating how the position of a detection area changes depending on the orientation of the information processing device. [Figure 3B] 10A and 10B are diagrams illustrating how the position of a detection area changes depending on the orientation of the information processing device. [Figure 4A] 10A and 10B are diagrams illustrating how the orientation of an information processing device changes depending on the orientation in which the information processing device is attached. [Figure 4B] 10A and 10B are diagrams illustrating how the orientation of an information processing device changes depending on the orientation in which the information processing device is attached. [Figure 5] FIG. 10 is a diagram showing a state in which the detectable range of a gesture is limited to a part of the detection area. [Figure 6] FIG. 10 is a diagram illustrating how the information processing device corrects the reference direction. [Figure 7] FIG. 10 is a diagram illustrating how the information processing device corrects the reference direction. [Figure 8] FIG. 1 is a diagram illustrating a configuration example of an information processing device according to an embodiment of the present disclosure. [Figure 9] FIG. 2 illustrates an example of the configuration of an input detection unit. [Figure 10] FIG. 10 is a diagram showing the correspondence between functions of an information processing device and gestures. [Figure 11] FIG. 10 is a diagram illustrating a pinch operation. [Figure 12] FIG. 10 is a diagram for explaining a hold operation. [Figure 13A] FIG. 10 is a diagram illustrating a left / right swipe. [Figure 13B] FIG. 10 is a diagram illustrating a left / right swipe. [Figure 14A] FIG. 10 is a diagram illustrating an up and down swipe. [Figure 14B] FIG. 10 is a diagram illustrating an up and down swipe. [Figure 15A] FIG. 10 is a diagram illustrating the action of holding a hand over the device. [Figure 15B] FIG. 10 is a diagram illustrating the action of holding a hand over the device. [Figure 16] FIG. 10 is a diagram illustrating a touch operation. [Figure 17] FIG. 10 is a diagram illustrating an action of moving the hand away. [Figure 18] 10 is a flowchart illustrating a command execution process according to the present embodiment. [Figure 19A] FIG. 10 is a diagram illustrating an example of a case where the gesture detection unit does not detect a gesture. [Figure 19B] FIG. 10 is a diagram illustrating an example of a case where the gesture detection unit does not detect a gesture. [Figure 20A] FIG. 10 is a diagram illustrating an example in which a gesture detection unit detects a gesture. [Figure 20B] FIG. 10 is a diagram illustrating an example in which a gesture detection unit detects a gesture. [Figure 21] FIG. 1 illustrates an example of the configuration of an information processing system. [Figure 22] FIG. 1 is a diagram illustrating a configuration example of an output device according to an embodiment of the present disclosure. [Figure 23] FIG. 1 is a diagram illustrating a configuration example of a terminal device according to an embodiment of the present disclosure. [Figure 24] FIG. 10 is a diagram showing how a user manually sets the size of the detectable range. [Figure 25] FIG. 10 is a diagram showing how a user manually sets the position of the detectable range. [Figure 26]FIG. 10 is a diagram showing a state in which gestures frequently used by users and their success rates are displayed. [Figure 27] FIG. 10 is a diagram illustrating an example of the configuration of an information processing device according to a third embodiment. [Figure 28] FIG. 10 is a diagram illustrating a state in which a user performs a gesture by physical contact. [Figure 29] 11 is a flowchart illustrating an example of a command execution process according to the third embodiment. [Figure 30] FIG. 10 is a diagram showing a user making a gesture in mid-air. [Figure 31] 11 is a flowchart showing another example of the command execution process according to the third embodiment. [Figure 32] FIG. 10 is a diagram showing how a command accompanied by an operation amount is input. [Figure 33] FIG. 10 is a diagram illustrating how the detectable range is changed depending on the distance to an obstacle. [Figure 34A] FIG. 10 is a diagram showing an example of a GUI for setting a detectable range. [Figure 34B] FIG. 10 is a diagram showing an example of a GUI for setting a detectable range. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0009] Furthermore, in this specification and drawings, multiple components having substantially the same functional configuration may be distinguished by adding different letters after the same reference numeral. For example, multiple elements having substantially the same functional configuration may be distinguished as necessary, such as output devices 20A and 20B. However, when there is no need to particularly distinguish between multiple elements having substantially the same functional configuration, only the same reference numeral is used. For example, when there is no need to particularly distinguish between output devices 20A and 20B, they will simply be referred to as output device 20.

[0010] The present disclosure will be described in the following order. 1. Overview of this Disclosure 1-1.Challenges 1-2. Solution 2. Embodiment 1 2-1. Configuration of information processing device 2-2. Gestures that can be detected by information processing devices 2-3. Operation of information processing device 3. Embodiment 2 3-1. Information processing system configuration 3-2.Output device configuration 3-3. Terminal device configuration 3-4. Operation of information processing system 4. Embodiment 3 4-1. Overview of the Third Embodiment 4-2.Configuration of information processing device 4-3. Operation of information processing device 5. Variations 6. Conclusion

[0011] <<1. Overview of this Disclosure>> First, an overview of the present disclosure will be described.

[0012] <1-1. Issues> In recent years, there has been active development of technologies that enable device operation using gestures. For example, technologies have been developed that enable users to use gestures to input commands (e.g., Next / Prev operations) into devices that can be worn by users, such as headphones or earphones.

[0013] Many devices capable of gesture detection include a detection unit (e.g., a proximity sensor) capable of detecting an object without contact, and detect a user's gesture based on the detection result (e.g., a sensor value) of the detection unit. In the following description, a device capable of gesture detection may be simply referred to as an information processing device.

[0014] 1 and 2 are diagrams illustrating how a user performs gestures. FIG. 1 illustrates earphones 10A as an example of an information processing device of this embodiment. FIG. 2 illustrates headphones 10B as an example of an information processing device of this embodiment. Device operation using gestures is performed by a user performing a predetermined gesture in a space near the information processing device. In the examples of FIGS. 1 and 2, the earphones 10A and the headphones 10B are worn on the user's ears, and the user U performs a gesture for device operation in a space near his or her ear. In the example of FIG. 1, the user U performs a gesture of moving his or her hand H from front to back, and in the example of FIG. 2, the user U performs a gesture of moving his or her hand H from back to front.

[0015] In this embodiment, the action of moving the hand includes the action of moving the fingers. In the following description, the term "hand" can be replaced with "finger" as appropriate.

[0016] In the following description, an XYZ coordinate system may be used to facilitate understanding. Here, the X-axis, Y-axis, and Z-axis directions are all determined relative to the user. The positive X-axis direction is the front direction of the user U, and the negative X-axis direction is the back direction of the user U. The positive Y-axis direction is the left-hand direction of the user U, and the negative Y-axis direction is the right-hand direction of the user U. The positive Z-axis direction is the upward direction, and the negative Z-axis direction is the downward direction. In this embodiment, the user U is assumed to be standing or sitting, and the X-axis and Y-axis are horizontal directions, and the Z-axis is vertical (direction of gravity). The X-axis, Y-axis, and Z-axis directions used in the following description may be interpreted appropriately according to the user's posture. For example, when the user is lying on his / her back, the X-axis direction is vertical, and the Y-axis and X-axis directions are horizontal.

[0017] In the following description, a gesture in which the user U moves the hand H in the horizontal direction (X-axis direction) may be referred to as a left / right swipe. In the following description, a gesture in which the user U moves the hand H in the vertical direction (Z-axis direction) may be referred to as an up / down swipe.

[0018] As described above, the information processing device detects a user's gesture based on the object detection result of the detection unit. When the information processing device detects a gesture, it executes a function corresponding to the gesture (for example, Next / Prev or Vol+ / Vol-).

[0019] While operating a device using gestures has the advantage of allowing a user to easily operate the device, it also has the problem of being prone to false detection. Assume here that the information processing device is a device that can be worn by the user. In this case, possible causes of false detection are (1-1-1) a change in the position of the detection area of the detection unit and (1-1-2) a change in the orientation of the information processing device. False detections caused by these factors will be described in detail below. In the following description, a device that can be worn by a user may be referred to as a wearable device for short.

[0020] (1-1-1) Change in the position of the detection area of the detection unit If the information processing device is a wearable device, the position of the detection area (also called the operating area) of the detector changes depending on the orientation of the information processing device. Figures 3A and 3B are diagrams showing how the position of the detection area changes depending on the orientation of the information processing device. Figure 3A is a diagram showing the earphone 10A when worn in a standard orientation, and Figure 3B is a diagram showing the earphone 10A when worn in an irregular orientation.

[0021] In the example of FIG. 3A, the earphones 10A are worn in a standard orientation, so the detection area OR is located in a certain range centered on the position of the user U's ear when viewed from the positive direction of the Y axis. That is, in the example of FIG. 3A, the detection area OR is located in the range intended by the device developer. On the other hand, in the example of FIG. 3B, the earphones 10A are worn in an irregular orientation, so the detection area OR is located in a range from slightly below the user U's ear to the temple of the head. That is, in the example of FIG. 3B, the detection area OR is not located in the range intended by the device developer. In other words, when a user intends to make a gesture near their ear, in the example of FIG. 3A, the detection area OR is located in a position that matches the user's sense, whereas in the example of FIG. 3B, the detection area OR is located in a position that does not match the user's sense.

[0022] If the detection area OR is located in a position that does not match the user's sense of touch, even actions that the user U did not intend to make as gestures will be detected as gestures. For example, in the example of FIG. 3B, the action of the user U bringing his / her hand to the side of his / her head to scratch his / her head is likely to be detected as an action within the detection area OR. In this case, the action may be detected as a gesture by the information processing device.

[0023] Furthermore, if the detection area OR is located in a position that does not match the user U's sense of touch, the movement that the user U intends to make a gesture may end up being outside the detection area OR. For example, in the example of FIG. 3B , a gesture made slightly below the earphone 10A is likely to be outside the detection area OR, even though it is close to the user's ear. In this case, the information processing device cannot detect the user's movement as a gesture. To prevent the user U's gesture from being outside the detection area OR, it is conceivable to increase the sensitivity of the detection unit to widen the detection area OR. However, in this case, while the detection area OR is fine in the direction below the user U's ear, it will expand to positions that do not match the user U's sense of touch in other directions. This will result in a problem similar to the one described above.

[0024] 3A and 3B show an earphone-type information processing device, but the change in the position of the detection area of the detection unit can also occur when the information processing device is a device other than an earphone-type device. For example, the change in the position of the detection area of the detection unit can also occur when the information processing device is a headphone-type device such as that shown in FIG. 2. Therefore, the same problem as described in (1-1-1) above can also occur when the information processing device is a device other than an earphone-type device.

[0025] (1-1-2) Change in orientation of information processing device When the information processing device is a wearable device, the orientation of the information processing device relative to the orientation of the user U changes depending on the orientation in which the information processing device is worn. Figures 4A and 4B are diagrams showing how the orientation of the information processing device changes depending on the orientation in which the information processing device is worn. Figure 4A is a diagram showing how the headphones 10B are worn in a standard wearing orientation, and Figure 4B is a diagram showing how the headphones 10B are worn in an irregular orientation.

[0026] Here, D1 to D4 in the figure are directions based on the device. More specifically, D1 to D4 in the figure indicate the upward, downward, forward, and backward directions when the information processing device is attached in the standard attachment orientation. D1 is the upward direction, D2 is the downward direction, D3 is the forward direction, and D4 is the backward direction. In the following description, D1 to D4 are referred to as device directions.

[0027] In the example of Fig. 4A, the headphones 10B are worn in a standard orientation, so the vertical direction (Z-axis direction) and device directions D1 and D2 coincide, and the horizontal direction (X-axis direction) and device directions D3 and D4 also coincide. On the other hand, in the example of Fig. 4B, the headphones 10B are worn in an irregular orientation, so the vertical direction (Z-axis direction) and device directions D1 and D2 do not coincide, and the horizontal direction (X-axis direction) and device directions D3 and D4 do not coincide either. In this way, when the information processing device is a wearable device, the orientation of the information processing device (e.g., device directions D3 and D4, device directions D1 and D2) relative to the orientation of the user U (e.g., X-axis direction, Z-axis direction) changes.

[0028] Here, assume that the user performs a left / right swipe as an input operation to the information processing device. Generally, the information processing device determines whether the gesture performed by the user is a left / right swipe based on the horizontal direction (device directions D3, D4) when normally worn. In the example of FIG. 4A, the horizontal direction (X-axis direction) and the horizontal direction (device directions D3, D4) of the information processing device when normally worn coincide with each other. Therefore, the information processing device can detect that the user's gesture is a left / right swipe without any problem.

[0029] However, in the example of FIG. 4B, the horizontal direction (X-axis direction) does not match the horizontal direction of the information processing device when normally worn (device directions D3, D4). Therefore, even if the user makes a gesture intending to swipe left or right, the information processing device cannot detect the gesture as a left or right swipe. In the example of FIG. 4B, the direction of the user's hand movement is close to the vertical direction of the information processing device when normally worn (device directions D1, D2). Therefore, in the example of FIG. 4B, the information processing device erroneously detects that the user has swiped up or down.

[0030] Although FIG. 4B shows a headphone-type information processing device, the change in the orientation of the information processing device may occur when the information processing device is a device other than a headphone-type device. For example, the change in the orientation of the information processing device may occur when the information processing device is a device other than a headphone-type device. Therefore, the same problem as described in (1-1-2) above may occur when the information processing device is a device other than a headphone-type device.

[0031] <1-2. Solution> Therefore, in this embodiment, the above problems (1-1-1) and (1-1-2) are solved by the following means. Examples of the solutions include (1-2-1) correcting the gesture detectable range and (1-2-2) correcting the reference direction for gesture detection. These solutions are described below. The information processing device may execute the following solutions in parallel or in combination.

[0032] (1-2-1) Correction of gesture detection range As described above, if the detection area OR is located in a position that does not match the user's sense, the information processing device may detect as a gesture even an action that the user does not intend as a gesture. Therefore, the information processing device of this embodiment limits the detectable range of a gesture to a partial area of the detection area OR.

[0033] FIG. 5 is a diagram showing a state in which the detectable range of a gesture is limited to a partial area of the detection area OR. In the example of FIG. 5, the detection area OR is formed in a certain range around the information processing device 10 when viewed from the positive direction of the Y axis. Note that the information processing device 10 may be an earphone 10A or a headphone 10B. In the example of FIG. 5, the detection area OR is divided into a detectable range AR and a non-detectable range NR. The detectable range AR is a range in which the information processing device 10 can detect a gesture, and the non-detectable range NR is a range in which the information processing device 10 does not detect a gesture. Both the detectable range AR and the non-detectable range NR are ranges within the detection area OR.

[0034] The information processing device 10 corrects the detectable range AR based on information related to a gesture of the user U. Here, the information related to the gesture may be information related to a movement of the user U related to the gesture. More specifically, the information related to the gesture may be information related to a movement of the user U in which the hand H enters the detectable range AR (hereinafter referred to as an entering movement). If the information processing device 10 is a device worn on the ear of the user U, the entering movement may be a movement of the user U in which the hand H is raised toward the ear (hereinafter referred to as a raising movement). The raising movement is, for example, a movement in which the user U holds the elbow joint still or moves it slowly forward, and raises the hand H of the user U toward the ear like a pendulum, with the elbow joint as a fulcrum. In the following description, the entering movement is assumed to be a movement in which the hand H of the user U is raised toward the ear.

[0035] When a user U raises his / her hand H, the approach direction AD of the user U's hand H into the detectable range AR is somewhat determined due to the structure of the human body. Although there are individual differences and differences depending on the gesture, in most cases, the approach direction AD is assumed to fall within a certain angle above and below the rear direction (negative X-axis direction). In the example of FIG. 5, the user's hand H enters the detectable range AR from a position slightly below the front of the user U. Therefore, in the example of FIG. 5, the approach direction AD is a direction slightly tilted upward from the rear direction (negative X-axis direction).

[0036] If the approach direction AD is known, the information processing device 10 can estimate where the user U will make the gesture. Therefore, the information processing device 10 corrects the detectable range AR based on the information about the approach direction AD so that it is narrower than the detection area OR. In the example of FIG. 5, as a result of the correction, the detectable range AR is set so that it is biased toward the detection area OR so that part of the edge of the detectable range AR contacts part of the edge of the detection area OR. More specifically, the detectable range AR is positioned biased toward a position where it is estimated that the user U has entered the detection area OR with their hand H (hereinafter referred to as the approach position). Here, the approach position, using the example of FIG. 5, is, for example, the intersection of the arrow indicating the approach direction AD and the dashed line indicating the edge of the detection area OR.

[0037] The information on the approaching motion is not limited to the approaching direction AD, but may be, for example, information on the approaching position. In this case, the information processing device 10 may correct the detectable range AR to be narrower than the detection area OR based on the information on the approaching position.

[0038] According to this solution, the information processing device 10 corrects the detectable range of the gesture based on information about the raising motion of the hand H of the user U. This reduces the number of cases where a motion that the user does not intend to make a gesture is detected as a gesture. Even if the information processing device 10 increases the sensitivity of the detection unit and widens the detection area OR, the possibility of erroneous detection is low.

[0039] (1-2-2) Correction of reference direction for gesture detection As described above, the orientation of the information processing device 10 relative to the orientation of the user U changes depending on the orientation in which the information processing device 10 is worn. Therefore, the information processing device 10 of this embodiment corrects the reference direction for gesture detection in accordance with the tilt of the device. Note that "detection" in gesture detection has a broad meaning that also includes gesture recognition. Here, the reference direction for gesture detection refers to a direction that is used as a reference when the information processing device 10 detects a gesture. For example, if the information processing device 10 is attempting to detect a left or right swipe, the reference direction is, for example, the horizontal direction.

[0040] FIG. 6 is a diagram illustrating how the information processing device 10 corrects the reference direction. In the example of FIG. 6, B1 to B4 are corrected reference directions. In the following description, B1 to B4 may be simply referred to as the "reference direction." Since the device directions D1 to D4 are directions based on the device, if the information processing device 10 is worn at an angle, the directions will be tilted from the horizontal or vertical direction. If the information processing device 10 detects a gesture using the device directions D1 to D4 as the reference direction, erroneous detection may occur, such as detecting a left-right swipe as an up-down swipe.

[0041] Various methods are conceivable for correcting the reference direction. For example, the information processing device 10 corrects the reference direction based on information related to a gesture. Here, the information related to the gesture may be information related to a movement of the user U related to the gesture. More specifically, the information related to the gesture may be information related to an approaching movement of the user U bringing the hand H into the detectable range AR. If the information processing device 10 is a device worn on the ear of the user U, the approaching movement may be an movement of the user U raising the hand H toward the ear (a raising movement).

[0042] As described above, when the user U raises his / her hand H, the approach direction AD of the user U's hand H into the detectable range AR is a somewhat determined direction. The information processing device 10 corrects the reference direction based on information about the approach direction AD. FIG. 7 is a diagram showing how the information processing device 10 corrects the reference direction. Since the device directions D1 to D4 are directions based on the device, the information processing device 10 naturally knows the device directions D1 to D4. The information processing device 10 corrects the reference direction based on information about the device directions D1 to D4 relative to the approach direction AD. As an example, the information processing device 10 can correct the reference direction in the following manner.

[0043] For example, the information processing device 10 calculates an angle R between at least one of the device directions D1 to D4 (device direction D2 in the example of FIG. 7) and the approach direction AD. The approach direction AD is, for example, the angle at position C where the hand H of the user U enters the detectable range AR. As described above, the approach direction AD is expected to be a direction slightly tilted upward from the back direction of the user U (the negative direction of the X-axis). Therefore, the information processing device 10 knows the angle of the device direction reference for the approach direction AD when normally worn. The information processing device 10 can estimate the wearing angle of the information processing device 10 from the difference between the angle of the approach direction AD when normally worn and the angle of the approach direction AD that is actually detected. Note that the information processing device 10 may use this angle difference as an estimation result as it is. Then, based on the estimation result of the wearing angle of the information processing device 10, the reference direction is corrected to one of the reference directions B1 to B4.

[0044] 7, the reference directions B1 to B4 are horizontal or vertical, but they do not necessarily have to be horizontal or vertical. Any direction that the information processing device 10 uses as a reference for detecting a gesture can be set as the reference direction. For example, if the information processing device 10 detects a swipe in a 45° diagonal direction, the 45° diagonal direction can be considered as the reference direction.

[0045] According to this solution, the information processing device 10 detects gestures based on the corrected reference directions B1 to B4 rather than using the device directions D1 to D4 as reference directions, which reduces erroneous detections such as detecting left / right swipes as up / down swipes.

[0046] (1-2-3) Other solutions In the above (1-2-1) and (1-2-2), the detectable range of a gesture and / or the reference direction for detecting a gesture is corrected based on information about the gesture of the user U. However, the information processing device 10 can also correct the detectable range and / or the reference direction using information other than information about a gesture.

[0047] For example, the information processing device 10 may correct at least one of the detectable range and the reference direction based on information about the state of the information processing device 10. For example, the information processing device 10 includes a sensor unit that detects the state of the information processing device 10, and corrects at least one of the detectable range and the reference direction based on information from the sensor unit.

[0048] Here, the sensor unit may be configured with one or more acceleration sensors or one or more gyro sensors. In this case, the information processing device 10 may be capable of correcting at least one of the detectable range and the reference direction based on the state of attachment of the information processing device 10 to the user U estimated based on information from the sensor unit.

[0049] Since the information processing device 10 is constantly subjected to gravitational acceleration, the information processing device 10 can estimate the direction of gravity based on information from the sensor unit. The information processing device 10 estimates the tilt of the information processing device 10 from a standard wearing state based on the estimated direction of gravity. Then, the information processing device 10 corrects at least one of the detectable range and the reference direction based on the estimated tilt. The information processing device 10 detects a gesture based on the corrected detectable range and / or the reference direction.

[0050] This method can also reduce false detection of the gestures of the user U, similar to the methods described in (1-2-1) and (1-2-2).

[0051] <<2. Embodiment 1>> The outline of this embodiment has been described above, and the information processing device 10 according to the first embodiment will be described below.

[0052] <2-1. Configuration of information processing device> First, the configuration of the information processing device 10 will be described.

[0053] The information processing device 10 is an information processing terminal that executes various functions based on user input operations. For example, the information processing device 10 is an audio output device that can be worn by a user, such as earphones or headphones. The information processing device 10 may also be a display device that can be worn by a user, such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses. In this embodiment, the information processing device 10 is assumed to be a device that can be worn by a user U and to have at least a part that is located at the ear of the user U when worn. Furthermore, it is assumed that this part is equipped with a detection unit that detects at least an object. The information processing device 10 is configured to be able to detect user gestures and executes functions such as music playback based on the user gestures.

[0054] Fig. 8 is a diagram illustrating an example configuration of an information processing device 10 according to an embodiment of the present disclosure. The information processing device 10 includes an input detection unit 11, a state detection unit 12, an output unit 13, a communication unit 14, a storage unit 15, and a control unit 16. Note that the configuration illustrated in Fig. 8 is a functional configuration, and the hardware configuration may be different from this. Furthermore, the functions of the information processing device 10 may be distributed and implemented in multiple physically separated configurations.

[0055] The input detection unit 11 is a detection unit that detects a user's input operation. For example, the input detection unit 11 is a non-contact detection unit that detects an object without contact. The input detection unit 11 includes one or more proximity sensors (also called non-contact sensors) and detects an object located within a detection area. The proximity sensor may be an optical sensor or a capacitance sensor. An optical proximity sensor is a sensor that detects light reflected by an object. A capacitance proximity sensor is a sensor that detects a change in capacitance that occurs between an object and the sensor.

[0056] When the input detection unit 11 is a non-contact type, the input detection unit 11 may be a 2D sensor or a 3D sensor. When the input detection unit 11 is a 2D sensor, the detection area formed by the input detection unit 11 is a two-dimensional area, and when the input detection unit 11 is a 3D sensor, the detection area formed by the input detection unit 11 is a three-dimensional area.

[0057] The input detection unit 11 is not limited to a non-contact type. The input detection unit 11 may also be a contact type detection unit. In this case, the input detection unit 11 includes one or more touch sensors, and detects an object that comes into contact with a predetermined location on the information processing device 10. If the information processing device 10 is an earphone-type device, the predetermined location is, for example, the side of the earphone (the side opposite the speaker surface). If the information processing device 10 is a headphone-type device, the predetermined location is, for example, the side of the headphone (the side opposite the speaker surface). In the following description, the side of the earphone or the side of the headphone may be referred to as the housing surface.

[0058] Fig. 9 is a diagram showing an example of the configuration of the input detection unit 11. Fig. 9 shows an earphone-type information processing device 10 as seen from the side of the earphone. Note that the configuration shown in Fig. 9 is merely an example, and the configuration of the input detection unit 11 is not limited to the configuration shown in Fig. 9.

[0059] 9, the input detection unit 11 includes an infrared emitter IR and four photosensors PD1, PD2, PD3, and PD4 as non-contact sensors. The infrared emitter ID is disposed at the center of the housing surface, and the four photosensors PD1, PD2, PD3, and PD4 are disposed at equal intervals in the circumferential direction so as to surround the infrared emitter ID. The input detection unit 11 detects an object by detecting infrared light emitted from the infrared emitter ID and reflected from the object surface with the four photosensors PD1, PD2, PD3, and PD4.

[0060] The input detection unit 11 also includes a touchpad TP as a contact sensor. The touchpad TP is composed of multiple touch sensors arranged on a surface. The infrared light emitter ID is arranged on the surface of the housing. The input detection unit 11 detects an object that comes into contact with the surface of the touchpad TP.

[0061] The state detection unit 12 is a sensor unit that detects the state of the information processing device 10. The state detection unit 12 is configured with one or more sensors that detect the state of the information processing device 10. The state detection unit 12 includes, for example, one or more acceleration sensors. The state detection unit 12 may also include one or more gyro sensors. The state detection unit 12 may also include one or more geomagnetic sensors. The state detection unit 12 may also include a motion sensor that combines these multiple types of sensors. The state detection unit 12 may also include a biosensor that acquires biometric information of the user, such as a blood pressure sensor or a heart rate sensor.

[0062] The state detection unit 12 detects the state of the information processing device 10 based on the detection results of these sensors. For example, the state detection unit 12 detects the direction of gravity based on the sensor values of these sensors. For example, since gravitational acceleration is always applied to the information processing device 10, the information processing device 10 can detect the direction of gravity by averaging the direction of acceleration detected by the acceleration sensor over a certain period of time.

[0063] The output unit 13 is an output interface that outputs information to the user. For example, the output unit 13 may be an acoustic device such as a speaker or a buzzer, or may be a vibration device such as a vibration motor. The output unit 13 may also be a display device such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED), or may be a lighting device such as an LED (Light Emitting Diode) lamp. The output unit 13 functions as output means of the information processing device 10. The output unit 13 outputs various types of information to the user under the control of the control unit 16.

[0064] The communication unit 14 is a communication interface for communicating with other devices. The communication unit 14 may be a network interface or a device connection interface. The communication unit 14 may also be a wired interface or a wireless interface. The communication unit 14 functions as a communication means of the information processing device 10. The communication unit 14 communicates with other devices under the control of the control unit 16. The other devices are, for example, terminal devices such as a music player or a smartphone.

[0065] The storage unit 15 is a data readable / writable storage device such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a hard disk, etc. The storage unit 15 functions as a storage means of the information processing device 10. The storage unit 15 stores, for example, settings related to a gesture detectable range AR.

[0066] The control unit 16 is a controller that controls each unit of the information processing device 10. The control unit 16 is realized by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). For example, the control unit 16 is realized by the processor executing various programs stored in a storage device inside the information processing device 10 using a RAM (Random Access Memory) or the like as a working area. Note that the control unit 16 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The CPU, MPU, ASIC, and FPGA can all be considered as controllers.

[0067] As shown in FIG. 8, the control unit 16 includes an acquisition unit 161, a gesture detection unit 162, a command execution unit 163, an output control unit 164, an estimation unit 165, and a correction unit 166. Each block (acquisition unit 161 to correction unit 166) constituting the control unit 16 is a functional block indicating a function of the control unit 16. These functional blocks may be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks may be a software module realized by software (including a microprogram), or may be a circuit block on a semiconductor chip (die). Of course, each functional block may be a processor or an integrated circuit. The functional blocks may be configured in any manner.

[0068] The control unit 16 may be configured with functional units different from the above-mentioned functional blocks. Also, some or all of the operations of the blocks (acquisition unit 161 to correction unit 166) constituting the control unit 16 may be performed by another device. For example, some or all of the operations of the blocks constituting the control unit 16 may be performed by a terminal device such as a music player or a smartphone. The operations of the blocks constituting the control unit 16 will be described later.

[0069] <2-2. Gestures that can be detected by information processing devices> The configuration of the information processing device 10 has been described above. Before describing the operation of the information processing device 10 according to this embodiment in detail, gestures that can be detected by the information processing device 10 will be described.

[0070] (2-2-1) Function and gesture correspondence FIG. 10 is a diagram showing the correspondence between the functions of the information processing device 10 and gestures. The information processing device 10 has a function related to answering a phone call in addition to a function related to playing music. For example, the information processing device 10 has the following functions (1) to (7). Note that "function" can be rephrased as "command." (1) Play / Pause (2) Answer Call / End (3) Next / Prev (4) Vol+ / Vol- (5) Cancel (6) Quick Attention (7) Ambient Control

[0071] The following describes the functions (1) to (7) and the gestures associated with each of them. Note that a user's action constituting one gesture may include a first action that triggers the start of the gesture and a second action that follows the first action. The first action is, for example, a wave-up action of the user U's hand H, and the second action is, for example, a swipe or other action that follows the first action. It is also possible to regard each of the first action and the second action as one gesture.

[0072] Note that the functions and gestures described here are merely examples. The functions and gestures possessed by the information processing device 10 are not limited to the functions and gestures shown below. The first and second actions are also not limited to the actions shown below. Furthermore, the combinations of functions and gestures are not limited to the combinations shown below. For example, although not described in (1) to (7) above, the Noise Cancelling function may be operated by a gesture.

[0073] In the following description, the information processing device 10 is assumed to be an earphone-type device worn in the user's ear, but the information processing device 10 is not limited to being an earphone-type device. For example, the information processing device 10 may be a headphone-type device. Furthermore, the information processing device 10 may have a portion worn in the user's ear and other portions. Furthermore, in the following description, the information processing device 10 is assumed to be worn in the user's ear, but the device worn in the user's ear may be a predetermined device other than the information processing device 10. The predetermined device may be, for example, an output device wirelessly connected to the information processing device 10 (for example, earphones, headphones, hearing aids, sound amplifiers, etc., regardless of whether they are wired, wireless, in-ear type, open-ear type, etc.). Note that if the information processing device 10 is worn in the user's ear, the predetermined device is the information processing device 10 itself. The predetermined device can be rephrased as a predetermined device.

[0074] Hereinafter, functions of the information processing device 10 and gestures associated with the functions will be described with reference to FIG.

[0075] (1) Play / Pause "Play" is a function for playing a song, and "Pause" is a function for pausing the playback of a song. "Play / Pause" is associated with a wave-up motion of the hand H as a first motion, and a pinch motion as a second motion. FIG. 11 is a diagram for explaining a pinch motion. As shown in FIG. 11, a pinch motion is a motion of pinching an object with fingers. Note that the information processing device 10 can detect that the user U has performed a pinch motion even if the user U is not actually pinching an object (for example, the information processing device 10).

[0076] (2) Answer Call / End "Answer Call" is a function for answering a call, and "Answer End" is a function for ending a call. "Answer Call / End" is associated with a wave-up action of the hand H as a first action, and a pinch action or a hold action as a second action. FIG. 12 is a diagram for explaining the hold action. As shown in FIG. 12, the hold action is an action of grabbing an object with the hand H. Note that the information processing device 10 can detect that the user U has performed a hold action even if the user U has not actually grabbed an object (for example, the information processing device 10).

[0077] (3) Next / Prev "Next" is a function for cueing the next song, and "Prev" is a function for cueing the previous song or the song currently being played. "Next / Prev" is associated with a gesture of raising the hand H as a first motion, and a left / right swipe as a second motion. Here, "Next" is associated with a left swipe (forward swipe) of left / right swipes as a second motion, and "Prev" is associated with a right swipe (backward swipe) of left / right swipes as a second motion. Note that "Next / Prev" may be associated with a gesture for performing the second motion without the first motion.

[0078] 13A and 13B are diagrams for explaining left and right swipes. FIG. 13A shows a left swipe, and FIG. 13B shows a right swipe. Unlike pinch and hold actions, left and right swipes are actions (gestures) that involve a movement width. Here, as shown in FIG. 13A, a left swipe is an action in which a user U slides their hand H forward (in the positive direction of the X-axis) by a predetermined movement width W1. Also, as shown in FIG. 13B, a right swipe is an action in which a user U slides their hand H backward (in the negative direction of the X-axis) by a predetermined movement width W2.

[0079] 13A and 13B are diagrams illustrating how the user U makes a gesture near the left ear. Unlike the examples of FIGS. 13A and 13B, when the user U makes a gesture near the right ear, it should be noted that a right swipe is an action of sliding the hand H forward, and a left swipe is an action of sliding the hand H backward. In this case, for example, a right swipe (forward swipe) is associated with "Next" as the second action, and for example, a left swipe (backward swipe) is associated with "Prev" as the second action.

[0080] (4) Vol+ / Vol- "Vol+" is a function to increase the volume, and "Vol-" is a function to decrease the volume. "Vol+ / Vol-" is associated with a waving motion of the hand H as a first motion, and an up / down swipe as a second motion. Here, "Vol+" is associated with an up swipe of the up / down swipe as a second motion, and "Vol-" is associated with a down swipe of the up / down swipe as a second motion. Note that "Vol+ / Vol-" may be associated with a gesture that performs the second motion without the first motion.

[0081] 14A and 14B are diagrams for explaining an up-and-down swipe. FIG. 14A shows a left swipe, and FIG. 14B shows a right swipe. Unlike pinch and hold actions, up-and-down swipes are actions (gestures) that involve a movement range. Here, an up-swipe is an action in which the user U slides their hand H upward (in the positive direction of the Z axis) by a predetermined movement range W3, as shown in FIG. 14A. A down-swipe is an action in which the user U slides their hand H downward (in the negative direction of the Z axis) by a predetermined movement range W4, as shown in FIG. 14B.

[0082] Note that "Vol+ / Vol-" is a function for adjusting the volume. Therefore, in order to execute this function, it is desirable that the information processing device 10 acquires not only information on raising or lowering the volume but also information on the amount of volume to be raised or lowered (amount of volume raised or lowered) from the user U. Therefore, in this embodiment, "Vol+ / Vol-" is a function that involves an operation amount (in the case of this function, the amount of volume raised or lowered). In this function, for example, the movement amount or movement speed of the hand H when swiping up or down is associated with the operation amount of "Vol+ / Vol-". Note that it is also possible to configure the information processing device 10 so that the volume is raised or lowered by a certain amount with one input operation, thereby making this function a function that does not involve an operation amount.

[0083] (5) Cancel "Cancel" is a function for canceling an operation performed by the user U. "Cancel" is associated with a wave-up action of the hand H as a first action, and with a holding-up action of the hand as a second action. FIGS. 15A and 15B are diagrams for explaining the holding-up action of the hand H. FIG. 15A is a diagram of the user U as seen from the left side, and FIG. 15B is a diagram of the user U as seen from the front. The holding-up action of the hand H is an action in which the user U holds out their open hand H toward the information processing device 10, as shown in FIGS. 15A and 15B.

[0084] (6) Quick Attention "Quick Attention" is a function that enables the user U to quickly hear surrounding sounds. Specifically, "Quick Attention" is a function that makes it easier to hear surrounding sounds by quickly lowering or stopping the volume of a sound that is being output (for example, music, call voice, or ringtone). "Quick Attention" is associated with a wave-up motion of the hand H as a first motion, and a touch motion as a second motion. FIG. 16 is a diagram for explaining the touch motion. As shown in FIG. 16, the touch motion is a motion in which the user U touches a part of the information processing device 10 (for example, the surface of the housing).

[0085] (7) Ambient Control "Ambient Control" is a function that enables the user U to listen to music while checking ambient sounds. Specifically, "Ambient Control" is a function that captures external sounds while music is being played. "Ambient Control" is associated with a first action, which is a gesture of raising the hand H, and a second action, which is a gesture of moving the hand H away. FIG. 17 is a diagram for explaining the action of moving the hand H away. The action of moving the hand H away is a gesture that involves a movement range, unlike a pinch action, a hold action, a touch action, and a hand-holding action. Here, as shown in FIG. 17, the action of moving the hand H away is an action in which the user U holds the hand H over the information processing device 10, and then moves the hand H in a predetermined moving away direction (the positive Y-axis direction in the example of FIG. 17) by a predetermined movement range W4.

[0086] (2-2-2) Correspondence between functions and correction processes The correspondence between functions and gestures has been described above. Next, the correspondence between functions and correction processes will be described. In addition to the correspondence between functions and gestures, FIG. 10 also shows the correspondence between functions and correction processes. Here, the correction processes are correction processes for the detectable range of a gesture and correction processes for the reference direction for detecting a gesture. In the example shown in FIG. 10, the correction processes differ for each function or each gesture.

[0087] The information processing device 10 may correct the detectable range and the reference direction immediately after the detection of the first movement is completed, or may correct the detectable range and the reference direction at the stage of performing gesture recognition processing after the acquisition of information on the series of user movements related to the gesture is completed.

[0088] For example, after the information processing device 10 has acquired information on both the first and second movements, it corrects the detectable range based on the information on the first movement. Then, the information processing device 10 determines whether the second movement was performed within the detectable range based on the information on the second movement that has been acquired. If the second movement was not performed within the detectable range, the information processing device 10 excludes the information on the second movement that has been acquired from the targets of gesture detection.

[0089] Furthermore, for example, after the information processing device 10 has completed acquiring information on both the first action and the second action, it corrects the reference direction based on the information on the first action. The information processing device 10 interprets the information on the second action, which has been acquired, based on the information on the reference direction corrected based on the information on the first action.

[0090] Hereinafter, the correspondence between the functions of the information processing device 10 and the correction processes will be described with reference to FIG.

[0091] (1) Play / Pause "Play / Pause" is associated with a waving motion of the hand H as a first motion, and a pinch motion as a second motion. In this function, the information processing device 10 limits the detectable range based on, for example, the first motion. Note that the information processing device 10 does not need to correct the reference direction. This is because a pinch motion is a gesture that does not involve a movement range, and therefore information on the reference direction is not necessarily required to detect the gesture.

[0092] (2) Answer Call / End "Answer Call / End" is associated with a wave of the hand H as the first action, and a pinch or hold action as the second action. In this function, the information processing device 10 limits the detectable range based on, for example, the first action. Note that the information processing device 10 does not need to correct the reference direction. This is because pinch and hold gestures do not necessarily require information on the reference direction to detect the gestures, since they are gestures that do not involve a movement range.

[0093] (3) Next / Prev "Next / Prev" is associated with a waving motion of the hand H as a first motion, and a left / right swipe as a second motion. In this function, the information processing device 10, for example, limits the detectable range based on the first motion. After limiting the detectable range based on the first motion, the information processing device 10 may, for example, correct the limited detectable range when detecting the start of a second motion. This improves the detection accuracy of the second motion. Furthermore, in this function, the information processing device 10 corrects the reference direction based on the first motion, for example.

[0094] (4) Vol+ / Vol- "Vol+ / Vol-" is associated with a waving motion of the hand H as the first motion, and with an up / down swipe as the second motion. In this function, the information processing device 10, for example, limits the detectable range based on the first motion. After limiting the detectable range based on the first motion, the information processing device 10 may, for example, correct the limited detectable range when detecting the start of the second motion. This improves the detection accuracy of the second motion. Furthermore, in this function, the information processing device 10 corrects the reference direction based on the first motion, for example.

[0095] (5) Cancel "Cancel" is associated with the action of raising the hand H as the first action, and the action of holding out the hand H as the second action. In this function, the information processing device 10 sets the detectable range based on, for example, information about the state of the information processing device 10. For example, the information processing device 10 sets the detectable range based on information from the state detection unit 12. In this function, the information processing device 10 does not necessarily need to correct the reference direction. This is because the action of holding out the hand H is a gesture that does not involve a movement range, and therefore information about the reference direction is not necessarily required to detect the gesture.

[0096] (6) Quick Attention "Quick Attention" is associated with a wave-up motion of the hand H as the first motion, and with a touch motion as the second motion. In this function, the information processing device 10 does not need to correct the detectable range and the reference direction. This is because information on the detectable range and the reference direction is not necessarily required to detect a touch motion.

[0097] (7) Ambient Control "Ambient Control" is associated with the action of raising the hand H as a first action, and the action of moving the hand H away as a second action. In this function, the information processing device 10, for example, limits the detectable range based on the first action. After limiting the detectable range based on the first action, the information processing device 10 may, for example, correct the limited detectable range when it detects the start of the second action. This improves the detection accuracy of the second action. Furthermore, in this function, the information processing device 10 corrects the reference direction based on the first action, for example.

[0098] <2-3. Operation of the information processing device> The gestures that can be detected by the information processing device 10 have been described above. Next, the operation of the information processing device 10 that can detect such gestures will be described.

[0099] 18 is a flowchart showing a command execution process according to this embodiment. The command execution process is a process in which the information processing device 10 executes a command input by the user U using a gesture. Here, a "command" refers to an instruction from inside or outside the device that causes the information processing device 10 to execute a function. Note that a "command" can also be considered as a word that refers to the function itself that the information processing device 10 has, such as Play / Pause. The term "command" that appears in the following description can be replaced with "function."

[0100] The command execution process is executed by the control unit 16 of the information processing device 10. The command execution process is started, for example, when the information processing device 10 is powered on.

[0101] First, the acquisition unit 161 of the control unit 16 acquires sensor values from the input detection unit 11 and the state detection unit 12 (step S101). For example, the acquisition unit 161 acquires information about an object in the detection area OR from a non-contact sensor such as a proximity sensor. The acquisition unit 161 also acquires information about contact with the housing surface from a contact sensor such as a touch sensor. The acquisition unit 161 also acquires information about the state of the information processing device 10 from sensors such as an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

[0102] Next, the acquisition unit 161 acquires information about the gesture of the user U based on the sensor value acquired in step S101 (step S102). For example, the acquisition unit 161 acquires information about the movement of the user U related to the gesture based on the sensor value from the input detection unit 11. More specifically, the acquisition unit 161 acquires information about an approaching motion of the user U bringing his / her hand H into the detectable range AR based on the sensor value of a non-contact sensor such as a proximity sensor. Note that if the information processing device 10 is a device worn on the ear of the user U, the approaching motion may be an up-movement in which the user U raises his / her hand H toward his / her ear. In this case, the information about the approaching motion may be information about the approaching direction AD of the hand H of the user U into the detectable range AR.

[0103] Next, the acquisition unit 161 acquires information about the state of the information processing device 10 based on the sensor value acquired in step S101 (step S103). For example, the acquisition unit 161 acquires information about the state of the information processing device 10 based on the sensor value from the state detection unit 12. For example, the acquisition unit 161 acquires information about the direction of gravity based on information from an acceleration sensor, a gyro sensor, or a geomagnetic sensor.

[0104] Next, the correction unit 166 of the control unit 16 executes a correction process for gesture detection (step 104). For example, the correction unit 166 corrects at least one of a detectable range of a gesture and a reference direction for detecting the gesture. The corrections performed by the correction unit 166 are broadly classified into two types: (1) corrections based on information about a gesture, and (2) corrections based on information about a device state.

[0105] (1) Correction based on gesture information First, correction based on gesture information will be described.

[0106] The correction unit 166 corrects at least one of the detectable range AR and the reference direction based on the information acquired in step S102. For example, the correction unit 166 corrects at least one of the detectable range AR and the reference direction based on information about the movement of the user U. As described in <1-2. Solution>, the movement of the user U may be a wave-up movement of the hand H. Also, as described in <1-2. Solution>, the wave-up movement may be an approaching movement in which the user U brings the hand H into the detectable range AR. In this case, the correction unit 166 may correct at least one of the detectable range AR and the reference direction based on information about the approaching direction AD of the hand H of the user U into the detectable range AR.

[0107] The information on the approaching motion may be information on the approaching direction AD of the hand H of the user U. In this case, the correction unit 166 may correct at least one of the detectable range AR and the reference direction based on the information on the approaching direction AD. More specifically, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on information on the wearing state of the information processing device 10 on the user U, which is estimated based on the information on the approaching direction AD. The information on the wearing state may be, for example, information on the wearing inclination. The wearing inclination is, for example, the current inclination of the information processing device 10 from a standard wearing state of the information processing device 10. The estimation unit 165 of the control unit 16 may estimate the wearing inclination based on the information on the approaching direction AD. Then, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on the estimation result of the estimation unit 165.

[0108] Note that a user's motion constituting one gesture may include a first motion that triggers the start of the gesture and a second motion that follows the first motion. Here, the first motion is a motion of raising the hand H of the user U. Then, after the raising motion is detected within the detectable range AR, the correction unit 166 may widen the detectable range for detecting the second motion than the detectable range AR when the raising motion was detected. More specifically, when the gesture to be detected (the second motion) is a gesture involving a movement range, the correction unit 166 may widen the detectable range for detecting the second motion than the detectable range AR when the raising motion was detected. A gesture involving a movement range is, for example, a swipe or a motion of moving the hand H away.

[0109] This makes it possible to improve gesture detection accuracy while suppressing false detection.

[0110] (2) Correction based on information about the equipment condition Next, correction based on information about the device state will be described.

[0111] The correction unit 166 corrects at least one of the detectable range AR and the reference direction based on the information acquired in step S103. For example, the correction unit 166 corrects at least one of the detectable range AR and the reference direction based on information about the wearing state of the information processing device 10 on the user U, which is estimated based on information from the state detection unit 12. The wearing tilt is, for example, the current tilt of the information processing device 10 from a standard wearing state of the information processing device 10. The wearing tilt may be estimated by the estimation unit 165 of the control unit 16. Then, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on the estimation result of the estimation unit 165.

[0112] Furthermore, the correction unit corrects at least one of the detectable range AR and the reference direction based on information about the posture of the user U estimated based on information from the state detection unit 12. The information about the posture of the user U is information indicating that the user is lying on his back, for example. The information about the posture of the user U may be information about the direction of gravity detected by an acceleration sensor or a gyro sensor. The posture of the user U may be estimated by the estimation unit 165 of the control unit 16. Then, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on the estimation result of the estimation unit 165.

[0113] This makes it possible to improve gesture detection accuracy while suppressing false detection.

[0114] Next, the gesture detection unit 162 of the control unit 16 determines whether a gesture has been detected (step S105). For example, after detecting a first action within the detection area OR, the gesture detection unit 162 determines whether a second action has been detected within the same detection area OR. If a gesture has not been detected (step S105: No), the gesture detection unit 162 returns the process to step S101.

[0115] Subsequently, when a gesture is detected (step S105: Yes), the gesture detection unit 162 determines whether the detected gesture is performed within the detectable range AR (step S106). For example, the gesture detection unit 162 determines whether a second motion is detected within the detectable range AR after a first motion is detected within the detectable range AR. If the second motion is a gesture involving a movement range such as a left or right swipe, it may be determined whether the gesture is performed within the detectable range AR based on whether both the start position and end position of the second motion are within the detectable range AR.

[0116] The detectable range AR for detecting the first movement and the detectable range AR for detecting the second movement may be different in size. For example, when the second movement is a gesture involving a movement range, the control unit 16 may set the detectable range for detecting the second movement to be wider than the detectable range AR when detecting the swing-up movement.

[0117] Furthermore, in order to make it easier for the user U to perform an input operation using a gesture, when the first movement direction is detected within the detectable range AR, the output control unit 164 may control the output unit 13 to notify the user that gesture detection is currently being accepted. At this time, the notification made by the output control unit 164 may be a sound.

[0118] If the gesture is not performed within the detectable range (step S106: No), the gesture detection unit 162 returns the process to step S101.

[0119] In addition, when the hand H of the user U enters the detectable range AR from the non-detectable range NR, the gesture detection unit 162 may not detect the gesture regardless of whether the second action is performed within the detectable range AR.

[0120] 19A and 19B are diagrams illustrating an example in which the gesture detection unit 162 does not detect a gesture. As shown in FIGS. 19A and 19B, the detectable range AR is biased within the detection area OR so that part of the edge of the detectable range AR contacts part of the edge of the detection area OR. More specifically, the detectable range AR is biased toward the front side of the user U. As shown in FIG. 19A, when the hand H of the user U enters the detectable range AR from the non-detectable range NR, it is assumed that the entry of the hand H of the user U into the detectable range AR is not due to a swing-up motion. Therefore, even if the second motion is performed within the detectable range AR, for example, as shown in FIG. 19B, the gesture detection unit 162 does not detect the gesture.

[0121] On the other hand, if the hand H of the user U enters the detectable range AR directly rather than from the non-detectable range NR, the gesture detection unit 162 detects the gesture on the condition that the second action is performed within the detectable range AR.

[0122] 20A and 20B are diagrams illustrating an example of a case where the gesture detection unit 162 detects a gesture. As shown in FIG. 20A, the user U moves his / her hand H directly into the detectable range AR, and then performs the second action as shown in FIG. 20B. In the example of FIG. 20B, the second action is a gesture involving a movement range, so the control unit 16 sets the detectable range for detecting the second action to be wider than the detectable range AR when the first action is detected (the detectable range AR shown in FIG. 20A). In the example of FIG. 20B, the second action is performed within the detectable range AR, so the gesture detection unit 162 detects the gesture.

[0123] In this way, when the hand H of the user U enters the detectable range AR from the detectable range NR, the gesture detection unit 162 does not detect the gesture, and when the hand H of the user U enters the detectable range AR directly, not from the non-detectable range NR, the gesture is detected, thereby further reducing the possibility of erroneous detection of the gesture.

[0124] 18, if the gesture is performed within the detectable range (step S106: Yes), the command execution unit 163 of the control unit 16 executes a command corresponding to the detected gesture (step S107). For example, the command execution unit 163 executes a function of the information processing device 10 associated with the detected gesture.

[0125] Next, the correction unit 166 executes a correction process related to gesture detection (step S108). For example, the correction unit 166 corrects at least one of the detectable range and the reference direction based on information about the gesture detected by the gesture detection unit 162. For example, the correction unit 166 widens or narrows the detectable range AR based on information about the start position and end position of the swipe. In addition, the correction unit 166 fine-tunes the reference direction based on the movement direction of the detected left / right swipe or up / down swipe.

[0126] When the correction process is completed, the control unit 16 returns the process to step S101 and executes the processes of steps S101 to S108 again.

[0127] According to this embodiment, the information processing device 10 corrects the detectable range, which reduces the number of cases where an action that the user does not intend to make as a gesture is detected as a gesture. In addition, the information processing device 10 corrects the reference direction, which reduces the number of cases where the user mistakes a gesture that he or she intends for another gesture.

[0128] <<3. Embodiment 2>> In the first embodiment, the information processing device 10 detects the movement of the user's hand H and executes the command. However, the device that detects the movement of the user's hand H and the device that executes the command may be different devices. The information processing system 1 of the second embodiment will be described below.

[0129] <3-1. Information Processing System Configuration> FIG. 21 is a diagram showing an example of the configuration of an information processing system 1. The information processing system 1 is a system that executes various functions based on a user's gestures. As shown in FIG. 21, the information processing system 1 includes an output device 20 and a terminal device 30. In the example of FIG. 21, the information processing system 1 includes an output device 20A and an output device 20B. Note that, in the example of FIG. 21, the output device 20 and the terminal device 30 are wirelessly connected, but the output device 20 and the terminal device 30 may be configured to be wired.

[0130] The information processing system 1 may include only one output device 20 or may include multiple output devices 20. For example, if the output device 20 is an earphone, the information processing system 1 may include a pair of output devices 20 that are wirelessly connected to the terminal device 30 and worn on the left and right ears of the user U. As an example, the output device 20A is an earphone worn on the left ear of the user U, and the output device 20B is an earphone worn on the right ear of the user U.

[0131] It should be noted that one output device 20 does not necessarily have to be one integrated device. It is also possible to regard multiple separate devices that are functionally or purposely related as one output device 20. For example, a pair of left and right earphones worn on the left and right ears of the user U may be regarded as one output device 20. Of course, one output device 20 may be one integrated earphone worn on one ear of the user U, or one integrated headphone worn on the left and right ears of the user U.

[0132] <3-2. Output device configuration> First, the configuration of the output device 20 will be described.

[0133] The output device 20 is an audio output device such as earphones or headphones that can be worn by a user. The information processing device 10 may also be a display device such as AR glasses or MR glasses that can be worn by a user. In this embodiment, the output device 20 is a device that can be worn by a user U and includes at least a portion that is located at the ear of the user U when worn. This portion is provided with at least a detection unit that detects an object. Here, taking the first embodiment as an example, the detection unit is a functional block that corresponds to the input detection unit 11. Note that if there are two portions located at the ears of the user U, one on each side, both portions may include a detection unit, or only one portion may include a detection unit.

[0134] 22 is a diagram illustrating an example configuration of an output device 20 according to an embodiment of the present disclosure. The output device 20 includes an input detection unit 21, a state detection unit 22, an output unit 23, a communication unit 24, and a control unit 26. Note that the configuration illustrated in FIG. 22 is a functional configuration, and the hardware configuration may be different from this. Furthermore, the functions of the output device 20 may be distributed and implemented in multiple physically separated configurations.

[0135] The input detection unit 21 is a detection unit that detects an input operation by the user. The state detection unit 22 is a sensor unit that detects the state of the output device 20. The output unit 23 is an output interface that outputs information to the user. The communication unit 24 is a communication interface for communicating with other devices such as the terminal device 30. The control unit 26 is a controller that controls each part of the output device 20.

[0136] The configurations of the input detection unit 21, the state detection unit 22, the output unit 23, the communication unit 24, and the control unit 26 are the same as the configurations of the input detection unit 11, the state detection unit 12, the output unit 13, the communication unit 14, and the control unit 16 provided in the information processing device 10. In this case, the description of the information processing device 10 may be replaced with the output device 20 as appropriate.

[0137] <3-3. Terminal Device Configuration> Next, the configuration of the terminal device 30 will be described.

[0138] The terminal device 30 is an information processing terminal capable of communicating with the output device 20. The terminal device 30 is a type of information processing device of this embodiment. The terminal device 30 is, for example, a mobile phone, a smart device (smartphone or tablet), a PDA (Personal Digital Assistant), or a personal computer. The terminal device 30 may be a device such as a commercial camera equipped with a communication function, or a mobile object equipped with a communication device such as an FPU (Field Pickup Unit). The terminal device 30 may be an M2M (Machine to Machine) device or an IoT (Internet of Things) device. The terminal device 30 controls the output device 20 from outside the output device 20 via a wired or wireless connection.

[0139] Fig. 23 is a diagram illustrating an example configuration of a terminal device 30 according to an embodiment of the present disclosure. The terminal device 30 includes an input unit 31, a state detection unit 32, an output unit 33, a communication unit 34, a storage unit 35, and a control unit 36. Note that the configuration illustrated in Fig. 23 is a functional configuration, and the hardware configuration may be different. Furthermore, the functions of the terminal device 30 may be distributed and implemented in multiple physically separated configurations.

[0140] The input unit 31 is an input interface that accepts input operations from the user. For example, the input unit 31 is a button or a touch panel. The state detection unit 32 is a sensor unit that detects the state of the terminal device 30. The output unit 33 is an output interface that outputs information to the user. The communication unit 34 is a communication interface for communicating with other devices such as the output device 20. The memory unit 35 is a storage device that can read and write data. The memory unit 35 stores, for example, information related to the detectable range AR of a gesture. The control unit 36 is a controller that controls each unit of the terminal device 30. The control unit 36 includes an acquisition unit 361, a gesture detection unit 362, a command execution unit 363, an output control unit 364, an estimation unit 365, and a correction unit 366.

[0141] The configurations of the input detection unit 21, the state detection unit 22, the output unit 23, the communication unit 24, and the control unit 26 are similar to the configurations of the input detection unit 11, the state detection unit 12, the output unit 13, the communication unit 14, and the control unit 16 provided in the information processing device 10. The configurations of the acquisition unit 361, the gesture detection unit 362, the command execution unit 363, the output control unit 364, the estimation unit 365, and the correction unit 366 are similar to the configurations of the acquisition unit 161, the gesture detection unit 162, the command execution unit 163, the output control unit 164, the estimation unit 165, and the correction unit 166 provided in the control unit 16 of the information processing device 10, except that the acquisition unit 361 acquires information from the input detection unit 21 and the state detection unit 22 via communication. In this case, the description of the information processing device 10 may be replaced with the output device 20 or the terminal device 30, as appropriate.

[0142] <3-4. Operation of the information processing system> The configuration of the information processing system 1 has been described above. Next, the operation of the information processing system 1 having such a configuration will be described.

[0143] (Command execution process) The information processing system 1 can execute command execution processing in the same manner as the information processing device 10 of embodiment 1. The command execution processing of the information processing system 1 is the same as the command execution processing of embodiment 1, except that the terminal device 30 acquires a sensor value from the output device 20. Hereinafter, as in embodiment 1, the command execution processing will be described with reference to FIG. 18.

[0144] 18 is a flowchart showing the command execution process according to this embodiment. The command execution process is executed by the control unit 36 of the terminal device 30. The command execution process is executed, for example, when the terminal device 30 establishes communication with the output device 20.

[0145] First, the acquisition unit 361 of the control unit 36 acquires sensor values from the input detection unit 21 and the state detection unit 22 of the output device 20 via the communication unit 34 (step S101). For example, the acquisition unit 361 acquires information about an object in the detection area OR from a non-contact sensor included in the output device 20. The acquisition unit 361 also acquires information about contact with the housing surface of the output device 20 from a contact sensor included in the output device 20. The acquisition unit 361 also acquires information about the state of the output device 20 from a sensor, such as an acceleration sensor, included in the output device 20.

[0146] Next, the acquisition unit 361 acquires information about the gesture of the user U based on the sensor value acquired in step S101 (step S102). For example, the acquisition unit 361 acquires information about the movement of the user U related to the gesture based on the sensor value from the input detection unit 21.

[0147] Next, the acquiring unit 361 acquires information about the state of the output device 20 based on the sensor value acquired in step S101 (step S103). For example, the acquiring unit 361 acquires information about the state of the output device 20 based on the sensor value from the state detecting unit 22.

[0148] Next, the correction unit 366 of the control unit 36 executes a correction process for gesture detection (step 104). For example, the correction unit 366 corrects at least one of the detectable range of a gesture and the reference direction for detecting the gesture.

[0149] Next, the gesture detection unit 362 of the control unit 36 determines whether a gesture has been detected (step S105). For example, after detecting a first action within the detection area OR, the gesture detection unit 362 determines whether a second action has been detected within the same detection area OR. If a gesture has not been detected (step S105: No), the gesture detection unit 362 returns the process to step S101.

[0150] Subsequently, if a gesture is detected (step S105: Yes), the gesture detection unit 362 determines whether the detected gesture is performed within the detectable range AR (step S106). For example, the gesture detection unit 362 determines whether a second action is detected within the detectable range AR after a first action is detected within the detectable range AR.

[0151] If the gesture is not performed within the detectable range (step S106: No), the gesture detection unit 362 returns the process to step S101.

[0152] On the other hand, if the gesture is performed within the detectable range (step S106: Yes), the command executing unit 363 of the control unit 16 executes a command corresponding to the detected gesture (step S107). For example, the command executing unit 363 executes a function of the output device 20 associated with the detected gesture.

[0153] Next, the correction unit 366 executes a correction process related to gesture detection (step S107). For example, the correction unit 366 corrects at least one of the detectable range and the reference direction based on information about the gesture detected by the gesture detection unit 362.

[0154] When the correction process is completed, the control unit 36 returns the process to step S101 and executes the processes of steps S101 to S108 again.

[0155] According to the present embodiment, the terminal device 30 corrects the detectable range, which reduces the possibility of detecting an action that the user does not intend as a gesture as a gesture. In addition, the terminal device 30 corrects the reference direction, which reduces the possibility of mistaking a gesture intended by the user U for another gesture.

[0156] (Manual setting of the detectable range) In the above-described embodiment, the terminal device 30 automatically sets the detectable range AR based on the sensor value. However, the terminal device 30 may be configured to allow the user U to manually set the detectable range AR.

[0157] 24 is a diagram showing how the user U manually sets the size of the detectable range AR. The terminal device 30 is configured so that the user U can change the setting of the size of the detectable range AR using a GUI (Graphical User Interface). Below, with reference to FIG. 24, a description will be given of the processing of the terminal device 30 that enables the manual setting of the size of the detectable range AR. The following processing is executed by the control unit 36 of the terminal device 30.

[0158] First, the acquisition unit 361 of the control unit 36 acquires information related to the detectable range AR from the storage unit 35. For example, information indicating the current setting of the detectable range AR is stored in the storage unit 35. The acquisition unit 361 acquires the information from, for example, the storage unit 35. Note that if information indicating the current setting of the detectable range AR is stored in another device (for example, the output device 20 or a device on the cloud), the acquisition unit 361 may acquire the information from the other device via the communication unit 34. Then, the output control unit 364 creates an image based on the acquired information so that the user can visualize the current setting of the detectable range AR, and displays the image on the output unit 33 (step S201).

[0159] In the example of FIG. 24, a part of the detection area OR is the detectable range AR. The acquisition unit 361 acquires operation information of the user U from the input unit 31 of the terminal device 30 (step S202). For example, the acquisition unit 361 acquires information of an operation by the user U to change the size of the detectable range AR from the input unit 31. In the example of FIG. 24, the terminal device 30 includes a touch panel display as the input unit 31, and the acquisition unit 361 acquires information of an operation by the user U to widen the detectable range AR. More specifically, in the example of FIG. 24, the acquisition unit 361 acquires information that the user U is pinching out his or her hand H on the detectable range AR displayed on the touch panel display.

[0160] The output control unit 364 creates an image of the detectable range AR related to the change operation of the user U based on the information acquired in step S202 so that the user U can see it, and displays it on the output unit 33 (step S203).

[0161] Then, when the control unit 36 detects an operation from the user U instructing a change in the detectable range AR (for example, pressing the OK button), the control unit 36 updates the setting of the detectable range AR stored in the memory unit 35 based on the information of the detectable range AR displayed in step S203.

[0162] The terminal device 30 may be configured so that the user U can manually change the position within the detection area OR of the detectable range AR.

[0163] 25 is a diagram showing how the user U manually sets the position of the detectable range AR. The terminal device 30 is configured so that the user U can change the setting of the position of the detectable range AR using a GUI. Below, with reference to FIG. 25, a description will be given of the processing of the terminal device 30 that enables the manual setting of the position of the detectable range AR. The following processing is executed by the control unit 36 of the terminal device 30.

[0164] First, the acquisition unit 361 of the control unit 36 acquires information about the detectable range AR from the storage unit 35. Then, the output control unit 364 creates an image of the current setting of the detectable range AR based on the acquired information so that the user can visualize it, and displays it on the output unit 33 (step S301).

[0165] In the example of FIG. 25, a part of the detection area OR is the detectable range AR. The acquisition unit 361 acquires operation information of the user U from the input unit 31 of the terminal device 30 (step S302). For example, the acquisition unit 361 acquires information of an operation by the user U to change the position of the detectable range AR from the input unit 31. In the example of FIG. 25, the terminal device 30 includes a touch panel display as the input unit 31, and the acquisition unit 361 acquires information of an operation by the user U to change the position of the detectable range AR. More specifically, in the example of FIG. 25, the acquisition unit 361 acquires information that the user U touches the detectable range AR displayed on the touch panel display and swipes their finger toward a position to which the user U wants to move the detectable range AR.

[0166] The output control unit 364 creates an image of the detectable range AR related to the change operation of the user U based on the information acquired in step S302 so that the user U can see it, and displays it on the output unit 33 (step S303).

[0167] Then, when the control unit 36 detects an operation from the user U instructing a change in the detectable range AR (for example, pressing the OK button), the control unit 36 updates the setting of the detectable range AR stored in the memory unit 35 based on the information of the detectable range AR displayed in step S303.

[0168] By allowing the user U to manually set the size and position of the detectable range AR, it becomes possible to set the detectable range AR to suit the individual characteristics of the user U. As a result, false gesture detection is reduced. Note that, in addition to the GUI (Graphical User Interface), the detectable range AR or non-detectable range NR may be adjusted in conjunction with, for example, a voice recognition function of the terminal device 30. For example, the detectable range AR may be changed in stages when the user's voice saying "widen the detectable range AR" is recognized.

[0169] (Manual setting of gesture assignment) In the above-described embodiment, the combinations of gestures and functions are predetermined. However, the terminal device 30 may be configured so that the user U can manually set the combinations of gestures and functions.

[0170] At this time, the terminal device 30 may be configured to store information on the operation history related to the gestures of the user U in the storage unit 35. Then, the terminal device 30 may tally up the gestures frequently used by the user U and their success rates based on the information on the operation history, and display the tally results so that the user U can visually recognize them. FIG. 26 is a diagram showing a display of the gestures frequently used by the user U and their success rates. In the example of FIG. 26, it can be seen that the user U frequently uses swipe motions, but the success rate of the swipe motions is low.

[0171] The terminal device 30 may be configured to allow the user U to change the association between a gesture and a function. For example, the terminal device 30 may have a GUI for changing the association between a gesture and a function. In the example of Fig. 26, the user U uses the GUI to change the association between a function (e.g., Next / Prev) associated with a swipe motion to a function associated with a pinch motion, for example.

[0172] The terminal device 30 may be configured to suggest the detectable range AR to the user based on information about the operation history. The terminal device 30 may also be configured to allow the user to change the detectable range AR based on the suggestion.

[0173] By allowing the user U to manually set the association between gestures and functions, it becomes possible to associate gestures and functions that suit the individuality of the user U. This allows the user U to associate gestures with a high success rate with frequently used functions, thereby reducing false detection of gestures.

[0174] <<4. Embodiment 3>> In the first and second embodiments, a non-contact sensor is mainly used to determine a user's operation on a device. In the third embodiment, a contact sensor and a non-contact sensor are combined to determine a user's operation on a device. An information processing device 40 of the third embodiment will be described below.

[0175] <4-1. Overview of the Third Embodiment> When detecting gestures using a non-contact sensor, the detectable range of the gesture changes depending on the orientation and posture of the user's face. As a result, the detection accuracy of gesture detection using a non-contact sensor is low. This is because, although the position of the gesture is absolutely determined by the placement of the sensor, the user perceives the gesture position relatively, regardless of their posture. To solve this problem, it is possible to widen the detectable range of the gesture. However, if the detectable range of the gesture is broadened unnecessarily, there is a possibility that the information processing device may mistakenly recognize other confusing actions as gestures. Therefore, it is desirable to limit the detectable range as much as possible.

[0176] Device operation using gestures (air operation) places a small burden on the user. However, gestures vary greatly from person to person (i.e., gesture positions are not reproducible), resulting in low accuracy in device operation. On the other hand, device operation using contact with the device or human body (physical operation) results in high accuracy in device operation. However, with physical operation, the operation surface is limited, resulting in a narrow operation area and a limited number of commands that can be input.

[0177] Therefore, in embodiment 3, a contact sensor and a non-contact sensor are combined. More specifically, the information processing device 40 of embodiment 3 performs correction regarding gesture detection or correction regarding device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor.

[0178] For example, the information processing device 40 detects a user's gesture using sensor information from at least a contact sensor, and estimates the gesture position based on sensor information from a non-contact sensor.The information processing device 40 then corrects the gesture detectable range based on the estimated gesture position information.This allows the gesture detectable range to be updated as needed in accordance with the user's usage situation, thereby improving gesture detection accuracy.

[0179] Furthermore, the information processing device 40 may be configured to detect a user's gesture using sensor information from at least a non-contact sensor. The information processing device 40 may then correct the detectable range of the gesture based on the sensor information from the non-contact sensor when the user touches the face or the information processing device 40. This allows the detectable range of the gesture to be corrected based on the position of physical contact, thereby improving the accuracy of gesture detection.

[0180] The information processing device 40 may be configured to detect a user's gesture using sensor information from at least a contact sensor. The information processing device 40 may then be configured to execute a command corresponding to the detected gesture. The command may include at least a command accompanied by an amount of operation. In this case, the information processing device 40 may determine the amount of operation based on sensor information from a non-contact sensor. This allows the information processing device 40 to achieve highly accurate gesture detection using a contact sensor while also determining the amount of operation using a non-contact sensor.

[0181] <4-2. Configuration of information processing device> The outline of the third embodiment has been described above. Next, the configuration of the information processing device 40 will be described.

[0182] FIG. 27 is a diagram showing an example of the configuration of an information processing device 40 according to the third embodiment. The information processing device 40 includes an input detection unit 41, a state detection unit 42, an output unit 43, a communication unit 44, a storage unit 45, and a control unit 46. Note that the configuration shown in FIG. 27 is a functional configuration, and the hardware configuration may be different. Furthermore, the functions of the information processing device 40 may be distributed and implemented in multiple physically separated configurations. For example, the functions of the information processing device 40 may be distributed and implemented in the output device 20 and the terminal device 30, as shown in the second embodiment.

[0183] The input detection unit 41 is a detection unit that detects an input operation by a user. The input detection unit 11 includes a non-contact type detection unit (hereinafter referred to as a non-contact sensor) that detects an object without contact, and a contact type detection unit (hereinafter referred to as a contact sensor) that detects contact of the user with the human body or an object.

[0184] The non-contact sensor may be composed of one or more proximity sensors that detect an object located within a detection area. The proximity sensor may be an optical sensor or a capacitance sensor. An optical proximity sensor is a sensor that detects light reflected by an object. A capacitance proximity sensor is a sensor that detects changes in capacitance that occur between the object and the sensor. The non-contact sensor may be a 2D sensor or a 3D sensor.

[0185] The non-contact sensor includes one or more touch sensors, and detects an object that comes into contact with a predetermined location on the information processing device 10. If the information processing device 40 is an earphone-type device, the predetermined location is, for example, the side of the earphone (the side opposite the speaker surface). If the information processing device 10 is a headphone-type device, the predetermined location is, for example, the side of the headphone (the side opposite the speaker surface). In the following description, the earphone side or headphone side may be referred to as the housing surface.

[0186] The input detection unit 11 also includes a touchpad TP as a contact sensor. The touchpad TP is composed of multiple touch sensors arranged on a surface. The infrared light emitter ID is arranged on the surface of the housing. The input detection unit 11 detects an object that comes into contact with the surface of the touchpad TP.

[0187] It should be noted that the non-contact sensor is not limited to a sensor that detects contact with a device. For example, the non-contact sensor may be a sensor that detects contact with the human body of a user wearing the information processing device 40. For example, if the information processing device 40 is an earphone-type or headphone-type device, the non-contact sensor may be a sensor that detects contact with the face of a user wearing the information processing device 40. In this case, the non-contact sensor may be configured to detect contact with the user's face by detecting vibrations when the user touches their hand to their face, for example.

[0188] The state detection unit 42 is a sensor unit that detects the state of the information processing device 40. The state detection unit 42 may be configured to detect the state of the user.

[0189] As shown in FIG. 27, the control unit 46 includes a first acquisition unit 461A, a second acquisition unit 461B, a gesture detection unit 462, a command execution unit 463, an output control unit 164, an estimation unit 165, and a correction unit 166. Each block (acquisition unit 161 to correction unit 166) constituting the control unit 46 is a functional block indicating a function of the control unit 46. These functional blocks may be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks may be a software module realized by software (including a microprogram), or may be a circuit block on a semiconductor chip (die). Of course, each functional block may be a processor or an integrated circuit. The functional blocks may be configured in any manner.

[0190] The control unit 16 may be configured with functional units different from the functional blocks described above. Also, some or all of the operations of the blocks (first acquisition unit 461A to correction unit 466) constituting the control unit 16 may be performed by another device. For example, some or all of the operations of the blocks constituting the control unit 46 may be performed by a terminal device such as a music player or a smartphone. The operations of the blocks constituting the control unit 46 will be described later.

[0191] In addition, the configurations of the input detection unit 41, the state detection unit 42, the output unit 43, the communication unit 44, and the control unit 46 are the same as the configurations of the input detection unit 11, the state detection unit 12, the output unit 13, the communication unit 14, and the control unit 16 provided in the information processing device 10.

[0192] <4-3. Operation of information processing device> The configuration of the information processing device 40 has been described above. Next, the operation of the information processing device 40 having such a configuration will be described.

[0193] As described above, in the third embodiment, a contact sensor and a non-contact sensor are combined. More specifically, the information processing device 40 performs correction regarding gesture detection or correction regarding device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor. Some specific examples are shown below.

[0194] <4-3-1. Example 1> First, Example 1 will be described.

[0195] (Contents of Example 1) In Example 1, a gesture is assumed to be made by physical contact with the user's face or a device. FIG. 28 is a diagram showing a user making a gesture by physical contact. In the example of FIG. 28, the detectable range A1 of the contact sensor is set to a part of the user's left cheek, and the detectable range A2 of the non-contact sensor is set to a part of the user's left cheek including the left cheek and its vicinity. In the example of FIG. 28, a part of the detectable range A1 and a part of the detectable range A2 overlap. In the example of FIG. 28, the user operates the device by tapping on the cheek.

[0196] In existing examples, a single operation area is set for contact, so the same gesture is recognized regardless of where the user taps on the face or device. This may limit the number of commands. Therefore, in Example 1, the information processing device 40 estimates the operation area (gesture position) at the time of contact based on sensor information from a non-contact sensor. This enables the information processing device 40 to determine in which area within the detectable range A1 the user performed the gesture.

[0197] Note that the size and position of the operation area vary from user to user. Therefore, the information processing device 40 corrects the detectable range of the gesture based on information about the estimated gesture position. For example, when the operation area (gesture position) covers a range equal to or greater than a predetermined percentage of the current detectable range A1, the information processing device 40 increases the size of the detectable range A1. Furthermore, when the operation area (gesture position) is located at the edge of the current detectable range A1, the information processing device 40 corrects the position of the detectable range A1 so that the operation area is located in the center. This allows the detectable range A1 of the gesture to be updated as needed in accordance with the user's usage situation, thereby improving the accuracy of gesture detection.

[0198] Note that a user's actions constituting one gesture may include a first action that triggers the start of the gesture and a second action that follows the first action. In this case, the information processing device 40 may narrow the detectable range of the second action based on information about the operation area (gesture position) of the first action. This makes it possible to prevent erroneous detection of gestures due to everyday actions.

[0199] The information processing device 40 executes a command corresponding to the detected gesture. At this time, the information processing device 40 may change the command to be executed if the gesture detected based on the sensor information from the contact sensor is the same gesture but the gesture position is different. That is, the information processing device 40 may determine the command to be executed based on the gesture detected based on the sensor information from the contact sensor and the information on the gesture position detected based on the sensor information from the non-contact sensor. This allows the information processing device 40 to assign many commands to one gesture while realizing highly accurate gesture detection based on physical contact.

[0200] Note that, when a part of the detectable range A1 and a part of the detectable range A2 overlap as shown in FIG. 28, the information processing device 40 can also detect a gesture made by physical contact using only the non-contact sensor.

[0201] (Processing example) 29 is a flowchart showing an example of a command execution process according to embodiment 3. The command execution process is executed by the control unit 46 of the information processing device 40. The command execution process is executed, for example, when the information processing device 40 is powered on.

[0202] First, the second acquisition unit 461B of the information processing device 40 acquires the sensor value of the non-contact sensor (step S201). Then, the estimation unit 465 of the information processing device 40 estimates the gesture position based on the sensor value of the non-contact sensor (step S202). Furthermore, the first acquisition unit 461A of the information processing device 40 acquires the sensor value of the contact sensor (step S203).

[0203] Next, the gesture detection unit 462 of the information processing device 40 determines whether a gesture has been detected based on the sensor value of the contact sensor (step S204). If a gesture has not been detected (step S204: No), the gesture detection unit 462 returns the process to step S201.

[0204] On the other hand, if a gesture is detected (step S204: Yes), the estimation unit 465 of the information processing device 40 estimates the device state (step S205). Then, the gesture detection unit 462 of the information processing device 40 determines the gesture based on the estimation result of the device state and the sensor value of the contact sensor (step S206). Then, the correction unit 466 of the information processing device 40 corrects the detectable range of the gesture (detectable range A1 in the example of FIG. 28) (step S207). Thereafter, the command execution unit 463 executes the command assigned to the determined gesture (step S208). After executing the command, the command execution unit 463 returns the process to step S201.

[0205] <4-3-2. Example 2> Next, Example 2 will be described.

[0206] (Contents of Example 2) In Example 2, a non-contact gesture by the user is assumed. FIG. 30 is a diagram showing a state in which a user is making a gesture by an air movement. In the example of FIG. 30, the detectable range A1 of the contact sensor is set to the information processing device 40 and its surroundings, and the detectable range A2 of the non-contact sensor is set to a position spaced apart to the left of the user's head. In the example of FIG. 30, the detectable range A1 and the detectable range A2 are spaced apart and do not overlap. In the example of FIG. 30, the user operates a device by making a predetermined air movement with the left side of his or her head.

[0207] When a non-contact sensor is mainly used to detect gestures, the gesture position may vary significantly depending on the user's state. Therefore, if non-contact gesture detection is not successful due to the variation in gesture position, the information processing device 40 prompts the user to touch a human body, an object, or an arbitrary spatial point. For example, the information processing device 40 prompts the user to tap on their face or physically touch the information processing device 40.

[0208] The information processing device 40 then calibrates the gesture detectable range A2 based on the user's physical touch action. For example, the information processing device 40 corrects the gesture detectable range A2 based on sensor information from a non-contact sensor when the user touches a human body, an object, or an arbitrary spatial point. More specifically, the information processing device 40 detects the user's touch with the human body, an object, or an arbitrary spatial point based on sensor information from a contact sensor, and corrects the gesture detectable range A2 based on sensor information from the non-contact sensor when the user's touch with the human body or the object is detected. This corrects the gesture detectable range A2 based on the position of physical contact, thereby improving the gesture detection accuracy.

[0209] After detecting contact of the user with the human body or an object, the information processing device 40 may provide feedback to the user by sound or the like when the user's hand is located within the corrected detectable range A2. Furthermore, if a gesture cannot be detected by the non-contact sensor due to the user's posture or context (for example, walking or running), the information processing device 40 may provide feedback to encourage a physical operation when it recognizes the user's hand within the detectable range A2 of the non-contact sensor.

[0210] (Processing example) 31 is a flowchart showing another example of the command execution process according to embodiment 3. The command execution process is executed by the control unit 46 of the information processing device 40. The command execution process is executed, for example, when the information processing device 40 is powered on.

[0211] First, the second acquiring unit 461B of the information processing device 40 acquires the sensor value of the non-contact sensor (step S301). Then, the gesture detecting unit 462 of the information processing device 40 determines whether a gesture has been detected based on the sensor value of the non-contact sensor (step S302).

[0212] If a gesture has not been detected (step S302: No), the first acquisition unit 461A of the information processing device 40 acquires a sensor value of the contact sensor (step S303). Then, the correction unit 466 of the information processing device 40 executes an operation related to calibration of the gesture detectable range A2 (step S304). For example, the correction unit 466 prompts the user to tap on their face or physically touch the information processing device 40. The correction unit 466 corrects the gesture detectable range A2 based on sensor information from the non-contact sensor when the user touches a human body or an object (step S305). When the correction is completed, the correction unit 466 returns the process to step S301.

[0213] On the other hand, if a gesture is detected (step S302: Yes), the estimation unit 465 of the information processing device 40 estimates the device state (step S306). Then, the gesture detection unit 462 of the information processing device 40 determines the gesture based on the estimation result of the device state and the sensor value of the contact sensor (step S307). Then, the command execution unit 463 executes the command assigned to the determined gesture (step S308). After executing the command, the command execution unit 463 returns the process to step S301.

[0214] <4-3-3. Example 3> Next, Example 3 will be described.

[0215] (Contents of Example 3) Some commands are accompanied by an operation amount. FIG. 32 is a diagram showing how a command accompanied by an operation amount is input. For example, if the command is a volume adjustment command (Vol+ / Vol-), the operation amount is the volume. In the case of a volume adjustment command, the direction of movement of the user's hand corresponds to the direction of change in the volume (whether to increase or decrease the volume), and the amount or speed of movement of the user's hand corresponds to the amount of change in the sound (input value).

[0216] In Example 3, a contact sensor and a non-contact sensor are combined to enable input of not only commands but also operation amounts (direction and amount of change). For example, the information processing device 40 detects the user's gesture using sensor information from at least the contact sensor. Then, the information processing device 40 determines the operation amount based on sensor information from the non-contact sensor. This allows the information processing device 40 to realize highly accurate gesture detection using the contact sensor, while also determining the operation amount using the non-contact sensor.

[0217] Note that a user's actions constituting one gesture may include a first action that triggers the start of the gesture and a second action that follows the first action. If the command is a volume adjustment command, the first action may be, for example, a face tap, and the second action may be, for example, a hand movement in a predetermined direction (for example, forward or backward) from the face tap. In this case, the information processing device 40 may determine the volume to be changed based on the movement direction, movement amount, and / or movement speed of the second action.

[0218] Note that the size and speed of gestures vary from person to person. Therefore, in the case of a command that requires an amount of operation, the amount of operation input may be inaccurate. Therefore, the information processing device 40 corrects the second action based on information about the speed and size of the first action (for example, the speed of the hand before tapping the face or the strength of the face tap).

[0219] For example, the information processing device 40 determines the speed of the first action based on information from a non-contact sensor. Then, the information processing device 40 corrects a setting value (e.g., a threshold value) for associating the second action with the amount of operation based on the information about the speed of the first action. Thereafter, the information processing device 40 determines the speed or movement amount of the second action based on sensor information from the non-contact sensor. Then, the information processing device 40 determines the amount of operation based on the corrected setting value and the information about the speed or movement amount of the second action. This allows the information processing device 40 to improve the accuracy of input of the amount of operation by the user.

[0220] A user who inputs gestures quickly may move their hand far away, so it is desirable that the range in which gestures can be detected be wide. On the other hand, a user who inputs gestures slowly may be less likely to move their hand far away, so it is desirable that the range in which gestures can be detected be narrower, taking into account the risk of false detection. Therefore, the information processing device 40 corrects the detection of the second movement based on information about the user's first movement.

[0221] For example, the information processing device 40 determines the speed of the first movement based on sensor information from a non-contact sensor. Then, the information processing device 40 corrects the detectable range of the second movement based on the information about the speed of the first movement. This makes it possible to improve the detection accuracy of the second movement.

[0222] <<5. Modifications>> The above-described embodiments are merely examples, and various modifications and applications are possible.

[0223] (Modifications regarding gesture detection) For example, the information processing device 10 of the first embodiment may change the detectable range AR according to the distance to an obstacle. Fig. 33 is a diagram showing how the detectable range AR is changed according to the distance to an obstacle. For example, the information processing device 10 may change the detectable range AR according to the distance to an obstacle by using a pair of input detection units 11. R , 11 L As shown in FIG. 33, the input detection unit 11 R An obstacle O1 is located to the right (negative Y-axis direction) of the input detector 11. L An obstacle O2 is located to the left (positive direction of the Y axis) of the obstacle O2. The obstacle may be a structure such as a wall, or may be an object independent of the structure.

[0224] Input detection unit 11 R and input detection unit 11 L detect the distance to each obstacle. In the example of FIG. 33, the distance to obstacle O1 is d1, and the distance to obstacle O2 is d2. Then, based on this information, the information processing device 10 adjusts the detectable range AR so that no obstacles fall within the detectable range AR. In the example of FIG. 33, the information processing device 10 adjusts the detectable range AR by detecting the input of the input detection unit 11. R Detection range AR R is corrected so that the maximum distance in the separation direction (negative Y-axis direction) is d3, which is shorter than d1. L is corrected so that the maximum distance in the separation direction (positive Y-axis direction) is d4, which is shorter than d2.

[0225] The information processing device 10 determines the detectable range AR so that d3 and d4 are the same distance. R and detectable range AR L The detectable range AR may be corrected so that d3 and d4 are different distances. R and detectable range AR L The information processing device 10 may also correct the detectable range AR R and detectable range AR L It is also possible to correct only one of the above. If the positions of the obstacle and / or the user U are known in advance, the information processing device 10 may correct the detectable range AR based on the position information.

[0226] This reduces the number of situations where the information processing device 10 is unable to detect a gesture due to an obstacle, thereby improving gesture detection accuracy. This modification is also applicable to the information processing system 1 of the second embodiment.

[0227] (Modifications regarding the execution of functions) The information processing device 10 of the first embodiment may include a pair of predetermined parts that are located on both ears of the user U when worn. In this case, the predetermined parts may be parts from which sound is output from earphones or headphones. When a swing-up motion of the user U is detected at both of the pair of predetermined parts, the information processing device 10 may not execute a function associated with the gesture even if a gesture is detected.

[0228] Similarly, the output device 20 of the second embodiment may include a pair of predetermined parts that are located on each ear of the user U when worn. In this case, the predetermined parts may be parts from which sound is output from earphones or headphones. When a swing-up motion of the user U is detected at both of the pair of predetermined parts, the terminal device 30 of the second embodiment may not execute a function associated with the gesture even if a gesture is detected.

[0229] When a swing-up motion is detected on both the left and right sides at the same time, it is assumed that the user U is not making a gesture but is removing a device such as earphones or headphones. By not executing a function in such a case, the information processing device 10 or the terminal device 30 can reduce actions that are not in line with the user's intention.

[0230] (Modifications regarding the amount of operation) The functions of the information processing device 10 of the first embodiment may include a predetermined function with an operation amount. In this case, the predetermined function with an operation amount may be a function related to volume operation (Vol+ / Vol-) or a function related to playback speed. Furthermore, the predetermined function with an operation amount is not limited to these, and may be, for example, a function related to fast forward, rewind, or slow playback. A gesture with a movement amount, such as a swipe, may be associated with the predetermined function. Then, when the gesture detected by the gesture detection unit 162 is a gesture with a movement amount, the information processing device 10 may determine the operation amount of the predetermined function (for example, the volume of the sound to be raised or lowered) based on the relative movement amount of the gesture with respect to the size of the detectable range.

[0231] Similarly, the functions of the terminal device 30 of the second embodiment may include a predetermined function with an operation amount. The predetermined function may be associated with a gesture with a movement range, such as a swipe. When the gesture detected by the gesture detection unit 362 is a gesture with a movement range, the terminal device 30 may determine the operation amount of the predetermined function (for example, the volume of the sound to be raised or lowered) based on the relative movement range of the gesture with respect to the size of the detectable range.

[0232] This allows the amount of operation to be input with one gesture, thereby improving the usability of the information processing device 10 or the terminal device 30.

[0233] (Modifications regarding device aspects) In the above-described embodiment, the information processing device 10 and the output device 20 are devices that can be worn by a user (wearable devices), but they do not necessarily have to be wearable devices. For example, the information processing device 10 and the output device 20 may be devices that are installed and used in a structure or a mobile object, such as a television, a car navigation system, a driver's seat, or various operation panels. Furthermore, the information processing device 10 and the output device 20 may be the mobile object itself.

[0234] Here, the mobile object may be a mobile terminal such as a smartphone, a mobile phone, a personal computer, a music player, or a portable television, or may be a remote controller for operating a device. The mobile object may be a mobile object that moves on land (e.g., a vehicle such as an automobile, bicycle, bus, truck, motorcycle, train, or linear motor car), or a mobile object that moves underground (e.g., in a tunnel) (e.g., a subway). The mobile object may be a mobile object that moves on water (e.g., a ship such as a passenger ship, cargo ship, or hovercraft), or a mobile object that moves underwater (e.g., a submersible vessel such as a submarine, submarine, or unmanned underwater vehicle). The mobile object may be a mobile object that moves within the atmosphere (e.g., an aircraft such as an airplane, airship, or drone), or a mobile object that moves outside the atmosphere (e.g., an artificial celestial body such as a space station).

[0235] Furthermore, examples of structures include buildings such as high-rise buildings, houses, steel towers, station facilities, airport facilities, port facilities, stadiums, etc. The concept of a structure includes not only buildings, but also non-building structures such as tunnels, bridges, dams, fences, and steel pillars, as well as equipment such as cranes, gates, and wind turbines. The concept of a structure also includes not only land-based or underground structures, but also structures on water such as piers and megafloats, and underwater structures such as oceanographic observation facilities.

[0236] (Variations regarding the sensors used) In the third embodiment, the information processing device 40 detects a user's gesture using both a non-contact sensor and a contact sensor, but it is not necessary to continue using both the non-contact sensor and the contact sensor. For example, the information processing device 40 may decide whether to continue using both the non-contact sensor and the contact sensor or to use either the non-contact sensor or the contact sensor depending on the remaining battery level. Furthermore, the information processing device 40 may change the settings related to the sensor to be used when there is a special environmental change, such as when the user is wearing a hat.

[0237] (Modifications regarding correction) In the third embodiment, the information processing device 40 corrects the detectable range of the sensor based on the sensor value, but the detectable range may be corrected using information other than the sensor value. For example, the information processing device 40 may correct the detectable range depending on the type of device of the information processing device 40 (headphones / earphones / head-mounted display, etc.). For example, if the type of device of the information processing device 40 is headphones, there is ample space in the area where the user can physically touch, so the correction of the detectable range A1 is made smaller. Furthermore, the user's state may be estimated using another sensor (for example, a biosensor such as a heart rate sensor), and the detectable range may be corrected based on the estimation result.

[0238] (Detectable range can be set by the user) In the third embodiment, the information processing device 40 corrects the detectable range of the sensor based on the sensor value. However, the information processing device 40 may be configured to allow a user to set the detectable range. FIGS. 34A and 34B are diagrams illustrating an example of a graphical user interface (GUI) for setting the detectable range. The GUI illustrated in FIG. 34A is a GUI for setting the detectable range in a planar direction (front-back, left-right, and right-side directions toward the user's ear E), and the GUI illustrated in FIG. 34B is a GUI for setting the detectable range in a depth direction (direction away from the user's ear E). Region A3 corresponds, for example, to the detectable range A1 of a contact sensor, and region A4 corresponds, for example, to the detectable range A2 of a non-contact sensor. Alternatively, region A3 corresponds, for example, to the detectable range A2 of a non-contact sensor, and region A4 corresponds, for example, to the detection range OR of a non-contact sensor. The user sets the detectable range of the sensor using the GUI illustrated in FIGS. 34A and 34B. The information processing device 40 may visualize the difference in the operation area between headphones and earphones. Furthermore, the information processing device 40 may provide a tutorial on the GUI when switching devices, for example.

[0239] (Other variations) The control device that controls the information processing device 10, the output device 20, or the terminal device 30 of this embodiment may be realized by a dedicated computer system or a general-purpose computer system.

[0240] For example, a communication program for executing the above-described operations is stored in a computer-readable recording medium such as an optical disk, a semiconductor memory, a magnetic tape, or a flexible disk and distributed. Then, for example, the program is installed in a computer and the above-described processing is executed to configure a control device. In this case, the control device may be a device (e.g., a personal computer) external to the information processing device 10, the output device 20, or the terminal device 30. Alternatively, the control device may be a device (e.g., the control unit 16, the control unit 26, or the control unit 36) internal to the information processing device 10, the output device 20, or the terminal device 30.

[0241] The communication program may also be stored in a disk device provided in a server device on a network such as the Internet, and may be downloaded to a computer. The above-mentioned functions may also be realized by cooperation between an OS (Operating System) and application software. In this case, the parts other than the OS may be stored on a medium and distributed, or may be stored in a server device and downloaded to a computer.

[0242] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0243] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0244] The above-described embodiments can be combined as appropriate within the scope of the present invention without causing any inconsistency in the processing content. The order of the steps shown in the flowcharts of the above-described embodiments can be changed as appropriate.

[0245] Furthermore, for example, this embodiment can also be implemented as any configuration that constitutes an apparatus or system, such as a processor as a system LSI (Large Scale Integration), a module using multiple processors, a unit using multiple modules, a set in which other functions are added to a unit, etc. (i.e., a configuration of a part of an apparatus).

[0246] In this embodiment, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0247] Furthermore, for example, this embodiment can be configured as a cloud computing system in which one function is shared and processed jointly by a plurality of devices via a network.

[0248] <<6. Conclusion>> As described above, according to an embodiment of the present disclosure, the information processing device 10 and the terminal device 30 correct the detectable range of a gesture based on information related to the gesture and information related to the device state. Therefore, the information processing device 10 and the terminal device 30 can reduce the number of cases where an action that the user does not intend as a gesture is detected as a gesture.

[0249] According to an embodiment of the present disclosure, the information processing device 10 and the terminal device 30 correct the reference direction for detecting a gesture based on information related to the gesture and information related to the device state. As a result, the information processing device 10 and the terminal device 30 can reduce erroneous detection, such as detecting a left / right swipe as an up / down swipe.

[0250] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0251] Furthermore, the effects of each embodiment described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0252] The present technology can also be configured as follows. (1) a gesture detection unit that detects a user's gesture made in a detection area of the detection unit that detects an object; a correction unit that corrects at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture, based on at least one of information regarding the gesture and information regarding a state of a predetermined device that includes the detection unit; and An information processing device comprising: (2) the correction unit corrects at least one of the detectable range and the reference direction based on information about the user's movement related to the gesture. The information processing device according to (1) above. (3) The information about the user's movement related to the gesture includes information about a first movement of the user's hand entering the detectable range, the correction unit corrects at least one of the detectable range and the reference direction based on information about the first operation. The information processing device according to (2) above. (4) the correction unit corrects the detectable range to be at least narrower than the detection area based on information about the first operation. The information processing device according to (3) above. (5) the user's actions constituting one gesture include the first action and a second action subsequent to the first action; After the first motion is detected in the detectable range, the correction unit widens the detectable range for detecting the second motion to be larger than the detectable range when the first motion is detected. The information processing device according to (4) above. (6) When the gesture to be detected is a gesture involving a movement range, the correction unit widens the detectable range for detecting the second movement compared to the detectable range when the first movement is detected. The information processing device according to (5) above. (7) the information about the first movement includes information indicating an approach direction of the hand entering the detectable range, the correction unit corrects at least one of the detectable range and the reference direction based on information indicating the hand entry direction. The information processing device according to any one of (3) to (6). (8) the predetermined device is wearable by the user, the correction unit corrects at least one of the detectable range and the reference direction based on information regarding a state in which the predetermined device is attached to the user, the state being estimated based on information indicating the hand approach direction. The information processing device according to (7) above. (9) the correction unit corrects at least one of the detectable range and the reference direction based on information about a wearing inclination of the predetermined device estimated based on information indicating the approach direction of the hand. The information processing device according to (8). (10) the predetermined device has a predetermined portion that is positioned in the ear of the user when worn; The detection unit is disposed at the predetermined location, the correction unit corrects at least one of the detectable range and the reference direction based on information about a swing-up motion in which the user swings their hand up toward their ear. The information processing device according to (8) or (9). (11) an execution unit that executes a function associated with the gesture when the gesture is detected; the predetermined device has a pair of predetermined portions that are positioned on both ears of the user when worn, When the swing-up motion is detected at both of the pair of predetermined parts, the execution unit does not execute the function associated with the gesture even if the gesture is detected. The information processing device according to (10) above. (12) the detectable range is disposed within the detection area such that a part of an edge of the detectable range contacts a part of an edge of the detection area, the gesture detection unit does not detect the gesture when the user's hand enters the detection area from a range other than the detectable range. The information processing device according to any one of (3) to (11) above. (13) an execution unit that executes a function associated with the gesture when the gesture is detected; The functions include at least a predetermined function involving an operation amount, a predetermined gesture having a movement range is associated with the predetermined function; When the gesture detected by the gesture detection unit is the predetermined gesture, the execution unit determines the amount of operation based on a magnitude of a movement width of the predetermined gesture relative to the detectable range. The information processing device according to any one of (1) to (12). (14) the correction unit corrects at least one of the detectable range and the reference direction based on information from a state detection unit that detects a state of the predetermined device. The information processing device according to any one of (1) to (13). (15) the predetermined device is wearable by the user, the correction unit corrects at least one of the detectable range and the reference direction based on information regarding a state of attachment of the predetermined device to the user estimated based on information from the state detection unit. The information processing device according to (14) above. (16) the predetermined device is wearable by the user, the correction unit corrects at least one of the detectable range and the reference direction based on information about the user's posture estimated based on information from the state detection unit. The information processing device according to (14) or (15). (17) the state detection unit includes at least one type of sensor selected from the group consisting of an acceleration sensor, a gyro sensor, and a biosensor; the correction unit corrects at least one of the detectable range and the reference direction based on information from one or more sensors included in the state detection unit. The information processing device according to any one of (14) to (16) above. (18) The predetermined device is a headphone or an earphone. The information processing device according to any one of (1) to (17). (19) The information processing device is the predetermined device or a device that controls the predetermined device from outside the predetermined device via a wired or wireless connection. The information processing device according to any one of (1) to (18). (20) Detecting a user gesture made in a detection area of a detection unit that detects an object; correcting at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture based on at least one of information regarding the gesture and information regarding a state of a predetermined device including the detection unit; Information processing methods. (twenty one) a first acquisition unit that acquires sensor information from a contact sensor that detects contact of a user with a human body or an object; a second acquisition unit that acquires sensor information from a non-contact sensor that detects the user's movement including the user's non-contact movement; a gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; a correction unit that performs correction regarding the detection of the gesture or correction regarding the device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor; An information processing device comprising: (twenty two) the gesture detection unit detects the gesture of the user using sensor information from at least the contact sensor; the correction unit corrects the detectable range of the gesture based on information about a gesture position indicating a position where the user performed the gesture and detected based on sensor information from the non-contact sensor. The information processing device according to (21) above. (twenty three) an execution unit that executes a command corresponding to the detected gesture, the execution unit determines a command to be executed based on the gesture detected based on sensor information from the contact sensor and information on a position of the gesture detected based on sensor information from the non-contact sensor. The information processing device according to (22). (twenty four) the gesture detection unit detects the gesture of the user using sensor information from at least the non-contact sensor; the correction unit corrects the detectable range of the gesture based on sensor information from the non-contact sensor when the user touches the human body or the object. The information processing device according to (21) above. (twenty five) the correction unit detects contact of the user with the human body or the object based on sensor information from the contact sensor, and corrects a detectable range of the gesture based on sensor information from the non-contact sensor when contact of the user with the human body or the object is detected. The information processing device according to (24). (26) an output control unit that, after detecting contact of the user with the human body or the object, provides feedback to the user when the user's hand is located within the corrected detectable range; The information processing device according to (25) above. (27) a command execution unit that executes a command corresponding to the detected gesture, The command includes at least a command accompanied by an operation amount, the gesture detection unit detects the gesture of the user using sensor information from at least the contact sensor; the command execution unit determines the operation amount based on sensor information from the non-contact sensor; The information processing device according to (21) above. (28) The user's actions constituting one gesture include a first action that triggers the start of the gesture and a second action that follows the first action; the command execution unit determines the amount of operation based on a speed or a movement amount of the second action determined based on sensor information from the non-contact sensor; the correction unit changes a set value for associating the second movement with the operation amount based on information about a speed of the first movement detected based on sensor information from the non-contact sensor. The information processing device according to (27) above. (29) the correction unit corrects a detectable range of the second movement based on information about a speed of the first movement detected based on sensor information from the non-contact sensor. The information processing device according to (28). (30) acquiring sensor information from a contact sensor that detects contact with the user's human body or an object; acquiring sensor information from a non-contact sensor that detects the user's movement, including the user's non-contact movement; detecting a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; performing a correction regarding the detection of the gesture or a correction regarding the device operation based on the sensor information from the contact sensor and the sensor information from the non-contact sensor; Information processing methods. (31) Computer, a gesture detection unit that detects a user's gesture performed in a detection area of the detection unit that detects an object; a correction unit that corrects at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture, based on at least one of information regarding the gesture and information regarding a state of a predetermined device that includes the detection unit; An information processing program that functions as a (32) Computer, a first acquisition unit that acquires sensor information from a contact sensor that detects contact of a user with a human body or an object; a second acquisition unit that acquires sensor information from a non-contact sensor that detects the user's movement including the user's non-contact movement; a gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; a correction unit that performs correction regarding the detection of the gesture or correction regarding the device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor; An information processing program that functions as a [Explanation of symbols]

[0253] 1. Information Processing Systems 10, 40 Information processing device 10A Earphones 10B headphones 11, 11 L , 11 R , 21, 41 Input detection section 12, 22, 32, 42 Status detection unit 13, 23, 33, 43 Output section 14, 24, 34, 44 Communications Department 15, 35, 45 storage section 16, 26, 36, 46 Control unit 20, 20A, 20B output device 30 Terminal Equipment 31 Input section 161, 361 Acquisition Department 162, 362, 462 Gesture detector 163, 363, 463 Command execution section 164, 364, 464 Output control section 165, 365, 465 Guessing part 166, 366, 466 correction section 461A First Acquisition Section 461B Second Acquisition Section AD approach direction OR detection area AR, A1, A2 detectable range A3, A4 area NR Non-detectable range B1~B4 Reference direction D1~D4 Device direction U User H hand O1, O2 obstacles

Claims

1. a gesture detection unit that detects a user's gesture made in a detection area of the detection unit that detects an object; a correction unit that corrects at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture, based on at least one of information regarding the gesture and information regarding a state of a predetermined device that includes the detection unit; and Equipped with the correction unit corrects at least one of the detectable range and the reference direction based on information about the user's movement related to the gesture; the information about the user's movement related to the gesture includes information about a first movement of the user's hand entering the detectable range; the correction unit corrects the detectable range to be at least narrower than the detection area based on information about the first operation. Information processing device.

2. the user's actions constituting one gesture include the first action and a second action subsequent to the first action, the correction unit, after the first motion is detected within the detectable range, widens the detectable range for detecting the second motion to be larger than the detectable range when the first motion is detected; The information processing device according to claim 1 .

3. When the gesture to be detected is a gesture involving a movement range, the correction unit widens the detectable range for detecting the second movement compared to the detectable range when the first movement is detected. The information processing device according to claim 2 .

4. A gesture detection unit that detects a user's gesture performed in a detection area of the detection unit that detects an object; a correction unit that corrects at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture, based on at least one of information regarding the gesture and information regarding a state of a predetermined device that includes the detection unit; and Equipped with the correction unit corrects at least one of the detectable range and the reference direction based on information about the user's movement related to the gesture; the information about the user's movement related to the gesture includes information about a first movement of the user's hand entering the detectable range; the correction unit corrects at least one of the detectable range and the reference direction based on information about the first operation; the information about the first movement includes information indicating an approach direction of the hand entering the detectable range, the correction unit corrects at least one of the detectable range and the reference direction based on information indicating the hand entry direction. Information processing device.

5. the predetermined device is wearable by the user, the correction unit corrects at least one of the detectable range and the reference direction based on information regarding a wearing state of the predetermined device on the user that is estimated based on the information indicating the hand approach direction, and information regarding a wearing inclination of the predetermined device that is estimated based on the information indicating the hand approach direction. The information processing device according to claim 4 .

6. the predetermined device has a predetermined portion that is positioned in the ear of the user when worn; The detection unit is disposed at the predetermined location, the correction unit corrects at least one of the detectable range and the reference direction based on information about a swing-up motion in which the user swings their hand up toward their ear. The information processing device according to claim 5 .

7. an execution unit that executes a function associated with the gesture when the gesture is detected; the predetermined device has a pair of predetermined portions that are positioned on both ears of the user when worn, When the swing-up motion is detected at both of the pair of predetermined parts, the execution unit does not execute the function associated with the gesture even if the gesture is detected. The information processing device according to claim 6 .

8. the detectable range is disposed within the detection area such that a part of an edge of the detectable range contacts a part of an edge of the detection area, the gesture detection unit does not detect the gesture when the user's hand enters the detection area from a range other than the detectable range. The information processing device according to claim 1 .

9. a gesture detection unit that detects a user's gesture made in a detection area of the detection unit that detects an object; a correction unit that corrects at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture, based on at least one of information regarding the gesture and information regarding a state of a predetermined device that includes the detection unit; and an execution unit that, when the gesture is detected, executes a function associated with the gesture, The functions include at least a predetermined function involving an operation amount, a predetermined gesture having a movement range is associated with the predetermined function; When the gesture detected by the gesture detection unit is the predetermined gesture, the execution unit determines the amount of operation based on a magnitude of a movement width of the predetermined gesture relative to the detectable range. Information processing device.

10. Detecting a user gesture made in a detection area of a detection unit that detects an object; correcting at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture based on at least one of information regarding the gesture and information regarding a state of a predetermined device including the detection unit; An information processing method, comprising: correcting at least one of the detectable range and the reference direction based on information about the user's movement related to the gesture; the information about the user's movement related to the gesture includes information about a first movement of the user's hand entering the detectable range; and correcting the detectable range to be at least narrower than the detection area based on information about the first operation. Information processing methods.

11. Detecting a user gesture made in a detection area of a detection unit that detects an object; correcting at least one of a detectable range of the gesture within the detection area and a reference direction for detecting the gesture based on at least one of information regarding the gesture and information regarding a state of a predetermined device including the detection unit; When the gesture is detected, performing a function associated with the gesture. An information processing method, comprising: The functions include at least a predetermined function involving an operation amount, a predetermined gesture having a movement range is associated with the predetermined function; If the detected gesture is the predetermined gesture, the operation amount is determined based on the magnitude of the movement width of the predetermined gesture relative to the detectable range. Information processing methods.

12. a first acquisition unit that acquires sensor information from a contact sensor that detects contact of a user with a human body or an object; a second acquisition unit that acquires sensor information from a non-contact sensor that detects the user's movement including the user's non-contact movement; a gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; a correction unit that performs correction regarding the detection of the gesture or correction regarding the device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor; Equipped with the gesture detection unit detects the gesture of the user using sensor information from at least the contact sensor; the correction unit corrects a detectable range of the gesture based on information about a gesture position indicating a position where the user performed the gesture and detected based on sensor information from the non-contact sensor; further comprising an execution unit that executes a command corresponding to the detected gesture, the execution unit determines a command to be executed based on the gesture detected based on sensor information from the contact sensor and information on a position of the gesture detected based on sensor information from the non-contact sensor. Information processing device.

13. a first acquisition unit that acquires sensor information from a contact sensor that detects contact of a user with a human body or an object; a second acquisition unit that acquires sensor information from a non-contact sensor that detects the user's movement including the user's non-contact movement; a gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; a correction unit that performs correction regarding the detection of the gesture or correction regarding the device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor; Equipped with the gesture detection unit detects the gesture of the user using sensor information from at least the non-contact sensor; the correction unit corrects the detectable range of the gesture based on sensor information from the non-contact sensor when the user touches the human body or the object. Information processing device.

14. the correction unit detects contact of the user with the human body or the object based on sensor information from the contact sensor, and corrects a detectable range of the gesture based on sensor information from the non-contact sensor when contact of the user with the human body or the object is detected. The information processing device according to claim 13.

15. an output control unit that, after detecting contact of the user with the human body or the object, provides feedback to the user when the user's hand is located within the corrected detectable range; The information processing device according to claim 14.

16. a first acquisition unit that acquires sensor information from a contact sensor that detects contact of a user with a human body or an object; a second acquisition unit that acquires sensor information from a non-contact sensor that detects the user's movement including the user's non-contact movement; a gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; a correction unit that performs correction regarding the detection of the gesture or correction regarding the device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor; a command execution unit that executes a command corresponding to the detected gesture, The command includes at least a command accompanied by an operation amount, the gesture detection unit detects the gesture of the user using sensor information from at least the contact sensor; the command execution unit determines the operation amount based on sensor information from the non-contact sensor; Information processing device.

17. The user's actions constituting one gesture include a first action that triggers the start of the gesture and a second action that follows the first action, the command execution unit determines the amount of operation based on a speed or a movement amount of the second action determined based on sensor information from the non-contact sensor; the correction unit changes a set value for associating the second action with the operation amount based on information on a speed of the first action detected based on sensor information from the non-contact sensor. The information processing device according to claim 16.

18. the correction unit corrects a detectable range of the second movement based on information about a speed of the first movement detected based on sensor information from the non-contact sensor. The information processing device according to claim 17.

19. acquiring sensor information from a contact sensor that detects contact with a user's body or an object; acquiring sensor information from a non-contact sensor that detects the user's movement, including the user's non-contact movement; detecting a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; performing a correction regarding the detection of the gesture or a correction regarding the device operation based on the sensor information from the contact sensor and the sensor information from the non-contact sensor; An information processing method, comprising: Detecting the gesture of the user using sensor information from at least the contact sensor; correcting a detectable range of the gesture based on information about a gesture position indicating a position where the user performed the gesture and detected based on sensor information from the non-contact sensor; Furthermore, executing a command corresponding to the detected gesture; and determining a command to be executed based on the gesture detected based on the sensor information from the contact sensor and information on the gesture position detected based on the sensor information from the non-contact sensor. Information processing methods.

20. acquiring sensor information from a contact sensor that detects contact with a user's body or an object; acquiring sensor information from a non-contact sensor that detects the user's movement, including the user's non-contact movement; detecting a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; performing a correction regarding the detection of the gesture or a correction regarding the device operation based on the sensor information from the contact sensor and the sensor information from the non-contact sensor; An information processing method, comprising: Detecting the gesture of the user using sensor information from at least the non-contact sensor; correcting a detectable range of the gesture based on sensor information from the non-contact sensor when the user touches the human body or the object; Information processing methods.

21. acquiring sensor information from a contact sensor that detects contact with a user's body or an object; acquiring sensor information from a non-contact sensor that detects the user's movement, including the user's non-contact movement; detecting a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor; performing a correction regarding the detection of the gesture or a correction regarding the device operation based on sensor information from the contact sensor and sensor information from the non-contact sensor; Executing a command corresponding to the detected gesture. An information processing method, comprising: The command includes at least a command accompanied by an operation amount, Detecting the gesture of the user using sensor information from at least the contact sensor; determining the operation amount based on sensor information from the non-contact sensor; Information processing methods.

Images