User interface for gesture detection

Abstract

The computer system detects gestures such as air gestures and performs actions based on those gestures.

Claims

1. It is a method, In a computer system that communicates with one or more input devices, including display generation components and physiological sensors, The system receives input via one or more input devices to display individual indicators corresponding to individual external devices from among a plurality of detected external devices, In response to receiving the input for displaying the individual indication corresponding to the individual external device, the display generation component displays the individual indication corresponding to the individual external device, While displaying the individual indication corresponding to the individual external device, the individual air gesture is detected via the physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that the individual indication corresponds to the first external device, the command corresponding to the individual air gesture is sent to the first external device without sending the command corresponding to the individual air gesture to the second external device. In accordance with the determination that the individual indication corresponds to a second external device different from the first external device, the command corresponding to the individual air gesture is sent to the second external device without sending the command corresponding to the individual air gesture to the first external device. Methods that include...

2. In accordance with the determination that the individual air gesture is a first air gesture, the command is a first command, In accordance with the determination that the individual air gesture is a second air gesture different from the first air gesture, the command is a second command different from the first command. The method according to claim 1.

3. The heart rate of the user of the computer system is detected via the physiological sensor, The heart rate is displayed via the aforementioned display generation component, The method according to claim 1 or 2, further comprising:

4. The method according to any one of claims 1 to 3, wherein the computer system is a wearable device worn by a user of the computer system.

5. The method according to any one of claims 1 to 4, wherein the input for displaying the individual indication corresponding to the individual external device is a double pinch air gesture.

6. The method according to any one of claims 1 to 4, wherein the input for displaying the individual indication corresponding to the individual external device includes a swipe gesture.

7. The plurality of detected external devices are selected based on the detected proximity of the plurality of detected external devices to the computer system. The order in which the indication of the plurality of detected external devices is determined is based on the state and / or proximity of the plurality of detected external devices. The method according to any one of claims 1 to 6.

8. The method according to any one of claims 1 to 7, wherein the individual air gesture is a swipe air gesture, and the command corresponding to the swipe air gesture is a navigation command for navigating the user interface of the individual external device.

9. While displaying the individual indication corresponding to the individual external device, a double pinch air gesture is detected via the physiological sensor. In response to the detection of the double pinch air gesture, the display of the indication for one of the detected external devices is stopped. The method according to any one of claims 1 to 4 and 6 to 8, further comprising:

10. In response to receiving the input for displaying the individual indication corresponding to the individual external device, the computer system transmits an indication to the individual external device that it is configured to control the individual external device. The method according to any one of claims 1 to 9, further comprising:

11. The individual air gesture includes a pinch-and-hold air gesture performed by the user's hand in the computer system, the individual air gesture includes a first magnitude of rotation of the user's hand in the computer system, and the command corresponding to the individual air gesture transmitted to the individual external device includes a second magnitude of change in the characteristics of the individual external device based on the first magnitude. The method according to any one of claims 1 to 10.

12. The method according to claim 11, wherein the first size is the amount of rotation of the user's hand in the computer system.

13. The method according to claim 11 or 12, wherein the first magnitude is the rotational speed of the user's hand in the computer system.

14. The method according to any one of claims 11 to 13, wherein the rate of change of the characteristics of the individual external devices is limited to a threshold rate based on the value of the characteristics of the individual external devices that exceeds a threshold.

15. In response to the computer system detecting the rotation of the user's hand as part of the individual air gestures, to provide tactile feedback, The method according to any one of claims 11 to 14, further comprising:

16. The method according to any one of claims 1 to 7 and 9 to 15, wherein the individual air gesture is a pinch air gesture.

17. The method according to any one of claims 1 to 7 and 9 to 15, wherein the individual air gesture is a pinch-and-hold air gesture, and the command corresponding to the pinch-and-hold air gesture is a navigation command for navigating back to the user interface of the individual external device.

18. The method according to any one of claims 1 to 7 and 9 to 15, wherein the individual air gesture is a tap of the user's finger on the computer system against the user's palm on the computer system.

19. While displaying the individual indication corresponding to the individual external device, the computer system detects speech via one or more microphones. In response to detecting the aforementioned utterance, In accordance with the determination that the individual indication corresponds to the first external device, the information based on the utterance is transmitted to the first external device without transmitting the information based on the utterance to the second external device. In accordance with the determination that the individual indication corresponds to the second external device, the information based on the utterance is transmitted to the second external device without transmitting the information based on the utterance to the first external device. The method according to any one of claims 1 to 18, further comprising:

20. The method according to any one of claims 1 to 19, wherein a first indication corresponds to a first application of a first external device, and a second indication, different from the first indication, corresponds to a second application of the first external device, different from the first application.

21. A non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including a display generation component and a physiological sensor, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 20.

22. A computer system configured to communicate with one or more input devices including display generation components and physiological sensors, wherein the computer system One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 20.

23. A computer system configured to communicate with one or more input devices, including display generation components and physiological sensors, A computer system comprising means for performing the method described in any one of claims 1 to 20.

24. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including a display generation component and a physiological sensor, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 20.

25. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including display generation components and physiological sensors, wherein the one or more programs are The system receives input via one or more input devices to display individual indications corresponding to individual external devices from among a plurality of detected external devices. Upon receiving the input for displaying the individual indication corresponding to the individual external device, the display generation component displays the individual indication corresponding to the individual external device. While displaying the individual indication corresponding to the individual external device, the individual air gesture is detected via the physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that the individual indication corresponds to the first external device, the command corresponding to the individual air gesture is sent to the first external device without sending the command corresponding to the individual air gesture to the second external device. A non-temporary computer-readable storage medium including instructions for transmitting the command corresponding to the individual air gesture to the second external device without transmitting the command corresponding to the individual air gesture to the first external device, in accordance with the determination that the individual indication corresponds to a second external device different from the first external device.

26. A computer system configured to communicate with one or more input devices, including display generation components and physiological sensors, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are The system receives input via one or more input devices to display individual indications corresponding to individual external devices from among a plurality of detected external devices. Upon receiving the input for displaying the individual indication corresponding to the individual external device, the display generation component displays the individual indication corresponding to the individual external device. While displaying the individual indication corresponding to the individual external device, the individual air gesture is detected via the physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that the individual indication corresponds to the first external device, the command corresponding to the individual air gesture is sent to the first external device without sending the command corresponding to the individual air gesture to the second external device. A computer system including instructions to send the command corresponding to the individual air gesture to the second external device without sending the command corresponding to the individual air gesture to the first external device, in accordance with the determination that the individual indication corresponds to a second external device different from the first external device.

27. A computer system configured to communicate with one or more input devices, including display generation components and physiological sensors, Means for receiving input via one or more input devices to display individual indications corresponding to individual external devices from among a plurality of detected external devices, In response to receiving the input for displaying the individual indication corresponding to the individual external device, means for displaying the individual indication corresponding to the individual external device via the display generation component, While displaying the individual indication corresponding to the individual external device, means for detecting individual air gestures via the physiological sensor, In response to the detection of the individual air gestures, Means for transmitting the command corresponding to the individual air gesture to the first external device without transmitting the command corresponding to the individual air gesture to the second external device, in accordance with the determination that the individual indication corresponds to the first external device, A means for transmitting the command corresponding to the individual air gesture to the second external device without transmitting the command corresponding to the individual air gesture to the first external device, in accordance with the determination that the individual indication corresponds to a second external device different from the first external device, A computer system equipped with the following features.

28. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that communicates with one or more input devices including a display generation component and a physiological sensor, wherein the one or more programs are The system receives input via one or more input devices to display individual indications corresponding to individual external devices from among a plurality of detected external devices. Upon receiving the input for displaying the individual indication corresponding to the individual external device, the display generation component displays the individual indication corresponding to the individual external device. While displaying the individual indication corresponding to the individual external device, the individual air gesture is detected via the physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that the individual indication corresponds to the first external device, the command corresponding to the individual air gesture is sent to the first external device without sending the command corresponding to the individual air gesture to the second external device. A computer program product including instructions that, in accordance with the determination that the individual indication corresponds to a second external device different from the first external device, transmits the command corresponding to the individual air gesture to the second external device without transmitting the command corresponding to the individual air gesture to the first external device.

29. It is a method, In a computer system that communicates with a display generation component and one or more input devices including one or more microphones, During the oral process, The system displays a transcription of the user's first utterance based on the audio data detected by one or more microphones via the display generation component. In accordance with the determination that no first type of air gesture has been detected, a transcription of the second utterance is displayed via the display generation component, based on the audio data of the second utterance detected by the one or more microphones, while the user is uttering the second utterance via the one or more input devices. In accordance with the determination that the first type of air gesture has been detected, the display of a transcription of the second utterance is discontinued via one or more input devices while the user is uttering the second utterance. Methods that include...

30. During the aforementioned oral process, In accordance with the determination that no first type of air gesture has been detected, a transcription of the third utterance is displayed via the display generation component, based on the audio data of the third utterance detected by the one or more microphones, while the user is uttering the third utterance via the one or more input devices. In accordance with the determination that the first type of air gesture has been detected, the display of a transcription of the third utterance is discontinued while the user is uttering the third utterance via one or more input devices. The method according to claim 29, further comprising:

31. The method according to claim 29 or 30, wherein the first type of air gesture is a pinch-and-hold air gesture.

32. During the aforementioned oral process, In accordance with the determination that the first type of air gesture has been detected via one or more input devices, the display generation component displays an indication that the audio has not been transcribed. The method according to any one of claims 29 to 31, further comprising:

33. During the aforementioned oral process, In accordance with the determination that the first type of air gesture has been detected, audio data is detected via one or more microphones. The method according to any one of claims 29 to 32, further comprising:

34. During the aforementioned oral process, In accordance with the determination that the first type of air gesture is detected via the one or more input devices, the detection of audio data by the one or more microphones is stopped. The method according to any one of claims 29 to 32, further comprising:

35. To detect the end of the aforementioned oral process, Upon detecting the termination of the aforementioned oral process, the display generation component shall display an option to send a message containing text based on the transcription. The method according to any one of claims 29 to 34, further comprising:

36. While displaying the option for sending the message, a pinch-and-hold air gesture is detected via one or more input devices. In response to detecting the aforementioned pinch-and-hold air gesture, the message is sent. The method according to claim 35, further comprising:

37. While the option for sending the message is displayed, touch input is detected via one or more input devices at locations corresponding to the option for sending the message. In response to detecting the aforementioned touch input, the message is sent, The method according to claim 35 or 36, further comprising:

38. The method according to any one of claims 29 to 37, wherein the computer system includes a touchscreen.

39. The method according to claim 38, wherein the transcription of the user's first utterance is not based on input received on the touchscreen.

40. The method according to any one of claims 29 to 39, wherein the first type of air gesture does not involve contact with the touchscreen.

41. The method according to any one of claims 29 to 40, wherein the computer system is a wearable computer system worn on the user's hand, and the first type of air gesture is performed by the user's hand on which the wearable computer system is worn.

42. A non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices including one or more microphones, wherein the one or more programs include instructions for performing the method according to any one of claims 29 to 41.

43. A computer system configured to communicate with a display generation component and one or more input devices including one or more microphones, wherein the computer system One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 29 to 41.

44. A computer system configured to communicate with a display generation component and one or more input devices, including one or more microphones, A computer system comprising means for performing the method described in any one of claims 29 to 41.

45. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices including one or more microphones, wherein the one or more programs include instructions for performing the method according to any one of claims 29 to 41.

46. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including display generation components and physiological sensors, wherein the one or more programs are During the oral process, Based on the audio data of the user's first utterance detected by one or more microphones, the display generation component displays a transcription of the first utterance. In accordance with the determination that no first type of air gesture has been detected, a transcription of the second utterance is displayed via the display generation component, based on the audio data of the second utterance detected by the one or more microphones, while the user is uttering the second utterance via the one or more input devices. A non-temporary computer-readable storage medium including a command to stop displaying a transcription of the second utterance via one or more input devices while the user is uttering the second utterance, in accordance with the determination that the first type of air gesture has been detected.

47. A computer system configured to communicate with a display generation component and one or more input devices, including one or more microphones, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are During the oral process, Based on the audio data of the user's first utterance detected by one or more microphones via the display generation component, a transcription of the first utterance is displayed. In accordance with the determination that no first type of air gesture has been detected, a transcription of the second utterance is displayed via the display generation component, based on the audio data of the second utterance detected by the one or more microphones of the second utterance, while the user is uttering the second utterance via the one or more input devices. A computer system including an instruction to stop displaying a transcription of the second utterance via one or more input devices while the user is uttering the second utterance, in accordance with the determination that the first type of air gesture has been detected.

48. A computer system configured to communicate with a display generation component and one or more input devices, including one or more microphones, During the oral process, Means for displaying a transcription of the user's first utterance based on audio data detected by one or more microphones of the user's first utterance via the display generation component, Means for displaying a transcription of the second utterance via the display generation component, based on audio data of the second utterance detected by the one or more microphones of the second utterance, while the user is uttering the second utterance via the one or more input devices, in accordance with the determination that a first type of air gesture has not been detected. Means for discontinuing the display of a transcription of the second utterance via one or more input devices while the user is uttering the second utterance, in accordance with the determination that the first type of air gesture has been detected. A computer system equipped with the following features.

49. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that communicates with a display generation component and one or more input devices including one or more microphones, wherein the one or more programs are During the oral process, Based on the audio data of the user's first utterance detected by one or more microphones via the display generation component, a transcription of the first utterance is displayed. In accordance with the determination that no first type of air gesture has been detected, a transcription of the second utterance is displayed via the display generation component, based on the audio data of the second utterance detected by the one or more microphones of the second utterance, while the user is uttering the second utterance via the one or more input devices. A computer program product that includes an instruction to stop displaying a transcription of the second utterance via one or more input devices while the user is uttering the second utterance, in accordance with the determination that the first type of air gesture has been detected.

50. It is a method, In a computer system that communicates with one or more input devices, including physiological sensors, Receiving input related to the voice assistant via one or more input devices, In response to receiving the input related to the voice assistant, the system outputs voice assistant data. Individual air gestures are detected via the aforementioned physiological sensor, In response to the detection of the individual air gestures, The operation corresponding to the individual air gesture is performed according to the determination that one or more conditions are met, including a first condition that is met when the individual air gesture is detected while the voice assistant data is being output. In accordance with the determination that one or more of the above set of conditions are not met, the execution of the operation corresponding to the individual air gesture is discontinued. Methods that include...

51. Outputting voice assistant data includes displaying an indication via a display generation component that the message is ready to be sent. The input related to the voice assistant is an utterance detected by one or more microphones of the computer system. The aforementioned message is based on the aforementioned utterance, The set of one or more conditions includes a second condition that is met when the individual air gesture is a pinch-and-hold air gesture detected beyond a threshold duration, The action corresponding to the individual air gesture is to send the message. The method according to claim 50.

52. In response to the detection of the individual air gestures, the display generation component displays a progress indicator that progresses over time, wherein the location of the progress indicator is based on the location of the indication that the message is ready to be sent. The method according to claim 51, further comprising:

53. The method according to claim 51 or 52, wherein when the pinch-and-hold air gesture is detected over the threshold duration, the progress indicator, which progresses over time, indicates completion.

54. The method according to any one of claims 51 to 53, wherein the computer system includes a touch-sensing surface.

55. The method according to claim 54, wherein the content of the message is not based on a touch gesture received via the touch sensing surface.

56. The method according to claim 54 or 55, wherein the pinch-and-hold air gesture does not include an input detected by the touch-sensing surface.

57. The method according to any one of claims 54 to 56, wherein touch input activation of the indication, detected via the touch-sensing surface, that the message is ready to be sent, causes the computer system to send the message.

58. In response to receiving the input related to the voice assistant, a first tactile output is output. The method according to any one of claims 50 to 57, further comprising:

59. In response to the termination of the output of the aforementioned voice assistant data, a second tactile output is output. The method according to any one of claims 50 to 58, further comprising:

60. While the computer system is outputting the voice assistant data, the physiological sensor detects a double pinch air gesture. In response to the detection of the aforementioned double pinch air gesture, the output of the voice assistant data is stopped. The method according to any one of claims 50 to 59, further comprising:

61. The method according to any one of claims 50 to 60, wherein performing the action corresponding to the individual air gesture includes performing a default action corresponding to the currently displayed user interface.

62. The method according to any one of claims 50 to 61, wherein the individual air gestures are single pinch air gestures.

63. The method according to any one of claims 50 to 61, wherein the individual air gesture is a double pinch air gesture.

64. The method according to any one of claims 50 to 63, wherein outputting voice assistant data includes outputting audio corresponding to a received message through one or more speakers.

65. The method according to claim 64, wherein performing the action corresponding to the individual air gestures includes initiating recording and / or transcription of audio as part of a response to the received message.

66. The method according to any one of claims 50 to 65, comprising outputting a request to the user of the computer system to confirm that outputting voice assistant data will perform the action corresponding to the individual air gesture.

67. The method according to any one of claims 50 to 66, wherein outputting voice assistant data includes outputting an indication that a proposed calendar event conflicts with an existing calendar event.

68. The method according to any one of claims 50 to 67, wherein the computer system is a wearable computer system worn on the user's hand, and the individual air gesture is a gesture performed by the user's hand on which the wearable computer system is worn.

69. A non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices, including a physiological sensor, wherein the one or more programs include instructions for performing the method according to any one of claims 50 to 68.

70. A computer system configured to communicate with one or more input devices, including physiological sensors, wherein the computer system One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 50 to 68.

71. A computer system configured to communicate with one or more input devices, including physiological sensors, A computer system comprising means for performing the method described in any one of claims 50 to 68.

72. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including physiological sensors, wherein the one or more programs include instructions for performing the method according to any one of claims 50 to 68.

73. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including physiological sensors, wherein the one or more programs are The system receives input related to the voice assistant via one or more input devices. In response to receiving the input related to the voice assistant, the system outputs voice assistant data. Individual air gestures are detected via the aforementioned physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that one or more sets of conditions are met, including a first condition that is met when the individual air gesture is detected while the voice assistant data is being output, the operation corresponding to the individual air gesture is executed. A non-temporary computer-readable storage medium including instructions for canceling the execution of the operation corresponding to the individual air gesture in accordance with the determination that one or more of the aforementioned set of conditions are not met.

74. A computer system configured to communicate with one or more input devices, including physiological sensors, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are The system receives input related to the voice assistant via one or more input devices. In response to receiving the input related to the voice assistant, the system outputs voice assistant data. Individual air gestures are detected via the aforementioned physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that one or more sets of conditions are met, including a first condition that is met when the individual air gesture is detected while the voice assistant data is being output, the operation corresponding to the individual air gesture is executed. A computer system including an instruction to cancel the execution of the operation corresponding to the individual air gesture in accordance with the determination that one or more of the aforementioned set of conditions are not met.

75. A computer system configured to communicate with one or more input devices, including physiological sensors, Means for receiving input related to a voice assistant via one or more input devices, A means for outputting voice assistant data in response to receiving the input related to the voice assistant, Means for detecting individual air gestures via the physiological sensor, In response to the detection of the individual air gestures, Means for performing an action corresponding to the individual air gesture, in accordance with a determination that one or more sets of conditions are met, including a first condition that is met when the individual air gesture is detected while the voice assistant data is being output; Means for stopping the execution of the operation corresponding to the individual air gesture in accordance with the determination that one or more of the above set of conditions are not met, A computer system equipped with the following features.

76. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including physiological sensors, wherein the one or more programs are The system receives input related to the voice assistant via one or more input devices. In response to receiving the input related to the voice assistant, the system outputs voice assistant data. Individual air gestures are detected via the aforementioned physiological sensor. In response to the detection of the individual air gestures, In accordance with the determination that one or more sets of conditions are met, including a first condition that is met when the individual air gesture is detected while the voice assistant data is being output, the operation corresponding to the individual air gesture is executed. A computer program product that includes instructions for ceasing to perform the operation corresponding to the individual air gesture in accordance with the determination that one or more of the aforementioned set of conditions are not met.

77. It is a method, In a computer system communicating with one or more input devices, Detecting a first input of a first type via one or more input devices, In response to the detection of the first input of the first type, In accordance with the determination that an individual air gesture was detected when the first input of the first type was detected, a first operation is performed based on the first input of the first type, Based on the determination that the individual air gesture was not detected when the first type of the first input was detected, a second operation different from the first operation is performed based on the first type of the first input. Methods that include...

78. The method according to claim 77, wherein the individual air gestures are pinch air gestures.

79. The method according to claim 77 or 78, wherein the first input of the first type is a stylus input along a path.

80. The method according to any one of claims 77 to 79, wherein the first operation is an erase operation.

81. The method according to any one of claims 77 to 80, wherein the second operation is a write operation.

82. The method according to any one of claims 77 to 81, wherein the first input of the first type is a keyboard input.

83. The method according to any one of claims 77 to 82, wherein the first operation is a text input operation using a first uppercase / lowercase character. In some embodiments, the first operation is the text input of a first character.

84. The method according to claim 83, wherein the second operation is a text input operation that uses a second set of uppercase and lowercase letters different from the first set of uppercase and lowercase letters. In some embodiments, the second operation is the text input of the first character.

85. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices, wherein the one or more programs include instructions for performing the method according to any one of claims 77 to 84.

86. A computer system configured to communicate with one or more input devices, wherein the computer system is One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 77 to 84.

87. A computer system configured to communicate with one or more input devices, A computer system comprising means for performing the method described in any one of claims 77 to 84.

88. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices, wherein the one or more programs include instructions for performing the method according to any one of claims 77 to 84.

89. A non-temporary computer-readable storage medium that stores one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices, wherein the one or more programs are A first input of a first type is detected via one or more input devices. In response to the detection of the first input of the first type, In accordance with the determination that a specific air gesture was detected when the first input of the first type was detected, the first operation is performed based on the first input of the first type. A non-temporary computer-readable storage medium including instructions for performing a second operation different from the first operation based on the first input of the first type, in accordance with the determination that the individual air gesture was not detected when the first input of the first type was detected.

90. A computer system configured to communicate with one or more input devices, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are A first input of a first type is detected via one or more input devices. In response to the detection of the first input of the first type, In accordance with the determination that a specific air gesture was detected when the first input of the first type was detected, the first operation is performed based on the first input of the first type. A computer system including instructions to perform a second operation different from the first operation based on the first input of the first type, in accordance with the determination that the individual air gesture was not detected when the first input of the first type was detected.

91. A computer system configured to communicate with one or more input devices, Means for detecting a first input of a first type via one or more input devices, In response to the detection of the first input of the first type, Means for performing a first operation based on the first input of the first type, in accordance with the determination that an individual air gesture was detected when the first input of the first type was detected, Means for performing a second operation different from the first operation based on the first input of the first type, in accordance with the determination that the individual air gesture was not detected when the first input of the first type was detected, A computer system equipped with the following features.

92. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices, wherein the one or more programs are A first input of a first type is detected via one or more input devices. In response to the detection of the first input of the first type, In accordance with the determination that a specific air gesture was detected when the first input of the first type was detected, the first operation is performed based on the first input of the first type. A computer program product including instructions that, based on the first input of the first type, execute a second operation different from the first operation, according to the determination that the individual air gesture was not detected when the first input of the first type was detected.

93. It is a method, In a computer system that communicates with one or more input devices, including physiological sensors, The physiological sensor detects a first finger slide gesture of the first finger of the user's hand, which slides in a first direction along a second finger different from the first finger of the user's hand, by a first magnitude. In response to detecting the first finger slide gesture, a first action is performed based on the first direction and / or the first magnitude of the first finger slide gesture, Methods that include...

94. The method according to claim 93, wherein the first action is based on the first direction and the first magnitude of the first finger slide gesture.

95. The physiological sensor detects a second finger slide gesture of the first finger of the user's hand, which slides in a second direction along the second finger of the user's hand by a second magnitude, In response to detecting the second finger slide gesture, a second action different from the first action is performed based on the second direction and / or second magnitude of the second finger slide gesture, The method according to claim 93 or 94, further comprising:

96. The method according to any one of claims 93 to 95, wherein the first operation includes a scrolling operation to scroll the content to be displayed via the display generation component of the computer system.

97. The method according to claim 96, wherein the scrolling direction of the scrolling operation is based on the first direction of the first finger slide gesture.

98. The method according to any one of claims 95 to 97, wherein the scroll size of the scrolling operation is based on the first size of the first finger slide gesture.

99. Detecting rotational input, including the direction and magnitude of rotation, via one or more input devices, In response to detecting the rotation input, the first operation is performed based on the rotation direction and / or magnitude of the rotation input, The method according to any one of claims 93 to 98, further comprising:

100. The method according to any one of claims 93 to 99, wherein the first size of the first finger slide gesture is the detected distance traveled by the first finger slide gesture.

101. The method according to any one of claims 93 to 99, wherein the first magnitude of the first finger slide gesture is the detected movement speed of the first finger slide gesture.

102. The method according to any one of claims 93 to 100, wherein the first action includes a size based on the first size of the first finger slide gesture.

103. The method according to claim 102, wherein the magnitude of the first operation is the amount of scrolling.

104. The method according to claim 102 or 103, wherein the magnitude of the first operation is the scrolling speed.

105. The method according to any one of claims 93 to 104, wherein the first operation is to send a command to the external device for traversing the user interface of the external device, the external device being selected from a plurality of external devices based on the fact that the user's hand is directed toward the external device when the first finger slide gesture is detected.

106. The method according to any one of claims 93 to 104, wherein the first operation is the movement of a cursor.

107. A non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices, including a physiological sensor, wherein the one or more programs include instructions for performing the method according to any one of claims 93 to 106.

108. A computer system configured to communicate with one or more input devices, including physiological sensors, wherein the computer system One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 93 to 106.

109. A computer system configured to communicate with one or more input devices, including physiological sensors, A computer system comprising means for performing the method described in any one of claims 93 to 106.

110. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including a physiological sensor, wherein the one or more programs include instructions for performing the method according to any one of claims 93 to 106.

111. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including physiological sensors, wherein the one or more programs are The physiological sensor detects a first finger slide gesture of the first finger of the user's hand, which slides in a first direction along a second finger different from the first finger of the user's hand, by a first magnitude. A non-temporary computer-readable storage medium containing instructions for performing a first action based on the first direction and / or the first magnitude of the first finger slide gesture in response to the detection of the first finger slide gesture.

112. A computer system configured to communicate with one or more input devices, including physiological sensors, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are The physiological sensor detects a first finger slide gesture of the first finger of the user's hand, which slides in a first direction along a second finger different from the first finger of the user's hand, by a first magnitude. A computer system including instructions for performing a first action based on the first direction and / or first magnitude of the first finger slide gesture in response to the detection of the first finger slide gesture.

113. A computer system configured to communicate with one or more input devices, including physiological sensors, Means for detecting a first finger slide gesture of the first finger of the user's hand, via the physiological sensor, which slides in a first direction along a second finger different from the first finger of the user's hand, by a first magnitude; In response to the detection of the first finger slide gesture, means for performing a first action based on the first direction and / or first magnitude of the first finger slide gesture, A computer system equipped with the following features.

114. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices including physiological sensors, wherein the one or more programs are The physiological sensor detects a first finger slide gesture of the first finger of the user's hand, which slides in a first direction along a second finger different from the first finger of the user's hand, by a first magnitude. A computer program product including instructions for performing a first action based on the first direction and / or the first magnitude of the first finger slide gesture in response to the detection of the first finger slide gesture.

Images