Computer, operating method, and operating program product

Abstract

The invention provides a computer, an operation method, and an operation program product capable of performing fine operation when operating with fingers. A computer is provided with: a touch panel display that displays a plurality of objects and a cursor that follows a finger; and an execution unit that executes a process corresponding to the object indicated by the cursor, and changes the relative position of the cursor based on the finger so that the cursor is positioned forward in the direction in which the finger moves.

Description

Technical Field

[0001] The present invention relates to a computer, an operating method and an operating program product. Background Art

[0002] Various methods have been proposed to improve the operability of touch operations. Patent document 1 describes a technique in which a cursor is displayed at a position separated from a touch position by a predetermined distance, and the cursor is positioned on an icon by operating the cursor, thereby performing a touch operation. Patent document 2 describes a technique in which a corner of a display unit can be indicated by making the touch panel unit larger than the display unit. Patent document 3 describes a technique in which, when the touch position reaches the end of the screen, the display position of a button indicating the touch position is changed to the inside of the screen, and the button touch state is maintained until the next touch operation. Patent document 4 describes a technique in which the display position of a cursor relative to a touch position (relative position based on the touch position) is changed by touching two fingers.

[0003] A structure that enables fine manipulation when operated with fingers is desired.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2002-123367

[0005] Patent Document 2: Japanese Patent Application Publication No. 2002-287904

[0006] Patent Document 3: Japanese Patent Application Publication No. 2003-186620

[0007] Patent Document 4: Japanese Patent Application Publication No. 2014-219726 Summary of the invention

[0008] A computer for solving the above-mentioned problems comprises: a touch panel display which displays a plurality of objects and a cursor which follows a finger; and an execution unit which performs processing corresponding to the object indicated by the cursor, thereby changing the relative position of the cursor with respect to the finger so that the cursor is located in front of the direction in which the finger moves.

[0009] The operation method for solving the above-mentioned problem includes the following operations, namely, detecting the position of the user's finger, displaying multiple objects and a cursor corresponding to both the position of the finger and the direction of movement of the finger, and performing processing corresponding to the object indicated by the cursor.

[0010] The operating program product for solving the above-mentioned problems enables the computer to perform the following operations, namely, detecting the position of the user's finger, displaying multiple objects and cursors corresponding to both the position of the finger and the direction of movement of the finger, and performing processing corresponding to the object indicated by the cursor. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A block diagram of a computer.

[0012] Figure 2 A diagram illustrating finger movement and cursor position.

[0013] Figure 3 A diagram illustrating the trajectory of the touch position and the cursor position.

[0014] Figure 4 A diagram illustrating finger movement and cursor position.

[0015] Figure 5 Flowchart for cursor display processing.

[0016] Figure 6 A diagram illustrating finger movement and cursor position. DETAILED DESCRIPTION

[0017] Here, embodiments of the present invention will be described in the following order.

[0018] (1) Computer structure:

[0019] (2) Cursor display processing:

[0020] (3) Other implementation methods:

[0021] (1) Computer structure:

[0022] Figure 1 1 is a block diagram showing the structure of an electronic device 100 as a computer according to an embodiment of the present invention. The computer in this embodiment is an electronic device having a touch panel display, and is assumed to be a tablet or a smartphone, for example.

[0023] The electronic device 100 includes a processor 120, a nonvolatile memory 130, a UI unit 140, and a communication unit 150. The processor 120 includes a CPU, a ROM, a RAM, etc. (not shown), and can execute various programs recorded in the nonvolatile memory 130 to control various parts of the electronic device 100. In addition, the processor 120 may be composed of a single chip or a plurality of chips. In addition, for example, an ASIC may be used instead of a CPU, or a structure in which the CPU and the ASIC cooperate may be used.

[0024] The UI unit 140 includes a touch panel display 141. The touch panel display 141 includes a display that displays an arbitrary image and a touch panel that overlaps the display and detects the contact of a finger to obtain the coordinates of the contact position. In addition, the UI unit 140 includes other input units such as a power button and a volume button, a microphone, and a speaker. The processor 120 can obtain input content corresponding to the operation of the touch panel display 141 and other input units of the UI unit 140.

[0025] The communication unit 150 includes a communication interface for communicating with external devices according to various wired or wireless communication protocols. The electronic device 100 can communicate with other devices via the communication unit 150. In addition, a keyboard, a mouse, a touch panel display, etc. are connected via the communication unit 150, and the processor 120 can also input and output various information via these devices.

[0026] A plurality of objects such as icons are displayed on the touch panel display 141. When the user inputs an instruction by a touch operation on the touch panel display 141, in the present embodiment, operations of the cursor operation mode and the touch operation mode can be performed. The touch operation mode is a mode in which the processor 120 can execute a process corresponding to the object by the user touching the position of the object where the operation target is displayed. The cursor operation mode is a mode in which the processor 120 can execute a process corresponding to the object indicated by the cursor by the user overlapping the cursor displayed at a position different from the touch position and following the movement of the touch position with the object of the operation target. The details of the cursor operation mode are described below. It can also be set that the user can switch between the cursor operation mode and the touch operation mode by a predetermined operation. For example, the switch from the touch operation mode to the cursor operation mode can be based on the touch to a specific area of ​​the touch panel display 141 as the starting point. In such a case, it can be set that the touch operation mode is restored by the finger that is touching leaving and a fixed time has passed or a touch outside the specific area is detected.

[0027] Figure 2 A schematic diagram for explaining the movement of a finger and the display position of a cursor in the cursor operation mode. Figure 2 The area g1 in FIG. 1 is an area for detecting touch and shows the range in which the cursor can move. In the cursor operation mode, when the user touches any position in the area g1 of the touch panel display 141 with a finger, the processor 120 Figure 2As shown in state 2a of , the cursor is displayed at position B1 at a distance C from touch position A1. The relative angle of the cursor position relative to the touch position at the start of the touch can be arbitrary, but since it is desired to display the cursor within region g1 at the start of the touch, the relative angle is selected so as to be a position at a distance C from A1 and within region g1. When the relative angle is defined as the angle between a line connecting the reference point and the cursor position and the reference line extending from the reference point toward the top in the vertical direction of the screen (clockwise is positive, counterclockwise is negative), this example shows a case where the cursor is displayed at position B1 at a relative angle of 0° and a distance C from touch position A1. When the user moves the finger while touching the touch panel display 141, the cursor follows the movement of the finger.

[0028] Specifically, the processor 120 changes the relative position of the cursor based on the finger so that the cursor is located ahead of the direction in which the finger moves. More specifically, the processor 120 displays the cursor at the intersection of a line extending along the direction in which the finger moves and a circle centered on the touch position of the finger. The processor 120 sets a threshold D for the movement distance (straight-line distance) of the finger used to move the cursor. The processor 120 sets the position A of the finger before the cursor moves. n When the straight-line distance between the position of the finger after the finger is moved while the finger is touching becomes greater than D, the cursor position is updated. n The finger position A when the straight-line distance becomes greater than D n+1 The extension line of the line extending in the direction of A n+1 The cursor is displayed at a position where the circumference of a circle with a radius C as the center overlaps. In this way, the cursor position can be set at a position that is always C away from the touch position and on the extension line of the finger's moving direction.

[0029] For example, Figure 2 State 2b shows the state after the finger is moved from state 2a. As shown in states 2a and 2b, when it is detected that the touch position moves from A1 to A2 at a distance D, the processor 120 makes the cursor that was previously displayed at B1 not displayed, and displays the cursor at the intersection B2 of the circumference of the circle with a radius C centered at A2 and the extension line of the line from A1 to A2. That is, when the touch position moves from A1 to A2 in the state of finger touch, the cursor moves from B1 to B2. Figure 2 State 2c shows the state after the finger is further moved from state 2b. When the touch position moves from A2 to A3, the cursor moves from B2 to B3 and is displayed. B3 is the intersection of the circle of radius C centered at the touch position A3 and the positive extension line of the line from A2 to A3.

[0030] Figure 3 A diagram illustrating a moving touch position and a trajectory of a cursor following the touch position (an example). The black circle in the figure is the touch position, and the white circle represents the cursor displayed on the touch panel display 141. The finger slides smoothly through A1, A2, A3, A4, A5, A6, A7, and A8. In response to the finger's contact starting at A1, the processor 120 displays the cursor at B1. When the finger slides from A1 in the above order, in response to detecting that the finger reaches position A2 at a distance D from A1, the processor 120 deletes the cursor that was displayed at B1 and redisplays the cursor at B2 (before the finger reaches A2, the cursor continues to be displayed at B1). In addition, in response to detecting that the finger reaches position A3 at a distance D from A2, the processor 120 deletes the cursor that was displayed at B2 and displays the cursor at B3. After that, by performing the same processing, in response to the finger sliding as described above, the cursor jumps and displays in the order of B4, B5, B6, B7, and B8. In this way, in response to the change in the moving direction of the finger, the cursor jumps and displays at a new relative position, which is similar to, for example, in response to detecting the change from A to B. n Move to A n+1 In the case of B n To B n+1 Compared with the case of performing moving animation, the user can feel that the cursor follows better (the cursor moves from B n Arrive at B n+1 less delay time).

[0031] Since the cursor is displayed on the circumference of a circle with a radius C centered on the finger touching the touch panel display 141, the distance between the finger and the cursor is always fixed, but the relative angle of the cursor based on the finger changes according to the moving direction of the finger. Since the distance between the finger and the cursor is fixed, it is easy for the user to understand the amount of finger movement required to move the cursor to the target position.

[0032] In addition, it can also be set that the above-mentioned distance C and distance D can be set. It can be configured so that it can be changed to a value desired by the user by changing the setting by the user. For example, when the display screen is large, the distance C is set to be larger than when the display screen is small, so that even if the finger is not moved significantly, the operation can easily reach a farther position on the screen. Or, even if it is a small screen like a smartphone, when operating with a thumb with one hand, for example, only touching the lower part of the screen, it is possible to set C in a way that is easy to operate to the upper part of the screen. In addition, with respect to the distance D, the smaller the value set, the more sensitive it can detect the movement of the finger and make the cursor follow, but it is also possible to detect the situation where the user accidentally moves the finger slightly and cause the cursor to jump and move frequently. On the other hand, with respect to the distance D, the larger the value set, the less smooth the movement of the cursor becomes and becomes intermittent. By being able to adjust D using settings, it is possible to provide an opportunity to achieve an operational feel that is suitable for the user.

[0033] By setting it as above, the user can move the cursor to the desired position. The user can execute the processing corresponding to the object indicated by the cursor by overlapping the cursor on the position of the object where the operation target is displayed and performing a predetermined operation. In such a case, the processor 120 functions as an execution unit. The predetermined operation can be various according to the mode of the touch panel display, the operating system, and the application. For example, it can be configured so that if the finger is not moved for a predetermined time in the state where the cursor is overlapped on the object of the operation target, the processing corresponding to the object is performed, and it can also be configured so that when the finger is detected to have left the state where the cursor overlaps with the object, the processing corresponding to the object is performed. Alternatively, it can also be configured so that when a single click or double click is detected in the state where the cursor overlaps with the object, the processing corresponding to the object is performed. In addition, in the case where the pressing force on the touch panel can be detected, it can also be configured so that when the finger is pressed strongly in the state where the cursor overlaps with the object of the operation target, the processing corresponding to the object is performed. According to the mode of the touch panel display, the operating system, and the application, what kind of processing is performed as the processing corresponding to the object can also be various. There can also be multiple predetermined operations and different processing for each operation.

[0034] In addition, when the cursor reaches the end of the cursor moving range, the processor 120 makes the cursor not to be displayed. The cursor moving range is the range in which the cursor can be displayed. Figure 4In this case, area g1 corresponds to the moving range of the cursor. The processor 120 sets an area g2 corresponding to the virtual screen outside the moving range of the cursor, i.e., area g1. Even if the cursor reaches the end of area g1, the cursor position can be calculated in the same way as above, and the cursor is considered to have moved to area g2 outside area g1. Since area g2 outside area g1 is an area not displayed on the touch panel display 141, the cursor is not displayed. For example, by touching the position from Figure 4 In state 4a, the cursor moves upward on the paper surface, and as shown in state 4b, the cursor moves outside the area g1 and becomes invisible. In this way, when the cursor reaches the end of the cursor movement range and is set to be invisible, there is no contradiction between the movement of the finger until it reaches the end and the position of the cursor, thereby preventing the user from feeling uncomfortable.

[0035] Even if the cursor is not displayed because it moves outside the area g1, the user can move the cursor to the target position in the area g1 by moving the finger toward the target position in the area g1. Figure 4 In the case where the cursor moves from the state 4b to the upper left end P1, when the finger is slid from the state of touching A3 to the direction of the upper left end P1, the cursor is displayed on the straight line P1A3. By sliding the finger on the straight line P1A3, the cursor reaches the upper left end P1 before the finger reaches the upper left end P1 of the region g1, so the cursor can be overlapped at the upper left end P1. Regardless of which position in the region g1 is set as the target position, the user can overlap the cursor at the target position by touching a position separated from the target position and moving the finger toward the target position.

[0036] In addition, assuming that in the case of a structure in which the relative position of the cursor based on the touch position of the finger is fixed, an area in which the cursor cannot be moved is generated in the area g1. In addition, assuming that in the case of a structure in which the relative position of the cursor based on the touch position of the finger is changed by operation of two fingers, etc., the cursor can be moved to any position in the area g1, but it will be troublesome to change the relative position by operation with two fingers. According to this embodiment, the processor 120 can display the cursor at a position that is a predetermined distance away from the touch position of the finger on the extension line of the sliding direction of the finger. Since the relative position of the cursor based on the finger can be changed according to the movement direction of the finger by following the movement of the finger, the cursor can be easily moved to a desired position without performing a special operation for changing the relative position. In addition, since the cursor displayed at a position different from the touch position of the finger can be overlapped with the object to perform processing corresponding to the object, more sophisticated operations can be performed compared to the touch operation mode.

[0037] (2) Cursor display processing:

[0038] The processor 120 can enable the computer to implement the above-mentioned cursor display function by executing the operation program recorded in the non-volatile memory 130 . Figure 5 1 is a flowchart showing the cursor display process. The cursor display process starts when the cursor operation mode is changed. When the cursor display process starts, the processor 120 waits until a touch is detected (step S100), and when a touch is detected, the cursor is displayed at a position B1 that is a predetermined distance C from the detected touch position A1 (step S105).

[0039] Next, the processor 120 waits until a displacement of the touch position is detected (step S110). That is, based on the touch position that serves as the reference for the position of the cursor currently displayed, it is determined whether the latest touch position has changed. In step S110, if it is determined that a change in the touch position is detected, the processor 120 determines whether the displacement amount is greater than D (step S115). The displacement amount is the straight-line distance from the touch position that serves as the reference for the cursor position currently displayed to the latest touch position. The touch position that serves as the reference for the cursor position currently displayed is referred to as A. n , the distance A n The latest touch position that is greater than the distance D is called A. n+1 .

[0040] In step S115, if it is not determined that the displacement amount is greater than D, the processor 120 returns to step S110. In step S115, if it is determined that the displacement amount is greater than D, the processor 120 obtains the latest touch position A. n+1 The circumference of a circle with radius C as its center is n A n+1 The displacement direction of the position A n+1 The intersection point B between the positive extension lines of the reference n+1 , as the cursor position (step S120). Next, the processor 120 determines whether the cursor position is within the region g1 (step S125), and displays the cursor at the cursor position when it is within the region g1 (step S130), and does not display the cursor when it is not within the region g1 (step S135). That is, in step S130, the cursor is displayed within the region g1, and in step S135, the cursor is not displayed within the region g1.

[0041] After executing step S130 or step S135, the processor 120 determines whether the finger has left (step S140), and returns to step S110 if it is not determined that the finger has left (the touch state continues). When it is determined that the finger has left in step S140, the processor 120 makes the cursor not displayed (step S145) and returns to step S100. As a result, the cursor is not displayed in area g1. In addition, the processing when changing from the touch state to the non-touch state can adopt various methods according to the method of the predetermined operation for performing the processing corresponding to the object indicated by the cursor. For example, it can also be set as follows, that is, when it is configured to perform the processing corresponding to the object when the finger is not moved in the state of overlapping the cursor with the object of the operation target and for a predetermined time, or when it is configured to perform the processing corresponding to the object when the finger is detected to leave the state where the cursor overlaps with the object, as shown in step S145, the cursor is not displayed according to the situation of becoming a non-touch state.

[0042] In addition, for example, it is also possible to configure the cursor display mode to be different between the state where the finger is touching and the state where the finger is not touching. In such a case, the processor 120 displays the cursor in a display mode indicating the touch state in steps S105 and S130. Then, when it is determined in step S140 that the finger has left, when the latest cursor position is within the area g1, the cursor display mode within the area g1 is changed from a mode indicating that the finger is in the touch state to a mode indicating that the finger is not in the touch state, and the process returns to step S100. The structure of changing the display mode of the cursor between the touch state and the non-touch state in this way can be adopted, for example, in the case of the following configuration, that is, when a single click or double click is detected in a state where the cursor overlaps with the object, processing corresponding to the object is performed.

[0043] Alternatively, the cursor display mode may not change between the touch state and the non-touch state. In this case, when it is determined in step S140 that the finger has left, the processor 120 maintains the state of step S130 or step S135 and returns to step S100.

[0044] Through the above-described processing, when the cursor is displayed and a predetermined operation is detected when the cursor points to a certain object, the processor 120 executes a process corresponding to the object.

[0045] (3) Other implementation methods:

[0046] The above embodiment is an example for implementing the present invention, and various other embodiments can also be adopted. For example, the present invention can be installed in an operating system of an electronic device, or can be implemented in a specific application program.

[0047] The cursor can be Figure 6 In this case, the direction of the arrow mark can be consistent with the direction of the finger movement. Figure 6 When the touch position moves from A1 to A2 as shown in states 6a and 6b, the control is performed so that an arrow mark that is in the same direction from A1 to A2 is displayed. When the touch position moves from A2 to A3 as shown in states 6b and 6c, an arrow mark that is in the same direction from A2 to A3 is displayed. When the user moves the finger slightly, the user may think that the direction of movement is different from the detected direction of movement. In such a case, by making the direction of the arrow mark of the cursor consistent with the direction of movement of the finger as in this example, the user can easily identify the direction of movement detected by the computer through the direction of the arrow mark.

[0048] In the above embodiment, although an example is given in which the cursor jumps to a new relative position according to a change in the movement direction of the finger, the present invention is not limited to this. For example, although the tracking performance is deteriorated, in order to easily understand where the cursor has moved from the previous display position, an animation of the cursor moving to the position of the cursor after the movement can be displayed, or a line or arrow mark indicating the movement trajectory from the cursor position before the movement to the cursor position after the movement can be displayed within a certain period after the movement on the basis of displaying the cursor after the movement. By displaying such a trajectory for a certain period of time while displaying the cursor after the movement, users who have already grasped the position of the cursor after the movement will not feel that the tracking performance has deteriorated, and users who have not yet grasped the position of the cursor after the movement can easily follow the trajectory to find the display position of the cursor after the movement.

[0049] In addition, the present invention can be applied to various electronic devices in addition to tablet terminals and smart phones. It can also be applied to operating panels of personal computers, electronic blackboards, and multifunction machines. In addition, the present invention can also be applied as a program or method executed by a computer. For example, the present invention is also established as an invention of an operation method including the following operations, that is, the position of the user's finger is detected, multiple objects and cursors corresponding to both the position of the finger and the movement direction of the finger are displayed, and processing corresponding to the object indicated by the cursor is performed. Such an operation method is not limited to electronic devices with touch panel displays, and can also be used in electronic devices without touch panel displays. The movement of the finger can be detected as movement in a two-dimensional plane or as movement in a three-dimensional space. The electronic device that detects the movement of the display and the finger can also be separate. In addition, the display is not necessarily required, and it can be a way for the object and the cursor to be projected onto the screen or in the air. The processing corresponding to the object can be an indication to an external device different from the electronic device that detects the movement of the finger, or it can be a structure in which the electronic device that detects the movement of the finger sends an indication command corresponding to the object to the external device to perform a predetermined processing.

[0050] In addition, the present invention also exists as an invention of an operating program that enables a computer to perform the following operations, namely, detecting the position of a user's finger, displaying multiple objects and cursors corresponding to both the position of the finger and the direction of movement of the finger, and performing processing corresponding to the object indicated by the cursor.

[0051] In addition, the above-mentioned systems, programs, and methods may be implemented by a single device or by components possessed by multiple devices, including various methods. In addition, such as being implemented in part by software and in part by hardware, appropriate changes can be made. In addition, the invention also exists as a recording medium for a program that controls the system. Of course, the recording medium of the program may be a magnetic recording medium or a semiconductor memory, and any recording medium developed in the future may be considered to be exactly the same.

[0052] Explanation of symbols

[0053] 100 ... electronic device; 120 ... processor; 130 ... nonvolatile memory; 140 ... UI unit; 141 ... touch panel display; 150 ... communication unit; A1, A2, A3 ... touch positions; B1, B2, B3 ... cursor positions.

Claims

1. A computer having: A touch panel display that displays multiple objects and a cursor that follows the finger; an execution unit that executes a process corresponding to the object indicated by the cursor, The relative position of the cursor with respect to the finger is changed so that the cursor is located forward in the direction in which the finger moves.

2. The computer according to claim 1, wherein: The distance between the finger and the cursor is fixed.

3. The computer according to claim 2, wherein: When the cursor reaches the end of the moving range of the cursor, the cursor is hidden.

4. The computer according to claim 2, wherein: The cursor is displayed at an intersection of a line extending in a moving direction of the finger and a circle centered at a touch position of the finger.

5. The computer according to claim 2, wherein: According to the change in the moving direction of the finger, the cursor jumps to the new relative position.

6. The computer according to claim 1, wherein: The cursor is set to be in the shape of an arrow mark, and the direction of the arrow mark is consistent with the direction of movement of the finger.

7. An operation method, comprising the following operations, namely, Detect the position of the user's finger. Displaying a plurality of objects and a cursor corresponding to both the position of the finger and the moving direction of the finger, A process corresponding to the object indicated by the cursor is executed.

8. An operating program product that enables a computer to execute the following operations, namely, Detect the position of the user's finger. Displaying a plurality of objects and a cursor corresponding to both the position of the finger and the moving direction of the finger, A process corresponding to the object indicated by the cursor is executed.

Images