Information processing device, information processing method, and program

Abstract

An information processing device according to the present technology comprises: an unevenness detection unit that detects unevenness information about an operation region used by a user for a touch operation on a region to be operated, which is a region on a screen; a determination processing unit that determines, on the basis of the unevenness information detected by the unevenness detection unit, whether the operation region is a first operation region which is a prescribed partial region of a hand of the user; and a correspondence processing unit that executes a prescribed correspondence process when it is determined that a region other than the first operation region of the hand of the user has been used for the touch operation.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program technology for acquiring unevenness information about a hand that touches an operation unit.

[0002] 2. Description of the Related Art Portable devices such as smartphones and tablet terminals employ a touch panel system in which the display unit also functions as an operation unit in order to eliminate operation buttons and operation devices (see, for example, Patent Document 1).

[0003] JP 2009-086184 A

[0004] To use multiple functions on such devices, various methods have been used, such as dividing the operation area on the display by providing an icon for each function, dividing the operation types (such as tapping and swiping), or dividing the hierarchy by function (such as menu items). However, as the number of functions increases, the number of icons increases, narrowing the operation area, increasing the number of operation types, and increasing the number of operations required to execute a function, which has not necessarily resulted in high usability. Furthermore, devices that use touch panels have the problem of causing the device to behave unintendedly if the user touches the touch panel unintentionally.

[0005] Therefore, an object of the present technology is to improve the usability of operations related to touch panels.

[0006] The information processing device according to the present technology includes: a concavo-convex detection unit that detects concavo-convex information about a user's operation area used for a touch operation on an operated area that is an area on a screen; a determination processing unit that determines whether the operation area is a first operation area that is a predetermined partial area of ​​the user's hand based on the concavo-convex information detected by the concavo-convex detection unit; and a response processing unit that executes a predetermined response process when it is determined that an area of ​​the user's hand other than the first operation area has been used for the touch operation. For example, the first operation area is an area that is normally used for touch operations. The area other than the first operation area is an area that is not normally used for touch operations.

[0007] 1 is a diagram showing the appearance of a smartphone terminal which is one aspect of an information processing device according to an embodiment of the present technology. FIG. 2 is a block diagram showing an example of a hardware configuration of the information processing device. FIG. 3 is a functional block diagram of a smartphone terminal according to a first embodiment. FIG. 4 is a diagram showing an example in which only an operated area on a display unit is made an emitting area. FIG. 5 is a diagram showing examples of a first operation area and a second operation area. FIG. 6 is a diagram showing a state in which a touch-down operation has been performed in a swipe operation. FIG. 7 is a diagram showing a state in which a slide operation has been performed in a swipe operation. FIG. 8 is a diagram showing a state in which a touch-up operation has been performed in a swipe operation. FIG. 9 is a diagram showing an example in which a specific function is executed separately depending on the direction of the swipe operation. FIG. 10 is a diagram showing an example in which a touch-up operation without a slide operation is detected in addition to the direction of the swipe operation, and a specific function is executed separately. FIG. 11 is a flowchart showing an example of processing executed by a CPU of the smartphone terminal according to the first embodiment. FIG. 12 is a flowchart showing an example of unevenness imaging processing. FIG. 13 is a diagram showing an example in which a specific function is executed separately depending on an operation area during a swipe operation. 10 is a diagram showing an example of an operated area in a touch operation using the ball area and distal joint area of ​​the thumb. FIG. 11 is a flowchart showing an example of processing executed by the CPU of the smartphone terminal in the second embodiment. FIG. 12 is an explanatory diagram showing an example where the detected operated area includes a second operation area. FIG. 13 is a flowchart showing an example of unevenness imaging processing in a modified example of the second embodiment. FIG. 14 is a diagram showing an example where the shape of the second operation area varies depending on the position on the display unit. FIG. 15 is a flowchart showing an example of processing executed by the CPU of the smartphone terminal in the second embodiment. FIG. 16 is a functional block diagram of a smartphone terminal in a modified example of the third embodiment. FIG. 17 is a diagram showing an example of an allocation screen. FIG. 18 is a diagram showing a state where some functions are allocated on the allocation screen. FIG. 19 is a diagram showing a state where a confirmation screen is superimposed.10 is a flowchart showing an example of processing executed by the CPU of a smartphone terminal to assign a function. FIG. 11 is a flowchart showing an example of processing executed by the CPU of a smartphone terminal in a modified example of the third embodiment. FIG. 12 is a flowchart showing another example of processing executed by the CPU of a smartphone terminal in a modified example of the third embodiment. FIG. 13 is a diagram showing another example of a specific function. FIG. 14 is a diagram showing an example of an image displayed in the operation area for registering concavo-convex information. FIG. 15 is a diagram showing a state in which a hand is placed in line with the image. FIG. 16 is an example of an image for performing an operation of lifting the palm. FIG. 17 is an example of an image for performing a procedure in which fingertips touching the operation area are brought closer to each other. FIG. 18 is a diagram showing an example of starting registration without the palm touching. FIG. 19 is an explanatory diagram for an operation of sliding the thumb forward. FIG. 19 is a diagram for explaining a change in the operation area for the thumb when sliding the thumb forward.

[0008] The embodiments will be described below in the following order: <1. Configuration of information processing device> <2. First embodiment> <2-1. Functional configuration> <2-2. Processing example> <2-3. Modified example> <3. Second embodiment> <3-1. Functional configuration> <3-2. Processing example> <3-3. Modified example> <4. Third embodiment> <4-1. Modified example> <5. Other modified examples> <6. Registration process of unevenness information> <7. Summary> <8. This technology>

[0009] <1. Configuration of Information Processing Device> A configuration example of an information processing device 1 according to the present technology will be described. Note that, here, a smartphone terminal 1A as a mobile terminal device is given as an example of the information processing device 1, but implementation of the present technology is not limited to this. For example, the information processing device 1 may be a mobile terminal device such as a tablet terminal in addition to the smartphone terminal 1A. Furthermore, although the smartphone terminal 1A is configured to include both a control unit and a display unit, the present technology can also be applied to an information processing device 1 such as a personal computer that includes a control unit and a separate display unit. Furthermore, the present technology can also be applied not only to a portable information processing device 1 but also to a stationary information processing device 1. In other words, the smartphone terminal 1A is merely one aspect for implementing the information processing device 1 of the present technology.

[0010] 1 shows the appearance of a smartphone terminal 1A as an information processing device 1. The smartphone terminal 1A is configured with a box-shaped housing 2 having an internal space and various components arranged inside and outside, a display 3 attached so as to be fitted into an opening formed in the housing 2, and various controls (not shown).

[0011] In addition, sensors that detect user operations are provided in the housing 2 and the display 3. For example, the sensor provided in the display 3 detects that the user has performed a touch operation or a swipe operation on the display 3. In other words, the display 3 functions as an operation unit 4 such as a so-called touch panel.

[0012] The display unit 3 serving as the operation unit 4 is a touch panel equipped with a full-screen fingerprint authentication function. The display unit 3 serving as a touch panel equipped with a full-screen fingerprint authentication function has sensors arranged over substantially the entire operation area of ​​the display unit 3 for reading fingerprints. This allows the fingerprint to be read and authenticated when the user performs a touch operation or the like at any position on the display unit 3.

[0013] In the smartphone terminal 1A, when a fingertip touches the operation unit 4, the fingerprint of the touching finger can be read as unevenness information. In addition, in the smartphone terminal 1A, when the pad of a finger or the distal joint portion serving as the first joint of a finger is touched to the operation unit 4, the fingerprint of the touched portion and the wrinkle shape of the distal joint can be read as unevenness information. Note that in the smartphone terminal 1A, when the palm portion excluding the fingers is touched to the operation unit 4, the wrinkles of the palm portion can also be read as unevenness information.

[0014] The smartphone terminal 1A appropriately executes processes such as registering the read fingerprint or wrinkle information as unevenness information, and collating the read unevenness information with the registered unevenness information.

[0015] In the following description, information indicating the undulations of fingerprints on fingers and wrinkles on knuckles and palms will be collectively referred to as "unevenness information."

[0016] In the process of detecting unevenness information, a light emitting section and a light receiving section arranged on the lower part (inner side) of the display section 3 are used.

[0017] The light-emitting unit emits visible light from inside the housing 2 toward the surface of the display unit 3. The light-receiving unit receives light reflected from the surface of the display unit 3 toward the interior of the smartphone terminal 1A. Depending on the amount of light reflected by the light-receiving unit, the ridges that form the fingerprint or the valleys between the ridges, or the raised and valley portions of the wrinkles, are detected.

[0018] The light-emitting units used for detecting unevenness information are the same as the light-emitting units used for displaying various images on the display unit 3. By using the same light-emitting units for image display and for detecting unevenness information, the number of light-emitting units can be reduced, which contributes to cost reduction.

[0019] In the following description, for convenience, the series of processes relating to light emission by the light-emitting unit and light reception by the light-receiving unit will be referred to as the "unevenness imaging process." Furthermore, unevenness information such as fingerprints obtained by the unevenness imaging process may also be referred to as the "unevenness image." The unevenness image does not need to be expressed in a specific image format, as long as it represents unevenness information of the hand.

[0020] The full-screen fingerprint authentication function of the display unit 3 does not have to be realized by including a light-emitting unit and a light-receiving unit. For example, the full-screen fingerprint authentication function of the display unit 3 may be realized by a capacitance method or an ultrasonic method.

[0021] The hardware configuration of an information processing device 1, such as a smartphone terminal 1A, will be described with reference to FIG. 2 . The information processing device 1 does not need to include all of the components described below; it may include only some of them. The CPU (Central Processing Unit) 71 of the information processing device 1 executes various processes according to programs stored in a ROM (Read Only Memory) 72 or a nonvolatile memory unit 74, such as an EEP-ROM (Electrically Erasable Programmable Read-Only Memory), or programs loaded from a storage unit 79 to a RAM 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes, as appropriate. The CPU 71, ROM 72, RAM 73, and nonvolatile memory unit 74 are interconnected via a bus 83. An input / output interface 75 is also connected to this bus 83.

[0022] An input unit 76 consisting of operators and operation devices is connected to the input / output interface 75. For example, the input unit 76 may be various operators and operation devices such as a keyboard, a mouse, keys, a dial, the above-mentioned touch panel, a touch pad, a remote controller, etc. The input unit 76 detects user operations, and the CPU 71 interprets signals corresponding to the input operations.

[0023] A display unit 77 (including the display unit 3 in FIG. 1 ) consisting of an LCD or an organic EL panel, etc., and an audio output unit 78 consisting of a speaker, etc., are connected to the input / output interface 75 either integrally or separately. The display unit 77 is a display unit that displays various types of information, and is configured, for example, by a display device provided in the housing of the computer device or a separate display device connected to the computer device. The display unit 77 displays images for various types of image processing, moving images to be processed, etc., on the display screen based on instructions from the CPU 71. The display unit 77 also displays various operation menus, icons, messages, etc., i.e., a graphical user interface (GUI), based on instructions from the CPU 71. That is, the CPU 71 functions as a display control unit that causes the display unit 77 to display predetermined images.

[0024] The input / output interface 75 may be connected to a storage unit 79 configured with a hard disk or solid-state memory, or a communication unit 80 configured with a modem or the like.

[0025] The communication unit 80 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, and the like.

[0026] A drive 81 is also connected to the input / output interface 75 as needed, and a removable storage medium 82 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately attached. The drive 81 allows data files such as image files and various computer programs to be read from the removable storage medium 82. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the audio output unit 78. In addition, computer programs and the like read from the removable storage medium 82 are installed in the storage unit 79 as needed.

[0027] In this computer device, for example, software for the processing of this embodiment can be installed via network communication by the communication unit 80 or via a removable storage medium 82. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, or the like.

[0028] The CPU 71 performs processing operations based on various programs, thereby realizing various processes described below in the smartphone terminal 1A. Note that the information processing device 1 is not limited to being configured with a single computer device as shown in FIG. 2 , but may also be configured with a system of multiple computer devices. The multiple computer devices may be systemized using a LAN or the like, or may be located in a remote location via a VPN or the like using the Internet or the like. The multiple computer devices may include computer devices serving as a server group (cloud) available through a cloud computing service.

[0029] 2. First Embodiment 2-1. Functional Configuration FIG. 3 shows the functional configuration realized by the CPU 71 of the smartphone terminal 1A executing a program.

[0030] By executing a program, the CPU 71 of the smartphone terminal 1A functions as an operated area detection unit F1, a specific operation detection unit F2, a concave / convex detection unit F3, a judgment processing unit F4, a response processing unit F5, and a UI (User Interface) processing unit F6.

[0031] The operated area detection unit F1 performs processing to detect and identify an area on the display unit 3 of the information processing device 1 that has been touched by the user. In the following description, the area on the display unit 3 that has been touched by the user will be referred to as the "operated area ArD."

[0032] The detection of the operated area ArD by the operated area detection unit F1 is performed by detecting a change in capacitance between the user's hand and the display unit 3, for example.

[0033] The specific operation detection unit F2 determines whether a specific operation has been performed in the operated area ArD detected by the operated area detection unit F1. For example, if the specific operation is a double tap, the specific operation detection unit F2 determines whether the operated area ArD has been detected twice in a short period of time, whether the two detected operated areas ArD are substantially the same area, and so on.

[0034] The specific operation may be a triple tap other than a double tap, or a long tap where the user presses and holds the touch panel for a predetermined period of time or longer. The specific operation detection unit F2 may determine that a specific operation has been performed when it detects that a specific operation has been performed in a predetermined area on the display unit 3.

[0035] When the specific operation detection unit F2 detects a specific operation, the unevenness detection unit F3 detects unevenness information about the user's hand that touches the operated area ArD. The detection of unevenness information is realized by the unevenness imaging process described above.

[0036] In addition, by configuring the unevenness imaging process to be executed when a specific operation is detected by the specific operation detection unit F2, and not to be executed in other cases, the number of times the unevenness imaging process is executed can be reduced, thereby suppressing power consumption.

[0037] In the following description, the area of ​​the user's hand that touches the operated area ArD is referred to as an operation area ArH.

[0038] The unevenness imaging process by the unevenness detection unit F3 is realized by illuminating the display unit 3 with white light. At this time, the unevenness detection unit F3 designates only a specific area on the display unit 3, specifically the operated area ArD, as the light-emitting area, as schematically shown in FIG. 4 . Furthermore, light is emitted in the unevenness imaging process only while the user's hand is touching the display unit 3. In other words, the unevenness detection unit F3 performs the unevenness imaging process by illuminating only the touched area and only while the user's hand is touching the display unit 3. This makes it possible to shorten the time that the light-emitting unit emits white light, thereby suppressing an increase in power consumption of the smartphone terminal 1A.

[0039] Furthermore, since only the area of ​​the display unit 3 that is hidden by the operation area ArH is illuminated in white light, leakage of white light from the gap between the finger and the display unit 3 is suppressed, and the user does not feel excessive glare or discomfort in a dark room at night, etc.

[0040] The determination processing unit F4 determines whether or not the operation area ArH used for the touch operation is a specific area.

[0041] In the present embodiment, the operation area ArH is exemplified by a first operation area ArH1 and a second operation area ArH2. The first operation area ArH1 is, for example, an area that is frequently used by the user for touch operations on the display unit 3. The first operation area ArH1 is, for example, the area of ​​the tip of the index finger or the tip of the thumb.

[0042] Alternatively, all five fingertip regions of the user's right hand (dominant hand) may be set as the first operation area ArH1, or all ten fingertip regions of both hands may be set as the first operation area ArH1, or the fingertip region of a specific finger of the user may be set as the first operation area ArH1.

[0043] The second operation area ArH2 is, for example, an area that is not often used by the user for touch operations on the display unit 3. The second operation area ArH2 is, for example, an area from the pad area of ​​the index finger or thumb to the distal phalanx (first joint). In other words, the area near the distal phalanx of the index finger may be the second operation area ArH2.

[0044] Alternatively, the pad regions or distal joint regions of all five fingers of the user's right hand (dominant hand) may be set as the second operation area ArH2, or the pad regions or distal joint regions of all ten fingers of both hands may be set as the second operation area ArH2, or the pad region or distal joint region of a specific finger of the user may be set as the second operation area ArH2.

[0045] Alternatively, the second operation area ArH2 may be the palm area of ​​the user or the joint area on the back of the hand. Alternatively, the entire area of ​​the user's hand other than the first operation area ArH1 may be the second operation area ArH2.

[0046] In the following description, an example will be given in which the tip region of the user's thumb is the first operation region ArH1, and the ball region or distal region of the thumb is the second operation region ArH2 (see FIG. 5).

[0047] The determination processor F4 determines whether the operation area ArH used for the touch operation is an area other than the first operation area ArH1. In other words, the determination processor F4 determines whether the operation area ArH is an area used for normal touch operations.

[0048] If the unevenness information detected by the unevenness detection unit F3 matches the unevenness information for the second operation area ArH2 stored in the memory unit 79, it is determined that an operation has been performed in an area other than the first operation area ArH1.

[0049] This determination process is realized, for example, by comparing the unevenness information detected by the unevenness detection unit F3 with the unevenness information of the first operation area ArH1 and the unevenness information of the second operation area ArH2 stored in the memory unit 79.

[0050] When the judgment processing unit F4 determines that the unevenness information detected by the unevenness detection unit F3 matches the unevenness information of the second operation area ArH2, the response processing unit F5 executes a response processing different from the normal tap processing.

[0051] The response processing unit F5 performs a command operation detection process and a start / stop process as response processes.

[0052] The instruction operation detection process is a process of detecting a user operation after a specific operation such as a double tap or triple tap as an instruction operation and grasping the content of the instruction given by the user.

[0053] The start / stop process is a process for starting or stopping a specific application or function.

[0054] For example, suppose that a user performs a swipe operation to the right after performing a specific operation (see FIGS. 6 and 7 ). The swipe operation can be decomposed into a touch-down operation in which the user touches the display unit 3 with a finger, a slide operation in which the user slides the finger in one direction while the finger is touching the display unit 3, and a touch-up operation in which the user removes the finger from the display unit 3.

[0055] FIG. 7 shows the relationship between the thumb and the display unit 3 after a slide operation, and FIG. 8 shows the relationship between the thumb and the display unit 3 after a touch-up operation.

[0056] 6 to 8 after the specific operation, the CPU 71 of the smartphone terminal 1A determines that a rightward swipe operation has been performed as an instruction operation. At this time, the handling processing unit F5 performs a process of starting up a specific application as a start / stop process in the handling process.

[0057] In the instruction operation detection process, several types of instruction operations are detected and distinguished. For example, as shown in Fig. 9 , in the instruction operation detection process by the response processing unit F5, an upward swipe operation, a downward swipe operation, a rightward swipe operation, and a leftward swipe operation are detected and distinguished from each other, starting from the operated area ArD where the specific operation was performed.

[0058] Then, in the start-stop processing by the corresponding processing unit F5, application App1 is started when an upward swipe operation is detected, application App2 is started when a downward swipe operation is detected, application App3 is started when a rightward swipe operation is detected, and application App4 is started when a leftward swipe operation is detected.

[0059] In this way, when a specific operation is performed using the second operation area ArH2 that is not used in normal operations, the response processing unit F5 detects a subsequent instruction operation (swipe operation) as a response process, and starts or stops an application according to the instruction operation. Alternatively, the response processing unit F5 may execute a response process when an instruction operation is performed using the second operation area ArH2 that is not used in normal operations after a specific operation is performed.

[0060] In the start / stop process, a process of stopping a currently running application may be executed. For example, a start / stop process corresponding to an upward or left / right swipe operation executes a process of starting a specific application, and a start / stop process corresponding to a downward swipe operation executes a process of stopping a specific application.

[0061] The start-up / stop processing may also include processing for setting the smartphone terminal 1A to a specific state, such as muting sound output from the smartphone terminal 1A, or shutting down the smartphone terminal 1A. Alternatively, the start-up / stop processing may include restart processing for successively shutting down and starting up the smartphone terminal 1A. In the following description, a specific application or a predetermined function that is to be started or stopped by the start-up / stop processing performed by the response processing unit F5 will be referred to as a "specific function."

[0062] The activation / stop process may include a process of activating the same application in different modes. For example, a payment application that enables cashless payment may have a first payment mode in which a camera for reading a two-dimensional code is activated, and a second payment mode in which a two-dimensional code for payment is displayed on the display unit 3.

[0063] The corresponding processing unit F5 may launch the payment application in a first payment mode when a swipe operation to the right is detected during the instruction operation detection process, and may launch the payment application in a second payment mode when a swipe operation to the left is detected.

[0064] In addition, in the instruction operation detection process executed by the response processing unit F5, an operation in which a touch-up operation is performed without a swipe operation may be detected as a single instruction operation. In this case, as shown in Fig. 10, five types of instruction operations are detected in the instruction operation detection process, and five types of applications are started or stopped in the start-stop process. Note that Fig. 10 shows an example in which application App5 is started in the start-stop process when an operation in which a touch-up operation is performed without a swipe operation is detected.

[0065] The UI processing unit F6 performs user interface processing for displaying on the display unit 3 icon images for starting applications, various controls (buttons), various notifications, and the like.

[0066] The UI processing unit F6 performs processing to provide a user interface for detecting and registering the uneven shape of a fingerprint, etc. The uneven shape information about the user's hand acquired via the user interface is stored in, for example, the storage unit 79, and is used for subsequent processing to verify the fingerprint, etc.

[0067] In the present embodiment, the unevenness information for both the first operation area ArH1 and the second operation area ArH2 is stored in the storage unit 79. In the following description, the unevenness information for the first operation area ArH1 is referred to as first unevenness information Das1, and the unevenness information for the second operation area ArH2 is referred to as second unevenness information Das2.

[0068] The UI processing unit F6 may perform user interface processing for issuing a notification to prompt fingerprint registration.

[0069] The UI processing unit F6 performs processing to provide a user interface for linking the type of instruction operation following a specific operation with the corresponding function. Specifically, the UI processing unit F6 performs processing to display a screen that allows the user to select the correspondence between the type of instruction operation detected in the instruction operation detection processing and the function to be activated or deactivated in the activation / deactivation processing.

[0070] 11 and 12 show examples of processing executed by the CPU 71 of the smartphone terminal 1A. Note that the order of processing shown in each figure is merely an example, and the order of processing may be reversed as appropriate.

[0071] The CPU 71 of the smartphone terminal 1A performs a process of detecting the operated area ArD in step S101, thereby realizing the function of the operated area detection unit F1.

[0072] The CPU 71 realizes the function of the specific operation detection unit F2 by determining whether a specific operation such as a double tap or a triple tap has been detected in step S102. Note that the determination process in step S102 is based on the results of the process in step S101, which is continuously executed up to that point, i.e., the results of continuous detection of the operated area ArD. This allows specific operations that require a certain amount of time, such as a double tap or a triple tap, to be detected in step S102.

[0073] If it is determined that a specific operation has not been detected, the CPU 71 proceeds to step S103 and executes processing according to the operation content. Specifically, if the detected operation content is a tap operation on an icon, the CPU 71 executes processing to launch a predetermined application linked to the icon. Furthermore, if a tap operation on a button UI is detected, the CPU 71 executes an animation of the button UI being pressed and activates a function linked to the button.

[0074] After completing the process of step S103, the CPU 71 returns to step S101. That is, the CPU 71 monitors the user's operation while continuously executing the process of step S101, and executes the process of step S103 as appropriate depending on the type of operation detected. Then, if it is determined in step S102 that a specific operation has been detected, the CPU 71 proceeds to step S104 and executes an unevenness imaging process to detect unevenness information about the user's hand.

[0075] FIG. 12 shows the specific processing contents of the unevenness imaging processing in step S104.

[0076] In the unevenness imaging process, the CPU 71 identifies the operated area ArD in step S201. If the operated area ArD has been detected in the immediately preceding step S101 in FIG. 11, the CPU 71 may avoid the process of step S201.

[0077] In the next step S202, the CPU 71 specifies the light-emitting element to be used for emitting light. This is processing for specifying the light-emitting element that will emit light to the specified operated area ArD.

[0078] The process of step S202 can be realized, for example, by the CPU 71 executing the following process: First, the CPU 71 generates a white image having substantially the same size and shape as the area detected in step S101. Next, the CPU 71 displays the generated white image so that it overlaps the detected operated area ArD.

[0079] In step S203, the CPU 71 causes the light-emitting unit of the light-emitting target to emit white light, thereby enabling the user's fingerprint or the like to be suitably detected.

[0080] In step S204, the CPU 71 detects concave / convex information about the user's operation area ArH.

[0081] In step S205, the CPU 71 determines whether or not the concavo-convex information of the operation area ArH has been acquired. If it is determined that the information has not been acquired (step S205: No determination), the CPU 71 returns to step S201 again. The reason for executing the process of step S201 again at this time is that it is possible that the user's finger or the like is moving on the display unit 3. Furthermore, since it takes a certain amount of time from when the user's finger begins to touch the display unit 3 until the contact state stabilizes, it is preferable to repeat the processes of steps S201 to S205 for several hundred milliseconds to several seconds.

[0082] On the other hand, if it is determined in step S205 that the unevenness information has been acquired (step S205: Yes determination), the CPU 71 ends the unevenness imaging process shown in FIG. 12 and proceeds to step S105 in FIG.

[0083] If unevenness information cannot be acquired for a predetermined time in the series of processes shown in Fig. 12, the CPU 71 may forcibly end the series of processes shown in Fig. 12. In this case, the CPU 71 may return to the process of step S101 in Fig. 11, thereby avoiding the processes from step S105 onwards.

[0084] 11, the CPU 71 compares the unevenness information detected in step S104 with the first unevenness information Das1 and the second unevenness information Das2 stored in the storage unit 79, and determines whether the detected unevenness information matches the second unevenness information Das2. Note that "match" here means that the detected unevenness information and the second unevenness information Das2 match in at least a partial area.

[0085] If it is determined that the detected unevenness information does not match the second unevenness information Das2 stored in the storage unit 79 (step S105: No determination), the CPU 71 returns to step S101.

[0086] On the other hand, if it is determined that the detected unevenness information matches the second unevenness information Das2 stored in the storage unit 79 (step S105: Yes determination), the CPU 71 proceeds to step S106 and executes an instruction operation detection process, which is part of the response process. As a result, the CPU 71 detects a swipe operation or the like by the user.

[0087] In step S107, the CPU 71 determines whether there is a process corresponding to the detected instruction operation. If it is determined that there is no corresponding process, i.e., that there is no function to be executed (step S107: No determination), the CPU 71 returns to step S101.

[0088] If it is determined that there is a corresponding process, that is, that there is a function to be executed (step S107: Yes determination), the CPU 71 activates the specific function in step S108.

[0089] After completing the process of step S108, the CPU 71 returns to the process of step S101 again. By executing the series of processes shown in Fig. 11 and 12, the CPU 71 of the smartphone terminal 1A can detect, for example, that the user has performed a double-tap operation as a specific operation and then performed a swipe operation using the pad of the finger, and can thereby activate a specific function.

[0090] There are several possible modes of a swipe operation following a double tap operation. For example, a touch-down operation of the swipe operation may be performed with a finger or the like removed from the display unit 3 after a double tap operation. Alternatively, a slide operation of the swipe operation may be performed with a finger or the like in contact with the display unit 3 during the second tap operation of the double tap operation. In this case, it is possible to consider the touch-down operation of the swipe operation as being omitted, or it is also possible to consider the swipe operation as being performed after a single tap operation. In other words, when a swipe operation is performed after a single tap operation, it is also possible to consider the double tap operation and the swipe operation as being performed in combination.

[0091] 12 is not essential. For example, regardless of whether the unevenness information was successfully acquired in step S204, the CPU 71 proceeds to step S105 and determines whether the unevenness information matches the second unevenness information.

[0092] If the unevenness information cannot be obtained normally in step S204, the CPU 71 determines "No" in step S105 and executes the processes from step S101 to step S104 again.

[0093] <2-3. Modification> A modification of the first embodiment will be described below. Note that the configuration described in this modification can also be applied to the other embodiments described below.

[0094] In the above example, the operation area ArH used for the slide operation in the swipe operation is not monitored, and a specific function is activated if the operation mode of the swipe operation is appropriate.

[0095] In this modified example, the operation area ArH on the user side is detected even during a slide operation, and the function to be activated is changed depending on the detection result.

[0096] Specifically, as shown in Figure 13, if the second operation area ArH2 is used in a user swipe operation performed after the judgment processing unit F4 determines that the second operation area ArH2 has been detected, the start / stop processing of the corresponding processing unit F5 starts or stops one of the applications App1, App2, App3, or App4 depending on the direction of the swipe operation.

[0097] When the first operation area ArH1 is used for the swipe operation, the start / stop processing of the corresponding processing unit F5 starts or stops one of the applications App6, App7, App8, or App9 (in parentheses in the figure) depending on the direction of the swipe operation.

[0098] That is, it is possible to detect and distinguish eight types of swipe operations as instruction operations, and to execute eight types of functions according to the user's operation mode.

[0099] An example of processing executed by the CPU 71 of the smartphone terminal 1A in this modified example is shown in Fig. 14. Note that the same steps as those in Fig. 11 are given the same step numbers, and descriptions thereof will be omitted where appropriate.

[0100] The CPU 71 of the smartphone terminal 1A identifies the operated area ArD in step S101, determines that a specific operation has been detected in step S102, and then performs unevenness imaging processing in step S104.

[0101] Subsequently, after determining in step S105 that the second operation area ArH2 has been detected, the CPU 71 detects an instruction operation such as a swipe operation in step S106.

[0102] Next, when it is determined in step S107 that a process corresponding to the instruction operation detected is being performed, the CPU 71 identifies in step S121 whether the operation area ArH used in the instruction operation is the first operation area ArH1 or the second operation area ArH2. This process is realized by continuously performing an unevenness imaging process while the instruction operation is being performed, and comparing the unevenness information acquired as a result with the first unevenness information Das1 and the second unevenness information Das2 stored in the storage unit 79.

[0103] Then, in step S108, the CPU 71 executes the corresponding start / stop process taking into account the operation area ArH used in the instruction operation.

[0104] In this modified example, the pointing operation using the first operation area ArH1 and the pointing operation using the second operation area ArH2 are detected as different pointing operations.

[0105] In addition, it is also possible to configure the device so that the start / stop process is not executed when the operation area ArH changes from the second operation area ArH2 to the first operation area ArH1 during a slide operation, thereby preventing the wrong function from being activated due to an erroneous operation.

[0106] The CPU 71 of the smartphone terminal 1A may function as the UI processing unit F6 to execute user interface processing that displays icons of applications to be launched or functions to be activated during a swipe (see FIG. 15).

[0107] In the example shown in Figure 15, icons indicate that an upward swipe operation as an instruction operation executes a volume increase function (application App1), a downward swipe operation executes a volume decrease function (application App2), a right swipe operation executes a search function (application App3), a left swipe operation executes a camera function (application App4), and a touch-up operation without a slide operation executes a power-off function (application App5).

[0108] When different functions are displayed depending on the operation area ArH used for the slide operation in the swipe operation, it is desirable to switch the icon display appropriately depending on whether the operation area ArH currently in contact with the display unit 3 is the first operation area ArH1 or the second operation area ArH2. This makes it possible to prevent the activation of functions that are not in line with the user's intention.

[0109] 3. Second embodiment A smartphone terminal 1A according to a second embodiment will be described. Note that the description of the same configuration as in the first embodiment will be omitted as appropriate.

[0110] In this embodiment, the CPU 71 of the smartphone terminal 1A does not function as a specific operation detection unit F2 for detecting specific operations such as double taps and triple taps, and does not execute corresponding processing in response to the detection of a specific operation.

[0111] Instead, the CPU 71 determines whether or not to execute the corresponding process based on the shape of the operated area ArD detected by the operated area detection unit F1.

[0112] <3-1. Functional Configuration> The CPU 71 in the second embodiment executes a program to function as an operated area detection unit F1, an operation area estimation unit F7, a concave / convex detection unit F3, a determination processing unit F4, a correspondence processing unit F5, and a UI processing unit F6 (see FIG. 16).

[0113] The operation area estimation unit F7 determines whether the operation area ArH used for the operation is the first operation area ArH1 or the second operation area ArH2 based on the shape of the operation area ArD detected by the operation area detection unit F1.

[0114] For example, Fig. 17 shows an example of the operated area ArD when the user performs a tap operation using the fingertip area of ​​the thumb (shown by the dashed line) as the first operation area ArH1. Fig. 18 shows an example of the operated area ArD when the user performs a tap operation using the pad area and distal area of ​​the thumb (shown by the dashed line) as the second operation area ArH2.

[0115] The operation area estimation unit F7 determines whether the shape of the operated area ArD detected by the operated area detection unit F1 is closer to the shape shown in Figure 17 or the shape shown in Figure 18, and if it determines that it is closer to the shape shown in Figure 18, the operation area estimation unit F7 determines that it has detected a touch operation using the second operation area ArH2.

[0116] In addition, this determination process may determine that the first operation area ArH1 has been detected if the shape of the operation area ArD detected by the operation area detection unit F1 is horizontal, and may determine that the second operation area ArH2 has been detected if the shape is vertical.

[0117] Alternatively, it may be determined that the first operation area ArH1 has been detected if the size of the shape of the operation area ArD detected by the operation area detection unit F1 is less than a predetermined size, and that the second operation area ArH2 has been detected if the size is greater than or equal to a predetermined size.

[0118] The unevenness detection unit F3 determines whether to execute an unevenness imaging process for detecting unevenness information according to the estimation result by the operation area estimation unit F7. That is, the unevenness detection unit F3 executes the unevenness imaging process when the operation area estimation unit F7 detects a touch operation using the second operation area ArH2.

[0119] In this way, by appropriately performing unevenness imaging processing according to the shape of the operated area ArD, it is possible to reduce the number of times the light-emitting unit emits light, thereby suppressing an increase in power consumption of the smartphone terminal 1A.

[0120] Furthermore, since the process of determining the shape of the operated area ArD takes less time than the process of detecting a specific operation, the time required to activate a specific function as a response process can be shortened. Therefore, a specific function can be quickly activated in response to a user operation, improving convenience. Furthermore, since the time required to activate a function is short, it is possible to assign a function that is preferably activated in a short time.

[0121] The unevenness detection unit F3 may be configured to perform unevenness imaging processing both when the first operation area ArH1 is detected by the operated area detection unit F1 and when the second operation area ArH2 is detected. Even in this case, the unevenness detection unit F3 performs unevenness imaging processing according to the shape of the operated area ArD detected by the operated area detection unit F1. That is, the light-emitting area is made different between the shape of the operated area ArD shown in FIG. 17 and the shape of the operated area ArD shown in FIG. 18. Therefore, the unevenness detection unit F3 illuminates only the area necessary for acquiring unevenness information with white light for the necessary time, thereby suppressing an increase in power consumption of the smartphone terminal 1A.

[0122] 3-2. Processing Example An example of processing executed by the CPU 71 of the smartphone terminal 1A in this embodiment is shown in Fig. 19. Note that processing similar to the processing shown in Fig. 11 is given the same step number, and description thereof will be omitted as appropriate.

[0123] After detecting the operated area ArD in step S101, the CPU 71 of the smartphone terminal 1A determines in step S141 whether or not the shape of the detected operated area ArD corresponds to the shape of the second operation area ArH2.

[0124] If it is determined that the shape of the detected operated area ArD does not correspond to the shape of the second operation area ArH2 (step S141: No determination), the CPU 71 proceeds to step S103 and executes processing according to the operation content.

[0125] On the other hand, if it is determined that the shape of the detected operated area ArD corresponds to the shape of the second operation area ArH2 (step S141: Yes determination), the CPU 71 proceeds to step S104 and performs unevenness imaging processing.

[0126] As a result, a specific function can be realized only when the shape of the detected operated area ArD corresponds to the shape of the second operation area ArH2.

[0127] <3-3. Modifications> A modification of the present embodiment will be described. In the present embodiment, the operation area estimation unit F7 determines whether the shape of the detected operated area ArD corresponds to the shape of the second operation area ArH2, thereby determining whether or not a corresponding process can be executed.

[0128] The operation area estimation unit F7 in this modified example determines whether or not the shape of the detected operated area ArD is a shape that can include the second operation area ArH2.

[0129] For example, a user may perform touch operations using the fingertip in normal operations, and may perform touch operations using the pad of the finger to activate a specific function. In this case, the operated area ArD detected by the touch operation using the pad of the finger may be larger than expected. In this case, simply comparing the shapes of the operated area ArD and the second operation area ArH2 may result in a mismatch between the shapes of the two areas, and the specific function may not be activated.

[0130] Therefore, the operation area estimation unit F7 in this modified example determines whether or not the shape of the detected operated area ArD is a shape that can include the second operation area ArH2.

[0131] For example, as shown in FIG. 20, if an operation area ArH (shown by a solid line), which is the area corresponding to the detected operated area ArD and is the area of ​​the finger that actually touched the display unit 3, includes a second operation area ArH2 (shown by a dotted line), the operation area estimation unit F7 determines that a touch operation has been performed using the second operation area ArH2.

[0132] This allows a specific function to be realized even without precise operation, thereby improving usability.

[0133] In addition, when the operation area ArH, which is the area that actually contacts the display unit 3, is larger than the second operation area ArH2, the unevenness imaging process may be performed on only a part of the operation area ArH, rather than on the entire operation area ArH.

[0134] Specifically, a modified example of the unevenness imaging process shown in Fig. 12 is shown in Fig. 21. Note that the same processes as those shown in Fig. 12 are given the same step numbers, and descriptions thereof will be omitted where appropriate.

[0135] After identifying the operated area ArD in step S201, the CPU 71 of the smartphone terminal 1A further identifies an area corresponding to the second operation area ArH2 from within the operated area ArD in step S241.

[0136] In the following step S202, the CPU 71 specifies a light-emitting unit for illuminating only the area corresponding to the second operation area ArH2.

[0137] In step S203, the CPU 71 performs light emission processing so that only the area corresponding to the second operation area ArH2 emits light in white.

[0138] In this way, by illuminating only a part of the operated area ArD rather than the entire area, the area illuminating in white can be reduced, and an increase in power consumption of the smartphone terminal 1A can be suppressed.

[0139] Another modification of the present embodiment will be described. In step S141 of Fig. 19, a process is performed to determine whether the shape of the operated area ArD matches or is similar to the shape of the second operation area ArH2.

[0140] At this time, the shape of the second operation area ArH2 compared with the operated area ArD may be different depending on the position of the detected operated area ArD.

[0141] An example is shown in Fig. 22. As shown in the figure, the shape of the operated area ArD when the second operation area ArH2 is touched at the right edge of the display unit 3, the shape of the operated area ArD when the second operation area ArH2 is touched at approximately the center of the display unit 3, and the shape of the operated area ArD when the second operation area ArH2 is touched near the bottom edge of the display unit 3 have different inclinations.

[0142] The operation area estimation unit F7 performs the determination process of step S141 in consideration of the shape (orientation) of the second operation area ArH2 to be compared according to the position of the operated area ArD identified in step S101 of FIG.

[0143] This allows the determination process of step S141 to be performed appropriately even if operations are performed at various positions on the display unit 3, making it possible to execute the corresponding start / stop process as intended by the user.

[0144] The position on the display unit 3 and the shape (orientation) of the operated area ArD when ArH2 touches it differ depending on the orientation of the smartphone terminal 1A, i.e., whether it is held horizontally or vertically. Therefore, by further considering the orientation of the smartphone terminal 1A, it is possible to increase the likelihood that the start / stop process will be executed as intended by the user.

[0145] 4. Third Embodiment A smartphone terminal 1A in a third embodiment is an example in which the first unevenness information Das1 is stored in the storage unit 79, but the second unevenness information Das2 is not stored.

[0146] In the third embodiment, the CPU 71 executes a program to function as an operated area detection unit F1, a specific operation detection unit F2, a concave / convex detection unit F3, a judgment processing unit F4, a response processing unit F5, and a UI processing unit F6 (see Figure 3).

[0147] The judgment processing unit F4 compares the unevenness information detected by the unevenness detection unit F3 when a specific operation such as a double tap is detected by the specific operation detection unit F2 with the first unevenness information Das1 stored in the memory unit 79.

[0148] If the unevenness information detected by the unevenness detection unit F3 does not match the first unevenness information Das1, the determination processing unit F4 determines that the operation area ArH used for the touch operation is the second operation area ArH2, which causes the response processing unit F5 to perform a response process, thereby realizing a specific function.

[0149] Note that even if an operation is unintentionally performed in the first operation area ArH1, the specific function will not be activated unless a subsequent instruction operation such as a swipe operation is performed. By performing the two-stage determination process in this manner, it is possible to prevent the specific function from being activated erroneously.

[0150] An example of processing executed by the CPU 71 of the smartphone terminal 1A in this embodiment is shown in Fig. 23. Note that the same steps as those in Fig. 11 are given the same step numbers, and descriptions thereof will be omitted where appropriate.

[0151] The CPU 71 of the smartphone terminal 1A identifies the operated area ArD in step S101, determines that a specific operation has been detected in step S102, and then performs unevenness imaging processing in step S104.

[0152] Next, in step S161, the CPU 71 determines whether the unevenness information of the operation area ArH acquired in the unevenness imaging process matches the first unevenness information Das1. If it is determined that the unevenness information of the acquired operation area ArH matches the first unevenness information Das1, it is determined that the user is performing a normal operation. Therefore, the CPU 71 proceeds to step S103 via steps S101 and S102, and executes processing according to the operation content.

[0153] On the other hand, if it is determined that the acquired unevenness information of the operation area ArH does not match the first unevenness information Das1, it is possible that the user is attempting to activate a specific function through a specific operation. Therefore, the CPU 71 proceeds to step S106 to detect the instruction operation, and then executes the corresponding start / stop process in steps S107 and S108 to activate the specific function.

[0154] <4-1. Modifications> Modifications of the present embodiment will be described.

[0155] In the third embodiment, as shown in step S161 of Fig. 23, when a user uses an area different from the first operation area ArH1 that is used for normal operations, a corresponding process can be executed regardless of the area used. That is, even if an operation is performed by someone other than the user who owns the smartphone terminal 1A, a specific application or the like is started as a corresponding process.

[0156] Incidentally, if the specific application is a payment application that allows cashless payment or an application for a bank, securities company, etc., it is undesirable for the specific application to be launched by an instruction operation from another person. In other words, for applications that are preferably launched with a certain level of security guaranteed, it is preferable not to assign the application as a specific function corresponding to an instruction operation.

[0157] However, this restriction may impair convenience for the user. In this modified example, when the function assigned in response to the instruction operation is a function for which it is preferable to ensure a certain level of security, the user is prompted to register the second unevenness information Das2.

[0158] As shown in FIG. 24, the smartphone terminal 1A according to this modified example functions as an operated area detection unit F1, a specific operation detection unit F2, a concave / convex detection unit F3, a judgment processing unit F4, a response processing unit F5, a UI processing unit F6, an allocation processing unit F8, and a recommendation processing unit F9.

[0159] The assignment processing unit F8 associates an instruction operation such as the swipe operation described above with a specific function to be executed in response to the instruction operation.

[0160] An example of the instruction operation and function allocation screen 100 is shown in Fig. 25. The allocation screen 100 is presented to the user by the UI processing unit F6.

[0161] The allocation screen 100 has an upper area, a status confirmation area 101 for confirming instruction operations and the allocation status of functions, and a lower area, a function list area 102 in which a list of allocatable functions is displayed as icons.

[0162] Five display fields 103 are arranged in the status confirmation area 101. The five display fields 103 are a display field 103A in which an icon image of a function to be executed when an upward swipe operation is performed as an instruction operation, a display field 103B corresponding to a downward swipe operation, a display field 103C corresponding to a rightward swipe operation, a display field 103D corresponding to a leftward swipe operation, and a display field 103E corresponding to a touch-up operation without a slide operation.

[0163] 25 , the functions arranged in the function list area 102 include a volume increase and decrease function, a search function, a camera function, a video capture function, a still image capture function, a cashless payment application, a banking application, a power-off function, a brightness increase and decrease function, an email application, launching the email application with a business account, and launching the email application with a private account, and are arranged as icons. Each function is arranged as an icon image 104.

[0164] The user can assign a function to a command operation by performing a drag operation starting from an icon arranged in the function list area 102 and ending at any of the display fields 103. As an example, FIG. 26 shows a state in which the volume increase function is assigned to an upward swipe operation as a command operation.

[0165] Here, assume that the user performs a drag operation starting from the icon image 104 of the cashless payment application and ending at one of the display fields 103. In this case, the user has issued an instruction to assign a function to launch the cashless payment application.

[0166] It is preferable that the cashless payment application be launched with a certain level of security guaranteed.

[0167] When an instruction to assign such a function is received, the recommendation processing unit F9 executes a process of prompting the registration of the second unevenness information Das2.

[0168] 27, the recommendation processing unit F9 displays a confirmation screen 200 on the display unit 3. The confirmation screen 200 includes a text sentence 201 prompting the user to register second unevenness information Das2, which is unevenness information for the second operation area ArH2, a registration instruction button 202 for instructing the user to register, and a cancel button 203 for rejecting the registration. Note that such a UI display may be realized by the UI processing unit F6.

[0169] By pressing the registration instruction button 202, the user starts the process of registering the unevenness information of the user's second operation region ArH2, for example, the finger pad region or distal joint region, as second unevenness information Das2.

[0170] Then, with the second concavo-convex information Das2 registered, a process for allocating a function that is desirably executed while ensuring a certain level of security is executed. In this case, when it is confirmed that the user's instruction operation has been performed using the second operation area ArH2, the function is realized.

[0171] On the other hand, to refuse to register the second unevenness information Das2, the user can press the cancel button 203. When the cancel button 203 is pressed, the CPU 71 of the smartphone terminal 1A may cancel the immediately preceding assignment operation or may complete the assignment operation.

[0172] If the cancel button 203 is pressed and the assignment operation is completed without registering the second unevenness information Das2, the function is executed appropriately in response to the instruction operation in an area other than the first operation area ArH1, even if it is a function that is desirably executed while ensuring a certain level of security.

[0173] FIG. 28 shows the process executed by the CPU 71 of the smartphone terminal 1A to assign functions in this modified example.

[0174] When detecting an operation by the user to assign a specific function to an instruction operation, the CPU 71 of the smartphone terminal 1A performs processing to display the assignment screen 100 on the display unit 3 in step S301 shown in FIG.

[0175] The CPU 71 performs the determination process of step S302 with the allocation screen 100 displayed on the display unit 3, that is, the determination process of determining whether or not an allocation operation such as a drag operation has been detected.

[0176] The CPU 71 repeats the process of step S302 until detecting an allocation operation by the user. If an operation to end the display of the allocation screen 100 is separately detected, the CPU 71 ends the series of processes shown in FIG.

[0177] If an allocation operation by the user is detected (step S302: Yes judgment), the CPU 71 judges in step S303 whether the specific function that is the target of the allocation operation is a function for which a specified security guarantee is recommended.

[0178] If the specific function targeted for the assignment operation is not one for which a predetermined security guarantee is recommended (step S303: No), the CPU 71 proceeds to step S308 and performs assignment processing according to the user's operation. As a result, the function selected by the user and the instruction operation are associated and stored in the storage unit 79 or the like.

[0179] On the other hand, if the specific function that is the target of the allocation operation is one for which a predetermined security guarantee is recommended (step S303: Yes judgment), the CPU 71 displays a confirmation screen 200 such as that shown in Figure 27 on the display unit 3 in step S304.

[0180] Next, the CPU 71 successively executes a process of determining whether or not the registration instruction button 202 has been pressed (step S305) and a process of determining whether or not the cancel button 203 has been pressed (step S306).

[0181] If it is determined in step S305 that the registration instruction button 202 has been pressed, the CPU 71 executes registration processing for the second operation area ArH2 in step S307. In this registration processing, second unevenness information Das2 is acquired by executing unevenness imaging processing while the user is holding the second operation area ArH2 in contact with the display unit 3. The acquired second unevenness information Das2 is stored in the storage unit 79.

[0182] After completing the registration process in step S307, the CPU 71 performs an allocation process in step S308.

[0183] Furthermore, if it is determined in step S306 that the cancel button 203 has been pressed, the CPU 71 executes the assignment process in step S308 without registering the second unevenness information Das2 for the second operation area ArH2.

[0184] Even if the answer to step S303 is Yes, if the second unevenness information Das2 for the user's second operation area ArH2 has already been registered, the allocation process of step S308 may be performed without performing the processes from step S304 to step S307.

[0185] Next, an example of processing executed by the CPU 71 of the smartphone terminal 1A to execute a specific function assigned in response to a user operation is shown in Fig. 29. Note that processing similar to that shown in Fig. 11 is assigned the same step number, and description thereof will be omitted as appropriate.

[0186] The CPU 71 of the smartphone terminal 1A detects the operated area ArD in step S101, and then detects a specific operation such as a double tap in step S102.

[0187] Then, when a specific operation is detected (step S102: Yes judgment), the CPU 71 performs unevenness imaging processing in step S104, and then judges whether the second unevenness information Das2 is stored in the memory unit 79 in step S181.

[0188] If it is determined that the second unevenness information Das2 is stored (step S181: Yes determination), the CPU 71 proceeds to step S182 and determines whether the unevenness information acquired in the unevenness imaging process of step S104 matches the second unevenness information Das2.

[0189] If the acquired unevenness information does not match the second unevenness information Das2 (step S182: No determination), the CPU 71 returns to step S101. That is, for example, if a user attempts to execute a function for which a certain level of security is recommended, but the user's operation in the second operation area ArH2 is not detected, the function is not executed and security is guaranteed.

[0190] Furthermore, if the acquired unevenness information matches the second unevenness information Das2 (step S182: Yes determination), the CPU 71 executes a process of detecting a pointing operation, i.e., a process of detecting a swipe operation in a specific direction, in step S106. In this case, when the user performs a predetermined swipe operation, a function that is recommended for ensuring a certain level of security is executed.

[0191] If it is determined in step S181 that the second unevenness information Das2 has not been registered, the CPU 71 determines in step S183 whether the unevenness information acquired in the unevenness imaging process in step S104 matches the registered first unevenness information Das1.

[0192] If it is determined that the acquired unevenness information in step S104 matches the registered first unevenness information Das1 (step S183: Yes determination), the CPU 71 determines that a normal operation by the user has been detected, and returns to the processing of step S101. In this case, the specific function is not executed.

[0193] Furthermore, if it is determined that the concavo-convex information acquired in step S104 does not match the registered first concavo-convex information Das1 (step S183: No determination), the CPU 71 proceeds to step S106 to detect a pointing operation. In this case, if an appropriate pointing operation is detected, a specific function can be executed even if the operation is performed with another person's finger.

[0194] As a modification of the example shown in FIG. 29, at least some of the processes in steps S181, S182, and S183 may be executed after the determination process in step S107.

[0195] Specifically, as shown in Fig. 30, the CPU 71 of the smartphone terminal 1A detects the operated area ArD in step S101, and detects a specific operation in step S102. If it is determined that a specific operation has been detected (step S102: Yes determination), the CPU 71 executes unevenness imaging processing in step S104.

[0196] Subsequently, in step S183, the CPU 71 determines whether or not the detected unevenness information matches the first unevenness information Das1.

[0197] If it is determined that the information matches the first unevenness information Das1 (step S183: Yes determination), the CPU 71 returns to step S101. That is, the CPU 71 determines that the operation is a normal operation and does not activate the specific function.

[0198] On the other hand, if it is determined that there is no match with the first unevenness information Das1 (step S183: No determination), the CPU 71 performs an instruction operation detection process in step S106, and determines whether or not a corresponding process exists in step S107.

[0199] If it is determined that a process corresponding to the instruction operation exists (step S107: Yes determination), the CPU 71 determines in step S184 whether the corresponding process is one for which a predetermined security guarantee is recommended.

[0200] If it is determined that security is not required (step S184: No), the CPU 71 executes the corresponding start-up / stop processing in step S108.

[0201] Furthermore, if it is determined that the corresponding processing is recommended for a specified security measure (step S184: Yes determination), the CPU 71 determines in step S181 whether the second unevenness information Das2 has already been stored, and determines in step S182 whether the second unevenness information Das2 matches the unevenness information detected in the unevenness imaging processing of step S104.

[0202] The CPU 71 executes the corresponding start-up / stop processing in step S108 only if it determines that the second unevenness information Das2 has been stored (step S181: Yes judgment) and that the second unevenness information Das2 matches the unevenness information detected in the unevenness imaging processing in step S104 (step S182: Yes judgment).

[0203] On the other hand, if the determination is No in either step S181 or step S182, the CPU 71 returns to the process of step S101 and does not activate the specific function.

[0204] With this configuration, for specific functions that require security, it is possible to require the user to perform operations in the second operation area ArH2.

[0205] The allocation process described in this embodiment can also be applied to the first and second embodiments.

[0206] 5. Other Modifications In the unevenness imaging process described above, an example has been shown in which the light emitting section is made to light up in accordance with the position of the operated area ArD that is in contact with the user's hand or the like.

[0207] However, the light emitted by the light-emitting unit may illuminate a fixed predetermined area on the display unit 3. The user can acquire the first unevenness information Das1 and the second unevenness information Das2 by bringing the first operation area ArH1 or the second operation area ArH2 of his or her hand into contact with the area indicated by the white light.

[0208] In this case, the CPU 71 can realize the above by executing the following process. First, the CPU 71 generates or reads a white image according to the shape and size of a predetermined area based on information about the predetermined area that has been prepared in advance. Next, the CPU 71 displays the generated or read white image in the predetermined area.

[0209] Furthermore, in the unevenness imaging process for acquiring the second unevenness information Das2, when the user touches the first operation area ArH1 to the display unit 3, the operation area ArH may be identified as the first operation area ArH1 from the detection results of the shape and unevenness shape, and the position on the display unit 3 where the second operation area ArH2 will be touched may be estimated. In this case, by performing the unevenness imaging process on the estimated position on the display unit 3 before the user touches the second operation area ArH2 to the display unit 3, it is possible to quickly read the unevenness information of the user's second operation area ArH2.

[0210] In the above example, the pad region of the finger and the distal joint region are given as examples of the second operation region ArH2 of the user, but these second operation regions ArH2 may be made unchangeable regardless of the user. For example, the palm region may be set as the second operation region ArH2 regardless of the user and may not be changed.

[0211] Alternatively, the first operation area ArH1 used by the user for normal operations may be identified by detecting the user's daily operations on the smartphone 1A and storing the detection results. This eliminates the need for the user to register the first operation area ArH1, thereby improving convenience. This is particularly effective when the user is not familiar with the operation of the smartphone 1A.

[0212] Furthermore, in the smartphone terminal 1A, it may be unnecessary to register the first unevenness information Das1 and the second unevenness information Das2.

[0213] For example, in the smartphone terminal 1A, it is predetermined that the first operation area ArH1 is the fingertip area and the second operation area ArH2 is the finger pad area or distal joint area. Furthermore, an operated area ArD is acquired through various operations by the user, and the acquired operated area ArD is input to a trained AI (Artificial Intelligence) model.

[0214] The AI ​​model estimates, based on input data, whether the operation area ArH that contacts the operated area ArD is the fingertip area, or the pad area or distal area. If the CPU 71 of the smartphone 1A obtains an estimation result that the operation area ArH used for the operation is the second operation area ArH2, the CPU 71 makes a "Yes" determination in the determination process, such as step S105 in FIG. 11 .

[0215] By providing the smartphone terminal 1A with such a configuration, the user does not need to perform an operation to register unevenness information, thereby improving convenience.

[0216] Furthermore, the trained AI model may be trained to be able to estimate whether the acquired unevenness information relates to the fingertip region or the pad region and distal joint region. That is, the trained AI model may be able to estimate whether the detected unevenness shape is the first unevenness information Das1 or the second unevenness information Das2. Even if the first unevenness information Das1 and the second unevenness information Das2 for each user are not registered, by using such an AI model, it is possible to estimate whether the user's operation is a normal operation.

[0217] Alternatively, an AI model may be used that estimates whether the operation area ArH that contacted the operated area ArD is a normal area. In this case, it is not necessary to estimate whether the operation area ArH used for the operation is an abdominal area or a distal area, and it is sufficient to estimate whether an unusual operation area ArH was used. Then, if an estimation result that an unusual operation area ArH was used is obtained, a "Yes" determination may be made in the determination process, for example, in step S105 of FIG. 11 .

[0218] Note that the estimation of whether the acquired unevenness information relates to the fingertip region or the abdominal region and distal joint region may be performed based on a rule base without using an AI model. In this case, in step S105 of Fig. 11, the features of the unevenness information detected in step S104 are compared with the features of the fingertip region and the abdominal region and distal joint region stored in the storage unit 79 to determine which of the first operation region ArH1 and the second operation region ArH2 was used for the operation. If it is determined that the features of the detected unevenness information do not match the features of the abdominal region and distal joint region, a "No" determination is made, and if they match, a "Yes" determination is made.

[0219] It should be noted that there are other possible configurations that do not require intentional registration processing of the unevenness information. For example, the first unevenness information Das1 used for normal operations can be acquired while the user is performing normal operations on the smartphone terminal 1A. This allows the user to register the first unevenness information Das1 for the first operation area ArH1 without performing an intentional registration operation, thereby improving convenience.

[0220] In the above example, the first operation area ArH1 and the second operation area ArH2 are described as different areas of the same finger, but they may be the same area of ​​different fingers. For example, the first operation area ArH1 may be the fingertip area of ​​the index finger, and the second operation area ArH2 may be the fingertip area of ​​the little finger. In this way, the second operation area ArH2 is not limited to the above examples as long as it is an area that is not usually used often.

[0221] In the first and second embodiments, an example has been shown in which both the first unevenness information Das1 and the second unevenness information Das2 are stored in the storage unit 79, but if only the second unevenness information Das2 is used in the determination process, it is not essential to store the first unevenness information Das1 in the storage unit 79. In other words, even if only the second unevenness information Das2 is stored in the storage unit 79, it is possible to execute a specific function as intended by the user.

[0222] The specific function executed in response to a predetermined instruction operation by the user may be a function called Side Sense (registered trademark).

[0223] Side sense is a function that displays a predetermined menu screen or the like in response to a user's operation on the edge of the display unit 3. In the example shown in Fig. 31 , the operation target area SA for the Side sense function is provided in approximately the center in the vertical direction on the right edge of the display unit 3.

[0224] For example, when a predetermined operation such as a tap operation is performed on the operation target area SA, the side sense function is executed.

[0225] Various functions can be assigned as the Side Sense function. For example, a specific application may be launched, or a screen with various menus may be displayed. Alternatively, the Side Sense function may be used to switch the display mode of various applications between window mode and full screen mode.

[0226] 6. Registration Process of Concave-Convex Information An example of the registration process of concave-convex information will be described. In this example of the registration process of concave-convex information, five fingers of one hand are simultaneously brought into contact with the touchable operation area 5 set on the display unit 3, thereby simultaneously registering the concave-convex information of the five fingers. Note that this example is an example in which the information processing device 1 is a tablet terminal 1B.

[0227] An image is displayed in the touched operation area 5 to guide the user to place the fingers of the right hand at predetermined positions.

[0228] FIG. 32 shows an example of an image displayed in the touched operation area 5 when starting to register the concave-convex information.

[0229] 32 and the following figures, images that are displayed in the touched operation area 5 and that resemble the outline of a hand or the shape of fingertips are depicted using dashed lines. Also, a user's hand placed in the touched operation area 5 for registering embossment information such as a fingerprint is depicted using a solid line.

[0230] As shown in the figure, an image G1 that resembles the outline of a hand is displayed in the touched operation area 5. When the user places their right hand on the touched operation area 5 as if placing their hand over this image G1 (see FIG. 33), an image G2 is displayed that prompts the user to change the posture of their hand by lifting their palm and clenching their fingers, as shown in FIG. 34. Image G2 is an image for starting the procedure of sliding their fingertips on the touched operation area 5. Specifically, image G2 is an image consisting of five circles or ellipses that indicate the positions of their fingertips. Note that the palm here refers to the part of the hand excluding the fingers.

[0231] When the right hand is placed over the image G1 as shown in FIG. 33, the pad region and distal joint region of the finger come into contact with the touch-operated region 5, making it possible to read unevenness information in the vicinity of the distal joint.

[0232] In addition, in the process of transitioning the instruction from FIG. 33 to FIG. 34, the fingertip area at the tip of the finger rather than the pad portion comes into contact with the touched operation area 5, making it possible to read the unevenness information of the fingertip area.

[0233] Furthermore, as shown in FIGS. 34 and 35, by deforming the image G2 to encourage the user to bring the fingertips closer together, it becomes possible to read the unevenness information of the tip regions of the fingers.

[0234] As shown in the figures, the unevenness information of each finger and palm can be acquired by displaying an image that mimics the outline of a hand and guides the user to touch a different area for each finger in the touch operation area 5. Furthermore, by reading the unevenness information of the fingers in this manner, it becomes possible to automatically identify the correspondence between the unevenness information of each finger and the finger type (thumb, index finger, etc.).

[0235] Even if the fingertip parts in each image differ from the operated area ArD actually touched by the fingers, by identifying whether the hand to be registered is a right hand or a left hand, it is possible to identify the order of the thumb, index finger, middle finger, ring finger, and little finger, and determine the correspondence between the unevenness information and the finger type. As a result, even if the user roughly places their hand on the touched operation area 5, it is possible to read the unevenness information of each finger with the finger type identified.

[0236] 36 , when registering only the concavo-convex information of the fingertip region without registering the concavo-convex information of the palm region or the finger pad region (distal region), image G2 may be displayed first without using image G1, and then image G2 may be deformed to guide the user so that only the fingertip region always comes into contact with the touched operation region 5. In other words, the user may be prompted to make an operation of bringing only the fingertip region into contact with the touched operation region 5, and then be guided to make an operation of puckering the fingers.

[0237] By reducing the contact area between the touched operation area 5 and the hand, the load of the unevenness imaging process can be reduced.

[0238] In order to make the user aware that the operation area ArH is shifting more toward the fingertip in the process of encouraging the user to pucker the fingers, the circle or ellipse that resembles the fingertip in image G2 may be deformed into a more squashed shape, as shown in FIG. 36 .

[0239] Furthermore, after prompting the user to touch only the fingertip to the touched operation area 5 (upper diagram in FIG. 36 ), the user may be prompted to slide only the thumb or slide only the thumb and little finger. That is, after image G2 is displayed in the touched operation area 5, image G2′ in which only the position of the thumb (or little finger included) is changed may be displayed (see FIG. 37 ).

[0240] By performing this procedure, the angle between the user's thumb and little finger will expand from about 20 degrees to about 79 to 90 degrees. At this time, as shown in Figure 38, the thumb changes from a state in which the side of the fingertip is the operation area ArH to a state in which the center of the fingertip is the operation area ArH. This makes it possible to appropriately read the unevenness information of the thumb.

[0241] When registering unevenness information for multiple fingers, in addition to the above-mentioned methods, conceivable methods include a method of acquiring unevenness information for each finger in turn, or a method of acquiring unevenness information for all ten fingers on both hands simultaneously.

[0242] 7. Summary As described in the above examples, the smartphone terminal 1A as the information processing device 1 of the present technology includes: an unevenness detection unit F3 that detects unevenness information about a user's operation area ArH used for a touch operation on an operated area ArD, which is an area on the screen (display unit 3); a determination processing unit F4 that determines whether the operation area ArH is a first operation area ArH1, which is a predetermined partial area of ​​the user's hand, based on the unevenness information detected by the unevenness detection unit F3; and a response processing unit F5 that executes a predetermined response process when it is determined that an area of ​​the user's hand other than the first operation area ArH1 has been used for the touch operation. For example, the first operation area ArH1 is an area normally used for touch operations. The area other than the first operation area ArH1 is an area not normally used for touch operations. In such a case, the predetermined response process is executed when a touch operation is performed on an area not normally used. That is, the user can execute the corresponding process simply by performing an operation using an area other than the first operation area ArH1, without performing an operation on an icon or menu. By executing the corresponding process when the user performs a touch operation using the operation area ArH that is not often used for touch operations, the corresponding process is not executed in normal touch operations, thereby preventing erroneous operations. Therefore, the convenience of user operations can be improved. The determination of whether the first operation area ArH1 is the first operation area may be determined based on stored fingerprint information or the like, but may also be performed by other methods. For example, when the first operation area ArH1 is determined to be a fingertip area, the operation area ArH is inferred using a trained AI model. Then, when it is estimated that the operation area ArH, which is the area of ​​the user's hand used for operation, is not a fingertip, the corresponding process may be executed. Furthermore, as described in other modified examples, the unevenness detection unit F3 may acquire unevenness information about the user's operation area ArH used in the touch operation on the fixed area as the operated area ArD when the operated area ArD targeted by the user's touch operation is a fixed specific area.

[0243] As described with reference to FIGS. 3 and 23 , the smartphone terminal 1A serving as the information processing device 1 includes a storage unit 79 that stores first unevenness information Das1, which is unevenness information for the first operation area ArH1. The determination processing unit F4 may determine that an area other than the first operation area ArH1 has been used for the touch operation if the unevenness information detected by the unevenness detection unit F3 differs from the first unevenness information Das1. The first unevenness information Das1 may be, for example, fingerprint information of a finger frequently used by the user for touch operations. The determination processing unit F4 determines whether the first unevenness information Das1 stored in the storage unit 79 matches the unevenness information detected by the unevenness detection unit F3, thereby identifying that the user has performed a touch operation using an operation area ArH that is not frequently used for touch operations. This allows the user to perform a predetermined response process at a desired timing and prevents the predetermined response process from being performed at an undesired timing.

[0244] As described with reference to FIGS. 2 and 3 , the smartphone terminal 1A serving as the information processing device 1 includes a storage unit 79 that stores first unevenness information Das1, which is unevenness information for the first operation area ArH1, and second unevenness information Das2, which is unevenness information for the second operation area ArH2 other than the first operation area ArH1, on the user's hand. The determination processing unit F4 may determine that an area other than the first operation area ArH1 has been used for a touch operation if the unevenness information detected by the unevenness detection unit F3 matches the second unevenness information Das2. This makes it possible to accurately determine that the user performed a touch operation using the infrequently used operation area ArH, for example, when the second operation area ArH2 is set as an operation area ArH that is not often used for touch operations. Therefore, it is possible to execute a predetermined response process in accordance with the user's intention, thereby improving convenience.

[0245] As described with reference to FIG. 5 and other figures, in the smartphone terminal 1A serving as the information processing device 1, the first operation area ArH1 may be an area frequently used for touch operations and may be a fingertip area of ​​a predetermined finger. This allows the predetermined response process to be executed when a touch operation is performed using an operation area ArH other than the fingertip area frequently used for touch operations. This prevents the predetermined response process from being unintentionally executed during a normal touch operation, improving user convenience.

[0246] As described with reference to FIG. 5 and other figures, in the smartphone terminal 1A serving as the information processing device 1, the first operation area ArH1 is an area frequently used for touch operations and may be the fingertip area of ​​a predetermined finger, while the second operation area ArH2 is an area less frequently used for touch operations than the fingertip area and may be the pad area of ​​a predetermined finger. That is, the first operation area ArH1 and the second operation area ArH2 are different areas of the same finger. This eliminates the need to switch the finger used for touch operations between performing a normal touch operation and executing a predetermined corresponding process. Therefore, a predetermined corresponding process can be executed without changing the grip of a mobile terminal device such as the smartphone terminal 1A or tablet terminal 1B serving as the information processing device 1, thereby preventing the mobile terminal device from being dropped, for example.

[0247] 3 and the like, the smartphone terminal 1A as the information processing device 1 includes a specific operation detection unit F2 that detects a specific operation by a user, and the unevenness detection unit F3 may detect unevenness information when the specific operation is detected. By configuring the smartphone terminal 1A so that a predetermined response process is executed only when a characteristic operation is detected, it is possible to prevent the predetermined response process from being executed unintentionally, thereby improving user convenience.

[0248] As described with reference to FIGS. 16 to 18 , the unevenness detection unit F3 in the smartphone terminal 1A serving as the information processing device 1 may detect unevenness information when the shape of the operated area ArD where a touch operation is performed corresponds to the shape of the second operation area ArH2. For example, to detect unevenness information in the operation area ArH, a process is performed in which a light-emitting unit disposed at the back (thickness direction) of the screen is illuminated. If the light-emitting unit is illuminated every time a user performs a touch operation, the battery consumption of the mobile terminal device serving as the information processing device 1 may increase and the continuous operating time may be shortened. In this configuration, by performing the unevenness information detection process when the shape of the operated area ArD corresponds to the shape of the second operation area ArH2, it is possible to suppress an increase in power consumption and reduce battery consumption.

[0249] 11 and the like, the determination processing unit F4 in the smartphone terminal 1A serving as the information processing device 1 may compare features common to a plurality of users for each of the first operation area ArH1 and the second operation area ArH2 other than the first operation area ArH1 with features included in the unevenness information detected by the unevenness detection unit F3, and may determine that the second operation area ArH2 has been used for a touch operation if the features included in the unevenness information include features for the second operation area ArH2. In other words, even if fingerprint information unique to each user is not stored in the storage unit 79, it is possible to determine whether the second operation area ArH2 has been used for a touch operation by determining whether the detected unevenness information includes features of the unevenness information for the fingertip region of the thumb or includes features for the pad region or the distal region.

[0250] As described with reference to FIGS. 9 and 10 , the response processing unit F5 in the smartphone terminal 1A serving as the information processing device 1 may execute, as predetermined response processing, a command operation detection process for detecting a slide operation and a touch-up operation, and a start / stop process for starting or stopping a corresponding application depending on the direction of the slide operation when a touch-up operation is detected. With this configuration, a user can start or stop a correspondingly assigned application by performing a touch-down operation, a slide operation, or a touch-up operation on the screen using an operation area ArH other than the first operation area ArH1. This allows different functions to be implemented depending on the slide direction. For example, if four slide directions, namely, up, down, left, and right, are distinguished, at least four functions can be provided. Note that three or fewer slide directions may be distinguished, or five or more slide directions may be distinguished.

[0251] As described with reference to FIG. 10 and other figures, the correspondence processing unit F5 in the smartphone terminal 1A serving as the information processing device 1 may detect a touch-up operation that does not involve a slide operation in the instruction operation detection process. This makes it possible to selectively provide five different functions when, for example, determining and distinguishing between four types of slide directions (upward, downward, leftward, and rightward). This further improves usability.

[0252] 9 and 10 , different applications may be started or stopped depending on the direction of the slide operation in the start / stop process executed by the smartphone terminal 1A as the information processing device 1. This allows the user to switch between starting and stopping many applications by changing the slide direction.

[0253] 9 and 10 , in the activation / stop process executed by the smartphone terminal 1A as the information processing device 1, the same application may be activated in different modes depending on whether the direction of the slide operation is a first direction or a second direction. For example, activation in different modes may involve activating a payment application in a first payment mode that reads a two-dimensional code and in a second payment mode that displays the two-dimensional code. This allows the user to activate the application in a desired mode, thereby improving convenience.

[0254] 24 to 29 , the smartphone terminal 1A as the information processing device 1 may include an assignment processing unit F8 that assigns an application corresponding to the direction of a slide operation. By providing the assignment processing unit F8, it becomes possible to assign the launch of an application that the user frequently uses as a predetermined response process. Therefore, it is possible to realize the response process that is optimal for the requirements of each user.

[0255] 27 to 29 , the smartphone terminal 1A as the information processing device 1 includes a storage unit 79 that stores first unevenness information Das1, which is unevenness information for the first operation area ArH1, and second unevenness information Das2, which is unevenness information for the second operation area ArH2 other than the first operation area ArH1 on the user's hand. The determination processing unit F4 determines that an area other than the first operation area ArH1 has been used for the touch operation when the unevenness information detected by the unevenness detection unit F3 matches the second unevenness information Das2, and may further include a recommendation processing unit F9 that determines whether the application assigned by the assignment processing unit F8 should ensure predetermined security and, if it is determined that the predetermined security should be ensured, prompts the user to register the second unevenness information Das2. When an application requiring payment processing, for example, is assigned, the user is prompted to register the second unevenness information Das2 for the second operation area ArH2 used for the touch operation that triggers the predetermined response processing. Therefore, it is possible to verify that the touch operation that triggers the execution of a specified response process was performed by the target user himself, and it is possible to prevent an application that should ensure high security from being launched by someone else.

[0256] As described with reference to Figures 4, 12, 21, etc., the unevenness detection unit F3 in the smartphone terminal 1A serving as the information processing device 1 may detect unevenness information by causing a light-emitting unit used for displaying images on the screen to emit light in a detection mode. The light-emitting unit emits white light in the detection mode to read a user's fingerprint, for example. The white light emitted by the light-emitting unit can be achieved by reusing the light-emitting unit used for displaying images, without the need to provide a new light-emitting unit for fingerprint detection. This makes it possible to suppress increases in the cost of the information processing device 1. Furthermore, because an increase in the number of light-emitting units can be suppressed, there is no need to increase the difficulty of design and manufacturing.

[0257] As described with reference to FIG. 4 and other figures, the unevenness detection unit F3 in the smartphone terminal 1A serving as the information processing device 1 may illuminate only the light-emitting units corresponding to the identified operation area ArD in a detection mode. This allows light-emitting units corresponding to areas other than the operation area ArD touched by the user's finger or the like to only emit light to display the image on the screen. This prevents white light from leaking from the gap between the screen and the finger, thereby reducing glare during nighttime use. It also prevents the misidentification of a malfunction of the information processing device 1 due to unnecessary white light emission from areas not touched by the finger or the like. Furthermore, when the pad and distal portion of the user's finger are defined as the second operation area ArH2 and a touch operation is performed using this area, an area larger than the fingertip is defined as the operation area ArD. Therefore, the illuminated area on the display unit 3 is relatively wide, allowing for optimal reading of unevenness information, thereby speeding up authentication processing and shortening the time required for the corresponding processing to be performed.

[0258] As described with reference to Figures 20 and 21, the smartphone terminal 1A as the information processing device 1 includes a storage unit 79 that stores first unevenness information Das1, which is unevenness information for the first operation area ArH1, and second unevenness information Das2, which is unevenness information for the second operation area ArH2 other than the first operation area ArH1 on the user's hand. The unevenness detection unit F3 may cause only the light-emitting unit corresponding to the second operation area ArH2 in the identified operation area ArD to emit light in a detection mode. For example, if the first operation area ArH1 is a fingertip area and the second operation area ArH2 is a finger pad area, and both the fingertip area and the finger pad area are touching the screen, according to this configuration, the light-emitting unit emits light so that only the finger pad area is brightly illuminated to read fingerprints, etc., in the finger pad area. This makes it possible to narrow the area emitting white light and suppress increases in power consumption.

[0259] As described in the modified examples, the unevenness detection unit F3 in the smartphone terminal 1A serving as the information processing device 1 may detect unevenness information about the user's operation area ArH used in a touch operation on the fixed area when the operation area ArD is a fixed area on the screen. That is, the process of identifying the operation area ArD touched by the user is unnecessary, thereby reducing the processing burden on the smartphone terminal 1A. Furthermore, since the user can be explicitly informed that unevenness information about the user's finger is being acquired, the user can be made aware of the need to touch the screen with a finger or the like in a manner that allows the unevenness information to be properly read, thereby avoiding errors in reading the unevenness information.

[0260] 19 and 22 , in the smartphone terminal 1A serving as the information processing device 1, the shape corresponding to the second operation area ArH2 may be different depending on the position on the screen (on the display unit 3). This makes it possible to properly detect a touch operation in the second operation area ArH2 even when a touch operation is performed at a different position on the screen, and to activate a predetermined function as intended by the user.

[0261] The information processing method in the present technology is performed by the information processing device 1 through the following steps: detecting a touch operation by a user and identifying an operated area ArD, which is the area on the screen (on the display unit 3) where the touch operation was performed; detecting unevenness information about the user's operation area ArH used for the touch operation on the operated area ArD; determining, based on the detected unevenness information, whether the operation area ArH is a first operation area ArH1 that is a predetermined partial area of ​​the user's hand; and executing a predetermined corresponding process when it is determined that an area on the user's hand other than the first operation area ArH1 was used for the touch operation.

[0262] The program in the present technology causes the information processing device 1 to perform the following processes: a process of detecting a touch operation by a user and identifying an operated area ArD, which is the area on the screen (on the display unit 3) where the touch operation was performed; a process of detecting unevenness information about the user's operation area ArH used for the touch operation on the operated area ArD; a process of determining, based on the detected unevenness information, whether the operation area ArH is a first operation area ArH1 which is a predetermined partial area of ​​the user's hand; and a process of executing a predetermined corresponding process when it is determined that an area on the user's hand other than the first operation area ArH1 was used for the touch operation.

[0263] Such a program and information processing method can also provide the various functions and effects described above.

[0264] The above-mentioned program can be pre-recorded on a hard disk drive (HDD) or a ROM in a microcomputer having a CPU, which serves as a recording medium built into a computer or other device. Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, CD-ROM, MO disk, DVD, Blu-ray Disc (registered trademark), magnetic disk, semiconductor memory, or memory card. Such removable recording medium can be provided as a so-called package software. In addition to being installed on a personal computer or the like from a removable recording medium, such a program can also be downloaded from a download site via a network such as a LAN or the Internet.

[0265] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0266] Furthermore, the above-described examples may be combined in any manner, and even when various combinations are used, the above-described various operational effects can be obtained.

[0267] <8. The Present Technology> The present technology may also be configured as follows. (1) An information processing device comprising: a concavo-convex detection unit that detects concavo-convex information about a user's operation area used for a touch operation on an operated area that is an area on a screen; a determination processing unit that determines whether the operation area is a first operation area that is a predetermined partial area of ​​the user's hand based on the concavo-convex information detected by the concavo-convex detection unit; and a response processing unit that executes predetermined response processing when it is determined that an area of ​​the user's hand other than the first operation area was used for the touch operation. (2) The information processing device described in (1) above includes: a storage unit that stores first concavo-convex information that is the concavo-convex information about the first operation area, and the determination processing unit determines that an area other than the first operation area was used for the touch operation when the concavo-convex information detected by the concavo-convex detection unit differs from the first concavo-convex information. (3) The information processing device according to (1), further comprising: a storage unit that stores first unevenness information that is the unevenness information about the first operation area and second unevenness information that is the unevenness information about a second operation area on a user's hand other than the first operation area, wherein the determination processing unit determines that an area other than the first operation area has been used for the touch operation when the unevenness information detected by the unevenness detection unit matches the second unevenness information. (4) The information processing device according to any of (1) to (3), further comprising: a first operation area that is an area frequently used for the touch operation and is a fingertip area of ​​a predetermined finger; and a second operation area that is an area less frequently used for the touch operation than the fingertip area and is a pad area of ​​the predetermined finger. (6) The information processing device according to any one of (1) to (5), further comprising: a specific operation detection unit that detects a specific operation by a user; and the unevenness detection unit detects the unevenness information when the specific operation is detected.(7) The information processing device according to any of (3) or (5), wherein the unevenness detection unit detects the unevenness information when a shape of the operated area where the touch operation has been performed corresponds to the second operation area. (8) The determination processing unit compares features common to a plurality of users for each of the first operation area and a second operation area other than the first operation area with features included in the unevenness information detected by the unevenness detection unit, and determines that the second operation area has been used for the touch operation when a feature of the second operation area is included in the features included in the unevenness information. (9) The information processing device according to any of (1) to (8), wherein the response processing unit executes, as the predetermined response processing, a pointing operation detection process that detects a slide operation and a touch-up operation, and a start / stop process that, when the touch-up operation has been detected, starts or stops a corresponding application depending on the direction of the slide operation. (10) The information processing device according to (9) above, wherein the corresponding processing unit detects a touch-up operation not accompanied by the slide operation in the instruction operation detection processing. (11) The information processing device according to any of (9) to (10) above, wherein, in the start-up / stop processing, different applications are started or stopped depending on the direction of the slide operation. (12) The information processing device according to any of (9) to (10) above, wherein, in the start-up / stop processing, the same application is started in different modes depending on whether the direction of the slide operation is a first direction or a second direction. (13) The information processing device according to any of (9) to (12) above, further comprising an assignment processing unit that assigns the corresponding application to the direction of the slide operation.(14) The information processing device according to any one of (1) to (14) above, further comprising: a storage unit that stores first unevenness information that is the unevenness information about the first operation area; and second unevenness information that is the unevenness information about a second operation area on a user's hand other than the first operation area, wherein the determination processing unit determines that an area other than the first operation area has been used for the touch operation when the unevenness information detected by the unevenness detection unit matches the second unevenness information, and a recommendation processing unit that determines whether an application assigned by the assignment processing unit should ensure predetermined security and prompts registration of the second unevenness information when it is determined that the predetermined security should be ensured. (15) The information processing device according to any one of (1) to (14) above, wherein the unevenness detection unit detects the unevenness information by causing a light-emitting unit used for image display on the screen to emit light in a detection mode. (16) The information processing device according to (15), wherein the unevenness detection unit causes only the light-emitting unit that corresponds to the identified operated area to emit light in the detection mode. (17) The information processing device according to any one of (1) to (17), further comprising: a storage unit configured to store first unevenness information that is the unevenness information about the first operation area, and second unevenness information that is the unevenness information about a second operation area on a user's hand other than the first operation area, wherein the unevenness detection unit causes only the light-emitting unit that corresponds to the second operation area among the identified operated areas to emit light in the detection manner. (18) The information processing device according to any one of (1) to (17), further comprising: when the operated area is a fixed area on the screen, the unevenness detection unit detects unevenness information about a user's operation area used for the touch operation on the fixed area. (19) An information processing method in which an information processing device performs the following processes: a process of detecting unevenness information about a user's operation area used for a touch operation on an operated area, which is an area on a screen; a process of determining, based on the detected unevenness information, whether or not the operation area is a first operation area that is a predetermined partial area of ​​the user's hand; and a process of executing a predetermined corresponding process when it is determined that an area of ​​the user's hand other than the first operation area was used for the touch operation.(20) A program that causes an information processing device to execute the following processes: a process of detecting unevenness information about a user's operation area used for a touch operation on an area to be operated, which is an area on a screen; a process of determining, based on the detected unevenness information, whether or not the operation area is a first operation area that is a predetermined partial area of ​​the user's hand; and a process of executing a predetermined corresponding process when it is determined that an area other than the first operation area of ​​the user's hand was used for the touch operation.

[0268] 1 Information processing device 1A Smartphone terminal 1B Tablet terminal 3 Display unit (screen) 79 Memory unit ArD Operated area ArH Operation area ArH1 First operation area ArH2 Second operation area Das1 First unevenness information Das2 Second unevenness information F1 Operated area detection unit F2 Specific operation detection unit F3 Unevenness detection unit F4 Determination processing unit F5 Correspondence processing unit F8 Allocation processing unit F9 Recommendation processing unit

Claims

1. An information processing device comprising: a bump detection unit that detects bump information about a user's operation area used for a touch operation on an operated area, which is an area on a screen; a determination processing unit that determines whether the operation area is a first operation area, which is a predetermined partial area of the user's hand, based on the bump information detected by the bump detection unit; and a response processing unit that executes predetermined response processing when it is determined that an area of the user's hand other than the first operation area was used for the touch operation.

2. An information processing device as described in claim 1, further comprising a memory unit that stores first unevenness information, which is the unevenness information for the first operation area, and wherein the judgment processing unit judges that an area other than the first operation area has been used for the touch operation when the unevenness information detected by the unevenness detection unit differs from the first unevenness information.

3. An information processing device as described in claim 1, further comprising a memory unit that stores first unevenness information, which is the unevenness information about the first operation area, and second unevenness information, which is the unevenness information about a second operation area other than the first operation area on the user's hand, and wherein the determination processing unit determines that an area other than the first operation area has been used for the touch operation when the unevenness information detected by the unevenness detection unit matches the second unevenness information.

4. The information processing device according to claim 1, wherein the first operation area is an area that is frequently used for the touch operation and is a fingertip area of a predetermined finger.

5. An information processing device according to claim 3, wherein the first operation area is an area used more frequently for the touch operation and is the fingertip area of a specified finger, and the second operation area is an area used less frequently for the touch operation than the fingertip area and is the pad area of the specified finger.

6. An information processing device according to claim 1, further comprising a specific operation detection unit that detects a specific operation by a user, wherein the unevenness detection unit detects the unevenness information when the specific operation is detected.

7. The information processing device according to claim 3, wherein the unevenness detection unit detects the unevenness information when the shape of the operated area where the touch operation is performed corresponds to the shape of the second operation area.

8. The information processing device according to claim 1, wherein the determination processing unit compares features common to a plurality of users for each of the first operation area and a second operation area other than the first operation area with features included in the unevenness information detected by the unevenness detection unit, and determines that the second operation area was used for the touch operation if the features included in the unevenness information include features for the second operation area.

9. The information processing device according to claim 1, wherein the response processing unit executes, as the predetermined response processing, an instruction operation detection processing for detecting a slide operation and a touch-up operation, and a start / stop processing for starting or stopping a corresponding application depending on the direction of the slide operation when the touch-up operation is detected.

10. The information processing device according to claim 9, wherein the corresponding processing unit detects a touch-up operation that does not involve the slide operation in the instruction operation detection process.

11. The information processing device according to claim 9, wherein the start / stop process starts or stops different applications depending on the direction of the slide operation.

12. The information processing device according to claim 9, wherein the start / stop process starts the same application in different modes depending on whether the direction of the slide operation is a first direction or a second direction.

13. The information processing device according to claim 9, further comprising an allocation processing unit that allocates the corresponding application to the direction of the slide operation.

14. An information processing device as described in claim 13, further comprising: a memory unit that stores first unevenness information, which is the unevenness information about the first operation area, and second unevenness information, which is the unevenness information about a second operation area other than the first operation area on the user's hand; the determination processing unit that determines that an area other than the first operation area was used for the touch operation when the unevenness information detected by the unevenness detection unit matches the second unevenness information; and a recommendation processing unit that determines whether or not the application assigned by the assignment processing unit should ensure a predetermined security, and prompts registration of the second unevenness information when it is determined that the predetermined security should be ensured.

15. The information processing device according to claim 1, wherein the unevenness detection unit detects the unevenness information by causing a light-emitting unit used for image display on the screen to emit light in a detection mode.

16. The information processing device according to claim 15, wherein the unevenness detection unit causes only the light-emitting unit corresponding to the identified operated area to emit light in the detection mode.

17. An information processing device as described in claim 15, further comprising a memory unit that stores first unevenness information, which is the unevenness information about the first operation area, and second unevenness information, which is the unevenness information about a second operation area other than the first operation area on the user's hand, and wherein the unevenness detection unit causes only the light-emitting unit that corresponds to the second operation area among the identified operated areas to emit light in the detection manner.

18. The information processing device according to claim 1, wherein the unevenness detection unit detects unevenness information about the user's operation area used for the touch operation on the fixed area when the operated area is a fixed area on the screen.

19. An information processing method in which an information processing device performs the following processes: a process of detecting unevenness information about a user's operation area used for a touch operation on an operated area, which is an area on a screen; a process of determining whether or not the operation area is a first operation area, which is a predetermined partial area of the user's hand, based on the detected unevenness information; and a process of executing a predetermined corresponding process when it is determined that an area of the user's hand other than the first operation area was used for the touch operation.

20. A program that causes an information processing device to execute the following processes: a process of detecting unevenness information about a user's operation area used for a touch operation on an operated area, which is an area on a screen; a process of determining whether or not the operation area is a first operation area, which is a predetermined partial area of the user's hand, based on the detected unevenness information; and a process of executing a predetermined corresponding process when it is determined that an area of the user's hand other than the first operation area was used for the touch operation.

Images