Information processing program, information processing method, and information processing apparatus

Abstract

To solve the problems in which there is a need for a mechanism of an input device for an ATM that does not need direct touch other than a touch panel that needs a touch on the device for operation, however, a special device, such as an aerial display, is costly or security problems, such as illegal access or eavesdropping, may arise from the use of a personal device or voice recognition for operation, resulting in difficulty of non-contact transaction operation on the ATM.SOLUTION: An information processing program causes a computer to execute processes of: detecting the shape of the hand of a user from video or an image captured by a camera device; identifying operation of a transaction associated with the detected shape of the hand; and executing the identified operation.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus.

Background Art

[0002] Currently, as an input device, an automatic teller machine (ATM) mainly uses a touch panel to perform operations for various transactions. Therefore, it is required that the user directly touches the touch panel provided in the ATM by hand. On the other hand, from the perspective of infectious disease prevention, there is a need to perform operations without touching the device as much as possible, which is used by a large number of unspecified users.

[0003] Since touching the device is inevitable in the operation on the touch panel, a mechanism is required, such as installing an input device that does not need to be directly touched other than the touch panel in the ATM, or performing operations with a personal-owned device such as a smartphone.

[0004] As an example of a means for operating without touching the device, a device called an air display has been developed, which displays a screen like a hologram in space and allows the user to operate by touching the screen in space. Also, although there is no implementation example in the ATM, a mechanism that enables operation in a language by voice recognition using a microphone has been implemented in other devices.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, special devices such as air displays are costly, and there are security issues such as unauthorized access and eavesdropping with personal owned devices or operations by voice recognition. Therefore, it is not easy to perform ATM transaction operations without physical contact.

[0007] On one aspect, an object of the ATM is to provide an information processing program, an information processing method, and an information processing apparatus that enable a user to more easily perform transaction operations without physical contact.

Means for Solving the Problem

[0008] In one embodiment, the information processing program causes a computer to execute a process of detecting a user's hand shape from a video or image captured by a camera device, specifying a transaction operation associated with the detected hand shape, and executing the specified operation.

[0009] Also, in another embodiment, a computer executes a process of detecting a user's hand shape from a video or image captured by a camera device, specifying a transaction operation associated with the detected hand shape, and executing the specified operation.

[0010] Furthermore, in yet another embodiment, the information processing apparatus has a control unit that executes a process of detecting a user's hand shape from a video or image captured by a camera device, specifying a transaction operation associated with the detected hand shape, and executing the specified operation.

Advantages of the Invention

[0011] On one aspect, the ATM can enable a user to more easily perform transaction operations without physical contact.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0013] Hereinafter, examples of an information processing program, an information processing method, and an information processing apparatus according to the present embodiment will be described in detail with reference to the drawings. Note that the present embodiment is not limited to these examples. Also, the respective examples can be appropriately combined within a non - contradictory range.

[0014] [OVERALL CONFIGURATION OF INFORMATION PROCESSING SYSTEM 1] First, an information processing system for implementing the present embodiment will be described. FIG. 1 is a diagram showing a configuration example of the information processing system 1 according to the present embodiment. As shown in FIG. 1, the information processing system 1 is a system in which an information processing apparatus 10 and ATMs 100 - 1 to 100 - n (n is an arbitrary natural number; hereinafter, collectively referred to as "ATM 100") are connected to be communicable with each other via a network 50.

[0015] The network 50 can adopt various communication networks such as the Internet, regardless of whether it is wired or wireless. Further, the network 50 is not a single network, and for example, an intranet and the Internet may be configured via a network device such as a gateway or other devices.

[0016] The information processing device 10 is an information processing device such as a server computer managed by, for example, a financial institution that manages the ATM 100. The information processing device 10 executes various transactions such as transfers in response to requests from the ATM 100 and transmits information such as the execution results of the transactions to the ATM 100.

[0017] In FIG. 1, the information processing device 10 is shown as a single computer, but it may also be a distributed computing system composed of multiple computers. Further, the information processing device 10 may be a cloud computer device managed by a service provider that provides cloud computing services.

[0018] The ATM 100 is a cash automated transaction device installed, for example, at an ATM corner of each branch of a financial institution. FIG. 2 is a diagram showing an example of the ATM 100 according to the present embodiment. As shown in FIG. 2, the ATM 100 includes, for example, a touch panel type display 150 for a user to perform a transaction operation, and a camera device 110. Note that the camera device 110 may be, for example, a camera built in the ATM 100 or an external camera communicably connected to the ATM 100.

[0019] The ATM100 detects the shape of a user's hand from, for example, video or an image captured by the camera device 110, identifies a transaction operation associated with the detected hand shape, and executes the identified operation. Here, the operations executed are, for example, the same operations as various operations that a user executes by operating a touch panel display 150, but various operations can be executed without contact by detecting the shape of the user's hand. Note that, for example, even in this case, various operations may be executed by a touch operation on the display 150. Also, strictly speaking, the video captured by the camera device 110 is composed of a plurality of images captured by the camera device 110, that is, a series of frames of a moving image. Therefore, the object for hand detection is the video or image captured by the camera device 110.

[0020] [Functional Configuration of ATM100] Next, the functional configuration of the ATM100 will be described. FIG. 3 is a diagram showing a configuration example of the ATM100 according to the present embodiment. As shown in FIG. 3, the ATM100 includes a communication unit 120, a storage unit 130, and a control unit 140.

[0021] The communication unit 120 is a processing unit that controls communication with other devices such as the information processing device 10, and is, for example, a communication interface such as a network interface card.

[0022] The storage unit 130 has a function of storing various data and programs executed by the control unit 140, and is realized by, for example, a storage device such as a memory or a hard disk. The storage unit 130 stores video information 131, transaction information 132, screen information 133, and the like.

[0023] The video information 131 stores, for example, video captured by the camera device 110 and information related to the video. The information related to the video stored in the video information 131 is, for example, the imaging date and time of the video, the identifier of the camera device 110 that captured the video, the identifier of the ATM100 that is communicably connected to the camera device 110, and the like.

[0024] The transaction information 132 stores information regarding transactions executed by the user using the ATM 100, for example. The information regarding transactions stored in the transaction information 132 includes, for example, information on various transactions such as transfers executed using the ATM 100, and account information of the user performing the transaction. For example, when the information regarding the transaction stored in the transaction information 132 is transmitted to the information processing device 10, various transactions are executed by the information processing device 10.

[0025] The screen information 133 stores information regarding various transaction screens such as transfer screens and menu screens for various transactions displayed on the display 150, for example. FIG. 4 is a diagram showing an example of a transaction selection screen according to the present embodiment. FIG. 4 is an example of a so-called menu screen, a transaction selection screen for the user to select a transaction to be performed, which is displayed on the display 150 of the ATM 100.

[0026] In the example of FIG. 4, buttons 160-1 to 160-4 (hereinafter collectively referred to as "buttons 160") for performing each transaction are displayed on the transaction selection screen. For example, when the user wants to make a deposit, by touching (pressing) the button 160-3, the user can transition to a screen for performing a deposit transaction and perform a deposit transaction operation.

[0027] In this case, since the display 150 part is touched (contacted), in the present embodiment, as shown in FIG. 4, furthermore, hand sign icons 170-1 to 170-4 (hereinafter collectively referred to as "icons 170") corresponding to each operation are displayed. For example, when the user wants to make a deposit non-contact with the display 150 part, the user forms a hand shape indicated by the hand sign icon 170-3 corresponding to the deposit with respect to the camera device 110. Thereby, for example, a deposit transaction operation similar to the case where the button 160-3 is touched (pressed) can be performed.

[0028] Note that the hand signs for performing each operation without contact are not limited to those in FIG. 4. FIG. 5 is a diagram showing an example of hand signs. Although FIG. 5 is also just an example, as shown in FIG. 5, various hand signs may be used to perform each operation without contact. Note that, as shown in FIG. 4, when displaying the hand sign icons 170 corresponding to each operation, each of the icons 170 may not be fixed to each operation but may be randomly associated and displayed. More specifically, in the example of FIG. 4, the thumbs-up icon 170-1 is displayed in association with the balance inquiry. However, for example, each time the transaction selection screen is displayed, different hand sign icons 170 may be selected and associated with each operation and displayed. Thereby, for example, even if a malicious person memorizes each operation on the screen and the corresponding hand signs, since the hand signs corresponding to each operation change each time the screen is displayed, it is impossible to determine what kind of transaction the user is performing, and security can be further enhanced.

[0029] Also, for example, each screen and the hand signs corresponding to each operation that can be performed on each screen may be fixedly or randomly associated and stored in the screen information 133. FIG. 6 is a diagram showing an example of the information stored in the screen information 133 according to the present embodiment. As shown in FIG. 6, for example, in the screen information 133, a "screen number" indicating the identifier of each screen, a "hand sign type" indicating the type of hand sign, and an "operation" indicating the operation associated with the hand sign may be associated and stored. Note that, for example, the first to fourth lines (records of screen number #0000) in the data record part of the screen information 133 shown in FIG. 6 are records corresponding to the transaction selection screen shown in FIG. 4.

[0030] Returning to the description of FIG. 3, note that the above information stored in the storage unit 130 is just an example, and the storage unit 130 can store various information other than the above information.

[0031] The control unit 140 is a processing unit that controls the entire ATM 100 and is, for example, a processor or the like. The control unit 140 includes a detection unit 141, an identification unit 142, an execution unit 143, and a display unit 144. Each processing unit is an example of an electronic circuit included in the processor or an example of a process executed by the processor.

[0032] The detection unit 141 detects the shape of the user's hand from, for example, the video or image captured by the camera device 110. The detection of the hand shape may be performed using, for example, existing object detection techniques.

[0033] The identification unit 142 identifies, for example, the transaction operation associated with the hand shape detected by the detection unit 141. More specifically, for example, the identification unit 142 identifies the transaction operation associated with the hand shape by comparing with the mapping data between the hand sign (hand shape) type for each screen and the corresponding operation as shown in FIG. 6.

[0034] Note that the identification of such a transaction operation may be, for example, a process of identifying the transaction operation associated with the change in the hand shape detected by the detection unit 141. The change in the hand shape may be, for example, a change from the opening of the hand sign to a thumbs up. Also, the identified transaction operation may be a financial transaction such as a transfer as shown in FIG. 4.

[0035] Also, the identification of such a transaction operation may be a process of identifying the transaction operation associated with the hand shape continuously detected by the detection unit 141 for a certain period of time in order to suppress misoperations and misdetections.

[0036] The execution unit 143 executes, for example, the operation identified by the identification unit 142. FIG. 7 is a diagram showing an example of the operation of the deposit transaction according to the present embodiment. FIG. 7 is an example showing the flow of the screen displayed on the display 150 of the ATM 100 when the deposit operation is executed by the execution unit 143. Also, the numbers such as #0000 shown in the upper left of each screen are screen numbers and correspond to, for example, the "screen number" of the information shown in FIG. 6.

[0037] For example, when performing a deposit transaction, as shown in FIG. 7, in accordance with the icon 170-3 on the screen indicated by the screen number #0000, the user makes a hand gesture into a finger point with respect to the camera device 110. Thereby, for example, when the deposit button 160-3 is touched (pressed), the screen transitions to the next screen indicated by the screen number #0100. Then, also on the screen indicated by the screen number #0100, in accordance with the icons 170-5 and 170-6 displayed in association with each operation, the user can perform the same operations as when touching the buttons 160-5 and 160-6 by showing a hand sign to the camera device 110.

[0038] FIG. 8 is a diagram showing an example of the operation of the withdrawal transaction according to the present embodiment. FIG. 8 is an example showing the flow of the screen displayed on the display 150 of the ATM 100 when a deposit withdrawal operation is executed by the execution unit 143. Also in the example of FIG. 8, similar to the example of FIG. 7, when performing a deposit withdrawal operation, in accordance with the icon 170-2 on the screen indicated by the screen number #0000, the user makes a hand gesture into a peace sign with respect to the camera device 110 and can perform the same operations as when touching the button 160-2 (pressing). Further, also on the next screen indicated by the screen number #0200, the user can perform the same operations as when touching the buttons 160-7 and 160-8 by showing a hand sign to the camera device 110 in accordance with the icons 170-7 and 170-8.

[0039] Returning to the description of FIG. 3, the display unit 144 displays, for example, via the display 150, in association, a display indicating an operation of a transaction associated with a hand shape and a display indicating the hand shape associated with the operation. More specifically, as shown in FIG. 4, for example, the display unit 144 displays, in association, a display indicating each operation such as a balance inquiry and a display indicating the hand shape associated with each operation, that is, 170-1 to 170-4. Note that the display indicating the hand shape displayed on each screen is not fixed for each operation and may be a display of a hand shape randomly associated with each operation. Further, for preventing misoperations and false detections, after a display indicating an operation of a transaction associated with a hand shape and a display indicating the hand shape associated with the operation are displayed, detection of the hand shape by the detection unit 141 may be started after a lapse of a certain period of time. Thereby, for example, even when the user makes a specific hand sign for an operation on the previous screen, the ATM 100 can prevent the specific hand sign from being detected immediately after the screen transition and an incorrect operation from being executed.

[0040] [Flow of processing] Next, the flow of the transaction operation execution process executed by the ATM 100 will be described. FIG. 9 is a flowchart showing the flow of the transaction operation execution process according to the present embodiment. The transaction operation execution process shown in FIG. 9 starts, for example, from when the previous user of the ATM 100 is detected.

[0041] First, as shown in FIG. 9, the ATM 100 displays a screen on the display 150, for example (step S101). The screen displayed here may be, for example, a transaction selection screen as shown in FIG. 4, that is, a so-called menu screen, if the user is detected immediately.

[0042] Next, the ATM 100 acquires, for example, an image captured by the camera device 110 (step S102). Note that, for example, the image acquired here may be a video captured by the camera device 110. However, in the description using FIG. 9, the description will be made with the "image" including the video. Further, since the image captured by the camera device 110 is stored in the video information 131 at any time, the ATM 100 acquires the image from the video information 131, for example.

[0043] Next, the ATM 100 determines, for example, whether or not the user's hand is detected from the image acquired in step S102 using an existing object detection technique (step S103). If the user's hand is not detected (step S103: No), the process returns to step S102, and the ATM 100 acquires the next image, for example.

[0044] On the other hand, if the user's hand is detected (step S103: Yes), the ATM 100 determines, for example, whether or not a transaction operation corresponding to the detected hand shape can be identified (step S104). If the transaction operation corresponding to the hand shape cannot be identified (step S104: No), the process returns to step S102, and the ATM 100 acquires the next image, for example.

[0045] On the other hand, if the transaction operation corresponding to the hand shape can be identified (step S104: Yes), the ATM 100 executes the identified operation, for example (step S105). After the execution of step S105, the transaction operation execution process shown in FIG. 9 ends. However, for example, if the identified operation is a screen transition or the like and there is a next operation, the process returns to step S101, the ATM 100 displays the next screen, and the process is repeated from step S102. Further, if the identified operation is an execution of a transfer or the like, the ATM 100 transmits, for example, information related to the transaction stored in the transaction information 132 to the information processing device 10, and the information processing device 10 executes the identified operation such as the execution of the transfer.

[0046] [Effect] The ATM 100 detects the shape of the user's hand from the video or image captured by the camera device 110, identifies the transaction operation associated with the detected hand shape, and executes the identified operation.

[0047] In this way, without having the user touch the display 150 of the ATM 100 or the like, the ATM 100 can identify and execute the transaction operation from the shape of the user's hand, enabling the user to perform the transaction operation more easily in a non-contact manner.

[0048] Also, the process of identifying the operation executed by the ATM 100 includes the process of identifying the transaction operation associated with the detected change in the hand shape.

[0049] Thereby, the ATM 100 can enable the user to perform the transaction operation more easily in a non-contact manner.

[0050] Also, the process of identifying the operation executed by the ATM 100 includes the process of identifying the operation of the financial transaction associated with the detected hand shape.

[0051] Thereby, the ATM 100 can enable the user to perform the operation of the financial transaction more easily in a non-contact manner.

[0052] Also, the process of identifying the operation executed by the ATM 100 includes the process of identifying the transaction operation associated with the hand shape continuously detected for a certain period of time.

[0053] Thereby, the ATM 100 can prevent the user's misoperation and misdetection of the hand.

[0054] Also, the ATM 100 executes a process of associating and displaying the display indicating the operation with the display indicating the hand shape associated with the operation.

[0055] Thereby, the ATM 100 can enable the user to perform the transaction operation more easily in a non-contact manner.

[0056] Further, the ATM 100 executes a process of associating and displaying a display indicating an operation with a display indicating a hand shape randomly associated with the operation.

[0057] Thereby, the ATM 100 can make it easier for the user to perform a transaction operation in a contactless manner while further enhancing security.

[0058] Also, the process of detecting a hand shape executed by the ATM 100 includes a process of starting to detect a hand shape from a video or an image after a certain period of time has elapsed since displaying a display indicating an operation and a display indicating a hand shape.

[0059] Thereby, the ATM 100 can prevent incorrect operations by the user and incorrect detection of the hand.

[0060] [System] The processing procedures, control procedures, specific names, information including various data and parameters shown in the above documents and drawings may be arbitrarily changed unless otherwise specified. Also, the specific examples, distributions, numerical values, etc. described in the embodiments are merely examples and may be arbitrarily changed.

[0061] Also, the specific forms of the distribution and integration of the components of each device are not limited to those shown in the drawings. That is, all or part of the components may be functionally or physically distributed and integrated in any unit according to various loads, usage situations, etc. Further, each processing function of each device may be realized in whole or in any part thereof by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

[0062] [Hardware] FIG. 10 is a diagram for explaining the hardware configuration of the ATM 100. As shown in FIG. 10, the ATM 100 includes a communication interface 100a, an HDD 100b, a memory 100c, a processor 100d, a display 100e, and a camera 100f. Further, each part shown in FIG. 10 is interconnected by a bus or the like.

[0063] The communication interface 100a is, for example, a network interface card or the like, and communicates with other information processing apparatuses such as the information processing apparatus 10. The HDD 100b stores, for example, programs and data for operating each function provided in the ATM 100.

[0064] The processor 100d is, for example, a hardware circuit that operates a process for executing each function by reading various programs for executing the processing of each function provided in the ATM 100 from the HDD 100b or the like and expanding them in the memory 100c. That is, this process executes the same functions as each processing unit included in the ATM 100.

[0065] In this way, the ATM 100 operates as an information processing apparatus that executes operation control processing by, for example, reading and executing various programs for executing the processing of each function of the ATM 100. Further, the ATM 100 can also realize each function provided in the ATM 100 by, for example, reading a program from a recording medium by a medium reading device and executing the read program. Note that the program for realizing each function is not limited to being executed by the ATM 100. For example, the present embodiment may be similarly applied when another computer executes the program, or when the ATM 100 and another computer cooperate to execute the program.

[0066] In addition, various programs for executing the processing of each function of the ATM 100 can be distributed via a network such as the Internet. Further, this program can be recorded on a computer-readable recording medium such as a hard disk, FD, CD-ROM, MO, DVD, etc., and can be executed by being read from the recording medium by a computer.

[0067] The display 100e is, for example, the display 150, and displays various visual information based on the control by the processor 100d. The display 100e may be a liquid crystal display (LCD), an OLED (Organic Light Emitting Diode), a so-called organic EL (Electro Luminescence) display, etc. Further, the display 100e may be a touch panel type display, and detects various input operations by the user such as a touch input operation for a program executed by the processor 100d. In addition to the touch input operation, various input operations may include inputs via an input unit such as a button or a proximity sensor.

[0068] The camera 100f is, for example, the camera device 110, and is a camera built in the ATM 100. Note that, for example, in the case where the camera device 110 is an external camera communicably connected to the ATM 100, the ATM 100 may not have the camera 100f.

Explanation of Signs

[0069] 1 Information processing system 50 Network 100 ATM 100a Communication interface 100b HDD 100c Memory 100d Processor 100e Display 100f Camera 110 Camera device 120 Communication unit 130 Memory unit 131 Video information 132 Transaction information 133 Screen information 140 Control unit 141 Detection unit 142 Identification unit 143 Execution unit 144 Display unit 150 Display 160 Button 170 Icon

Claims

1. Detect the shape of the user's hand from the video or image captured by the camera device, Identify the operation of the transaction associated with the detected hand shape, Execute the identified operation An information processing program characterized by causing a computer to execute the processing.

2. The process of identifying the operation Includes the process of identifying the operation of the transaction associated with the detected change in the hand shape The information processing program according to claim 1, characterized in that it includes the processing.

3. The process of identifying the operation Includes the process of identifying the operation of the financial transaction associated with the detected hand shape The information processing program according to claim 1 or 2, characterized in that it includes the processing.

4. The process of identifying the operation Includes the process of identifying the operation of the transaction associated with the hand shape continuously detected for a certain period of time The information processing program according to claim 1, characterized in that it includes the processing.

5. Causing the computer to execute the process of associating and displaying the display indicating the operation and the display indicating the hand shape associated with the operation The information processing program according to claim 1, characterized in that it includes the processing.

6. Causing the computer to execute the process of associating and displaying the display indicating the operation and the display indicating the hand shape randomly associated with the operation The information processing program according to claim 1, characterized in that it includes the processing.

7. The process of detecting the hand shape Includes the process of starting the detection of the hand shape from the video or the image after a certain period of time has elapsed since the display indicating the operation and the display indicating the hand shape are displayed The information processing program according to claim 5 or 6, characterized in that it includes the processing.

8. Detect the shape of the user's hand from the video or image captured by the camera device, Identify the operation of the transaction associated with the detected hand shape, Execute the identified operation An information processing method characterized by a computer executing the processing.

9. Detect the shape of the user's hand from the video or image captured by the camera device, Identify the operation of the transaction associated with the detected hand shape, Execute the identified operation An information processing device characterized by having a control unit that executes the processing.

Images