Settlement terminal and settlement assistance method

Abstract

This settlement terminal capable of settlement using a code includes: a camera that can capture an image of a code used for settlement and captures an image of a subject; and a processor that processes the captured image captured by the camera. The processor determines whether a second person other than a first person using the settlement terminal is captured together with the first person in the captured image, estimates a relative positional relationship between the first person and the second person when the second person is captured, and determines whether or not the second person is the peeping person on the basis of the relative positional relationship.

Description

Payment terminal and payment support method

[0001] The present disclosure relates to a payment terminal and a payment support method.

[0002] There are devices that use an imaging device to detect whether or not someone is peeping depending on the user's surroundings and automatically prevent peeping. Patent Document 1 discloses a display image control unit that prevents others from peeping at an image displayed on a display screen without unnecessarily reducing operability. This display image control unit includes: a person detection unit that processes an image captured by an imaging device capturing an image in front of the display and detects a person viewing the image displayed on the display screen; and an image size setting unit that sets the image displayed on the display screen to a normal size if the person detection unit detects that only one person is viewing the image displayed on the display screen; and a reduced size that is smaller than the normal size if the person detection unit detects that multiple people are viewing the image displayed on the display screen.

[0003] Japanese Patent Application Publication No. 2012-27641

[0004] If the display image control unit of Patent Document 1 is applied to a payment terminal, for example, if both a camera for code payment and a camera for peeping detection are installed in the payment terminal, it may become difficult to maintain the compact size of the payment terminal.

[0005] The present disclosure provides a payment terminal and a payment support method that can improve security while maintaining the compactness of the payment terminal.

[0006] One aspect of the present disclosure is a payment terminal capable of making payments using a code, the payment terminal being capable of capturing an image of the code used for the payment, the payment terminal comprising: a camera that captures an image of a subject; and a processor that processes the captured image captured by the camera, wherein the processor determines whether a second person other than a first person using the payment terminal is captured in the captured image together with the first person; if the second person is captured, the processor estimates the relative positional relationship between the first person and the second person; and determines whether the second person is a peeping person based on the relative positional relationship.

[0007] One aspect of the present disclosure is a payment assistance method executed by a payment terminal capable of making payments using a code, the payment assistance method including the steps of: acquiring an image captured by a camera provided in the payment terminal, the camera being capable of capturing an image of the code used for the payment and capturing an image of a subject; determining whether a second person other than a first person using the payment terminal is captured in the captured image together with the first person; if the second person is captured, estimating a relative positional relationship between the first person and the second person; and determining whether the second person is a peeping person based on the relative positional relationship.

[0008] According to the present disclosure, security can be improved while maintaining the compact size of the payment terminal.

[0009] Schematic diagram showing an example of the appearance of a payment terminal according to an embodiment. Block diagram showing an example of the electrical configuration of the payment terminal. Flowchart showing an example of the operation of the payment terminal. Flowchart showing an example of the operation of the payment terminal for code payment, which is an example of payment without a PIN number. Schematic diagram showing the state of code reading by a camera Schematic diagram showing an embodiment in which it is determined whether a third party is a peeper based on the distance between the user and a third party in peeping detection mode Schematic diagram showing an embodiment in which it is determined whether a third party is a peeper based on the ratio of the size of the user's face to the size of the third party's face in peeping detection mode Schematic diagram showing an embodiment in which it is determined whether a third party is a peeper based on the angle formed between the user and the third party in peeping detection mode Schematic diagram showing the relationship between the display and the line of sight of the user and the third party Diagram showing an example of angle θ1 Schematic diagram showing an area on the display where a third party can peek

[0010] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0011] FIG. 1 is a top view showing an example of the appearance of a payment terminal 100 according to an embodiment. The payment terminal 100 has a main body 10. The main body 10 has, for example, a generally flat plate shape. FIG. 1 shows a front surface 1A of the payment terminal 100. For example, a customer (user) who is a purchaser of a product operates the payment terminal 100 from the front surface 1A. The customer may be a customer of a store. The payment terminal 100 may be a stationary terminal placed on a stand or the like in a store, or may be a portable mobile terminal.

[0012] The payment terminal 100 includes a first card slot 16s, a second card slot 17s, a touch panel 22, a display 23, and a camera 25. The display 23 is a display device that displays images (including, for example, still images and videos). The touch panel 22 is an input means through which the user inputs information. The touch panel 22 and the display 23 are arranged superimposed on each other. As will be described later, the camera 25 is capable of capturing an image of the payment code as well as the user and their surroundings. The content of the captured code is read.

[0013] The payment terminal 100 is a composite type that allows a customer to specify multiple payment methods. The multiple payment methods include, for example, credit card payment, electronic money payment, code (e.g., QR Code (registered trademark)) payment, and cash payment. Credit card payment includes, for example, magnetic card payment, contact IC card payment, and contactless IC card payment. There may be multiple types of electronic money. Electronic money payment and contactless IC card payment are performed using contactless communication (e.g., near field communication (NFC)), and are therefore collectively referred to here as contactless payment.

[0014] The multiple payment methods are each performed using a corresponding multiple payment interface, such as a credit card, an electronic money card, a code, and cash. The credit card interface includes, for example, a magnetic card, a contact IC card, and a contactless IC credit card.

[0015] A magnetic card is inserted into the first card slot 16s and used for payment. A contact IC card is inserted into the second card slot 17s and used for payment. A contactless IC credit card and an electronic money card are used for payment by bringing them close to the NFC antenna 15 (see FIG. 2). Contactless IC credit cards and electronic money cards are collectively referred to as "contactless IC cards."

[0016] Furthermore, a camera 25 is disposed in the payment terminal 100. The camera 25 may be disposed on the same plane as the touch panel 22 or the display 23, or may be disposed below the touch panel 22 or the display 23, but the location is not limited to these.

[0017] Although the touch panel 22 is shown as the user's input means, the payment terminal 100 may also be provided with other input devices such as the keyboard 19 (see FIG. 2).

[0018] Furthermore, the payment terminal 100 has a security protection area SR (see FIG. 2 ) in a portion of the payment terminal 100. The security protection area SR is an area with higher security than areas of the payment terminal 100 other than the security protection area SR. For example, the security protection area SR is tamper-resistant. The security of the security protection area SR may be logically increased by software processing, or may be physically increased by hardware structure or mechanism. The security protection area SR mainly stores components used in payment processing.

[0019] FIG. 2 is a block diagram showing an example of the electrical configuration of the payment terminal 100. As shown in FIG.

[0020] Payment terminal 100 has a security protection area SR. Within security protection area SR, payment terminal 100 includes a first CPU (Central Processing Unit) 11, an NFC_IC 14, an NFC antenna 15, a first card reader 16, a second card reader 17, a memory 18, and a keyboard 19. Outside security protection area SR, payment terminal 100 includes a second CPU 21, a touch panel 22, a display 23, a camera 25, an external terminal 27, and a switch 29.

[0021] The first CPU 11 realizes various functions by executing programs stored in the memory 18. The first CPU 11 controls all parts within the security protection area SR. The first CPU 11 performs, for example, processing related to credit card payments and electronic money payments. The first CPU 11 functions as a payment CPU that performs processing related to payments. The first CPU 11 also transmits and receives data to and from the second CPU 21, and cooperates with the second CPU 21. Note that the first CPU 11 is an example of a processor, and may be another processor.

[0022] The NFC_IC 14 performs processing for the NFC antenna 15. For example, the NFC_IC 14 controls the power supply to the NFC antenna 15 and data communication via the NFC antenna 15.

[0023] The NFC antenna 15 is formed of, for example, a loop coil. The NFC antenna 15 receives power from the NFC IC 14 and generates radio waves. An area within a predetermined distance from the NFC antenna 15 is an area where contactless communication with the NFC antenna 15 is possible (contactless communication area). When a contactless IC card C3 (an example of a communication medium capable of contactless communication) is placed within the contactless communication area, radio waves from the NFC antenna 15 reach the contactless IC card C3, and the NFC antenna 15 provides activation power to the contactless IC card C3. As a result, the NFC antenna 15 communicates data with the contactless IC card C3.

[0024] Furthermore, the center position of the surface defined by the NFC antenna 15 may be approximately the same as the center position of the surface defined by the touch panel 22. In this case, by holding the non-contact IC card over the touch panel 22, data can be communicated between the NFC antenna 15 and the non-contact IC card C3, making the operation easier for the user to understand.

[0025] The NFC_IC 14 acquires (reads) information held by the contactless IC credit card via the NFC antenna 15. The NFC_IC 14 sends this information to the first CPU 11 as contactless read information. The information held by the contactless IC credit card and the contactless read information include, for example, the identification number of the credit card. The NFC_IC 14 also acquires (reads) information held by the electronic money card via the NFC antenna 15. The NFC_IC 14 sends this information to the first CPU 11 as electronic money read information. The information held by the electronic money card and the electronic money read information include, for example, the identification number of the electronic money card.

[0026] The first card reader 16 is a magnetic card reader. The first card reader 16 reads information stored on a magnetic card inserted into the first card slot 16s and sends the read information as magnetically read information to the first CPU 11. The information stored on the magnetic card and the magnetically read information include, for example, a credit card identification number.

[0027] The second card reader 17 is a contact IC card reader. The second card reader 17 reads information stored in a contact IC card inserted into the second card slot 17s and sends the read information as contact read information to the first CPU 11. The information stored in the contact IC card and the contact read information include, for example, a credit card identification number.

[0028] The memory 18 may be configured to include, for example, a read-only memory (ROM) and a random access memory (RAM). The memory 18 stores various data, information, programs, and the like.

[0029] The keyboard 19 includes physical keys. The user may input various information (e.g., a personal identification number) using the keyboard 19 instead of the touch panel 22. The payment terminal 100 does not have to include the keyboard 19. The input information input by an input operation on the keyboard 19 may be peeked by a third party other than the user who performed the input operation.

[0030] The second CPU 21 realizes various functions by executing programs stored in the memory 28. The second CPU 21 controls all parts outside the security protection area SR. The second CPU 21 performs, for example, processing related to code payment and cash payment, as well as processing related to non-payment. Therefore, the second CPU 21 functions as a general-purpose CPU that performs general-purpose processing. The second CPU 21 also transmits and receives data to and from the first CPU 11 and cooperates with the first CPU 11. The second CPU 21 is an example of a processor and may be another processor. The second CPU 21 may have higher performance than the first CPU 11.

[0031] The second CPU 21 acquires, for example, an image of a subject including a code displayed on a customer's mobile terminal (not shown) from the camera 25. The second CPU 21 analyzes the image and recognizes the identification information required for payment indicated by the code. The second CPU 21 performs payment processing related to the code payment using the identification information.

[0032] Furthermore, the second CPU 21 performs various processes related to peeping detection and peeping prevention based on the captured image acquired from the camera 25. The processes related to peeping detection and peeping prevention will be described in detail later.

[0033] Therefore, in payment terminal 100, camera 25 can be used not only for code payment but also for peeping detection. Therefore, payment terminal 100 can maintain security while being miniaturized without increasing the number of cameras 25.

[0034] The touch panel 22 has an input detection function that electrically detects contact or proximity of a customer's finger (an example of an object) as an input (input operation). The touch panel 22 accepts, for example, an input operation by a customer. The touch panel 22 accepts, for example, an operation to input a PIN for credit card payments, an operation to input a personal identification number, or an operation to enter an electronic signature. Note that input information entered into the touch panel 22 by an input operation may be peeked by a third party other than the user who performed the input operation.

[0035] For example, the touch panel 22 displays a preview image based on an image captured by the camera 25. The customer holds the display surface (not shown) of the mobile terminal facing the camera 25 to allow the camera 25 to capture an image. As a result, even if the customer has difficulty directly viewing the display surface of the mobile terminal, the customer can check the status of the code displayed on the mobile terminal relative to the camera 25 by checking the displayed preview image. The mobile terminal is, for example, a smartphone, tablet terminal, or other mobile terminal, and has a display surface that displays the code CD.

[0036] The camera 25 captures an image of a subject and obtains the captured image. For example, the camera 25 captures an image of a code displayed on a customer's mobile terminal and sends the captured image to the second CPU 21. In this case, the subject is the code. The camera 25 can also capture an image of the user of the payment terminal 100 and their surroundings (including people) as the subject. For example, by capturing an image of the user or a third party other than the user during payment by the camera 25, it is possible to determine whether peeping is occurring based on the captured image.

[0037] The external terminal 27 can be connected to various cables (for example, a LAN (Local Area Network) cable, a USB (Universal Serial Bus) cable, etc.). Therefore, the external terminal 27 can be connected to various external devices (for example, a POS or a cash drawer) via various cables, and can communicate with them. Therefore, the external terminal 27 functions as a communication unit. Note that the communication unit may perform wireless communication (for example, wireless LAN communication or Bluetooth (registered trademark) communication) without going through the external terminal 27.

[0038] The memory 28 may include, for example, a ROM and a RAM, and stores various data, information, and programs.

[0039] The switch 29 is connected to the first CPU 11 or the second CPU 21 based on, for example, an instruction from the first CPU 11, and switches control of the touch panel 22. When the switch 29 is connected to the first CPU 11, the touch panel 22 performs input under the control of the first CPU 11. When the switch 29 is connected to the second CPU 21, the touch panel 22 performs input under the control of the second CPU 21 (see FIG. 14B ). In other words, the switch 29 is used when the touch panel 22 and the display 23 are shared between a payment block (security block) including components within the security protection area SR and an application block including components outside the security protection area SR. For example, the switch 29 sets the connection of the touch panel 22 to the second CPU 21 before the start of a payment, switches the connection of the touch panel 22 from the second CPU 21 to the first CPU 11 when the payment PIN is entered after the start of the payment, and switches the connection of the touch panel 22 back to the second CPU 21 after the payment PIN is entered.

[0040] Next, an example of the operation of the payment terminal 100 will be described.

[0041] FIG. 3 is a flowchart showing an example of the operation of the payment terminal 100.

[0042] First, when, for example, a store clerk operates the touch panel 22 of the payment terminal 100 to input information to start payment, the second CPU 21 starts payment (S1). In this case, the second CPU 21 receives input from the store clerk via the touch panel 22, for example, and specifies one of the payment methods from among credit card payment, electronic money payment, code (e.g., QR Code (registered trademark)) payment, cash payment, etc.

[0043] The second CPU 21 determines whether the payment method requires the input of a personal identification number (S2), and if a personal identification number is not required (step S2; No), performs payment without a personal identification number (step S3), and ends the process. If a personal identification number is required (step S2; Yes), the process proceeds to step S4 and subsequent steps.

[0044] When a PIN is required for payment (Step S2: Yes), a third party H2 around the user H1 should be wary of. This is because the third party H2 may spy on the user H1 entering his or her PIN. Therefore, the second CPU 21 activates the peeping detection mode (S4). The peeping detection mode is an operating mode in which the camera 25 widens the image capture range IR (e.g., maximizes the angle of view). In the peeping detection mode, the second CPU 21 widens the image capture range IR of the camera 25 compared to when a payment method that does not require PIN entry is specified. By activating the peeping detection mode, the second CPU 21 determines whether a person other than the user H1 of the payment terminal 100 (a third party H2) is captured in the image captured by the camera 25 and determines whether the person is a peeper watching the user H1's operation (S5). The detailed processing of Steps S4 and S5 will be described later ( FIGS. 6 to 12 ).

[0045] If another person is a peeping person (step S5; Yes), the second CPU 21 activates the anti-peeping mode (S6A) because there is a high risk that the PIN input will be peeped. The anti-peeping mode is an operating mode that performs various processes to prevent peeping. The anti-peeping mode may include notifying the peeping person by displaying an image or narrowing the viewing angle of the display 23. In this case, a viewing angle control filter may be used or the color contrast may be adjusted. The viewing angle control filter may be realized by a software blind function, or a physical filter provided in the payment terminal 100 may be inserted and turned on. In addition, the second CPU 21 may perform pixel processing or color processing using known techniques, for example, to make the display 23 less visible when viewed from an angle.

[0046] On the other hand, if no other person is captured in the captured image in step S5, or if the person is not a peeping person (step S5; No), the risk of someone peeping at the input of the PIN is low. In this case, the second CPU 21 maintains the display 23 in the normal mode (S6B). In other words, in this case, the second CPU 21 does not narrow the viewing angle of the display 23 as in the anti-peeping mode.

[0047] After processing step S6A or step S6B, the second CPU 21 acquires input information such as a PIN number by user H1 operating the touch panel 22 via the touch panel 22 (S7), and then terminates the peeping detection mode (S8). This is because, after acquiring input information such as a PIN number, no PIN number or other information that requires security is entered, and widening the imaging range IR reduces the need to monitor third parties H2 present around user H1. When the second CPU 21 terminates the peeping detection mode, it also terminates the peeping prevention mode.

[0048] After the peeping detection mode ends in step S8, the second CPU 21 stops the operation of the camera 25 (S9), executes the process after the PIN input (S10), and ends the process of Fig. 3. The process after the PIN input, for example, executes a predetermined payment process using the PIN. This payment process is performed in cooperation with, for example, a server device of an external payment center.

[0049] 4 is a flowchart showing an example of the operation of the payment terminal 100 for code payment, which is an example of payment without a PIN number in step S3. When the code payment payment method is specified, the second CPU 21 activates the code payment mode as the operating mode (step S31) and activates the camera 25 (step S32). The camera 25 reads the code (step S33), and the second CPU 21 executes code payment processing (step S34).

[0050] When reading a code in the code payment mode, the second CPU 21 sets a code reading range CR that includes the range in the captured image where the code CD exists and is narrower than the imaging range IR (image range). In other words, the second CPU 21 reduces the effective pixels that are the target for code reading. This reduces the processing load on the payment terminal 100 when reading a code, allowing for faster code reading.

[0051] FIG. 5 is a schematic diagram showing how the camera 25 reads the code CD.

[0052] The camera 25 can change the imaging range IR within which the camera 25 can capture images. During code payment in FIG. 4 , the second CPU 21 sets the imaging range IR to imaging range IR1. The second CPU 21 sets the code reading range CR of the code CD to be narrower than the imaging range IR1. The second CPU 21 extracts (cuts out) the code reading range CR from the imaging range IR1. That is, when reading the code CD, the second CPU 21 optimizes the imaging range IR1 using techniques such as clipping or angle of view adjustment (narrowing the angle of view) to limit the analysis area of ​​the code CD (code reading range CR) within the imaging range IR1. This allows the payment terminal 100 to improve the processing speed of reading and analyzing the code CD. Note that while the code CD in FIG. 5 is a two-dimensional code such as a QR code (registered trademark), it may also be a one-dimensional code. The actual imaging range IR of the camera 25 during code payment is set to a range that includes the code reading range CR and is included in the maximum imaging range that can be captured.

[0053] 6 is a schematic diagram showing an example of an image capturing range IR2 that can be captured by the camera 25 in the peeping detection mode, and determining whether the third party H2 is a peeping person based on the distance between the user H1 and the third party H2. The second CPU 21 performs various processes on the captured image captured by the camera 25.

[0054] As described above, the second CPU 21 can perform payment according to multiple payment methods. The multiple payment methods are broadly divided into a first payment method that uses input information (e.g., a personal identification number or an electronic signature) for payment authentication, and a second payment method that does not use input information. When a payment method that uses a personal identification number (first payment method) is specified, the second CPU 21 widens the imaging range IR of the camera 25 compared to when a payment method that does not use a personal identification number (second payment method) is specified.

[0055] Specifically, in the second payment method that does not use a personal identification number, such as the code payment shown in Figures 4 and 5, the second CPU 21 sets the imaging range IR to IR1. On the other hand, in the first payment method that uses a personal identification number, the second CPU 21 sets the imaging range IR to IR2, which is wider than IR1.

[0056] Therefore, even when payment is made using a PIN or electronic signature, the payment terminal 100 makes it easier for peepers who may be present around the user H1 of the payment terminal 100 to be captured on camera, thereby strengthening monitoring of peepers and further improving the reliability of security.

[0057] Furthermore, when the second payment method is payment using the code CD and the second payment method is specified, the second CPU 21 sets the image capture range IR to an image capture range IR1 that is narrower than the image capture range IR2. Furthermore, the code reading range CR is set to a range narrower than the image capture range IR1. This allows the payment terminal 100 to narrow the image capture range of the camera 25, reduce the processing load on the second CPU 21, and speed up reading of the code CD.

[0058] The second CPU 21 also determines whether or not there is a peeping person based on the captured image. In this case, the camera 25 captures an image of a subject around the camera 25. The second CPU 21 determines whether or not a third party H2 other than the user H1 using the payment terminal 100 is captured in the image captured by the camera 25. If the third party H2 is captured, the second CPU 21 estimates the relative positional relationship between the user H1 and the third party H2. The second CPU 21 determines whether or not the third party H2 is a peeping person based on the relative positional relationship. The relative positional relationship may include the distance between the user H1 and the third party H2. The second CPU 21 may detect people (e.g., the user H1, the third party H2) in the captured image or may detect the faces of people by, for example, performing image processing on the captured image. The second CPU 21 may, for example, estimate the distance between user H1 and third party H2 based on the size of the faces of user H1 and third party H2, as described below, or may estimate the distance between user H1 and third party H2 based on the number of pixels in the area corresponding to the faces of user H1 and third party H2 in the captured image.

[0059] The second CPU 21 determines whether the third party H2 is a peeper based on the estimated distance. If the distance is short, there is a high possibility that the third party H2 is a peeper. Therefore, for example, if the estimated distance is equal to or less than a predetermined threshold th1 (predetermined distance), the second CPU 21 determines that the third party H2 is a peeper. For example, if the estimated distance is longer than the predetermined threshold th1 (predetermined distance), the second CPU 21 determines that the third party H2 is not a peeper.

[0060] FIG. 7 is a schematic diagram showing an example in which, in the peeping detection mode, it is determined whether or not a third party H2 is a peeper based on the ratio between the face size of the user H1 and the face size of the third party H2.

[0061] The second CPU 21 may estimate this distance by calculating the distance between the user H1 and the third party H2 based on the ratio of their facial sizes. That is, the second CPU 21 extracts the facial contours of each person, the user H1 and the third party H2, and calculates the ratio of the facial size of the third party H2 to the facial size of the user H1. The facial size of each person and the distance between each person are roughly proportional to each other. Therefore, the second CPU 21 may calculate (estimate) the distance between each person based on the facial ratio, which is one of the differences in the facial sizes of each person.

[0062] When a third party H2 appears in the captured image, the second CPU 21 may estimate the ratio between the face size D1 of the user H1 and the face size D2 of the third party H2, and determine whether the third party H2 is a peeping person based on this ratio. If the ratio of the face size D1 of the user H1 to the face size D2 of the third party H2 is equal to or less than a predetermined threshold th2, the second CPU 21 determines that the user H1 and the third party H2 are close to each other at a distance equal to or less than the predetermined threshold th1, and that the third party H2 is a peeping person. If the ratio of the face size D1 of the user H1 to the face size D2 of the third party H2 is equal to or less than the predetermined threshold th2, the second CPU 21 determines that the user H1 and the third party H2 are close to each other at a distance equal to or less than the predetermined threshold th1, and that the third party H2 is a peeping person. If the facial size D1 of the user H1 is larger than the ratio of the facial size D2 of the third party H2 by a predetermined threshold th2, the second CPU 21 determines that the user H1 and the third party H2 are at a distance longer than the predetermined threshold th1 and are not in close proximity, and that the third party H2 is not a peeping person. Note that it may also be possible to directly determine whether the third party H2 is a peeping person based on the facial size D1 of the user H1 and the facial size D2 of the third party H2, without estimating the distance between the user H1 and the third party H2.

[0063] In this way, the payment terminal 100 can easily determine whether the third party H2 is a peeping person or not based on the face size D1 of the user H1 and the face size D2 of the third party H2.

[0064] FIG. 8 is a schematic diagram showing an example in which, in the peeping detection mode, it is determined whether or not a third party H2 is a peeping person based on the angle formed between a user H1 and a third party H2.

[0065] As described above, the second CPU 21 estimates the relative positional relationship between the user H1 and the third party H2. The relative positional relationship may include the angle θ1 formed between the user H1 and the third party H2 with the display 23 as the base point. In other words, the second CPU 21 may estimate the angle θ1 and determine whether the third party H2 is a peeping person based on this angle θ1.

[0066] For example, if the angle θ1 is small, there is a high possibility that the third party H2 is a peeper attempting to peek from the same direction as the user H1. Therefore, for example, if the estimated angle θ1 is equal to or smaller than a predetermined threshold th3 (predetermined angle), the second CPU 21 determines that the third party H2 is a peeper. On the other hand, for example, if the estimated angle θ1 is greater than the predetermined threshold th3 (predetermined angle), the second CPU 21 determines that the third party H2 is not a peeper. This is because the positions of the user H1 and the third party H2 relative to the display 23 are significantly different, and it can be inferred that it would be difficult for the third party H2 to peek at the display 23 from their position.

[0067] This makes it easy for a third party H2 approaching the payment terminal 100 from a direction along the display surface of the display 23 and standing near the user H1 of the payment terminal 100 to peek at the payment terminal 100. The payment terminal 100 can determine that such a third party H2 is a peeping person.

[0068] 6 to 8, the second CPU 21 may determine that the third party H2 is a peeper. Furthermore, to further improve the accuracy of the determination, the second CPU 21 may re-determine whether the third party H2 is a peeper by analyzing the facial direction or line of sight of the third party H2.

[0069] Specifically, if the second CPU 21 determines that the third party H2 is a peeper after going through the processing processes shown in FIGS. 6 to 8 , it may set an analysis target area AR that is narrower than the image capture range IR of the camera 25, as shown in the figures. The analysis target area AR is an area in which information about the third party H2 (e.g., the facial orientation or line of sight of the third party H2) is analyzed. The analysis target area AR includes the user H1 and one or more third parties H2. The second CPU 21 extracts (cuts out) the analysis target area AR from the image capture range IR. The second CPU 21 analyzes and detects in detail the facial orientation or line of sight of the user H1 and the third parties H2 in the analysis target area AR. Then, the second CPU 21 may re-determine and confirm whether the third party H2 is a peeper based on the analyzed facial orientation or line of sight of the third party H2.

[0070] This allows the payment terminal 100 to determine whether or not a third party H2 is peeking based on the direction in which the third party H2 is actually looking, thereby improving the accuracy of determining whether or not a third party H2 is peeking.

[0071] Furthermore, the second CPU 21 may continuously (continuously) determine whether the third party H2 is a peeper by going through the processes shown in Figures 6 to 8 while processing the payment for user H1. In this case, the positions of user H1 and third party H2 may change over time. In this case, the second CPU 21 may continuously (continuously) set an analysis target area AR including user H1 and third party H2 while processing the payment for user H1. Therefore, the position, size, range, etc. of the analysis target area AR within the image capture range IR of the camera 25 may be dynamically changed.

[0072] By dynamically changing the analysis target area AR of the captured image, the payment terminal 100 can optimize the analysis target area AR and reduce the analysis load. Therefore, even if the performance of the second CPU 21 is low, the payment terminal 100 can smoothly determine whether or not peeping is occurring using the analysis target area AR. In other words, by clipping the effective pixels used in this analysis to within the analysis target area AR, the payment terminal 100 can reduce the processing load compared to when the entire imaging range IR is always used as the analysis target area AR.

[0073] Next, an example of determining whether or not peeping is occurring using the line-of-sight directions of the user H1 and the third party H2 will be described.

[0074] FIG. 9 is a schematic diagram showing the relationship between the display 23 and the line of sight of the user H1 and the third person H2.

[0075] In FIG. 9 , user H1 and third parties H2 (H2A, H2B, H2C) are looking at the same display 23. The display 23 can display various payment-related information, including input information used for payment (e.g., a personal identification number or electronic signature). FIG. 9 shows the line-of-sight directions LS of user H1 and each third party H2. Specifically, the line-of-sight direction LS1 of user H1, the line-of-sight direction LSA of third party H2A, the line-of-sight direction LSB of third party H2B, and the line-of-sight direction LSC of third party H2C are shown. FIG. 9 also shows the peeking range PR of a specific third party H2. Specifically, the peeking range PRA of third party H2A and the peeking range PRB of third party H2B are shown. Each person's line-of-sight direction LS varies. If the line-of-sight direction LS of third party H2 falls within the peeking range PR, third party H2 can see the content displayed on the display 23. That is, depending on the line of sight LS, the third party H2 may or may not be able to see the display 23. Note that since the user H1 uses the touch panel 22 and the display 23 while looking directly ahead, the line of sight LS basically coincides with the normal direction to the display surface of the display 23.

[0076] Fig. 10 is a diagram showing an example of the angle θ1 formed between a user H1 and a third party H2, with the display 23 as the base point. Fig. 10 shows, as an example, the angle θ1 formed between a user H1 and a third party H2B, with the display 23 (for example, the center point of the display surface) as the base point.

[0077] The angle θ1 may coincide with the angle formed between the line of sight LS1 of the user H1 and the line of sight LS (e.g., line of sight LSB) of the third party H2, with the display 23 as the base point. In other words, the angle θ1 may coincide with the angle formed between the line of sight LS1 of the user H1 and the line of sight LS (e.g., line of sight LSB) of the third party H2, with the display 23 as the base point. Furthermore, since the user H1 is basically positioned directly in front of the display 23 as described above, the angle θ1 may coincide with the angle formed between the normal to the display surface of the display 23 and the line of sight LS of the third party H2B.

[0078] FIG. 11 is a schematic diagram showing an example of a map MP on the display 23 that indicates an area that can be viewed by a third party.

[0079] In the map MP, a position P2 is defined on the x-axis. The position P2 is the position of the third party H2 relative to the user H1 in the direction along the display surface of the display 23 (the x1 direction in FIG. 10). In the map MP, the position P2 takes a positive value when the third party H3 is located to the right of the user H1 when looking at the display 23, and the position P2 takes a negative value when the third party H3 is located to the left of the user H1 when looking at the display 23. In addition, an angle θ1 is defined on the y-axis. The angle θ1 is the angle described in FIG. 10.

[0080] For example, the second CPU 21 detects a third party H2 appearing in the captured image by image processing. Then, the second CPU 21 detects the position of the third party H2 (for example, the position of the face) and calculates the position P2 and angle θ1. The angle θ1 is derived using the line of sight LS, and is therefore derived by analyzing the analysis target area AR.

[0081] A map position HP, which is a position on the map MP, is determined for each third party H2 by the position P2 and angle θ1. For example, a map position HPA is uniquely determined by the position P2 and angle θ1 for a third party H2A. A map position HPB is uniquely determined by the position P2 and angle θ1 for a third party H2B. A map position HPC is uniquely determined by the position P2 and angle θ1 for a third party H2C.

[0082] Furthermore, region R10 on the map MP indicates the peeping range PR of each third party H2, i.e., the range of the position P2 and angle θ1 at which peeping is possible. In other words, if the map position HP of the third party H2 is present within region R10, the third party H2 is able to peek at the display 23, i.e., is a peeper. The second CPU 21 determines whether the third party H2 is a peeper based on this position P2 and angle θ1. Specifically, if the map position HP of the third party H2 is within the peeping range PR, the second CPU 21 determines that the third party H2 is a peeper. If the map position HP of the third party H2 is not within the peeping range PR, the second CPU 21 determines that the third party H2 is not a peeper. Note that although region R10 is depicted in FIG. 11 as being surrounded by straight lines, it may also be a region at least partially surrounded by curved lines.

[0083] Furthermore, region R20 on the map MP is a region where peeping is impossible due to the viewing angle limit of the display 23 (peeping-prohibited region). The range where the absolute value of the angle θ1 of the third party H2 is equal to or greater than a predetermined threshold th4 is the peeping-prohibited region. This is because, if the absolute value of the angle θ1 of the third party H2 is equal to or greater than the predetermined threshold th4, the third party H2 views the display 23 from a substantially sideways direction, making the display 23 unviewable due to the viewing angle limit. On the other hand, if the absolute value of the angle θ1 of the third party H2 is less than the predetermined threshold th4, i.e., if the line of sight of the third party H2 does not exceed the viewing angle limit, the third party H2 can see the contents displayed on the display 23. Therefore, if the map position HP of the third party H2 is within region R10 but outside region R20, the second CPU 21 can determine that the third party H2 is a peeper.

[0084] 11, the line of sight direction LS when viewing the display 23 from directly in front of the display 23 is set to 0°. The line of sight direction LS when viewing the display 23 from directly to the right of the display 23 (i.e., the x1 direction in FIG. 10) is set to 90°. The line of sight direction LS when viewing the display 23 from directly to the left of the display 23 (i.e., the opposite direction from the x1 direction in FIG. 10) is set to -90°. Note that, although it has been described in FIG. 11 that the angle θ1 is derived based on the line of sight direction LS, the angle θ1 may also be derived based on the direction of the face instead of the line of sight direction.

[0085] Fig. 12 is a flowchart showing an example of the operation of the payment terminal 100 for determining whether or not a third party H2 appearing in the captured image is a peeping person in step S5 of Fig. 3. Here, it is assumed that a third party H2 other than the user H1 appears in the captured image, as in Figs. 6 to 8.

[0086] First, the second CPU 21 performs a peeping determination based on a threshold (step S51). That is, the second CPU 21 determines whether the positional relationship between the user H1 and the third party H2 captured in the captured image satisfies a predetermined relative positional relationship. The relative positional relationship here is as follows for each of the examples shown in FIGS. 6 to 8. Specifically, in the case of FIG. 6, the distance between the user H1 and the third party H2 is equal to or less than a predetermined threshold th1. In the case of FIG. 7, the ratio of the face size D1 of the user H1 to the face size D2 of the third party H2 is equal to or less than a predetermined threshold th2. In the case of FIG. 8, the angle formed between the user H1 and the third party H2 is equal to or less than a predetermined threshold th3.

[0087] If the above relative positional relationship is satisfied, the third party H2 is likely to be a peeper, and if the above relationship is not satisfied, the third party H2 is unlikely to be a peeper. Therefore, if the above relative positional relationship is not satisfied (step S51; No), the second CPU 21 determines that the third party H3 is not a peeper (step S52).

[0088] If the above relative positional relationship is satisfied (step S51; Yes), the second CPU 21 performs a detailed peeping analysis based on the facial orientation or line of sight of the third party H2 in order to further improve the accuracy of peeping determination (step S53). In the detailed peeping analysis, the second CPU 21 may cut out (crop) the analysis target area AR shown in Figures 6 to 8 from the imaging range IR to reduce the analysis load, and then detect the face of the third party H2 and the facial orientation or line of sight of the third party H2.

[0089] The second CPU 21 again determines whether the third party H2 is a peeper based on the facial orientation or line of sight of the third party H2 (step S54). The second CPU 21, for example, derives (e.g., calculates) the position P2 and angle θ1 described above and derives (e.g., calculates) the map position HP of the third party H2 on the map MP shown in FIG. 11. If the map position HP of the third party H2 is located within region R10, the second CPU 21 determines that the third party H2 is a peeper (step S55). On the other hand, if the map position HP of the third party H2 is not located within region R10, the second CPU 21 determines that the third party H2 is not a peeper (step S52). The second CPU 21 proceeds from step S55 to step S6A in FIG. 3 , and from step S52 to step S6B in FIG. 3 .

[0090] 3 is the activation of the anti-peeping mode when it is determined (including re-determination) that the third party H2 is a peeping person. When the second CPU 21 finally determines that the third party H2 is a peeping person, it activates the anti-peeping mode, which changes the settings of the display 23, the touch panel 22, etc. This makes it possible to prevent the third party H2 from peeping.

[0091] 12, the process of step S54 may be omitted. In this case, the peeping determination by the second CPU 21 is performed in the process of step S51, and this becomes the final determination result.

[0092] 13 is a schematic diagram showing an example of a display in which the user H1 and the third party H2 are displayed in different display modes on the display 23, which is one mode of the anti-peeping mode. The second CPU 21 causes the user H1 and the third party H2 to be displayed in different display modes on the display 23. The different display modes include, for example, displaying the user H1 and the third party H2 in different colors, or displaying only the third party H2 in a frame, etc. This allows the user H1 to more reliably recognize the presence of a peeping person by checking the display in this display mode.

[0093] In one aspect of the anti-peeping mode, the second CPU 21 may narrow the viewing angle of the display 23 compared to before the third party H2 is determined to be a peeper. This makes it more difficult for the peeper to peek at the payment terminal 100 when a peeper is present.

[0094] In addition, in one aspect of the anti-peeping mode, the second CPU 21 may display information indicating the presence of a peeping person on the display 23. Such information may be, for example, a message notifying the user of the presence of a peeping person, a pop-up display alerting the user to the presence of a peeping person, or the like. This message or pop-up display may include warning information such as "You may be being peeped at." In this way, the payment terminal 100 allows the user H1 to recognize the presence of a peeping person by checking the display regarding the peeping person.

[0095] In addition, in one aspect of the anti-peeping mode, the second CPU 21 may stop accepting input information via the touch panel 22. This allows the payment terminal 100 to prevent input information such as a PIN number from being entered when a peeping person is present.

[0096] In one aspect of the anti-peeping mode, the second CPU 21 may display a cancel button on the display 23 that can be input via the touch panel 22 and that stops the payment process. In this case, the second CPU 21 positions the cancel button so that it is within a visible viewing angle range even when the viewing angle of the display 23 is narrowed. Furthermore, the second CPU 21 may display the cancel button after the above-mentioned warning pop-up display, avoiding the position of the pop-up display. This allows the payment terminal 100 to cancel the payment process at the user H1's discretion if a peeping person is present.

[0097] 14A, 14B, and 14C are sequence diagrams showing the transmission and reception of information between components of payment terminal 100. FIG. 14A mainly shows the process from the start of payment to the PIN input operation. FIG. 14B mainly shows various processes related to peeping. FIG. 14C mainly shows the process from the peeping notification and PIN input operation to the completion of payment.

[0098] As shown in Fig. 14A, user H1 operates the touch panel 22 to start payment (S101). The process of step S101 includes the selection of a payment method (e.g., the first payment method) and corresponds to step S1 in Fig. 3. When the touch panel 22 receives an operation to start payment (a payment start operation), it transmits payment start operation information to the second CPU 21 (S102).

[0099] When the second CPU 21 receives the payment start operation information, it sends a payment start instruction to the first CPU 11 (S201). The payment start instruction here is, for example, a payment according to the first payment method that requires input of a personal identification number (Yes in step S2 of FIG. 3).

[0100] When the first CPU 11 receives the payment start instruction, it generates an activation signal for the first card reader 16 or the second card reader 17 (hereinafter simply referred to as the "card reader") (S301) and sends the activation signal to the card reader (S302). The card reader reads data from the card (e.g., a magnetic card or a contact IC card) of user H1 (S401) and sends the read information to the first CPU 11 (S402). When the first CPU 11 receives the read information, it generates a communication completion indication indicating that communication with the card reader has been completed (S303) and sends the communication completion information to the second CPU 21 (S304).

[0101] Furthermore, the first CPU 11 generates a touch panel control switching signal for switching the control source of the touch panel 22 from the second CPU 21 to the first CPU 11 (S305), and transmits the touch panel control switching signal including a touch panel switching instruction to the switch 29 (S306). Furthermore, the first CPU 11 transmits a PIN input instruction to the touch panel 22 to prompt the user H1 to input a PIN (S307), and the touch panel 22 displays information for accepting the input of a PIN and waits for the user H1 to input the PIN (S103).

[0102] Meanwhile, in parallel with the processing in steps S301 to S103 described above, after transmitting the payment start instruction described above (S201), the second CPU 21 generates a camera activation signal for activating the camera 25 (S202) and sends the camera activation signal to the camera 25 (S203). Upon receiving the camera activation signal, the camera 25 activates and starts capturing images of subjects around the camera 25 (S501), and sends the captured images to the second CPU 21 (S502).

[0103] When the second CPU 21 receives the captured image, it activates the peeping detection mode (S503). Step S503 corresponds to step S4 in Fig. 3. After step S503, the second CPU 21 executes peeping-related processing (see Fig. 14B).

[0104] As shown in FIG. 14B , the second CPU 21 branches depending on conditions during the anti-peeping process. If there are no people other than the user H1 in the captured image, or if a predetermined relationship (predetermined relative positional relationship) is not satisfied between the user H1 and the third party H2 (e.g., No in step S5 of FIG. 3 or No in step S51 of FIG. 12 ), the second CPU 21 instructs the display 23 to continue the normal display mode (S204). Step S204 corresponds to the transmission of display data for the normal display mode from the second CPU 21 to the display 23, and corresponds to step S6B of FIG. 3 . In the normal display mode, normal display is performed without displaying at a narrow viewing angle, as in the anti-peeping mode.

[0105] On the other hand, if a third party H2 other than the user H1 is present in the captured image and a predetermined relationship is satisfied between the user H1 and the reflected third party H2 (Yes in step S51 of FIG. 12 ), the second CPU 21 performs a detailed peeping analysis (S205). In the detailed peeping analysis, if the second CPU 21 determines that the third party H2 is not a peeper based on the facial orientation or line of sight of the third party H2 (S206), the second CPU 21 instructs the display 23 to continue in the normal display mode (S207). Step S206 corresponds to No in step S54 of FIG. 12 .

[0106] In the detailed peeping analysis, if the second CPU 21 determines that the third party H2 is a peeping person based on the facial orientation or line of sight of the third party H2 (Yes in step S54 of FIG. 12), the second CPU 21 activates the peeping prevention mode (S208). Step S208 corresponds to step S6A of FIG. 3. The second CPU 21 sends, for example, display data under the peeping prevention mode or a control signal for the peeping prevention mode to the display 23 (S209). Such display data or control signal may include, for example, narrowing the viewing angle of the display 23 or a message alerting the presence of a peeping person.

[0107] Furthermore, when the second CPU 21 determines that a third party is a peeping person, it sends a peeping notification to the first CPU 11 (S210).

[0108] 14C , when the first CPU 11 receives the peeping notification, it suspends the input of the PIN (S308). In this case, the first CPU 11 suspends (stops) the acceptance of input information of the PIN via, for example, the touch panel 22. However, if the peeping stops, the first CPU 11 may resume the processing.

[0109] After step S103, when the touch panel 22 accepts the input of the PIN, the touch panel 22 notifies the first CPU 11 that the PIN has been input (S104), and the first CPU 11 confirms the PIN (S309). The first CPU 11 then generates a touch panel control switching signal for switching control of the touch panel 22 from the first CPU 11 to the second CPU 21 (S310) and sends the touch panel control switching signal, including a touch panel switching instruction, to the switch 29 (S311). The first CPU 11 then sends the PIN input information to the second CPU 21 (S312). Upon receiving the PIN input information, the first CPU 11 terminates the peeping detection mode (S211). Step S211 corresponds to step S8 in FIG. 3 .

[0110] The second CPU 21 generates a camera stop signal for stopping the operation of the camera 25 (step S212), and sends the camera stop signal to the camera 25 (step S213). Upon receiving the camera stop signal, the camera 25 stops operations such as capturing images.

[0111] The first CPU 11 and the second CPU 21 perform payment processing using the acquired PIN input information (step S601). When the payment processing is completed, the first CPU 11 sends a payment completion notice to the second CPU 21 (step S313). When the second CPU 21 receives the payment completion notice, it sends a payment completion display signal including an instruction to display that payment has been completed to the display 23 (step S214). When the display 23 receives the payment completion display signal, it displays information indicating that payment has been completed.

[0112] The payment terminal 100 of this embodiment determines whether or not a third party H2 is peeping based on the image captured by the camera 25 and the circumstances surrounding the user H1, specifically the relative positional relationship between the user H1 and the third party H2. If peeping is detected, the anti-peeping mode is automatically activated, eliminating the need for the user H1 or a store clerk to set the blind function each time. Furthermore, the payment terminal 100 can use the camera 25 for both code payment and peeping detection, allowing for a more compact size. The payment terminal 100 can also improve security for the user H1 by controlling the viewing angle and visibility or displaying a warning pop-up.

[0113] Furthermore, when entering a PIN, payment terminal 100 can improve the monitoring function for the surroundings of user H1 by widening the imaging range of camera 25. By dynamically changing analysis target area AR within imaging range IR, payment terminal 100 can optimize analysis target area AR and reduce the analysis load, thereby providing a monitoring function even when the performance of second CPU 21 is low.

[0114] (Variations) In the present embodiment, the payment terminal 100 is mainly illustrated as having one touch panel 22 and one display 23, but this is not limited thereto. For example, the payment terminal 100 may be a payment terminal 100A in which the customer side and the store clerk side are used separately, with each side provided with a touch panel and a display. In other words, the payment terminal 100A has a payment block that performs payment-related processing and an application block that performs image-related processing, each of which has a display and a touch panel. The first CPU 11 and the second CPU are connected so as to be able to communicate with each other.

[0115] 15 is a block diagram showing a first variant (payment terminal 100A) of the electrical configuration of payment terminal 100. In payment terminal 100A, the payment block includes first CPU 11, first touch panel 12, first display 13, NFC IC 14, NFC antenna 15, first card reader 16, second card reader 17, and memory 18. In payment terminal 100A, the application block includes second CPU 21, second touch panel 22A, second display 23A, speaker 24, camera 25, printer 26, external terminal 27, and memory 28.

[0116] Alternatively, payment terminal 100 may be payment terminal 100B that performs payment-related processing, and an application terminal 300 that performs image-related processing may be provided separately from payment terminal 100. Payment terminal 100B and application terminal 300 may be communicably connected via their respective communication units (not shown) to form payment system 500. In other words, payment terminal 100B has a payment block, application terminal 300 has an application block, and payment terminal 100B and application terminal 300 each have their own display and touch panel.

[0117] 16 is a block diagram showing a second variant (payment terminal 100B) of the electrical configuration of payment terminal 100. Payment terminal 100B includes a first CPU 11, a first touch panel 12, a first display 13, an NFC IC 14, an NFC antenna 15, a first card reader 16, a second card reader 17, and a memory 18. The application terminal 300 includes a second CPU 21, a second touch panel 22A, a second display 23A, a speaker 24, a camera 25, a printer 26, an external terminal 27, and a memory 28. Note that although camera 25 is externally located in payment terminal 100B, it is electrically connected to second CPU 21.

[0118] As a result, the present disclosure describes at least the following: Note that, in parentheses, examples of components corresponding to those in the above-described embodiments are given, but the present disclosure is not limited to these.

[0119] (Item 1) A payment terminal (payment terminal 100) capable of making payments using a code (code CD), comprising: a camera (camera 25) that captures an image of a subject; and a processor (second CPU 21) that processes the captured image captured by the camera, wherein the processor determines whether a second person (third party H2) other than a first person (user H1) using the payment terminal is captured in the captured image, estimates the relative positional relationship between the first person and the second person, and determines whether the second person is a peeping person based on the relative positional relationship.

[0120] This allows the payment terminal to use the camera not only for code payments but also for peeping detection, allowing the payment terminal to be made smaller while maintaining security.

[0121] (Item 2) The payment terminal described in Item 1, wherein the processor is capable of making payments according to a plurality of payment methods, and when a first payment method is specified that makes a payment using input information (PIN number, electronic signature) that is entered into a touch panel (touch panel 22) and used to authenticate the payment, the processor widens the imaging range (imaging range IR) of the camera compared to when a second payment method is specified that makes a payment without using the input information.

[0122] As a result, when a payment method that requires input information is specified, the payment terminal widens the imaging range of the camera, making it easier to capture peepers who may be around the user of the payment terminal. This makes it easier to detect peepers.

[0123] (Item 3) The payment terminal according to Item 2, wherein the processor sets the imaging range of the camera to the imaging range of the camera with the maximum angle of view when the first payment method is specified.

[0124] This allows the payment terminal to maximize the imaging range of the camera, so that it can capture as many peepers as possible around the user of the payment terminal.

[0125] (Item 4) The payment terminal according to any one of items 1 to 3, wherein the processor, when it is determined that the second person is a peeping person, sets an area to be analyzed (analysis area AR) that is narrower than the imaging range of the camera and that includes the first person and the second person, analyzes the facial direction or line of sight of the second person, and determines again whether the second person is a peeping person based on the analyzed facial direction or line of sight of the second person.

[0126] This allows the payment terminal to determine whether or not the second person is peeping based on the direction the second person is actually looking, improving the accuracy of peeping detection. Also, by setting the analysis target area narrower than the imaging range so as to include the person (e.g., the person's face), the payment terminal can reduce the processing load compared to analyzing the entire imaging range and speed up peeping detection.

[0127] (Item 5) The payment terminal according to any one of items 2 to 4, wherein the second payment method is payment using the code, and when the second payment method is specified, the processor sets a code reading range for reading the code that is narrower than the imaging range, and reads the code in the code reading range.

[0128] This allows the payment terminal to narrow the imaging range of the camera, reducing the processing load on the processor and speeding up code reading.

[0129] (Item 6) The payment terminal according to any one of items 1 to 5, wherein the relative positional relationship includes a distance between the first person and the second person.

[0130] For example, if a first person and a second person are close to each other, there is a possibility that the information entered by the first person can be viewed from a short distance. Therefore, the payment terminal can accurately determine whether or not the person is a peeping person by taking into account the distance between the first person and the second person as a relative positional relationship.

[0131] (Item 7) The payment terminal according to Item 6, wherein the processor, when the second person is photographed, calculates a ratio between a face size of the first person and a face size of the second person, and estimates the distance based on the ratio.

[0132] The size of the faces of multiple people captured in a captured image is proportional to the distance of each person's face from the camera. Therefore, by taking into account the ratio of each person's size, the payment terminal can estimate the distance of each person with high accuracy.

[0133] (Item 8) The payment terminal according to any one of items 1 to 7, wherein the relative positional relationship includes an angle formed between the first person and the second person with the camera as a base point.

[0134] For example, if the angle between the first person and the second person is small, the first person and the second person are at a similar angle to the display 23, and therefore there is a possibility that the information entered by the first person can be viewed from a close distance. Therefore, the payment terminal can accurately determine whether or not a person is a peeping person by taking into account the distance between the first person and the second person as a relative positional relationship.

[0135] (Item 9) The payment terminal described in any one of items 1 to 8, wherein, when the second person is determined to be the peeping person, the processor narrows the viewing angle of the display device (display 23) compared to before the second person was determined to be the peeping person.

[0136] This allows the payment terminal to be adjusted so that it is difficult for a peeper to peek at the payment terminal if one is present.

[0137] (Item 10) The payment terminal according to any one of items 1 to 9, wherein the processor, when it is determined that the second person is the peeping person, causes a display device to display information indicating the presence of the peeping person.

[0138] This allows the payment terminal to make the user aware of the presence of a peeping person.

[0139] (Item 11) The payment terminal according to any one of items 1 to 10, wherein the processor, when it is determined that the second person is the peeping person, causes a display device to display the first person and the second person in different display modes.

[0140] This allows the payment terminal to more reliably make the user aware of the presence of a peeping person.

[0141] (Item 12) The payment terminal according to any one of items 1 to 11, wherein the processor stops accepting the input information via the touch panel when it is determined that the second person is the peeping person.

[0142] This allows the payment terminal to prevent input information such as a PIN number that may be spied on from being entered, and prevents the input information from actually being used in processing after entry, thereby ensuring security.

[0143] (Item 13) A payment assistance method executed by a payment terminal capable of making payments using a code, comprising the steps of: acquiring an image captured by a camera provided in the payment terminal, the camera being capable of capturing an image of the code used for the payment and capturing an image of a subject; determining whether a second person other than a first person using the payment terminal is captured in the captured image together with the first person; if the second person is captured, estimating a relative positional relationship between the first person and the second person; and determining whether the second person is a peeping person based on the relative positional relationship.

[0144] This allows the payment terminal that executes the payment support method to use the camera not only for code payment but also for peeping detection, thereby maintaining security while miniaturizing the payment terminal.

[0145] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0146] In addition, the above embodiment may also be applicable to a program that realizes the functions of the payment support method, which is supplied to an electronic device that is a computer (e.g., a payment terminal) via a network or various storage media, and which is read and executed by the processor of this electronic device, as well as the storage medium on which this program is stored.

[0147] This disclosure is based on a Japanese patent application (Patent Application No. 2023-092730) filed on June 5, 2023, the contents of which are incorporated by reference into this disclosure.

[0148] The present disclosure is useful for a payment terminal and a payment support method that can improve security while maintaining the compactness of the payment terminal.

[0149] REFERENCE SIGNS LIST 11 First CPU 14 NFC_IC 15 NFC antenna 16 First card reader 17 Second card reader 18 Memory 19 Keyboard 21 Second CPU 22 Touch panel 23 Display 25 Camera 27 External terminal 28 Memory 29 Switch 100 Payment terminal AR Analysis target area CD Code CR Code reading range H1 User H2 Third party IR Imaging range

Claims

1. A payment terminal having a first area that is a security protected area within a device body and a second area outside the security protected area, and capable of processing payments using a plurality of payment methods, a first processor disposed in the first area and configured to process a first payment method requiring input of a personal identification number; a camera disposed in the second area for capturing an image of a subject; a second processor that is disposed in the second area, is connected to the first processor, and processes an image captured by the camera; a touch panel that displays a screen for inputting the PIN; Equipped with The first processor When the first payment method is designated among the plurality of payment methods, A payment process requiring the input of the PIN is performed, and the touch panel Display the PIN entry screen, When receiving a peep notice indicating that the input of the personal identification number has been peeped, the reception of the input of the personal identification number is stopped, The second processor When a second payment method using a code is designated among the plurality of payment methods, the payment is processed using the code captured by the camera, When the first payment method is designated among the plurality of payment methods, In the captured image, a first person using the payment terminal is determining whether a second person other than the first person is present in the image; If the second person is included in the image, a relative positional relationship between the first person and the second person is estimated; determining whether the second person is a peeper based on the relative positional relationship; When it is determined that the second person is a spectator, the spectator notification is sent to the first processor. Payment terminal.

2. The second processor When the first payment method is specified, the imaging range of the camera is made wider than when the second payment method is specified. The payment terminal of claim 1 .

3. When the first payment method is specified, the processor sets the imaging range of the camera to the imaging range of the camera with the maximum angle of view. The payment terminal according to claim 2 .

4. The second processor If the second person is determined to be a peeper, the image capturing range of the camera is narrower than the image capturing range of the camera and includes the first person and the second person. Set the area to be analyzed, Analyzing a facial direction or a line of sight of the second person with respect to the analysis target area; determining again whether the second person is a peeper based on the analyzed face direction or line of sight of the second person; The payment terminal according to any one of claims 1 to 3.

5. the relative positional relationship includes a distance between the first person and the second person; A payment terminal according to any one of claims 1 to 4.

6. The second processor If the second person is included in the image, a ratio between the size of the face of the first person and the size of the face of the second person is calculated; estimating the distance based on the ratio; The payment terminal according to claim 5 .

7. the relative positional relationship includes an angle formed between the first person and the second person with the camera as a base point; The payment terminal according to any one of claims 1 to 4.

8. The second processor When the second person is determined to be the peeping person, narrowing the viewing angle of the display device compared to before the second person was determined to be the peeping person. The payment terminal according to any one of claims 1 to 4.

9. The second processor If the second person is determined to be the peeping person, displaying information indicating the presence of the peeping person on a display device. The payment terminal according to any one of claims 1 to 4.

10. The second processor When the second person is determined to be the peeping person, the first person and the second person are displayed in different display modes on a display device. The payment terminal according to any one of claims 1 to 4.

11. A payment support method executed by a payment terminal capable of processing payments by a plurality of payment methods, the method comprising: a first processor located within a security protected area within a device body; and a second processor located outside the security protected area within the device body; When a first payment method is designated among the plurality of payment methods, a step of the first processor performing a payment process requiring input of a personal identification number and displaying a screen for inputting the personal identification number; acquiring an image captured by a camera located outside the secure area; determining whether a second person other than a first person using the payment terminal is captured in the captured image together with the first person; If the second person is included in the image, estimating a relative positional relationship between the first person and the second person; determining whether the second person is a peeper based on the relative positional relationship; When the second processor determines that the second person is a spectator, sending a spectator notification to the first processor; and a step of the first processor stopping acceptance of the input of the personal identification number upon receiving the peeping notification, When a second payment method is designated among the plurality of payment methods, a step in which the second processor performs code payment processing using the code captured by the camera; A payment support method having the above.

12. If the second person is determined to be the peeping person, displaying information indicating the presence of the peeping person on a display device of the payment terminal. The payment support method according to claim 11.

13. If the second person is determined to be the peeping person, the first person and the second person are displayed in different display modes on a display device of the payment terminal. The payment support method according to claim 11.