Program and determination system

Abstract

To improve accuracy in determining whether or not a user who performs an operation related to a financial transaction is suspected of being involved in a crime or trying to perform other actions to be restricted, as compared with a case where a change in biological information before and after a warning is presented is not used.SOLUTION: The moving image acquisition unit 111 acquires moving image data obtained by photographing the face of the user. The biological information measurement unit 112 measures biological information based on the moving image data. The reception unit 114 receives an operation of the user for a procedure related to a financial transaction from the operation unit 14. When the condition is satisfied, the presentation unit 115 causes the display unit 15 to present a warning. The biological information acquisition unit 116 acquires biological information measured before and after the date and time when the warning is presented. The determination unit 117 determines whether the change from the first biological information to the second biological information satisfies the determination condition.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a program and a determination system for determining whether a user performing an operation related to a financial transaction is suspected of being involved in a crime. [Background technology]

[0002] Patent Document 1 discloses a security device that acquires an image of a user of a device that executes cash transactions, extracts a biological area contained in the image, detects a signal waveform in which components of specific frequency bands other than those in which pulse waves can be obtained among each wavelength component are offset from a signal of a representative value for each wavelength component of each pixel contained in the biological area, and issues an alarm if the detected waveform is abnormal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6003752 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 compares the heart rate measured when, for example, the "Transfer" menu is operated with a standard heart rate and calculates an abnormality score based on the degree of deviation. Here, the standard heart rate refers to the standard value in the customer's personal history if one exists, or the standard value in the history of a standard person if one does not exist. If the abnormality score does not fall below a predetermined threshold, this technology either measures the heart rate again or determines that an abnormality has occurred and calls an operator.

[0005] However, when a user's heart rate deviates from the standard value, it does not necessarily mean that the user is suspected of being involved in a crime. For example, the heart rate of a user who is ill or unwell tends to be higher than that of a normal person. Furthermore, for example, after climbing stairs or running, the user's heart rate is generally higher than the user's normal heart rate. In these cases, the technology described in this document may determine that the user performing the operation is suspected of being involved in a crime, even if they are not involved in a crime, simply because the biometric information during the operation deviates from the standard.

[0006] One of the objectives of the present invention is to improve the accuracy of determining whether a user performing an operation related to a financial transaction is suspected of being involved in a crime or of attempting to perform other acts that should be restricted, compared to when changes in biometric information before and after the presentation of a warning are not used. [Means for solving the problem]

[0007] In one aspect, the present invention provides a program for causing a computer to execute the steps of accepting an operation related to a financial transaction from a user, presenting a warning to the user, acquiring first biometric information of the user based on a video of the user's face taken before the warning is presented, acquiring second biometric information of the user based on a video of the user's face taken after the warning is presented, and determining whether the change from the first biometric information to the second biometric information satisfies a predetermined condition. [Effects of the Invention]

[0008] According to the present invention, it is possible to improve the accuracy of determining whether a user performing an operation related to a financial transaction is suspected of being involved in a crime or of attempting to perform other acts that should be restricted, compared to when changes in biometric information before and after the presentation of a warning are not used. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram showing an example of the overall configuration of a determination system 9. [Figure 2] FIG. 1 is a diagram showing an example of the configuration of a determination device 1. [Figure 3] FIG. 10 is a diagram showing an example of a video DB 121. [Figure 4] FIG. 1 is a diagram for explaining an example of the concept of video data. [Figure 5] FIG. 2 is a diagram showing an example of a biometric information DB 122. [Figure 6] FIG. 10 is a diagram showing an example of a user attribute DB 123. [Figure 7] FIG. 4 is a diagram showing an example of a condition DB 124. [Figure 8] FIG. 10 is a diagram showing an example of a warning DB 125. [Figure 9] FIG. 10 is a diagram showing an example of a presentation history DB 126. [Figure 10] FIG. 2 is a diagram showing an example of the configuration of a server device 2. [Figure 11] FIG. 2 is a diagram showing an example of the functional configuration of the determination device 1. [Figure 12] FIG. 3 is a diagram showing an example of the flow of operations of the determination device 1. [Figure 13] FIG. 4 is a diagram showing an example of a display area R0. [Figure 14] FIG. 10 is a diagram showing an example of a display when shooting a selfie video. [Figure 15] FIG. 10 is a diagram showing an example of an operation screen for accepting procedures related to financial transactions. [Figure 16] FIG. 10 is a diagram showing an example of a warning display. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Embodiment> <Overall configuration of the judgment system> FIG. 1 is a diagram showing an example of the overall configuration of a determination system 9. The determination system 9 includes a determination device 1, a server device 2, and a communication line 3. The determination device 1 is an information processing device used by a user to perform procedures related to financial transactions, such as an automated teller machine, a smartphone, or a personal computer. The determination device 1 accepts operations related to procedures related to financial transactions from a user, and determines whether the user performing the operations is suspected of being involved in a crime or of attempting to perform other acts that should be restricted.

[0011] The server device 2 is an information processing device that provides financial transaction services by executing procedures according to operations accepted by the determination device 1. The server device 2 is not limited to one device, and may be configured with multiple information processing devices. The server device 2 may include, for example, a credit data transmission system, a settlement network, an electronic debt recording institution, or other networks of various financial institutions, or may cooperate with these to provide financial transaction services.

[0012] The determination device 1 and the server device 2 are communicatively connected via a communication line 3. The communication line 3 may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), an intranet, the Internet, or a combination thereof.

[0013] <Configuration of the determination device> 2 is a diagram showing an example of the configuration of the determination device 1. The determination device 1 is an information processing device that issues instructions to a server device 2 connected via a communication line 3 to execute procedures related to financial transactions.

[0014] This determination device 1 has a function for measuring the physical condition of the user based on a video of the user's face, and uses this function to determine whether the user is suspected of being involved in a crime or of attempting to perform other restricted activities. The determination device 1 shown in Figure 2 is a mobile terminal, such as a smartphone. Furthermore, the "physical condition" referred to here mainly refers to the state and condition of the mind and body resulting from autonomic nervous system disorders.

[0015] 2, the determination device 1 includes a processor 11, a memory 12, a communication unit 13, an operation unit 14, a display unit 15, and a camera 16. These are connected by a bus.

[0016] The processor 11 reads and executes a program stored in the memory 12 to control each part of the determination device 1. The processor 11 is, for example, a CPU (Central Processing Unit).

[0017] The communication unit 13 is a communication circuit that communicatively connects the determination device 1 to other devices via wire or wirelessly. For example, the communication unit 13 may have a circuit that complies with a wireless communication system standard such as IMT-2000, IMT-Advanced, or IMT-2020. Furthermore, for example, the communication unit 13 may have a circuit that complies with a wireless LAN standard such as IEEE802.11.

[0018] Furthermore, for example, the communication unit 13 may include a module that realizes near field communication (NFC). Examples of NFC standards include ISO / IEC18092 (NFCIP-1), ISO / IEC14443, ISO / IEC15693, and IEEE802.15.

[0019] The operation unit 14 is equipped with operation buttons, a touch panel, and other operators for issuing various instructions, and receives an operation and sends a signal corresponding to the operation content to the processor 11. This operation is, for example, pressing a button, making a gesture on the touch panel, or the like.

[0020] The operation unit 14 may have a microphone for collecting voice. In this case, the processor 11 may perform voice recognition processing on the voice data representing the user's voice collected by the microphone, and accept the recognition result as the user's operation.

[0021] The display unit 15 has a display screen such as a liquid crystal display, and displays images under the control of the processor 11. A transparent touch panel of the operation unit 14 may be placed on top of the display screen.

[0022] The display unit 15 has a display screen that displays images and the like that are visually perceived by the user, but may also have, in addition to this, for example, a speaker that emits sound.

[0023] The camera 16 is an imaging means, such as a digital still camera, that captures an image of the surroundings of the determination device 1. The camera 16 includes an optical system such as a lens, as well as an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0024] The camera 16 shown in Fig. 2 has a common configuration with, for example, the "RGB camera" of Patent Document 1 mentioned above. That is, this camera 16 is a device that acquires image data by arranging a plurality of detection elements, each of which has sensitivity to light intensity in a wavelength range corresponding to each of the colors R (red), G (green), and B (blue), as one pixel unit. The wavelength ranges corresponding to each color are preferably, for example, 580 nm to 680 nm for R (red), 500 nm to 630 nm for G (green), and 410 nm to 530 nm for B (blue).

[0025] The camera 16 may have a polarizing plate as part of its optical system. This polarizing plate is placed in the direction in which the camera 16 photographs the object. This allows the camera 16 to remove reflected components from the surface of the object.

[0026] Furthermore, the camera 16 may detect light in a wavelength range other than the above-mentioned ranges. For example, the camera 16 may be a five-band camera that adds two sensitivities, cyan (C) and orange (O), to a normal RGB camera.

[0027] The memory 12 is a storage means for storing an operating system, various programs, data, etc. that are loaded into the processor 11. The memory 12 includes a RAM (Random Access Memory) and a ROM (Read Only Memory).

[0028] The memory 12 may include a solid state drive, a hard disk drive, etc. The memory 12 shown in FIG.

[0029] <Video DB configuration> Fig. 3 is a diagram showing an example of the video DB 121. The video DB 121 is a database that stores data of videos (also called video data) that capture the user's face. The video DB 121 shown in Fig. 3 has a table structure that stores values ​​for each item that is associated with each other. The video DB 121 shown in Fig. 3 has the following items: video ID, start date and time, length, and video data.

[0030] In this video DB 121, the value of the video data is the video data itself. The value of the video ID is identification information for uniquely identifying the corresponding video data. The value of the start date and time is information on the date and time when filming of the corresponding video data started (i.e., the start date and time). The value of the length is a numerical value indicating the length of the corresponding video data. The unit of this length is, for example, a time unit such as minutes or seconds.

[0031] FIG. 4 is a diagram illustrating an example of the concept of video data. Video data is made up of multiple still images captured at regular intervals. One frame of these multiple still images is called a frame. The number of still images captured per unit time is called the frame rate. As shown in FIG. 4, video data is made up of multiple ordered frames each associated with image data indicating the still image captured at each instant. Note that video data may be compressed as long as the image data for each frame can be extracted and acquired.

[0032] <Biometric information DB configuration> FIG. 5 is a diagram showing an example of the biological information DB 122. The biological information DB 122 is a database that stores vital signs (hereinafter also referred to as biological information) that indicate the user's vital signs and physical condition measured based on video data. The biological information DB 122 shown in FIG. 5 has a table structure that stores values ​​for each associated item. The biological information DB 122 shown in FIG. 5 associates video IDs with biological information. This biological information consists of seven items: pulse, heart rate variability, respiratory rate, LF, HF, LF / HF, and TP. Of these items, the measured biological information values ​​are five items: pulse, heart rate variability, respiratory rate, LF, and HF. Furthermore, of these items, the index values ​​calculated by the determination device 1 from the values ​​of the biological information are two items: LF / HF and TP.

[0033] The moving image ID is identification information for identifying the moving image data that was the subject of measurement, and is the same information as that stored in the moving image DB 121. From the moving image data identified by this moving image ID, five corresponding items of biological information are measured. For example, the determination device 1 separates the pigment components from each of the image data included in the moving image data, and measures the pulse rate, etc. from the change over time in the distribution of the pigment components derived from hemoglobin in the blood.

[0034] The pulse value is the number of times the blood vessels in the user's face beat per minute. The pulse is equivalent to the number of times the heart beats per minute, so it is synonymous with the heart rate.

[0035] Heart rate variability (HRV) is a numerical value that represents the fluctuations observed in the time intervals between heartbeats. Heart rate variability reflects information related to biological control, such as the autonomic nervous system, and its characteristics are known to change with age and the presence or absence of illness.

[0036] Respiratory rate is the number of breaths per minute measured from the user's facial image.

[0037] The LF and HF values ​​are both stored as numerical values ​​calculated as a result of power spectrum analysis of heartbeat interval fluctuations. For example, the LF value is stored as the sum (integral value) of the intensities of low-frequency components, and the HF value is stored as the sum (integral value) of the intensities of high-frequency components. It is said that the low-frequency components correspond to blood pressure fluctuations, and the high-frequency components correspond to respiratory fluctuations.

[0038] The determination device 1 then calculates index values ​​used to identify the user's physical and mental state from the measured values ​​of the biological information, and stores the calculated index values ​​in the biological information DB 122. In the example shown in Fig. 5, the biological information DB 122 stores two calculated index values. LF / HF is a numerical value obtained by dividing LF by HF, and is an index value representing the "ratio of sympathetic nerves to parasympathetic nerves." TP is an index value representing the "level of activity of the entire autonomic nerves." TP is calculated, for example, by the following formula (1):

[0039] [Number 1] TP=LF / HF+HF ……(1) That is, TP is expressed as the sum of the value of "LF / HF", which is controlled by the sympathetic and parasympathetic nerves, and the value of "HF", which is controlled by the parasympathetic nerves.

[0040] <Attribute DB configuration> Fig. 6 is a diagram showing an example of the user attribute DB 123. The user attribute DB 123 is a database that stores the attributes of each user (referred to as user attributes). The user attribute DB 123 shown in Fig. 6 stores attribute items and attribute information in association with each other.

[0041] The attribute items include the user's age, sex, height, weight, etc. The attribute items may also include the user's language, facial contour features, whether or not the user wears glasses or a mask, etc.

[0042] The attribute information is information indicating the content of the corresponding attribute item. The attribute information is, for example, estimated from an operation received from the user by the operation unit 14. The attribute information may also be information input in advance by the user via the operation unit 14. The attribute information may also be estimated based on biometric information measured from a video of the user.

[0043] <Configuration of the condition database> 7 is a diagram showing an example of the condition DB 124. The condition DB 124 is a database that stores, for each type of warning, a condition for presenting the warning, a condition for determining that a change in the user's biometric information before and after the warning is presented is an abnormality, and a process ID that is identification information for the process when an abnormality is determined. The condition DB 124 has the following items: attribute condition, operation condition, warning ID, determination condition, and process ID.

[0044] The attribute condition is a condition for displaying a warning that relates to user attributes. The operation condition is a condition for displaying a warning that relates to an operation accepted from the user. The warning ID is identification information for the warning to be displayed.

[0045] For example, when the estimated or acquired user attribute satisfies this attribute condition and the operation accepted from the user satisfies this operation condition, the determination device 1 presents a warning identified by the corresponding warning ID.

[0046] The judgment conditions stored in the condition DB 124 are criteria used to determine whether a user performing an operation is suspected of being involved in a crime or of attempting to perform other restricted activities based on changes in biometric information measured before and after the issuance of a warning identified by a corresponding warning ID. If the changes in biometric information before and after the issuance of a warning satisfy these judgment conditions, the judgment device 1 determines that the user is suspected of being involved in a crime or of attempting to perform other restricted activities. For example, a user attempting to transfer money through a bank transfer fraud is not committing a crime, but is suspected of being involved in a crime because the user is about to become a victim of a crime. Furthermore, a user's attempt to withdraw money transferred through a bank transfer fraud from an account or an attempt to open a bank account used for bank transfer fraud or money laundering may directly constitute a crime or may not constitute a crime because the user is unaware of it and therefore is considered to be involved in a crime or an activity that a financial institution intends to restrict.

[0047] The process ID is identification information of a process to be executed when the determination device 1 determines that the user is suspected of being involved in a crime or of attempting to perform other acts that should be restricted.

[0048] <Warning DB configuration> FIG. 8 is a diagram showing an example of the warning DB 125. The warning DB 125 is a database that stores, for each warning ID, warning data, which is the content of the warning identified by that warning ID. The warning DB 125 shown in FIG. 8 stores warning text, warning images, warning sounds, and effects as warning data, each associated with a warning ID. A warning refers to a warning that alerts a user regarding a transaction that the user is about to perform through an operation, and is not limited to the warning data and its content exemplified here, as long as it has such content. Note that presenting a warning to a user refers to any act of showing a warning to a user, and this includes not only showing a warning text or warning image, but also sounding a warning sound.

[0049] The warning message is a sentence that indicates the content of the warning to be presented. For example, the warning message corresponding to warning ID "W1" is "Is this a bank transfer fraud?" This warning message is a sentence that assumes that the user performing the operation is a victim who has been caught up in the crime of bank transfer fraud. Also, for example, the warning message corresponding to warning ID "W3" is "Are you trying to illegally buy or sell an account?" This warning message is a sentence that assumes that the user performing the operation is not a victim, but a criminal who is trying to open an account for illegal buying or selling. In this way, warning messages are set according to the type of crime, the user's position, and the situation.

[0050] The warning image is an image such as an icon or illustration used to present a warning. The warning sound is a sound used to present a warning. This sound is output from an output device such as a speaker provided on the display unit 15 at the timing when the warning message is displayed on the display unit 15. The effect is information indicating the effect to be applied when presenting the warning message, warning image, etc. The effect item contains parameters for the effect to be used for presentation, such as blinking the warning message at a predetermined cycle or changing the color of the warning image.

[0051] <Configuration of the submission history DB> FIG. 9 is a diagram showing an example of the presentation history DB 126. The presentation history DB 126 is a database that stores identification information of presented warnings for each date and time when the warning was presented. For example, the presentation history DB 126 shown in FIG. 9 stores the presentation date and time "24 / 11 / 25 13:55" in association with the warning ID "W2" as the latest history. This indicates that the determination device 1 presented the warning identified by the warning ID "W2" at 13:55 on November 25, 2024.

[0052] <Server device configuration> Fig. 10 is a diagram showing an example of the configuration of the server device 2. As shown in Fig. 10, the server device 2 has a processor 21, a memory 22, and a communication unit 23. These are connected by a bus.

[0053] The processor 21 reads and executes a program stored in the memory 22 to control each part of the determination device 1. The processor 21 is, for example, a CPU.

[0054] The communication unit 23 is a communication circuit that connects the server device 2 to the determination device 1 via the communication line 3 in a wired or wireless manner so that communication is possible.

[0055] The memory 22 is a storage means for storing an operating system, various programs, data, etc. that are read into the processor 21. The memory 22 includes a RAM and a ROM.

[0056] The memory 22 may include a solid state drive, a hard disk drive, or the like.

[0057] The server device 2 exchanges information with the determination device 1 and executes the procedures for the financial transaction indicated by the operation accepted by the determination device 1. At this time, the server device 2 may connect to the networks of various financial institutions via the communication unit 23 and relay or mediate the above-mentioned procedures. If the determination device 1 determines that the user performing the operation is suspected of being involved in a crime or of attempting to perform other acts that should be restricted, the server device 2 receives instructions from the determination device 1 to suspend, suspend, or wait for the procedures for the financial transaction.

[0058] <Functional configuration of the determination device> Fig. 11 is a diagram illustrating an example of the functional configuration of the determination device 1. The processor 11 of the determination device 1 reads and executes a program stored in the memory 12, thereby functioning as a moving image acquisition unit 111, a biological information measurement unit 112, an estimation unit 113, a reception unit 114, a presentation unit 115, a biological information acquisition unit 116, and a determination unit 117 shown in Fig. 11.

[0059] The video acquisition unit 111 acquires video data showing a video of a user's own face captured by the camera 16. The video data acquired by the video acquisition unit 111 is stored in the video DB 121 of the memory 12.

[0060] The biological information measurement unit 112 reads video data from the video DB 121 and measures values ​​of biological information including the pulse based on the video data. The values ​​of the biological information measured by the biological information measurement unit 112 are stored in the biological information DB 122 together with a video ID that identifies the video data, as shown in FIG.

[0061] The estimation unit 113 estimates the user attributes and stores the result in the user attribute DB 123 of the memory 12. The estimation unit 113 may estimate the user attributes from, for example, measured biometric information. Alternatively, the estimation unit 113 may estimate the user attributes from, for example, a user operation accepted by the acceptance unit 114 described below. Note that if the user attributes are stored in the user attribute DB 123 by previously registering and editing, the processor 11 does not need to function as the estimation unit 113. In this case, the processor 11 may simply read and acquire the user attributes from the user attribute DB 123.

[0062] The reception unit 114 receives user operations for procedures related to financial transactions from the operation unit 14. The reception unit 114 is an example of a reception unit that receives operations for procedures related to financial transactions from the user. A financial transaction is a transaction between a user and a financial institution such as a bank, and includes, for example, remittance, withdrawal, account opening, etc.

[0063] The receiving unit 114, for example, refers to the condition DB 124, and if the received user operation satisfies the operation conditions shown in the condition DB 124 and the estimated user attribute satisfies the attribute conditions shown in the condition DB 124, notifies the presentation unit 115 of this.

[0064] When the user's operation and the user attributes satisfy the respective conditions, the presentation unit 115 causes the display unit 15 to present the associated warning. The presentation unit 115 is an example of a presentation unit that presents a warning to the user.

[0065] When the presentation unit 115 instructs the display unit 15 to present a warning, the presentation unit 115 stores the date and time (presentation date and time) and the identification information (warning ID) of the presented warning as history in the presentation history DB 126 of the memory 12.

[0066] The biometric information acquisition unit 116 reads out the latest history stored in the presentation history DB 126, and acquires from the biometric information DB 122 the biometric information measured before and after the date and time at which the warning indicated in the history was presented. The biometric information measured before the warning was presented is referred to as first biometric information. The biometric information measured after the warning was presented is referred to as second biometric information.

[0067] Therefore, when acquiring the first biometric information, the biometric information acquisition unit 116 is an example of a first acquisition unit that acquires the first biometric information of the user based on a video captured of the user's face before the warning is presented. Also, when acquiring the second biometric information, the biometric information acquisition unit 116 is an example of a second acquisition unit that acquires the second biometric information of the user based on a video captured of the user's face after the warning is presented.

[0068] The determination unit 117 refers to a determination condition associated with the identification information (warning ID) of the presented warning in the condition DB 124, and determines whether or not the change from the first biological information to the second biological information acquired by the biological information acquisition unit 116 satisfies the determination condition. The determination unit 117 is an example of a determination unit that determines whether or not the change from the first biological information to the second biological information satisfies a predetermined condition. The above-mentioned determination condition is specified, for example, by a threshold value of the change in heart rate before and after the presentation of the warning.

[0069] Then, when determining that this change satisfies the determination condition, determination unit 117 executes the process identified by the associated process ID in condition DB 124. For example, determination unit 117 may issue an instruction to display a predetermined image on display unit 15. In this case, determination unit 117 may display an additional operation screen for continuing the procedure related to the financial transaction.

[0070] Also, for example, the determination unit 117 may notify the server device 2 or a user who has operated an information processing device of a financial institution or the like via the communication unit 13 and the communication line 3 shown in Fig. 1 as to whether or not there is suspicion that the user is involved in a crime or is about to perform other acts that should be restricted. Furthermore, together with or separately from the notification as to whether or not there is suspicion that the user is involved in a crime or is about to perform other acts that should be restricted, the determination unit 117 may instruct the server device 2 to suspend, suspend, or wait for the procedure.

[0071] <Operation of the determination device> 12 is a diagram showing an example of the flow of operations of the determination device 1. The processor 11 of the determination device 1 reads and executes a program from the memory 12. When the program is started, it displays a screen prompting the user to take a picture of their own face, a so-called selfie.

[0072] Fig. 13 is a diagram showing an example of a display area R0. As shown in Fig. 13, a display unit 15 and a camera 16 are provided on the front surface of the determination device 1. As described above, the display unit 15 has a display screen such as a liquid crystal display. The upper part of this display screen is provided with a general information display area for displaying radio wave reception sensitivity, remaining battery power, time, etc., and the lower part is provided with a general operation area for receiving operations for the operating system. The display area R0 is provided between these areas.

[0073] 14 is a diagram showing an example of a display when shooting a selfie video. When processor 11 prompts the user to take a selfie, a cancel icon area R1, an explanation area R2, a face shooting area R3, and a measurement start button area R4 are displayed in display area R0, as shown in FIG.

[0074] The cancel icon area R1 is an area for accepting an instruction to cancel selfie video shooting.

[0075] Explanation area R2 is an area for displaying text explaining the steps for taking a selfie video. In the example shown in Figure 14, explanation area R2 displays the following explanation: "Hold the smartphone directly in front of your face, align your face with the circle, and press the 'Start measurement' button."

[0076] The face shooting area R3 is an area that indicates the range in which the user's own face should fit within the image area that simultaneously displays the selfie video being shot.

[0077] The measurement start button area R4 is an area that displays a button for starting selfie video shooting. When the user presses the measurement start button area R4, the determination device 1 starts shooting the selfie video. Note that the determination device 1 may start shooting the selfie video without receiving a user operation. For example, the processor 11 of the determination device 1 may analyze the characteristics of an image within a predetermined range (such as the face shooting area R3 described above) in an image captured by the camera 16, and start shooting the selfie video when it recognizes that the user's face is within this predetermined range.

[0078] Returning to the description of the operation shown in Fig. 12, when the user presses the measurement start button in response to the above-mentioned screen to start taking a selfie, the processor 11 starts acquiring the video supplied from the camera 16 (step S101).

[0079] In this embodiment, the user starts a selfie to capture a picture of their own face, but the subject of the picture is not limited to the face, and may be the user's own body. In addition, the processor 11 continues to execute the process of acquiring the video in the subsequent steps.

[0080] Next, the processor 11 monitors the operation unit 14 and accepts an operation related to a financial transaction from the user (step S102). This step S102 is an example of a step of accepting an operation related to a financial transaction from the user.

[0081] Fig. 15 is a diagram showing an example of an operation screen for accepting procedures related to financial transactions. As shown in Fig. 15, the determination device 1 displays in the display area R0 the title "Welcome to the XX Bank app," followed by explanatory text such as "You will now be setting up your use of XX Bank," and "To use this app, you will need an XX Bank account." The determination device 1 also displays in the display area R0 an account opening button area R5 with the string "Open an account" written on it, and a transaction start button area R6 with the string "Start a transaction" written on it.

[0082] The processor 11 acquires the user attributes (step S103). At this time, the processor 11 may, for example, estimate the user attributes from the content of the operation received from the user and acquire the estimation result. Alternatively, the processor 11 may read and acquire the user attributes from the user attribute DB 123 that has been input and edited in advance by the user. Alternatively, the processor 11 may acquire the user attributes estimated based on the video that began to be shot and acquired in step S101.

[0083] The processor 11 determines whether or not the conditions for presenting a warning are met (step S104). At this time, the processor 11 may determine, for example, whether or not the user attributes meet a predetermined condition. In this case, the program executed by the processor 11 is an example of a program that presents a warning in a manner according to the user attributes.

[0084] At this time, processor 11 may determine, for example, whether the accepted user operation satisfies a predetermined condition. In this case, the program executed by processor 11 is an example of a program that presents a warning in a manner according to the content of the accepted operation.

[0085] When it is determined that the condition for issuing a warning is not met (step S104; NO), the processor 11 proceeds to step S110. On the other hand, when it is determined that the condition for issuing a warning is met (step S104; YES), the processor 11 acquires first biometric information, which is biometric information before the warning is issued (step S105). Step S105 is an example of a step of acquiring the first biometric information of the user based on a video captured of the user's face before the warning is issued.

[0086] Then, the processor 11 presents a warning to the user (step S106). Step S106 is an example of a step of presenting a warning to the user.

[0087] Fig. 16 is a diagram showing an example of a warning presentation. For example, when the user presses the transaction start button area R6 shown in Fig. 15, the processor 11 causes the display unit 15 to display an operation screen for transactions (not shown). This operation screen for transactions includes buttons on which text such as "Withdraw," "Deposit," "Balance Inquiry," "Transfer," "Term Deposit," and "Remittance" is displayed. The functions of these buttons are, for example, as follows:

[0088] The "Withdraw" button allows the user to withdraw money from their own account. The "Deposit" button is a button that accepts an operation to deposit money into an account in the user's own name. The "Balance Inquiry" button is a button that accepts an operation to display the balance of money deposited in the user's own account. The "Transfer" button is an operation to transfer money from an account in the user's own name to an account in someone else's name. The "Fixed Term Deposit" button is a button that accepts an operation to deposit money into an account in the user's own name for a pre-specified period of time. The "Remit Money" button is an operation to send money from an account in the user's own name to another person. In this case, the destination of the money does not have to be another person's account, but may be, for example, another person's app, electronic money, etc.

[0089] For example, when a user presses the "Transfer" button or the "Remit" button on the transaction operation screen, processor 11 refers to condition DB 124 stored in memory 12 and determines that the conditions for presenting a warning when the user is involved in a bank transfer fraud are met. Then, a warning associated with this condition is presented. As a result, for example, a balloon image as shown in FIG. 16(a) is displayed near the transaction start button area R6 pressed by the user. This balloon image shown in FIG. 16(a) contains a warning message saying, "Is this a bank transfer fraud?" This warning message may be, for example, "Are you being asked by someone to transfer money?" or "Beware of bank transfer fraud." Furthermore, the warning message is not limited to bank transfer fraud, and may be something like, "Beware of special frauds."

[0090] The timing at which this balloon image is displayed is not limited to the timing at which the "Transfer" button or the "Remittance" button is pressed on the transaction operation screen. For example, this balloon image may be displayed when the user presses the transaction start button area R6.

[0091] The timing at which the balloon image shown in Fig. 16(a) is displayed may be when the user inputs the amount on the operation screen after pressing the "Transfer" button or the "Remit" button, or when the user presses the execute button. In other words, it is desirable that the balloon image shown in Fig. 16(a) be displayed at any time during the series of operations that instruct a transfer or remittance.

[0092] Furthermore, for example, when a user presses a "Withdraw" button on the transaction operation screen, processor 11 may cause display unit 15 to display a balloon image shown in FIG. 16(b) near this button. This balloon image shown in FIG. 16(b) contains a warning message saying, "Is this an unauthorized withdrawal?" The timing at which this balloon image shown in FIG. 16(b) is displayed is not limited to the above, and may be when the user presses transaction start button area R6, or at any time during a series of operations instructing a withdrawal.

[0093] Furthermore, for example, when the user presses the account opening button region R5 shown in FIG. 15, the processor 11 refers to the condition DB 124 stored in the memory 12 and determines that the condition for presenting a warning when the user is attempting to open a fraudulent account for buying and selling has been met. The processor 11 then presents a warning associated with this condition. As a result, for example, a balloon image shown in FIG. 16(c) is displayed near the account opening button region R5 pressed by the user. The balloon image shown in FIG. 16(c) contains the warning message, "Are you attempting to fraudulently buy or sell an account?"

[0094] The timing at which the speech bubble image shown in Fig. 16(c) is displayed is not limited to the above, and may be the timing when the user inputs the information required to open a new account, the timing when the user presses the final application button when opening a new account, etc. In other words, the timing at which the speech bubble image shown in Fig. 16(c) is displayed may be the timing at which any of a series of operations instructing the opening of a new account is performed. Note that an image warning against fraudulent account opening, like the speech bubble image shown in Fig. 16(c), may be displayed in a dedicated app for account opening.

[0095] Various studies, including polygraph tests, have revealed that a person's biometric information changes with a high probability when that person recalls some kind of memory after coming into contact with text, an image, a sound, etc. Changes in biometric information (also known as biological reactions) associated with memory recall include, for example, fluctuations in heart rate and changes in breathing patterns.

[0096] Generally, regardless of the level of consciousness, people have some degree of awareness of the situation they are in and remember that situation. Therefore, if the presented warning is similar to the user's situation, memory recall will occur in the user, resulting in a characteristic change in the biometric information. For example, if the user of this determination system 9 has even the slightest awareness that they may be involved in a crime such as bank transfer fraud, receiving the warning shown in Figure 16(a) will cause a change in the user's biometric information due to memory recall.

[0097] Furthermore, for example, if the user of this determination system 9 is even slightly aware that he or she plays the role of a so-called "depositor," when the user receives the warning shown in Figure 16(b), a change in the user's biometric information occurs due to memory recall. Here, a "depositor" is a person who withdraws cash from a victim's account in a bank transfer fraud.

[0098] Furthermore, for example, if a user of this determination system 9 has even the slightest awareness that he or she is committing a crime such as opening an account for fraudulent buying and selling, the user will be presented with the warning shown in Figure 16(c), which will cause a change in the user's biometric information due to memory recall.

[0099] When the warning is presented to the user, the processor 11 acquires second biometric information, which is biometric information after the warning (step S107). Step S107 is an example of a step of acquiring the second biometric information of the user based on a video captured of the user's face after the warning is presented.

[0100] Then, the processor 11 identifies a change from the first biometric information to the second biometric information, and determines whether or not this change satisfies a predetermined condition (step S108). This step S108 is an example of a step of determining whether or not the change from the first biometric information to the second biometric information satisfies a predetermined condition. When it is determined that the identified change in the biometric information does not satisfy the predetermined condition (step S108; NO), the processor 11 proceeds to the process of step S110.

[0101] On the other hand, when it is determined that the change in the biometric information satisfies the predetermined condition (step S108; YES), the processor 11 determines that there is an abnormality in the operation received from the user, and executes processing for the abnormality (step S109).

[0102] The processing in the event of an abnormality may be, for example, a process of sending a notification to an information processing device of the administrator of the determination system 9 via the communication unit 13. In this case, the program executed by the processor 11 is an example of a program that causes a computer to execute a step of sending a notification when it is determined that the change in the biometric information before and after the presentation of a warning satisfies the condition.

[0103] Furthermore, the processing in the event of an abnormality may be, for example, processing such as instructing the server device 2 to stop, suspend, or wait for an operation regarding a procedure for a financial transaction via the communication unit 13. In this case, the program executed by the processor 11 is an example of a program that causes a computer to execute a step of stopping, suspending, or waiting for an operation accepted from a user when it is determined that a change in biometric information before and after presenting a warning satisfies a condition.

[0104] If it is determined in step S104 of FIG. 12 that the conditions for presenting a warning are not met, and if it is determined in step S108 that the change in the user's biometric information before and after the warning does not meet the predetermined conditions, the processor 11 determines that there is no abnormality in the operation received from the user, executes normal processing (step S110), and returns the processing to step S101.

[0105] Through the operations described above, the determination device 1 presents a specific warning corresponding to a condition when the condition is met, giving the user an opportunity to recall a specific memory, and monitors changes in biometric information before and after the presentation. Therefore, the determination device 1 can capture changes in biometric information related to the presented warning and discover that the user is in a situation that corresponds to the warning. In other words, the determination system 9 including the determination device 1 can improve the accuracy of determining whether a user performing an operation related to a financial transaction is suspected of being involved in a crime or of attempting to engage in other restricted behavior, compared to a case where changes in biometric information before and after the presentation of a warning are not used.

[0106] <Modification> The above is a description of the embodiment, but the contents of this embodiment can be modified as follows. In addition, the following modifications can be combined with each other.

[0107] In the above-described embodiment, the examples were given of remittance, withdrawal, and account opening in bank transfer fraud, but the present invention is not limited to the above, and may be applied to any situation where a determination is made as to whether or not a user performing operations related to financial transactions is suspected of being involved in a crime or of attempting to perform other acts that should be restricted.

[0108] In the above-described embodiment, the determination device 1 has the processor 11 configured as a CPU, but the control means for controlling the determination device 1 may have other configurations. That is, the determination device 1 may have, as the processor 11, various types of processors other than a CPU, such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a programmable logic device, etc.

[0109] The operations of the processors in the above-described embodiments may not only be performed by a single processor, but may also be performed by a plurality of processors located at physically separate locations working together.

[0110] Furthermore, the order of the operations of the processor is not limited to the order described in the above embodiment, and may be changed as appropriate.

[0111] In the above-described embodiment, the processor 11 determines whether or not the change from the first biometric information to the second biometric information satisfies a predetermined condition, but various predetermined conditions can be adopted.

[0112] For example, the processor 11 may determine whether the above change satisfies a condition according to the user's attributes. That is, if the user's gender is male and the user's age is between 20 and 30, the determination device 1 may use, as the threshold value of the above condition, a statistical value such as an arithmetic mean or median of changes in biometric information obtained in advance through experiments on these users. In this case, the program executed by the processor 11 is an example of a program for executing a step of determining whether the change from the first biometric information to the second biometric information satisfies a condition according to the user's attributes.

[0113] Furthermore, for example, the processor 11 may determine whether the above-mentioned change satisfies a condition according to the content of the operation received from the user. That is, if the content of the operation received from the user is, for example, an operation for a transfer procedure exceeding a certain amount, the determination device 1 may adopt, as the threshold value of the above-mentioned condition, a statistical value such as an arithmetic mean value or a median value of the change in biometric information obtained in advance by an experiment on the user when performing this operation. In this case, the program executed by the processor 11 is an example of a program for executing a step of determining whether the change from the first biometric information to the second biometric information satisfies a condition according to the content of the received operation.

[0114] In the above-described embodiment, the determination device 1 presents a warning identified by a corresponding warning ID when the estimated or acquired user attribute satisfies the attribute condition and the operation received from the user satisfies the operation condition. However, the conditions for the determination device 1 to present a warning are not limited to these. For example, the determination device 1 may present a warning regardless of the user attribute. In this case, the determination device 1 does not need to estimate, acquire, or store the user attribute.

[0115] In the above-described embodiment, the processor 11 of the determination device 1 realizes the above-described functions by reading and executing a program from the memory 12. However, the processor 11 may realize only some, but not all, of these functions. In this case, the processor 21 of the server device 2 may realize, in place of the processor 11, any of the functional configurations realized by the processor 11 in the embodiment.

[0116] The program for executing each of the above steps may be provided in a state stored in a computer-readable recording medium such as a magnetic recording medium such as a magnetic tape or a magnetic disk, an optical recording medium such as an optical disk, a magneto-optical recording medium, a semiconductor memory, etc. This program may also be downloaded via a communication line such as the Internet. [Explanation of symbols]

[0117] 1...determination device, 11...processor, 111...video acquisition unit, 112...biometric information measurement unit, 113...estimation unit, 114...reception unit, 115...presentation unit, 116...biometric information acquisition unit, 117...determination unit, 12...memory, 121...video DB, 122...biometric information DB, 123...user attribute DB, 124...condition DB, 125...warning DB, 126...presentation history DB, 13...communication unit, 14...operation unit, 15...display unit, 16...camera, 2...server device, 21...processor, 22...memory, 23...communication unit, 3...communication line, 9...determination system, R0...display area, R1...cancel icon area, R2...explanation area, R3...face capture area, R4...measurement start button area, R5...account opening button area, R6...transaction start button area.

Claims

1. On the computer, receiving an operation related to a financial transaction from a user; presenting a warning to the user; acquiring first biometric information of the user based on a video of the user's face captured before presenting the warning; acquiring second biometric information of the user based on a video of the user's face after presenting the warning; determining whether a change from the first biometric information to the second biometric information satisfies a predetermined condition; A program to execute.

2. Presenting the warning in a manner according to the attributes of the user The program according to claim 1.

3. The warning is presented in a manner corresponding to the content of the received operation. The program according to claim 1.

4. The computer, and if it is determined that the change satisfies the condition, issuing a notification. The program according to claim 1.

5. The condition is a condition according to the attributes of the user. The program according to claim 1.

6. The condition is a condition according to the content of the received operation. The program according to claim 1.

7. The computer, If it is determined that the change satisfies the condition, the step of canceling, suspending, or waiting for the operation is executed. The program according to claim 1.

8. a reception unit that receives operations for procedures related to financial transactions from a user; a notification unit that presents a warning to the user; a first acquisition unit that acquires first biometric information of the user based on a video captured of the face of the user before the warning is presented; a second acquisition unit that acquires second biometric information of the user based on a video captured of the face of the user after the warning is presented; a determination unit that determines whether a change from the first biometric information to the second biometric information satisfies a predetermined condition; A determination system having the following.

Images