Transaction identity confirmation method and device
By collecting user's face and iris images on the ATM and using multimodal neural network for identity confirmation, the problem of insufficient security of ATM in large-value transaction scenarios is solved, and effective prevention of fraud is achieved.
Patent Information
- Application Number
- CN202311482249.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-08
- Publication Date
- 2025-05-13
AI Technical Summary
ATMs have insufficient security in large-value transaction scenarios, which cannot effectively prevent fraud, affecting users' enthusiasm for using cards.
A transaction identity confirmation method is adopted to collect user's face images and iris images through the camera, extract feature information and input it to the multimodal neural network for correlation processing, and verify transaction information to start the transaction process.
Provide security guarantees for large-scale transaction scenarios of ATMs, expand business scope, reduce pressure on bank outlets, and effectively prevent fraud.
Smart Images
Figure CN119991126A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, in particular the field of deep learning, and can also be applied to the financial field, and specifically relates to a transaction identity confirmation method and device. Background Art
[0002] With the in-depth development of today's financial electronic construction and the improvement of bank customers' requirements for financial service quality, ATMs are being used more and more widely in the financial industry. ATMs are playing an irreplaceable role in narrowing the distance between customers and banks, expanding business outlets, improving the card-using environment, providing all-weather and comprehensive financial services, reducing operating costs, and improving the service quality and comprehensive competitiveness of the financial industry.
[0003] However, many ATMs have not fully utilized their advanced functions. Some ATMs often malfunction and cannot provide services; some ATMs often make mistakes or swallow cards; some ATMs have even become simple decorations that no one cares about. In particular, there are frequent illegal acts of fraud and theft using ATMs, which have dampened people's enthusiasm for using cards. Advanced technology plays a vital role in maintaining and improving customer relationships. At present, ATMs have the following problems:
[0004] Firstly, due to certain security issues, large transactions cannot be performed at ATMs, which results in some limitations for ATMs, making them unable to be quick and convenient in some emergency scenarios.
[0005] In addition, some criminals use ATMs to commit fraud, pretending to be relatives of victims and asking some victims to transfer money to them at the ATMs. Summary of the invention
[0006] In response to the problems in the prior art, the present application provides a transaction identity confirmation method and device, which can provide security for large-value transaction scenarios of ATMs, expand the business scope of ATMs, reduce pressure on bank branch counters, and prevent fraud.
[0007] In order to solve at least one of the above problems, the present application provides the following technical solutions:
[0008] According to a first aspect of an embodiment of the present application, the present application provides a transaction identity confirmation method, comprising:
[0009] In response to receiving transaction information input by a user, verifying the transaction information, and collecting a facial image of the user through a camera if the verification result passes;
[0010] Acquire feature information of the face image, and extract feature vectors of the face image;
[0011] Capturing the user's iris image through a camera and extracting feature information of the iris image;
[0012] Inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain association information between the face image and the iris image;
[0013] The associated information is checked against the transaction information, and the transaction process is started if the check result passes.
[0014] According to any implementation manner of the present application, the acquiring feature information of the face image and extracting a feature vector of the face image includes:
[0015] Acquire feature information of the face image based on a preset convolutional neural network, wherein the feature information includes edge, texture, and shape information;
[0016] The facial image is mapped to a high-order feature space and input into a pre-trained twin network to obtain a feature vector of the facial image.
[0017] According to any implementation of the present application, the training method of the twin network includes:
[0018] Based on the comparison loss function, the twin network is controlled to minimize the distance between feature vectors of the same face image, and maximize the distance between feature vectors of different face images, until a distinguishable feature representation result is obtained.
[0019] According to any embodiment of the present application, the step of extracting feature information of the iris image includes:
[0020] Positioning the iris according to the boundary position of the iris in the iris image;
[0021] The iris image is normalized and enhanced, and feature information of the iris image is extracted based on wavelet transform.
[0022] According to any embodiment of the present application, the step of inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the face image and the iris image includes:
[0023] Inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to connect to a shared feature representation layer of the multimodal neural network;
[0024] The shared feature representation layer processes the feature vector of the face image and the feature information of the iris image based on the attention mechanism, and inputs the processing result to the fully connected layer to obtain the association information between the face image and the iris image.
[0025] According to any embodiment of the present application, it also includes:
[0026] Extracting time-series multimodal feature information from the feature vectors of the face images at different countdowns and the feature information of the iris images, and performing feature extraction and normalization processing on the time-series multimodal feature information of each countdown;
[0027] Performing feature recognition on the multimodal feature information based on a machine learning method to obtain a first prediction result of the user emotion;
[0028] Performing feature recognition on the multimodal feature information based on a convolutional neural network to obtain a second prediction result of the user emotion;
[0029] Performing feature recognition on the multimodal feature information based on a recursive neural network to obtain a third prediction result of the user emotion;
[0030] The first prediction result, the second prediction result and the third prediction result are integrated and input into a preset classification model to obtain a final prediction result of the user emotion.
[0031] According to any embodiment of the present application, it also includes:
[0032] In response to the final prediction result of the user's emotion being abnormal emotion, the user's customer information is sent to a business person at a transaction outlet, and the business person is prompted to determine whether the user is in a state of being defrauded.
[0033] According to a second aspect of an embodiment of the present application, the present application provides a transaction identity confirmation device, including:
[0034] The transaction information verification module is used to: in response to receiving the transaction information input by the user, verify the transaction information, and if the verification result passes, collect the face image of the user through the camera;
[0035] A facial feature determination module, used to: obtain feature information of the facial image and extract a feature vector of the facial image;
[0036] An iris feature determination module, used to: collect the user's iris image through a camera and extract feature information of the iris image;
[0037] A multimodal authentication module, used to: input the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the face image and the iris image;
[0038] The transaction verification module is used to: check the associated information with the transaction information and start the transaction process if the verification result is passed.
[0039] According to any implementation of the present application, the facial feature determination module includes:
[0040] A facial feature information acquisition unit, used to: acquire feature information of the facial image based on a preset convolutional neural network, wherein the feature information includes edge, texture and shape information;
[0041] A facial feature vector acquisition unit is used to: map the facial image to a high-order feature space and input it into a pre-trained twin network to obtain a feature vector of the facial image.
[0042] According to any implementation of the present application, the iris feature determination module includes:
[0043] An iris positioning unit, used to: locate the iris according to the boundary position of the iris in the iris image;
[0044] The iris feature information acquisition unit is used to: perform normalization and enhancement processing on the iris image, and extract feature information of the iris image based on wavelet transform.
[0045] According to any implementation of the present application, the multimodal authentication module includes:
[0046] A feature input unit, used to: input the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network, so as to connect to a shared feature representation layer of the multimodal neural network;
[0047] The associated information determination unit is used to: the shared feature representation layer processes the feature vector of the face image and the feature information of the iris image based on the attention mechanism, and inputs the processing result to the fully connected layer to obtain the associated information of the face image and the iris image.
[0048] According to any embodiment of the present application, it also includes a sentiment prediction module, which is used to:
[0049] Extracting time-series multimodal feature information from the feature vectors of the face images at different countdowns and the feature information of the iris images, and performing feature extraction and normalization processing on the time-series multimodal feature information of each countdown;
[0050] Performing feature recognition on the multimodal feature information based on a machine learning method to obtain a first prediction result of the user emotion;
[0051] Performing feature recognition on the multimodal feature information based on a convolutional neural network to obtain a second prediction result of the user emotion;
[0052] Performing feature recognition on the multimodal feature information based on a recursive neural network to obtain a third prediction result of the user emotion;
[0053] The first prediction result, the second prediction result and the third prediction result are integrated and input into a preset classification model to obtain a final prediction result of the user emotion.
[0054] According to any embodiment of the present application, a fraud prompt module is also included, which is used to:
[0055] In response to the final prediction result of the user's emotion being abnormal emotion, the user's customer information is sent to a business person at a transaction outlet, and the business person is prompted to determine whether the user is in a state of being defrauded.
[0056] According to a third aspect of an embodiment of the present application, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the transaction identity confirmation method when executing the program.
[0057] According to a fourth aspect of the embodiments of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the transaction identity confirmation method when executed by a processor.
[0058] According to a fifth aspect of the embodiments of the present application, the present application provides a computer program product, including a computer program / instruction, which implements the steps of the transaction identity confirmation method when executed by a processor.
[0059] It can be seen from the above technical scheme that the present application provides a transaction identity confirmation method and device, which collects the user's facial image through a camera when the verification result passes, obtains the feature information of the facial image, and extracts the feature vector of the facial image; collects the user's iris image through a camera, and extracts the feature information of the iris image; inputs the feature vector of the facial image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the facial image and the iris image; verifies the association information with the transaction information, and starts the transaction process when the verification result passes; it can provide security for large-value transaction scenarios of ATMs, expand the business scope of ATMs, reduce pressure on bank branch counters, and prevent fraud. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the above drawings without paying creative work.
[0061] Figure 1 This is one of the flow charts of the transaction identity confirmation method in the embodiment of the present application;
[0062] Figure 2 This is a second flow chart of the transaction identity confirmation method in the embodiment of the present application;
[0063] Figure 3 A twin network diagram of the transaction identity confirmation method in an embodiment of the present application;
[0064] Figure 4 The third flowchart of the transaction identity confirmation method in the embodiment of the present application;
[0065] Figure 5 This is a fourth flow chart of the transaction identity confirmation method in the embodiment of the present application;
[0066] Figure 6 This is a fifth flow chart of the transaction identity confirmation method in the embodiment of the present application;
[0067] Figure 7 This is one of the structural diagrams of the transaction identity confirmation device in the embodiment of the present application;
[0068] Figure 8 This is the second structural diagram of the transaction identity confirmation device in the embodiment of the present application;
[0069] Fig. 9 This is the third structural diagram of the transaction identity confirmation device in the embodiment of the present application;
[0070] Fig.10 This is the fourth structural diagram of the transaction identity confirmation device in the embodiment of the present application;
[0071] Fig.11 This is the fifth structural diagram of the transaction identity confirmation device in the embodiment of the present application;
[0072] Fig.12 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0074] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.
[0075] The present application provides a transaction identity confirmation method and device, which provide security protection for large-value transaction scenarios of ATMs, expand the business scope of ATMs, reduce pressure on bank branch counters, and prevent fraud.
[0076] In order to provide security for large-value transactions at ATMs, expand the business scope of ATMs, reduce pressure on bank counters, and prevent fraud, this application provides an embodiment of a transaction identity confirmation method, see Figure 1 The transaction identity confirmation method specifically includes the following contents:
[0077] Step S101: in response to receiving transaction information input by a user, verifying the transaction information, and capturing a facial image of the user through a camera if the verification result passes.
[0078] First, the system will take action based on the transaction information input from the user. The transaction information input by the user includes the transaction details to be executed, such as the transaction amount, transaction type, as well as the user's identity information, transaction password, etc.
[0079] After that, the transaction information provided by the user is verified. This includes checking whether the transaction information is legitimate, whether the user's account has sufficient funds to execute the transaction, etc. Once the system verifies the transaction information provided by the user, the camera is enabled to capture the user's facial image to ensure that the user is authenticated before executing large transactions, to help ensure that only authenticated users can complete the transaction and improve transaction security.
[0080] For example, when a user conducts a transaction, the bank card password information is first checked. If the password is correct and the customer wants to conduct a large transaction, the enhanced security authentication step is entered. The face information is collected using a camera with a collection size of 224x224x3, and the photo is transmitted through the network to a cloud server where the authentication device is deployed.
[0081] Step S102: Acquire feature information of the face image, and extract a feature vector of the face image.
[0082] Among them, relevant feature information can be extracted from the captured user's face image. The above feature information may include key attributes of the face, such as the position of the eyes, mouth, nose, facial contour, etc.
[0083] After that, a feature vector is further obtained, which is a mathematical representation that encodes the characteristic information of a face into a series of numbers. The vector is unique and can be used for subsequent comparison and identity verification.
[0084] In one embodiment of the transaction identity confirmation method of the present application, see Figure 2 , the step of obtaining the feature information of the face image and extracting the feature vector of the face image may further specifically include the following contents:
[0085] Step S102A: Acquire feature information of the face image based on a preset convolutional neural network, where the feature information includes edge, texture, and shape information.
[0086] Step S102B: Map the facial image to a high-order feature space and input it into a pre-trained twin network to obtain a feature vector of the facial image.
[0087] For example, first, a convolutional neural network (CNN), such as Resnet-18, can be used to extract key features from the input face image. The above features capture visual information such as edges, textures, shapes, etc. of the image, providing support for subsequent similarity comparisons.
[0088] At the same time, at this stage, the pre-trained CNN network can efficiently map the face image to a high-dimensional feature space, in which the visual information of the image is represented in numerical form, which is more suitable for subsequent analysis and processing.
[0089] Then, the feature vector of the face image is obtained by using the twin network architecture. The twin network consists of two identical sub-networks, each of which corresponds to an input image and generates a corresponding feature vector. The feature vector is used to compare the similarity between images through a distance metric, such as cosine distance or Euclidean distance.
[0090] In an optional embodiment, the training method of the twin network includes:
[0091] Based on the comparison loss function, the twin network is controlled to minimize the distance between feature vectors of the same face image, and maximize the distance between feature vectors of different face images, until a distinguishable feature representation result is obtained.
[0092] For example, Figure 3 As shown in the figure, the twin network forms a distinguishable feature representation by learning to bring the feature vectors of the same face closer together and push the feature vectors of different faces farther apart. In terms of loss function design, contrast loss is used here. Contrast loss aims to minimize the distance between feature vectors of the same face image while maximizing the distance between feature vectors of different face images. In this way, the twin network is guided to learn how to effectively encode face images for subsequent comparison and recognition.
[0093] Step S103: collecting the user's iris image through a camera and extracting feature information of the iris image.
[0094] Among them, a camera device is used to capture the user's iris image, and the iris is a special structure in the human eye, which has unique texture and characteristics, and each person's iris is unique. The camera will be positioned at the position of the user's eye and capture the image of the iris.
[0095] Afterwards, the iris image is processed to extract relevant feature information, including the texture, structure and other unique properties of the iris, for subsequent iris recognition.
[0096] In one embodiment of the transaction identity confirmation method of the present application, see Figure 4 , the extracting of feature information of the iris image may further specifically include the following contents:
[0097] Step S103A: locating the iris according to the boundary position of the iris in the iris image;
[0098] Step S103B: normalizing and enhancing the iris image, and extracting feature information of the iris image based on wavelet transform.
[0099] For example, the present application further integrates the collection and authentication process of iris information on the basis of realizing an image identity authentication device based on face information. The customer's iris information is collected through a camera, and the collected iris photo is transmitted through the network to a cloud server where the authentication device is deployed.
[0100] First, in the step of locating the inner and outer edges of the iris, the system accurately locates the iris area by detecting the inner and outer boundaries of the iris, thereby ensuring that subsequent processing targets the correct part of the iris. Subsequently, normalization processing is performed to eliminate the scale and rotation differences between different images by mapping the iris image to a standardized size and shape, making subsequent feature comparisons more accurate and reliable. In addition, in the image enhancement processing, a series of technologies are used to enhance the quality of the iris image, reduce noise and interference, and improve the effect of subsequent feature extraction and matching.
[0101] Afterwards, wavelet transform is used to capture the unique texture information in the iris image, obtaining the iris image in a higher dimensional feature space and generating a more informative representation for subsequent feature matching.
[0102] Step S104: inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain association information between the face image and the iris image.
[0103] First, the feature vector of the face image and the feature information of the iris image are used as input data and input into a multimodal neural network, which is specially designed to process the association information between different biometric features for identity authentication.
[0104] The two different types of feature information are processed by a multimodal neural network, and a result containing association information is output. The association information can be used to determine the relationship between the face image and the iris image to verify the identity of the user. This association information indicates whether the two features are consistent (i.e., belong to the same user) or inconsistent.
[0105] In one embodiment of the transaction identity confirmation method of the present application, see Figure 5 , the step of inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the face image and the iris image may also specifically include the following contents:
[0106] Step S104A: Inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to connect to a shared feature representation layer of the multimodal neural network.
[0107] Step S104B: The shared feature representation layer processes the feature vector of the face image and the feature information of the iris image based on the attention mechanism, and inputs the processing result to the fully connected layer to obtain the association information between the face image and the iris image.
[0108] This application introduces a multimodal neural network structure, which takes the features of the face and iris as input, connects a shared feature representation layer in series, and achieves the final identity determination with higher robustness and accuracy. This includes the features of the face and iris, which are then connected to the shared feature representation layer.
[0109] The shared feature representation layer uses the attention mechanism to realize the interaction of information of different modalities, and finally connects to the fully connected layer to realize the final identity prediction. The shared feature representation layer allows the model to fuse information of different modalities to better understand cross-modal data, that is, the association between iris images and face images, thereby improving the accuracy of identity verification.
[0110] Step S105: Check the association information with the transaction information, and start the transaction process if the check result passes.
[0111] The result of the multimodal feature authentication of the operating user is compared with the transaction information (eg, bank card user information). If the identification result matches successfully, the verification is passed.
[0112] From the above description, it can be seen that the transaction identity confirmation method provided in the embodiment of the present application can provide security for large-value transaction scenarios of ATMs, expand the business scope of ATMs, reduce pressure on bank branch counters, and prevent fraud.
[0113] In one embodiment of the transaction identity confirmation method of the present application, see Figure 6 , also includes:
[0114] Step S106: extracting time-series multimodal feature information from the feature vectors of the face images at different countdowns and the feature information of the iris images, and performing feature extraction and normalization processing on the time-series multimodal feature information of each countdown;
[0115] Step S107: performing feature recognition on the multimodal feature information based on a machine learning method to obtain a first prediction result of the user emotion;
[0116] Step S108: performing feature recognition on the multimodal feature information based on a convolutional neural network to obtain a second prediction result of the user emotion;
[0117] Step S109: performing feature recognition on the multimodal feature information based on a recursive neural network to obtain a third prediction result of the user emotion;
[0118] Step S110: Integrate the first prediction result, the second prediction result and the third prediction result, and input them into a preset classification model to obtain a final prediction result of the user emotion.
[0119] In anti-fraud scenarios, this application can further provide fraud warnings through deep emotion recognition.
[0120] Current emotion recognition methods are mainly divided into two categories: recognition based on non-physiological signals and recognition based on physiological signals.
[0121] Emotion recognition methods based on non-physiological signals mainly include the recognition of facial expressions and voice intonation. Facial expression recognition methods identify different emotions based on the correspondence between expressions and emotions. Under specific emotional states, people will have specific facial muscle movements and expression patterns, such as when they are happy, the corners of their mouths will turn up, and circular wrinkles will appear around their eyes; when they are angry, they will frown and open their eyes wide, etc.
[0122] At present, facial expression recognition is mostly achieved by image recognition. Voice and intonation recognition is achieved based on the different ways people express themselves in different emotional states. For example, when you are in a good mood, your tone of voice will be more cheerful, while when you are upset, your tone of voice will be more dull. The advantage of the recognition method based on non-physiological signals is that it is easy to operate and does not require special equipment.
[0123] However, the disadvantage of the above scheme is that the reliability of emotion recognition cannot be guaranteed. This is because people can disguise their true emotions by disguising their facial expressions and voice intonation, and such disguise is often difficult to detect.
[0124] To this end, this application utilizes the temporal multimodal feature information of face images and irises, and combines machine learning methods, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Stacking integration to achieve emotion recognition and abnormal emotion detection.
[0125] First, temporal multimodal feature information was extracted from face images and iris data at different countdowns. For each countdown, the above features were extracted and normalized; then, the multimodal features of each countdown were averaged and the average features were used for emotion recognition.
[0126] Use traditional machine learning methods, such as random forests, to identify emotions and obtain prediction result A. Then, for the multimodal features of each countdown, deep feature extraction is performed based on the representative image processing method convolutional neural network, and then the emotions are identified through the fully connected layer to obtain prediction result B. Finally, in order to consider the timing information, a recursive neural network is used to identify the multimodal features at different times to obtain prediction result C.
[0127] The emotion recognition result A of the traditional machine learning method (random forest), the emotion recognition result B of CNN and the emotion recognition result C of RNN are stacked and integrated, and the Logistic classification model is used to learn and predict to determine whether there is abnormal emotion.
[0128] The innovation of this solution lies in the ability to integrate a variety of different technologies and models, fully considering information in different time and space to improve the accuracy and robustness of emotion recognition.
[0129] Furthermore, in a preferred embodiment, in response to the final prediction result of the user's emotion being abnormal emotion, the user's customer information is sent to a business person at a transaction outlet, and the business person is prompted to determine whether the user is in a state of being defrauded.
[0130] Among them, if it is judged that there are abnormal emotions, the customer is deemed to be at risk of being defrauded, and the customer information is sent to the staff of nearby branches. The staff who have received anti-fraud training will analyze the customer's behavior and determine whether he or she is at risk of being defrauded.
[0131] This application uses facial recognition and iris recognition to conduct extremely convenient and secure identity authentication in large-value transactions at ATMs, and uses deep emotion recognition to provide fraud warnings in anti-fraud scenarios.
[0132] In order to provide security for large-value transactions at ATMs, expand the business scope of ATMs, reduce pressure on bank counters, and prevent fraud, the present application provides an embodiment of a transaction identity confirmation device for implementing all or part of the transaction identity confirmation method, see Figure 7 The transaction identity confirmation device specifically includes the following contents:
[0133] The transaction information verification module 1101 is used to: in response to receiving the transaction information input by the user, verify the transaction information, and if the verification result passes, collect the face image of the user through the camera;
[0134] The face feature determination module 1102 is used to: obtain feature information of the face image and extract a feature vector of the face image;
[0135] The iris feature determination module 1103 is used to: collect the iris image of the user through a camera and extract feature information of the iris image;
[0136] The multimodal authentication module 1104 is used to: input the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the face image and the iris image;
[0137] The transaction verification module 1105 is used to check the associated information with the transaction information and start the transaction process if the check result is passed.
[0138] According to any embodiment of the present application, see Figure 8 , the facial feature determination module includes:
[0139] The facial feature information acquisition unit 1002A is used to: acquire feature information of the facial image based on a preset convolutional neural network, wherein the feature information includes edge, texture and shape information;
[0140] The facial feature vector acquisition unit 1002B is used to: map the facial image to a high-order feature space and input it into a pre-trained twin network to obtain a feature vector of the facial image.
[0141] According to any embodiment of the present application, see Fig. 9 , the iris feature determination module includes:
[0142] An iris positioning unit 1003A is used to: locate the iris according to the boundary position of the iris in the iris image;
[0143] The iris feature information acquisition unit 1003B is used to normalize and enhance the iris image, and extract feature information of the iris image based on wavelet transform.
[0144] According to any embodiment of the present application, see Fig.10 , the multimodal authentication module comprises:
[0145] The feature input unit 1004A is used to: input the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to connect to the shared feature representation layer of the multimodal neural network;
[0146] The associated information determination unit 1004B is used to: the shared feature representation layer processes the feature vector of the face image and the feature information of the iris image based on the attention mechanism, and inputs the processing result to the fully connected layer to obtain the associated information of the face image and the iris image.
[0147] According to any embodiment of the present application, it also includes a sentiment prediction module, which is used to:
[0148] Extracting time-series multimodal feature information from the feature vectors of the face images at different countdowns and the feature information of the iris images, and performing feature extraction and normalization processing on the time-series multimodal feature information of each countdown;
[0149] Performing feature recognition on the multimodal feature information based on a machine learning method to obtain a first prediction result of the user emotion;
[0150] Performing feature recognition on the multimodal feature information based on a convolutional neural network to obtain a second prediction result of the user emotion;
[0151] Performing feature recognition on the multimodal feature information based on a recursive neural network to obtain a third prediction result of the user emotion;
[0152] The first prediction result, the second prediction result and the third prediction result are integrated and input into a preset classification model to obtain a final prediction result of the user emotion.
[0153] According to any embodiment of the present application, a fraud prompt module is also included, which is used to:
[0154] In response to the final prediction result of the user's emotion being abnormal emotion, the user's customer information is sent to a business person at a transaction outlet, and the business person is prompted to determine whether the user is in a state of being defrauded.
[0155] From the above description, it can be seen that the transaction identity confirmation device provided in the embodiment of the present application can provide security for large-value transaction scenarios of ATMs, expand the business scope of ATMs, reduce pressure on bank branch counters, and prevent fraud.
[0156] In order to further illustrate the present solution, the present application also provides a specific application example of using the above transaction identity confirmation device to implement the transaction identity confirmation method, see Fig.11 , specifically including the following contents:
[0157] First, through the transaction information input device 201, when the user conducts a transaction, the bank card password information is first checked. If the password is correct and the customer wants to conduct a large-amount transaction, the enhanced security authentication step is entered.
[0158] In the image acquisition device 202, a camera is used to collect facial information, and the photo is transmitted through the network to a cloud server where an authentication device is deployed.
[0159] Afterwards, when it is confirmed that the user conducts a large-value transaction, the corresponding feature vector is generated by the image identity confirmation device 203. The collected iris photo is transmitted to the cloud server where the authentication device is deployed through the network by the iris information collection device 204, and the iris image is mapped to a standardized size and shape by the iris information confirmation device 205, and the unique texture information in the iris image is captured using wavelet transform.
[0160] Afterwards, a multimodal neural network structure is introduced into the large-value transaction security authentication device 206 to achieve the final identity prediction, and the result of the multimodal feature authentication of the operating user is compared with the bank card user information. If the recognition result matches successfully, the verification is passed.
[0161] Finally, emotion recognition and abnormal emotion detection are realized through the emotion analysis device 207.
[0162] First, in step 6, time-series multimodal feature information is extracted from the face images and iris data of different countdowns. If it is determined that there is abnormal emotion, it is determined that the customer is likely to be defrauded, and the customer information is sent to the staff of nearby outlets through the fraud warning device 208. The staff who have received anti-fraud training will analyze the customer's behavior and determine whether he is likely to be defrauded.
[0163] From the hardware level, in order to provide security for large-value transaction scenarios of ATMs, expand the business scope of ATMs, reduce pressure on bank branch counters, and prevent fraud, the present application provides an embodiment of an electronic device for implementing all or part of the content of the transaction identity confirmation method, and the electronic device specifically includes the following content:
[0164] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the transaction identity confirmation device and the core business system, user terminal and related database and other related devices; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the transaction identity confirmation method and the embodiment of the transaction identity confirmation device in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.
[0165] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.
[0166] In practical applications, part of the transaction identity confirmation method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.
[0167] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.
[0168] Fig.12 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig.12 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.12 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.
[0169] In one embodiment, the transaction identity confirmation method function may be integrated into the central processor 9100. The central processor 9100 may be configured to perform the following control:
[0170] Step S101: In response to receiving transaction information input by a user, verify the transaction information, and if the verification result passes, collect the user's face image through a camera.
[0171] Step S102: Acquire feature information of the face image, and extract a feature vector of the face image;
[0172] Step S103: collecting the user's iris image through a camera, and extracting feature information of the iris image;
[0173] Step S104: inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain association information between the face image and the iris image;
[0174] Step S105: The association information is compared with the transaction information, and the transaction process is started if the comparison result passes.
[0175] From the above description, it can be seen that the electronic device provided in the embodiment of the present application provides security for large-value transaction scenarios of ATMs, expands the business scope of ATMs, reduces pressure on bank branch counters, and plays a preventive role in fraud.
[0176] In another embodiment, the transaction identity confirmation device may be configured separately from the central processor 9100. For example, the transaction identity confirmation device may be configured as a chip connected to the central processor 9100, and the transaction identity confirmation method function is implemented under the control of the central processor.
[0177] like Fig.12 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.12 In addition, the electronic device 9600 may also include Fig.12 For components not shown, reference may be made to the prior art.
[0178] like Fig.12 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device 9600.
[0179] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.
[0180] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.
[0181] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.
[0182] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).
[0183] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.
[0184] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.
[0185] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the transaction identity confirmation method in the above embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the transaction identity confirmation method in the above embodiment, where the execution subject is a server or a client, are implemented. For example, when the processor executes the computer program, the following steps are implemented:
[0186] Step S101: In response to receiving transaction information input by a user, verify the transaction information, and if the verification result passes, collect the user's face image through a camera.
[0187] Step S102: Acquire feature information of the face image, and extract a feature vector of the face image;
[0188] Step S103: collecting the user's iris image through a camera, and extracting feature information of the iris image;
[0189] Step S104: inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain association information between the face image and the iris image;
[0190] Step S105: Check the association information with the transaction information, and start the transaction process if the check result passes.
[0191] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application provides security for large-value transaction scenarios of ATMs, expands the business scope of ATMs, reduces pressure on bank branch counters, and plays a preventive role in fraud.
[0192] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the transaction identity confirmation method in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the transaction identity confirmation method are implemented. For example, the computer program / instruction implements the following steps:
[0193] Step S101: In response to receiving transaction information input by a user, verify the transaction information, and if the verification result passes, collect the user's face image through a camera.
[0194] Step S102: Acquire feature information of the face image, and extract a feature vector of the face image;
[0195] Step S103: collecting the user's iris image through a camera, and extracting feature information of the iris image;
[0196] Step S104: inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain association information between the face image and the iris image;
[0197] Step S105: Check the association information with the transaction information, and start the transaction process if the check result passes.
[0198] From the above description, it can be seen that the computer program product provided in the embodiment of the present application provides security for large-value transaction scenarios of ATMs, expands the business scope of ATMs, reduces pressure on bank branch counters, and plays a preventive role in fraud.
[0199] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0200] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. The above-mentioned computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0201] The above-mentioned computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0202] The above-mentioned computer program instructions can also be loaded into a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0203] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A transaction identity confirmation method, characterized in that: The method comprises: In response to receiving transaction information input by a user, verifying the transaction information, and collecting a facial image of the user through a camera if the verification result passes; Acquire feature information of the face image, and extract feature vectors of the face image; Capturing the user's iris image through a camera and extracting feature information of the iris image; Inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain association information between the face image and the iris image; The associated information is checked against the transaction information, and the transaction process is started if the check result passes.
2. The transaction identity confirmation method according to claim 1, characterized in that: The step of obtaining feature information of the face image and extracting a feature vector of the face image includes: Acquire feature information of the face image based on a preset convolutional neural network, wherein the feature information includes edge, texture, and shape information; The facial image is mapped to a high-order feature space and input into a pre-trained twin network to obtain a feature vector of the facial image.
3. The transaction identity confirmation method according to claim 2, characterized in that: The training method of the twin network includes: Based on the comparison loss function, the twin network is controlled to minimize the distance between feature vectors of the same face image, and maximize the distance between feature vectors of different face images, until a distinguishable feature representation result is obtained.
4. The transaction identity confirmation method according to claim 1, characterized in that: The step of extracting feature information of the iris image comprises: Positioning the iris according to the boundary position of the iris in the iris image; The iris image is normalized and enhanced, and feature information of the iris image is extracted based on wavelet transform.
5. The transaction identity confirmation method according to claim 1, characterized in that: The step of inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the face image and the iris image includes: Inputting the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to connect to a shared feature representation layer of the multimodal neural network; The shared feature representation layer processes the feature vector of the face image and the feature information of the iris image based on the attention mechanism, and inputs the processing result to the fully connected layer to obtain the association information between the face image and the iris image.
6. The transaction identity confirmation method according to claim 1, characterized in that: Also includes: Extracting time-series multimodal feature information from the feature vectors of the face images at different countdowns and the feature information of the iris images, and performing feature extraction and normalization processing on the time-series multimodal feature information of each countdown; Performing feature recognition on the multimodal feature information based on a machine learning method to obtain a first prediction result of the user emotion; Performing feature recognition on the multimodal feature information based on a convolutional neural network to obtain a second prediction result of the user emotion; Performing feature recognition on the multimodal feature information based on a recursive neural network to obtain a third prediction result of the user emotion; The first prediction result, the second prediction result and the third prediction result are integrated and input into a preset classification model to obtain a final prediction result of the user emotion.
7. The transaction identity confirmation method according to claim 1, characterized in that: Also includes: In response to the final prediction result of the user's emotion being abnormal emotion, the user's customer information is sent to a business person at a transaction outlet, and the business person is prompted to determine whether the user is in a state of being defrauded.
8. A transaction identity confirmation device, characterized in that: The device comprises: The transaction information verification module is used to: in response to receiving the transaction information input by the user, verify the transaction information, and if the verification result passes, collect the face image of the user through the camera; A facial feature determination module, used to: obtain feature information of the facial image and extract a feature vector of the facial image; An iris feature determination module, used to: collect the user's iris image through a camera and extract feature information of the iris image; A multimodal authentication module, used to: input the feature vector of the face image and the feature information of the iris image into a preset multimodal neural network to obtain the association information between the face image and the iris image; The transaction verification module is used to: check the associated information with the transaction information and start the transaction process if the verification result is passed.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the transaction identity confirmation method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the transaction identity confirmation method according to any one of claims 1 to 7 are implemented.