Face-swiping payment method, device, electronic device and storage medium
By identifying face-scanning and gesture information, generating payment intention values and determining target users, the problem of low security in multi-user scenarios is solved, and the accuracy and user experience of face-scanning payment are improved.
Patent Information
- Application Number
- CN202110935725.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-16
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-08-16
AI Technical Summary
Face-scan payment method is difficult to judge the user's willingness to scan face in multiple user scenarios, resulting in misidentifying the target user and reducing the security of payment.
By collecting image information, identifying the user's face-scanning information and gesture information, using the multi-task detection model and gesture classification model to generate payment intention values, and determining the target user with face-scanning payment intention to make face-scanning payments.
It effectively avoids the situation of misidentifying the target user, improves the security of face-scanning payment, and enriches the user's interactive experience.
Smart Images

Figure CN113657903B_ABST
Abstract
Description
[Technical field]
[0001] The embodiments of this specification relate to the field of electronic payment, and in particular to a face-scanning payment method, device, electronic device and storage medium. [Background technology]
[0002] With the development of network and terminal technology, the payment method has evolved from traditional cash payment to cashless electronic payment, and electronic payment has also evolved from code scanning payment to more convenient face payment. Face payment is a payment method that uses facial recognition as identity authentication. Due to its convenience, it is adopted by more and more users.
[0003] However, while face-scanning payment provides users with convenient services, it also has some security risks. When multiple users appear in front of the face-scanning payment device, it is difficult for the face-scanning payment device to determine which user is willing to scan their face, and the target user may be misidentified, resulting in the wrong face-scanning payment, thereby reducing the security of face-scanning payment. [Summary of the invention]
[0004] In view of this, the embodiments of this specification provide a face-scanning payment method, device, electronic device and storage medium for improving the security of face-scanning payment.
[0005] In a first aspect, the embodiments of this specification provide a face-scanning payment method, including:
[0006] Collect image information;
[0007] identifying user behavior information of at least one user according to the image information, wherein the user behavior information includes face scanning information, or the user behavior information includes face scanning information and gesture information;
[0008] If at least one gesture information is recognized, generating a willingness to pay value for each user according to the user behavior information of the at least one user;
[0009] If a target user is determined from the at least one user according to the payment willingness value of each user, face recognition payment is performed on the target user.
[0010] In a possible implementation, identifying user behavior information of at least one user based on the image information, wherein the user behavior information includes face scanning information and gesture information, includes:
[0011] Input the image information into a pre-trained multi-task detection model, and output face scanning information, a face frame image, a gesture frame image, and a correspondence between the face frame image and the gesture frame image of at least one user; track the face frame image by a target tracking technology to generate a face tracking sequence, and track the gesture frame image by a target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate a gesture tracking sequence corresponding to the face tracking sequence;
[0012] The gesture tracking sequence is input into a pre-trained gesture classification model, and the gesture information is output.
[0013] In a possible implementation, the training method of the gesture classification model includes:
[0014] Collect video data of different gestures;
[0015] Calibrate gesture labels for video data of different gestures;
[0016] The gesture classification model is trained based on the video data of different gestures and gesture labels.
[0017] In a possible implementation, the gesture classification model is a neural network model.
[0018] In a possible implementation, generating the willingness-to-pay value of each user according to the user behavior information of the at least one user includes:
[0019] User behavior information of at least one user is input into a pre-trained willingness recognition model, and a willingness-to-pay value of each user is output.
[0020] In a possible implementation, the training method of the intention recognition model includes:
[0021] Collect face recognition video data with different intentions;
[0022] Assign willingness labels to face recognition video data with different intentions;
[0023] Extract gesture information, face position information, face posture information and face key point information from face scanning video data with different intentions;
[0024] The intention recognition model is trained according to the gesture information, face position information, face posture information, face key point information and intention label.
[0025] In a possible implementation, the intention recognition model is a neural network model.
[0026] In a possible implementation, determining a target user from the at least one user according to the willingness-to-pay value of each user includes:
[0027] Determining whether there is a willingness-to-pay value greater than a set threshold among the willingness-to-pay values of at least one user;
[0028] If it is determined that at least one user's willingness to pay value has a willingness to pay value greater than a set threshold, determining a maximum willingness to pay value from the willingness to pay values greater than the set threshold;
[0029] The user corresponding to the maximum willingness to pay value is determined as the target user.
[0030] In a possible implementation, the face scanning information includes at least one of face position information, face posture information or face key point information.
[0031] In a second aspect, the embodiments of this specification provide a face-scanning payment device, including:
[0032] An acquisition module, used for acquiring image information;
[0033] an identification module, configured to identify user behavior information of at least one user based on the image information, wherein the user behavior information includes face scanning information, or the user behavior information includes face scanning information and gesture information;
[0034] A generating module, configured to generate a willingness-to-pay value for each user according to user behavior information of the at least one user if the identifying module identifies at least one gesture information;
[0035] A determination module, configured to determine a target user from the at least one user according to the willingness to pay value of each user;
[0036] The payment module is used to perform face-scanning payment for the target user.
[0037] In a possible implementation, when the user behavior information includes face scanning information and gesture information, the recognition module includes:
[0038] A first generating submodule is used to input the image information into a pre-trained multi-task detection model, and output face scanning information, a face frame image, a gesture frame image, and a correspondence between the face frame image and the gesture frame image of at least one user;
[0039] A tracking submodule, configured to track the face frame image by using a target tracking technology to generate a face tracking sequence, and to track the gesture frame image by using a target tracking technology according to a correspondence between the face frame image and the gesture frame image to generate a gesture tracking sequence corresponding to the face tracking sequence;
[0040] The second generating submodule is used to input the gesture tracking sequence into a pre-trained gesture classification model and output the gesture information.
[0041] In a possible implementation, the generation module is specifically configured to input user behavior information of at least one user into a pre-trained willingness recognition model, and output a willingness-to-pay value of each of the users.
[0042] In a possible implementation, the determining module includes:
[0043] A judgment submodule, used to judge whether there is a willingness to pay value greater than a set threshold value among the willingness to pay values of at least one user;
[0044] A first determination submodule, configured to determine a maximum willingness-to-pay value from among the willingness-to-pay values that are greater than the set threshold value if the determination submodule determines that at least one user's willingness-to-pay value has a willingness-to-pay value that is greater than the set threshold value;
[0045] The second determination submodule is used to determine the user corresponding to the maximum willingness to pay value as the target user.
[0046] In a third aspect, an embodiment of the present specification provides an electronic device, including:
[0047] One or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, enable the electronic device to execute the face-swiping payment method in the first aspect or any possible implementation of the first aspect.
[0048] In a fourth aspect, an embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the face-swiping payment method in the first aspect or any possible implementation of the first aspect.
[0049] In the technical solution provided in the embodiments of this specification, user behavior information of at least one user is identified based on the collected image information, wherein the user behavior information includes face scanning information or the user behavior information includes face scanning information and gesture information. If at least one gesture information is identified, a willingness to pay value for each user is generated based on the user behavior information of at least one identified user. If a target user is determined from at least one user based on the willingness to pay value of each user, face scanning payment is performed for the target user. In the embodiments of this specification, the face scanning payment process is awakened by identifying gesture information, a willingness to pay value is generated based on the face scanning information and gesture information, and face scanning payment is performed for the target user determined based on the willingness to pay value, thereby avoiding misidentification of the target user, thereby improving the security of face scanning payment.
Brief Description of the Drawings
[0050] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0051] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;
[0052] Figure 2 A flow chart of a face-scanning payment method provided in an embodiment of this specification;
[0053] Figure 3 A schematic diagram of a face-scanning payment method provided in an embodiment of this specification;
[0054] Figure 4 A flowchart for identifying face scanning information and gesture information provided in the embodiments of this specification;
[0055] Figure 5 A schematic diagram of facial gesture detection provided in an embodiment of this specification;
[0056] Figure 6 Another schematic diagram of facial gesture detection provided in the embodiments of this specification;
[0057] Figure 7 Another schematic diagram of facial gesture detection provided in the embodiments of this specification;
[0058] Figure 8 Another schematic diagram of facial gesture detection provided in the embodiments of this specification;
[0059] Fig. 9 A flowchart for generating a willingness to pay value provided in an embodiment of this specification;
[0060] Fig.10 A schematic diagram of the structure of a face-scanning payment device provided in an embodiment of this specification;
[0061] Fig.11 A schematic diagram of the structure of the identification module provided in the embodiments of this specification;
[0062] Fig.12 A schematic diagram of the structure of the determination module provided in the embodiments of this specification;
[0063] Fig.13 A schematic diagram of an electronic device provided in an embodiment of this specification. [Specific implementation method]
[0064] In order to better understand the technical solution of this specification, the embodiments of this specification are described in detail below with reference to the accompanying drawings.
[0065] It should be clear that the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.
[0066] The terms used in the embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit this specification. The singular forms "a", "said" and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0067] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0068] The embodiment of the present specification can implement the face-scanning payment method through an electronic device. The electronic device is a device with a face-scanning payment function, for example, the electronic device can be independently set or set in an automatic vending machine. Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification, such as Figure 1As shown, the electronic device may include a face-scanning payment device 11, and the face-scanning payment device 11 may include a display screen 12 and a camera 13. For example, the camera 13 may be arranged on the top of the display screen 12. The display screen 12 may be used to display a display interface. For example, when the face-scanning payment device 11 does not perform face-scanning payment, the display interface may be used to display advertising information; when the face-scanning payment device 11 performs face-scanning payment, the display interface may be used to display the user's image information and / or payment information, etc. For example, the payment information may include the payment account, the payment amount, etc. In actual applications, the display interface may also display other screens as needed, which are not listed here one by one. The camera 13 may be used to collect the user's image information.
[0069] like Figure 1 As shown, further, the electronic device may also include a bracket 14, which is arranged at the bottom of the face-scanning payment device 11. The bracket 14 is used to support the face-scanning payment device 11 so that the electronic device can be placed on the checkout counter.
[0070] Based on the electronic device provided by the embodiments of this specification, for example Figure 1 The electronic device shown in the figure provides a face-swiping payment method in an embodiment of the present specification. Figure 2 A flow chart of a face-scanning payment method provided in an embodiment of this specification, Figure 3 A schematic diagram of a face-swiping payment method provided in an embodiment of this specification, such as Figure 2 and Figure 3 As shown, the method includes:
[0071] Step 102: Collect image information.
[0072] Each step in the embodiments of this specification can be executed by an electronic device.
[0073] In the embodiments of this specification, the electronic device can collect image information in real time, that is, the electronic device maintains real-time pre-collection of image information before executing the face-swiping payment process. When a user is in front of the electronic device, the electronic device can collect the user's image information through its own camera. The number of users in front of the electronic device can be one or more. If the number of users is one, the image information collected by the electronic device includes the image of one user; if the number of users is multiple, the image information collected by the electronic device includes the images of multiple users.
[0074] Step 104: Identify user behavior information of at least one user based on the image information, where the user behavior information includes face scanning information, or the user behavior information includes face scanning information and gesture information.
[0075] In the embodiments of this specification, when the user only stands in front of the electronic device and does not make any gestures, the user behavior information recognized by the electronic device includes face scanning information; when the user stands in front of the electronic device and makes gestures, the user behavior information recognized by the electronic device includes face scanning information and gesture information.
[0076] In the embodiments of this specification, the face scanning information includes at least one of face position information, face posture information, or face key point information. The face position information may include face distance information and / or image position information, and the face posture may be the head rotation angle. Among them, the face distance information may be the distance between the user's face and the display screen; the image position information may be the position of the face in the display interface, for example, the face is centered, to the left, to the right, to the top, or to the bottom in the display interface.
[0077] The face position information may include face distance information. The face distance information may be used to indicate whether the user is willing to pay by face scanning. For example, if a user stands in front of an electronic device and the face distance information of the user is large, the user is far away from the display screen, indicating that the user is not willing to pay by face scanning. For example, if multiple users stand in front of an electronic device, the user with the smaller face distance information among the multiple users indicates that the user is closer to the display screen and has a stronger willingness to pay by face scanning.
[0078] Face location information may include image location information. Image location information may be used to indicate whether the user is willing to pay by face scanning. For example, if a user stands in front of an electronic device and the face is centered in the display interface, it indicates that the user is willing to pay by face scanning. For example, if a user stands in front of an electronic device and the face is lower in the display interface, it indicates that the user is not willing to pay by face scanning. For example, if multiple users stand in front of an electronic device, the more centered the face is in the display interface, the stronger the user's willingness to pay by face scanning.
[0079] Face location information may include face distance information and image location information. Both face distance information and image location information can be used to indicate whether the user is willing to pay by face scanning. For example, if a user stands in front of an electronic device, the user's face distance information is large and the face in the display interface is lower or higher, it indicates that the user does not have the willingness to pay by face scanning. For example, if a user stands in front of an electronic device, the user's face distance information is small and the face in the display interface is centered, it indicates that the user has the willingness to pay by face scanning. For example, if multiple users stand in front of an electronic device, the user with smaller face distance information and more centered face in the display interface among the multiple users indicates that the user has a stronger willingness to pay by face scanning.
[0080] The facial posture information is the head rotation angle. The head rotation angle can be used to indicate whether the user has the willingness to pay by face scanning. Among them, the head rotation angle can be a rotation angle in a preset direction, for example, the head rotation angle includes the rotation angle of the head in three directions: pitch, yaw, and roll. Pitch refers to rotation around the preset X-axis, and the angle of rotation around the preset X-axis is also called the pitch angle; yaw refers to rotation around the preset Y-axis, and the angle of rotation around the preset Y-axis is also called the yaw angle; roll refers to rotation around the preset Z-axis, and the angle of rotation around the preset Z-axis is also called the roll angle. Among them, the size of the rotation angle in the preset direction is directly related to the user's willingness to pay by face scanning. For example, if a user stands in front of an electronic device, when the head rotation angle is large, the electronic device cannot capture all parts of the user's face, which indicates that the user does not have the willingness to pay by face scanning. For example, if a user stands in front of an electronic device, when the head rotation angle is small, the electronic device can capture all parts of the user's face, which indicates that the user has the willingness to pay by face scanning. For example, if multiple users stand in front of an electronic device, the user with a smaller head rotation angle among multiple users is a user with a stronger willingness to pay by face scanning.
[0081] In the embodiment of the present specification, the gesture information may be the type of gesture made by the user. For example, the gesture information may include an OK gesture, a thumbs-up gesture, or a waving gesture.
[0082] Figure 4 This is a flowchart of recognizing face scanning information and gesture information provided in the embodiments of this specification, such as Figure 4 As shown, the user behavior information includes face scanning information and gesture information, and step 104 may specifically include:
[0083] Step 1042: input the image information into a pre-trained multi-task detection model, and output face scanning information, a face frame image, a gesture frame image, and a correspondence between the face frame image and the gesture frame image of at least one user.
[0084] In the embodiments of this specification, the multi-task detection model may be a deep learning model. The deep learning model may be a neural network model, for example, the neural network model may be a convolutional neural network (CNN) model.
[0085] In the embodiment of this specification, each gesture frame image may correspond to a face frame image, that is, the gesture frame image and the face frame image have a corresponding relationship. At this time, one or more users stand in front of the electronic device, and one or more of the users make gestures, so the face frame image and the gesture frame image of the one or more users making gestures can be detected, and the detected face frame image and gesture frame image of each user have a corresponding relationship.
[0086] It should be noted that: in the embodiment of this specification, when the user behavior information includes face scanning information, the step 104 of identifying the user behavior information of at least one user based on the image information of at least one user may specifically include: inputting the image information of at least one user into a pre-trained multi-task detection model, and outputting the face scanning information and face frame image of at least one user. At this time, one or more users stand in front of the electronic device, but no user makes a gesture, so only the face scanning information and face frame image of each user are detected, and the gesture frame image is not detected.
[0087] Combine the following Figures 5 to 8 ,The facial gesture detection process is described in detail through four examples.
[0088] The first case: Figure 5 As shown, user A stands in front of the electronic device and does not make any gestures, then the face scanning information and face frame image 15 of user A are detected.
[0089] The second case: Figure 6 As shown, user A stands in front of the electronic device and makes a gesture, and the face scanning information, face frame image 15 and gesture frame image 16 of user A are detected. Among them, the face frame image 15 of user A and the gesture frame image 16 of user A have a corresponding relationship, and the corresponding relationship between the face frame image 15 and the gesture frame image 16 of user A can be detected.
[0090] The third situation: Figure 7 As shown, user A and user B stand side by side in front of the electronic device, and user A makes a gesture, while user B does not make a gesture, then the face scanning information, face frame image 15 and gesture frame image 16 of user A are detected, and the face scanning information and face frame image 17 of user B are detected. Among them, the face frame image 15 of user A and the gesture frame image 16 of user A have a corresponding relationship, and the corresponding relationship between the face frame image 15 of user A and the gesture frame image 16 can be detected.
[0091] The fourth case: Figure 8 As shown, user A and user B stand in front of the electronic device, user A is closer to the display screen than user B, and both user A and user B make gestures, then the face scanning information, face frame image 15 and gesture frame image 16 of user A are detected, and the face scanning information, face frame image 17 and gesture frame image 18 of user B are detected. Among them, the face frame image 15 of user A and the gesture frame image 16 of user A have a corresponding relationship, and the face frame image 17 of user B and the gesture frame image 18 of user B have a corresponding relationship, then the corresponding relationship between the face frame image 15 and the gesture frame image 16 of user A and the corresponding relationship between the face frame image 17 and the gesture frame image 18 of user B can be detected.
[0092] Step 1044: Track the face frame image using the target tracking technology to generate a face tracking sequence, and track the gesture frame image using the target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate a gesture tracking sequence corresponding to the face tracking sequence.
[0093] In the embodiments of this specification, the face tracking sequence may include multiple frames of face frame images, and the gesture tracking sequence may include multiple frames of gesture frame images. Since the face frame images and the gesture frame images have a corresponding relationship, during the target tracking process, the face tracking sequence and the corresponding gesture tracking sequence can be tracked according to the corresponding relationship between the face frame images and the gesture frame, and the face tracking sequence and the gesture tracking sequence of the same user tracked have a corresponding relationship.
[0094] like Figure 6 As shown, the face frame 15 of user A is tracked by target tracking technology to generate a face tracking sequence of user A, and the gesture frame image 16 of user A is tracked by target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate a gesture tracking sequence of user A corresponding to the face tracking sequence of user A.
[0095] like Figure 7 As shown, the face frame 15 of user A is tracked by the target tracking technology to generate the face tracking sequence of user A, and the gesture frame image 16 of user A is tracked by the target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate the gesture tracking sequence of user A corresponding to the face tracking sequence of user A. It should be noted that: since user B did not make a gesture, although the face frame image 17 of user B was detected in step 1042, the gesture frame image and the correspondence between the undetected face frame image and the gesture frame image were not detected, and therefore the gesture tracking sequence of user B was not tracked.
[0096] like Figure 8 As shown, the face frame 15 of user A is tracked by the target tracking technology to generate the face tracking sequence of user A, and the gesture frame image 16 of user A is tracked by the target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate the gesture tracking sequence of user A corresponding to the face tracking sequence of user A. The face frame 17 of user B is tracked by the target tracking technology to generate the face tracking sequence of user B, and the gesture frame image 18 of user B is tracked by the target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate the gesture tracking sequence of user B corresponding to the face tracking sequence of user B.
[0097] It should be noted that: Figure 5As shown, since user A did not make any gesture, although the face frame image 15 of user A was detected in step 1042, the gesture frame image and the correspondence between the undetected face frame image and the gesture frame image were not detected, and therefore the gesture tracking sequence of user A was not tracked.
[0098] Step 1046: input the gesture tracking sequence into a pre-trained gesture classification model and output gesture information.
[0099] In the embodiments of this specification, the gesture classification model may be a deep learning model. The deep learning model may be a neural network model, for example, the neural network model may be a CNN model.
[0100] In the embodiment of this specification, the training method of the gesture classification model includes:
[0101] Step 1048a: Collect video data of different gestures.
[0102] Step 1048b: calibrate gesture labels for video data of different gestures.
[0103] For example, manually calibrating gesture labels for the collected video data of different gestures.
[0104] Step 1048c: train a gesture classification model based on the video data and gesture labels of different gestures.
[0105] For example, a gesture classification model can be trained by training video data and gesture labels of different gestures through CNN.
[0106] Step 106: If at least one gesture information is recognized, generate a willingness to pay value for each user according to user behavior information of at least one user.
[0107] In the embodiment of this specification, if the electronic device recognizes at least one gesture information, the face-scanning payment process can be awakened. In this step, if the electronic device recognizes at least one gesture information, it can trigger the step of generating a payment willingness value for each user based on the user behavior information of at least one user.
[0108] For example, Figure 5 As shown, since user A standing in front of the electronic device did not make a gesture, the electronic device did not detect the gesture frame image, and therefore did not recognize the gesture information. If the electronic device does not recognize the gesture information, the face-scanning payment process cannot be awakened, that is, the subsequent face-scanning payment process will not be executed.
[0109] For example, Figure 6 and Figure 7As shown, since user A standing in front of the electronic device makes a gesture, the electronic device detects the gesture frame image, and thus recognizes the gesture information of user A. Since the electronic device recognizes the gesture information, the face-swiping payment process is awakened. Figure 6 and Figure 7 The gesture information of user A identified in is the ok gesture.
[0110] For example, Figure 8 As shown, since user A and user B standing in front of the electronic device both make gestures, the electronic device detects the gesture frame image of user A and the gesture frame image of user B, and thus recognizes the gesture information of user A and the gesture information of user B. When the electronic device recognizes the gesture information, the face-swiping payment process is awakened. Figure 8 The gesture information of user A and the gesture information of user B identified in the figure are both ok gestures.
[0111] In this step, if at least one gesture information is recognized, the user behavior information of at least one user is input into the pre-trained willingness recognition model, and the willingness to pay value of each user is output. The user's willingness to pay value is used to indicate the degree of the user's willingness to pay. For example, the larger the value of the user's willingness to pay value, the higher the user's willingness to pay degree.
[0112] In the embodiments of this specification, the intention recognition model may be a deep learning model. The deep learning model may be a neural network model, for example, the neural network model may be a CNN model.
[0113] In the embodiments of this specification, the training method of the intention recognition model includes:
[0114] Step 1062: Collect face recognition video data of different intentions.
[0115] Step 1064: calibrate intention labels for face recognition video data with different intentions.
[0116] For example, the collected face scanning video data with different intentions are manually calibrated with intention labels.
[0117] Step 1066: extract gesture information, face position information, face posture information and face key point information from the face scanning video data of different intentions.
[0118] Step 1068: Train the intention recognition model based on the gesture information, face position information, face posture information, face key point information and intention label.
[0119] For example, gesture information, face position information, face posture information, face key point information and intention labels are trained through CNN to train an intention recognition model.
[0120] In the embodiment of this specification, as another optional solution, a face recognition button can also be displayed on the display interface of the display screen 12. The user clicks the face recognition button to wake up the face recognition payment process, which can trigger the step of generating the payment willingness value of each user according to the user behavior information of at least one user. In this case, since the user does not make a gesture, the user behavior information of at least one user may not include gesture information.
[0121] Step 108: If a target user is determined from at least one user according to each user's willingness to pay value, face recognition payment is performed on the target user.
[0122] Fig. 9 The flowchart for generating willingness to pay value provided in the embodiment of this specification is as follows: Fig. 9 As shown, determining the target user from at least one user according to the willingness to pay value of each user in step 108 may specifically include:
[0123] Step 1082: determine whether there is a willingness-to-pay value greater than a set threshold among the willingness-to-pay values of at least one user. If so, execute step 1084; if not, end the process.
[0124] In the embodiment of the present specification, if it is determined that at least one user's willingness to pay value has a willingness to pay value greater than a set threshold, indicating that there is at least one user who is willing to pay by face recognition, then step 1084 can be executed; if it is determined that at least one user's willingness to pay value does not have a willingness to pay value greater than the set threshold, that is, all willingness to pay values are less than the set threshold, indicating that there is no user who is willing to pay by face recognition, then the subsequent process will no longer be executed.
[0125] For example, when only user A stands in front of the electronic device, only the willingness to pay value of user A is generated in step 106. For example, if the willingness to pay value of user A is 0.8 and the threshold is set to 0.5, the willingness to pay value determined to be greater than the set threshold is the willingness to pay value of user A, and step 1084 is continued.
[0126] For example, when user A and user B stand in front of the electronic device, the willingness to pay value of user A and the willingness to pay value of user B are generated in step 106. For example, the willingness to pay value of user A is 0.8, the willingness to pay value of user B is 0.6, and the threshold is set to 0.5, then the willingness to pay values judged to be greater than the set threshold include the willingness to pay value of user A and the willingness to pay value of user B, and step 1084 is continued.
[0127] For example, when user A stands in front of the electronic device, the willingness to pay value of user A is generated in step 106. For example, the willingness to pay value of user A is 0.2, and the threshold is set to 0.5. Since the willingness to pay value of user A is less than the set threshold, it is determined that there is no willingness to pay value greater than the set threshold, and the process ends. In this case, the target user cannot be determined later.
[0128] In the embodiment of this specification, users whose willingness to pay values are greater than the set threshold are users who are willing to pay by face recognition.
[0129] Step 1084: Determine the maximum willingness to pay value from the willingness to pay values that are greater than the set threshold.
[0130] In the embodiment of this specification, if the number of willingness-to-pay values greater than the set threshold is one, the willingness-to-pay value greater than the set threshold is determined as the maximum willingness-to-pay value. For example, if the willingness-to-pay value greater than the set threshold is the willingness-to-pay value of user A, the willingness-to-pay value of user A is determined as the maximum willingness-to-pay value.
[0131] In the embodiment of this specification, if there are multiple willingness-to-pay values greater than the set threshold, the largest value among the multiple willingness-to-pay values greater than the set threshold is determined as the maximum willingness-to-pay value. For example, if the willingness-to-pay values greater than the set threshold are the willingness-to-pay value of user A and the willingness-to-pay value of user B, and the willingness-to-pay value of user A is 0.8 and the willingness-to-pay value of user B is 0.6, the willingness-to-pay value of user A is determined as the maximum willingness-to-pay value.
[0132] Step 1086: Determine the user corresponding to the maximum willingness to pay value as the target user.
[0133] In the embodiment of this specification, the largest willingness to pay value indicates that the willingness to pay by face recognition is the strongest, and the target user is the user with the strongest willingness to pay by face recognition among users who have the willingness to pay by face recognition.
[0134] In the embodiment of this specification, after the target user is selected, the target user performs face-swiping payment. Then the face-swiping payment for the target user in step 108 may specifically include: the electronic device sends the image information of the target user to the server, and the server compares the face in the image information of the target user with the standard face. If the comparison is successful, the payment process is completed.
[0135] As an optional solution, after step 108, the electronic device displays information indicating that the target user has made a successful payment.
[0136] In the technical solution provided in the embodiments of this specification, user behavior information of at least one user is identified based on the collected image information, wherein the user behavior information includes face scanning information or the user behavior information includes face scanning information and gesture information. If at least one gesture information is identified, a willingness to pay value for each user is generated based on the user behavior information of at least one identified user. If a target user is determined from at least one user based on the willingness to pay value of each user, face scanning payment is performed for the target user. In the embodiments of this specification, the face scanning payment process is awakened by identifying gesture information, a willingness to pay value is generated based on the face scanning information and gesture information, and face scanning payment is performed for the target user determined based on the willingness to pay value, thereby avoiding misidentification of the target user, thereby improving the security of face scanning payment.
[0137] In the embodiments of the present specification, when the user is making a face-scanning payment, the face-scanning payment process can be awakened by gesture information, without the need for the user to click a face-scanning button displayed on the display screen to awaken the face-scanning payment process, thereby preventing the user from touching the display screen during the face-scanning payment process. As a result, users who are worried about the display screen being dirty and do not want to touch the display screen can also complete the face-scanning payment process, thereby enriching the user's interactive experience when making a face-scanning payment.
[0138] Fig.10 A schematic diagram of a face-scanning payment device provided in an embodiment of this specification is shown in FIG. Fig.10 As shown, the device includes: a collection module 21, an identification module 22, a generation module 23, a determination module 24 and a payment module 25.
[0139] The acquisition module 21 is used to acquire image information. The recognition module 22 is used to identify user behavior information of at least one user based on the image information, where the user behavior information includes face scanning information, or the user behavior information includes face scanning information and gesture information. The generation module 23 is used to generate a willingness to pay value for each user based on the user behavior information of at least one user if the recognition module 22 identifies at least one gesture information. The determination module 24 is used to determine a target user from at least one user based on the willingness to pay value of each user. The payment module 25 is used to perform face scanning payment for the target user.
[0140] Fig.11 Schematic diagram of the structure of the identification module in the embodiment of this specification, such as Fig.11As shown, when the user behavior information includes face scanning information and gesture information, the recognition module 22 includes: a first generation submodule 221, a tracking submodule 222, and a second generation submodule 223. The first generation submodule 221 is used to input the image information into a pre-trained multi-task detection model, and output the face scanning information, face frame image, gesture frame image, and the correspondence between the face frame image and the gesture frame image of at least one user. The tracking submodule 222 is used to track the face frame image through the target tracking technology to generate a face tracking sequence, and to track the gesture frame image according to the correspondence between the face frame image and the gesture frame image through the target tracking technology to generate a gesture tracking sequence corresponding to the face tracking sequence. The second generation submodule 223 is used to input the gesture tracking sequence into a pre-trained gesture classification model and output gesture information.
[0141] like Fig.10 As shown, in the embodiment of this specification, the device also includes: a first calibration module 26 and a first training module 27. The acquisition module 21 is also used to collect video data of different gestures. The first calibration module 26 is used to calibrate gesture labels for video data of different gestures. The first training module 27 is used to train a gesture classification model based on video data and gesture labels of different gestures.
[0142] In the embodiment of this specification, the generation module 23 is specifically used to input the user behavior information of at least one user into a pre-trained willingness recognition model, and output the willingness to pay value of each user.
[0143] like Fig.10 As shown, in the embodiment of this specification, the device also includes: a second calibration module 28, an extraction module 29 and a second training module 30. The acquisition module 21 is also used to collect face-scanning video data of different intentions. The second calibration module 28 is used to calibrate the intention labels for the face-scanning video data of different intentions. The extraction module 29 is used to extract gesture information, face position information, face posture information and face key point information from the face-scanning video data of different intentions. The second training module 30 is used to train the intention recognition model based on the gesture information, face position information, face posture information, face key point information and intention labels.
[0144] Fig.12 This is a schematic diagram of the structure of the module in the embodiment of this specification. Fig.12As shown, the determination module 24 includes: a judgment submodule 241, a first determination submodule 242, and a second determination submodule 243. The judgment submodule 241 is used to determine whether there is a willingness to pay value greater than a set threshold value among the willingness to pay values of at least one user. The first determination submodule 242 is used to determine the maximum willingness to pay value from the willingness to pay values greater than the set threshold value if the judgment submodule determines that there is a willingness to pay value greater than the set threshold value among the willingness to pay values of at least one user. The second determination submodule 243 is used to determine the user corresponding to the maximum willingness to pay value as the target user.
[0145] In the technical solution provided in the embodiments of this specification, user behavior information of at least one user is identified based on the collected image information, wherein the user behavior information includes face scanning information or the user behavior information includes face scanning information and gesture information. If at least one gesture information is identified, a willingness to pay value for each user is generated based on the user behavior information of at least one identified user. If a target user is determined from at least one user based on the willingness to pay value of each user, face scanning payment is performed for the target user. In the embodiments of this specification, the face scanning payment process is awakened by identifying gesture information, a willingness to pay value is generated based on the face scanning information and gesture information, and face scanning payment is performed for the target user determined based on the willingness to pay value, thereby avoiding misidentification of the target user, thereby improving the security of face scanning payment.
[0146] An embodiment of the present specification provides a computer-readable storage medium, which includes a stored program, wherein when the program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the embodiment of the above-mentioned face-swiping payment method.
[0147] An embodiment of the present specification provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, enable the electronic device to execute an embodiment of the above-mentioned face-swiping payment method.
[0148] Fig.13 Schematic diagram of an electronic device provided in an embodiment of this specification. Fig.13 As shown, the electronic device 3 of this embodiment includes: a processor 31, a memory 32, and a computer program 33 stored in the memory 32 and executable on the processor 31. When the computer program 33 is executed by the processor 31, the face-swiping payment method in the embodiment is implemented. To avoid repetition, they are not described one by one here.
[0149] The electronic device 3 includes, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will appreciate that Fig.13It is only an example of the electronic device 3 and does not constitute a limitation of the electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 3 may also include input and output devices, network access devices, buses, etc.
[0150] The processor 31 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0151] The memory 32 may be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 32 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 3. Further, the memory 32 may also include both an internal storage unit and an external storage device of the electronic device 3. The memory 32 is used to store computer programs and other programs and data required by the electronic device 3. The memory 32 may also be used to temporarily store data that has been output or is to be output.
[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0153] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0155] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0156] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a computer-readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (Processor) to perform some steps of the method described in each embodiment of this specification. The aforementioned computer-readable storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0157] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.
Claims
1. A face-scanning payment method, comprising: Collect image information; identifying user behavior information of at least one user according to the image information, wherein the user behavior information includes face scanning information and gesture information; If at least one gesture information is recognized, generating a willingness to pay value for each user according to the user behavior information of the at least one user; If a target user is determined from the at least one user according to the willingness to pay value of each user, performing face recognition payment for the target user; The step of identifying user behavior information of at least one user according to the image information includes: Inputting the image information into a pre-trained multi-task detection model, and outputting face scanning information, a face frame image, a gesture frame image, and a correspondence between the face frame image and the gesture frame image of at least one user; Tracking the face frame image by target tracking technology to generate a face tracking sequence, and tracking the gesture frame image by target tracking technology according to the correspondence between the face frame image and the gesture frame image to generate a gesture tracking sequence corresponding to the face tracking sequence; The gesture tracking sequence is input into a pre-trained gesture classification model, and the gesture information is output.
2. The method according to claim 1, wherein: The training method of the gesture classification model includes: Collect video data of different gestures; Calibrate gesture labels for video data of different gestures; The gesture classification model is trained based on the video data and gesture labels of different gestures.
3. The method according to claim 1, wherein: The gesture classification model is a neural network model.
4. The method according to claim 1, wherein: Generating the willingness to pay value of each user according to the user behavior information of the at least one user includes: User behavior information of at least one user is input into a pre-trained willingness recognition model, and a willingness-to-pay value of each user is output.
5. The method according to claim 4, wherein: The training method of the intention recognition model includes: Collect face recognition video data with different intentions; Assign willingness labels to face recognition video data with different intentions; Extract gesture information, face position information, face posture information and face key point information from face scanning video data with different intentions; The intention recognition model is trained according to the gesture information, face position information, face posture information, face key point information and intention label.
6. The method according to claim 4, wherein: The intention recognition model is a neural network model.
7. The method according to claim 1, wherein: The step of determining a target user from the at least one user according to the willingness to pay value of each user includes: Determining whether there is a willingness-to-pay value greater than a set threshold among the willingness-to-pay values of at least one user; If it is determined that at least one user's willingness to pay value has a willingness to pay value greater than a set threshold, determining a maximum willingness to pay value from the willingness to pay values greater than the set threshold; The user corresponding to the maximum willingness to pay value is determined as the target user.
8. The method according to any one of claims 1 to 7, wherein: The face scanning information includes at least one of face position information, face posture information or face key point information.
9. A face-scanning payment device, comprising: An acquisition module, used for acquiring image information; A recognition module, used to recognize user behavior information of at least one user according to the image information, wherein the user behavior information includes face scanning information and gesture information; A generating module, configured to generate a willingness-to-pay value for each user according to user behavior information of the at least one user if the identifying module identifies at least one gesture information; A determination module, configured to determine a target user from the at least one user according to the willingness to pay value of each user; A payment module, used to perform face-scanning payment for the target user; The identification module comprises: A first generating submodule is used to input the image information into a pre-trained multi-task detection model, and output face scanning information, a face frame image, a gesture frame image, and a correspondence between the face frame image and the gesture frame image of at least one user; A tracking submodule, configured to track the face frame image by using a target tracking technology to generate a face tracking sequence, and to track the gesture frame image by using a target tracking technology according to a correspondence between the face frame image and the gesture frame image to generate a gesture tracking sequence corresponding to the face tracking sequence; The second generating submodule is used to input the gesture tracking sequence into a pre-trained gesture classification model and output the gesture information.
10. The device according to claim 9, wherein: The generation module is specifically used to input user behavior information of at least one user into a pre-trained willingness recognition model, and output the willingness to pay value of each of the users.
11. The device according to claim 9, wherein: The determination module comprises: A judgment submodule, used to judge whether there is a willingness to pay value greater than a set threshold value among the willingness to pay values of at least one user; A first determination submodule, configured to determine a maximum willingness-to-pay value from among the willingness-to-pay values that are greater than the set threshold value if the determination submodule determines that at least one user's willingness-to-pay value has a willingness-to-pay value that is greater than the set threshold value; The second determination submodule is used to determine the user corresponding to the maximum willingness to pay value as the target user.
12. An electronic device comprising: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, enable the electronic device to execute the face-scanning payment method described in any one of claims 1 to 8.
13. A computer-readable storage medium, the computer-readable storage medium comprising a stored program, wherein: When the program is running, the electronic device where the computer-readable storage medium is located is controlled to execute the face-scanning payment method described in any one of claims 1 to 8.
Citation Information
Cited By
Face scanning payment management method and system
CN119624459A
A face-scanning payment management method and system
CN119624459B
