Face payment method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a face payment method and device, electronic equipment, a storage medium and a program product, and relates to the technical fields of cloud technology, Internet of Things and the like. When a first device initiates a face scanning payment request, a first face image is collected and sent to a server, and a payment result sent by the server is displayed. The payment result is determined based on the judgment result of the first equipment and the first face image, and the judgment result is obtained by judging false swiping of the first equipment based on the payment process video acquired from the second equipment, so that the second equipment is associated equipment of the first equipment. The payment process video is acquired by the second equipment in response to a target triggering operation associated with the target order in the first equipment, and the triggering time of the target triggering operation is before the initiation time of the face scanning payment request; the judgment result represents whether the first face image is the face image of the first object initiating the face scanning payment request or not. The safety and reliability of face scanning payment can be improved.

Description

Technical Field

[0001] The present application relates to technical fields such as cloud technology and the Internet of Things, and the present application relates to a facial payment method, device, electronic device, storage medium and program product. Background Art

[0002] Facial payment is a payment method based on facial recognition technology. Facial recognition technology belongs to biometric recognition technology, which distinguishes individuals based on the facial features of organisms (generally referring to humans). In some facial payment application scenarios, some payment devices can be used to scan the face of self-service cashiers (such as face-scanning self-service cashiers) to jointly realize the payment process.

[0003] In current face-scanning payment scenarios, human costs are usually required for on-duty operation. If there is no one on duty, it is easy to lose orders, swipe by mistake, or escape from the order. For example, user A enters the product at the face-scanning self-service checkout counter, and then clicks on face-scanning payment. At this time, user B in front of the camera may be mistakenly identified and user B may be deducted, resulting in a payment error. No effective solution has been proposed for the above problem. Therefore, there is an urgent need for a face-based payment method to solve the above problem. Summary of the invention

[0004] The present application provides a facial payment method, device, electronic device, storage medium and program product, which can effectively solve the problem of mistakenly swiping the face of another object to pay when swiping the face to pay, and ensure the security and reliability of face payment. The technical solution is as follows:

[0005] In one aspect, a facial payment method is provided, the method being performed by a first device; the first device being any one of a plurality of payment devices deployed in a target space, each payment device being an associated device of another payment device in the target space, and an associated device of a payment device being used to collect a payment process video of the payment device;

[0006] The method comprises:

[0007] In response to detecting that a face recognition payment request for a target order is initiated through the first device, a facial image is captured through an image capture device of the first device, and the captured first facial image is sent to a server;

[0008] Display the payment result sent by the server;

[0009] The payment result is determined based on a determination result of the first device and the first facial image, and the determination result is obtained by determining a false swipe of the first device based on a payment process video obtained from a second device, and the second device is an associated device of the first device;

[0010] The payment process video is obtained by the second device performing video acquisition in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-scanning payment request, and the target trigger operation is an operation associated with the target order;

[0011] The determination result indicates whether the first facial image is a facial image of a first object, where the first object is the object that initiates the face-swiping payment request.

[0012] On the other hand, a facial payment method is provided, the method being executed by a server; the server establishing communication connections with a first device and a second device respectively; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; the second device is an associated device of the first device;

[0013] The method comprises:

[0014] receiving a payment process video sent by a second device, and receiving a first facial image sent by a first device in response to a face payment request for a target order;

[0015] The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order;

[0016] Performing a false swipe determination on the first device based on the payment process video to obtain a determination result, wherein the determination result indicates whether the first facial image is a facial image of a first object, the first object being the object that initiates a facial swipe payment request for a target order;

[0017] Determining the payment result based on the determination result and the first facial image;

[0018] The payment result is sent to the first device.

[0019] On the other hand, a facial payment method is provided, the method being performed by a second device; the second device is an associated device of a first device, the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device;

[0020] The method comprises:

[0021] Based on the communication connection with the first device, acquiring device information of the first device, and based on the device information of the first device and the second device, sending a binding instruction to the server, wherein the binding instruction indicates that the first device and the second device are bound as associated devices;

[0022] In response to a target triggering operation on a target order in the first device, performing video acquisition through an image acquisition device of the second device, and sending the acquired payment process video to a server;

[0023] Among them, the triggering time of the target triggering operation is before the initiation time of the face-scanning payment request; the face-scanning payment request is a request initiated by the first device to perform face-scanning payment for the target order, and the target triggering operation is an operation associated with the target order.

[0024] In one possible implementation, the method further includes:

[0025] In response to receiving a stop acquisition instruction sent by the server, the video acquisition of the first device is stopped, and the payment process video of the first device is stopped from being sent to the server.

[0026] On the other hand, a facial payment device is provided, the device being executed by a first device; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device;

[0027] The device comprises:

[0028] A facial image sending module, configured to, in response to detecting that a facial payment request for a target order is initiated through the first device, collect a facial image through an image collection device of the first device, and send the collected first facial image to a server;

[0029] The payment result display module is used to display the payment result sent by the server;

[0030] The payment result is determined based on a determination result of the first device and the first facial image, and the determination result is obtained by determining a false swipe of the first device based on a payment process video obtained from a second device, and the second device is an associated device of the first device;

[0031] The payment process video is obtained by the second device performing video acquisition in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-scanning payment request, and the target trigger operation is an operation associated with the target order;

[0032] The determination result indicates whether the first facial image is a facial image of a first object, where the first object is the object that initiates the face-swiping payment request.

[0033] In a possible implementation, the payment result display module is used to:

[0034] If the determination result is a first determination result, a prompt message sent by the server is displayed, the prompt message prompting to re-initiate a face recognition payment request for the target order, and the first determination result indicates that the first facial image is not a facial image of the first object;

[0035] The payment result corresponding to the face recognition payment request re-initiated based on the prompt message is displayed. In a possible implementation, the device further includes a determination result determination module, and the determination result determination module, when determining the determination result, includes:

[0036] A receiving unit, configured to receive the payment process video sent by the second device;

[0037] a posture analysis unit, configured to perform posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, wherein the posture information indicates whether an action of face scanning failure is performed, and the at least one object includes at least a first object;

[0038] The determination unit is configured to obtain the determination result based on the posture information of the at least one object.

[0039] In a possible implementation, the posture analysis unit is used to:

[0040] Identify a first object among various objects included in the payment process video, and perform joint point detection on the first object to obtain various joint points of the first object;

[0041] determining, based on relative positions between joints of the first object, posture information of the first object, the posture information indicating whether an action of performing face scanning failure is performed;

[0042] The determination unit is used for:

[0043] If the posture information of the first object is the first posture information, the determination result is determined to be the first determination result, the first posture information indicates that the first object has failed to perform a face scanning action, and the first determination result indicates that the first facial image is not the facial image of the first object.

[0044] In a possible implementation, the posture analysis unit is used for at least one of the following:

[0045] Based on the angle between the line connecting the first joint point of the first object and the line connecting the hip joint point, determining whether the first object is bending over, and obtaining the posture information based on the bending over determination result; the action of failed face scanning includes bending over, and the first joint point includes at least one of a shoulder joint point or a knee joint point;

[0046] Based on the rotation angle corresponding to the line connecting the second joint points of the first object, determining whether the first object has turned around, and obtaining the posture information based on the turning determination result; the face scanning failed action includes turning around, and the second joint point includes at least one of a shoulder joint point or a hip joint point;

[0047] Based on the moving distance corresponding to the third joint point of the first object, determine whether the first object has moved, and obtain the posture information based on the movement determination result; the action of failed face scanning includes movement, and the third joint point includes a torso joint point.

[0048] In a possible implementation, the determination result is obtained by determining whether the first device has been swiped incorrectly based on the payment process video and the first facial image;

[0049] The determination unit is used for:

[0050] Acquiring a second facial image from a first device, and determining an image similarity between the first facial image and the second facial image;

[0051] The first facial image is an image captured by a facial recognition page displayed by the first device when a facial payment request is initiated; the second facial image is an image captured after the first facial image is captured;

[0052] The determination result is obtained based on the posture information and the image similarity of the at least one object.

[0053] In a possible implementation, the first device corresponds to a plurality of associated devices in the target area, and the second device is any one of the plurality of associated devices of the first device; each associated device is used to capture a payment process video of the first device from a perspective corresponding to the associated device;

[0054] The determination result determination module, when determining the determination result, is further used to:

[0055] Determining a panoramic video corresponding to the first device based on the payment process videos respectively corresponding to the multiple associated devices;

[0056] Performing a false swipe determination on the first device based on the panoramic video to obtain the determination result.

[0057] In one possible implementation, the target trigger operation includes any one of an operation of selecting a face-scanning payment method or an operation of entering an item; the payment process video includes a video screen of a process of entering item information in the first device, and a process of initiating a face-scanning payment request to capture a facial image.

[0058] On the other hand, a facial payment device is provided, the device is executed by a server; the server establishes communication connections with a first device and a second device respectively; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; the second device is an associated device of the first device;

[0059] The device comprises:

[0060] A video receiving module, used for receiving a payment process video sent by a second device;

[0061] A facial image receiving module, configured to receive a first facial image sent by the first device in response to a facial payment request for a target order;

[0062] The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order;

[0063] a false swipe determination module, configured to determine a false swipe of the first device based on the payment process video to obtain a determination result, wherein the determination result indicates whether the first facial image is a facial image of a first object, the first object being an object that initiates a face-swiping payment request for a target order;

[0064] a payment result determination module, configured to determine the payment result based on the determination result and the first facial image;

[0065] A sending module is used to send the payment result to the first device.

[0066] In a possible implementation, the false-swipe determination module is configured to:

[0067] Performing posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, wherein the posture information indicates whether an action of face scanning failure is performed, and the at least one object includes at least a first object;

[0068] The determination result is obtained based on the posture information of the at least one object.

[0069] In a possible implementation, the facial image receiving module is used to:

[0070] receiving a facial image sent by at least one payment device, the at least one payment device including the first device;

[0071] Determine a first device associated with the second device based on the associated stored device information and the device information of the second device, and determine a first facial image sent by the first device from the facial images sent by each payment device;

[0072] The false-brush determination module is used to:

[0073] Based on the payment process video and the first facial image, a mistaken swipe determination is performed on the first device to obtain a determination result.

[0074] In a possible implementation, the false-swipe determination module is configured to:

[0075] Acquire a second facial image from the first device, and determine an image similarity between the first facial image and the second facial image;

[0076] The first facial image is an image captured by a facial recognition page displayed by the first device when a facial payment request is initiated; the second facial image is an image captured after the first facial image is captured;

[0077] The determination result is obtained based on the posture information and the image similarity of the at least one object.

[0078] In a possible implementation, the device further includes:

[0079] a binding module, configured to receive a binding instruction sent by at least one of the first device or the second device;

[0080] an association storage module, configured to bind the first device and the second device as associated devices based on the binding instruction sent by the at least one device, and to associate and store device information of the first device and the second device;

[0081] The binding instruction sent by the at least one device carries device information of the first device and the second device, and the device information of the first device and the second device is obtained based on the communication connection between the first device and the second device.

[0082] In a possible implementation, the video receiving module is used to:

[0083] In response to receiving a video capture instruction sent by the first device, forwarding the video capture instruction to the second device, and receiving the payment process frequency returned by the second device based on the video capture instruction, wherein the video capture instruction is triggered by a target trigger operation on a target order in the first device.

[0084] In a possible implementation, the device further includes:

[0085] a stop instruction sending module, configured to send a stop acquisition instruction to the second device in response to detecting a stop acquisition event corresponding to the target order, so that the second device stops capturing video of the first device and stops sending the payment process video of the first device based on the stop acquisition instruction;

[0086] The stop acquisition event includes at least one of the following:

[0087] Determine a second determination result based on the payment process video sent by the second device;

[0088] receiving a first facial image sent by a first device;

[0089] determining that the payment is successful based on the second determination result and the first facial image;

[0090] A third facial image re-captured and sent by the first device based on the prompt message is received.

[0091] On the other hand, a facial payment device is provided, the device being executed by a second device; the second device being an associated device of a first device, the first device being any one of a plurality of payment devices deployed in a target space, each payment device being an associated device of another payment device in the target space, and an associated device of a payment device being used to collect a payment process video of the payment device;

[0092] The device comprises:

[0093] a binding instruction sending module, configured to obtain device information of the first device based on a communication connection with the first device, and send a binding instruction to a server based on the device information of the first device and the second device, wherein the binding instruction indicates that the first device and the second device are bound as associated devices;

[0094] a video sending module, configured to respond to a target triggering operation on a target order in the first device, to perform video acquisition through an image acquisition device of the second device, and to send the acquired payment process video to the server;

[0095] Among them, the triggering time of the target triggering operation is before the initiation time of the face-scanning payment request; the face-scanning payment request is a request initiated by the first device to perform face-scanning payment for the target order, and the target triggering operation is an operation associated with the target order.

[0096] In a possible implementation, the device further includes:

[0097] The stop instruction sending module is used to stop capturing video of the first device and stop sending the payment process video of the first device to the server in response to receiving the stop capturing instruction sent by the server.

[0098] On the other hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-mentioned facial payment method.

[0099] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the facial payment method described above is implemented.

[0100] On the other hand, a computer program product is provided, including a computer program, which implements the above-mentioned facial payment method when executed by a processor.

[0101] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0102] The facial payment method provided in the present application deploys multiple payment devices in the target space, and deploys each payment device as an associated device of another payment device. The associated device of one payment device is used to collect the payment process video of the payment device; the first device, as any one of the multiple payment devices, can collect and send the first facial image to the server when initiating a face payment request, and display the payment result sent by the server.

[0103] Among them, the payment result is determined based on the judgment result of the first device and the first facial image. Since the judgment result is obtained by judging the first device as being mistakenly swiped based on the payment process video obtained from the second device, the second device is an associated device of the first device. In addition, the payment process video is collected by the second device in response to the target trigger operation associated with the target order in the first device, because the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request. The judgment result represents whether the first facial image is the facial image of the first object that initiates the face-swiping payment request. Therefore, the payment process video may include a video screen of the process of initiating the face-swiping payment request to collect facial images; based on the payment process video, it can be effectively determined whether the first device mistakenly swipes the facial image of other objects, which effectively solves the problem of mistakenly swiping the face of other objects for payment during face-swiping payment, and ensures the security and reliability of face-swiping payment. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0105] Figure 1 A schematic diagram of an implementation environment of a facial payment method provided in an embodiment of the present application;

[0106] Figure 2 A schematic diagram of a facial payment method provided in an embodiment of the present application;

[0107] Figure 3 A schematic diagram of a payment device provided in an embodiment of the present application;

[0108] Figure 4 A schematic diagram of a device deployment method provided in an embodiment of the present application;

[0109] Figure 5 A schematic diagram of a device deployment method provided in an embodiment of the present application;

[0110] Figure 6 A schematic diagram of a method for determining a determination result provided in an embodiment of the present application;

[0111] Figure 7 A schematic diagram of various joint points of an object provided in an embodiment of the present application;

[0112] Figure 8 A signaling interaction diagram of a facial payment method provided in an embodiment of the present application;

[0113] Fig. 9 A schematic diagram of the overall process of facial payment provided in an embodiment of the present application;

[0114] Fig.10 A schematic diagram of a facial payment system architecture provided in an embodiment of the present application;

[0115] Fig.11 A schematic diagram of the structure of a facial payment device provided in an embodiment of the present application;

[0116] Fig.12 A schematic diagram of the structure of a facial payment device provided in an embodiment of the present application;

[0117] Fig.13 A schematic diagram of the structure of a facial payment device provided in an embodiment of the present application;

[0118] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0119] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0120] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. The terms "include" and "comprise" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, and operations, but do not exclude the implementation as other features, information, data, steps, operations, etc. supported by the technical field.

[0121] It can be understood that in the specific implementation of the present application, any object-related data such as facial images, payment process videos, face payment requests, target orders, entered item information, objects or object joints in the payment process videos, etc., when the above embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the object, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0122] Figure 1 A schematic diagram of the implementation environment of a facial payment method provided in this application. Figure 1 As shown, the implementation environment includes: a server 101, a first device 102 and a second device 103. The server 101 can be a background server that provides face-swiping payment services.

[0123] The first device 102 and the second device 103 are payment devices that support the face-swiping payment function. In this application, the first device 102 is any one of the multiple payment devices deployed in the target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect the payment process video of the payment device; the second device 103 is an associated device of the first device 102.

[0124] In a possible scenario, the first device 102 may detect that a face-scanning payment request for a target order is initiated through the first device. When initiating the face-scanning payment request, the first device 102 may collect and send a first facial image to the server, and display the payment result sent by the server. The second device 103 may collect a payment process video of the first device 102, and send the payment process video to the server 101. The server 101 may perform face-scanning judgment on the first device based on the payment process video, and determine the payment result based on the judgment result and the first facial image. The server 101 sends the payment result to the first device 102. If the judgment result indicates that the first device 102 may have mistakenly swiped an object other than the payment object, the server 101 may send a prompt message to the first device 102. The first device 102 receives and displays the prompt message to prompt the user to re-initiate the face-scanning payment request for the target order.

[0125] It should be noted that the first device 102 and the second device 103 can be any payment device that supports the face payment function. For example, cash register equipment used for face payment in various scenarios such as medical care, supermarkets, banks, gas stations, convenience stores, and restaurants, such as supermarket cash register equipment, medical system payment equipment, gas station refueling equipment, etc.; it can also be an intelligent robot with face payment function. For example, a self-service cash register terminal can store the unit price, inventory information, and sales list of goods, and can support functions such as printing sales receipts, issuing invoices, and obtaining information about goods by scanning the barcode of goods (for example, the unit price of goods, purchase quantity, inventory information, etc.). Exemplarily, intelligent robots can be used as intelligent interactive devices used in places such as commodity retail enterprises, restaurants, and bank counters that handle counter business. This application is only illustrated by the examples listed above, but the payment device is not specifically limited.

[0126] The server 101 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, and big data and artificial intelligence platforms. The first device 102, the second device 103 and the server 101 may be directly or indirectly connected via wired or wireless communications, and may also be determined based on actual application scenario requirements, which is not limited here.

[0127] Figure 2 A schematic diagram of a facial payment method provided in an embodiment of the present application. The execution subject of the method may be a first device. Figure 2 As shown, the facial payment method includes the following steps:

[0128] Step 201: In response to detecting that a face-scanning payment request for a target order is initiated through the first device, the first device captures a facial image through an image capture device of the first device, and sends the captured first facial image to a server.

[0129] Exemplarily, the face payment request is used to request payment by face payment for a target order. Exemplarily, the target order may be an order initiated in the first device that needs to be paid. The target order may include information of multiple items to be paid, such as the name, quantity, price, etc. of the items.

[0130] In one possible manner, the first device may make a corresponding change in the page display in response to the face payment request. Exemplarily, step 201 may include step S1: Step S1, the first device displays a facial recognition page in response to detecting a face payment request for a target order initiated by the first device. The facial recognition page includes a face prompt area, and the collected first facial image is displayed in the face prompt area. The face prompt area can be used to prompt the face to be collected to appear in the face prompt area for identification; thereby facilitating the first device to collect facial images. The face prompt area can be an area enclosed by a circular or rectangular frame located in the center of the facial recognition page.

[0131] In this step, when a payment initiation operation for a target order is detected, the first device displays the facial recognition page; in the facial recognition page, facial images are captured by an image capture device, and the captured first facial image is sent to the server.

[0132] Among them, the payment initiation operation may be an operation for triggering face-scanning payment for the target order. For example, if face-scanning payment has been selected before the item information in the order is entered, the payment initiation operation may be an operation for triggering the completion of entry or a payment operation, such as clicking the entry completion button or the payment button in the order page to initiate face-scanning payment. For another example, if face-scanning payment is not selected before the item information is entered, when it is detected that the click to pay button or the entry completion button in the order page is clicked, the first device may display multiple payment methods to be selected, and the payment initiation operation may also be an operation for selecting face-scanning payment from the multiple payment methods.

[0133] In one example, the first device can execute the first payment operation process to complete the order payment. In the first payment operation process, the payment method of face recognition payment is first selected, and then the item information is entered and the face recognition payment is made. The process specifically includes: the first device displays multiple payment methods to be selected, such as face recognition payment, mobile phone scanning payment, etc.; when it is detected that the payment method of face recognition payment is selected, the item entry page is displayed; the item is scanned based on the item scanning area of ​​the first device, and the name, quantity, price and other information of the entered item are displayed, so as to enter the information of each item in the target order into the first device; when it is detected that the entry completion button or payment button in the item entry page is clicked, the face recognition page is directly displayed.

[0134] In another example, the first device can also execute the second payment operation process to realize order payment. In the second payment operation process, the item information is entered first, and then the payment method of face payment is selected and paid by face payment. The process specifically includes: the first device displays the item entry page, scans the item based on the item scanning area of ​​the first device, and displays the name, quantity, price and other information of the entered item to enter the information of each item in the target order into the first device; when it is detected that the entry completion button or the payment button is clicked in the item entry page, multiple payment methods to be selected can be displayed, and when the face payment operation is selected among the multiple payment methods, the face recognition page can be directly displayed.

[0135] Step 202: The first device displays the payment result sent by the server.

[0136] The payment result is the result of face payment for the target order based on the first facial image. For example, the payment result may include payment success or payment failure.

[0137] In the present application, the payment result is determined based on the determination result of the first device and the first facial image, wherein the determination result is obtained by determining the mis-swipe of the first device based on the payment process video obtained from the second device.

[0138] The payment process video is obtained by the second device performing video capture in response to the target trigger operation in the first device. The trigger time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order.

[0139] The determination result indicates whether the first facial image is a facial image of a first object. The first object is the object that initiates the facial payment request, that is, the object that initiates the facial payment request for the target order in step 201, and is also the payment object corresponding to the target order. The mis-swipe determination can be to determine whether the first device mistakenly swipes an object other than the first object, that is, to determine whether the first facial image is a facial image of another object that was mistakenly swiped.

[0140] In some embodiments, if the determination result indicates that the first facial image is a facial image of the first object, that is, the first device did not mistakenly swipe other objects, the server may perform face-swiping payment based on the first facial image and return the result of the face-swiping payment. Exemplarily, step 202 may include: the first device displays the result of face-swiping payment based on the first facial image. The result is the result of face-swiping payment when the determination result is the second determination result, and the result may be payment success or payment failure. The second determination result indicates that the first facial image is a facial image of the first object. For example, if the verification based on the first facial image passes and the corresponding resource transfer process is successfully executed, the payment success is displayed. Otherwise, if the verification based on the first facial image fails or the resource transfer fails to be successfully executed, such as when the balance in the resource account of the first object is insufficient or there is an abnormality in the resource account, the payment failure is displayed.

[0141] In some embodiments, if the determination result indicates that the first facial image is not the facial image of the first object, that is, the first device mistakenly swipes the facial image of an object other than the first object; the first device may also prompt to re-initiate the facial payment process in order to obtain an accurate facial image and then make the payment. Exemplarily, step 202 may include the following steps 2021-2022:

[0142] Step 2021: If the determination result is the first determination result, the first device displays a prompt message sent by the server; the prompt message prompts to re-initiate a face-scanning payment request for the target order, and the first determination result indicates that the first facial image is not a facial image of the first object.

[0143] Step 2022: The first device displays the payment result corresponding to the face recognition payment request re-initiated based on the prompt message.

[0144] For example, if the result is the first determination, a prompt message may be used to prompt the user to re-initiate the face-scanning payment process so as to obtain the facial image of the first object. For example, the prompt message may be: prompting that the face-scanning payment is abnormal, or that the face-scanning payment fails and the user needs to re-initiate the face-scanning payment process.

[0145] Exemplarily, when the first device detects a face-scanning payment request re-initiated based on the prompt message, the first device may re-capture the facial image based on the image acquisition device based on the re-initiated face-scanning payment request, and send the re-captured facial image to the server. The server may re-determine the mis-scanning of the first device based on the payment process video, and upon obtaining the second determination result, perform face-scanning payment based on the re-captured facial image, and return the result of the face-scanning payment. The first device displays the result of the face-scanning payment returned by the server.

[0146] In step 2022, based on the payment process video, the process of re-initiating the face-swiping payment in the first device can be judged as an erroneous swipe. If there is no erroneous swipe in the process of re-initiating the face-swiping payment, the face-swiping payment is performed based on the re-collected facial image. Otherwise, if there is still an erroneous swipe in the process of re-initiating the face-swiping payment, a prompt message is sent to the first device again to prompt the process of re-initiating the face-swiping payment, or the payment result of the payment failure can be directly displayed.

[0147] In some embodiments, the second device is an associated device of the first device, and the payment process video of the first device is obtained by capturing video of the first device by the second device; the payment process video can be used to record the process of the first object paying for the target order by face recognition on the first device.

[0148] In this application, multiple payment devices can be deployed in the target space; the first device is any one of the multiple payment devices deployed in the target space, each payment device is an associated device of another payment device in the target space, and the associated device of a payment device is used to collect the payment process video of the payment device. That is, each payment device has an associated device corresponding to the target space, and the associated device can be used to provide the payment process video of the payment device.

[0149] Each payment device may be equipped with an image acquisition device, and the associated device may collect video of the payment device through the image acquisition device to obtain a payment process video of the payment device. The payment process video of a payment device may be used to record the process of the subject performing face payment on the payment device.

[0150] For example, the image acquisition device of each payment device can acquire images within a certain range of the target space. Figure 3As shown, the payment device is pre-configured with a display screen, an image acquisition device, a code scanning area for scanning and entering item information, etc. For example, the image acquisition device can be a 3D camera; if the image acquisition device is located above the display screen, the image acquisition device can be used to capture the facial image of the object within the spatial acquisition range of the display screen; for example, a 3D image of the face of an object standing facing the display screen is captured. Of course, when the payment device is used as an associated device, it can also capture the video of the corresponding spatial acquisition range.

[0151] In this application, the spatial collection range corresponding to the associated device of a payment device overlaps with the spatial collection range corresponding to the payment device. For example, after placing various payment devices in the target space, if there is an overlap between the spatial collection ranges corresponding to two payment devices, then the two devices can be considered to constitute mutually associated devices.

[0152] It should be noted that the deployment positions of a payment device and its associated devices in the target space have a certain spatial position relationship.

[0153] In one example, if a payment device corresponds to an associated device, the payment device and its associated device can be placed opposite each other in the target space. Figure 4 As shown, device A is placed on the opposite side of device B, so that the image acquisition devices of device A and device B are also placed opposite each other. Device A and device B can respectively capture corresponding payment process videos through their respective image acquisition devices.

[0154] In another example, if a payment device corresponds to multiple associated devices, the spatial collection range corresponding to the payment device overlaps with the spatial collection ranges corresponding to the multiple associated devices. Figure 5 As shown, the associated devices of device C include device D and device E, and device C, device D and device E are respectively deployed at the vertices of a triangle; for device C, device D and device E can respectively capture video of device C from corresponding perspectives; for any device among device D and device E, similarly, the other two devices can capture video of device D or device E from their respective perspectives.

[0155] Below Figure 6 Taking the flowchart shown as an example, how to get the judgment result is introduced:

[0156] Figure 6 1 is a flow chart of a method for determining a determination result provided by an embodiment of the present application. The method can be executed by a server. The method includes the following steps.

[0157] Step 601: The server receives the payment process video sent by the second device.

[0158] The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order. Exemplarily, the payment process video at least includes a video screen of the process of initiating the face-swiping payment request to capture a facial image.

[0159] In one possible implementation, the target triggering operation includes any one of a selection operation of a face-scanning payment method or an item entry operation; the payment process video also includes a video screen of a process of entering item information in the first device.

[0160] In the present application, the target trigger operation is an operation for the target order that occurs before the payment initiation operation. The payment process video may also include a video screen of the item information entry process. The target trigger operation and payment process video are illustrated below through Example 1 and Example 2.

[0161] Example 1: In the first payment operation process, the target triggering operation may be an operation of selecting a face-scanning payment method; or, it may be an operation of entering an item.

[0162] For example, in the first payment operation process, the payment method of face payment is selected first, and then the item information is entered. After the entry is completed, the face payment request is initiated through the payment initiation operation. Before the payment initiation operation, a face payment selection operation is included. The target triggering operation can be a face payment selection operation or an item entry operation before the payment initiation operation. Accordingly, for the first payment operation process, the payment process video may include video screens of the item information entry process and the face payment request initiation process.

[0163] Example 2: In the second payment operation process, the target triggering operation may also be an item entry operation.

[0164] For example, in the second payment operation process, the item information is entered first, and then the payment method of face-scanning payment is selected. The payment initiation operation may be an operation of selecting face-scanning payment from the multiple payment methods. Then the target triggering operation may be an item entry operation before selecting face-scanning payment. Accordingly, for the second payment operation process, the payment process video may include video footage of the item information entry process and the face-scanning payment request initiation process.

[0165] In a possible implementation, step 601 may be implemented through interaction between the first device, the second device, and the server. The interaction process of step 601 may include the following steps 6011-6014:

[0166] Step 6011: When the first device detects a target trigger operation for a target order, the first device sends a video capture instruction to the server;

[0167] Step 6012: The server determines a second device associated with the first device based on the video capture instruction, and sends the video capture instruction to the second device;

[0168] Step 6013: The second device performs video acquisition through the image acquisition device in the second device based on the video acquisition instruction, and sends the acquired payment process video of the first device to the server;

[0169] Step 6014: The server receives the payment process video sent by the second device.

[0170] In one possible example, a long connection may be established between the second device and the server. The second device may start the video acquisition step of the first device upon receiving the video acquisition instruction, and transmit the acquired video stream to the server in the form of an information stream according to a certain transmission period. For example, the transmission period may be 1 second, 0.5 second, 20 milliseconds, etc., which is not limited in this application.

[0171] In the present application, the second device may also be in use when receiving the video capture instruction; for example, a user enters a product on the second device, or triggers a payment initiation operation to initiate a face-scanning payment request, etc. In one possible manner, if the second device processes the video capture instruction and the face-scanning payment request at the same time, the process may include: the second device may perform video capture and facial image capture through an image capture device based on the video capture instruction and the face-scanning payment request, and directly send the captured data including the facial image to the server; and after performing facial blurring processing on the data including the facial image, send it to the server as a payment process video of the first device.

[0172] For example, the second device may have the functions of data uploading and video diversion. Among them, for the function of video diversion, the second device can divide the data collected by the current camera into two paths, and one video stream continues to be transmitted to the server based on the transmission path of face payment. In the second path, the facial information in the video stream can be blurred (for example, mosaic processing), and then the blurred video stream is sent to the server. It should be noted that for the processing flow of the second path, an independent sub-process can be used to execute. For example, two sub-processes are used to execute the step processes corresponding to the two paths respectively.

[0173] Step 602: The server performs posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, where the at least one object includes at least a first object.

[0174] In this step, the payment process video includes multiple video frames of the face-swiping payment process on the first device. The server can perform posture analysis based on the joints of at least one object in the multiple video frames to obtain the posture of the at least one object in each video frame, and identify the movement change process of at least one object based on the posture of the at least one object in each video frame to obtain the posture information of the at least one object. The posture information is used to indicate whether the object performs a failed face-swiping action. The failed face-swiping action can be an action that causes the first device to fail to successfully capture the facial image of the first object.

[0175] In a possible implementation, the at least one object may include other objects besides the first object, and the server may first identify the first object among the objects and perform posture analysis on the first object. Exemplarily, step 602 may include the following steps 6021-6022:

[0176] Step 6021: The server identifies a first object among various objects included in the payment process video, and performs joint point detection on the first object to obtain various joint points of the first object;

[0177] Step 6022: The server determines the posture information of the first object based on the relative positions between the joints of the first object, where the posture information indicates whether the face scanning action has failed.

[0178] Exemplarily, the first object among multiple objects can be identified based on preconfigured payment object identification conditions; for example, the first object includes but is not limited to: an object that interacts with the first device, or is closest to the first device; or an object that is at the top of the object queue corresponding to the first device, etc. For example, a handheld item is placed in the item scanning area of ​​the first device for scanning. For example, the object that performs the item information entry operation in the payment process video is taken as the first object; or, the object that initiates the payment request is taken as the first object; or, the object directly in front of the display screen of the first device can also be taken as the first object; or, the first device can correspond to a queue of objects waiting in line for payment, and the object ranked first in the queue can be taken as the first object. This application only exemplifies the method of identifying the first object in the above manner, but does not limit the identification method.

[0179] Exemplarily, the server may use a neural network model to detect the joint points of an object. The process may include: the server inputs the payment process video into a detection model for detecting the joint points of an object in an image; through the detection model, the joint points of the first object in the payment process video are detected and determined. For example, the first object may be marked as an object to be detected for joint points, and the marked payment process video may be input into the detection model to obtain the joint point positions of the first object in each frame of the video.

[0180] Exemplarily, the server may determine the posture of the first object in each frame of the video based on the position information of the joint points of the first object in each frame of the video, and determine the action performed by the first object based on the posture of the first object in the multiple frames of the video. For example, if the first object is in a bending posture in 100 consecutive video frames, the posture information indicates that the first object has performed a bending action, which is an action that causes the first device to fail to scan the face.

[0181] In another possible implementation, posture information of multiple objects may be identified, and the first device may be judged as being swiped incorrectly based on the posture information of the multiple objects. Exemplarily, step 602 may include: the server identifies each object in the payment process video, and performs joint point detection on each object to obtain each joint point of each object, and determines the posture information of each object based on the relative position between each joint point of each object.

[0182] In another possible implementation, based on a pre-configured object screening condition, objects meeting the condition may be screened out from various objects included in the payment process video, and posture information of the screened objects may be obtained.

[0183] Exemplarily, the action of the failed face scan may include at least one of bending, turning or moving. For different actions, the server may identify the joint points corresponding to the different actions so as to obtain posture information through posture analysis. Accordingly, the implementation method of step 6021 knee may include at least one of the following methods 1 to 3.

[0184] Method 1: based on the angle between the line connecting the first joint points of the first object and the line connecting the hip joint points, determine whether the first object is bending over, and obtain the posture information based on the bending determination result; the action of failed face scanning includes bending over, and the first joint point includes at least one of a shoulder joint point or a knee joint point.

[0185] In an example, Figure 7As shown, the shoulder joint points may include joint points corresponding to the left shoulder and the right shoulder respectively, and the line connecting the shoulder joint points is the line connecting one joint point on each side; correspondingly, the hip joint point may also be the line connecting the joint points corresponding to the left hip and the right hip respectively. In the case of normal standing, the line corresponding to the shoulder joint points of the human body tends to be parallel to the line corresponding to the hip joint points, and the angle corresponding to the two lines is small, tending to 0 or being within the first threshold range close to 0 degrees. In addition, the angle between the line connecting the knee joint points of the human body and the line connecting the hip joint points is also small. In the case of bending over, both angles will become larger. If the angle between the line connecting the first joint point and the line connecting the hip joint points exceeds the first threshold range, it is judged that the first object has performed a bending action.

[0186] Method 2: based on the rotation angle corresponding to the line connecting the second joint points of the first object, determine whether the first object has turned around, and obtain the posture information based on the turning determination result; the action of failed face scanning includes turning around, and the second joint point includes at least one of a shoulder joint point or a hip joint point.

[0187] In an example, Figure 7 As shown, in the case of turning, the position of the line connecting the shoulder joint points in the continuous multi-frame video images is rotating and changing, and the hip joint points are also rotating and changing. Therefore, if it is detected that the second joint point presents a certain rotation angle in the continuous multi-frame video images, it can also be determined that the first object has performed a turning action. The same is true for the hip joint points.

[0188] Method three: based on the movement distance corresponding to the third joint point of the first object, determine whether the first object has moved, and obtain the posture information based on the movement determination result; the action of the failed face scanning includes movement, and the third joint point includes a torso joint point.

[0189] The trunk joints may include but are not limited to: at least one of a spine joint, a shoulder joint, a hip joint or a limb joint. If it is detected that the position of the third joint continues to change in a continuous multi-frame video, it can be determined that the first object has moved.

[0190] It should be noted that this application only uses the above examples to illustrate how to use joint points to identify postures or actions. In addition, this application only uses bending, turning, moving and other actions to illustrate the actions of face scanning failure. However, this application does not specifically limit how to identify postures or actions, and what actions are involved in the face scanning failure.

[0191] Step 603: The server obtains the determination result based on the posture information of the at least one object.

[0192] In this step, the server may combine the posture information of the first object to make a false swipe determination on the first device. In a possible implementation, step 603 may include: if the posture information of the first object is the first posture information, determining that the determination result is the first determination result, the first posture information indicates that the first object has failed to perform a face swiping action, and the first determination result indicates that the first facial image is not the facial image of the first object.

[0193] The first posture information may include but is not limited to at least one of posture information indicating bending, posture information indicating turning, or posture information indicating moving. Accordingly, the server determines that the first device has been swiped by mistake, that is, obtains a first determination result.

[0194] Exemplarily, step 603 further includes: if the posture information of the first object is second posture information, determining the determination result as a second determination result. The second posture information indicates that the first object has not performed a failed face scanning action, and the second determination result indicates that the first facial image is a facial image of the first object.

[0195] In one possible implementation, the first device corresponds to multiple associated devices in the target area, and the second device is any one of the multiple associated devices of the first device; each associated device is used to collect the payment process video of the first device from the perspective corresponding to the associated device. In the present application, the payment process videos corresponding to multiple associated devices can also be combined to implement the judgment of the mis-swipe of the first device to obtain the judgment result. Exemplarily, taking the server as the execution subject as an example, the method for determining the judgment result may include the following steps 701-702:

[0196] Step 701: The server determines the panoramic video corresponding to the first device based on the payment process videos corresponding to the multiple associated devices respectively.

[0197] Step 702: The server determines whether the first device has been swiped incorrectly based on the panoramic video to obtain a determination result.

[0198] Exemplarily, each associated device may perform video capture on the first device from a corresponding viewing angle, wherein the viewing angle corresponding to each associated device may be a capture range when the image capture device of the associated device captures the first device.

[0199] Exemplarily, the server may send video capture instructions to multiple associated devices at the same time, based on which the payment process videos respectively sent by the multiple associated devices may be videos captured from the same starting time. The server may match the video frames with the same timestamp in each payment process video one by one; and the server performs panoramic stitching on multiple frames of video frames with the same timestamp or in the same time period, thereby obtaining a panoramic video of the first device. Based on the panoramic video, the posture information of at least one object can be identified from the video frame with a 360° panoramic perspective, and the determination result can be obtained based on at least one posture information. The implementation method of performing posture information recognition based on the panoramic video and obtaining the determination result is the same as the above steps 602-603, and will not be repeated here.

[0200] For example, Figure 5 As shown, for device C, device D and device E are located in front of the left and right of device C, respectively. Then device D can capture the payment process video of device C from the perspective of the left front, and device E can capture the payment process video of device C from the perspective of the right front. Based on this, the payment process videos of device C in the right front and left front can be obtained; the two videos are stitched together frame by frame to obtain a 360° video of device C. Based on the 360° panoramic video in the panoramic video, the posture information of the object is identified, and judgment is made based on the identified posture information, which can improve the accuracy of the judgment result, thereby ensuring the security and reliability of the payment process.

[0201] In some embodiments, the facial image collected by the first device and the payment process video may be combined to determine the mis-swipe; that is, the determination result is obtained by determining the mis-swipe of the first device based on the payment process video and the first facial image. Exemplarily, the determination method of the determination result may include the following steps A1-A3:

[0202] Step A1: The server performs posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, where the at least one object includes at least a first object.

[0203] The implementation of step A1 is similar to that of step 602 and will not be described in detail here.

[0204] Step A2: The server obtains a second facial image from the first device, and determines an image similarity between the first facial image and the second facial image.

[0205] Among them, the first facial image is an image captured by the facial recognition page displayed by the first device when the face payment request is initiated; the second facial image is an image captured after the first facial image is captured.

[0206] For example, the first device can continuously collect facial images and transmit the facial images to the server in real time. The first facial image can be the facial image collected when the facial recognition page is started; the second facial image can be the facial image collected after the first facial image is started and sent; the server can receive the second facial image at a preset time after receiving the first facial image; for example, the facial image received 1 second or 2 seconds after receiving the first facial image. For another example, the server can also send an instruction to control the first device to collect the second facial image again. The process may include: the server can send a collection instruction to the first device to re-collect the facial image; the first device returns the second facial image collected again to the server based on the collection instruction. The server obtains the second facial image.

[0207] Exemplarily, the higher the image similarity, the greater the possibility that the first face image and the second face are from the same object; the object collected has not changed during the facial image collection process, and the possibility of mistakenly scanning other objects is smaller. The server can use computer vision technology to determine the image similarity. In one example, first, the server can preprocess the data of the first face image and the second face image; for example, use a computer vision library such as OpenCV to convert the two face images into digital matrices. Secondly, the server can use computer vision technology to extract features from the two face images; for example, obtain data of dimensions such as color histograms, local binary patterns, SIFT, etc. corresponding to each face image, and extract feature vectors based on the obtained multi-dimensional data. Then, the feature vectors of the two face images can be used to calculate the similarity or vector distance between the two feature vectors, such as Euclidean distance, cosine similarity, histogram-based similarity, etc., to obtain the image similarity.

[0208] In one example, when the first device jumps to the facial recognition page based on the face payment request, facial image acquisition can be continued during the display of the facial recognition page, and the obtained multiple facial images can be continuously sent to the server. For example, the multiple facial images are continuously transmitted in the form of image data streams. The server can perform similarity calculations on the multiple facial images to analyze the similarities between the multiple facial images. Furthermore, the server can also classify the multiple facial images according to the similarity between every two facial images in the multiple facial images, and classify the multiple facial images with a similarity higher than a specified threshold into one category. Of course, a clustering algorithm can also be used to cluster multiple facial images with high similarity into one category.

[0209] In one possible manner, the server may execute step A2 based on the posture information of the first object. For example, if the posture information of the first object is the first posture information, the process of step A2 is executed. For another example, the posture information of the first object may include the action corresponding to the first object in each frame of the video and the action probability belonging to the action. If the action probability of the action belonging to the failed face scanning exceeds the target probability threshold, the server executes the process of step A2.

[0210] Step A3: The server obtains the determination result based on the posture information and image similarity of the at least one object.

[0211] Exemplarily, if the posture information of the first object is the first posture information and the image similarity does not exceed the target similarity threshold, the determination result is determined to be the first determination result. If the posture information of the first object is the second posture information and the image similarity exceeds the target similarity threshold, the determination result is determined to be the second determination result. In addition, if the posture information of the first object is the first posture information, or the image similarity does not exceed the target similarity threshold, the determination result may also be determined to be the first determination result.

[0212] In one possible way, if the posture information includes the action of the first object and the action probability of the action. If the action belongs to an action that fails to scan the face, such as bending, turning, etc., the server can determine the score of the first device based on the action probability and the target similarity, and the score represents the possibility of the first device mistakenly scanning other objects. The higher the score, the higher the possibility of mistaken scanning. For example, the higher the action probability, the lower the corresponding action score; the higher the similarity, the higher the corresponding image score; the action score and the image score can be weighted to obtain the score of the first device.

[0213] In some embodiments, the second device may stop sending the payment process video of the first device at any time based on the instruction of the server. Exemplarily, the implementation method of the server controlling the second device to stop sending the payment process video may include the following steps 604:

[0214] Step 604: In response to detecting a stop collection event corresponding to the target order, the server sends a stop collection instruction to the second device.

[0215] Exemplarily, the second device may stop capturing video of the first device and stop sending the payment process video of the first device to the server in response to a stop capturing instruction sent by the server.

[0216] The stop acquisition event may be an event indicating to stop acquiring the payment process video. Exemplarily, the stop acquisition event may include but is not limited to at least one of the following:

[0217] Determine a second determination result based on the payment process video sent by the second device;

[0218] receiving a first facial image sent by a first device;

[0219] Determining that the payment is successful based on the second determination result and the first facial image;

[0220] A third facial image re-captured and sent by the first device based on the prompt message is received.

[0221] Exemplarily, based on several possible forms of the stop acquisition event, the implementation of step 604 may correspond to at least one of the following modes 1 to 4.

[0222] Method 1: When the server obtains the second determination result based on the payment process video sent by the second device, the server may control the second device to stop collecting and stop sending the payment process video.

[0223] The server determines that the first device has not mistakenly swiped other objects, that is, the first facial image is the facial image of the first object. At this time, there is no need to perform video acquisition, but the face payment process is performed based on the received first facial image.

[0224] Method 2: When the server receives the first facial image sent by the first device, the server can control the second device to stop collecting and stop sending the payment process video.

[0225] When the server receives the first facial image, it controls the second device to stop collecting the image, and the payment process video also includes a video screen of the collection process of obtaining the first facial image.

[0226] It should be noted that when the server receives the first facial image, the payment process video collected and sent by the second device may include the collection process of the first facial image collected by the first device. Based on the payment process video, it is possible to identify whether the first device mistakenly swipes the facial image of another object during the collection process, thereby obtaining a determination result. Therefore, the second device can stop video collection.

[0227] Mode 3: When the server determines that the payment is successful based on the second determination result and the first facial image, the server may control the second device to stop collecting and stop sending the payment process video.

[0228] Mode 4: When the server receives the third facial image that is recaptured and sent by the first device based on the prompt message, the server may control the second device to stop capturing and stop sending the payment process video.

[0229] Among them, the second device can continue to capture video in response to the target trigger event of the first device. If the first device receives a prompt message sent by the server, the first device can display the prompt message to prompt the re-initiation of the face-swiping payment request. The first device can re-capture the third facial image based on the re-initiated face-swiping payment request, and send the third facial image to the server. When the server receives the third facial image, the payment process video sent by the second device at this time includes the acquisition process of the first device re-capturing the third facial image. Based on the payment process video, it is possible to identify whether the first device mistakenly swipes the facial image of other objects during the re-capture process, thereby obtaining a judgment result. Therefore, the second device can stop video capture.

[0230] It should be noted that the above-mentioned methods 1 to 4 can also be used in combination. For example, the server can be implemented in combination with methods 1 and 2; accordingly, step 604 includes: when the server receives the first facial image sent by the first device, the server makes a false swipe judgment based on the payment process video sent by the second device, and when the second judgment result is obtained, controls the second device to stop collecting and stop sending the payment process video. Among them, when the server receives the first facial image, the payment process video received at this time includes the collection process of the first facial image by the first device; if it is determined based on the payment process video that the first device does not have a false swipe, the second device can stop collecting.

[0231] The facial payment method provided in the present application deploys multiple payment devices in the target space, and deploys each payment device as an associated device of another payment device. The associated device of one payment device is used to collect the payment process video of the payment device; the first device, as any one of the multiple payment devices, can collect and send the first facial image to the server when initiating a face payment request, and display the payment result sent by the server.

[0232] Among them, the payment result is determined based on the judgment result of the first device and the first facial image. Since the judgment result is obtained by judging the first device as being mistakenly swiped based on the payment process video obtained from the second device, the second device is an associated device of the first device. In addition, the payment process video is collected by the second device in response to the target trigger operation associated with the target order in the first device, because the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request. The judgment result represents whether the first facial image is the facial image of the first object that initiates the face-swiping payment request. Therefore, the payment process video may include a video screen of the process of initiating the face-swiping payment request to collect facial images; based on the payment process video, it can be effectively determined whether the first device mistakenly swipes the facial image of other objects, which effectively solves the problem of mistakenly swiping the face of other objects for payment during face-swiping payment, and ensures the security and reliability of face-swiping payment.

[0233] Figure 8 A signaling interaction diagram of a facial payment method provided in an embodiment of the present application. Figure 8 As shown, the method can be interactively executed by the first device, the server, and the second device. Figure 8 As shown, the method comprises the following steps:

[0234] Step 801: In response to a target trigger operation on a target order in the first device, the second device performs video acquisition through an image acquisition device of the second device, and sends the acquired payment process video to a server;

[0235] Among them, the trigger time of the target trigger operation is before the initiation time of the face-scanning payment request; the face-scanning payment request is a request for face-scanning payment for the target order initiated by the first device, and the target trigger operation is an operation associated with the target order.

[0236] In one possible manner, the second device may start collecting data based on the instruction sent by the server. The process may include steps E1-E4:

[0237] Step E1: In response to detecting a target triggering operation, the first device sends a video capture instruction to the server;

[0238] Step E2: In response to receiving the video capture instruction sent by the first device, the server forwards the video capture instruction to the second device;

[0239] Step E3: The second device receives the video acquisition instruction sent by the server, performs video acquisition through the image acquisition device of the second device based on the video acquisition instruction, and sends the collected payment process video to the server;

[0240] Step E4: The server receives the payment process video returned by the second device based on the video capture instruction, wherein the video capture instruction is triggered by the target trigger operation of the target order in the first device. Step E4 is the step in which the server receives the payment process video sent by the second device in step 803 described below.

[0241] In a possible manner, before step 801, the second device may be bound to the first device as mutually associated devices through the following steps C1:

[0242] Step C1: The second device obtains device information of the first device based on the communication connection with the first device, and sends a binding instruction to the server based on the device information of the first device and the second device, wherein the binding instruction indicates to bind the first device and the second device as associated devices.

[0243] For example, the communication connection may be a Bluetooth connection, the second device may establish a Bluetooth connection with the first device in advance, and the second device may implement a process of sending device information based on the Bluetooth connection.

[0244] The device information may be a SN serial number of the device, which is a string serial number and is an ID that can uniquely identify a device.

[0245] Accordingly, the process of the server performing binding based on the binding instruction may include steps D1-D2:

[0246] Step D1, the server receives a binding instruction sent by at least one of the first device or the second device;

[0247] Step D2: the server binds the first device and the second device as associated devices based on the binding instruction sent by the at least one device, and stores device information of the first device and the second device in an associated manner;

[0248] The binding instruction sent by the at least one device carries device information of the first device and the second device, and the device information of the first device and the second device is obtained based on the communication connection between the first device and the second device.

[0249] It should be noted that the above description is only based on the example of the second device sending a binding instruction to the server to implement the binding process of the associated device. Of course, the above step C1 can also be performed by the first device. Alternatively, the first device and the second device can both send a binding instruction to the server. This application does not limit the binding process.

[0250] Step 802: In response to detecting that a face-scanning payment request for a target order is initiated through the first device, the first device captures a facial image through an image capture device of the first device, and sends the captured first facial image to a server.

[0251] Step 803: The server receives the payment process video sent by the second device, and receives the first facial image sent by the first device in response to the face payment request for the target order.

[0252] Among them; the payment process video is obtained by the second device performing video capture in response to the target trigger operation in the first device, the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order.

[0253] In one possible manner, a server receives a first facial image sent by a first device in response to a face-scanning payment request for a target order, including: the server receives a facial image sent by at least one payment device, the at least one payment device including the first device; the server determines the first device associated with the second device based on associated stored device information and device information of the second device, and determines the first facial image sent by the first device from the facial images sent by each payment device.

[0254] The server first starts to receive each frame of the payment process video sent by the second device, and then receives the face payment request from the first device. For example, the server can query the first device from multiple payment devices based on the device information of the second device, and obtain the first facial image sent by the first device.

[0255] Step 804: The server determines whether the first device has been swiped incorrectly based on the payment process video to obtain a determination result.

[0256] The determination result indicates whether the first facial image is a facial image of a first object, where the first object is the object that initiates a face-swiping payment request for a target order.

[0257] In one possible manner, an implementation method of step 804 may include: the server performs a posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, and the at least one object includes at least a first object; the server obtains the determination result based on the posture information of the at least one object.

[0258] In another possible manner, the server may also combine the first facial image and the payment process video to obtain the determination result. Exemplarily, step 804 may include: the server performs a false swipe determination on the first device based on the payment process video and the first facial image to obtain a determination result.

[0259] In a possible example, the server may also combine the second facial image, the first facial image and the payment process video to make a mistaken swipe judgment. Accordingly, the process may include: the server obtains the second facial image from the first device, and determines the image similarity between the first facial image and the second facial image; wherein the first facial image is an image captured by the facial recognition page displayed by the first device when the face-swiping payment request is initiated; the second facial image is an image captured after the first facial image is captured; the server obtains the judgment result based on the posture information and image similarity of the at least one object. The way in which the server combines the first facial image, the second facial image and the payment process video to make a mistaken swipe judgment is the same process as steps A1-A2, and will not be repeated here.

[0260] Step 805: The server determines the payment result based on the determination result and the first facial image.

[0261] If the determination result is the second determination result, the server may perform face-scanning payment based on the first facial image and return the result of the face-scanning payment.

[0262] If the determination result is the first determination result, the server may send a prompt message to the first device to prompt it to re-initiate a face recognition payment request for the target order.

[0263] Step 806: The server sends the payment result to the first device.

[0264] Step 807: The first device displays the payment result sent by the server.

[0265] In one possible manner, if the determination result obtained in step 804 is the first determination result, the server and the first device may perform the following interactive steps B1-B4:

[0266] Step B1: If the determination result is the first determination result, the server sends a prompt message to the first device.

[0267] The prompt message prompts to re-initiate a face-scanning payment request for the target order, and the first determination result indicates that the first facial image is not a facial image of the first object.

[0268] Step B2: The first device displays the prompt message sent by the server;

[0269] Step B3: The first device detects the face-scanning payment request re-initiated based on the prompt message, re-captures the facial image, and sends the re-captured third facial image to the server.

[0270] Step B4: The server receives the third facial image, and based on the third facial image and the payment process video sent by the second device, determines the payment result corresponding to the third facial image; and sends the payment result corresponding to the third facial image to the first device.

[0271] Step B4: The first device receives and displays the payment result corresponding to the third facial image.

[0272] It should be noted that the above description only takes the example of the server sending a video acquisition instruction and a stop acquisition instruction to the second device. In some other possible embodiments, the first device can also control the second device to start or stop collecting the payment process video. Exemplarily, taking the video acquisition instruction as an example, the process of the first device controlling the second device to start acquisition includes: a Bluetooth connection can be established between the first device and the second device, when the first device detects a target trigger operation for a target order, the first device sends a video acquisition instruction to the second device, the second device performs video acquisition on the first device based on the video acquisition instruction of the first device, and sends the collected payment process video to the first device.

[0273] Below Fig. 9 The process shown introduces the overall process of facial payment in this application.

[0274] like Fig. 9 As shown, the overall process may include:

[0275] 1. After the payment device is put into use, the merchant needs to place the payment devices side by side, enable the Bluetooth function of the two payment devices, and then pair them. For example, the payment device can be a self-service cashier terminal device in a store or supermarket that supports facial payment.

[0276] 2. When the Bluetooth pairing connection is successful, the SN serial numbers of the two currently paired payment devices will be sent to the backend for recording, and then returned to the device end. Subsequently, the two payment devices become paired devices, binding the two payment devices.

[0277] 3. After device A is powered on, it will establish a persistent command push channel with the backend service to receive control commands from the backend. Device B can also establish a persistent connection in this way.

[0278] 4. After device B is started, the auxiliary detection module of device A will be started to continuously collect video of device A.

[0279] 5. Device B will continue to blur the face in the payment process video and send it to the backend server. The backend server will continue to determine whether the user hits the detection rules and obtain the determination result; and send the prompt message based on the determination result to device A.

[0280] 6. When the user performs face payment on device A, if the first judgment result is hit, a prompt message will be displayed. For example, the prompt message may indicate that the current face payment may be swiped by mistake, and the face payment needs to be re-initiated or further confirmed. After the user confirms or re-initiates the face payment, the payment is executed.

[0281] Fig.10This is a schematic diagram of a system architecture provided by this application. Fig.10 Take the face payment method of this application as an example to introduce. Fig.10 As shown, the system may include: face recognition device A, face recognition device B and a backend server.

[0282] 1. For face recognition devices:

[0283] For example, the face recognition devices A and B can be IoT (Internet of Things) face terminal devices, such as self-service cashier terminals. Fig.10 As shown, the face scanning device may include the following structure:

[0284] (1) 3D camera. In this application, in order to enhance the security of user facial recognition, a 3D camera is used on the IoT face terminal. The data output by the camera includes not only RGB images but also related information such as depth images. The scanning terminal is a common QR code scanning terminal. For example, the first facial image, the second facial image, and the third facial image can be depth images acquired based on the 3D camera.

[0285] (2) Bluetooth and network modules: The Bluetooth module is used to establish a Bluetooth connection between payment devices and confirm the identity of each other's devices. The network module, in this application, can be used to send the payment process video to the backend server and receive prompt information from the backend server. For example, the prompt information can be in the form of a control instruction to control the first device to re-initiate the face-swiping payment process.

[0286] (3) APP that supports face-scanning payment, the core of which includes face-scanning payment module, auxiliary detection module, command module, Bluetooth pairing module, etc.

[0287] (3.1) The face payment module mainly includes face collection and optimization, which are:

[0288] Face acquisition - used to call the 3D camera for acquisition, obtain RGB image stream, depth image stream, infrared image stream, etc.

[0289] Face optimization - select the best face image through comprehensive evaluation of factors such as face size, face angle, image contrast, image brightness and clarity.

[0290] After successful collection and optimization, the IoT facial terminal will send the data to the backend for recognition through the network module.

[0291] (3.2) The auxiliary detection module has two core functions: data upload and video diversion. Video diversion mainly means that if the current device has the ability to set up paired detection, the data collected by the current camera needs to be divided into two paths. One video stream maintains the original path, that is, the path for face payment. The other video is to desensitize the facial information in the current video stream (blur the face or mosaic, etc.), call the data upload module, and independently send the video stream to the backend to determine whether it hits the relevant detection rules. The data upload module is an independent sub-process, which is mainly used to upload the video stream generated by the detection device.

[0292] (3.3) Instruction module, a module for receiving dispatch instructions from the backend service. This application is mainly for receiving control instructions from the backend server so as to execute the operation process of secondary confirmation and re-payment when the server identifies that there is a risk in the payment device.

[0293] 2. For the backend server, such as Fig.10 As shown:

[0294] The backend server can provide related services, including facial payment services and device-assisted detection services, as follows:

[0295] (1) A facial recognition service, which is used to extract features from the current image after receiving facial data uploaded by the payment device, and compare the relevant facial data with the faces in the backend database to verify the first facial image.

[0296] (2) Equipment auxiliary detection services, mainly including equipment detection binding services, command push services, and auxiliary detection analysis services.

[0297] (2.1) Device detection and binding service, used to receive binding instructions from payment devices for mutual binding, which may include the SN serial numbers of payment devices A and B, so as to associate and store the SN serial numbers.

[0298] (2.2) Command push service: After the backend server receives the payment process video from the payment device, the video can be transmitted in the form of a video stream; the backend server can continuously analyze the payment process video and push control commands based on the judgment results. If device A is bound to device B, device B will bypass the collection of device A's detection video and upload it. When the backend determines that there may be a mis-swipe in the video uploaded by device B, it will query the device SN bound to device B and send the control command to device A based on the SN.

[0299] (2.3) Auxiliary detection and analysis service. It should be noted that in one possible scenario of the present application, multiple objects may line up around the payment device to pay. Specifically, the present application provides a variety of ways to determine false swipes in order to solve the problem that the front-row user bends over after initiating the face-scanning payment, causing the back-row user to be identified. For example, identifying whether the object bends over or turns around, or identifying whether the first facial image collected when initiating the face-scanning request is similar to the second facial image collected after the initiation, etc., which will not be described in detail here.

[0300] The facial payment method provided in the present application deploys multiple payment devices in the target space, and deploys each payment device as an associated device of another payment device. The associated device of one payment device is used to collect the payment process video of the payment device; the first device, as any one of the multiple payment devices, can collect and send the first facial image to the server when initiating a face payment request, and display the payment result sent by the server.

[0301] Among them, the payment result is determined based on the judgment result of the first device and the first facial image. Since the judgment result is obtained by judging the first device as being mistakenly swiped based on the payment process video obtained from the second device, the second device is an associated device of the first device. In addition, the payment process video is collected by the second device in response to the target trigger operation associated with the target order in the first device, because the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request. The judgment result represents whether the first facial image is the facial image of the first object that initiates the face-swiping payment request. Therefore, the payment process video may include a video screen of the process of initiating the face-swiping payment request to collect facial images; based on the payment process video, it can be effectively determined whether the first device mistakenly swipes the facial image of other objects, which effectively solves the problem of mistakenly swiping the face of other objects for payment during face-swiping payment, and ensures the security and reliability of face-swiping payment.

[0302] The facial payment method provided in this application involves technologies such as the Internet of Things, cloud IoT, medical cloud, and artificial intelligence. For example, in this application, the merchant's payment device can be connected to the server based on IoT interconnection technology to realize the process of face payment.

[0303] It is understandable that the Internet of Things (IOT) refers to the real-time collection of any object or process that needs to be detected, connected, and interacted through various devices and technologies such as various information sensors, radio frequency identification technology, global positioning system, infrared sensor, laser scanner, etc., and the collection of various required information such as sound, light, heat, electricity, mechanics, chemistry, biology, location, etc., and the realization of ubiquitous connection between objects and objects, objects and people through various possible network access, and the realization of intelligent perception, identification and management of objects and processes. The Internet of Things is an information carrier based on the Internet, traditional telecommunications networks, etc. It allows all ordinary physical objects that can be independently addressed to form an interconnected network.

[0304] Cloud IOT aims to connect the information sensed and instructions received by sensor devices in traditional IoT to the Internet, truly realize networking, and achieve massive data storage and computing through cloud computing technology. Due to the characteristics of IoT, objects are connected to each other and the current operating status of each "object" is perceived in real time. A large amount of data information will be generated in this process. How to aggregate this information and how to filter out useful information from the massive information to support decision-making for subsequent development have become key issues affecting the development of IoT. IoT cloud based on cloud computing and cloud storage technology has therefore become a strong support for IoT technology and applications.

[0305] Medical cloud refers to the use of cloud computing to create a cloud platform for medical and health services based on new technologies such as cloud computing, mobile technology, multimedia, 4G communications, big data, and the Internet of Things, combined with medical technology, to achieve the sharing of medical resources and the expansion of medical scope. Because of the application and combination of cloud computing technology, medical cloud has improved the efficiency of medical institutions and facilitated residents' medical treatment. For example, hospital appointment registration and medical insurance are the products of the combination of cloud computing and the medical field. Medical cloud also has the advantages of data security, information sharing, dynamic expansion, and overall layout.

[0306] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0307] Fig.11A schematic diagram of the structure of a facial payment device provided in an embodiment of the present application, the device is executed by a first device; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device;

[0308] like Fig.11 As shown, the device comprises:

[0309] The facial image sending module 1101 is used for, in response to detecting that a facial payment request for a target order is initiated through the first device, collecting a facial image through an image collection device of the first device, and sending the collected first facial image to the server;

[0310] The payment result display module 1102 is used to display the payment result sent by the server;

[0311] The payment result is determined based on a determination result of the first device and the first facial image, and the determination result is obtained by determining a false swipe of the first device based on a payment process video obtained from a second device, and the second device is an associated device of the first device;

[0312] The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-scanning payment request, and the target trigger operation is an operation associated with the target order;

[0313] The determination result indicates whether the first facial image is a facial image of a first object, where the first object is the object that initiates the face-swiping payment request.

[0314] In one possible implementation, the

[0315] Payment result display module, used for:

[0316] If the determination result is a first determination result, a prompt message sent by the server is displayed, the prompt message prompting to re-initiate a face recognition payment request for the target order, and the first determination result indicates that the first facial image is not a facial image of the first object;

[0317] The payment result corresponding to the face recognition payment request re-initiated based on the prompt message is displayed. In a possible implementation, the device further includes a determination result determination module, and the determination result determination module, when determining the determination result, includes:

[0318] A receiving unit, configured to receive the payment process video sent by the second device;

[0319] a posture analysis unit, configured to perform posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, wherein the at least one object includes at least a first object;

[0320] The determination unit is used to obtain the determination result based on the posture information of the at least one object.

[0321] In a possible implementation, the posture analysis unit is used to:

[0322] Identify a first object among various objects included in the payment process video, and perform joint point detection on the first object to obtain various joint points of the first object;

[0323] Determining posture information of the first object based on relative positions between joints of the first object, where the posture information indicates whether an action of performing face scanning failure is performed;

[0324] The determination unit is used for:

[0325] If the posture information of the first object is the first posture information, the determination result is determined to be the first determination result, the first posture information indicates that the first object has failed to perform a face scanning action, and the first determination result indicates that the first facial image is not the facial image of the first object.

[0326] In a possible implementation, the posture analysis unit is used for at least one of the following:

[0327] Based on the angle between the line connecting the first joint point of the first object and the line connecting the hip joint point, determining whether the first object is bending over, and obtaining the posture information based on the bending over determination result; the action of the failed face scanning includes bending over, and the first joint point includes at least one of a shoulder joint point or a knee joint point;

[0328] Based on the rotation angle corresponding to the line connecting the second joint points of the first object, determining whether the first object turns around, and obtaining the posture information based on the turning determination result; the face scanning failure action includes turning around, and the second joint point includes at least one of a shoulder joint point or a hip joint point;

[0329] Based on the movement distance corresponding to the third joint point of the first object, it is determined whether the first object has moved, and the posture information is obtained based on the movement determination result; the action of the failed face scanning includes movement, and the third joint point includes a torso joint point.

[0330] In a possible implementation, the determination result is obtained by performing a false swipe determination on the first device based on the payment process video and the first facial image;

[0331] The determination unit is used for:

[0332] Acquiring a second facial image from the first device, and determining an image similarity between the first facial image and the second facial image;

[0333] The first facial image is an image captured by a facial recognition page displayed by the first device when a facial payment request is initiated; the second facial image is an image captured after the first facial image is captured;

[0334] The determination result is obtained based on the posture information and the image similarity of the at least one object.

[0335] In a possible implementation, the first device corresponds to a plurality of associated devices in the target area, and the second device is any one of the plurality of associated devices of the first device; each associated device is used to capture a payment process video of the first device from a perspective corresponding to the associated device;

[0336] The determination result determination module, when determining the determination result, is further used to:

[0337] Determine the panoramic video corresponding to the first device based on the payment process videos respectively corresponding to the multiple associated devices;

[0338] Based on the panoramic video, a false swipe determination is performed on the first device to obtain the determination result.

[0339] In one possible implementation, the target trigger operation includes any one of an operation of selecting a face-scanning payment method or an operation of entering an item; the payment process video includes a video screen of a process of entering item information in the first device, and a process of initiating a face-scanning payment request to capture a facial image.

[0340] The facial payment method provided in the present application deploys multiple payment devices in the target space, and deploys each payment device as an associated device of another payment device. The associated device of one payment device is used to collect the payment process video of the payment device; the first device, as any one of the multiple payment devices, can collect and send the first facial image to the server when initiating a face payment request, and display the payment result sent by the server.

[0341] Among them, the payment result is determined based on the judgment result of the first device and the first facial image. Since the judgment result is obtained by judging the first device as being mistakenly swiped based on the payment process video obtained from the second device, the second device is an associated device of the first device. In addition, the payment process video is collected by the second device in response to the target trigger operation associated with the target order in the first device, because the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request. The judgment result represents whether the first facial image is the facial image of the first object that initiates the face-swiping payment request. Therefore, the payment process video may include a video screen of the process of initiating the face-swiping payment request to collect facial images; based on the payment process video, it can be effectively determined whether the first device mistakenly swipes the facial image of other objects, which effectively solves the problem of mistakenly swiping the face of other objects for payment during face-swiping payment, and ensures the security and reliability of face-swiping payment.

[0342] Fig.12 A structural diagram of a facial payment device provided in an embodiment of the present application. The device is executed by a server; the server establishes communication connections with a first device and a second device respectively; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; the second device is an associated device of the first device.

[0343] like Fig.12 As shown, the device comprises:

[0344] The video receiving module 1201 is used to receive the payment process video sent by the second device;

[0345] The facial image receiving module 1202 is used to receive a first facial image sent by the first device in response to a facial payment request for a target order;

[0346] The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-scanning payment request, and the target trigger operation is an operation associated with the target order;

[0347] The mis-swipe determination module 1203 is used to determine the mis-swipe of the first device based on the payment process video to obtain a determination result, wherein the determination result indicates whether the first facial image is a facial image of a first object, and the first object is the object that initiates the face-swiping payment request for the target order;

[0348] A payment result determination module 1204, configured to determine the payment result based on the determination result and the first facial image;

[0349] The sending module 1205 is used to send the payment result to the first device.

[0350] In a possible implementation, the false-swipe determination module is used to:

[0351] Performing posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, where the at least one object includes at least a first object;

[0352] The determination result is obtained based on the posture information of the at least one object.

[0353] In a possible implementation, the facial image receiving module is used to:

[0354] receiving a facial image sent by at least one payment device, the at least one payment device including the first device;

[0355] Determine a first device associated with the second device based on the associated stored device information and the device information of the second device, and determine a first facial image sent by the first device from the facial images sent by each payment device;

[0356] The false-brush determination module is used to:

[0357] Based on the payment process video and the first facial image, a mistaken swipe determination is performed on the first device to obtain a determination result.

[0358] In a possible implementation, the false-swipe determination module is used to:

[0359] Acquiring a second facial image from the first device, and determining an image similarity between the first facial image and the second facial image;

[0360] The first facial image is an image captured by a facial recognition page displayed by the first device when a facial payment request is initiated; the second facial image is an image captured after the first facial image is captured;

[0361] The determination result is obtained based on the posture information and the image similarity of the at least one object.

[0362] In a possible implementation, the device further includes:

[0363] A binding module, configured to receive a binding instruction sent by at least one of the first device or the second device;

[0364] an association storage module, configured to bind the first device and the second device as associated devices based on the binding instruction sent by the at least one device, and to associate and store device information of the first device and the second device;

[0365] The binding instruction sent by the at least one device carries device information of the first device and the second device, and the device information of the first device and the second device is obtained based on the communication connection between the first device and the second device.

[0366] In a possible implementation, the video receiving module is used to:

[0367] In response to receiving a video capture instruction sent by the first device, forwarding the video capture instruction to the second device, and receiving the payment process frequency returned by the second device based on the video capture instruction, wherein the video capture instruction is triggered by a target trigger operation on a target order in the first device.

[0368] In a possible implementation, the device further includes:

[0369] A stop instruction sending module, configured to send a stop acquisition instruction to the second device in response to detecting a stop acquisition event corresponding to the target order; so that the second device stops capturing video of the first device and stops sending the payment process video of the first device based on the stop acquisition instruction;

[0370] The stop collection event includes at least one of the following:

[0371] Determine a second determination result based on the payment process video sent by the second device;

[0372] receiving a first facial image sent by a first device;

[0373] Determining that the payment is successful based on the second determination result and the first facial image;

[0374] A third facial image re-captured and sent by the first device based on the prompt message is received.

[0375] The facial payment device provided by the present application deploys multiple payment devices in the target space, and deploys each payment device as an associated device of another payment device. The associated device of a payment device is used to collect the payment process video of the payment device; the first device, as any one of the multiple payment devices, can collect and send the first facial image to the server when initiating a face payment request, and display the payment result sent by the server.

[0376] Among them, the payment result is determined based on the judgment result of the first device and the first facial image. Since the judgment result is obtained by judging the first device as being mistakenly swiped based on the payment process video obtained from the second device, the second device is an associated device of the first device. In addition, the payment process video is collected by the second device in response to the target trigger operation associated with the target order in the first device, because the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request. The judgment result represents whether the first facial image is the facial image of the first object that initiates the face-swiping payment request. Therefore, the payment process video may include a video screen of the process of initiating the face-swiping payment request to collect facial images; based on the payment process video, it can be effectively determined whether the first device mistakenly swipes the facial image of other objects, which effectively solves the problem of mistakenly swiping the face of other objects for payment during face-swiping payment, and ensures the security and reliability of face-swiping payment.

[0377] Fig.13 A schematic diagram of the structure of a facial payment device provided in an embodiment of the present application. The device is executed by a second device; the second device is an associated device of the first device, the first device is any device among multiple payment devices deployed in the target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device;

[0378] like Fig.13 As shown, the device comprises:

[0379] The binding instruction sending module 1301 is used to obtain device information of the first device based on the communication connection with the first device, and send a binding instruction to the server based on the device information of the first device and the second device, wherein the binding instruction indicates to bind the first device and the second device as associated devices;

[0380] The video sending module 1302 is used to respond to the target triggering operation of the target order in the first device, to perform video acquisition through the image acquisition device of the second device, and to send the acquired payment process video to the server;

[0381] Among them, the trigger time of the target trigger operation is before the initiation time of the face-scanning payment request; the face-scanning payment request is a request for face-scanning payment for the target order initiated by the first device, and the target trigger operation is an operation associated with the target order.

[0382] In a possible implementation, the device further includes:

[0383] The stop sending module is used to stop capturing video of the first device and stop sending the payment process video of the first device to the server in response to receiving a stop capturing instruction sent by the server.

[0384] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0385] Fig.14 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.14 As shown, the electronic device includes: a memory, a processor and a computer program stored in the memory, and the processor executes the above computer program to implement the steps of the facial payment method. Compared with the related art, it can achieve:

[0386] The facial payment device provided by the present application deploys multiple payment devices in the target space, and deploys each payment device as an associated device of another payment device. The associated device of a payment device is used to collect the payment process video of the payment device; the first device, as any one of the multiple payment devices, can collect and send the first facial image to the server when initiating a face payment request, and display the payment result sent by the server.

[0387] Among them, the payment result is determined based on the judgment result of the first device and the first facial image. Since the judgment result is obtained by judging the first device as being mistakenly swiped based on the payment process video obtained from the second device, the second device is an associated device of the first device. In addition, the payment process video is collected by the second device in response to the target trigger operation associated with the target order in the first device, because the trigger time of the target trigger operation is before the initiation time of the face-swiping payment request. The judgment result represents whether the first facial image is the facial image of the first object that initiates the face-swiping payment request. Therefore, the payment process video may include a video screen of the process of initiating the face-swiping payment request to collect facial images; based on the payment process video, it can be effectively determined whether the first device mistakenly swipes the facial image of other objects, which effectively solves the problem of mistakenly swiping the face of other objects for payment during face-swiping payment, and ensures the security and reliability of face-swiping payment.

[0388] In an alternative embodiment, an electronic device is provided, such as Fig.14 As shown, Fig.14The electronic device 1400 shown includes: a processor 1401 and a memory 1403. The processor 1401 and the memory 1403 are connected, such as through a bus 1402. Optionally, the electronic device 1400 may also include a transceiver 1404, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 1404 is not limited to one, and the structure of the electronic device 1400 does not constitute a limitation on the embodiments of the present application.

[0389] Processor 1401 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0390] The bus 1402 may include a path to transmit information between the above components. The bus 1402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.14 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0391] The memory 1403 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium\other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0392] The memory 1403 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 1401. The processor 1401 is used to execute the computer program stored in the memory 1403 to implement the steps shown in the above method embodiment.

[0393] Among them, electronic equipment includes but is not limited to: servers, payment devices, self-service checkout counters, cashier terminals or cloud computing center equipment, etc.

[0394] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0395] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0396] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. The terms "include" and "comprise" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, and operations, but do not exclude the implementation as other features, information, data, steps, operations, etc. supported by the technical field.

[0397] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0398] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0399] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A facial payment method, characterized in that: The method is performed by a first device; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; The method comprises: In response to detecting that a face recognition payment request for a target order is initiated through the first device, a facial image is captured through an image capture device of the first device, and the captured first facial image is sent to a server; Display the payment result sent by the server; The payment result is determined based on a determination result of the first device and the first facial image, and the determination result is obtained by determining a false swipe of the first device based on a payment process video obtained from a second device, and the second device is an associated device of the first device; The payment process video is obtained by the second device performing video acquisition in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-scanning payment request, and the target trigger operation is an operation associated with the target order; The determination result indicates whether the first facial image is a facial image of a first object, where the first object is the object that initiates the face-swiping payment request.

2. The method according to claim 1, characterized in that The payment result sent by the display server includes: If the determination result is a first determination result, a prompt message sent by the server is displayed, the prompt message prompting to re-initiate a face recognition payment request for the target order, and the first determination result indicates that the first facial image is not a facial image of the first object; The payment result corresponding to the face recognition payment request re-initiated based on the prompt message is displayed.

3. The method according to claim 1, characterized in that: The determination result is obtained in the following manner: receiving a payment process video sent by the second device; Performing posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, wherein the posture information indicates whether an action of face scanning failure is performed, and the at least one object includes at least a first object; The determination result is obtained based on the posture information of the at least one object.

4. The method according to claim 3, characterized in that The performing posture analysis on at least one object in the payment process video to obtain posture information of the at least one object includes: Identify a first object among various objects included in the payment process video, and perform joint point detection on the first object to obtain various joint points of the first object; determining posture information of the first object based on relative positions between joints of the first object, the posture information indicating whether an action of performing face scanning failure is performed; The obtaining the determination result based on the posture information of the at least one object includes: If the posture information of the first object is the first posture information, the determination result is determined to be the first determination result, the first posture information indicates that the first object has failed to perform a face scanning action, and the first determination result indicates that the first facial image is not the facial image of the first object.

5. The method according to claim 4, characterized in that The determining the posture information of the first object based on the relative positions between the joints of the first object includes at least one of the following: Based on the angle between the line connecting the first joint point of the first object and the line connecting the hip joint point, determining whether the first object is bending over, and obtaining the posture information based on the bending over determination result; the action of failed face scanning includes bending over, and the first joint point includes at least one of a shoulder joint point or a knee joint point; Based on the rotation angle corresponding to the line connecting the second joint points of the first object, determining whether the first object has turned around, and obtaining the posture information based on the turning determination result; the face scanning failed action includes turning around, and the second joint point includes at least one of a shoulder joint point or a hip joint point; Based on the moving distance corresponding to the third joint point of the first object, determine whether the first object has moved, and obtain the posture information based on the movement determination result; the action of failed face scanning includes movement, and the third joint point includes a torso joint point.

6. The method according to claim 3 or 4, characterized in that: The determination result is obtained by determining an incorrect swipe of the first device based on the payment process video and the first facial image; The obtaining the determination result based on the posture information of the at least one object includes: Acquiring a second facial image from a first device, and determining an image similarity between the first facial image and the second facial image; The first facial image is an image captured by a facial recognition page displayed by the first device when a facial payment request is initiated; the second facial image is an image captured after the first facial image is captured; The determination result is obtained based on the posture information and the image similarity of the at least one object.

7. The method according to claim 1, characterized in that The first device corresponds to a plurality of associated devices in the target area, and the second device is any one of the plurality of associated devices of the first device; each associated device is used to collect a payment process video of the first device from a perspective corresponding to the associated device; The determination result is obtained in the following manner: Determining a panoramic video corresponding to the first device based on the payment process videos respectively corresponding to the multiple associated devices; Performing a false swipe determination on the first device based on the panoramic video to obtain the determination result.

8. The method according to claim 1, characterized in that The target trigger operation includes any one of the operations of selecting a face-scanning payment method or entering an item; the payment process video includes a video screen of the process of entering item information in the first device, and a process of initiating a face-scanning payment request to capture a facial image.

9. A facial payment method, characterized in that: The method is executed by a server; the server establishes communication connections with a first device and a second device respectively; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; The second device is an associated device of the first device; The method comprises: receiving a payment process video sent by a second device, and receiving a first facial image sent by a first device in response to a face payment request for a target order; The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order; Performing a false swipe determination on the first device based on the payment process video to obtain a determination result, wherein the determination result indicates whether the first facial image is a facial image of a first object, the first object being the object that initiates a facial swipe payment request for a target order; Determining the payment result based on the determination result and the first facial image; The payment result is sent to the first device.

10. The method according to claim 9, characterized in that The performing a mistaken swipe determination on the first device based on the payment process video to obtain a determination result includes: Performing posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, wherein the posture information indicates whether an action of face scanning failure is performed, and the at least one object includes at least a first object; The determination result is obtained based on the posture information of the at least one object.

11. The method according to claim 10, characterized in that The receiving a first facial image sent by the first device in response to a face payment request for a target order includes: receiving a facial image sent by at least one payment device, the at least one payment device including the first device; Determine a first device associated with the second device based on the associated stored device information and the device information of the second device, and determine a first facial image sent by the first device from the facial images sent by each payment device; The performing a mistaken swipe determination on the first device based on the payment process video to obtain a determination result includes: Based on the payment process video and the first facial image, a mistaken swipe determination is performed on the first device to obtain a determination result.

12. The method according to claim 11, characterized in that The obtaining the determination result based on the posture information of the at least one object includes: Acquire a second facial image from the first device, and determine an image similarity between the first facial image and the second facial image; The first facial image is an image captured by a facial recognition page displayed by the first device when a facial payment request is initiated; the second facial image is an image captured after the first facial image is captured; The determination result is obtained based on the posture information and the image similarity of the at least one object.

13. The method according to claim 9, characterized in that Before receiving the payment process video sent by the second device, the method further includes: receiving a binding instruction sent by at least one of the first device or the second device; Based on the binding instruction sent by the at least one device, bind the first device and the second device as associated devices, and associate and store device information of the first device and the second device; The binding instruction sent by the at least one device carries device information of the first device and the second device, and the device information of the first device and the second device is obtained based on the communication connection between the first device and the second device.

14. The method according to claim 9, characterized in that The receiving the payment process video sent by the second device includes: In response to receiving a video capture instruction sent by the first device, forwarding the video capture instruction to the second device, and receiving the payment process frequency returned by the second device based on the video capture instruction, wherein the video capture instruction is triggered by a target trigger operation on a target order in the first device.

15. The method according to claim 9, characterized in that The method further comprises: In response to detecting a stop acquisition event corresponding to the target order, sending a stop acquisition instruction to the second device; so that the second device stops capturing video of the first device and stops sending the payment process video of the first device based on the stop acquisition instruction; The stop acquisition event includes at least one of the following: Determine a second determination result based on the payment process video sent by the second device; receiving a first facial image sent by a first device; determining that the payment is successful based on the second determination result and the first facial image; A third facial image re-captured and sent by the first device based on the prompt message is received.

16. A facial payment method, characterized in that: The method is performed by a second device; the second device is an associated device of the first device, the first device is any one of a plurality of payment devices deployed in the target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; The method comprises: Based on the communication connection with the first device, acquiring device information of the first device, and based on the device information of the first device and the second device, sending a binding instruction to the server, wherein the binding instruction indicates that the first device and the second device are bound as associated devices; In response to a target triggering operation on a target order in the first device, performing video acquisition through an image acquisition device of the second device, and sending the acquired payment process video to a server; Among them, the triggering time of the target triggering operation is before the initiation time of the face-scanning payment request; the face-scanning payment request is a request initiated by the first device to perform face-scanning payment for the target order, and the target triggering operation is an operation associated with the target order.

17. A facial payment device, characterized in that: The apparatus is executed by a first device; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; The device comprises: A facial image sending module, configured to, in response to detecting that a facial payment request for a target order is initiated through the first device, collect a facial image through an image collection device of the first device, and send the collected first facial image to a server; The payment result display module is used to display the payment result sent by the server; The payment result is determined based on a determination result of the first device and the first facial image, and the determination result is obtained by determining a false swipe of the first device based on a payment process video obtained from a second device, and the second device is an associated device of the first device; The payment process video is obtained by the second device performing video acquisition in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-scanning payment request, and the target trigger operation is an operation associated with the target order; The determination result indicates whether the first facial image is a facial image of a first object, where the first object is the object that initiates the face-swiping payment request.

18. A facial payment device, characterized in that: The device is executed by a server; the server establishes communication connections with a first device and a second device respectively; the first device is any one of a plurality of payment devices deployed in a target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; The second device is an associated device of the first device; The device comprises: A video receiving module, used for receiving a payment process video sent by a second device; A facial image receiving module, configured to receive a first facial image sent by the first device in response to a facial payment request for a target order; The payment process video is obtained by the second device performing video capture in response to a target trigger operation in the first device, the triggering time of the target trigger operation is before the initiation time of the face-swiping payment request, and the target trigger operation is an operation associated with the target order; a false swipe determination module, configured to determine a false swipe of the first device based on the payment process video to obtain a determination result, wherein the determination result indicates whether the first facial image is a facial image of a first object, the first object being an object that initiates a face-swiping payment request for a target order; a payment result determination module, configured to determine the payment result based on the determination result and the first facial image; A sending module is used to send the payment result to the first device. In a possible implementation, the false-swipe determination module is configured to: Performing posture analysis on at least one object in the payment process video to obtain posture information of the at least one object, wherein the posture information indicates whether an action of face scanning failure is performed, and the at least one object includes at least a first object; The determination result is obtained based on the posture information of the at least one object.

19. A facial payment device, characterized in that: The apparatus is executed by a second device; the second device is an associated device of the first device, the first device is any one of a plurality of payment devices deployed in the target space, each payment device is an associated device of another payment device in the target space, and an associated device of a payment device is used to collect a payment process video of the payment device; The device comprises: a binding instruction sending module, configured to obtain device information of the first device based on a communication connection with the first device, and send a binding instruction to a server based on the device information of the first device and the second device, wherein the binding instruction indicates that the first device and the second device are bound as associated devices; a video sending module, configured to respond to a target triggering operation on a target order in the first device, to perform video acquisition through an image acquisition device of the second device, and to send the acquired payment process video to the server; Among them, the triggering time of the target triggering operation is before the initiation time of the face-scanning payment request; the face-scanning payment request is a request initiated by the first device to perform face-scanning payment for the target order, and the target triggering operation is an operation associated with the target order.

20. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the facial payment method according to any one of claims 1 to 16.

Images