Face-scanning payment method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of artificial intelligence, and relates to a face-scanning payment method, which comprises the following steps: when a face-scanning payment request is received, obtaining a first image of a customer through a POS terminal; performing image reconstruction on the first image through an image super-resolution reconstruction algorithm to obtain a high-resolution second image; inputting the second image into a face detection model to obtain a face region image in the second image; performing feature extraction on the face region image through a local binary pattern algorithm to obtain a face feature image; performing face recognition in a preset face feature database based on the face feature image to obtain an identity verification result of the face scanning payment request; and when the identity verification result shows that the client identity is legal, responding to the face scanning payment request to complete the face scanning payment operation. The invention further provides a face-scanning payment device, computer equipment and a storage medium. According to the invention, the accuracy of face-scanning payment of the POS terminal is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a face-scanning payment method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of mobile payment, face-scanning payment, as a new payment method, has gradually gained market favor due to its convenience and efficiency. However, there are some technical difficulties and shortcomings in the actual application of face-scanning payment technology.

[0003] In order to ensure the accuracy and stability of face recognition, face-swiping payment devices need to adopt high-performance hardware configurations, such as high-resolution cameras and large-capacity memory. Such high-configuration devices are often expensive and are not conducive to promotion and application in a wide range of markets. For small and micro merchants, the high cost of equipment may become a major obstacle for them to accept face-swiping payment. Therefore, most merchants use simple POS machines. Most of the existing face-swiping payment technologies rely on local processing, that is, the processing and recognition tasks of facial images are handed over to the device itself. For simple POS machines with relatively ordinary hardware configurations, their processing power and storage space are limited, and it is difficult to meet the needs of complex face recognition algorithms, which may lead to problems such as slower recognition speed and lower recognition rate. Secondly, in actual payment scenarios, light factors are an important factor affecting the accuracy of face recognition. Due to changes in ambient light, such as insufficient indoor light and direct sunlight outdoors, the quality of facial images may decrease. Therefore, the accuracy of existing POS machines in face-swiping payment is low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a face-scanning payment method, apparatus, computer equipment and storage medium to solve the problem of low accuracy of face-scanning payment by POS machines.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a face-scanning payment method, which adopts the following technical solution:

[0006] When receiving a face payment request, obtaining a first image of the customer through the POS terminal;

[0007] Reconstructing the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image;

[0008] Inputting the second image into a face detection model to obtain a face region image in the second image;

[0009] Extracting features from the face region image using a local binary pattern algorithm to obtain a face feature image;

[0010] Performing face recognition in a preset face feature database based on the face feature image to obtain an identity authentication result of the face payment request;

[0011] When the identity verification result indicates that the customer's identity is legitimate, a response is made to the face-scanning payment request to complete the face-scanning payment operation.

[0012] In order to solve the above technical problems, the embodiment of the present application also provides a face-scanning payment device, which adopts the following technical solution:

[0013] A first acquisition module, configured to acquire a first image of the customer through a POS terminal when receiving a face-scanning payment request;

[0014] An image reconstruction module, used to reconstruct the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image;

[0015] A face detection module, used for inputting the second image into a face detection model to obtain a face region image in the second image;

[0016] A feature extraction module, used to extract features from the face region image by using a local binary pattern algorithm to obtain a face feature image;

[0017] A face recognition module, used to perform face recognition in a preset face feature database based on the face feature image, and obtain an identity authentication result of the face payment request;

[0018] The payment response module is used to respond to the face-scanning payment request when the identity authentication result indicates that the customer identity is legal, so as to complete the face-scanning payment operation.

[0019] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0020] When receiving a face payment request, obtaining a first image of the customer through the POS terminal;

[0021] Reconstructing the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image;

[0022] Inputting the second image into a face detection model to obtain a face region image in the second image;

[0023] Extracting features from the face region image using a local binary pattern algorithm to obtain a face feature image;

[0024] Performing face recognition in a preset face feature database based on the face feature image to obtain an identity authentication result of the face payment request;

[0025] When the identity verification result indicates that the customer's identity is legitimate, a response is made to the face-scanning payment request to complete the face-scanning payment operation.

[0026] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0027] When receiving a face payment request, obtaining a first image of the customer through the POS terminal;

[0028] Reconstructing the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image;

[0029] Inputting the second image into a face detection model to obtain a face region image in the second image;

[0030] Extracting features from the face region image using a local binary pattern algorithm to obtain a face feature image;

[0031] Performing face recognition in a preset face feature database based on the face feature image to obtain an identity authentication result of the face payment request;

[0032] When the identity verification result indicates that the customer's identity is legitimate, a response is made to the face-scanning payment request to complete the face-scanning payment operation.

[0033] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: when a face-swiping payment request is received, a first image of the customer is obtained through the POS terminal; the first image is reconstructed through an image super-resolution reconstruction algorithm, so that a high-quality, high-resolution second image can be obtained even under a low-resolution camera, thereby ensuring the accuracy of subsequent face recognition; the second image is input into a face detection model to obtain a face area image in the second image; feature extraction is performed on the face area image through a local binary pattern algorithm to obtain a face feature image, thereby enhancing the robustness of the system to environmental factors such as lighting changes; face recognition is performed in a preset face feature database based on the face feature image, the customer is quickly authenticated and an identity authentication result of the face-swiping payment request is generated; when the identity authentication result indicates that the customer's identity is legal, it means that the customer has opened the face-swiping payment service, and can respond to the face-swiping payment request to complete the face-swiping payment operation, thereby ensuring the security and efficiency of the payment process, and solving the problems of high equipment cost and low recognition accuracy in the face-swiping payment technology of POS terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0036] Figure 2 is a flow chart of an embodiment of a face-scanning payment method according to the present application;

[0037] Figure 3 is a structural schematic diagram of an embodiment of a face-scanning payment device according to the present application;

[0038] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0042] like Figure 1As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0043] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0044] The terminal devices 101, 102, 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc. In the present application, the terminal device may also be a POS terminal, i.e., a POS machine.

[0045] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0046] It should be noted that the face-scanning payment method provided in the embodiment of the present application is generally executed by a server, and accordingly, the face-scanning payment device is generally set in the server.

[0047] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0048] Continue to refer Figure 2 , shows a flow chart of an embodiment of a face-scanning payment method according to the present application. The face-scanning payment method comprises the following steps:

[0049] Step S201, when a face payment request is received, a first image of the customer is obtained through the POS terminal.

[0050] In this embodiment, the electronic device (e.g. Figure 1 The server shown in the figure) can communicate with the terminal device through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0051] Specifically, at the beginning of the face payment process, the user selects face payment as a payment method and triggers a face payment request. The camera on the POS terminal (POS, i.e., point of sale terminal) is activated to capture the facial image of the current customer and obtain a first image.

[0052] Considering that the memory of a simple POS terminal is relatively small, the POS terminal sends the first image to the server, and the server in the cloud performs image processing.

[0053] Step S202: reconstruct the first image using an image super-resolution reconstruction algorithm to obtain a high-resolution second image.

[0054] Specifically, since the camera of the POS terminal in actual application may be limited by factors such as cost and volume, its resolution may not be sufficient to meet the needs of high-precision face recognition. Therefore, after acquiring the first image, the server uses an image super-resolution reconstruction algorithm to process the image. The image super-resolution reconstruction algorithm can utilize redundant information and prior knowledge in the image to reconstruct the low-resolution first image into a high-resolution second image through interpolation, learning, etc. The high-resolution second image can provide more details, which helps to improve the accuracy of face recognition.

[0055] Step S203: input the second image into a face detection model to obtain a face region image in the second image.

[0056] Specifically, after obtaining the high-resolution second image, it is input into a pre-trained face detection model. The face detection model is a deep learning model that can automatically detect the position and size of the face in the image, and then accurately crop the face area image from the second image.

[0057] The face detection model can adopt the YOLOv3 network structure, and the YOLOv3 network consists of a feature extraction network, a multi-scale detection network and a YOLO layer. In this application, all parts of the output of the last bneck module in the lightweight model MobileNetv3 are selected as submodules for extracting feature information. Then the multi-scale detection network composed of convolutional layers and upsampling layers needs to establish a transfer layer in MobileNetv3 for FPN connection, and select the feature maps of the 10th bneck module, the 12th bneck module and the last bneck module in MobileNetv3 as transfer layers connected to FPN.

[0058] In order to better perform multi-scale feature fusion, a feature fusion algorithm based on an efficient channel attention mechanism is introduced into the detection network, and an attention block is added before concat; finally, the YOLO layer performs detection output.

[0059] The image size of the first image is expected to be 416×416, and the depth of the output feature map is 3×(5+1)=18. There are 3 candidate boxes. Each candidate box contains 5 parameters (x, y, w, h, confidence), namely the center coordinates, width, height and confidence. After a series of bottleneck modules (bneck), the feature map dimension output by the 15th bottleneck layer is 13×13×160. After the convolution processing of the detection network, the number of channels is reduced to 18, and the output result of the YOLO layer Output1 is obtained. Its dimension is 13×13×18, which is suitable for detecting large-scale faces; the feature map dimension output by the 12th bottleneck layer is 26×26×112, which is concatenated with the feature map of the small-scale 13×13×160 and the feature map of 26×26×40 obtained by convolution and upsampling, and the feature map is 26×26×152. After the convolution layer, the feature map is processed. After processing, the number of channels is reduced to 18, and the output result Output2 of the YOLO layer is obtained, whose dimension is 13×13×18, which is suitable for detecting medium-sized faces; the feature map scale output after the 10th bottleneck layer is 52×52×80, and the feature map of the medium scale 26×26×112 is convolved and up-sampled to 52×52×20, and then spliced ​​together to obtain a feature map of 52×52×100. After processing the convolution layer, the number of channels is reduced to 18, and the output result Output3 of the YOLO layer is obtained, whose dimension is 52×52×18, which is suitable for detecting small-scale faces. Through face detection of different scales and extraction of facial feature values, the facial feature values ​​are compared with those in the library.

[0060] Step S204, extracting features from the face region image using a local binary pattern algorithm to obtain a face feature image.

[0061] Specifically, facial feature extraction is a key step in face recognition. The server uses the Local Binary Patterns (LBP) algorithm to extract features from the face area image. The LBP algorithm generates a code that can describe the local texture features of the image by comparing the grayscale value of each pixel in the image with the surrounding pixels. This code is robust to changes in lighting and can effectively extract the key features of the face to form a facial feature image.

[0062] Step S205, performing face recognition in a preset face feature database based on the face feature image to obtain an identity authentication result of the face payment request.

[0063] Specifically, after obtaining the facial feature image, it is compared with the data in the preset facial feature database. The facial feature database contains the facial feature information of the stored customers. The stored customers may be customers who have activated face-scanning payment. Based on the facial feature image, the server queries the facial feature database to see whether the customer is a customer who has activated face-scanning payment, and then generates an identity authentication result for the face-scanning payment request.

[0064] Step S206, when the identity verification result indicates that the customer's identity is legitimate, respond to the face-scanning payment request to complete the face-scanning payment operation.

[0065] Specifically, the server responds based on the identity verification result. If the identity verification result indicates that the client's identity is legitimate, that is, the client is a registered user in the preset facial feature database (that is, a client who has activated face-scanning payment), the face-scanning payment request is accepted and corresponding payment operations are performed, such as deducting money, recording transaction information, etc., to complete the entire face-scanning payment process.

[0066] In this embodiment, when a face-swiping payment request is received, a first image of the customer is obtained through the POS terminal; the first image is reconstructed through an image super-resolution reconstruction algorithm, so that a high-quality, high-resolution second image can be obtained even under a low-resolution camera, thereby ensuring the accuracy of subsequent face recognition; the second image is input into a face detection model to obtain a face area image in the second image; features are extracted from the face area image through a local binary pattern algorithm to obtain a face feature image, thereby enhancing the robustness of the system to environmental factors such as lighting changes; face recognition is performed in a preset face feature database based on the face feature image, the customer is quickly authenticated and an identity authentication result of the face-swiping payment request is generated; when the identity authentication result indicates that the customer's identity is legitimate, it means that the customer has opened the face-swiping payment service, and can respond to the face-swiping payment request to complete the face-swiping payment operation, thereby ensuring the security and efficiency of the payment process, and solving the problems of high equipment cost and low recognition accuracy in the face-swiping payment technology of POS terminals.

[0067] Furthermore, the above step S202 may include: inputting the first image into a shallow feature extraction network to obtain shallow features of the first image; inputting the shallow features into a convolutional attention network for feature extraction to obtain attention weighted features; performing sub-pixel convolution on the attention weighted features to perform image reconstruction to obtain a high-resolution second image.

[0068] Specifically, the present application realizes image super-resolution reconstruction through a lightweight multi-attention fusion network strategy, and enhances the first image through the super-resolution reconstructed image. The present application can adopt a CNN-based super-resolution reconstruction algorithm to enhance the low-resolution image, which mainly consists of three parts: a shallow feature extraction network (SFE), a convolutional attention network (including several multi-scale convolutional attention layers MCAB) and an image upsampling reconstruction network (sub-pixel).

[0069] First, the first image is input into a shallow feature extraction network. The shallow feature extraction network usually contains several initial convolutional layers and pooling layers to quickly capture the basic structure and edge information of the image and generate shallow features of the first image:

[0070] I0=C 3×3 (X) (1)

[0071] Where X represents the first image, C 3×3 represents a 3×3 convolution operation, and I0 represents shallow features. Shallow features provide the necessary foundation for subsequent high-level feature extraction.

[0072] Then the shallow features are input into the convolutional attention network. The convolutional attention network is a network structure that combines convolution operations and attention mechanisms. It can automatically learn which areas in the image are more important for subsequent tasks (such as face recognition) and give higher weights to these areas. Through the convolutional attention network, the attention-weighted feature I can be obtained. deep ,The attention weighted features not only contain the basic information of the image, but also highlight the key areas that are closely related to the recognition task.

[0073] Finally, the attention-weighted features are upsampled using sub-pixel convolution technology to achieve super-resolution reconstruction of the image. Sub-pixel convolution is an efficient image magnification method that rearranges and combines the pixels of multiple low-resolution feature maps in a sub-pixel shuffle layer through sub-pixel (the specific component that implements the sub-pixel convolution method) to generate a high-resolution output image. Sub-pixel convolution not only retains important details of the image, but also avoids the blur and jagged effects that may be introduced by traditional interpolation methods. Through this step, a high-resolution second image is obtained, which provides high-quality input data for subsequent face detection and recognition. The formula is as follows:

[0074] I SR =F up (I deep +I0) (2)

[0075] Among them, I SR represents the high-resolution second image, F up () indicates sub-pixel function, I deep represents the attention-weighted feature, and I0 represents the shallow feature.

[0076] In this embodiment, shallow feature extraction lays a data foundation for subsequent processing; the convolutional attention network improves the robustness to environmental changes and individual differences by automatically learning and strengthening key area features; sub-pixel convolution achieves high-quality image magnification and retains more detail information that is conducive to recognition; the introduction of the image super-resolution reconstruction algorithm can significantly improve the image quality and face recognition accuracy in the face-swiping payment process when the resolution of the first image collected by the POS terminal is low.

[0077] Furthermore, the above-mentioned step of inputting shallow features into the convolutional attention network for feature extraction to obtain attention-weighted features may include: inputting shallow features into the convolutional attention network to perform multi-scale feature extraction and attention weighting on the shallow features through each convolutional attention layer in the convolutional attention network to obtain attention feature maps corresponding to each convolutional attention layer; fusing each attention feature map to obtain a multi-scale attention feature map; performing a convolution operation on the multi-scale attention feature map to obtain a convolution feature; and adding the convolution feature to the shallow feature to obtain an attention-weighted feature.

[0078] Specifically, the shallow features (containing the basic information of the image) are input into the convolutional attention network. The convolutional attention network consists of multiple convolutional attention layers MCAB, each of which is designed with different convolution kernel sizes and strides to achieve multi-scale feature extraction of shallow features. This multi-scale design helps capture information of different scales in the image, thereby improving the richness and robustness of the features.

[0079] In the convolutional attention layer, in addition to the convolution operation, an attention mechanism is also introduced, which allows the neural network to automatically assign different weights according to the importance of the features during the convolution process, that is, to perform attention weighting. In this way, the neural network can learn which features are more critical to the current task, and thus give them higher priority in subsequent processing. After being processed by each convolutional attention layer, a series of corresponding attention feature maps will be obtained. These attention feature maps not only contain the information of the original shallow features, but also incorporate the effect of attention weighting, making the features more focused on key areas.

[0080] Through specific fusion strategies (such as feature concatenation, weighted summation, etc.), these attention feature maps are integrated into a multi-scale attention feature map, thereby integrating feature information at different scales to form a more comprehensive and powerful feature representation.

[0081] A further convolution operation is performed on the multi-scale attention feature map to further extract its deep information and generate convolution features. In one embodiment, a 1×1 convolution operation may be performed on the multi-scale attention feature map first, and then a 3×3 convolution operation may be performed to obtain the convolution features.

[0082] Subsequently, the convolutional features are added to the original shallow features (this process is also called feature fusion or residual connection) to retain the useful information in the shallow features and introduce new contextual information to obtain the attention-weighted features. The formula is as follows:

[0083] I deep =C 3×3 (C 1×1 (F cat (I1,I2,…,I k )))+I0 (4)

[0084] Among them, C 1×1 represents a 1×1 convolution operation; C 3×3 Represents a 3×3 convolution operation; I k is the attention feature map output by the kth MCAB, F cat () is the fusion function; I deep represents the attention weighted feature.

[0085] The attention-weighted features retain the original details of the image and also incorporate high-level semantic information extracted by the attention mechanism and multi-layer convolution, providing strong support for subsequent task processing.

[0086] In this embodiment, through multi-scale feature extraction and attention weighting, the extracted features are more focused on the key areas of the image, while retaining rich contextual information and improving the expressiveness of the features; multi-scale feature extraction helps to better cope with interference factors such as scale changes and deformations in the image, while the attention mechanism enables the model to ignore unimportant information interference, thereby enhancing the robustness and generalization ability of the model; by combining the convolutional attention network and multi-layer feature fusion, deep processing of shallow image features is achieved, thereby improving the feature expression ability.

[0087] In this application, MCAB consists of a branch residual multi-scale convolutional network (BRMB), a series of nonlinear free activation networks (NAFB), and a cascaded channel attention network (CA) and enhanced spatial attention network (ESA). It uses three different channels: 1×1 convolution, single 3×3 convolution, and double 3×3 convolution (two 3×3 convolutions are equivalent to one 5×5 convolution) to extract features at different scales of view and enrich the feature information of the image; at the same time, the single 3×3 convolution and double 3×3 convolution channels respectively add residual connections to enhance the original feature information for better backward transmission, and prevent the problem of gradient disappearance due to network deepening. The feature I obtained after the input feature passes through BRMB BRMB It can be expressed as:

[0088] I BRMB =F cat (C 1×1 (I in ),C 3×3 (I in )+I in ,C 3×3 (C 3×3 (I in )+I in )) (5)

[0089] Among them, I in represents the input feature, F cat () is the fusion function, C 1×1 represents a 1×1 convolution operation; C 3×3 Represents a 3×3 convolution operation.

[0090] NAFB consists of layer normalization layer (layerNorm), point convolution layer and depth convolution layer, SimpleGate module and simple channel attention (SCA). SCA consists of average pooling layer, 1×1 convolution layer and Sigmoid layer, which can effectively guide the network to focus on important feature information through simple network layers; SimpleGate module divides the input features into two equally divided channels and multiplies them. Compared with using convolution operation to reduce the number of channels, SimpleGate can greatly reduce the computational cost. The calculation formula is as follows:

[0091] I DC =C 1×1 (F SCA (F Simple (F D3×3 (C 1×1 (F LN (I BRMB )))))))+I BRMB (6)

[0092] I NAFB =C 1×1 (F Simple (C 1×1 (F LN (I DC ))))+I DC (7)

[0093] Among them, F LN () represents the normalization function, F D3×3 () represents a 3×3 depth-wise separable convolution function, F Simple () and F SCA () are channel fusion and simple channel attention functions respectively.

[0094] The features obtained by NAFB are further extracted through cascaded channel attention (CA) and enhanced spatial attention (ESA). CA uses average pooling and maximum pooling to extract image channel features, and then combines with the ESA module. ESA pays more attention to local spatial information and can better extract high-frequency information. Therefore, the output feature I of MCAB out It is expressed as:

[0095] I out =F ESA (F CA (I NAFB )) (8)

[0096] Among them, F CA () and F ESA () represent CA and ESA functions respectively.

[0097] Furthermore, the above-mentioned step S204 may include: calculating the local binary pattern value of each central pixel in the face area image by means of a circular local binary pattern algorithm; generating a local binary pattern image or a local binary pattern histogram of the face area image according to the local binary pattern value of each central pixel; and determining the local binary pattern image or the local binary pattern histogram as a face feature image.

[0098] Specifically, considering the different venues and environmental light factors involved in daily payment scenarios; different light factors lead to large differences between face images and registered images in the database (existing face feature maps), resulting in the intra-class distance being greater than the inter-class distance, affecting the final recognition result. This application introduces the local binary pattern (LBP) algorithm to eliminate the recognition error caused by light to the face through a feature extraction algorithm.

[0099] This application can capture subtle texture features and structural information in facial area images through the Circular Local Binary Patterns (CLBP) algorithm. The CLBP algorithm traverses every pixel in the facial area image, takes each pixel as the center, and defines a circular neighborhood range around it (this range can be set according to specific needs, such as the radius size). Then, the algorithm compares the grayscale value of the central pixel with the pixel points in its circular neighborhood, and performs binarization based on the comparison result (usually greater than, less than or equal to), generating a binary code around the central pixel, that is, the local binary pattern value of the point.

[0100] This process takes into account the spatial relationship between pixels and also enhances the algorithm's robustness to rotation changes through a circular neighborhood. Since different starting points produce different binary combinations, the resulting LBP values ​​(local binary pattern values) are also different. This application performs rotation calculations on the circular neighborhood, compares each set of LBP values ​​obtained in this process, and takes the minimum value as the LBP value of the pixel. The rotation-invariant CLBP algorithm not only has grayscale invariance, but also has good robustness to image rotation.

[0101] Subsequently, a new image, namely a local binary pattern image (CLBP Image), can be generated based on the local binary pattern values ​​of all central pixels in the image, where the value of each pixel is directly mapped from its local binary pattern code, or the frequency of occurrence of different local binary pattern codes can be counted to generate a local binary pattern histogram (CLBP Histogram). Both forms effectively characterize the texture features of the face area and provide an important basis for subsequent face recognition or classification.

[0102] In this embodiment, the accuracy and robustness of face recognition are improved by introducing a circular local binary pattern algorithm for facial feature extraction; the circular local binary pattern algorithm considers the spatial relationship and grayscale differences between pixels to generate a feature image or histogram with rich texture information, which can effectively distinguish different faces and resist interference from external factors such as changes in lighting and expression to a certain extent; in addition, the circular neighborhood definition method is also adaptable to image rotation, ensuring the accuracy of subsequent face recognition.

[0103] Furthermore, the above-mentioned step S205 may include: obtaining all stored facial feature images from a preset facial feature database; calculating the similarity between the facial feature image and each stored facial feature image; obtaining the highest similarity among each similarity; when the highest similarity is greater than a preset similarity threshold, using the customer identity corresponding to the stored facial feature image corresponding to the highest similarity as the identity authentication result of the face-swiping payment request.

[0104] Specifically, a database containing a large amount of facial feature information of registered customers is set up in the server. This feature information is usually stored in a digital form (such as a feature vector), and each feature vector corresponds to a specific customer identity. The database is constructed based on the key features extracted by the face recognition algorithm based on the face image provided by the customer during the registration or authorization process.

[0105] When the system receives a face payment request, it captures the customer's current facial image through the camera of the POS terminal and uses the face recognition algorithm to extract key facial feature images from the image.

[0106] Next, the system compares the extracted facial feature image with each stored facial feature map in the preset facial feature database and calculates the similarity between them. The calculation of similarity relies on cosine similarity, Euclidean distance, etc., which aims to quantify the closeness between two feature maps.

[0107] Among all the calculated similarities, the highest one is selected. The highest similarity represents the maximum matching degree between the current facial features and a facial feature stored in the database. This highest similarity is compared with the preset similarity threshold. If the highest similarity is greater than or equal to this threshold, the system considers that the facial features of the current customer are highly similar to those of a registered customer in the database, and uses the identity of the registered user as the authentication result of the face payment request, ensuring the security of the payment process and preventing unauthorized access.

[0108] In this embodiment, through precise facial feature comparison and similarity threshold determination, the customer's identity authentication result is generated to confirm whether the customer has activated face-scanning payment, thereby preventing identity impersonation and fraud and ensuring the security of the face-scanning payment process.

[0109] Furthermore, after the above-mentioned step of obtaining the identity authentication result of the face-swiping payment request, it can also include: when the identity authentication result contains more than one stored customer identity, sending an identity error notification to the POS terminal, and displaying the identity error notification through the POS terminal.

[0110] Specifically, when the identity verification result contains more than one stored customer identity, it may cause misidentification or fraud risks. The system will send an identity error notification to the POS terminal, informing the POS terminal that the current identity verification process has encountered a problem and requires further processing or intervention.

[0111] The system will display the identity error notification to the customer through the POS terminal so that the customer can re-perform facial recognition, or enter other identity authentication information (such as password or fingerprint), or contact customer service for resolution.

[0112] In this embodiment, when the identity authentication result contains more than one stored customer identity, it indicates that identity confusion may occur, and an identity error notification is sent and displayed to the POS terminal to prevent possible mispayment or fraud, thereby improving the security and accuracy of face-swiping payment.

[0113] Furthermore, the above-mentioned step of responding to the face-scanning payment request may include: sending first verification information to a customer terminal held by the customer; obtaining second verification information input by the customer through the POS terminal; when the first verification information and the second verification information match, verifying the face-scanning payment request from a preset verification dimension to obtain a multi-dimensional verification result; when it is determined that the face-scanning payment request has passed the verification according to the multi-dimensional verification result, responding to the face-scanning payment request.

[0114] Specifically, the system sends the first verification information to the client terminal (such as a smart phone) held by the client. This information may be a verification code, a temporary password, or a specific encrypted information based on the client's identity. The client needs to check and record this information on the client terminal for subsequent verification.

[0115] Next, the system obtains the second verification information input by the customer through the POS terminal. This usually requires the customer to enter the first verification information received previously on the POS terminal, or submit a certificate matching the first verification information through other means (such as scanning a QR code, NFC touch, etc.).

[0116] After confirming that the first verification information and the second verification information match, the multi-dimensional verification process begins. This verification process comprehensively evaluates the face-scanning payment request from multiple preset verification dimensions, including but not limited to whether the second verification information is expired, whether the account balance is sufficient, whether the customer's payment account status is normal, whether the payment amount exceeds the limit, etc. Each dimension can be set with different verification rules and thresholds to ensure the rationality and security of the payment request.

[0117] Finally, when the results of all verification dimensions indicate that the face payment request is safe and reasonable, the system will respond to the request and complete the payment process. The system can also return the payment result to the POS terminal for display and broadcast, print vouchers, etc.; customers can receive and view consumption records at the customer terminal.

[0118] In this embodiment, sending and verifying the first verification information through the client terminal can effectively prevent unauthorized access and fraud, ensuring that only legitimate users can initiate payment requests; multi-dimensional verification further refines the verification process of payment requests, evaluates their rationality and risk from multiple angles, effectively reduces possible misjudgments and loopholes in the payment process, and improves the security and protection capabilities of the face-swiping payment system.

[0119] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned first image, the above-mentioned first image can also be stored in a node of a blockchain.

[0120] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0121] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0122] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0124] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0125] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a face-scanning payment device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0126] like Figure 3 As shown, the face-swiping payment device 300 described in this embodiment includes: a first acquisition module 301, an image reconstruction module 302, a face detection module 303, a feature extraction module 304, a face recognition module 305 and a payment response module 306, wherein:

[0127] The first acquisition module 301 is used to acquire a first image of the customer through the POS terminal when receiving a face payment request.

[0128] The image reconstruction module 302 is used to reconstruct the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image.

[0129] The face detection module 303 is used to input the second image into the face detection model to obtain a face region image in the second image.

[0130] The feature extraction module 304 is used to extract features from the face region image by using a local binary pattern algorithm to obtain a face feature image.

[0131] The face recognition module 305 is used to perform face recognition in a preset face feature database based on the face feature image to obtain the identity authentication result of the face payment request.

[0132] The payment response module 306 is used to respond to the face-scanning payment request when the identity verification result indicates that the customer's identity is legal, so as to complete the face-scanning payment operation.

[0133] In this embodiment, when a face-swiping payment request is received, a first image of the customer is obtained through the POS terminal; the first image is reconstructed through an image super-resolution reconstruction algorithm, so that a high-quality, high-resolution second image can be obtained even under a low-resolution camera, thereby ensuring the accuracy of subsequent face recognition; the second image is input into a face detection model to obtain a face area image in the second image; features are extracted from the face area image through a local binary pattern algorithm to obtain a face feature image, thereby enhancing the robustness of the system to environmental factors such as lighting changes; face recognition is performed in a preset face feature database based on the face feature image, the customer is quickly authenticated and an identity authentication result of the face-swiping payment request is generated; when the identity authentication result indicates that the customer's identity is legitimate, it means that the customer has opened the face-swiping payment service, and can respond to the face-swiping payment request to complete the face-swiping payment operation, thereby ensuring the security and efficiency of the payment process, and solving the problems of high equipment cost and low recognition accuracy in the face-swiping payment technology of POS terminals.

[0134] In some optional implementations of this embodiment, the image reconstruction module 302 may include:

[0135] The shallow extraction submodule is used to input the first image into the shallow feature extraction network to obtain the shallow features of the first image.

[0136] The feature extraction submodule is used to input shallow features into the convolutional attention network for feature extraction to obtain attention-weighted features.

[0137] The image reconstruction submodule is used to perform sub-pixel convolution on the attention-weighted features to reconstruct the image and obtain a high-resolution second image.

[0138] In some optional implementations of this embodiment, the feature extraction submodule may include:

[0139] The feature extraction unit is used to input shallow features into the convolutional attention network, so as to perform multi-scale feature extraction and attention weighting on the shallow features through each convolutional attention layer in the convolutional attention network, and obtain attention feature maps corresponding to each convolutional attention layer.

[0140] The feature map fusion unit is used to fuse the attention feature maps to obtain a multi-scale attention feature map.

[0141] The feature map convolution unit is used to perform convolution operations on multi-scale attention feature maps to obtain convolution features.

[0142] The feature addition unit is used to add the convolutional features to the shallow features to obtain the attention-weighted features.

[0143] In some optional implementations of this embodiment, the feature extraction module 304 may include:

[0144] The calculation submodule is used to calculate the local binary pattern value of each central pixel point in the face area image through a circular local binary pattern algorithm.

[0145] The generating submodule is used to generate a local binary pattern image or a local binary pattern histogram of the face area image according to the local binary pattern value of each central pixel point.

[0146] The determination submodule is used to determine the local binary pattern image or the local binary pattern histogram as a facial feature image.

[0147] In some optional implementations of this embodiment, the face recognition module 305 may include:

[0148] The stored acquisition submodule is used to obtain all stored facial feature maps from a preset facial feature database.

[0149] The similarity calculation submodule is used to calculate the similarity between the face feature image and each stored face feature image.

[0150] The highest acquisition submodule is used to obtain the highest similarity among various similarities.

[0151] The identity determination submodule is used to use the customer identity corresponding to the stored facial feature map corresponding to the highest similarity as the identity authentication result of the face-swiping payment request when the highest similarity is greater than a preset similarity threshold.

[0152] In some optional implementations of this embodiment, the face-scanning payment device 300 may further include:

[0153] The error notification module is used to send an identity error notification to the POS terminal when the identity authentication result contains more than one stored customer identity, and display the identity error notification through the POS terminal.

[0154] In some optional implementations of this embodiment, the payment response module 306 may include:

[0155] The information sending submodule is used to send the first verification information to the client terminal held by the client.

[0156] The information receiving submodule is used to obtain the second verification information input by the customer through the POS terminal.

[0157] The request verification submodule is used to verify the face-swiping payment request from a preset verification dimension when the first verification information and the second verification information match, and obtain a multi-dimensional verification result.

[0158] The request response submodule is used to respond to the face-scanning payment request when it is determined that the face-scanning payment request has passed the verification based on the multi-dimensional verification results.

[0159] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0160] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0161] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0162] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the face-swiping payment method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0163] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the face-swiping payment method.

[0164] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0165] The computer device provided in this embodiment can execute the above-mentioned face-scanning payment method. Here, the face-scanning payment method can be the face-scanning payment method of each of the above-mentioned embodiments.

[0166] In this embodiment, when a face-swiping payment request is received, a first image of the customer is obtained through the POS terminal; the first image is reconstructed through an image super-resolution reconstruction algorithm, so that a high-quality, high-resolution second image can be obtained even under a low-resolution camera, thereby ensuring the accuracy of subsequent face recognition; the second image is input into a face detection model to obtain a face area image in the second image; features are extracted from the face area image through a local binary pattern algorithm to obtain a face feature image, thereby enhancing the robustness of the system to environmental factors such as lighting changes; face recognition is performed in a preset face feature database based on the face feature image, the customer is quickly authenticated and an identity authentication result of the face-swiping payment request is generated; when the identity authentication result indicates that the customer's identity is legitimate, it means that the customer has opened the face-swiping payment service, and can respond to the face-swiping payment request to complete the face-swiping payment operation, thereby ensuring the security and efficiency of the payment process, and solving the problems of high equipment cost and low recognition accuracy in the face-swiping payment technology of POS terminals.

[0167] The present application also provides another implementation, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the face-swiping payment method as described above.

[0168] In this embodiment, when a face-swiping payment request is received, a first image of the customer is obtained through the POS terminal; the first image is reconstructed through an image super-resolution reconstruction algorithm, so that a high-quality, high-resolution second image can be obtained even under a low-resolution camera, thereby ensuring the accuracy of subsequent face recognition; the second image is input into a face detection model to obtain a face area image in the second image; features are extracted from the face area image through a local binary pattern algorithm to obtain a face feature image, thereby enhancing the robustness of the system to environmental factors such as lighting changes; face recognition is performed in a preset face feature database based on the face feature image, the customer is quickly authenticated and an identity authentication result of the face-swiping payment request is generated; when the identity authentication result indicates that the customer's identity is legitimate, it means that the customer has opened the face-swiping payment service, and can respond to the face-swiping payment request to complete the face-swiping payment operation, thereby ensuring the security and efficiency of the payment process, and solving the problems of high equipment cost and low recognition accuracy in the face-swiping payment technology of POS terminals.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0170] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A face-scanning payment method, characterized in that: The steps include: When receiving a face payment request, obtaining a first image of the customer through the POS terminal; Reconstructing the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image; Inputting the second image into a face detection model to obtain a face region image in the second image; Extracting features from the face region image using a local binary pattern algorithm to obtain a face feature image; Performing face recognition in a preset face feature database based on the face feature image to obtain an identity authentication result of the face payment request; When the identity verification result indicates that the customer's identity is legitimate, a response is made to the face-scanning payment request to complete the face-scanning payment operation.

2. The face recognition payment method according to claim 1, characterized in that: The step of reconstructing the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image comprises: Inputting the first image into a shallow feature extraction network to obtain shallow features of the first image; Inputting the shallow features into the convolutional attention network for feature extraction to obtain attention weighted features; Sub-pixel convolution is performed on the attention weighted features to perform image reconstruction to obtain a high-resolution second image.

3. The face recognition payment method according to claim 2, characterized in that: The step of inputting the shallow features into the convolutional attention network for feature extraction to obtain the attention weighted features comprises: Inputting the shallow features into a convolutional attention network, so as to perform multi-scale feature extraction and attention weighting on the shallow features through each convolutional attention layer in the convolutional attention network, and obtaining attention feature maps corresponding to each convolutional attention layer; Fuse each attention feature map to obtain a multi-scale attention feature map; Performing a convolution operation on the multi-scale attention feature map to obtain a convolution feature; The convolutional features are added to the shallow features to obtain attention-weighted features.

4. The face recognition payment method according to claim 1, characterized in that: The step of extracting features from the face region image by using a local binary pattern algorithm to obtain a face feature image comprises: Calculating the local binary pattern value of each central pixel in the face region image by a circular local binary pattern algorithm; Generating a local binary pattern image or a local binary pattern histogram of the face region image according to the local binary pattern values ​​of each central pixel point; The local binary pattern image or the local binary pattern histogram is determined as a facial feature image.

5. The face recognition payment method according to claim 1, characterized in that: The step of performing face recognition in a preset face feature database based on the face feature image to obtain the identity authentication result of the face payment request includes: Obtain all stored facial feature images from a preset facial feature database; Calculating the similarity between the facial feature image and each stored facial feature image; Obtain the highest similarity among all similarities; When the highest similarity is greater than a preset similarity threshold, the customer identity corresponding to the stored facial feature image corresponding to the highest similarity is used as the identity authentication result of the face-swiping payment request.

6. The face recognition payment method according to claim 1, characterized in that: After the step of obtaining the identity authentication result of the face-scanning payment request, the method further includes: When the identity verification result includes more than one stored customer identity, an identity error notification is sent to the POS terminal, and the identity error notification is displayed by the POS terminal.

7. The face recognition payment method according to claim 1, characterized in that: The step of responding to the face recognition payment request includes: Sending first verification information to a client terminal held by the client; Acquiring second verification information input by the customer through the POS terminal; When the first verification information matches the second verification information, verify the face-scanning payment request from a preset verification dimension to obtain a multi-dimensional verification result; When it is determined that the face-scanning payment request has passed the verification according to the multi-dimensional verification result, the face-scanning payment request is responded to.

8. A face-scanning payment device, characterized in that: include: A first acquisition module, configured to acquire a first image of the customer through a POS terminal when receiving a face-scanning payment request; An image reconstruction module, used to reconstruct the first image by using an image super-resolution reconstruction algorithm to obtain a high-resolution second image; A face detection module, used for inputting the second image into a face detection model to obtain a face region image in the second image; A feature extraction module, used to extract features from the face region image by using a local binary pattern algorithm to obtain a face feature image; A face recognition module, used to perform face recognition in a preset face feature database based on the face feature image, and obtain an identity authentication result of the face payment request; The payment response module is used to respond to the face-scanning payment request when the identity authentication result indicates that the customer identity is legal, so as to complete the face-scanning payment operation.

9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the face-swiping payment method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the face-scanning payment method as described in any one of claims 1 to 7.

Images