A method, device and equipment for identifying willingness to pay by face scanning

Abstract

The embodiments of this specification disclose a method, device and equipment for identifying willingness to pay by face recognition. The scheme includes: obtaining a face recognition image, and determining candidates to be identified in the face recognition image; generating corresponding mask images according to the areas where each candidate is located in the face recognition image to distinguish the areas from other areas in the face recognition image; extracting features of the face recognition image, and obtaining fused features according to the features of the face recognition image and the mask image; and identifying whether each candidate has the willingness to pay by face recognition according to the fused features. It can improve the security of face recognition payment.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and equipment for identifying willingness to pay by face recognition. Background Art

[0002] With the development of computer and Internet technology, many businesses can be conducted online, which has promoted the development of various online business platforms. Among them, face payment refers to a new payment method based on artificial intelligence, machine vision, 3D sensing, big data and other technologies. By using face recognition as a payment method for identity authentication, it brings great convenience to users and is widely loved by users.

[0003] At present, in the face-scanning payment scenario, after the user to pay turns on the face-scanning payment, he needs to stand in front of the device with the face-scanning payment function to perform face recognition. However, during the face-scanning process, there may be multiple users standing in front of the device, which will cause multiple users to appear in the face-scanning image collected by the device. At this time, when the device performs face recognition on the face-scanning image, it is difficult to determine which user is the current user to pay, that is, which user has the willingness to pay by face-scanning. In other words, only the current user to pay has the willingness to pay by face-scanning, while other users do not have the willingness to pay by face-scanning.

[0004] Based on this, face-scanning payment intention recognition is an important part of face-scanning security in the payment system, which helps to improve the face-scanning security experience. However, if the device recognizes other users and identifies other users, mistaken face-scanning payment will occur, thereby reducing the security of face-scanning payment.

[0005] Based on this, a more secure identification solution is needed for face-scanning payment. Summary of the invention

[0006] One or more embodiments of this specification provide a method, device, equipment and storage medium for identifying willingness to pay by face recognition, so as to solve the following technical problem: a more secure identification solution is needed for face recognition payment.

[0007] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0008] One or more embodiments of this specification provide a method for identifying willingness to pay by face recognition, including:

[0009] Acquire a face scan image, and determine a candidate to be identified in the face scan image;

[0010] According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image;

[0011] Extracting features of the face scan image, and obtaining fusion features according to the features of the face scan image and the mask image;

[0012] According to the fusion features, it is identified whether each of the candidates is willing to pay by face recognition.

[0013] One or more embodiments of this specification provide a face-scanning payment willingness recognition device, including:

[0014] An acquisition module acquires a face scan image and determines a candidate to be identified in the face scan image;

[0015] A generation module, which generates corresponding mask images according to the regions where the candidates are located in the face-scanning image, so as to distinguish the regions from other regions in the face-scanning image;

[0016] An extraction module extracts features of the face scan image and obtains fusion features according to the features of the face scan image and the mask image;

[0017] The identification module identifies whether each of the candidates is willing to pay by face recognition according to the fusion features.

[0018] One or more embodiments of this specification provide a face-scanning payment willingness recognition device, including:

[0019] at least one processor; and,

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0022] Acquire a face scan image, and determine a candidate to be identified in the face scan image;

[0023] According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image;

[0024] Extracting features of the face scan image, and obtaining fusion features according to the features of the face scan image and the mask image;

[0025] According to the fusion features, it is identified whether each of the candidates is willing to pay by face recognition.

[0026] One or more embodiments of this specification provide a non-volatile computer storage medium for face-scanning payment willingness recognition, storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0027] Acquire a face scan image, and determine a candidate to be identified in the face scan image;

[0028] According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image;

[0029] Extracting features of the face scan image, and obtaining fusion features according to the features of the face scan image and the mask image;

[0030] According to the fusion features, it is identified whether each of the candidates is willing to pay by face recognition.

[0031] At least one of the above technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects:

[0032] By generating a corresponding mask map for the area where each candidate is located in the face scanning image, the feature information of the candidate can be made more distinct, thereby increasing the difference between those who have the willingness to pay by face scanning and those who do not. By fusing features, it is possible to identify whether each candidate has the willingness to pay by face scanning, thereby achieving an enhanced image contrast effect, thereby focusing attention on candidates who have the willingness to pay by face scanning, and being able to more accurately distinguish between candidates who have the willingness to pay by face scanning and candidates who do not have the willingness to pay in the face scanning image, and more specifically identify the willingness to pay by face scanning of candidates in the face scanning image, thereby enhancing the face scanning security experience and effectively ensuring the available security of the face scanning system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 A flowchart of a method for identifying willingness to pay by face recognition provided in one or more embodiments of this specification;

[0035] Figure 2 A schematic diagram of a framework of a face-scanning payment willingness recognition system provided in one or more embodiments of this specification;

[0036] Figure 3 A flowchart of a method for recognizing willingness to pay by face recognition based on end-to-end learning of a deep convolutional neural network provided for one or more embodiments of this specification;

[0037] Figure 4 A schematic diagram of the structure of a device for identifying willingness to pay by face recognition provided in one or more embodiments of this specification;

[0038] Figure 5 A schematic diagram of the structure of a face-scanning payment willingness recognition device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0039] The embodiments of this specification provide a method, device, equipment and storage medium for identifying willingness to pay by face recognition.

[0040] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0041] Figure 1 A flowchart of a method for identifying willingness to pay by face recognition provided in one or more embodiments of this specification. The process can be executed by an electronic device with a face recognition payment function, which can be a terminal with an image data processing function, for example, a mobile terminal such as a mobile phone, a tablet, a notebook, or a fixed terminal such as a desktop or a server. Some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0042] Figure 1 The process in may include the following steps:

[0043] S102: Acquire a face recognition image, and determine a candidate to be recognized in the face recognition image.

[0044] In one or more embodiments of the present specification, the electronic device may obtain the face-scanning image through a pre-installed camera device after receiving the face-scanning payment instruction, or the electronic device may generate the face-scanning payment instruction according to the payment order and obtain the face-scanning image through the camera device. The face-scanning image may be a single-frame image obtained from a video or image.

[0045] The candidates to be identified refer to users who need to pay relevant fees. It should be noted that if the candidates want to make face payment, they need to register their identity information on the corresponding client and enter their facial information. When the candidates turn on face payment, the client will recognize that the candidates are willing to make face payment and then authenticate the candidates through the pre-registered facial information.

[0046] That is, the face scanning image includes the candidate's facial information. By identifying the facial information, it can be determined whether the candidate is willing to pay by face scanning, and then the candidate's identity can be authenticated through the facial information.

[0047] The number of candidates in front of the camera device may be one or more. When the number of candidates is more than one, the face recognition image includes multiple candidates. When the number of candidates is one, the face recognition image includes one candidate. At the same time, the face recognition image includes not only the facial information of the candidate, but also other feature information of the candidate, such as torso information and limb information, and may also include other objects that do not need to be recognized, such as tables, chairs, hanging objects, etc. included in the environment where the candidate is located.

[0048] In addition, under normal circumstances, when an electronic device executes a single face-scanning payment instruction, it is to authenticate the identity of the specific candidate who currently enables face-scanning payment, and the specific candidate usually also has the willingness to pay by face-scanning. That is to say, when executing a single face-scanning payment instruction, even if the face-scanning image includes multiple candidates, not all of the multiple candidates have the willingness to pay by face-scanning. Only the specific candidate has the willingness to pay by face-scanning. It can be considered that the payment willingness of the specific candidate is safe, and the payment willingness of other candidates is unsafe.

[0049] For example, in public places, offline Internet of Things (IoT) face-scanning devices are usually used for face-scanning payment. Among them, face-scanning IoT devices in public places refer to devices with face-scanning functions set up in public consumption scenarios such as supermarkets / convenience stores / catering / hotels / campus education and medical care / campus education.

[0050] If candidate A clicks on face payment, face payment will be enabled, and the IoT face payment device will receive the face payment instruction and obtain the face image through the camera. Since it is in an open public place, there are multiple candidates queuing to pay, so the face image obtained by the IoT device is likely to include multiple candidates. However, among the multiple candidates, only candidate A is actually willing to pay by face. At this time, it is necessary to identify candidate A's willingness to pay by face and authenticate candidate A through his facial information, that is, other candidates do not actually have the willingness to pay by face.

[0051] Furthermore, if candidate B is ranked behind candidate A, then even if candidate B has no intention to pay by face recognition, during the face recognition payment authentication process, the camera device may capture candidate B at the same time as candidate A, resulting in the captured face recognition image including both candidate A and candidate B. If the electronic device does not identify candidate A as the face recognition user during the face recognition image recognition process, but mistakenly recognizes candidate B, then the electronic device will not identify whether candidate B has the intention to pay by face recognition, but directly authenticates candidate B. After the authentication is passed, payment will be made through candidate B's account, thereby mistakenly swiping candidate B's assets, resulting in candidate B's asset loss.

[0052] S104: Generate corresponding mask images according to the regions where the candidates are located in the face recognition image to distinguish the regions from other regions in the face recognition image.

[0053] In one or more embodiments of the present specification, the region may include appearance feature information of the candidate, such as face information, torso information, and limb information, but in order to increase the accuracy of the recognition result, the region mainly includes the face information of the candidate. At the same time, the region may be determined based on the position information of the candidate in the face scan image.

[0054] Since the mask operation of an image refers to recalculating the value of each pixel in the image through a mask kernel operator, the mask kernel operator characterizes the degree of influence of the domain pixel points on the new pixel value, and at the same time performs weighted averaging of the pixels according to the weight factor in the mask operator. Image mask operations are often used in image smoothing, edge detection, feature analysis and other areas. Therefore, mask operations can be used to distinguish the area where the candidate is located in the face scan image from other areas in the face scan image.

[0055] It should be noted that the area where a single candidate is located in the face-scanning image corresponds to a single mask image. That is to say, if there are multiple candidates in the face-scanning image, a corresponding mask image is generated for each candidate, and multiple mask images are finally obtained.

[0056] That is to say, in a single mask image, the area where the user is located can be distinguished from other areas, for example, the filling value of the area where the user is located is set to 1, and the filling value of other areas is set to 0 (other filling values ​​with higher discrimination can also be used). That is, by generating a mask image corresponding to each candidate, the characteristic information of the candidate can be more distinct, thereby increasing the difference between those who are willing to pay by face scanning and those who are not willing to pay by face scanning.

[0057] S106: extracting features of the face scan image, and obtaining fusion features according to the features of the face scan image and the mask image.

[0058] In one or more embodiments of this specification, how to extract features of the face scan image is not limited here. For example, the features of the face scan image are extracted by a feature extraction model. The features may include facial features, torso features, and limb features of each candidate. Among them, the facial features may be global features of the candidate's face. Recognizing the face by global features can improve the accuracy of the recognition result.

[0059] Of course, after obtaining the mask image, the fused features can be obtained by inputting the features of the face scan image and the mask image into a fused feature extraction model.

[0060] By extracting the features of the face scan image and combining the features of the face scan image with the mask image, it is equivalent to adding a new channel to the features of the face scan image, that is, increasing the number of channels of the features of the face scan image, thereby obtaining fused features. In the fused features, more attention is paid to the facial features of the corresponding candidates in the face scan image, so that the candidates with the willingness to pay by face scan and the candidates who do not have the willingness to pay by face scan can be more accurately distinguished in the face scan image.

[0061] S108: Identify, based on the fusion features, whether each of the candidates is willing to pay by face recognition.

[0062] In one or more embodiments of this specification, face image information is determined based on the fusion features, and the face image information includes the feature information of the candidate. Among them, more attention is paid to the feature information of the candidate corresponding to the mask image of the fusion feature, and the feature information mainly includes the face information of the corresponding candidate. That is, when a mask image is generated, a recognition process is performed to identify whether the candidate corresponding to the mask image has the willingness to pay by face recognition.

[0063] It should be noted that the preset rules can be used to identify whether the corresponding candidate has the willingness to pay by face recognition. For example, if the candidate's facial area is identified to be located in the middle area, the candidate is considered to have the willingness to pay by face recognition; or, if the candidate's facial area is identified to occupy most of the face recognition image, the candidate is considered to have the willingness to pay by face recognition; or, if the candidate's facial area is identified to occupy most of the face recognition image and the face angle meets the preset angle threshold, the candidate is considered to have the willingness to pay by face recognition.

[0064] Furthermore, the fused features can be input into the willingness to pay recognition model, the face recognition image information can be identified by the willingness to pay recognition model, and the processing results can be output according to preset rules. Then, the face recognition willingness to pay probability value can be generated according to the processing results, and the face recognition willingness to pay probability value can be used to determine whether the candidate has the willingness to pay by face recognition.

[0065] The processing result may be a vector generated by the willingness to pay recognition model and used to represent the probability value of the willingness to pay by face recognition.

[0066] For example, if the probability value is greater than a preset probability threshold, it can be considered that the candidate has the willingness to pay by face recognition, that is, the face recognition payment instruction of the electronic device is generated after the candidate turns on the face recognition payment. If the probability value is less than or equal to the preset probability threshold, it can be considered that the candidate does not have the willingness to pay by face recognition. That is, the face recognition payment instruction of the electronic device is not generated after the candidate turns on the face recognition payment, but is generated after other candidates turn on the face recognition payment.

[0067] Furthermore, if there are multiple candidates with probability values ​​greater than the preset probability, it means that the result of this face-scanning payment willingness recognition is not credible, and the authentication will fail. In any case, if there is no candidate with a probability value greater than the preset probability, it also means that the result of this face-scanning payment willingness recognition is not credible, and the authentication will fail.

[0068] pass Figure 1 The method generates a corresponding mask map for the area where each candidate is located in the face scanning image, which can make the feature information of the candidate more distinct, increase the difference between those with the willingness to pay by face scanning and those without the willingness to pay by face scanning, and identify whether each candidate has the willingness to pay by face scanning by fusing features, which can enhance the image contrast effect, thereby focusing attention on the candidates with the willingness to pay by face scanning, so as to more accurately distinguish the candidates with the willingness to pay by face scanning and the candidates without the willingness to pay by face scanning in the face scanning image, and more specifically identify the willingness to pay by face scanning of the candidates in the face scanning image, thereby enhancing the face scanning security experience.

[0069] based on Figure 1 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0070] In one or more embodiments of this specification, when generating a corresponding mask map,

[0071] Specifically, after the candidate is determined, the face area of ​​the candidate in the face scanning image is extracted. For example, the face of the candidate in the face scanning image is first extracted by a face extraction model, and then the face area of ​​the candidate is determined by the position information of the face. The face area is then processed to determine the face area selection box. The face area selection box can be displayed in a variety of ways, such as a circular box, a rectangular box, an irregular polygonal box, etc. However, there is a prerequisite that in order to ensure the accuracy of the recognition result, the face area selection box must completely surround the face area of ​​the candidate.

[0072] After obtaining the face area selection frame, the first filling area of ​​the mask image corresponding to the candidate will be determined according to the candidate's face area selection frame. The shape of the first filling area can be displayed in a variety of ways, which are not limited here, such as a circular area, a rectangular area, an irregular polygonal area, etc. However, there is a prerequisite that in order to ensure the accuracy of the recognition result, when determining the first filling area, it is necessary to combine the face area selection frame and be as close to the actual face area as possible.

[0073] After the first filling area is determined, a second filling area other than the first filling area is further determined in the face scanning image, and different filling values ​​are assigned to the first filling area and the second filling area.

[0074] It should be noted that in order to make the first filling area in the mask image as consistent as possible with the face area in the face recognition image, after assigning different filling values ​​to the first filling area and the second filling area, a mask image with a resolution consistent with the resolution of the face recognition image is generated.

[0075] Furthermore, since most of the face region is a circular region or an elliptical region, in order to make the first filling region more closely fit the candidate's face region, the first filling region is used as a circular region.

[0076] Specifically, when processing the face region and determining the face region selection frame, the face region selection frame is determined as a rectangular frame. After obtaining the face region of the rectangular frame, the face frame width and the face frame height are calculated according to the position of the rectangular frame in the face scan image, and the radius of the circular region is calculated according to the face frame width and the face frame height.

[0077] Among them, when calculating the radius of the circular area, since the face area is a circular area or an elliptical area, when the rectangular frame is initially generated, the face area is similar to the inscribed circle of the rectangular frame. Therefore, in order to restore the face area as much as possible and ensure that the first filling area includes the entire face area as much as possible, the maximum value between half the width of the rectangular frame and half the height of the frame is taken as the radius of the circular area.

[0078] Therefore, the center of the rectangular box is taken as the center of the circle, half the length of the longest side of the rectangular box is determined as the radius, and then the circular area formed by the center of the circle and the radius is determined as the first filling area of ​​the mask map corresponding to the candidate.

[0079] For example, suppose the position of the candidate's face rectangle in the face scan image is (x1, y1, x2, y2), where x1 and x2 are the position coordinates of the rectangle width on the x-axis, and y1 and y2 are the position coordinates of the rectangle height on the y-axis.

[0080] Then the expression for calculating the face frame width is w=x2-x1, where w is the face frame width. It should be noted that the position coordinate of x1 is smaller than the position coordinate of x2.

[0081] The expression for calculating the face frame height is h=y2-y1, where h is the face frame height. It should be noted that the position coordinate of y1 is smaller than the position coordinate of y2.

[0082] The expression for determining the radius of the circular area is Where R is the radius of the circular area.

[0083] Based on this, the position coordinates of the center of the rectangular box are R is

[0084] In one or more embodiments of this specification, more intuitively, combined with Figure 2-Figure 3 , a more specific description of this scheme is given.

[0085] Figure 2 A schematic diagram of the framework of a face-scanning payment willingness recognition system provided for one or more embodiments of this specification.

[0086] In one or more embodiments of the present specification, in order to more accurately identify the candidate's willingness to pay by face recognition, it is proposed to use a deep convolutional neural network end-to-end learning approach for the face images obtained during the face payment process, so as to achieve security detection of willingness to pay by face recognition, and introduce a candidate area attention mechanism to integrate the mask map into network learning, so as to more specifically identify the candidate's willingness to pay by face recognition in the face images, thereby enhancing the face recognition security experience.

[0087] like Figure 2 As shown, the face-scanning payment willingness recognition system is implemented by end-to-end learning of a deep convolutional neural network. The system includes a face-scanning image, a mask map generated by the area where the candidate is located in the face-scanning image, a first convolutional network module, an attention mechanism implementation module for the candidate's area, a third convolutional module, and a network output.

[0088] It should be noted that the first convolutional network and the third convolutional network have a specific corresponding relationship, that is, the network types involved in the first convolutional network and the third convolutional network are not limited here, but the two are not separated, for example, the first convolutional network and the third convolutional network belong to different parts of the same recognition network type.

[0089] Among them, the face recognition image and the generated mask image are used as input data of the deep convolutional neural network, and the network output is the probability value of willingness to pay by face recognition, that is, the probability of willingness to be safe and the probability of willingness to be unsafe.

[0090] Based on this, in the process of face-scanning payment willingness recognition, the first convolutional network is used to extract the features of the face-scanning image, and the features and mask map of the face-scanning image are input into the candidate's area attention mechanism implementation module, which processes the features and mask map of the face-scanning image and outputs fused features. Finally, the fused features are input into the third convolutional network, and the fused features are processed by the third convolutional network to obtain the processing results.

[0091] The following will explain how the attention mechanism implementation module of the candidate's area processes the features and mask map of the face scan image and outputs the fused features.

[0092] like Figure 2 As shown, the attention mechanism implementation module of the candidate's area includes the features of the face image, the mask image after resolution reduction processing, the second convolutional network module, and the fusion feature.

[0093] Specifically, in the process of face-scanning payment intention recognition, the face-scanning image is first input into the first convolutional network in the first convolutional network module. The features of the face-scanning image are extracted through the first convolutional network, and the mask image is down-sampled to obtain a mask image after down-sampled to adapt to the features of the face-scanning image.

[0094] Then, the features of the face scanned image and the mask image after the resolution reduction processing are input into the second convolutional neural network module. Through the second convolutional network, the features of the face scanned image and the mask image after the resolution reduction processing are fused to obtain the fused features.

[0095] Through end-to-end learning of deep convolutional neural networks, the recognition of candidates' willingness to pay by face recognition is realized. Through the implementation module of the attention mechanism of the candidate's area, the network can learn to judge the willingness of candidates in the face recognition image in a more targeted manner. Especially in public places, it can effectively prevent candidate A from enabling face recognition and mistakenly swiping the assets of candidate B.

[0096] More intuitively, Figure 3 A flowchart of a method for recognizing willingness to pay by face recognition based on end-to-end learning of a deep convolutional neural network is provided for one or more embodiments of this specification.

[0097] Figure 3 The process in may include the following steps:

[0098] S302: Acquire a face recognition image, and determine a candidate to be recognized in the face recognition image.

[0099] S304: Generate corresponding mask images according to the regions where the candidates are located in the face recognition image to distinguish the regions from other regions in the face recognition image.

[0100] S306: Extracting features of the face scanned image through a first convolutional network.

[0101] It should be noted that the first convolutional network is obtained in advance through supervised training.

[0102] S308: performing resolution reduction processing on the mask image to adapt to the features of the face scanning image.

[0103] For example, the resolution of the mask image is reduced through nearest neighbor sampling, thereby generating a mask image with the same resolution as the face image, which can adapt to the features of the face image so that the second convolutional network can process it.

[0104] S310: Using a second convolutional network, the features of the face recognition image and the mask image after the resolution reduction process are fused to obtain fused features. It should be noted that the second convolutional network is obtained in advance through supervised training.

[0105] In one or more embodiments of the present specification, in the process of obtaining fused features, the features of the face scanned image and the mask image after resolution reduction processing are connected according to the channel dimension, and the connected features are input into the second convolutional network for processing to obtain fused features.

[0106] The number of convolutional layers of the second convolutional network is not specifically limited here. That is, the number of convolutional layers of the second convolutional network can be composed of one or more convolutional layers. At the same time, the resolution of the fused feature is the same as the feature of the face scan image, and the number of feature channels of the fused feature is the same as the feature of the face scan image.

[0107] S312: Input the fused features into a third convolutional network corresponding to the first convolutional network for processing to obtain a processing result, wherein the first convolutional network and the third convolutional network are pre-split from the same convolutional network. It should be noted that the third convolutional network is pre-obtained through supervised training.

[0108] In one or more embodiments of the present specification, the first convolutional network and the third convolutional network are pre-split from the same convolutional network, for example, the convolutional network is resent, ShUffleNet V2, etc.

[0109] During the splitting process, the first convolutional network and the third convolutional network can be respectively regarded as the front and back parts of the same convolutional network.

[0110] The split position can be determined by the resolution of the features of the face image. Even if the features of the face image have not yet been extracted (assuming that the model has not yet been constructed), there can also be an expected resolution for the features of the face image (called the target resolution, for example, the resolution of the face image). ) in order to split the convolutional network accordingly and complete the model construction, so that the convolutional network in the previous part (the first convolutional network) can just output the features of this resolution.

[0111] In the convolutional layers in the same convolutional network, determine the convolutional layer that matches the target resolution. Finally, use the matching convolutional layer as the split point to split the same convolutional network into the first part and the second part, with the first part as the first convolutional network and the second part as the third convolutional network.

[0112] S314: Generate a probability value according to the processing result to indicate whether the corresponding candidate is willing to pay by face recognition.

[0113] For example, the probability value is compared with the set threshold probability. If the probability value is greater than the set threshold probability, it is determined that the candidate's willingness is safe, that is, the candidate has the willingness to pay by face recognition. If the probability value is less than or equal to the set threshold probability, it is determined that the candidate's willingness is unsafe, that is, the candidate does not have the willingness to pay by face recognition.

[0114] Based on the previous description, the following will explain how to perform supervised training on the first convolutional network, the second convolutional network, and the third convolutional network.

[0115] In one or more embodiments of the present specification, a training data set needs to be established first, and then network training is performed using the training data set.

[0116] Specifically, when establishing a training data set, first obtain a face-scanning sample image that contains a confirmed face-scanning user, that is, the candidate enables face-scanning payment, and collects a face-scanning image through a camera device. Regardless of whether there are multiple candidates to be identified in the face-scanning image, in the face-scanning image, the candidate is regarded as a face-scanning user, and the face-scanning image is used as a face-scanning sample image. For example, if a face-scanning image is collected by a camera on an offline IoT face-scanning device, and if candidate A enables face-scanning payment, candidate A is regarded as a face-scanning user in the face-scanning image, and the face-scanning image is used as a face-scanning sample image.

[0117] It should be noted that each time face-scanning payment is enabled, a corresponding face-scanning image will be collected. For example, if candidate A enables face-scanning payment, the IoT face-scanning device will collect images of nearby users to obtain a single face-scanning image. If candidate B enables face-scanning payment, the IoT face-scanning device will collect images of nearby users again to obtain a single face-scanning image.

[0118] After obtaining the face scanning sample image, the positive sample is obtained by marking the face scanning user as having the willingness to pay by face scanning and generating the corresponding mask map. When generating the corresponding mask map, the position of the face selection box of the face scanning user in the face scanning image is selected from the face scanning image, and then the corresponding mask map is generated by the position. When marking the face scanning user as having the willingness to pay by face scanning, it can be marked by the willingness marking label, for example, the willingness label is {0,1}, where 1 represents the willingness to pay by face scanning and 0 represents the lack of willingness to pay by face scanning.

[0119] After obtaining the face scanning sample image, since there may be multiple candidates to be identified in the face scanning sample image, if the face scanning sample image also contains other users who were photographed by the way, the other users are marked as not willing to pay by face scanning, and the corresponding mask image is generated to obtain a negative sample. For example, if candidate A enables face scanning payment, candidate B is marked as not willing to pay by face scanning in the face scanning image.

[0120] Finally, according to the obtained positive samples and negative samples, supervised training is performed on the first convolutional network, the second convolutional network, and the third convolutional network.

[0121] In the case of supervised training, firstly, the positive samples and negative samples are randomly sampled in the training data set to generate training batches and their corresponding labeled labels. Then, the training batches and their corresponding labeled labels are input into the initial deep convolutional neural network. The initial deep convolutional neural network includes the untrained first convolutional network, the second convolutional network, and the third convolutional network.

[0122] Then, the initial deep convolutional neural network outputs the probability value, calculates the loss function through the probability value and the corresponding label, and performs network training without interrupting the optimization of the loss function through the gradient descent method, thereby completing the supervised training and obtaining the deep convolutional neural network. Among them, the rules for identifying whether the candidate has the willingness to pay by face recognition can be obtained through network training. For example, if the deep convolutional neural network recognizes that the candidate's face area is located in the middle area, it is considered that the candidate has the willingness to pay by face recognition.

[0123] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 4 , Figure 5shown.

[0124] Figure 4 A schematic diagram of a device for identifying willingness to pay by face recognition provided in one or more embodiments of this specification, the device comprising:

[0125] An acquisition module 402 acquires a face scan image and determines a candidate to be identified in the face scan image;

[0126] A generating module 404 generates a corresponding mask image according to the region where each candidate is located in the face scan image to distinguish the region from other regions in the face scan image;

[0127] An extraction module 406 extracts features of the face scan image and obtains fusion features according to the features of the face scan image and the mask image;

[0128] The identification module 408 identifies whether each of the candidates is willing to pay by face recognition according to the fusion features.

[0129] Optionally, the generating module 404 executes, for each of the determined candidates, the following steps:

[0130] Determine, according to the face region selection box of the candidate, a first filling region of the mask image corresponding to the candidate, and a second filling region outside the first filling region;

[0131] By assigning different filling values ​​to the first filling area and the second filling area, the mask image having a resolution consistent with a resolution of the face scanning image is generated.

[0132] Optionally, the face area selection frame is a rectangular frame;

[0133] The generating module 404 determines the center of the rectangular frame as the center of the circle, and determines half the length of the longest side of the rectangular frame as the radius;

[0134] A circular area formed by the center of the circle and the radius is determined as the first filling area of ​​the mask image corresponding to the candidate.

[0135] Optionally, the extraction module 406 extracts features of the face scanning image through a first convolutional network;

[0136] Performing resolution reduction processing on the mask image to adapt to the features of the face scanning image;

[0137] The features of the face scanned image and the mask image after the resolution reduction processing are fused through the second convolutional network to obtain fused features.

[0138] Optionally, the extraction module 406 connects the features of the face scanned image and the mask image after the resolution reduction processing according to the channel dimension;

[0139] The connected features are input into the second convolutional network for processing to obtain fused features.

[0140] Optionally, the recognition module 408 inputs the fused features into a third convolutional network corresponding to the first convolutional network for processing to obtain a processing result, wherein the first convolutional network and the third convolutional network are pre-split from the same convolutional network;

[0141] A probability value is generated according to the processing result to indicate whether the corresponding candidate is willing to pay by face recognition.

[0142] Optionally, the recognition module 408 determines a target resolution so as to serve as a resolution of features of the face scanned image;

[0143] Among the convolutional layers in the same convolutional network, determining a convolutional layer that matches the target resolution;

[0144] Taking the matched convolution layer as a splitting point, the same convolution network is split into a front part and a rear part, the front part serves as the first convolution network, and the rear part serves as the third convolution network.

[0145] Optionally, the device further comprises a supervised training module, wherein the supervised training module obtains a face scanning sample image containing a confirmed face scanning user;

[0146] The face-scanning user is marked as having a willingness to pay by face-scanning, and a corresponding mask image is generated to obtain a positive sample;

[0147] If the face-scanning sample image also includes other users who are photographed incidentally, the other users are marked as not willing to pay by face-scanning, and a corresponding mask image is generated to obtain a negative sample;

[0148] According to the obtained samples, supervised training is performed on the first convolutional network, the second convolutional network, and the third convolutional network.

[0149] Optionally, the face scanning image contains at least two faces.

[0150] Optionally, the device is applied to an offline IoT face recognition device, and the face recognition image is collected by the IoT face recognition device for nearby users.

[0151] Figure 5A schematic diagram of a face-scanning payment willingness recognition device provided in one or more embodiments of this specification, the device comprising:

[0152] at least one processor; and,

[0153] a memory communicatively connected to the at least one processor; wherein,

[0154] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0155] Acquire a face scan image, and determine a candidate to be identified in the face scan image;

[0156] According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image;

[0157] Extracting features of the face scan image, and obtaining fusion features according to the features of the face scan image and the mask image;

[0158] According to the fusion features, it is identified whether each of the candidates is willing to pay by face recognition.

[0159] Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium for face-scanning payment willingness recognition corresponding to the above method, storing computer executable instructions, and the computer executable instructions are set to:

[0160] Acquire a face scan image, and determine a candidate to be identified in the face scan image;

[0161] According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image;

[0162] Extracting features of the face scan image, and obtaining fusion features according to the features of the face scan image and the mask image;

[0163] According to the fusion features, it is identified whether each of the candidates is willing to pay by face recognition.

[0164] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0165] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0166] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0167] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0168] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0169] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0172] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0173] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0174] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0175] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0176] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0177] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0178] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0179] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A method for identifying willingness to pay by face recognition, comprising: Acquire a face scan image, and determine a candidate to be identified in the face scan image; According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image; Extracting features of the face-scanning image, and obtaining fused features according to the features of the face-scanning image and the mask map, specifically comprising: extracting features of the face-scanning image through a first convolutional network, performing resolution reduction processing on the mask map to adapt to the features of the face-scanning image, and fusing the features of the face-scanning image and the mask map after the resolution reduction processing through a second convolutional network to obtain fused features; According to the fused features, identifying whether each of the candidates is willing to pay by face recognition, specifically comprising: inputting the fused features into a third convolutional network corresponding to the first convolutional network for processing to obtain a processing result, wherein the first convolutional network and the third convolutional network are pre-split from the same convolutional network, and generating a probability value according to the processing result to indicate whether the corresponding candidate is willing to pay by face recognition.

2. The method according to claim 1, wherein generating corresponding mask images according to the regions where each candidate is located in the face scan image comprises: For each of the determined candidates, execute: Determine, according to the face region selection box of the candidate, a first filling region of the mask image corresponding to the candidate, and a second filling region outside the first filling region; By assigning different filling values ​​to the first filling area and the second filling area, the mask image having a resolution consistent with a resolution of the face scanning image is generated.

3. The method according to claim 2, wherein the face area selection frame is a rectangular frame; The step of determining the first filling area of ​​the mask image corresponding to the candidate according to the face area selection box of the candidate specifically includes: The center of the rectangular box is determined as the center of the circle, and half the length of the longest side of the rectangular box is determined as the radius; a circular area formed by the center of the circle and the radius is determined as the first filling area of ​​the mask image corresponding to the candidate.

4. The method according to claim 1, wherein the second convolutional network is used to fuse the features of the face recognition image and the mask image after the resolution reduction process to obtain the fused features, specifically comprising: According to the channel dimension, the features of the face scanned image and the mask image after the resolution reduction processing are connected; the features obtained by the connection are input into the second convolutional network for processing to obtain fused features.

5. The method according to claim 1, wherein the splitting comprises: Determining a target resolution so as to serve as a resolution of features of the face scanned image; Among the convolutional layers in the same convolutional network, determining a convolutional layer that matches the target resolution; Taking the matched convolution layer as a splitting point, the same convolution network is split into a front part and a rear part, the front part serves as the first convolution network, and the rear part serves as the third convolution network.

6. The method according to claim 1, before identifying whether each candidate has a willingness to pay by face recognition based on the fusion feature, the method further comprises: Obtain a sample image of a face scanned user who has been confirmed to be a face scanned user; The positive sample is obtained by marking the face-scanning user as having the willingness to pay by face-scanning and generating a corresponding mask map; If the face-scanning sample image also includes other users who are photographed incidentally, the other users are marked as not willing to pay by face-scanning, and a corresponding mask image is generated to obtain a negative sample; According to the obtained samples, supervised training is performed on the first convolutional network, the second convolutional network, and the third convolutional network.

7. The method according to any one of claims 1 to 6, wherein the face scanning image contains at least two faces.

8. The method according to any one of claims 1 to 6, wherein the method is applied to an offline IoT face recognition device, and the face recognition image is collected by the IoT face recognition device for nearby users.

9. A face-scanning payment intention recognition device, comprising: An acquisition module acquires a face scan image and determines a candidate to be identified in the face scan image; A generation module, which generates corresponding mask images according to the regions where the candidates are located in the face-scanning image, so as to distinguish the regions from other regions in the face-scanning image; The extraction module extracts the features of the face-scanning image, and obtains fused features according to the features of the face-scanning image and the mask map, specifically comprising: extracting the features of the face-scanning image through a first convolutional network, performing resolution reduction processing on the mask map to adapt to the features of the face-scanning image, and fusing the features of the face-scanning image and the mask map after the resolution reduction processing through a second convolutional network to obtain fused features; The identification module identifies whether each of the candidates is willing to pay by face recognition based on the fused features, specifically comprising: inputting the fused features into a third convolutional network corresponding to the first convolutional network for processing to obtain a processing result, wherein the first convolutional network and the third convolutional network are pre-split from the same convolutional network, and generating a probability value based on the processing result to indicate whether the corresponding candidate is willing to pay by face recognition.

10. The apparatus according to claim 9, wherein the generating module performs, for each of the determined candidates, the following steps: Determine, according to the face region selection box of the candidate, a first filling region of the mask image corresponding to the candidate, and a second filling region outside the first filling region; By assigning different filling values ​​to the first filling area and the second filling area, the mask image having a resolution consistent with a resolution of the face scanning image is generated.

11. The device according to claim 10, wherein the face area selection frame is a rectangular frame; The generating module determines the center of the rectangular frame as the center of the circle, and determines half the length of the longest side of the rectangular frame as the radius; A circular area formed by the center of the circle and the radius is determined as the first filling area of ​​the mask image corresponding to the candidate.

12. The device according to claim 9, wherein the extraction module connects the features of the face scanned image and the mask image after the resolution reduction processing according to the channel dimension; The connected features are input into the second convolutional network for processing to obtain fused features.

13. The apparatus according to claim 9, wherein the recognition module determines a target resolution so as to serve as a resolution of features of the face scan image; Among the convolutional layers in the same convolutional network, determining a convolutional layer that matches the target resolution; Taking the matched convolution layer as a splitting point, the same convolution network is split into a front part and a rear part, the front part serves as the first convolution network, and the rear part serves as the third convolution network.

14. The device according to claim 9, further comprising a supervised training module, wherein the supervised training module obtains a sample face image containing a confirmed face-scanning user; The positive sample is obtained by marking the face-scanning user as having the willingness to pay by face-scanning and generating a corresponding mask map; If the face-scanning sample image also includes other users who are photographed incidentally, the other users are marked as not willing to pay by face-scanning, and a corresponding mask image is generated to obtain a negative sample; According to the obtained samples, supervised training is performed on the first convolutional network, the second convolutional network, and the third convolutional network.

15. The device according to any one of claims 9 to 14, wherein the face scanning image contains at least two human faces.

16. The device according to any one of claims 9 to 14, wherein the device is applied to an offline IoT face recognition device, and the face recognition image is collected by the IoT face recognition device for nearby users.

17. A face-scanning payment willingness recognition device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Acquire a face scan image, and determine a candidate to be identified in the face scan image; According to the region where each candidate is located in the face-scanning image, a corresponding mask image is generated to distinguish the region from other regions in the face-scanning image; Extracting features of the face-scanning image, and obtaining fused features according to the features of the face-scanning image and the mask map, specifically comprising: extracting features of the face-scanning image through a first convolutional network, performing resolution reduction processing on the mask map to adapt to the features of the face-scanning image, and fusing the features of the face-scanning image and the mask map after the resolution reduction processing through a second convolutional network to obtain fused features; According to the fused features, identifying whether each of the candidates is willing to pay by face recognition, specifically comprising: inputting the fused features into a third convolutional network corresponding to the first convolutional network for processing to obtain a processing result, wherein the first convolutional network and the third convolutional network are pre-split from the same convolutional network, and generating a probability value according to the processing result to indicate whether the corresponding candidate is willing to pay by face recognition.

Images