Face scanning payment management method and system

Abstract

The invention relates to the technical field of payment management, and particularly discloses a face scanning payment management method and system, and the method comprises the steps: collecting humidity sensor data, light sensor data and a face image; inputting the data of the humidity sensor, the data of the light sensor and the face image into a pre-constructed oil-water detection model, and outputting an environment mark; marking at a corresponding position of the face image according to the environment mark to obtain an original image; performing image restoration according to the original image to obtain a clear portrait; building and extracting the clear portrait to obtain a complete feature point set; and inputting the complete feature point set and the database template into a pre-constructed payment permission judgment model to obtain a living body judgment value and a similarity score. According to the method, the recognition accuracy is ensured through intelligent repair methods such as an integrated sensing technology, an image processing algorithm and a deep learning model, and the problem of water-oil interference possibly encountered in a facial recognition payment system is solved.

Description

Technical Field

[0001] The present application relates to the field of payment management technology, and in particular to a face-scanning payment management method and system. Background Art

[0002] In the current digital payment field, especially in mobile payment and contactless payment systems, user experience and transaction security are crucial considerations. With the advancement of technology, biometric technology, especially facial recognition technology, has become an important means of secure payment due to its convenience and difficulty in counterfeiting. For example, the Chinese patent specification embodiment with announcement number CN113657903B provides a face-swiping payment method, device, electronic device and storage medium, the method comprising: collecting image information; identifying user behavior information of at least one user according to the image information, the user behavior information includes face-swiping information, or the user behavior information includes face-swiping information and gesture information; if at least one gesture information is identified, generating a willingness to pay value for each user according to the user behavior information of at least one user; if a target user is determined from at least one user according to the willingness to pay value of each user, performing face-swiping payment on the target user. The embodiment of the specification awakens the face-swiping payment process by identifying gesture information, generates a willingness to pay value according to the face-swiping information and gesture information to determine the target user, and performs face-swiping payment on the target user determined according to the willingness to pay value, thereby avoiding the situation of misidentifying the target user, thereby improving the security of face-swiping payment. However, facial recognition technology is often affected by multiple factors in actual environments. In particular, when the payment environment is outdoors, it is extremely susceptible to interference from water and oil. For example, in the cafeteria, when students wash their hands after a meal and use facial recognition payment, some water and oil may get on the camera of the facial consumption machine, causing the facial recognition payment system to fail in payment verification. Summary of the invention

[0003] In order to solve the above problems, an embodiment of the present invention provides a face-scanning payment management method, the method comprising:

[0004] Collect humidity sensor data, light sensor data and face images;

[0005] Input humidity sensor data, light sensor data and face images into the pre-built oil and water detection model and output the environmental sign;

[0006] Marking the corresponding position of the face image according to the environmental mark to obtain the original image;

[0007] Perform image restoration based on the original image to obtain a clear portrait;

[0008] Extract the clear portrait and obtain a complete set of feature points;

[0009] Input the complete feature point set and database template into the pre-built payment permission determination model to obtain the liveness determination value and similarity score;

[0010] The payment result is output based on the liveness determination value and similarity score.

[0011] The oil-water detection model includes an oil-water interference identification model and a frequency domain analysis mathematical model;

[0012] The frequency domain analysis mathematical model includes:

[0013] The face image is converted into a grayscale image f(x,y); the frequency domain representation F(u,v) of the image is obtained by calculating the grayscale image f(x,y). The calculation method of F(u,v) includes:

[0014]

[0015] Where f(x,y) is the grayscale value of the image in the time domain; F(u,v) is the complex value in the frequency domain, representing the frequency component; M and N are the width and height of the image; u and v are the frequency domain coordinates, corresponding to the horizontal and vertical frequencies respectively; j is the imaginary unit;

[0016] Generate a spectrum diagram based on the frequency domain representation F(u,v) and perform logarithmic transformation to enhance the contrast;

[0017] Perform high-pass filtering on the spectrum graph, and then inversely transform it back to the face image, that is, filter the face image, input the filtered face image into the database, and perform threshold segmentation on the filtered face image to obtain an image containing only high-frequency characteristics, that is, a time-series highlight image;

[0018] The inverse transform calculation method includes:

[0019]

[0020] The training method of the oil-water interference recognition model includes:

[0021] All time series highlight images are used as input of the oil-water interference recognition model. The oil-water interference recognition model uses the environmental signs corresponding to the predictions of each group of time series highlight images as output, uses the actual environmental signs corresponding to each group of time series highlight images as prediction targets, and uses minimizing the sum of the first prediction accuracies of all predicted environmental signs as a training target. The oil-water interference recognition model is trained until the sum of the first prediction accuracies reaches convergence, and the training is stopped. The oil-water interference recognition model is a convolutional neural network model.

[0022] The method for obtaining the clear portrait comprises:

[0023] Analyze and repair the marked areas on the original image:

[0024] Environmental markers include no interference, water interference, oil interference and oil-water mixed interference; if the marked environmental marker is no interference, no operation is performed; if the marked environmental marker is water interference, the area with the environmental marker as water interference, i.e. the water interference area, is input into the pre-built adversarial network model, and a clear area map is output; if the marked environmental marker is oil interference, the filtered face image at the corresponding position is output from the database; if the marked environmental marker is oil-water mixed interference, the filtered face image at the corresponding position in the database is input into the pre-built adversarial network model, and a clear area map is output; when all the marked areas on the original image are analyzed and repaired, a clear portrait is obtained.

[0025] The adversarial network model includes a generator and a discriminator;

[0026] The generator includes an initial layer, a downsampling layer, a residual block, an upsampling layer and a first output layer; the generator adopts a U-Net architecture; the initial layer includes a first convolutional layer and an activation layer; the downsampling layer implements downsampling through convolution with a step size greater than 1; the residual block includes a second convolutional layer, a third convolutional layer and a jump connection, and the second and third convolutional layers are followed by batch normalization and ReLU activation; the upsampling layer includes a transposed convolutional layer, and the transposed convolutional layer is followed by batch normalization and ReLU; the first output layer includes a fourth convolutional layer and a Tanh activation function;

[0027] The discriminator includes an input layer, a feature extraction layer, an intermediate layer and a second output layer; the input layer is used to input images; the feature extraction layer includes multiple convolutional layers and a LeakyReLU activation function; the intermediate layer is located in the middle of the feature extraction layer and is a batch normalization layer; the second output layer includes multiple fully connected layers and output neurons.

[0028] The training method of the adversarial network model includes:

[0029] The discriminator and generator are trained in a loop. When both the generator loss function and the discriminator loss function converge, the loop ends and the training is completed.

[0030] The training method of the payment permission determination model includes:

[0031] All complete feature point sets and database templates are used as inputs of the payment permission determination model. The payment permission determination model uses the liveness determination value and similarity score corresponding to each set of complete feature point sets and the database template prediction as outputs, uses the actual liveness determination value and similarity score corresponding to each set of complete feature point sets and the database template as prediction targets, and uses the sum of the second prediction accuracies of minimizing all predicted liveness determination values ​​and similarity scores as a training target; the payment permission determination model is trained until the sum of the second prediction accuracies reaches convergence, and the training is stopped; the payment permission determination model is a convolutional neural network model.

[0032] On the other hand, the present application also provides a face-scanning payment management system, including:

[0033] A collection module, the collection module is used to collect humidity sensor data, light sensor data and face images;

[0034] A pre-labeling module, which is used to input humidity sensor data, light sensor data and face images into a pre-built oil-water detection model and output an environmental sign;

[0035] A marking module, which is used to mark the corresponding position of the face image according to the environmental mark to obtain the original image;

[0036] An image restoration module, wherein the image restoration module is used to perform image restoration based on the original image to obtain a clear portrait;

[0037] A feature extraction module, which is used to extract the clear portrait and obtain a complete feature point set;

[0038] A liveness determination module, which inputs the complete feature point set and the database template into a pre-built payment permission determination model to obtain a liveness determination value and a similarity score;

[0039] A payment output module is used to output a payment result according to the liveness determination value and the similarity score.

[0040] The technical effects and advantages of the face-scanning payment management method and system provided by the present invention are as follows:

[0041] The present invention ensures recognition accuracy by integrating intelligent repair methods such as sensing technology, image processing algorithms and deep learning models, and solves the water and oil interference problem that may be encountered in facial recognition payment systems. In addition, through intelligent repair and precise liveness detection technology, the user's payment security is greatly enhanced, the risk of fraud is reduced, and the user's payment experience is improved; the humidity and light sensors are integrated to monitor environmental changes in real time and quickly identify possible interference conditions, such as high humidity may indicate water drop interference, and abnormal light enhancement may be due to oil film reflection; the deep learning algorithm is used to combine sensor data and image analysis to accurately calibrate the interfered area and distinguish the interference type (water, oil or mixed); the frequency domain characteristics of the interfered image are transformed and analyzed, which is effective Effectively identify and isolate high-frequency interference, such as light spots caused by oil film; use generative adversarial networks, especially U-Net architecture, to perform targeted repairs on damaged areas to ensure accurate face recognition; obtain a detailed set of facial feature points through structure extraction technology, and use depth information for reconstruction even when some feature points are damaged; combine liveness detection values ​​and similarity scores to accurately determine payment authorization, and ensure that only legitimate and real users can perform payments through efficient comparison with database templates; all technologies and models are integrated in a way that is not easily perceived by users, without interfering with user operations and improving user experience; ensure that all processing steps such as interference detection, image repair and feature analysis are completed within seconds to achieve fast feedback and operation. . BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a face-scanning payment management method in Embodiment 1;

[0043] Figure 2 This is a schematic diagram of the connection of the adversarial network model in Example 1;

[0044] Figure 3 This is a connection diagram of a face-scanning payment management system in Example 2. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Embodiment 1:

[0047] See also Figure 1 As shown, the face-scanning payment management method described in this embodiment includes:

[0048] Collect humidity sensor data, light sensor data and face images;

[0049] Input humidity sensor data, light sensor data and face images into the pre-built oil and water detection model and output the environmental sign;

[0050] Marking the corresponding position of the face image according to the environmental mark to obtain the original image;

[0051] Perform image restoration based on the original image to obtain a clear portrait;

[0052] Extract the clear portrait and obtain a complete set of feature points;

[0053] Input the complete feature point set and database template into the pre-built payment permission determination model to obtain the liveness determination value and similarity score;

[0054] The payment result is output based on the liveness determination value and similarity score.

[0055] The humidity sensor data and light sensor data are obtained by the sensors of the face consumption machine. The humidity sensor data is used to quickly filter the possibility of no water interference and reduce the burden of subsequent analysis. The light sensor data is used to capture abnormal reflections and preliminarily determine whether oil or water may exist. For example, high humidity (for example >70%): there may be water interference; low humidity (for example <40%): the possibility of water interference is low; abnormal reflected light intensity (increased local light spots and reflections): it may be caused by oil film reflection or water droplet focusing; no obvious reflection abnormality: the reflection characteristics are normal, without oil or water interference; the face image is used as the original data to verify the interference type, which is used to accurately determine the specific interference type (oil, water, oil and water mixture); due to the difference between water droplet characteristics and oil film characteristics, the oil and water detection model is a mixed recognition.

[0056] Water droplet characteristics:

[0057] Local image blur: The convex lens effect of water droplets will cause blur and distortion in local areas.

[0058] Optical properties of water droplet edges: Scattered halos may appear around water droplets.

[0059] Oil film characteristics:

[0060] Local highlights in the image: The mirror reflection effect of the oil film will cause strong reflective highlights to appear in some areas of the image.

[0061] Global blurring of the image: A thin oil film may cause overall light scattering, making the edges of the image unclear.

[0062] Environmental signs include no interference, water interference, oil interference and oil-water mixed interference.

[0063] The oil-water detection model includes the oil-water interference identification model and the frequency domain analysis (FFT) mathematical model.

[0064] Frequency domain analysis (FFT) mathematical models include:

[0065] The face image is converted into a grayscale image f(x,y); the frequency domain representation F(u,v) of the image is obtained by calculating the grayscale image f(x,y). The calculation method of F(u,v) includes:

[0066]

[0067] Where f(x, y) is the grayscale value (pixel value) of the image in the time domain; F(u, v) is the complex value in the frequency domain, representing the frequency component; M and N are the width and height of the image; u and v are the frequency domain coordinates, corresponding to the horizontal and vertical frequencies respectively; j is the imaginary unit.

[0068] Generate a spectrum diagram based on the frequency domain representation F(u,v) and perform logarithmic transformation to enhance the contrast;

[0069] The formula for enhancing contrast is as follows:

[0070] Spectrum(u,v)=log(1+|F(u,v)|);

[0071] Structure code example:

[0072]

[0073]

[0074] The spectrum graph is high-pass filtered and then inversely transformed back into a face image, i.e., a filtered face image. The filtered face image is input into a database, and a threshold segmentation is performed on the filtered face image to obtain an image containing only high-frequency characteristics, i.e., a time series highlight image.

[0075] The inverse transform calculation method includes:

[0076]

[0077] The operation method of high-pass filtering is to create a mask with a "center-shielded low-frequency area" to set the low-frequency components in the frequency domain to zero, leaving only the high-frequency components in the periphery.

[0078] High-frequency characteristics describe details with high-frequency changes in the image, such as edges, textures, and bright spots, which can be used to assist in interference classification in subsequent machine learning models (such as distinguishing between oil, water, and oil-water mixtures); the recognition logic is:

[0079] Water Drops:

[0080] Local blur and diffuse halo.

[0081] The high-frequency characteristics are more dispersed, and the boundaries of the bright spot area are blurred.

[0082] Oil film:

[0083] Highlight areas caused by strong reflections.

[0084] The high-frequency characteristics are concentrated, and the bright spot areas are dot-shaped or strip-shaped.

[0085] It is particularly noted that the time series highlight image can be segmented by a threshold value to detect and identify the highlight reflection area.

[0086] Structure code example:

[0087]

[0088]

[0089] The training method of the oil-water interference recognition model includes:

[0090] All time series highlight images are used as the input of the oil-water interference recognition model. The oil-water interference recognition model predicts the corresponding environmental signs for each group of time series highlight images as output, and the actual environmental signs corresponding to each group of time series highlight images are used as prediction targets. Minimizing the sum of the first prediction accuracies of all predicted environmental signs is used as the training goal.

[0091] Among them, the calculation formula for the first prediction accuracy is: EW v =(EQ v -ET v ) 2 , where v is the number of each set of time series highlight images, EW v is the first prediction accuracy, EQ v is the predicted environmental symbol corresponding to the vth group of time series highlight images, ET v is the actual environmental sign corresponding to the vth group of time series highlight images; the oil-water interference recognition model is trained until the sum of the first prediction accuracy reaches convergence and the training is stopped; the oil-water interference recognition model is a convolutional neural network model.

[0092] Methods for obtaining clear portraits include:

[0093] Analyze and repair the marked areas on the original image:

[0094] If the marked environmental sign is no interference, no operation is performed; if the marked environmental sign is water interference, the area marked as water interference, i.e. the water interference area, is input into the pre-built adversarial network model, and a clear area map is output; if the marked environmental sign is oil interference, the filtered face image at the corresponding position is output from the database; if the marked environmental sign is oil-water mixed interference, the filtered face image at the corresponding position in the database is input into the pre-built adversarial network model, and a clear area map is output; when all the marked areas on the original image are analyzed and repaired, a clear portrait is obtained.

[0095] like Figure 2 As shown, the adversarial network model includes a generator and a discriminator;

[0096] The generator includes the initial layer, downsampling layer, residual block, upsampling layer and the first output layer; the generator adopts the U-Net architecture; the initial layer includes the first convolution layer and the activation layer, the convolution layer is used to extract the features of the image, and the activation layer uses the ReLU activation function to increase the nonlinear processing capability; the downsampling layer achieves downsampling through convolution with a step size greater than 1, reducing the spatial dimension of the feature map and increasing the number of channels; the residual block includes the second convolution layer, the third convolution layer and the jump connection, and the second and third convolution layers are followed by batch normalization and ReLU Activation, batch normalization and ReLU are standard activation and normalization methods; skip connections can help information propagate in the network and reduce information loss during training; the upsampling layer includes a transposed convolution layer, which is followed by batch normalization and ReLU. The transposed convolution layer is used to gradually enlarge the feature map and restore the image dimension; the first output layer includes the fourth convolution layer and the Tanh activation function. The fourth convolution layer uses a convolution layer with a smaller convolution kernel size to produce the final image. The Tanh activation function ensures that the pixel values ​​of the output image are within an appropriate range.

[0097] The discriminator includes an input layer, a feature extraction layer, an intermediate layer and a second output layer; the input layer is used to input images; the feature extraction layer includes multiple convolutional layers and LeakyReLU activation functions. The number of convolutional layers depends on the complexity of the data and is not fixed. After each layer of convolution, the spatial dimension of the feature map will be gradually reduced (using convolution with a step size greater than 1) and the number of channels will be increased. The LeakyReLU activation function provides a slight negative slope to help the model learn more features; the intermediate layer is located in the middle of the feature extraction layer and is a batch normalization layer. Batch normalization is added to deeper convolutional layers to help stabilize the training process and improve the effect; the second output layer includes multiple fully connected layers and output neurons. After flattening the convolutional feature map, it is processed through one or more fully connected layers, that is, multiple fully connected layers. The number of multiple fully connected layers depends on the complexity of the data. The neurons use the Sigmoid activation function to output a value in the range of [0, 1], indicating the probability that the input image is real.

[0098] It is particularly noted that data complexity refers to the diversity, dimension, size, and structural complexity of the input data. These factors directly affect the design of the model structure and the training strategy. If the labeled environment is less interference-free, that is, the more complex the image is, the more processing elements there are, that is, the greater the data complexity, the greater the number of layers;

[0099] Training methods for adversarial network models include:

[0100] The discriminator and generator are trained in a loop. When both the generator loss function and the discriminator loss function converge, the loop ends and the training is completed.

[0101] Discriminator training: Use the filtered face images generated by the generator, the water interference areas generated by the generator, the real filtered face images and the real water interference areas to train the discriminator, and update the discriminator parameters (weights and biases) to maximize the discriminator loss function.

[0102] Generator training: Freeze the parameters of the discriminator and update the generator parameters (weights and biases) by trying to "fool" the discriminator to minimize the generator loss function.

[0103] Generator loss function L G The calculation methods include:

[0104] L G = -log(D(G(z)));

[0105] Where D(G(z)) is the discriminator’s discrimination result on the image generated by the generator.

[0106] Discriminator loss function L D The calculation methods include:

[0107] L D =-[d·log(D(l))+(1-d)log(1-D(G[z]))];

[0108] Where d is the label, which is 11 for the real filtered face image and water interference area, and 10 for the filtered face image and water interference area generated by the generator; l is the real filtered face image and water interference area; G(z) is the filtered face image and water interference area generated by the generator.

[0109] Code example:

[0110]

[0111]

[0112]

[0113] Feature extraction uses image recognition technology to extract feature points (nose, mouth, eyes, etc.). If the feature points are missing, the depth information (infrared or ToF) is used to reconstruct the feature points of the occluded area. Although the feature points of the face have been supplemented and clarified in the previous steps, there may still be omissions or places that cannot be repaired at all, so the places where there is a very small possibility of missing information are reconstructed here, but the loss of such reconstructed information will be very serious. If the depth information is used directly for reconstruction at the beginning, the authenticity of the entire face image will be seriously questionable, and information extraction will be seriously interfered with.

[0114] The database template is a pre-set or collected face image library that contains facial feature data of registered users. In the facial recognition system, these templates are used as a benchmark for comparison with newly captured face data.

[0115] The training method of the payment permission determination model includes:

[0116] All complete feature point sets and database templates are used as inputs of the payment permission determination model. The payment permission determination model uses the liveness determination value and similarity score corresponding to each set of complete feature point sets and the database template predictions as outputs, uses the actual liveness determination value and similarity score corresponding to each set of complete feature point sets and the database template as prediction targets, and uses minimizing the sum of the second prediction accuracies of all predicted liveness determination values ​​and similarity scores as training targets.

[0117] Among them, the calculation formula for the second prediction accuracy is: FY H =(FV H -FP H ) 2 , where H is the number of each set of complete feature points and database template, FY H is the second prediction accuracy, FV H is the predicted liveness judgment value and similarity score corresponding to the Hth group of complete feature point sets and the database template, FP H is the actual liveness determination value and similarity score corresponding to the Hth group of complete feature point sets and the database template; the payment permission determination model is trained until the sum of the second prediction accuracy reaches convergence; the payment permission determination model is a convolutional neural network model.

[0118] The liveness determination value is 100 and 111, 100 means it is not a live person, and 111 means it is a live person; this prevents the use of pictures for face payment; the similarity score range is [0, 1], 0 means it is not a real face at all, and 1 means it is a database template; the payment result is either granting payment authorization or denying payment authorization; if the liveness determination value is 111 and the similarity score is greater than or equal to the preset threshold, the payment result is granting payment authorization, and all other payment results are denying payment authorization and returning an error.

[0119] Embodiment 2:

[0120] like Figure 3 As shown, based on the same inventive concept as the face-scanning payment management method in the aforementioned embodiment, the present application provides a face-scanning payment management system, and the system and method embodiments in the embodiments of the present application are based on the same inventive concept. The system includes:

[0121] A collection module, used to collect humidity sensor data, light sensor data and face images;

[0122] A pre-labeling module is used to input humidity sensor data, light sensor data and face images into a pre-built oil-water detection model and output environmental labels;

[0123] The annotation module is used to annotate the corresponding position of the face image according to the environmental mark to obtain the original image;

[0124] An image restoration module is used to perform image restoration based on the original image to obtain a clear portrait;

[0125] The feature extraction module is used to extract the clear portrait and obtain a complete feature point set;

[0126] The liveness determination module inputs the complete feature point set and the database template into the pre-built payment permission determination model to obtain the liveness determination value and similarity score;

[0127] The payment output module is used to output the payment result according to the liveness determination value and the similarity score.

[0128] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0129] What has been described above is only a preferred specific implementation manner of the embodiments of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions or changes according to the technical scheme and concept of the present application within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A facial recognition payment management method, characterized in that: Methods include: Collect humidity sensor data, light sensor data and face images; Input humidity sensor data, light sensor data and face images into the pre-built oil and water detection model and output the environmental sign; Marking the corresponding position of the face image according to the environmental mark to obtain the original image; Perform image restoration based on the original image to obtain a clear portrait; Extract the clear portrait and obtain a complete set of feature points; Input the complete feature point set and database template into the pre-built payment permission determination model to obtain the liveness determination value and similarity score; The payment result is output based on the liveness determination value and similarity score.

2. A facial recognition payment management method as claimed in claim 1, characterized in that: The oil-water detection model includes an oil-water interference identification model and a frequency domain analysis mathematical model; The frequency domain analysis mathematical model includes: The face image is converted into a grayscale image f(x,y); the frequency domain representation F(u,v) of the image is obtained by calculating the grayscale image f(x,y). The calculation method of F(u,v) includes: Where f(x,y) is the grayscale value of the image in the time domain; F(u,v) is the complex value in the frequency domain, representing the frequency component; M and N are the width and height of the image; u and v are the frequency domain coordinates, corresponding to the horizontal and vertical frequencies respectively; j is the imaginary unit; Generate a spectrum diagram based on the frequency domain representation F(u,v) and perform logarithmic transformation to enhance the contrast; Perform high-pass filtering on the spectrum graph, and then inversely transform it back to the face image, that is, filter the face image, input the filtered face image into the database, and perform threshold segmentation on the filtered face image to obtain an image containing only high-frequency characteristics, that is, a time-series highlight image; The inverse transform calculation method includes:

3. A facial recognition payment management method as claimed in claim 2, characterized in that: The training method of the oil-water interference recognition model includes: All time series highlight images are used as input of the oil-water interference recognition model. The oil-water interference recognition model uses the environmental signs corresponding to the predictions of each group of time series highlight images as output, uses the actual environmental signs corresponding to each group of time series highlight images as prediction targets, and uses minimizing the sum of the first prediction accuracies of all predicted environmental signs as a training target. The oil-water interference recognition model is trained until the sum of the first prediction accuracies reaches convergence, and the training is stopped. The oil-water interference recognition model is a convolutional neural network model.

4. A facial recognition payment management method as claimed in claim 2, characterized in that: The method for obtaining the clear portrait comprises: Analyze and repair the marked areas on the original image: The environmental signs include no interference, water interference, oil interference and oil-water mixed interference; if the marked environmental sign is no interference, no operation is performed; if the marked environmental sign is water interference, the area marked as water interference, i.e., the water interference area, is input into the pre-built adversarial network model, and a clear area map is output; if the marked environmental sign is oil interference, the filtered face image at the corresponding position is output from the database; if the marked environmental sign is oil-water mixed interference, the filtered face image at the corresponding position in the database is input into the pre-built adversarial network model, and a clear area map is output; When all the marked areas on the original image are analyzed and repaired, a clear portrait is obtained.

5. A facial recognition payment management method as claimed in claim 4, characterized in that: The adversarial network model includes a generator and a discriminator; The generator includes an initial layer, a downsampling layer, a residual block, an upsampling layer and a first output layer; the generator adopts a U-Net architecture; the initial layer includes the first convolutional layer and an activation layer; the downsampling layer achieves downsampling through convolution with a step size greater than 1; the residual block includes the second convolutional layer, the third convolutional layer and a jump connection, and the second and third convolutional layers are followed by batch normalization and ReLU activation; the first output layer includes the fourth convolutional layer and a Tanh activation function; The discriminator includes an input layer, a feature extraction layer, an intermediate layer and a second output layer; the input layer is used to input images; the feature extraction layer includes multiple convolutional layers and a LeakyReLU activation function; the intermediate layer is located in the middle of the feature extraction layer and is a batch normalization layer; the second output layer includes multiple fully connected layers and output neurons.

6. A facial recognition payment management method as claimed in claim 5, characterized in that: The training method of the adversarial network model includes: training the discriminator and the generator in a loop, and when the generator loss function and the discriminator loss function converge, the loop ends and the training is completed.

7. A facial recognition payment management method as claimed in claim 1, characterized in that: The training method of the payment permission determination model includes: taking all complete feature point sets and database templates as inputs of the payment permission determination model, the payment permission determination model uses the liveness determination value and similarity score corresponding to each set of complete feature point sets and the database template prediction as outputs, uses the actual liveness determination value and similarity score corresponding to each set of complete feature point sets and the database template as prediction targets, and uses the sum of the second prediction accuracies of minimizing all predicted liveness determination values ​​and similarity scores as a training target; training the payment permission determination model until the sum of the second prediction accuracies reaches convergence, and stopping the training; the payment permission determination model is a convolutional neural network model.

8. A face-scanning payment management system, characterized in that: include: A collection module, the collection module is used to collect humidity sensor data, light sensor data and face images; A pre-labeling module, which is used to input humidity sensor data, light sensor data and face images into a pre-built oil-water detection model and output an environmental sign; A marking module, wherein the marking module is used to mark corresponding positions of the face image according to the environmental mark to obtain the original image; An image restoration module, wherein the image restoration module is used to perform image restoration based on the original image to obtain a clear portrait; A feature extraction module, which is used to extract the clear portrait and obtain a complete feature point set; A liveness determination module, which is used to input the complete feature point set and the database template into a pre-built payment permission determination model to obtain a liveness determination value and a similarity score; A payment output module is used to output a payment result according to the liveness determination value and the similarity score.

Images