A remote real-name authentication method and system based on image recognition

Abstract

The present application relates to remote real-name authentication technical field, specifically to a kind of remote real-name authentication method and system based on image recognition, comprising: receiving user uploaded real-time live face image and identity card chip photo.The real-time live face image is detected by micro-expression texture analysis and light reflection feature extraction, and the facial texture feature is extracted to form live feature descriptor;Certificate chip photo is subjected to moire filter, light equalization processing and perspective distortion correction, to generate standardized certificate face image.Both are input into double-branch feature comparison network coding, to calculate the cosine similarity of feature vector, and the result is output by authentication decision engine in combination with the live detection result.The present application can avoid the risk of fake face fraud, reduce the comparison deviation, and improve the authentication security and accuracy.

Description

Technical Field

[0001] This invention relates to the field of remote real-name authentication technology, and in particular to a remote real-name authentication method and system based on image recognition. Background Technology

[0002] Remote real-name authentication is widely used in finance, government affairs, internet services, and other fields. Its core is to verify the authenticity of a user's identity by comparing their live face with their ID card information using image recognition technology. Currently, in conventional remote real-name authentication methods, liveness detection often employs methods such as action command verification and single facial feature comparison. This involves requiring users to perform specified actions such as blinking or turning their head, or simply extracting basic facial surface features to determine if the image originates from a real, living person. Preprocessing of ID card chip photos often involves simple cropping and brightness or contrast adjustments, performing only basic image optimization without addressing the specific challenges of capturing chip photos.

[0003] Conventional liveness detection methods struggle to effectively distinguish between genuine live faces and highly sophisticated forged faces, such as those in paper photographs or displayed on electronic screens. These are easily exploited for identity theft, leading to insufficient authentication security. During the capture process, ID card chip photos are prone to perspective distortion due to shooting angle deviations, and may also exhibit moiré patterns and uneven lighting. Simple preprocessing cannot eliminate these interferences, causing discrepancies in feature comparison between ID card facial images and live face images, affecting authentication accuracy. Furthermore, the lack of standardization in conventionally preprocessed ID card images further reduces the consistency of feature comparison, resulting in unreliable authentication results. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a remote real-name authentication method and system based on image recognition.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a remote real-name authentication method based on image recognition, comprising: The system receives real-time live face images and ID card chip photos uploaded by users, performs liveness detection on the real-time live face images, and determines whether the real-time live face images are from real live people through micro-expression texture analysis and light reflection feature extraction, and generates liveness detection results. Feature point localization is performed on the real-time live face image that has passed the liveness detection, and texture features of facial contour, periorbital region and nasolabial fold region are extracted to form a liveness feature descriptor; The ID card chip photo is preprocessed by removing moiré filtering and applying illumination equalization to correct perspective distortion caused by the shooting angle and generate a standardized ID card face image. The liveness feature descriptor and the standardized ID card face image are input into a dual-branch feature matching network, and deep semantic encoding is performed on the two types of images respectively. An initial identity matching score is obtained by calculating the cosine similarity of the feature vectors output by the dual-branch feature matching network. The initial identity matching score and the liveness detection result are input into the authentication decision engine, which generates a judgment result including authentication status and confidence level according to the preset multi-factor judgment rules.

[0006] As a further aspect of the present invention, liveness detection is performed on the real-time live face image, and the determination of whether the real-time live face image originates from a real live person is made through micro-expression texture analysis and light reflection feature extraction, including: The real-time live face image is captured by the front-facing camera of the mobile terminal, and the photo of the ID card chip is read from the contactless chip of the ID card through a near-field communication protocol; The real-time live human face image is converted into a grayscale image, and its gradient magnitude and orientation histogram are calculated at different scales. Extract time-series data of eyelid closure frequency and mouth corner pulling amplitude from the real-time live human face image, and input the time-series data into a temporal convolutional network to identify unnatural blinking patterns or stiff facial expressions. The chromaticity component is separated from the RGB channels of the real-time live human face image, and the specular reflection spot distribution on the skin surface is extracted within a specific wavelength range; By comparing the geometric shape of the specular reflection spot distribution with a preset real skin lighting model, the regular reflection marks left by the screen photo or printed photo can be identified. The output of the temporal convolutional network is logically ANDed with the optical reflection analysis result. Only when both the output of the temporal convolutional network and the optical reflection analysis result are determined to be true, is the liveness detection result marked as passed.

[0007] As a further aspect of the present invention, the ID card chip photo is preprocessed by performing demoiré filtering and illumination equalization to correct perspective distortion caused by the shooting angle, thereby generating a standardized ID card face image, including: The ID card chip photo is processed using a frequency domain band-stop filter to filter out periodic moiré patterns caused by digital re-photography. After filtering out moiré patterns, the white balance of the ID card chip photo is adjusted using a color constancy algorithm to eliminate color cast due to ambient light. After completing the illumination correction, the outer border outline of the ID photo area is identified using an edge detection algorithm; Calculate the coordinates of the four corner points of the outer frame outline, and calculate the original shooting angle of the ID card chip photo based on the corner point coordinates; Based on the calculated original shooting angle, the perspective transformation matrix is ​​applied to project the ID card chip photo into the rectangular frame of the standard ID photo to complete the perspective distortion correction. The image after perspective transformation is cropped and resampled to make its resolution consistent with the preset standard size, thereby generating the standardized ID card face image.

[0008] As a further aspect of the present invention, the liveness feature descriptor and the standardized ID card face image are input into a dual-branch feature comparison network, and deep semantic encoding is performed on the two types of images respectively, including: A twin convolutional neural network architecture is constructed as the backbone of the dual-branch feature comparison network, the twin convolutional neural network architecture containing two feature extraction branches with shared weights; The liveness feature descriptor is input into the first branch of the Siamese convolutional neural network architecture. After multiple convolution and pooling operations, a high-dimensional liveness semantic vector is output. The standardized ID card face image is input into the second branch of the twin convolutional neural network architecture. After the same convolution and pooling operations, a high-dimensional ID card semantic vector is output. A fully connected layer is added at the tail of the twin convolutional neural network architecture to fuse the difference information between the liveness semantic vector and the document semantic vector; The vector containing the fused difference information is processed by a normalized exponential function and directly mapped to a preset score range to obtain the initial identity matching score.

[0009] As a further aspect of the present invention, the initial identity matching score and the liveness detection result are input into the authentication decision engine. The authentication decision engine generates a determination result including authentication status and confidence level based on preset multi-factor determination rules, including: The authentication decision engine internally maintains a two-dimensional judgment rule table, where the horizontal axis of the judgment rule table is the Boolean value of the liveness detection result, and the vertical axis is the interval segmentation of the initial identity matching score; When the liveness detection result is unsuccessful, regardless of the initial identity matching score, the authentication decision engine directly sets the authentication status to failure and the confidence level to the lowest level. When the liveness detection result is passed, the authentication decision engine queries the operation instruction corresponding to the interval of the initial identity matching score in the judgment rule table; Based on the retrieved operation instructions, the authentication decision engine performs a weighted correction on the initial identity matching score, and determines whether the authentication status is successful or unsuccessful based on the corrected score, while also determining the corresponding confidence level.

[0010] As a further aspect of the present invention, it also includes: determining whether to initiate the subsequent inference calculation process based on the confidence level in the determination result; If inference computation is initiated, a state transition observation sequence is constructed based on the determination result and the current authentication session log; Using the state transition observation sequence, parameter estimation and state inversion are performed on the hidden Markov model to deduce the user's intent confidence distribution during the authentication process; If inference computation is initiated, a state transition observation sequence is constructed based on the determination result and the current authentication session log, including: The authentication session logs were parsed to extract each user interaction during image capture, including the time of clicking the camera button, the amplitude of shaking measured by the device gyroscope, and the network transmission delay. The authentication status and confidence level in the judgment result are used as state variables, and the specific value of each interaction behavior is used as observation variables. The state variables and observation variables are concatenated in chronological order to form a one-dimensional joint state vector. The joint state vector is processed by a sliding window, and all joint state vectors within a fixed-length window are combined into the state transition observation sequence, which is used to characterize the dynamic evolution of the entire authentication process.

[0011] As a further aspect of the present invention, the state transition observation sequence is used to perform parameter estimation and state inversion on the hidden Markov model, and the user's intent confidence distribution during the authentication process is calculated, including: Initialize a hidden Markov model with three hidden states, which are defined as normal user, uncertain user and malicious attacker, respectively; The emission probability matrix and transition probability matrix of the Hidden Markov Model are re-estimated using the state transition observation sequence, so that the Hidden Markov Model can be adapted to the current business scenario data. After parameter estimation is completed, the forward algorithm is used to calculate the joint probability of the state transition observation sequence in all hidden states, and the backward algorithm is used to calculate the conditional probability from the end of the state transition observation sequence to each state. By combining the forward and backward probabilities, the posterior probabilities of the three hidden states at each time step are calculated. The posterior probability at the last time step is normalized to obtain the intent confidence distribution representing the user's final intent.

[0012] As a further aspect of the present invention, it also includes a step of monitoring abnormal states of the hidden Markov model: While calculating the conditional probability using the backward algorithm, the maximum likelihood path generated by the hidden Markov model during the state transition process is recorded. Calculate the transition probability between adjacent hidden states in the maximum likelihood path. When a transition probability is lower than a very small threshold, it is determined to be an abnormal implicit state transition. When the number of abnormal implicit state transitions exceeds the preset number, even if the intent confidence distribution shows a normal user, the authentication status will be forcibly set to require manual verification, and the time point of the transition and the corresponding observation data will be recorded.

[0013] As a further aspect of the present invention, the vector containing the fused difference information is processed by a normalized exponential function and directly mapped to a preset score interval, including: The input to the normalization exponential function is defined as the vector of fused difference information; The vector containing the fused difference information is compressed into a one-dimensional scalar value through multiple nonlinear transformations. The scalar value obtained by compression is used as the independent variable and input into the normalized exponential function; The normalized exponential function maps input values ​​within any real number range to a closed interval between zero and one by combining exponential operations and linear transformations. The numerical values ​​between zero and one obtained from the mapping are converted into a specific score range with upper and lower bounds through linear scaling and translation, thus obtaining the initial identity matching score.

[0014] As a further aspect of the present invention, the present invention also includes a remote real-name authentication system based on image recognition, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements the steps of the remote real-name authentication method based on image recognition as described above.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This liveness detection method employs micro-expression texture analysis and light reflection feature extraction. By capturing subtle micro-expression changes and the light reflection characteristics of the skin surface, it can accurately distinguish between real and fake faces. Compared to conventional action command verification and single facial feature comparison, it effectively avoids the risk of identity theft caused by fake faces on paper photos or electronic screens, reduces the probability of false positives in liveness detection, improves the security of remote real-name authentication, and makes the authentication results more credible. Micro-expression texture analysis can capture subtle muscle activities that fake faces cannot simulate, while light reflection feature extraction can distinguish the differences in reflection between real skin and photos or screens. The combination of these two methods forms a liveness detection method that verifies the authenticity of liveness from multiple dimensions, overcoming the limitations of conventional liveness detection methods.

[0016] The process involves demoiré filtering and illumination equalization of the ID card chip photo, correcting perspective distortion caused by the shooting angle, and generating a standardized ID card face image. Compared to conventional simple preprocessing methods that only perform cropping, brightness, or contrast adjustments, this method effectively eliminates the interference of moiré patterns on the facial features of the ID card, equalizes the image illumination distribution, and corrects deviations in facial contours and proportions caused by perspective distortion. This results in clearer and more realistic texture features in the ID card face image, leading to greater consistency with the features of live face images. The standardized ID card face image provides a unified benchmark for subsequent feature comparisons, reducing comparison deviations caused by image quality issues, making identity matching scores more accurate, and improving the accuracy and stability of remote real-name authentication. Attached Figure Description

[0017] Figure 1 This is a flowchart of a remote real-name authentication method based on image recognition according to the present invention; Figure 2 This is a flowchart for liveness detection; Figure 3 The flowchart for the authentication decision engine's determination; Figure 4 A time trend analysis chart of certification status and confidence level; Figure 5 This is a time-series analysis diagram of multi-dimensional features for liveness detection. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] See Figure 1 The system receives real-time live face images uploaded by users via a client and photos of ID card chip images read from the contactless chip of ID cards. Liveness detection is performed on the real-time live face images, using micro-expression texture analysis and light reflection feature extraction to determine if the image originates from a real living person, generating a Boolean liveness detection result. Feature point localization is performed on the live face images that pass liveness detection, extracting texture features from the facial contours, periorbital region, and nasolabial folds, integrating these features to form a liveness feature descriptor. Simultaneously, the ID card chip image is preprocessed, including demoiré filtering and illumination equalization to improve image quality and correct perspective distortion caused by the shooting angle, ultimately generating a standardized ID card face image. The liveness feature descriptor and the standardized ID card face image are input into a two-branch feature comparison network. The two branches of this network perform deep semantic encoding on the two types of inputs and output corresponding feature vectors. An initial identity matching score is obtained by calculating the cosine similarity between these two feature vectors. The initial identity matching score and liveness detection result are input into the authentication decision engine. The authentication decision engine generates a final decision result that includes authentication status and confidence level based on its internal preset multi-factor judgment rules.

[0021] In one embodiment of the present invention, see [reference] Figure 2Real-time live face images are captured by the front-facing camera of the user's mobile terminal, while the ID card chip photo is read from the contactless chip of the ID card via a near-field communication protocol. In one example scenario, the user opens an application on their mobile terminal and, following the prompts, holds their ID card close to the near-field communication sensing area of ​​the mobile terminal; the ID card chip photo is then directly read. The user then faces the front-facing camera of the mobile terminal, which captures a real-time live face image containing multiple frames or video clips. During liveness detection of the real-time live face image, it is converted to a grayscale image, and the gradient magnitude and orientation histogram of the grayscale image at different scales are calculated. Time-series data of eyelid closure frequency and mouth corner traction amplitude are extracted from the real-time live face image. For example, during a 3-second acquisition process, the number of complete eyelid closures per second and the change curve of mouth corner muscle movement amplitude are recorded to form time-series data of eyelid closure frequency and mouth corner traction amplitude. These time-series data are input into a temporal convolutional network. The network extracts local and global features from the time-series data through its internal convolutional layers, identifying unnatural blinking patterns or stiff facial expressions. Unnatural blinking patterns may manifest as abnormally fixed blinking frequency or incomplete blinking amplitude, while stiff facial expressions may manifest as a lack of smooth transition in the time-series data of the corners of the mouth being pulled.

[0022] In some embodiments, chromaticity components are separated from the RGB channels of a real-time live face image, and the specular reflection distribution on the skin surface is extracted within a specific wavelength range. One data comparison scenario is that the specular reflection distribution on the surface of real facial skin exhibits irregular and dispersed characteristics, while the regular reflective marks left by screen-captured or printed photos may present a grid-like or striped distribution. The geometry of the extracted specular reflection distribution is compared with a preset real skin lighting model, which defines the statistical characteristic range of the area, shape, and distribution density of specular reflection spots on the real skin surface under standard lighting conditions. Regular reflective marks left by screen-captured or printed photos are identified. The output of the temporal convolutional network is logically ANDed with the optical reflectance analysis result. A liveness detection result is marked as passed only when both the temporal convolutional network output and the optical reflectance analysis result determine that the subject is a real liveness. Both the temporal convolutional network output and the optical reflectance analysis result determining the subject is a real liveness is a necessary condition for generating a passed liveness detection result.

[0023] Optionally, the calculation of optical reflection analysis results can be based on the following formula: in: The true value representing the result of optical reflection analysis. This represents the total number of specular reflection spots detected in real-time live face images. Representing the The shape feature vector of a specular reflection spot The function represents the shape feature reference vector of the preset real skin lighting model. calculate and The matching index between them This represents a preset matching threshold. When the mean matching index is greater than the threshold, the optical reflection analysis result is true.

[0024] In practical implementation, when preprocessing ID card chip photos, a frequency domain bandstop filter is used to filter out periodic moiré patterns caused by digital re-photography. One example scenario is that after an ID card chip photo is read via a near-field communication protocol, moiré patterns may appear due to interference between the image sensor of the reading device and the original printed dots. The frequency domain bandstop filter attenuates or reduces the specific frequency components representing the moiré patterns to zero in the frequency domain of the image, thus outputting a moiré-filtered image in the spatial domain. After filtering out the moiré patterns, a color constancy algorithm is used for white balance adjustment to eliminate ambient light color cast. The color constancy algorithm estimates the color of the scene light source based on the color information of white or neutral-colored areas in the image and performs color correction on the entire image, ensuring that white areas present a consistent color under different lighting conditions, generating a white-balanced image.

[0025] In some embodiments, after illumination correction, the outer border contour of the ID photo area is identified using an edge detection algorithm. Edge detection algorithms, such as the Canny operator, detect points of drastic change in pixel intensity in the white-balanced image by calculating image gradients, connecting these points to form a closed contour line. The coordinates of the four corner points of the outer border contour are calculated, and the original shooting angle of the ID chip photo is calculated based on these coordinates. The original shooting angle is obtained by comparing the positional deviation between the identified outer border contour and a standard rectangle. Based on the calculated original shooting angle, a perspective transformation matrix is ​​applied to project the ID chip photo into the rectangular frame of the standard ID photo, completing perspective distortion correction. The perspective transformation matrix is ​​a 3x3 matrix calculated based on the four corner points of the original outer border contour and the four target corner points of the standard rectangular frame, mapping the tilted quadrilateral region in the original image to a standard rectangular region through matrix operations. The image after perspective transformation is cropped and resampled to match its resolution with a preset standard size, generating a standardized ID face image. The cropping operation is performed based on the coordinates of a standard rectangular frame, and the resampling operation, such as bilinear interpolation, is used to adjust the resolution of the cropped image to a preset fixed size.

[0026] In one embodiment of the present invention, a Siamese convolutional neural network architecture is constructed as the backbone of a two-branch feature comparison network. The Siamese convolutional neural network architecture includes two feature extraction branches with shared weights. A liveness feature descriptor is input into the first branch of the Siamese convolutional neural network architecture. The liveness feature descriptor is a multi-dimensional data matrix containing texture features of facial contours, the periorbital region, and the nasolabial fold region. After multiple convolutional and pooling operations, the first branch outputs a high-dimensional liveness semantic vector, which is a fixed-length feature representation of the liveness feature descriptor after deep semantic encoding. A standardized ID card face image is input into the second branch of the Siamese convolutional neural network architecture. The standardized ID card face image is a pre-processed, uniformly sized color or grayscale image. After multiple convolutional and pooling operations with the same structure and weight parameters as the first branch, the second branch outputs a high-dimensional ID card semantic vector, which is a fixed-length feature representation of the standardized ID card face image after deep semantic encoding.

[0027] In some embodiments, both feature extraction branches of the Siamese convolutional neural network architecture consist of alternating stacks of multiple convolutional layers, batch normalization layers, activation function layers, and max pooling layers. The convolutional layers use small kernels to extract local features of the image, while the pooling layers reduce the spatial dimensionality of the feature maps and enhance the translation invariance of the features. A fully connected layer is added at the tail of the Siamese convolutional neural network architecture. The neurons in the fully connected layer are connected to the high-dimensional vectors output from the two branches, used to fuse the difference information between the liveness semantic vector and the document semantic vector. The fully connected layer learns the non-linear relationship between the liveness semantic vector and the document semantic vector in the feature space and encodes the difference information into a new vector that fuses the difference information.

[0028] Optionally, the input to the normalized exponential function can be defined as a vector incorporating differential information. This vector is then compressed into a one-dimensional scalar value through a multi-layer nonlinear transformation. This multi-layer nonlinear transformation can consist of one or more fully connected layers and activation functions, with the final output layer containing only one neuron. The compressed scalar value is then used as the independent variable and input to the normalized exponential function. The normalized exponential function, through a combination of exponential operations and linear transformations, maps input values ​​within any real number range to a closed interval between zero and one. This mapping process can be expressed in the following form: in: The independent variable representing the scalar value obtained from compression. This represents the output value within the range of zero to one after mapping. Represents the natural constant. It is the slope parameter that controls the steepness of the function curve. It is the position parameter of the function's center point. and These are linear scaling and translation parameters used to adjust the upper and lower bounds of the function output, ensuring that the function's range is confined to the (0,1) interval. The mapped values ​​between zero and one are converted into a specific score interval with upper and lower bounds through linear scaling and translation, resulting in the initial identity matching score. The linear scaling and translation operation can be represented as... ,in It is the initial identity matching score. and These are the upper and lower bounds of the preset score range, respectively.

[0029] In some embodiments, the Siamese convolutional neural network architecture uses a dataset containing a large number of labeled live face and ID card face matching pairs and non-matching pairs during the training phase. The training objective is to minimize the distance between the two semantic vectors of a matching pair and maximize the distance between the two semantic vectors of a non-matching pair. The loss function typically employs contrastive loss or triplet loss. The trained Siamese convolutional neural network architecture is able to map the live features and ID card features of the same person to locations close to each other in the feature space. It can be understood that the two branches of the Siamese convolutional neural network architecture share weights, meaning they use the exact same set of convolutional kernel parameters, which ensures that the live feature descriptors and the standardized ID card face images are processed by the same feature transformation rules. The live semantic vectors and ID card semantic vectors have the same dimension, making it possible to subsequently calculate cosine similarity or fuse differences through fully connected layers.

[0030] In one embodiment of the present invention, see [reference] Figure 3 The authentication decision engine internally maintains a two-dimensional decision rule table. The horizontal axis of the table represents the Boolean value of the liveness detection result, and the vertical axis represents the interval segments of the initial identity matching score. In one example scenario, the Boolean value of the liveness detection result includes two states: "true" and "false." The initial identity matching score interval segments can be pre-divided into four continuous intervals: [0.0, 0.3), [0.3, 0.6), [0.6, 0.8), and [0.8, 1.0]. Each cell in the two-dimensional decision rule table stores a preset operation instruction corresponding to that Boolean value and score interval. The preset operation instruction defines how to correct the score and what authentication status and confidence level to output.

[0031] In some embodiments, when the liveness detection result is negative, regardless of the initial identity matching score range, the authentication decision engine directly sets the authentication status to failure and the confidence level to the lowest level. For example, even if the initial identity matching score is as high as 0.95, if the liveness detection result is marked as "false," indicating that the system determines the current face image is not from a real live person, the authentication decision engine will ignore the high score matching degree and directly output a failed authentication status and the lowest confidence level according to the rules. When the liveness detection result is positive, the authentication decision engine queries the two-dimensional decision rule table for the operation instruction corresponding to the range of the initial identity matching score. For example, if the initial identity matching score is 0.75, which falls within the range [0.6, 0.8), the authentication decision engine will locate the cell corresponding to the liveness detection result being "true" and the score range being [0.6, 0.8), and read the preset operation instruction stored in that cell.

[0032] The preset operation instructions include logic for weighted correction of the initial identity matching score, which can be implemented using a linear transformation formula: in: This represents the initial identity matching score. This represents the weighted score. This represents a correction factor related to the score range and business strategy. This represents the adjusted offset related to the score range and business strategy. Adjustment Factor and correct offset The value is defined in the corresponding cell by a two-dimensional decision rule table. Based on the retrieved preset operation instructions, the authentication decision engine performs a weighted correction on the initial identity matching score and determines whether the authentication status is successful or failed based on the corrected score, while also determining the corresponding confidence level. For example, for the range [0.6, 0.8), the preset operation instructions may specify a correction factor. Correct offset The score is then fine-tuned upwards to pass the certification and assigned a "moderate" confidence level; for the interval [0.3, 0.6), the preset operation instruction may specify a correction factor. Correct offset The score will be adjusted downwards, triggering a failure or requiring manual review status, and a "low" confidence level will be assigned.

[0033] In some embodiments, the preset operation instructions corresponding to different score ranges in the two-dimensional judgment rule table are pre-configured through business rules. The confidence level can be designed as three levels: "high," "medium," and "low," each corresponding to different subsequent processing flows. A successful authentication status with a "high" confidence level allows direct authentication; a successful authentication status with a "medium" confidence level may require recording more detailed audit logs; and a failed authentication status directly rejects the authentication request. It can be understood that the logic of the authentication decision engine is deterministic, and its output is entirely determined by the input liveness detection result boolean value, the initial identity matching score, and the predefined two-dimensional judgment rule table. Weighted correction of the initial identity matching score is a step in which the authentication decision engine performs secondary calibration of the matching score according to business security policies, provided that the liveness detection has passed.

[0034] In one embodiment of the present invention, the system determines whether to initiate subsequent inference calculations based on the confidence level in the judgment result. One example scenario is that the authentication decision engine outputs a judgment result containing "failure, low confidence level." Since the confidence level is the lowest, the system determines that no further inference calculations are needed. Another example scenario is that the authentication decision engine outputs a judgment result containing "success, medium confidence level." Since the confidence level is medium, the system determines that further inference calculations are needed for more in-depth behavioral analysis. If inference calculations are initiated, a state transition observation sequence is constructed based on the judgment result and the current authentication session log. The authentication session log is a structured file containing timestamps and various event data recorded by the system background during a single authentication session.

[0035] The authentication session logs were analyzed to extract each user interaction during image capture. These interactions included the time the camera button was clicked, the amplitude of shaking measured by the device gyroscope, and the network transmission latency. The time the camera button was clicked recorded precisely in milliseconds from the start of the authentication process to the user manually triggering the photo capture. The amplitude of shaking measured by the device gyroscope was expressed as a numerical value of angular velocity or angle change, reflecting the stability of the user's handheld device during face capture. The network transmission latency was the time difference between the image data being uploaded from the client to the server and receiving a response. The authentication status and confidence level in the judgment result were used as state variables. The authentication status could be encoded as discrete values ​​such as "success = 1, failure = 0", and the confidence level could be quantified as discrete values ​​such as "high = 3, medium = 2, low = 1". The specific values ​​of each interaction were used as observation variables, which were the original or normalized values ​​of the time the camera button was clicked, the amplitude of shaking measured by the device gyroscope, and the network transmission latency.

[0036] In some embodiments, consecutive state variables and observation variables are concatenated in chronological order to form a one-dimensional joint state vector. The construction of the joint state vector follows a fixed order. For example, the first element is the encoded value of the authentication status, the second element is the quantized value of the confidence level, and subsequent elements are the time point when the photo button was clicked, the shaking amplitude measured by the device gyroscope, and the network transmission delay. Assuming that at a certain time point t, the state variables are successful authentication status (encoded 1) and medium confidence level (quantized 2), and the observation variables are click time 5000 milliseconds, shaking amplitude 0.5 radians / second, and delay 200 milliseconds, then the joint state vector at that moment can be represented as [1, 2, 5000, 0.5, 200]. Refer to Table 1, which shows a simplified authentication session log fragment and an example of its constructed joint state vector.

[0037] Table 1: Examples of Authentication Session Log Fragments and Joint State Vectors Optional, joint state vector At the point of time The construction can be formally represented as: in: Represents a point in time The constructed joint state vector, Represents a point in time The authentication status code value, Represents a point in time The confidence level quantification value, Represents a point in time The parsed time value of the clicked camera button. Represents a point in time The analyzed value of the shaking amplitude measured by the device's gyroscope. Represents a point in time The parsed network transmission latency value.

[0038] The joint state vector is processed using a sliding window, where the sliding window has a fixed length. All joint state vectors within a fixed-length window are combined into a state transition observation sequence, which is used to characterize the dynamic evolution of the entire authentication process. For example, the sliding window length is set... For the consecutive time points in Table 1, the three joint state vectors [[1,2,1500,0.8,150],[1,2,3200,0.3,180],[1,3,5000,0.1,120]] from t1 to t3 will be combined into a state transition observation sequence. This sequence captures the continuous changes in state and observation data from the start to the end of the authentication process.

[0039] See Figure 4 This is a time trend analysis chart of authentication status and confidence level, used to illustrate the dynamic relationship between authentication status and confidence level over time in the real-name authentication process. The authentication status remains successful for most of the time, only briefly failing at points t4 and t7, indicating high overall stability. The confidence level reaches a high level at t3, t6, t8, and t9, a medium level at t1, t2, t5, and t10, and drops to a low level at t4 and t7. The fluctuation of the confidence level is highly synchronized with the authentication status; when the authentication status fails, the confidence level inevitably drops to its lowest point, while when the authentication status is successful, the confidence level fluctuates between medium and high levels. There is a strong causal relationship between authentication status and confidence level; failure of liveness detection or feature comparison will directly lead to a collapse in the confidence level. During the t1-t3 period, the authentication status remained successful, and the confidence level rose from medium to high. The authentication status failed at t4 and t7, and the confidence level simultaneously dropped to its lowest point, representing risky points in the process. The T5-T6 and T8-T10 authentication statuses have been successfully restored, and the confidence level has quickly recovered to a medium / high level.

[0040] In one embodiment of the present invention, the Hidden Markov Model (HMM) is used to estimate parameters and invert states using state transition observation sequences to deduce the user's intent confidence distribution during the authentication process. An HMM with three hidden states is initialized, defined as normal user, uncertain user, and malicious attacker, respectively. An example scenario is in the remote account opening process of financial services. The hidden state of normal user represents the intent of the real identity holder to perform compliant operations, the hidden state of uncertain user represents the intent of slightly abnormal behavior due to network latency or unfamiliarity with operation, and the hidden state of malicious attacker represents the intent to use forged biometrics or automated scripts to launch attacks. Data comparison shows that the authentication session of normal user often produces a smooth and expected state transition observation sequence, while the session of malicious attacker may produce a sequence containing abrupt or contradictory observation data. The emission probability matrix and transition probability matrix of the Hidden Markov Model (HMM) are re-estimated using state transition observation sequences to adapt the HMM to the current business scenario data. The emission probability matrix defines the probability distribution of generating a specific joint state vector from each hidden state, and the transition probability matrix defines the probability of mutual transformation between the three hidden states. The re-estimate process adopts an iterative approach using the expectation-maximization algorithm, updating the parameters of the HMM using the actual collected state transition observation sequences until the log-likelihood of the model converges.

[0041] In some embodiments, after parameter estimation, the forward algorithm is used to calculate the joint probability of the state transition observation sequence in all hidden states. The forward algorithm recursively calculates the forward probability of reaching each time step and each hidden state starting from the beginning time step of the sequence. Then, the backward algorithm is used to calculate the conditional probability from the end of the state transition observation sequence to each state. The backward algorithm recursively calculates the conditional probability from each time step to the end of the sequence starting from the end time step of the sequence. Combining the forward and backward probabilities, the posterior probability of the three hidden states at each time step is calculated. The posterior probability reflects the probability that each time step belongs to a certain hidden state given the entire observation sequence. The posterior probability of the last time step is normalized to obtain the intent confidence distribution representing the user's final intent. The intent confidence distribution is a three-dimensional probability vector, whose three components correspond to the probability values ​​of normal users, uncertain users, and malicious attackers, respectively, and their sum is one. For example, after an authentication, the intent confidence distribution may be [0.92, 0.05, 0.03].

[0042] Optional, posterior probability At time step For hidden state The calculation can be formally represented as: in: Represents the time step Hidden state The posterior probability, The time step is calculated using the forward algorithm. Hidden state The forward probability, The time step is calculated using the backward algorithm. Hidden state The backward probability, This involves summing the indices and traversing the three hidden states. While calculating the conditional probabilities using the backward algorithm, the maximum likelihood path generated by the Hidden Markov Model during state transitions is recorded. The maximum likelihood path is obtained through Viterbi decoding, which uses dynamic programming to find the sequence of hidden states most likely to generate the entire state transition observation sequence. The transition probabilities between adjacent hidden states in the maximum likelihood path are calculated. When a transition probability falls below a very small threshold, it is considered an abnormal hidden state jump. For example, the probability of jumping directly from a normal user's hidden state to a malicious attacker's hidden state is usually extremely low in a well-trained Hidden Markov Model. If the probability of such a transition in the actual path is lower than a preset threshold, such as 0.005, an anomaly marker is triggered.

[0043] In some embodiments, the cumulative number of abnormal implicit state transitions is implemented using a counter. Each time a transition with a probability below a threshold is detected, the counter increments by one. When the cumulative number of abnormal implicit state transitions exceeds a preset number, even if the final calculated intent confidence distribution indicates a normal user, the system will forcibly set the authentication status to require manual review and record the time of the transition and the corresponding observation data. For example, if three low-probability transitions from an uncertain user to a malicious attacker are detected in a session, even though the final intent confidence distribution shows a normal user probability of 0.88, the system will still trigger a manual review process and record the specific timestamp of each transition, the time the camera button was clicked, the shaking amplitude measured by the device gyroscope, and the network transmission delay.

[0044] See Figure 5This is a time-series analysis chart of multi-dimensional features for liveness detection, used to demonstrate the dynamic changes of micro-expressions and optical features over time and the final detection results during the real-name authentication liveness detection process. The eyelid closure frequency continuously increases from 0 to 2.5 seconds, reaching a peak of 1.0 at 2.5 seconds, and then gradually decreases to 0 at 5 seconds, consistent with the physiological rhythm of natural blinking. The corner-of-mouth pulling amplitude shows a generally gradual trend of first increasing and then decreasing, reaching a peak of 0.5 at 2.5 seconds, without drastic abnormal fluctuations, indicating a natural facial expression. The similarity of specular reflection spot distribution continuously increases from 0.3 at 0 seconds to 0.95 at 2.5 seconds, then slowly decreases, highly matching the lighting model of real skin, and no regular reflection traces from copied / printed photos were found. The liveness detection result is 0 (failed) from 0 to 1.5 seconds, at which point the micro-expressions and optical features have not yet accumulated to a sufficient threshold for judgment. The detection result is 1 (pass) after 2-3.5 seconds. At this time, the eyelid closure frequency, the corner of the mouth traction amplitude, and the optical similarity are all at a high level, which meets the judgment condition that "the output of the temporal convolutional network and the optical reflection analysis are both true". The detection result drops back to 0 (fail) after 4-5 seconds, and the corresponding feature value decreases synchronously, and the liveness detection window ends.

[0045] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An image recognition-based remote real-name authentication method, characterized in that, include: The system receives real-time live face images and ID card chip photos uploaded by users, performs liveness detection on the real-time live face images, and determines whether the real-time live face images are from real live people through micro-expression texture analysis and light reflection feature extraction, and generates liveness detection results. Feature point localization is performed on the real-time live face image that has passed the liveness detection, and texture features of facial contour, periorbital region and nasolabial fold region are extracted to form a liveness feature descriptor; The ID card chip photo is preprocessed by removing moiré filtering and applying illumination equalization to correct perspective distortion caused by the shooting angle and generate a standardized ID card face image. The liveness feature descriptor and the standardized ID card face image are input into a dual-branch feature matching network, and deep semantic encoding is performed on the two types of images respectively. An initial identity matching score is obtained by calculating the cosine similarity of the feature vectors output by the dual-branch feature matching network. The initial identity matching score and the liveness detection result are input into the authentication decision engine, which generates a judgment result including authentication status and confidence level according to the preset multi-factor judgment rules.

2. The remote real-name authentication method based on image recognition according to claim 1, characterized in that, Liveness detection is performed on the real-time live face image. Through micro-expression texture analysis and light reflection feature extraction, it is determined whether the real-time live face image originates from a real live person, including: The real-time live face image is captured by the front-facing camera of the mobile terminal, and the photo of the ID card chip is read from the contactless chip of the ID card through a near-field communication protocol; The real-time live human face image is converted into a grayscale image, and its gradient magnitude and orientation histogram are calculated at different scales. Extract time-series data of eyelid closure frequency and mouth corner pulling amplitude from the real-time live human face image, and input the time-series data into a temporal convolutional network to identify unnatural blinking patterns or stiff facial expressions. The chromaticity component is separated from the RGB channels of the real-time live human face image, and the specular reflection spot distribution on the skin surface is extracted within a specific wavelength range; By comparing the geometric shape of the specular reflection spot distribution with a preset real skin lighting model, the regular reflection marks left by the screen photo or printed photo can be identified. The output of the temporal convolutional network is logically ANDed with the optical reflection analysis result. Only when both the output of the temporal convolutional network and the optical reflection analysis result are determined to be true, is the liveness detection result marked as passed.

3. The remote real-name authentication method based on image recognition of claim 2, wherein, The ID card chip photo is preprocessed by performing demoiré filtering and illumination equalization to correct perspective distortion caused by the shooting angle, generating a standardized ID card face image, including: The ID card chip photo is processed using a frequency domain band-stop filter to filter out periodic moiré patterns caused by digital re-photography. After filtering out moiré patterns, the white balance of the ID card chip photo is adjusted using a color constancy algorithm to eliminate color cast due to ambient light. After completing the illumination correction, the outer border outline of the ID photo area is identified using an edge detection algorithm; Calculate the coordinates of the four corner points of the outer frame outline, and calculate the original shooting angle of the ID card chip photo based on the corner point coordinates; Based on the calculated original shooting angle, the perspective transformation matrix is ​​applied to project the ID card chip photo into the rectangular frame of the standard ID photo to complete the perspective distortion correction. The image after perspective transformation is cropped and resampled to make its resolution consistent with the preset standard size, thereby generating the standardized ID card face image.

4. The remote real-name authentication method based on image recognition of claim 3, wherein, The liveness feature descriptor and the standardized ID card face image are input into a two-branch feature comparison network, and deep semantic encoding is performed on the two types of images respectively, including: A twin convolutional neural network architecture is constructed as the backbone of the dual-branch feature comparison network, the twin convolutional neural network architecture containing two feature extraction branches with shared weights; The liveness feature descriptor is input into the first branch of the Siamese convolutional neural network architecture. After multiple convolution and pooling operations, a high-dimensional liveness semantic vector is output. The standardized ID card face image is input into the second branch of the twin convolutional neural network architecture. After the same convolution and pooling operations, a high-dimensional ID card semantic vector is output. A fully connected layer is added at the tail of the twin convolutional neural network architecture to fuse the difference information between the liveness semantic vector and the document semantic vector; The vector containing the fused difference information is processed by a normalized exponential function and directly mapped to a preset score range to obtain the initial identity matching score.

5. The remote real-name authentication method based on image recognition of claim 4, wherein, The initial identity matching score and the liveness detection result are input into the authentication decision engine. The authentication decision engine, based on preset multi-factor judgment rules, generates a judgment result including authentication status and confidence level, including: The authentication decision engine internally maintains a two-dimensional judgment rule table, where the horizontal axis of the judgment rule table is the Boolean value of the liveness detection result, and the vertical axis is the interval segmentation of the initial identity matching score; When the liveness detection result is unsuccessful, regardless of the initial identity matching score, the authentication decision engine directly sets the authentication status to failure and the confidence level to the lowest level. When the liveness detection result is passed, the authentication decision engine queries the operation instruction corresponding to the interval of the initial identity matching score in the judgment rule table; Based on the retrieved operation instructions, the authentication decision engine performs a weighted correction on the initial identity matching score, and determines whether the authentication status is successful or unsuccessful based on the corrected score, while also determining the corresponding confidence level.

6. The remote real-name authentication method based on image recognition as described in claim 5, characterized in that, Also includes: Based on the confidence level in the judgment result, a decision is made on whether to initiate the subsequent inference calculation process; If inference computation is initiated, a state transition observation sequence is constructed based on the determination result and the current authentication session log; Using the state transition observation sequence, parameter estimation and state inversion are performed on the hidden Markov model to deduce the user's intent confidence distribution during the authentication process; If inference computation is initiated, a state transition observation sequence is constructed based on the determination result and the current authentication session log, including: The authentication session logs were parsed to extract each user interaction during image capture, including the time of clicking the camera button, the amplitude of shaking measured by the device gyroscope, and the network transmission delay. The authentication status and confidence level in the judgment result are used as state variables, and the specific value of each interaction behavior is used as observation variables. The state variables and observation variables are concatenated in chronological order to form a one-dimensional joint state vector. The joint state vector is processed by a sliding window, and all joint state vectors within a fixed-length window are combined into the state transition observation sequence, which is used to characterize the dynamic evolution of the entire authentication process.

7. The remote real-name authentication method based on image recognition as described in claim 6, characterized in that, Using the state transition observation sequence, parameter estimation and state inversion are performed on the Hidden Markov Model to deduce the user's intent confidence distribution during the authentication process, including: Initialize a hidden Markov model with three hidden states, which are defined as normal user, uncertain user and malicious attacker, respectively; The emission probability matrix and transition probability matrix of the Hidden Markov Model are re-estimated using the state transition observation sequence, so that the Hidden Markov Model can be adapted to the current business scenario data. After parameter estimation is completed, the forward algorithm is used to calculate the joint probability of the state transition observation sequence in all hidden states, and the backward algorithm is used to calculate the conditional probability from the end of the state transition observation sequence to each state. By combining the forward and backward probabilities, the posterior probabilities of the three hidden states at each time step are calculated. The posterior probability at the last time step is normalized to obtain the intent confidence distribution representing the user's final intent.

8. The remote real-name authentication method based on image recognition as described in claim 7, characterized in that, It also includes the step of monitoring abnormal states of the hidden Markov model: While calculating the conditional probability using the backward algorithm, the maximum likelihood path generated by the hidden Markov model during the state transition process is recorded. Calculate the transition probability between adjacent hidden states in the maximum likelihood path. When a transition probability is lower than a very small threshold, it is determined to be an abnormal implicit state transition. When the number of abnormal implicit state transitions exceeds the preset number, even if the intent confidence distribution shows a normal user, the authentication status will be forcibly set to require manual verification, and the time point of the transition and the corresponding observation data will be recorded.

9. The remote real-name authentication method based on image recognition as described in claim 8, characterized in that, The vector containing the fused difference information is processed using a normalized exponential function and directly mapped to a preset score interval, including: The input to the normalization exponential function is defined as the vector of fused difference information; The vector containing the fused difference information is compressed into a one-dimensional scalar value through multiple nonlinear transformations. The scalar value obtained by compression is used as the independent variable and input into the normalized exponential function; The normalized exponential function maps input values ​​within any real number range to a closed interval between zero and one by combining exponential operations and linear transformations. The numerical values ​​between zero and one obtained from the mapping are converted into a specific score range with upper and lower bounds through linear scaling and translation, thus obtaining the initial identity matching score.

10. A remote real-name authentication system based on image recognition, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the remote real-name authentication method based on image recognition as described in any one of claims 1 to 9.

Images