Identity identification method and device, storage medium and equipment

Abstract

The application discloses an identity recognition method and device, a storage medium and equipment, and belongs to the field of biological recognition. The application utilizes the correlation between eyeball deflection and face posture deflection to quickly and accurately find user posture deflection. First, eyeball deflection is judged based on an iris image to determine whether the face posture is deflected, then the deflection direction and angle of the face posture are judged based on a face image, and correction is performed to obtain an iris image of a to-be-recognized object when the eyes are in orthoptic state, which meets the actual application requirements. The application improves the success rate and efficiency of iris image acquisition, and further improves the accuracy of iris recognition, and can realize detection and correction of large-angle face posture deflection, and can meet the requirements of face posture diversity in the actual application process.

Description

Technical Field

[0001] This invention relates to the field of biometrics, and in particular to an identity recognition method, apparatus, storage medium and device. Background Technology

[0002] With the continuous development of the internet in modern society, people's demands for the security and stability of information, especially personal information, are constantly increasing. Applications of identity authentication utilizing the unique physiological or behavioral characteristics inherent in the human body are becoming increasingly widespread. For example, fingerprint, iris, facial, and vein authentication methods and devices are now widely used in daily life and security.

[0003] However, each single biometric identification technology currently has its limitations. For example, people with damaged fingerprints cannot use fingerprint recognition, meaning that single biometric identification technologies cannot meet everyone's needs. Therefore, multimodal biometric identification technology has become a hot topic in recent years. It overcomes the limitations of single biometric identification to some extent, and the overall recognition accuracy of multimodal biometric identification technology is usually higher than that of single biometric identification technology.

[0004] In multimodal biometrics, face and iris recognition is the most promising biometric technology. The process of face and iris multimodal biometrics recognition first requires acquiring iris and face images that meet certain requirements. The quality of these images directly affects the speed and accuracy of the recognition. However, a significant issue during image acquisition is that factors such as head tilt or user rotation can cause facial and eye deviations, leading to a decrease in the quality of the acquired face and iris images.

[0005] Therefore, in the process of multimodal facial iris recognition, how to detect user pose deviations and promptly alert the user is a pressing technical problem. Existing technologies use integral projection to locate key points in facial images (including the left inner eye point, right inner eye point, nose tip, and corners of the mouth), then calculate the tilt angle based on the two inner eye points, and finally correct the facial image using an affine transformation function. However, this method can only effectively detect angles within a range of approximately 25°, which cannot meet the requirements of diverse facial poses in practical applications. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an identity recognition method, apparatus, storage medium, and device, which improves the success rate and efficiency of iris image acquisition, thereby enhancing the accuracy of iris recognition. Furthermore, it enables the detection and correction of large-angle facial pose deviations, meeting the requirements of diverse facial poses in practical applications.

[0007] The technical solution provided by this invention is as follows:

[0008] In a first aspect, the present invention provides an identity recognition method, the method comprising:

[0009] Acquire the iris image of the object to be identified;

[0010] The pupil boundary is fitted with an ellipse curve on the iris image, and the eyeball deflection of the object to be identified is determined based on the fitted ellipse curve.

[0011] If it is determined that the object to be identified has deflected its eyeballs, then the face image of the object to be identified is acquired, the face pose of the object to be identified is determined based on the face image, and the iris image of the object to be identified when its eyes are looking straight ahead is acquired based on the face pose of the object to be identified; if it is determined that the object to be identified has not deflected its eyeballs, then the iris image is determined to be the iris image of the object to be identified when its eyes are looking straight ahead.

[0012] The identity of the object to be identified is determined based on the iris image of the object when the human eye is looking straight ahead.

[0013] Furthermore, the step of fitting an ellipse curve to the pupil boundary on the iris image and determining whether the object to be identified has undergone eyeball deflection based on the fitted elliptical curve includes:

[0014] The pupil is located on the iris image to obtain the pupil boundary;

[0015] Gradient information from the iris image is used to correct points on the pupil boundary and remove noise, resulting in a valid set of points on the pupil boundary.

[0016] Elliptic curve fitting is performed based on the effective point set on the pupil boundary, and the flatness of the elliptic curve is used to determine whether the object to be identified has undergone eyeball deflection.

[0017] Furthermore, the step of using gradient information from the iris image to correct points on the pupil boundary and remove noise to obtain a valid set of points on the pupil boundary includes:

[0018] Spot detection is performed on the iris image, and spot removal is achieved using biquadratic interpolation;

[0019] The annular region image with center (y,x) and radius range [r1,r2] on the iris image after removing light spots is expanded into a rectangular region image by taking values ​​at certain angle intervals;

[0020] Where (y,x) represents the pupil center obtained by locating the pupil on the iris image, r1 = a1r, r2 = a2r, r is the pupil radius obtained by locating the pupil on the iris image, and a1 and a2 are set coefficients, 0 <a1<1<a2;

[0021] Smoothing filtering is performed on the rectangular region image, and alternating subtraction is performed. Values ​​greater than 0 after subtraction are retained, and values ​​less than or equal to 0 are set to 0 to obtain the gradient image.

[0022] Obtain the maximum pixel value and its coordinates for each row of the gradient image. Replace the maximum pixel value of each row with the maximum pixel value of that row and the average value of several pixels near the maximum pixel value in that row. Then, the rows whose maximum pixel value after replacement is not less than a set threshold are considered as valid rows.

[0023] The coordinates of the maximum pixel values ​​in the valid rows are transformed back onto the annular region of the iris image to obtain the set of valid points on the pupil boundary.

[0024] Furthermore, the step of fitting an elliptic curve based on the effective set of points on the pupil boundary includes:

[0025] Get the points (y_max, xx2) and (y_min, xx1) with the maximum and minimum ordinates on the effective point set on the pupil boundary, and the points (yy2, x_max) and (yy1, x_min) with the maximum and minimum abscissas.

[0026] Calculate the initial center point (y_point, x_point), the initial major axis a, and the initial minor axis b;

[0027] Where y_point=(y_min+y_max) / 2, x_point=(x_min+x_max) / 2, a=max(a_point,b_point), b=min(a_point,b_point), a_point is the Euclidean distance between (y_min,xx1) and (y_max,xx2), and b_point is the Euclidean distance between (yy1,x_min) and (yy2,x_max);

[0028] Define the traversal range of the center point, major axis, and minor axis based on the initial center point, initial major axis, and initial minor axis;

[0029] For each value within the traversal range of the center point, major axis, and minor axis, perform elliptic curve fitting, and find the elliptic curve that matches the set of valid points on the pupil boundary most often as the fitted elliptic curve.

[0030] Furthermore, the step of obtaining the iris image of the object to be identified when its eyes are looking straight ahead based on the facial pose of the object to be identified includes:

[0031] The iris image is corrected based on the facial posture to obtain the iris image of the object to be identified when the human eye is looking straight ahead;

[0032] Alternatively, based on the facial pose prompting the object to be identified to adjust its posture, the process returns to the step of obtaining the iris image of the object to be identified, until the iris image of the object to be identified when its eyes are looking straight ahead is obtained.

[0033] Furthermore, the method is used for face and iris multimodal identity recognition.

[0034] Furthermore, determining the facial pose of the object to be identified based on the facial image includes:

[0035] The face image is input into a convolutional neural network to obtain the face pose of the object to be identified;

[0036] The convolutional neural network includes an image preprocessing part and a first branch and a second branch located in parallel after the image preprocessing part;

[0037] The face image undergoes a series of convolution operations, activation operations, absolute value operations, and pooling operations in the image preprocessing part to obtain a preprocessed feature map.

[0038] The preprocessed feature map undergoes a series of convolution operations, activation operations, absolute value operations, and inner product operations in the first branch, and the coordinates of the facial feature points are calculated by combining the width and height of the face image.

[0039] The first set of three-dimensional facial pose angles is calculated based on the coordinates of the facial feature points.

[0040] The preprocessed feature map is processed through a series of convolution operations, BN operations, activation operations, and inner product operations in the second branch to obtain the second set of three-dimensional face pose angles.

[0041] The facial pose of the object to be identified is obtained based on the first set of three-dimensional facial pose angles and the second set of three-dimensional facial pose angles.

[0042] In a second aspect, the present invention provides an identity recognition device, the device comprising:

[0043] The image acquisition module is used to acquire the iris image of the object to be identified;

[0044] An eye deflection detection module is used to fit an ellipse curve to the pupil boundary on the iris image, and to determine whether the object to be identified has undergone eye deflection based on the fitted ellipse curve.

[0045] The iris image acquisition module is used to acquire a face image of the subject if it is determined that the subject has deflected its eyeballs, determine the face pose of the subject based on the face image, and acquire an iris image of the subject when its eyes are looking straight ahead based on the face pose of the subject; if it is determined that the subject has not deflected its eyeballs, the module determines that the iris image is an iris image of the subject when its eyes are looking straight ahead.

[0046] The identity recognition module is used to identify the object to be identified based on the iris image of the object when the human eye is looking straight ahead.

[0047] Furthermore, the eye deflection determination module includes:

[0048] The pupil boundary extraction unit is used to locate the pupil on the iris image and obtain the pupil boundary;

[0049] The effective point set determination unit is used to correct the points on the pupil boundary and remove noise using the gradient information of the iris image to obtain the effective point set on the pupil boundary.

[0050] The fitting and turning judgment unit is used to perform elliptic curve fitting based on the effective point set on the pupil boundary, and to determine whether the object to be identified has undergone eyeball deflection based on the flatness of the elliptic curve.

[0051] Furthermore, the effective point set determination unit includes:

[0052] A spot removal unit is used to detect spots in the iris image and remove spots using biquadratic interpolation.

[0053] The rectangular region image unfolding unit is used to unfold the annular region image with center (y,x) and radius range [r1,r2] on the iris image after removing light spots into a rectangular region image by taking values ​​at certain intervals of angle;

[0054] Where (y,x) represents the pupil center obtained by locating the pupil on the iris image, r1 = a1r, r2 = a2r, r is the pupil radius obtained by locating the pupil on the iris image, and a1 and a2 are set coefficients, 0 <a1<1<a2;

[0055] The gradient image acquisition unit is used to perform smoothing filtering on the rectangular region image and perform interlaced subtraction, retaining the values ​​greater than 0 after subtraction and setting the values ​​less than or equal to 0 to 0, thereby obtaining the gradient image;

[0056] The valid row determination unit is used to obtain the maximum pixel value and its coordinates of each row of the gradient image, replace the maximum pixel value of each row with the maximum pixel value of the row and the average value of several pixels near the maximum pixel value in the row, and take the row with the maximum pixel value after replacement not less than a set threshold as a valid row.

[0057] The coordinate transformation unit is used to transform the coordinates of the maximum pixel value of the effective row back to the annular region image of the iris image, so as to obtain the effective point set on the pupil boundary.

[0058] Furthermore, in the fitting and conversion judgment unit, elliptic curve fitting is performed based on the effective point set on the pupil boundary, including:

[0059] The data acquisition unit is used to acquire the points (y_max, xx2) and (y_min, xx1) with the maximum and minimum vertical coordinates in the effective point set on the pupil boundary, and the points (yy2, x_max) and (yy1, x_min) with the maximum and minimum horizontal coordinates.

[0060] The initial parameter calculation unit is used to calculate the initial center point (y_point, x_point), the initial major axis a, and the initial minor axis b;

[0061] Where y_point=(y_min+y_max) / 2, x_point=(x_min+x_max) / 2, a=max(a_point,b_point), b=min(a_point,b_point), a_point is the Euclidean distance between (y_min,xx1) and (y_max,xx2), and b_point is the Euclidean distance between (yy1,x_min) and (yy2,x_max);

[0062] The traversal range determination unit is used to set the traversal range of the center point, major axis, and minor axis based on the initial center point, initial major axis, and initial minor axis;

[0063] The traversal unit is used to perform elliptic curve fitting for each value within the traversal range of the center point, major axis, and minor axis, and find the elliptic curve that matches the set of valid points on the pupil boundary the most.

[0064] Furthermore, in the iris image acquisition module, acquiring the iris image of the object being identified when its eyes are looking straight ahead, based on the facial pose of the object to be identified, includes:

[0065] The correction unit is used to correct the iris image according to the facial posture to obtain the iris image of the object to be identified when the eyes are looking straight ahead;

[0066] Alternatively, the adjustment unit is used to prompt the object to be identified to adjust its posture according to the facial posture and then return to the step of obtaining the iris image of the object to be identified until the iris image of the object to be identified when its eyes are looking straight ahead is obtained.

[0067] Furthermore, the device is used for multimodal facial and iris recognition.

[0068] Furthermore, in the emphyseal image acquisition module, determining the facial pose of the object to be identified based on the facial image includes:

[0069] The face image is input into a convolutional neural network to obtain the face pose of the object to be identified;

[0070] The convolutional neural network includes an image preprocessing part and a first branch and a second branch located in parallel after the image preprocessing part;

[0071] The face image undergoes a series of convolution operations, activation operations, absolute value operations, and pooling operations in the image preprocessing part to obtain a preprocessed feature map.

[0072] The preprocessed feature map undergoes a series of convolution operations, activation operations, absolute value operations, and inner product operations in the first branch, and the coordinates of the facial feature points are calculated by combining the width and height of the face image.

[0073] The first set of three-dimensional facial pose angles is calculated based on the coordinates of the facial feature points.

[0074] The preprocessed feature map is processed through a series of convolution operations, BN operations, activation operations, and inner product operations in the second branch to obtain the second set of three-dimensional face pose angles.

[0075] The facial pose of the object to be identified is obtained based on the first set of three-dimensional facial pose angles and the second set of three-dimensional facial pose angles.

[0076] Thirdly, the present invention provides a computer-readable storage medium for identity verification, including a memory for storing processor-executable instructions, which, when executed by the processor, implement the steps of the identity verification method described in the first aspect.

[0077] Fourthly, the present invention provides an apparatus for identity recognition, comprising at least one processor and a memory storing computer-executable instructions, wherein the processor, when executing the instructions, implements the steps of the identity recognition method described in the first aspect.

[0078] The present invention has the following beneficial effects:

[0079] This invention first acquires the iris image of the object to be identified, then uses an elliptical curve fitted to the pupil boundary of the iris image to determine whether the object has undergone eye movement. When the object has undergone eye movement, it indicates that the face posture has been deflected. Next, the face posture of the object to be identified is determined by the acquired face image, and the direction and angle of the face deflection are obtained. Based on the face posture of the object to be identified, the iris image of the object when its eyes are looking straight ahead is acquired, and the identity recognition is performed using the iris image of the object when its eyes are looking straight ahead.

[0080] This invention utilizes the correlation between eye movement and facial pose deviation to quickly and accurately detect user pose deviation. First, eye movement is assessed based on iris images to determine if facial pose deviation exists. Then, the direction and angle of facial pose deviation are determined based on the facial image and corrected to obtain an iris image of the subject looking straight ahead, meeting practical application requirements. This invention improves the success rate and efficiency of iris image acquisition, thereby enhancing the accuracy of iris recognition. Furthermore, it enables the detection and correction of large-angle facial pose deviations, meeting the diverse facial pose requirements of practical applications. Attached Figure Description

[0081] Figure 1 A schematic diagram of a normal iris image;

[0082] Figure 2 A schematic diagram of the iris image showing eye movement;

[0083] Figure 3 This is a flowchart of the identity recognition method of the present invention;

[0084] Figure 4 A schematic diagram showing the initial pupil localization in an iris image;

[0085] Figure 5 A schematic diagram showing the iris image after removing light spots;

[0086] Figure 6 This is a schematic diagram of a rectangular region image;

[0087] Figure 7 This is a schematic diagram of a gradient image;

[0088] Figure 8 A schematic diagram of the effective point set on the pupil boundary;

[0089] Figure 9 This is a schematic diagram of an ellipse;

[0090] Figure 10 This is a schematic diagram of coordinate transformation;

[0091] Figure 11 A schematic diagram of the three-dimensional pose angles of a face;

[0092] Figure 12 This is a schematic diagram of the identity recognition device of the present invention. Detailed Implementation

[0093] To make the technical problems, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. The components of the embodiments of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0094] Example 1:

[0095] This invention provides an identity recognition method, such as... Figure 3 As shown, the method includes:

[0096] S100: Acquire the iris image of the object to be identified.

[0097] S200: Fit the pupil boundary to an ellipse on the iris image, and determine whether the object to be identified has undergone eyeball deflection based on the fitted elliptical curve.

[0098] An iris image without eye deviation, such as Figure 1 As shown, the pupil boundary (i.e., the inner circle of the iris) is circular, and the iris image when the eyeball deflects is as follows. Figure 2 As shown, the pupil boundary is elliptical. Therefore, by fitting a curve to the pupil boundary as an ellipse, the result of the fitted elliptical curve can be used to determine whether the object being identified has experienced eyeball deflection.

[0099] S300: If it is determined that the object to be identified has undergone eye movement, then the face image of the object to be identified is acquired, the face posture of the object to be identified is determined based on the face image, and the iris image of the object to be identified when its eyes are looking straight ahead is acquired based on the face posture of the object to be identified; if it is determined that the object to be identified has not undergone eye movement, then the iris image is determined to be the iris image of the object to be identified when its eyes are looking straight ahead.

[0100] Since eye movement and facial pose deviation are related, facial pose deviation will cause eye movement. Therefore, when the object to be identified moves its eyes, it means that the facial pose of the object to be identified has deviated. However, the specific direction and angle of facial pose deviation cannot be further determined based on the elliptic curve alone. It is necessary to collect facial images to further determine the facial pose.

[0101] When acquiring a face image, it can be acquired simultaneously with the iris image in S100, or the face image can be acquired at a short interval from the acquisition of the iris image in S100.

[0102] This invention does not limit the method for determining facial pose. Facial pose can be represented by pose angles. The direction and angle of facial deflection can be obtained from the pose angles, and the iris image of the subject when looking straight ahead can be obtained by correcting the pose angles. Specifically, there are two possible implementation methods:

[0103] 1. Correct the iris image according to the facial posture to obtain the iris image of the object to be identified when the eyes are looking straight ahead.

[0104] This invention does not limit the specific correction method; various methods for correcting facial posture in the prior art can be used.

[0105] 2. After the object to be identified adjusts its posture according to the facial posture prompt, return to the step of obtaining the iris image of the object to be identified, until the iris image of the object to be identified when its eyes are looking straight ahead is obtained.

[0106] For example, when the face is judged to be turning left, the user is prompted to turn their head to the right; when the face is judged to be turning right, the user is prompted to turn their head to the left; when the face is judged to be tilted upward, the user is prompted to adjust their posture downward; when the face is judged to be tilted downward, the user is prompted to adjust their posture upward.

[0107] After the user corrects their facial posture according to the prompts, the iris image is captured until the iris image of the object to be identified is obtained when the eyes are looking straight ahead.

[0108] S400: Identify the object to be identified based on the iris image of the object when it is looking straight ahead.

[0109] Identification using iris images taken when the human eye is looking straight ahead can improve accuracy. If necessary, the S300 can also simultaneously acquire facial images taken when the human eye is looking straight ahead, which can then be used together with the iris images taken when the human eye is looking straight ahead for multimodal facial and iris recognition.

[0110] This invention first acquires the iris image of the object to be identified, then uses an elliptical curve fitted to the pupil boundary of the iris image to determine whether the object has undergone eye movement. When the object has undergone eye movement, it indicates that the face posture has been deflected. Next, the face posture of the object to be identified is determined by the acquired face image, and the direction and angle of the face deflection are obtained. Based on the face posture of the object to be identified, the iris image of the object when its eyes are looking straight ahead is acquired, and the identity recognition is performed using the iris image of the object when its eyes are looking straight ahead.

[0111] This invention utilizes the correlation between eye movement and facial pose deviation to quickly and accurately detect user pose deviation. First, eye movement is assessed based on iris images to determine if facial pose deviation exists. Then, the direction and angle of facial pose deviation are determined based on the facial image and corrected to obtain an iris image of the subject looking straight ahead, meeting practical application requirements. This invention improves the success rate and efficiency of iris image acquisition, thereby enhancing the accuracy of iris recognition. Furthermore, it enables the detection and correction of large-angle facial pose deviations, meeting the diverse facial pose requirements of practical applications.

[0112] As one implementation of this invention, the aforementioned S200 includes:

[0113] S210: Locate the pupil on the iris image to obtain the pupil boundary.

[0114] This step is used to locate the pupil on the iris image, obtaining the pupil center, pupil radius, and pupil boundary. The specific implementation includes:

[0115] S211: Binarize the iris image and use the curve fitting method of symmetrical radial transformation to locate the pupil, thereby obtaining the pupil center and pupil radius.

[0116] In this step, binarization is first performed to obtain a binarized image. Then, using algorithms such as Sobel, the pupil boundary is initially located as a circle through symmetric radial transformation to obtain parameters (y, x, r), where (y, x) are the coordinates of the pupil center and r is the pupil radius.

[0117] One example of the location results is as follows: Figure 4 As shown, the white arc line is the boundary of the pupil, which is the boundary of a circle, and the white cross is the center of the pupil circle.

[0118] S212: The iris image is segmented into several connected regions using an eight-way connected labeling algorithm. The pupil region is determined by comparing the area ratio of each connected region and the distance between the centroid of each connected region and the center of the pupil.

[0119] After initial localization of the pupil as a circle in S211, this step performs fine localization. Specific methods for fine localization may include: removing interference regions such as eyelashes from the binarized image; segmenting the iris image using an eight-way connected component labeling algorithm; dividing the iris image into several connected regions; and determining the accurate pupil region by comparing the area ratio of each connected region and the distance between the centroid of each connected region and the initially localized pupil center.

[0120] S213: Use an edge detection algorithm to extract the contour of the pupil region and obtain the pupil boundary.

[0121] In this step, the Sobel edge detection algorithm can be used to extract the contour of the pupil region and obtain the pupil boundary.

[0122] It should be noted that other edge detection algorithms, such as the Canny edge detection algorithm, can also be used in this invention, and this invention does not impose any special limitations on them. By processing the binarized image using an edge detection algorithm, the boundary points of the pupil can be obtained.

[0123] This invention improves the accuracy of pupil positioning by coarsely locating the pupil to determine its approximate boundary and then finely locating it.

[0124] S220: Use the gradient information of the iris image to correct the points on the pupil boundary and remove noise to obtain the effective point set on the pupil boundary.

[0125] The aforementioned S210 positions the pupil as a circle, resulting in a circular pupil boundary. However, the circular pupil boundary may have inaccurate positioning. For example, when the eyeball deflects or strabismus occurs, the positioned pupil boundary will be inaccurate.

[0126] Therefore, this step corrects the points on the pupil boundary using the gradient information of the iris image. The gradient information can reflect the degree of change around each pixel in the image. The pixel changes are drastic at the real pupil boundary point, so the pupil boundary can be corrected using the gradient information to obtain the real pupil boundary point.

[0127] The pupil boundary obtained by S210 localization is also susceptible to noise, which consists of false boundary points formed by occlusion on the pupil edge. The true pupil boundary point is located between the pupil and the iris, while false pupil boundary points (i.e., noise) are often located between the pupil and the eyelid or light spot, and need to be removed. Similarly, gradient information from the iris image can be used to remove noise; if necessary, light spot detection technology can also be combined to remove noise.

[0128] S230: Perform elliptic curve fitting based on the effective point set on the pupil boundary, and determine whether the object to be identified has undergone eyeball deflection based on the flatness of the elliptic curve.

[0129] After obtaining the effective point set on the pupil boundary, elliptic curve fitting can be performed. This invention does not limit the method of elliptic curve fitting. For example, the edge points on the pupil boundary can be calculated based on the least squares method or the Hough transform elliptic curve fitting method. Using the effective point set on the pupil boundary, elliptic curve fitting is performed to obtain the edge of the fitted elliptic curve. Then, the nonlinear least squares method is used to find the fitted elliptic curve edge with the smallest Euclidean distance to the actual edge, thus obtaining the fitted elliptic curve.

[0130] After fitting the elliptic curve, the closer the elliptic curve is to a circle, the smaller the pupil tilt and the milder the eye movement. Conversely, the flatter the elliptic curve, the greater the pupil tilt and the more severe the eye movement. Therefore, the degree of eye movement can be represented by parameters such as the flatness of the elliptic curve, and compared with a set threshold to determine whether the subject has experienced eye movement.

[0131] After the elliptic curve fitting is completed, the parameters of the final elliptic curve include the length of the major axis a_finial, the length of the minor axis b_finial, and the two foci F1 and F2 on the major axis and the distance c between them.

[0132] Then, based on the parameters of the elliptic curve obtained from the final fitting, it can be determined whether the object to be identified has undergone eye movement:

[0133] 1. Calculate the ratio of the distance between the two foci on the major axis to the length of the major axis to obtain the first ratio; or, calculate the ratio of the length of the major axis to the length of the minor axis to obtain the second ratio.

[0134] 2. When the first ratio is greater than the set first threshold or when the second ratio is greater than the set second threshold, eye deflection is determined, and the degree of eye deflection is determined based on the first ratio or the second ratio.

[0135] Taking the two foci F1 and F2 on the major axis of an elliptic curve as an example, the closer F1 and F2 are (i.e., the smaller the distance between F1 and F2), the closer the shape of the elliptic curve is to a circle, and the less tilted the pupil. Conversely, the farther apart F1 and F2 are, the greater the tilt of the pupil and the more severe the eyeball deflection. Therefore, the degree of eyeball deflection can be determined by calculating the ratio of the distance between F1 and F2 to the entire major axis. Similarly, the ratio of the major axis length to the minor axis length can also determine the degree of eyeball deflection.

[0136] When the ratio of the distance between F1 and F2 to the entire long axis is greater than a first threshold, and the ratio of the long axis to the short axis is greater than a second threshold, eye deviation is determined. At this point, the system can issue a prompt to remind the user to adjust the angle and re-acquire the iris image. The first and second thresholds can be values ​​set by those skilled in the art based on experience.

[0137] This invention first performs pupil localization on the acquired iris image to determine the pupil boundary. Then, it uses the gradient information of the iris image to correct the points on the pupil boundary and remove noise, obtaining a set of effective points on the pupil boundary. Next, it performs elliptic curve fitting based on the set of effective points, and judges whether the object to be identified has undergone eye deflection based on the flatness of the elliptic curve, thereby improving the accuracy of elliptic curve fitting and ultimately improving the accuracy of eye deflection judgment.

[0138] In step S220, this invention utilizes gradient information from the iris image to correct points on the pupil boundary and remove noise, obtaining a set of effective points on the pupil boundary, thus improving the accuracy of effective points on the pupil boundary. Specific implementation methods include:

[0139] S221: Perform spot detection on the iris image and remove the spot using biquadratic interpolation.

[0140] When acquiring iris images, factors such as supplemental lighting can easily cause light spots to appear within the pupil, such as... Figure 4 As shown, the light spot has a relatively large gradient at the pupil boundary, which has a significant impact on the pupil boundary fitting. Therefore, it is necessary to remove the light spot noise.

[0141] One specific implementation of this step to remove light spots is as follows: the iris image is binarized using a set binarization threshold, and dilation is performed to obtain the light spot region. Then, biquadratic interpolation is used to fill the light spot region on the iris image to remove the light spots.

[0142] For example, using a grayscale value of 250 as the binarization threshold, the image is binarized. Pixels with a grayscale value greater than 250 are assigned a value of 1, and pixels with a grayscale value less than or equal to 250 are assigned a value of 0. The resulting image shows a light spot at a position where the pixel is 1, which can roughly determine the location and size of the light spot. The binarization threshold of 250 is merely an example and is not intended to limit the invention.

[0143] Dilation is performed on the binarized image to locate the spot region.

[0144] Dilation is similar to "neighborhood expansion," which expands the highlighted or white areas in an image, resulting in a larger highlighted area than the original image. This step uses dilation to remove holes within light spots or jagged edges on the light spots.

[0145] The pixels within the light spot on the iris image are subjected to biquadratic interpolation with their adjacent pixels. The values ​​of the pixels surrounding the light spot are then used to fill the area within the light spot, thus removing the light spot. The result after light spot removal is as follows: Figure 5 As shown.

[0146] S222: Take the annular region image with center (y,x) and radius range [r1,r2] on the iris image after removing the light spots, and expand it into a rectangular region image by taking values ​​at certain angle intervals.

[0147] Where (y,x) represents the pupil center obtained by locating the pupil on the iris image, r1 = a1r, r2 = a2r, r is the pupil radius obtained by locating the pupil on the iris image, and a1 and a2 are set coefficients, 0 <a1<1<a2。

[0148] This step unfolds the image based on the pupil center (y, x) and pupil radius r obtained from the aforementioned localization. Specifically:

[0149] The pupil in the iris image is a ring-shaped region with a radius variation of [r1, r2] centered at (y, x). The [-π, π] ring-shaped region images are expanded into rectangular region images with rows (r2-r1+1) and columns P respectively using coordinate transformation.

[0150] When expanding, the center (y,x) is taken as the origin, and the value is taken once at a certain angle interval. For example, when the value is taken every 2°, the column is 180 rows, when the value is taken every 3°, the column is 120 rows, when the value is taken every 4°, the column is 90 rows, and so on. The angle intervals should not be too dense or too sparse. Preferably, the value is taken once at 3°, and the expanded area is a rectangular region image with 120 columns.

[0151] With (y,x) as the center, the traversal range of r is [r1,r2], where r1 = a1r and r2 = a2r. In one example, a1 = 0.5 and a2 = 1.5. It should be noted that a1 = 0.5 and a2 = 1.5 are merely illustrative examples and are not intended to limit the invention.

[0152] Taking a 3° interval as an example, with the center (y,x) as the origin, the annular region between radii r1 = 0.5r and r2 = 1.5r is expanded into a rectangular region every 3° interval, transforming it into... Figure 6 The rectangular region image shown has 120 columns and r2-r1+1 rows.

[0153] S223: Perform smoothing filtering on the rectangular region image and perform interlaced subtraction, keeping the values ​​greater than 0 after subtraction and setting the values ​​less than or equal to 0 to 0, to obtain the gradient image;

[0154] In this step, a 3x3 or 5x5 smoothing filter can be applied to the expanded rectangular region image. Then, alternating row subtraction is performed, that is, the (n+1)th row is subtracted from the (n-1)th row. Values ​​greater than 0 are retained, while values ​​less than or equal to 0 are set to 0, resulting in the gradient image, as shown below. Figure 7 As shown.

[0155] S224: Obtain the maximum pixel value and coordinates of each row of the gradient image, replace the maximum pixel value of each row with the maximum pixel value of the row and the average value of several pixels near the maximum pixel value in the row, and take the row with the maximum pixel value after replacement not less than the set threshold as the valid row.

[0156] This step, based on the gradient image, uses rows as a reference to obtain the maximum value of each row of pixels and its coordinates. To prevent the influence of noise, the maximum value of each row can be replaced by the maximum value of the row itself and the average of several pixels near the maximum value in that row.

[0157] For example, if the m-th pixel in a row is the maximum pixel value, then the average pixel value of the five points (m-2, m-1, m, m+1, m+2) can be used to replace the maximum value at position m.

[0158] Then set a threshold T. If the maximum pixel value in a row is not less than T, the row is considered a valid row; otherwise, it is considered an invalid row.

[0159] S225: Transform the coordinates of the maximum pixel value of the valid row back onto the annular region image of the iris image to obtain the set of valid points on the pupil boundary.

[0160] Assuming there are n valid rows, we obtain the coordinates of the maximum pixel value in all valid rows: (m1,1), (m2,2), (m3,3), ... . Then, these coordinates are transformed back to the original iris image. Figure 6 , Figure 7 By corresponding the angles one-to-one with those in the original iris image and transforming the coordinates back to the original iris image based on the angle values, the pupil coordinates on the annular region image are obtained, which is the effective set of points on the pupil boundary.

[0161] Assumption Figure 7 There are 20 rows. For each row, the maximum pixel value and its coordinates are obtained. Then, the average of 5 nearby pixels is taken from the coordinates of the maximum pixel value and used to replace the original maximum pixel value. Simultaneously, assuming a threshold T of 20, if the maximum pixel value in a row is less than 20, that row is considered invalid. Taking 3 invalid rows as an example, there are 17 valid rows. The actual number of coordinates of the maximum pixel value obtained is 17, and the set of valid points on the pupil boundary after coordinate transformation is also 17.

[0162] After obtaining the effective point set, elliptic curve fitting is performed in S230 based on the effective point set. A specific implementation of elliptic curve fitting may include:

[0163] S231: Obtain the points (y_max, xx2) and (y_min, xx1) with the maximum and minimum ordinates on the effective point set on the pupil boundary, and the points (yy2, x_max) and (yy1, x_min) with the maximum and minimum abscissas.

[0164] Assume the coordinates of the effective point set on the pupil boundary in the iris image are (y1, x1), (y2, x2), (y3, x3)...(yn, xn), as follows Figure 8 As shown. Take the maximum and minimum values ​​of y1, y2, y3...yn and denote them as (y_max, xx2) and (y_min, xx1). Take the maximum and minimum values ​​of x1, x2, x3...xn and denote them as (yy2, x_max) and (yy1, x_min).

[0165] S232: Calculate the initial center point (y_point, x_point), the initial major axis a, and the initial minor axis b.

[0166] Where y_point=(y_min+y_max) / 2, x_point=(x_min+x_max) / 2, a=max(a_point,b_point), b=min(a_point,b_point), a_point is the Euclidean distance between (y_min,xx1) and (y_max,xx2), and b_point is the Euclidean distance between (yy1,x_min) and (yy2,x_max);

[0167] S233: Set the traversal range of the center point, major axis, and minor axis based on the initial center point, initial major axis, and initial minor axis;

[0168] In one example, the range to be traversed is:

[0169] The traversal range in the y-direction is [y_point-5, y_point+5], the traversal range in the x-direction is [x_point-5, x_point+5], the traversal range of the major axis is [a-10, a+10], and the traversal range of the minor axis is [b-10, b+10]. It should be noted that the values ​​of 5 and 10 in this paragraph are merely illustrative and are not intended to limit the scope of this invention.

[0170] S234: For each value within the traversal range of the center point, major axis, and minor axis, perform elliptic curve fitting, and find the elliptic curve that matches the set of valid points on the pupil boundary the most, as the fitted elliptic curve.

[0171] In this step, the system iterates through the range described above, with (y_point, x_point) as the center point and a = max(a_point, b_point) as the major axis and b = min(a_point, b_point) as the minor axis. For each value within the range, an elliptic curve is fitted using the standard elliptic curve equation formula:

[0172]

[0173] Until the elliptic curve that best matches the set of valid points on the pupil boundary is found, such as... Figure 9 As shown, the parameters of the elliptic curve can be obtained.

[0174] It should be noted that coordinate translation and transformation are required in elliptic curve fitting to transform the coordinate system of the original iris image to the coordinate system of the center of the elliptic curve. As the center point changes, the coordinates of all valid points on the pupil boundary must also change to ensure that the coordinates of the center point are always 0.

[0175] The process of transforming the coordinate system of the original iris image to the coordinate system of the center of the elliptic curve is as follows:

[0176] like Figure 10 As shown, given a point P(y,x) in the coordinate system XOY of the original iris image, and the coordinate system X of the center of the elliptic curve... θ O'Y θ The elliptic curve center coordinate system has its origin at (y_point, x_point), and the Y-axis makes an angle θ (θ>0) with the initial major axis. The elliptic curve center coordinate system X... θ O'Y θ After rotating counterclockwise by θ, a new coordinate system X'O'Y' is obtained. The coordinates y' and x' of P(y,x) in coordinate system X'O'Y' can then be calculated as follows:

[0177] y'=y·cos(θ)-x·sin(θ)-y_point;

[0178] x'=x·cos(θ)+y·sin(θ)-x_point.

[0179] After fitting, the parameters of the final elliptic curve can include the length of the major axis a_finial, the length of the minor axis b_finial, and the two foci F1 and F2 on the major axis and the distance c between them.

[0180] As an improvement to this embodiment of the invention, in the aforementioned S300, determining the facial pose of the object to be identified based on the facial image includes:

[0181] The face image is input into a convolutional neural network to obtain the face pose of the object to be identified.

[0182] If necessary, the face image can be preprocessed. This preprocessing includes: performing face detection on the face image to obtain the coordinates (x, y) of the top-left corner of the face, as well as the width and height of the face. Based on the face detection results, the face image is cropped, specifically by taking (x, y) as the starting point and moving left and down by width and height pixels to obtain the cropped image. The differences between the cropped images are then normalized to a size of 40*40 and used as the input to a CNN (Convolutional Neural Network).

[0183] The convolutional neural network includes an image preprocessing part and a first branch and a second branch located in parallel after the image preprocessing part.

[0184] The specific processing steps of a convolutional neural network are as follows:

[0185] S310: The face image undergoes a series of convolution operations, activation operations, absolute value operations, and pooling operations in the image preprocessing section to obtain the preprocessed feature map.

[0186] For example, a 3-channel face image with a width and height of 40 pixels is used as the input image for the image preprocessing part of a CNN. First, the two pixels around the face image are padded with 0. Then, a 5x5 convolution kernel with a stride of 1 is used to perform a convolution operation, resulting in a 16-channel feature map of size 40x40. Then, the Tanh activation function is used for activation. The Tanh function has the following form:

[0187]

[0188] The absolute value of the activated neuron is used as the input to the next layer. Then, a non-overlapping max pooling operation is performed on the neurons after the absolute value is taken, where the pooling kernel size is 2*2 and the stride is 2. After the above series of operations, a 16-channel feature map of size 20*20 is obtained.

[0189] The above process yields 16-channel features of size 20*20. Figure 4One pixel in the circle is filled with 0, and then a 3*3 convolution kernel is used to perform a convolution operation with a stride of 1 to obtain a 48-channel feature map of size 20*20. Then, Tanh activation is performed and the absolute value operation is performed. Then, a 3*3 max pooling operation is performed with a stride of 1 to obtain a 48-channel feature map of size 10*10.

[0190] The 48-channel feature map of size 10*10 obtained by the above process is convolved with a 3*3 kernel with a stride of 1 to obtain a 64-channel feature map of size 8*8. Then, Tanh activation is performed and the absolute value is taken. Next, 3*3 max pooling is performed with a stride of 2 to obtain a 64-channel feature map of size 3*3, which is the preprocessed feature map.

[0191] The image preprocessing section described above processes a 40*40 color face image to obtain a 3*3 64-channel preprocessed feature map.

[0192] S320: The preprocessed feature map undergoes a series of convolution operations, activation operations, absolute value operations, and inner product operations in the first branch, and the coordinates of the facial feature points are calculated by combining the width and height of the face image.

[0193] For example, a 64-channel preprocessed feature map of size 3*3 is convolved with a 2*2 kernel with a stride of 1 to obtain a 64-channel feature map of size 3*3. Then, Tanh activation and absolute value operation are performed. The result after absolute value is used to perform inner product operation to obtain a feature vector of dimension 256. The obtained feature vector is activated with Tanh and its absolute value is taken.

[0194] To obtain the localization results of 83 feature points, the feature vectors after absolute value union need to be mapped to a 166-dimensional vector using an inner product operation to obtain a 166-dimensional feature vector.

[0195] This 166-dimensional feature vector represents the compensation values ​​for the horizontal and vertical coordinates of the 83 facial feature points. The coordinates corresponding to the face image are:

[0196]

[0197]

[0198] Where width and height are the width and height of the face image, respectively. i ,y i The coordinates of the 83 facial feature points are used. The first branch uses L2 loss for parameter training.

[0199] S330: The first set of three-dimensional facial pose angles is calculated based on the coordinates of facial feature points.

[0200] The 83 facial feature points in this step can be used to calculate 3 facial poses (pitch2, yaw2, roll2) using the PNP method, which are denoted as the first group of 3D facial pose angles.

[0201] S340: The preprocessed feature map is processed through a series of convolution operations, BN operations, activation operations, and inner product operations in the second branch to obtain the second set of three-dimensional face pose angles.

[0202] For example, a 64-channel preprocessed feature map of size 3*3 is convolved using a 2*2 kernel with a stride of 1. Then, BN is used for feature normalization and ReLU is used for feature activation. Next, an inner product is used to obtain a 128-dimensional feature vector. Finally, an inner product is used on the 128-dimensional feature vector to obtain a 3-dimensional feature vector (pitch1, yaw1, roll1) as the pose output, denoted as the second set of three-dimensional face pose angles.

[0203] The three-dimensional pose angles of a face (pitch, yaw, roll) are the three true yaw angles of the face. These three yaw angles are as follows: Figure 11 As shown.

[0204] S350: Obtain the facial pose of the object to be identified based on the first set of three-dimensional facial pose angles and the second set of three-dimensional facial pose angles.

[0205] To obtain a more stable face pose, if any angle in the second set of three-dimensional face pose angles is greater than 20 degrees, the final face pose is considered to be the second set of three-dimensional face pose angles (pitch1, yaw1, roll1). Otherwise, the final face pose is considered to be the average of the second set of three-dimensional face pose angles (pitch1, yaw1, roll1) and the first set of three-dimensional face pose angles (pitch2, yaw2, roll2).

[0206] Example 2:

[0207] This invention provides an identity recognition device, such as... Figure 12 As shown, the device includes:

[0208] Image acquisition module 100 is used to acquire the iris image of the object to be identified.

[0209] The eye deflection judgment module 200 is used to fit the pupil boundary to an ellipse on the iris image and determine whether the object to be identified has deflected its eye based on the fitted elliptical curve.

[0210] The iris image acquisition module 300 is used to acquire the face image of the object to be identified if it is determined that the object to be identified has deflected its eyeballs, determine the face posture of the object to be identified based on the face image, and acquire the iris image of the object to be identified when its eyes are looking straight ahead based on the face posture of the object to be identified; if it is determined that the object to be identified has not deflected its eyeballs, the iris image is determined to be the iris image of the object to be identified when its eyes are looking straight ahead.

[0211] The identity recognition module 400 is used to identify the object based on the iris image of the object when the human eye is looking straight ahead.

[0212] This invention first acquires the iris image of the object to be identified, then uses an elliptical curve fitted to the pupil boundary of the iris image to determine whether the object has undergone eye movement. When the object has undergone eye movement, it indicates that the face posture has been deflected. Next, the face posture of the object to be identified is determined by the acquired face image, and the direction and angle of the face deflection are obtained. Based on the face posture of the object to be identified, the iris image of the object when its eyes are looking straight ahead is acquired, and the identity recognition is performed using the iris image of the object when its eyes are looking straight ahead.

[0213] This invention utilizes the correlation between eye movement and facial pose deviation to quickly and accurately detect user pose deviation. First, eye movement is assessed based on iris images to determine if facial pose deviation exists. Then, the direction and angle of facial pose deviation are determined based on the facial image and corrected to obtain an iris image of the subject looking straight ahead, meeting practical application requirements. This invention improves the success rate and efficiency of iris image acquisition, thereby enhancing the accuracy of iris recognition. Furthermore, it enables the detection and correction of large-angle facial pose deviations, meeting the diverse facial pose requirements of practical applications.

[0214] The aforementioned eye deflection detection module includes:

[0215] The pupil boundary extraction unit is used to locate the pupil on the iris image and obtain the pupil boundary.

[0216] The effective point set determination unit is used to correct the points on the pupil boundary and remove noise using the gradient information of the iris image, so as to obtain the effective point set on the pupil boundary.

[0217] The fitting and turning judgment unit is used to fit an elliptic curve based on the effective point set on the pupil boundary, and to determine whether the object to be identified has undergone eyeball deflection based on the flatness of the elliptic curve.

[0218] The effective point set determination unit includes:

[0219] The spot removal unit is used to detect spots in the iris image and remove spots using biquadratic interpolation.

[0220] The rectangular region image unfolding unit is used to unfold a ring-shaped region image with center (y,x) and radius range [r1,r2] on the iris image after removing light spots into a rectangular region image by taking values ​​at certain intervals of angle.

[0221] Where (y,x) represents the pupil center obtained by locating the pupil on the iris image, r1 = a1r, r2 = a2r, r is the pupil radius obtained by locating the pupil on the iris image, and a1 and a2 are set coefficients, 0 <a1<1<a2。

[0222] The gradient image acquisition unit is used to perform smoothing filtering on the rectangular region image and perform interlaced subtraction, retaining the values ​​greater than 0 after subtraction and setting the values ​​less than or equal to 0 to 0, thus obtaining the gradient image.

[0223] The valid row determination unit is used to obtain the maximum pixel value and coordinates of each row of the gradient image, replace the maximum pixel value of each row with the maximum pixel value of the row and the average value of several pixels near the maximum pixel value in the row, and take the row with the maximum pixel value after replacement not less than a set threshold as a valid row.

[0224] The coordinate transformation unit is used to transform the coordinates of the maximum pixel value of the effective row back to the annular region image of the iris image, so as to obtain the effective point set on the pupil boundary.

[0225] In the aforementioned fitting and conversion judgment unit, elliptic curve fitting is performed based on the effective point set on the pupil boundary, including:

[0226] The data acquisition unit is used to acquire the points (y_max, xx2) and (y_min, xx1) with the maximum and minimum vertical coordinates, and the points (yy2, x_max) and (yy1, x_min) with the maximum and minimum horizontal coordinates, respectively, in the effective point set on the pupil boundary.

[0227] The initial parameter calculation unit is used to calculate the initial center point (y_point, x_point), the initial major axis a, and the initial minor axis b.

[0228] Where y_point=(y_min+y_max) / 2, x_point=(x_min+x_max) / 2, a=max(a_point,b_point), b=min(a_point,b_point), a_point is the Euclidean distance between (y_min,xx1) and (y_max,xx2), and b_point is the Euclidean distance between (yy1,x_min) and (yy2,x_max).

[0229] The traversal range determination unit is used to set the traversal range of the center point, major axis, and minor axis based on the initial center point, initial major axis, and initial minor axis.

[0230] The traversal unit is used to perform elliptic curve fitting for each value within the traversal range of the center point, major axis, and minor axis, and find the elliptic curve that matches the set of valid points on the pupil boundary the most.

[0231] In the iris image acquisition module, the iris image of the subject being identified when the eyes are looking straight ahead is acquired based on the subject's facial pose, including:

[0232] The correction unit is used to correct the iris image according to the facial posture to obtain the iris image of the object to be identified when the human eye is looking straight ahead.

[0233] Alternatively, an adjustment unit may be used to prompt the object to be identified to adjust its posture based on the facial pose and then return to the step of acquiring the iris image of the object to be identified, until the iris image of the object to be identified when its eyes are looking straight ahead is acquired.

[0234] The device of the present invention can be used for multimodal identity recognition of face and iris.

[0235] Furthermore, in the iris image acquisition module, the facial pose of the object to be identified is determined based on the facial image, including:

[0236] The face image is input into a convolutional neural network to obtain the face pose of the object to be identified.

[0237] The convolutional neural network includes an image preprocessing part and a first branch and a second branch located in parallel after the image preprocessing part.

[0238] The face image undergoes a series of convolution operations, activation operations, absolute value operations, and pooling operations in the image preprocessing section to obtain a preprocessed feature map.

[0239] The preprocessed feature map undergoes a series of convolution operations, activation operations, absolute value operations, and inner product operations in the first branch, and the coordinates of the facial feature points are calculated by combining the width and height of the face image.

[0240] The first set of three-dimensional facial pose angles is calculated based on the coordinates of facial feature points.

[0241] The preprocessed feature map is processed through a series of convolution operations, BN operations, activation operations, and inner product operations in the second branch to obtain the second set of three-dimensional face pose angles.

[0242] The facial pose of the object to be identified is obtained based on the first set of three-dimensional facial pose angles and the second set of three-dimensional facial pose angles.

[0243] The device provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment 1. For the sake of brevity, any parts not mentioned in this device embodiment can be referred to the corresponding content in the aforementioned method embodiment 1. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the aforementioned device and unit can all be referred to the corresponding processes in the aforementioned method embodiment 1, and will not be repeated here.

[0244] Example 3:

[0245] The method described in Embodiment 1 of this invention can implement business logic through a computer program and record it on a storage medium. This storage medium can be read and executed by a computer, achieving the effects of the scheme described in Embodiment 1 of this specification. Therefore, this invention also provides a computer-readable storage medium for identity verification, including a memory for storing processor-executable instructions. When executed by a processor, the instructions implement the steps of the identity verification method of Embodiment 1.

[0246] This invention utilizes the correlation between eye movement and facial pose deviation to quickly and accurately detect user pose deviation. First, eye movement is assessed based on iris images to determine if facial pose deviation exists. Then, the direction and angle of facial pose deviation are determined based on the facial image and corrected to obtain an iris image of the subject looking straight ahead, meeting practical application requirements. This invention improves the success rate and efficiency of iris image acquisition, thereby enhancing the accuracy of iris recognition. Furthermore, it enables the detection and correction of large-angle facial pose deviations, meeting the diverse facial pose requirements of practical applications.

[0247] The storage medium may include a physical device for storing information, typically digitizing the information and then storing it using electrical, magnetic, or optical methods. The storage medium may include: devices that store information using electrical energy, such as various types of memory, like RAM and ROM; devices that store information using magnetic energy, such as hard disks, floppy disks, magnetic tapes, magnetic core memory, bubble memory, and USB flash drives; and devices that store information using optical methods, such as CDs or DVDs. Of course, there are other readable storage media, such as quantum memories and graphene memories.

[0248] The storage medium described above may also include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects of this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, which will not be repeated here.

[0249] Example 4:

[0250] The present invention also provides a device for identity recognition. This device may be a standalone computer, or it may include an actual operating device that uses one or more of the methods or embodiments described in this specification. The identity recognition device may include at least one processor and a memory storing computer-executable instructions. When the processor executes the instructions, it implements the steps of any one or more of the identity recognition methods described in Embodiment 1.

[0251] This invention utilizes the correlation between eye movement and facial pose deviation to quickly and accurately detect user pose deviation. First, eye movement is assessed based on iris images to determine if facial pose deviation exists. Then, the direction and angle of facial pose deviation are determined based on the facial image and corrected to obtain an iris image of the subject looking straight ahead, meeting practical application requirements. This invention improves the success rate and efficiency of iris image acquisition, thereby enhancing the accuracy of iris recognition. Furthermore, it enables the detection and correction of large-angle facial pose deviations, meeting the diverse facial pose requirements of practical applications.

[0252] The device described above may include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects of this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, which will not be repeated here.

[0253] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An identity recognition method, characterized in that, The method includes: Acquire the iris image of the object to be identified; The pupil boundary is fitted with an ellipse curve on the iris image, and the eyeball deflection of the object to be identified is determined based on the fitted ellipse curve. If it is determined that the object to be identified has deflected its eyeballs, then the face image of the object to be identified is acquired, the face pose of the object to be identified is determined based on the face image, and the iris image of the object to be identified when its eyes are looking straight ahead is acquired based on the face pose of the object to be identified; if it is determined that the object to be identified has not deflected its eyeballs, then the iris image is determined to be the iris image of the object to be identified when its eyes are looking straight ahead. The identity of the object to be identified is determined based on the iris image of the object when the human eye is looking straight ahead; The step of fitting an ellipse curve to the pupil boundary on the iris image and determining whether the object to be identified has undergone eyeball deflection based on the fitted ellipse curve includes: The pupil is located on the iris image to obtain the pupil boundary; Gradient information from the iris image is used to correct points on the pupil boundary and remove noise, resulting in a valid set of points on the pupil boundary. Elliptic curve fitting is performed based on the effective point set on the pupil boundary, and the flatness of the elliptic curve is used to determine whether the object to be identified has undergone eyeball deflection. The step of obtaining the iris image of the object to be identified when its eyes are looking straight ahead based on the facial pose of the object to be identified includes: The iris image is corrected based on the facial posture to obtain the iris image of the object to be identified when the human eye is looking straight ahead; Alternatively, based on the facial pose prompting the object to be identified to adjust its posture, the process returns to the step of obtaining the iris image of the object to be identified, until the iris image of the object to be identified when its eyes are looking straight ahead is obtained.

2. The identity recognition method according to claim 1, characterized in that, The step of using gradient information from the iris image to correct points on the pupil boundary and remove noise to obtain a valid set of points on the pupil boundary includes: Spot detection is performed on the iris image, and spot removal is achieved using biquadratic interpolation; The annular region image with center (y, x) and radius range [r1, r2] on the iris image after removing light spots is expanded into a rectangular region image by taking values ​​at certain intervals of angle. Where (y, x) represents the pupil center obtained by locating the pupil on the iris image, r1 = a1r, r2 = a2r, r is the pupil radius obtained by locating the pupil on the iris image, and a1 and a2 are set coefficients, 0 <a1<1<a2; Smoothing filtering is performed on the rectangular region image, and alternating subtraction is performed. Values ​​greater than 0 after subtraction are retained, and values ​​less than or equal to 0 are set to 0 to obtain the gradient image. Obtain the maximum pixel value and its coordinates for each row of the gradient image. Replace the maximum pixel value of each row with the maximum pixel value of that row and the average value of several pixels near the maximum pixel value in that row. Then, the rows whose maximum pixel value after replacement is not less than a set threshold are considered as valid rows. The coordinates of the maximum pixel values ​​in the valid rows are transformed back onto the annular region of the iris image to obtain the set of valid points on the pupil boundary.

3. The identity recognition method according to claim 1, characterized in that, The step of fitting an elliptic curve based on the effective set of points on the pupil boundary includes: Obtain the points (y_max, xx2) and (y_min, xx1) with the maximum and minimum ordinates on the valid point set on the pupil boundary, and the points (yy2, x_max) and (yy1, x_min) with the maximum and minimum abscissas. Calculate the initial center point (y_point, x_point), the initial major axis a, and the initial minor axis b; Where y_point = (y_min+y_max) / 2, x_point = (x_min+x_max) / 2, a = max(a_point, b_point), b = min(a_point, b_point), a_point is the Euclidean distance between (y_min, xx1) and (y_max, xx2), and b_point is the Euclidean distance between (yy1, x_min) and (yy2, x_max); Define the traversal range of the center point, major axis, and minor axis based on the initial center point, initial major axis, and initial minor axis; For each value within the traversal range of the center point, major axis, and minor axis, perform elliptic curve fitting, and find the elliptic curve that matches the set of valid points on the pupil boundary most often as the fitted elliptic curve.

4. The identity recognition method according to claim 1, characterized in that, The method is used for multimodal identity recognition based on face and iris.

5. The identity recognition method according to claim 1, characterized in that, Determining the facial pose of the object to be identified based on the facial image includes: The face image is input into a convolutional neural network to obtain the face pose of the object to be identified; The convolutional neural network includes an image preprocessing part and a first branch and a second branch located in parallel after the image preprocessing part; The face image undergoes a series of convolution operations, activation operations, absolute value operations, and pooling operations in the image preprocessing part to obtain a preprocessed feature map. The preprocessed feature map undergoes a series of convolution operations, activation operations, absolute value operations, and inner product operations in the first branch, and the coordinates of the facial feature points are calculated by combining the width and height of the face image. The first set of three-dimensional facial pose angles is calculated based on the coordinates of the facial feature points. The preprocessed feature map is processed through a series of convolution operations, BN operations, activation operations, and inner product operations in the second branch to obtain the second set of three-dimensional face pose angles. The facial pose of the object to be identified is obtained based on the first set of three-dimensional facial pose angles and the second set of three-dimensional facial pose angles.

6. An identity recognition device, characterized in that, The device includes: The image acquisition module is used to acquire the iris image of the object to be identified; An eye deflection detection module is used to fit an ellipse curve to the pupil boundary on the iris image, and to determine whether the object to be identified has undergone eye deflection based on the fitted ellipse curve. The iris image acquisition module is used to acquire a face image of the subject if it is determined that the subject has deflected its eyeballs, determine the face pose of the subject based on the face image, and acquire an iris image of the subject when its eyes are looking straight ahead based on the face pose of the subject; if it is determined that the subject has not deflected its eyeballs, the module determines that the iris image is an iris image of the subject when its eyes are looking straight ahead. The identity recognition module is used to identify the object to be identified based on the iris image of the object when the human eye is looking straight ahead; The eye deflection detection module includes: The pupil boundary extraction unit is used to locate the pupil on the iris image and obtain the pupil boundary; The effective point set determination unit is used to correct the points on the pupil boundary and remove noise using the gradient information of the iris image to obtain the effective point set on the pupil boundary. The fitting and turning judgment unit is used to fit an elliptic curve based on the effective point set on the pupil boundary, and to determine whether the object to be identified has undergone eyeball deflection based on the flatness of the elliptic curve. In the iris image acquisition module, acquiring the iris image of the subject when its eyes are looking straight ahead, based on the subject's facial posture, includes: The correction unit is used to correct the iris image according to the facial posture to obtain the iris image of the object to be identified when the eyes are looking straight ahead; Alternatively, the adjustment unit is used to prompt the object to be identified to adjust its posture according to the facial posture and then return to the step of obtaining the iris image of the object to be identified until the iris image of the object to be identified when its eyes are looking straight ahead is obtained.

7. A computer-readable storage medium for identity verification, characterized in that, It includes a memory for storing processor-executable instructions, which, when executed by the processor, implement the steps of the identification method according to any one of claims 1-5.

8. A device for identity recognition, characterized in that, It includes at least one processor and a memory storing computer-executable instructions, wherein the processor, when executing the instructions, implements the steps of the identity recognition method according to any one of claims 1-5.

Images