A method for liveness detection based on binocular left and right image alignment and correction

Abstract

The present invention provides a method for liveness detection by aligning and correcting binocular left and right images, including: S1, acquiring a checkerboard pattern; S2, performing single-target localization; S3, correcting and removing distortion from the left and right images; S4, aligning the left and right images through perspective transformation; and S5, cropping the face frame. Single-target localization is performed for a single camera by acquiring ten left and right checkerboard patterns. The intrinsic parameter matrix and distortion matrix are calculated. The left and right cameras use the same set of intrinsic parameters and distortion matrices to remove distortion from the original images. Perspective transformation is then used to achieve row-wise alignment of the left and right images.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for performing living body detection by aligning and correcting binocular left and right images. Background Art

[0002] With the advancement of technology, existing facial liveness recognition systems inevitably experience differences in imaging parameters and assembly accuracy, as most binocular cameras on the market use different modules and inherent differences between the two cameras. Therefore, binocular left and right image alignment is necessary to correct for distortion in camera images.

[0003] The existing binocular alignment method is to perform single-target positioning on each camera separately, obtain the intrinsic parameters and distortion matrix of each camera, obtain the reprojection matrix and the rotation and translation matrix through binocular positioning, and then eliminate the distortion to align the left and right images in the row direction to achieve binocular left and right image alignment.

[0004] However, conventional binocular alignment methods currently rely on binocular calibration. This method requires capturing over 20 checkerboard images from different angles, performing single-target calibration for each camera, and finally correcting them for distortion. This calibration process is cumbersome and requires strict control over the captured checkerboard images, which can affect the calibration results.

[0005] In addition, the commonly used terms in the prior art are as follows:

[0006] Calibration Target: A flat plate with a fixed-pitch pattern array is a calibration target. By capturing the target with a camera and applying a calibration algorithm to it, we can derive the camera's geometric model, resulting in highly accurate measurement and reconstruction results.

[0007] Single target calibration: Obtain the homography matrix corresponding to each calibration plate, then jointly optimize all calibration plate data to obtain the camera intrinsic parameter matrix, and then obtain the extrinsic parameters corresponding to each calibration plate.

[0008] Intrinsic parameter matrix: It transforms 3D camera coordinates into 2D homogeneous image coordinates. It is determined by the camera itself and will not change due to the external environment. It is related to the focal length f of the camera.

[0009] Distortion parameters: caused by irregular lens and incomplete parallelism between the lens and the photosensitive film during installation. Image correction: The process of removing distortion from the original image through the internal parameter matrix and distortion parameters to make the image regular.

[0010] Perspective transformation: Perspective transformation refers to the transformation that uses the condition that the perspective center, image point and target point are collinear, and according to the law of perspective rotation, the receiving surface (perspective surface) is rotated around the trace line (perspective axis) by a certain angle, destroying the original projection light beam, while still keeping the projected geometric figure on the receiving surface unchanged.

[0011] OPENCV: A cross-platform computer vision and machine learning software library released under the Apache 2.0 license (open source), it runs on Linux, Windows, Android, and Mac OS. It is lightweight and efficient—consisting of a series of C functions and a small number of C++ classes. It also provides interfaces for languages ​​like Python, Ruby, and MATLAB, implementing many common algorithms for image processing and computer vision. Summary of the Invention

[0012] In order to solve the problems in the above-mentioned prior art, the purpose of this application is to complete the single-target positioning of a camera by collecting about ten checkerboard images, calculate the intrinsic parameter matrix and distortion matrix, and use a set of intrinsic parameters and distortion matrices for the left and right cameras to remove the distortion information on the original image, and realize the alignment of the left and right images in the row direction through perspective transformation.

[0013] Specifically, the present invention provides a method for performing liveness detection by aligning and correcting binocular left and right images, the method comprising the following steps:

[0014] S1. Collect chessboard image:

[0015] The camera captures four or more chessboard images at different angles to complete single-target positioning. The chessboard must be completely imaged in the image. Testing shows that the camera exposure threshold is set to 0.3 when the distance from the lens is within 1 meter. A value below 0.3 (i.e., less than or equal to 0.3) is considered normal exposure.

[0016] S2. Complete single target setting:

[0017] Extract corner point information from the checkerboard image taken by the left camera or the right camera and determine whether the corner point position meets the calibration requirements. The calibration requirement is that all corner points in the checkerboard must be detected without missing any, otherwise the corner point information cannot be extracted normally. In order to obtain accurate corner point information, sub-pixel corner point precision is performed, that is, sub-pixel corner point detection using find4QuadCornerSubpix() in opencv is used to determine the corner point information. After determining the corner point information, one of the cameras is calibrated to obtain the camera's intrinsic parameter matrix and distortion parameters. The calibration results are evaluated. The camera's intrinsic parameter matrix and distortion parameters are used to reproject the three-dimensional points in space to obtain new projection coordinates. The deviation between the projection coordinates and the sub-pixel corner point coordinates is calculated. The smaller the deviation, the better the calibration result.

[0018] S3. Correct and remove distortion of the left and right images:

[0019] During the distortion correction process, the left and right cameras use the same set of camera parameters;

[0020] S4. Perspective transformation aligns the left and right images:

[0021] Perspective change optimizes the height difference in the row direction, aligning the left and right images in the row direction and reducing the error caused by solving parallax.

[0022] S5. Perform face frame cropping:

[0023] By setting the weight of the left and right width difference based on the center coordinates of the face detection frame, the left and right face cropped images are used to replace the original image information, reducing the image resolution and making the calculated facial parallax effect more obvious.

[0024] In step S1, ten checkerboard images are collected, and the calibration result will be more accurate. When the exposure threshold is above 0.3, the exposure is serious, and the imaged checkerboard image cannot be overexposed or too dark. When selecting the checkerboard image, as long as the human eye can normally distinguish the black and white squares of the checkerboard, otherwise the checkerboard corners cannot be found and calibration cannot be performed.

[0025] In step S2, the intrinsic parameter matrix of the camera is composed of the following parameters: f / dx, f / dy, u0, v0; wherein, f is the focal length, in millimeters; dx is the width of the pixel in the x-direction, in millimeters; 1 / dx is the number of pixels within 1 millimeter in the x-direction; f / dx is the length of the focal length in the x-axis direction described in pixels; f / dy is the length of the focal length in the y-axis direction described in pixels; u0 and v0 represent the horizontal and vertical pixel differences between the center pixel coordinates of the image and the pixel coordinates of the image origin; the distortion parameters are: k1, k2, k3 are radial distortion, and p1 and p2 are tangential distortion coefficients; radial distortion occurs in the process of converting the camera coordinate system to the physical coordinate system, and tangential distortion occurs because the lens is not completely parallel to the image.

[0026] The implementation of the single target targeting in step S2 includes:

[0027] 1) Extract corner points using findChessboardCorners();

[0028] int cvFindChessboardCorners(const void*image,

[0029] CvSize pattern_size,

[0030] CvPoint2D32f*corners,

[0031] int*corner_count=NULL,

[0032] int flags=CV_CALIB_CB_ADAPTIVE_THRESH);

[0033] This function attempts to determine whether the input image is a checkerboard pattern and determine the locations of the corners; if all corners are detected and they are arranged in a certain order, the function returns a non-zero value, otherwise if the function cannot find all corners or record them, the function returns 0;

[0034] 2) Sub-pixel precision, use find4QuadCornerSubpix(); Set CV_EXPORTS boolfind4QuadCornerSubpix(InputArray img,

[0035] InputOutputArray corners,

[0036] Size region_size);

[0037] In order to improve the calibration accuracy, it is necessary to further extract sub-pixel information based on the initially extracted corner information to reduce the camera calibration deviation. This function is specifically used to obtain the precise position of the inner corner points on the chessboard image.

[0038] 6) Visualize the inner corners using the drawChessboardCorners() function; set CV_EXPORTS_W voiddrawChessboardCorners(InputOutputArray image,

[0039] Size patternSize,

[0040] InputArray corners,

[0041] bool patternWasFound);

[0042] This function draws all the corner points found onto the provided image;

[0043] 7) Camera calibration, using the calibrateCamera() function; set

[0044] CV_EXPORTS_W double calibrateCamera(

[0045] InputArrayOfArrays objectPoints,

[0046] InputArrayOfArrays imagePoints,

[0047] Size imageSize,

[0048] CV_OUT InputOutputArray cameraMatrix,

[0049] CV_OUT InputOutputArray distCoeffs,

[0050] OutputArrayOfArrays rvecs,

[0051] OutputArrayOfArrays tvecs,

[0052] int flags=0,

[0053] TermCriteria criteria=TermCriteria(

[0054] TermCriteria::COUNT+TermCriteria::EPS,30,DBL_EPSILON));

[0055] This function performs calibration and calculates the camera's internal parameters and distortion coefficients. Before using this function for calibration, the spatial coordinate system position coordinates of each inner corner point on the chessboard need to be initialized. The calibration result is the camera's internal parameter matrix cameraMatrix and the camera's five distortion coefficients distCoeffs. In addition, each image will generate its own translation vector and rotation vector; 8) Calibration effect evaluation, using the projectPoints() function; Set

[0056] void cv::projectPoints(InputArray_opoints,

[0057] InputArray_rvec,

[0058] InputArray_tvec,

[0059] InputArray_cameraMatrix,

[0060] InputArray_distCoeffs,

[0061] OutputArray_ipoints,

[0062] OutputArray_jacobian,

[0063] double aspectRatio)

[0064] This function performs reverse projection on the three-dimensional coordinate points in space.

[0065] The step S3 further comprises:

[0066] Use a set of intrinsic parameters and distortion parameters for the left and right cameras to dedistort and correct the image. Use the initUndistortRectifyMap function on the collected left and right images, that is, use the obtained intrinsic parameters and distortion parameters of the camera to correct the image distortion and obtain the output mapping on the x-axis and y-axis. The dedistortion information is obtained using the initUndistortRectifyMap() function; Set cv::initUndistortRectifyMap(InputArray_cameraMatrix,

[0067] InputArray_distCoeffs,

[0068] InputArray_matR,

[0069] InputArray_newCameraMatrix,

[0070] Size size,

[0071] int m1type,

[0072] OutputArray_map1,

[0073] OutputArray_map2)

[0074] This function corrects the distortion of the image;

[0075] Then, the original image is remapped geometrically according to the specified mapping form by the remap function; the remap geometric transformation uses the Remap() function to remap the corrected image.

[0076] void remap(InputArray src,

[0077] OutputArray dst,

[0078] InputArray map1,

[0079] InputArray map2,

[0080] int interpolation,

[0081] int borderMode=BORDER_CONSTANT,

[0082] const Scalar&borderValue=Scalar())

[0083] This function performs a remapping geometric transformation on the rectified image.

[0084] The step S4 further comprises:

[0085] Perform corner detection on a chessboard that has been corrected for alignment, determine the positions of the four outermost corner points in the left and right images, and use the left image as a reference to remap the positions of the four corner points of the right image to the positions of the corner points of the left image through perspective transformation, so that the positions of the left and right images are aligned.

[0086] The step S5 further comprises:

[0087] A face detection frame is obtained through face detection, and the detection frame is expanded to make the cropped face information more complete. The corrected left image is cropped with the expanded detection frame to obtain a left cropped image. The width of the two images in the column direction is calculated through the center coordinates of the face detection frame. Finally, the calculated width difference is added to the left end of the right cropping frame, and the right image is re-cropped with the cropping frame. The final right cropped image will be aligned with the left cropped image. The aligned left and right images are applied to stereo vision for liveness detection.

[0088] The process of implementing the face frame cropping method includes:

[0089] First, calculate the width and height of the face detection and the coordinates of its center point. Then, expand the detection frame up, down, left, and right based on the center point. The expanded size is half of the maximum side of the detection frame width and height. The expansion to the left of the center point is 1.5 times the expansion in other directions. The final aspect ratio after expansion is 5:4. This ratio is used because there is parallax between the left and right images, which will provide a more complete liveness detection effect. The width difference between the left and right images is calculated based on the expanded size. The purpose of this is to make the parallax information of the left and right images more obvious. Finally, the left image is cropped with the cropping frame, and the cropping frame is horizontally translated by the width difference units and cropped on the right image.

[0090] The process further comprises:

[0091] 5) Face detection frame, calculate height H and width W;

[0092] The coordinates of the upper left corner of the face detection frame are: (x1, x2);

[0093] The coordinates of the lower right corner of the face detection frame are: (x1, x2);

[0094] W = x2 - x1;

[0095] H = y2 - y1;

[0096] 6) Calculate the center coordinates of the detection box (center_x, center_y);

[0097] The center coordinates of the detection box are:

[0098] center_x = (x1 + x2) / 2;

[0099] center_y = (y1 + y2) / 2;

[0100] 7) Calculate the cropping length radius, Max(H,W) / 2;

[0101] Radius = max(H,W) / 2;

[0102] 8) Expand the cropping frame, the detection frame ratio is 5:4,

[0103] Upper left point x: center_x-1.5*radius

[0104] Upper left point y: center_y-radius

[0105] Lower right point x: center_x+radius

[0106] Lower right point y: center_y+radius;

[0107] 4A) Crop the left image with the expanded cropping frame; at the same time,

[0108] 4B) Calculate the width difference, shift the cropping frame horizontally, and crop the right image;

[0109] Width difference diff:4.0*radius / 64.0;

[0110] Upper left point x: center_x-1.5*radius+diff;

[0111] Upper left point y: center_y-radius;

[0112] Lower right point x: center_x+radius+diff;

[0113] Lower right point y: center_y + radius.

[0114] Therefore, the advantages of this application are:

[0115] 1. Collect a small number of checkerboard images and perform single-target calibration. The left and right cameras use the same calibration parameters. This reduces the complexity of the calibration process while ensuring calibration accuracy.

[0116] 2. Through perspective transformation, the positions of the left and right images are made more precise. This step aims to optimize the image after single-target positioning correction, prevent excessive differences in the row direction between the left and right images, minimize the resulting height difference, and ensure the disparity map obtained by binocular stereo matching.

[0117] 3. By cropping the left and right images, the efficiency of the algorithm is greatly improved in the process of solving the disparity. At the same time, the width difference between the left and right cropped images is calculated, which improves the accuracy of the disparity map. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0119] Figure 1It is a flowchart of the method of the present invention.

[0120] Figure 2 1 is a schematic diagram of a single-target positioning flow chart of a method embodiment of the present invention.

[0121] Figure 3 (1) is the corrected left image of the calibration correction effect diagram in the method of the present invention.

[0122] Figure 3 (2) is the corrected right image of the calibration correction effect diagram in the method of the present invention.

[0123] FIG4(1) is the corrected left image of the perspective transformation result image in the method of the present invention.

[0124] FIG4(2) is the right image of the perspective transformation result image after correction in the method of the present invention.

[0125] Figure 5 This is a schematic diagram of the face frame cropping process in the method of the present invention.

[0126] Figure 6 1 is a flow chart of an embodiment of the method of the present invention. DETAILED DESCRIPTION

[0127] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail with reference to the accompanying drawings.

[0128] The method of the present invention relates to a method for liveness detection by aligning and correcting binocular left and right images, such as Figure 6 As shown, the method includes:

[0129] S1 collects a checkerboard image;

[0130] S2 completes single target positioning;

[0131] S3 corrects the left and right images to remove distortion;

[0132] S4 perspective transformation aligns the left and right images;

[0133] S5 performs face frame cropping.

[0134] like Figure 1 The flowchart of the method is shown in FIG. , and the specific implementation steps of the method are described as follows:

[0135] Step S1. Collecting chessboard images

[0136] The camera captures checkerboard images from different angles. Four or more images are required. In theory, four images are sufficient for calibration. More images yield more accurate results. In practice, capturing around ten images yields more accurate results, and 10 images are sufficient for even more precise calibration. The checkerboard must be fully imaged and positioned appropriately, neither too close nor too far from the lens. The camera also places certain requirements on the camera: exposure should be appropriate, ensuring the imaged checkerboard pattern is neither overexposed nor too dark. Brightness analysis is not performed on overly dark images. When selecting checkerboard images, it is sufficient that the human eye can distinguish between the black and white squares. Calibration is therefore impossible if the checkerboard corners cannot be identified. Testing was performed with a distance of less than 1 meter from the lens, and the camera exposure threshold was set to 0.3. A value below 0.3 (i.e., less than or equal to 0.3) is considered normal exposure.

[0137] Among them, step S1 includes: before collecting the chessboard, it is necessary to determine the rationality of the camera exposure and to determine that the distance between the chessboard and the camera is appropriate, which plays a decisive role in the obtained parameter matrix.

[0138] Step S2. Complete single target positioning

[0139] Extract corner information from the checkerboard image taken by the left or right camera and determine whether the corner position meets the calibration requirements. In order to obtain more accurate corner information, perform sub-pixel corner refinement, that is, use the find4QuadCornerSubpix() sub-pixel corner detection in opencv to determine the corner information. Without accurate corner information, the camera calibration result will not be accurate. Finally, the left and right cameras share a calibration parameter information, which will affect the alignment of the left and right pictures. After determining the corner information, calibrate one of the cameras to obtain the camera's intrinsic parameter matrix and distortion parameters, evaluate the calibration results, use the camera parameters to reproject the three-dimensional points in space, obtain new projection coordinates, and calculate the deviation between the projection coordinates and the sub-pixel corner coordinates. The smaller the deviation, the better the calibration result.

[0140] Step S3: Correct and remove distortion of the left and right images

[0141] A set of internal parameters and distortion parameters are used for the left and right cameras to dedistort and correct the image. The initUndistortRectifyMap function is used to obtain the output mapping on the x-axis and y-axis for the collected left and right images. The remap function is used to remap the original image according to the specified mapping form. Figures 3(1)(2) show the effect diagram of this method.

[0142] Among them, step S3 includes: in the process of correcting and removing distortion, the left and right cameras use the same set of camera parameters, which simplifies the tedious calibration steps and also improves the efficiency of the algorithm.

[0143] like Figure 2 As shown in FIG, it is a flowchart of the single-target positioning method, which covers the contents of steps S1, S2, and S3:

[0144] Collect chessboard images;

[0145] Extract corner points using findChessboardCorners;

[0146] Sub-pixel precision find4QuadCornerSubpix;

[0147] Inner corner visualization drawChessboardCorners;

[0148] Camera calibration calibrateCamera;

[0149] Calibration effect evaluation projectPoints;

[0150] Dedistortion information initUndistortRectifyMap;

[0151] Remap geometric transformation Remap.

[0152] Further including:

[0153] The implementation of the single target targeting in step S2 includes:

[0154] 1) Extract corner points using findChessboardCorners();

[0155] int cvFindChessboardCorners(const void*image,

[0156] CvSize pattern_size,

[0157] CvPoint2D32f*corners,

[0158] int*corner_count=NULL,

[0159] int flags=CV_CALIB_CB_ADAPTIVE_THRESH);

[0160] This function attempts to determine whether the input image is a checkerboard pattern and determine the locations of the corners; if all corners are detected and they are arranged in a certain order, the function returns a non-zero value, otherwise if the function cannot find all corners or record them, the function returns 0;

[0161] 2) Sub-pixel precision, use find4QuadCornerSubpix(); Set CV_EXPORTS boolfind4QuadCornerSubpix(InputArray img,

[0162] InputOutputArray corners,

[0163] Size region_size);

[0164] In order to improve the calibration accuracy, it is necessary to further extract sub-pixel information based on the initially extracted corner information to reduce the camera calibration deviation. This function is specifically used to obtain the precise position of the inner corner points on the chessboard image.

[0165] 3) Visualize the inner corners using the drawChessboardCorners() function; set CV_EXPORTS_W voiddrawChessboardCorners(InputOutputArray image,

[0166] Size patternSize,

[0167] InputArray corners,

[0168] bool patternWasFound);

[0169] This function draws all the corner points found onto the provided image;

[0170] 4) Camera calibration, using the calibrateCamera() function; set

[0171] CV_EXPORTS_W double calibrateCamera(

[0172] InputArrayOfArrays objectPoints,

[0173] InputArrayOfArrays imagePoints,

[0174] Size imageSize,

[0175] CV_OUT InputOutputArray cameraMatrix,

[0176] CV_OUT InputOutputArray distCoeffs,

[0177] OutputArrayOfArrays rvecs,

[0178] OutputArrayOfArrays tvecs,

[0179] int flags=0,

[0180] TermCriteria criteria=TermCriteria(

[0181] TermCriteria::COUNT+TermCriteria::EPS,30,DBL_EPSILON));

[0182] This function performs calibration and calculates the camera's internal parameters and distortion coefficients. Before using this function for calibration, the spatial coordinate system position coordinates of each inner corner point on the chessboard need to be initialized. The calibration result is the camera's internal parameter matrix cameraMatrix and the camera's five distortion coefficients distCoeffs. In addition, each image will generate its own translation vector and rotation vector; 5) Calibration effect evaluation, using the projectPoints() function; Set

[0183] void cv::projectPoints(InputArray_opoints,

[0184] InputArray_rvec,

[0185] InputArray_tvec,

[0186] InputArray_cameraMatrix,

[0187] InputArray_distCoeffs,

[0188] OutputArray_ipoints,

[0189] OutputArray_jacobian,

[0190] double aspectRatio)

[0191] This function performs reverse projection on the three-dimensional coordinate points in space.

[0192] The step S3 further comprises:

[0193] Use a set of intrinsic parameters and distortion parameters for the left and right cameras to dedistort and correct the image. Use the initUndistortRectifyMap function on the collected left and right images, that is, use the obtained intrinsic parameters and distortion parameters of the camera to correct the image distortion and obtain the output mapping on the x-axis and y-axis. The dedistortion information is obtained using the initUndistortRectifyMap() function; Set cv::initUndistortRectifyMap(InputArray_cameraMatrix,

[0194] InputArray_distCoeffs,

[0195] InputArray_matR,

[0196] InputArray_newCameraMatrix,

[0197] Size size,

[0198] int m1type,

[0199] OutputArray_map1,

[0200] OutputArray_map2)

[0201] This function can correct the distortion of the image;

[0202] Then, the original image is remapped geometrically according to the specified mapping form by the remap function; the remap geometric transformation uses the Remap() function to remap the corrected image.

[0203] void remap(InputArray src,

[0204] OutputArray dst,

[0205] InputArray map1,

[0206] InputArray map2,

[0207] int interpolation,

[0208] int borderMode=BORDER_CONSTANT,

[0209] const Scalar&borderValue=Scalar())

[0210] This function performs a remapping geometric transformation on the rectified image.

[0211] Step S4. Perspective transformation to align the left and right images

[0212] Perform corner detection on a chessboard that has been corrected for alignment, determine the positions of the four outermost corner points in the left and right images, and use the left image as a reference to remap the four corner points of the right image to the positions of the left image corner points through perspective transformation, so that the positions of the left and right images are aligned. Attached Figures 4(1)(2) are the effect diagrams achieved by this method. The "one" here refers to any one of a stack of left and right images, because here we only need to determine the positions of the four corner points of a left image and the four corner points of a right image, and use the left image as a reference to map the four corner points of the right image to the four points of the left image.

[0213] Step S4 includes: optimizing the height difference in the row direction by perspective change, so that the left and right images are aligned in the row direction, and reducing the error caused by solving the parallax.

[0214] Step S5. Crop the face frame

[0215] A face detection frame is obtained through face detection, and the detection frame is expanded to make the cropped face information more complete. The corrected left image is cropped with the expanded detection frame to obtain a left cropped image. The width of the two images in the column direction is calculated through the center coordinates of the face detection frame. Finally, the calculated width difference is added to the left end of the right cropping frame, and the right image is re-cropped with the cropping frame. The final right cropped image will be aligned with the left cropped image. The aligned left and right images are applied to stereo vision for liveness detection.

[0216] Among them, step S5 includes: setting the weight of the left and right width difference through the coordinates of the center point of the face detection frame, replacing the information of the original image with the left and right face cropped images, reducing the resolution of the image, making the calculated facial parallax effect more obvious, and improving the efficiency of the algorithm.

[0217] First, calculate the width and height of the face detection and the coordinates of its center point. Then, expand the detection frame up, down, left, and right based on the center point. The expanded size is half of the maximum side of the detection frame width and height. The expansion to the left of the center point is 1.5 times the expansion in other directions. The final aspect ratio after expansion is 5:4. This ratio is used because there is parallax between the left and right images, which will provide a more complete liveness detection effect. The width difference between the left and right images is calculated based on the expanded size. The purpose of this is to make the parallax information of the left and right images more obvious. Finally, the left image is cropped with the cropping frame, and the cropping frame is horizontally translated by the width difference units and cropped on the right image.

[0218] like Figure 5 FIG. 1 is a flowchart of implementing the face frame cropping process, wherein the process further includes: 1) detecting the face frame and calculating the height H and width W;

[0219] The coordinates of the upper left corner of the face detection frame are: (x1, x2);

[0220] The coordinates of the lower right corner of the face detection frame are: (x1, x2);

[0221] W = x2 - x1;

[0222] H = y2 - y1;

[0223] 2) Calculate the center coordinates of the detection box (center_x, center_y);

[0224] The center coordinates of the detection box are:

[0225] center_x = (x1 + x2) / 2;

[0226] center_y = (y1 + y2) / 2;

[0227] 3) Calculate the cropping length radius, Max(H,W) / 2;

[0228] Radius = max(H,W) / 2;

[0229] 4) Expand the cropping frame, the detection frame ratio is 5:4,

[0230] Upper left point x: center_x-1.5*radius

[0231] Upper left point y: center_y-radius

[0232] Lower right point x: center_x+radius

[0233] Lower right point y: center_y+radius;

[0234] 4A) Crop the left image with the expanded cropping frame; at the same time,

[0235] 4B) Calculate the width difference, shift the cropping frame horizontally, and crop the right image;

[0236] Width difference diff:4.0*radius / 64.0;

[0237] Upper left point x: center_x-1.5*radius+diff;

[0238] Upper left point y: center_y-radius;

[0239] Lower right point x: center_x+radius+diff;

[0240] Lower right point y: center_y + radius.

[0241] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for liveness detection by aligning and correcting binocular left and right images, characterized in that: The method comprises the following steps: S1. Collect chessboard image: The camera captures four or more chessboard images at different angles to complete single-target positioning. The chessboard must be completely imaged in the image. Testing shows that the camera exposure threshold is set to 0.3 when the distance from the lens is within 1 meter. A value below 0.3 (i.e., less than or equal to 0.3) is considered normal exposure. S2. Complete single target setting: Extract corner point information from the checkerboard image taken by the left camera or the right camera and determine whether the corner point position meets the calibration requirements. The calibration requirement is that all corner points in the checkerboard must be detected without missing any, otherwise the corner point information cannot be extracted normally. In order to obtain accurate corner point information, sub-pixel corner point precision is performed, that is, sub-pixel corner point detection using find4QuadCornerSubpix() in opencv is used to determine the corner point information. After determining the corner point information, one of the cameras is calibrated to obtain the camera's intrinsic parameter matrix and distortion parameters. The calibration results are evaluated. The camera's intrinsic parameter matrix and distortion parameters are used to reproject the three-dimensional points in space to obtain new projection coordinates. The deviation between the projection coordinates and the sub-pixel corner point coordinates is calculated. The smaller the deviation, the better the calibration result. S3. Correct and remove distortion of the left and right images: During the distortion correction process, the left and right cameras use the same set of camera parameters; S4. Perspective transformation aligns the left and right images: Perspective change optimizes the height difference in the row direction, aligning the left and right images in the row direction and reducing the error caused by solving parallax. S5. Perform face frame cropping: By setting the weight of the left and right width difference based on the center coordinates of the face detection frame, the left and right face cropped images are used to replace the original image information, reducing the image resolution and making the calculated facial parallax effect more obvious.

2. The method for liveness detection by aligning and correcting binocular left and right images according to claim 1, characterized in that: In step S1, ten checkerboard images are collected, and the calibration result will be more accurate. When the exposure threshold is above 0.3, the exposure is serious, and the imaged checkerboard image cannot be overexposed or too dark. When selecting the checkerboard image, as long as the human eye can normally distinguish the black and white squares of the checkerboard, otherwise the checkerboard corners cannot be found and calibration cannot be performed.

3. The method for liveness detection by aligning and correcting binocular left and right images according to claim 1, characterized in that: In step S2, the intrinsic parameter matrix of the camera is composed of the following parameters: f / dx, f / dy, u0, v0; wherein, f is the focal length, in millimeters; dx is the width of the pixel in the x-direction, in millimeters; 1 / dx is the number of pixels within 1 millimeter in the x-direction; f / dx is the length of the focal length in the x-axis direction described in pixels; f / dy is the length of the focal length in the y-axis direction described in pixels; u0 and v0 represent the horizontal and vertical pixel differences between the center pixel coordinates of the image and the pixel coordinates of the image origin; the distortion parameters are: k1, k2, k3 are radial distortion, and p1 and p2 are tangential distortion coefficients; radial distortion occurs in the process of converting the camera coordinate system to the physical coordinate system, and tangential distortion occurs because the lens is not completely parallel to the image.

4. The method for liveness detection by aligning and correcting binocular left and right images according to claim 1, characterized in that: The implementation of the single target targeting in step S2 includes: 1) Extract corner points using findChessboardCorners(); int cvFindChessboardCorners(const void*image, CvSize pattern_size, CvPoint2D32f*corners, int*corner_count=NULL, int flags=CV_CALIB_CB_ADAPTIVE_THRESH); This function attempts to determine whether the input image is a checkerboard pattern and determine the locations of the corners; if all corners are detected and they are arranged in a certain order, the function returns a non-zero value, otherwise if the function cannot find all corners or record them, the function returns 0; 2) Sub-pixel precision, using find4QuadCornerSubpix(); set CV_EXPORTS bool find4QuadCornerSubpix(InputArray img, InputOutputArray corners, Size region_size); In order to improve the calibration accuracy, it is necessary to further extract sub-pixel information based on the initially extracted corner information to reduce the camera calibration deviation. This function is specifically used to obtain the precise position of the inner corner points on the chessboard image. 3) Visualize the inner corners using the drawChessboardCorners() function; CV_EXPORTS_W void drawChessboardCorners(InputOutputArray image, Size patternSize, InputArray corners, bool patternWasFound); This function draws all the corner points found onto the provided image; 4) Camera calibration, using the calibrateCamera() function; set CV_EXPORTS_W double cal ibrateCamera( InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, Size imageSize, CV_OUT InputOutputArray cameraMatrix, CV_OUT InputOutputArray distCoeffs, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=0, TermCriteria criteria=TermCriteria( TermCriteria::COUNT+TermCriteria::EPS,30,DBL_EPSILON)); This function performs calibration and calculates the camera's intrinsic parameters and distortion coefficients. Before using this function for calibration, the spatial coordinate system coordinates of each inner corner point on the chessboard need to be initialized. The calibration result is the camera's intrinsic parameter matrix cameraMatrix and the camera's five distortion coefficients distCoeffs. In addition, each image will generate its own translation vector and rotation vector. 5) Calibration effect evaluation, using projectPoints() function; void cv::projectPoints(InputArray_opoints, InputArray_rvec, InputArray_tvec, InputArray_cameraMatrix, InputArray_distCoeffs, OutputArray_ipoints, OutputArray_jacobian, double aspectRatio) This function performs reverse projection on the three-dimensional coordinate points in space.

5. The method for liveness detection by aligning and correcting binocular left and right images according to claim 4, characterized in that: The step S3 further comprises: Use a set of intrinsic parameters and distortion parameters for the left and right cameras to dedistort and correct the image. Use the initUndistortRectifyMap function on the collected left and right images. That is, use the obtained intrinsic parameters and distortion parameters of the camera to correct the image distortion and obtain the output mapping on the x-axis and y-axis. The dedistortion information uses the initUndistortRectifyMap() function. cv::initUndistortRectifyMap(InputArray_cameraMatrix, InputArray_distCoeffs, InputArray_matR, InputArray_newCameraMatrix, Size size, int m1type, OutputArray_map1, OutputArray_map2) This function corrects the distortion of the image; Then, the original image is remapped geometrically according to the specified mapping form by the remap function; the remap geometric transformation uses the Remap() function to remap the corrected image. void remap(InputArray src, OutputArray dst, InputArray map1, InputArray map2, int interpolation, int borderMode=BORDER_CONSTANT, const Scalar&borderValue=Scalar()) This function performs a remapping geometric transformation on the rectified image.

6. The method for liveness detection by aligning and correcting binocular left and right images according to claim 1, characterized in that: The step S4 further comprises: Perform corner detection on a chessboard that has been corrected for alignment, determine the positions of the four outermost corner points in the left and right images, and use the left image as a reference to remap the positions of the four corner points of the right image to the positions of the corner points of the left image through perspective transformation, so that the positions of the left and right images are aligned.

7. The method for liveness detection by aligning and correcting binocular left and right images according to claim 1, characterized in that: The step S5 further comprises: A face detection frame is obtained through face detection, and the detection frame is expanded to make the cropped face information more complete. The corrected left image is cropped with the expanded detection frame to obtain a left cropped image. The width of the two images in the column direction is calculated through the center coordinates of the face detection frame. Finally, the calculated width difference is added to the left end of the right cropping frame, and the right image is re-cropped with the cropping frame. The final right cropped image will be aligned with the left cropped image. The aligned left and right images are applied to stereo vision for liveness detection.

8. The method for liveness detection by aligning and correcting binocular left and right images according to claim 7, characterized in that: The process of implementing the face frame cropping method includes: First, calculate the width and height of the face detection and the coordinates of its center point. Then, expand the detection frame up, down, left, and right based on the center point. The expanded size is half of the maximum side of the detection frame width and height. The expansion to the left of the center point is 1.5 times the expansion in other directions. The final aspect ratio after expansion is 5:

4. This ratio is used because there is parallax between the left and right images, which will provide a more complete liveness detection effect. The width difference between the left and right images is calculated based on the expanded size. The purpose of this is to make the parallax information of the left and right images more obvious. Finally, the left image is cropped with the cropping frame, and the cropping frame is horizontally translated by the width difference units and cropped on the right image.

9. The method for liveness detection by aligning and correcting binocular left and right images according to claim 8, characterized in that: The process further comprises: 1) Face detection frame, calculate height H and width W; The coordinates of the upper left corner of the face detection frame are: (x1, x2); The coordinates of the lower right corner of the face detection frame are: (x1, x2); W = x2 - x1; H = y2 - y1; 2) Calculate the center coordinates of the detection box (center_x, center_y); The center coordinates of the detection box are: center_x = (x1 + x2) / 2; center_y = (y1 + y2) / 2; 3) Calculate the cropping length radius, Max(H,W) / 2; Radius = max(H,W) / 2; 4) Expand the cropping frame, the detection frame ratio is 5:4, Upper left point x: center_x-1.5*radius Upper left point y: center_y-radius Lower right point x: center_x+radius Lower right point y: center_y+radius; 4A) Crop the left image with the expanded cropping frame; at the same time, 4B) Calculate the width difference, shift the cropping frame horizontally, and crop the right image; Width difference diff:4.0*radius / 64.0; Upper left point x: center_x-1.5*radius+diff; Upper left point y: center_y-radius; Lower right point x: center_x+radius+diff; Lower right point y: center_y + radius.

Images