Image acquisition processing device and method for acquiring face iris image

Abstract

The invention provides an image acquisition and processing device and method for face iris image acquisition, and the device comprises an image acquisition and processing unit which employs an image period parallel synchronous logic time sequence working mode to acquire images, and the image period parallel synchronous logic time sequence working mode is in a current image acquisition period TAn + 2, and synchronously executing the image calculation period TCn + 1 and the image processing period TPn in parallel. According to the face iris image acquisition and processing device and the face iris image acquisition and processing method, synchronous parallel processing of image acquisition, image calculation and image processing is realized, and the processing efficiency is improved.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application number 202211555663.0, application date December 6, 2022, and invention name “Device and method for acquiring face and iris images”, and all contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the technical field of optical image acquisition, and in particular to a device and method for acquiring face and iris images. Background Art

[0003] Faced with the increasing demand for long-distance and wide-angle iris image acquisition, the current mainstream solution is a 2-axis rotating iris optical imaging system or a high-pixel resolution iris optical imaging system.

[0004] The invention patent application with application number 202011131179 discloses a device and method for acquiring iris images of multiple target groups. The device includes a base, a vertical motor module, a horizontal motor module, a reflector, a stage module, a motion limiter and a control system. The base supports and fixes the overall two-dimensional turntable; the vertical motor module includes a vertical motor and a vertical motion adapter plate; the horizontal motor module includes a horizontal motor, a support plate, a horizontal motion adapter plate and a rolling bearing; the reflector is fixed on the stage module, and the stage module includes a loading plate, a loading support and a fixing part, which is fixed to the horizontal motor module; there are multiple motion limiters installed at different positions; the control system includes a driver, a PLC controller and a PC control card to control the movement of the turntable; the iris camera is installed vertically. The device can effectively increase the iris recognition field of view, automatically adapt to targets of different heights at different directions and distances, and has fast speed and strong robustness.

[0005] Application No. 202110588620 discloses an image information processing method and system for use with a specific object, the system comprising a first module for collecting image or video data; a second module for determining face range data from the image and video data collected by the first module; a third module for extracting feature data from the face range data, comprising the following units: a first unit for determining the detection point position of facial features from the face range data, and extracting feature data from the detection point position; a second unit for determining the detection area position of facial skin features from the face range data, and extracting feature data from the detection area position; a third unit for determining the detection area position of facial iris features from the face range data, and extracting feature data from the detection point position; a fourth module for processing and analyzing the facial features, the facial skin features, and the facial iris features, comprising the following units: a fourth unit for comparing and analyzing the facial features with pre-stored facial features and obtaining results ; A fifth unit is used to compare and analyze the facial skin feature data with the pre-stored facial skin feature data and obtain a result; a sixth unit is used to compare and analyze the facial iris feature data with the pre-stored facial iris feature data and obtain a result; a fifth module is used to integrate the comparison and analysis results of the facial five-feature feature data, the comparison and analysis results of the facial skin feature data, and the comparison and analysis results of the facial iris feature data, and obtain a determination result of face recognition, including the following units: a seventh unit is used to normalize the comparison and analysis results of the facial five-feature feature data, the comparison and analysis results of the facial skin feature data, and the comparison and analysis results of the facial iris feature data, respectively; an eighth unit is used to comprehensively process the normalized results of the comparison and analysis results of the facial five-feature feature data, the normalized results of the comparison and analysis results of the facial skin feature data, and the normalized results of the comparison and analysis results of the facial iris feature data, and obtain a determination result of face recognition; a sixth module is used to make a response process according to the comprehensive determination result of face recognition.

[0006] As in the above two patent applications, in the prior art, in view of the future trend towards high pixel resolution, such as 64M pixel resolution or above. And when the prior art collects videos and performs face and iris processing, face images and iris images are collected separately; when the face state changes, face images and iris images need to be collected again separately, which results in very large image data. On this basis, image data collection and transmission, transmission bandwidth challenges are brought about. At 64M pixel resolution, it is less than 10fps, which obviously cannot meet the frame rate requirements of real-time image processing. The lower the frame rate, the more seriously it affects the image acquisition speed and subsequent recognition performance. In addition, high pixel resolution images are also a huge challenge to the image processing speed itself. The huge amount of image data processing seriously affects the image calculation processing speed performance.

[0007] In addition, the high pixel resolution image is accompanied by such a small pixel scale for the same image plane, which makes it difficult to achieve high-speed and accurate optical imaging of the iris image. The currently known traditional autofocus uses the focus image to search for the maximum focus quality, and iterates 2-3 search cycles through the coarse to fine strategy, which takes at least 1 second. A further serious problem is that the depth of field of the image that meets the iris diameter standard is extremely small, resulting in the inability to guarantee focus accuracy. Therefore, how to accurately capture high-quality iris images at high speed is also a huge challenge.

[0008] In order to solve the problems existing in the prior art: first, the high-pixel resolution iris image has the problem of slow acquisition speed and poor subsequent recognition performance; second, the problem of focusing accuracy when acquiring high-pixel resolution iris images. Therefore, a device and method for acquiring face iris images are urgently needed to solve the above problems. Summary of the invention

[0009] The purpose of the present invention is to solve the problems of slow acquisition speed and poor subsequent recognition performance of high-pixel resolution iris images in the prior art, and to provide a device and method for acquiring facial iris images.

[0010] In order to achieve the above-mentioned purpose of the present invention, the following technical scheme is adopted:

[0011] A device for collecting face and iris images, comprising:

[0012] Face optical imaging unit, iris optical imaging unit, optical imaging association unit, image acquisition and processing unit;

[0013] The image acquisition processing unit is respectively connected to the face optical imaging unit, the iris optical imaging unit, and the optical imaging association unit, and controls the face optical imaging unit, the iris optical imaging unit, and the optical imaging association unit;

[0014] The face optical imaging unit outputs a global area face image;

[0015] The optical imaging association unit is used to establish a synchronous association relationship between the output of the global-local area image between the predetermined face optical imaging unit and the iris optical imaging unit;

[0016] The iris optical imaging unit outputs a corresponding local area iris image according to the synchronous association relationship;

[0017] The imaging field of view of the face optical imaging unit is greater than or equal to the imaging field of view of the iris optical imaging unit.

[0018] Further,

[0019] The optical imaging association unit includes: a human eye detection unit, an optical mapping unit, and a local area configuration unit;

[0020] The face optical imaging unit is used to collect global area face images;

[0021] The eye detection unit is used to detect the center pixel positions of both eyes of the face from the global area face image collected by the face optical imaging unit;

[0022] The optical mapping unit is used to transform the object-image optical geometric mapping relationship corresponding to the iris optical imaging unit in response to the face optical imaging unit according to the center pixel positions of both eyes of the face detected by the eye detection unit;

[0023] The local area configuration unit is used to synchronously and dynamically configure the local area of ​​the iris image with a fixed pixel resolution corresponding to the iris optical imaging unit according to the object-image optical geometric mapping relationship obtained by the optical mapping unit;

[0024] The iris optical imaging unit is used to collect the local area iris image according to the local area of ​​the iris image with a fixed pixel resolution synchronized by the local area configuration unit;

[0025] Further, when the optical mapping unit transforms the object-image optical geometric mapping relationship of the face optical imaging unit in response to the iris optical imaging unit according to the center pixel positions of the two eyes of the face of the eye detection unit, the calculation method of the transformation factor OMTF is:

[0026] OMTF=Fi*PSe / (Fe*PSi)

[0027] =(PXi-cPXi / 2) / (PXe-cPXe / 2)

[0028] =(PYi-cPYi / 2) / (PYe-cPYe / 2)

[0029] Among them, OMTF is a transformation factor with an inherent optical geometric mapping relationship between the object and the image; Fi is the imaging optical focal length of the iris optical imaging unit; PSe is the unit pixel resolution of the face optical imaging unit; Fe is the imaging optical focal length of the face optical imaging unit; PSi is the unit pixel resolution of the iris optical imaging unit; (PXi, PYi) is the XY coordinates of the center pixel position of both eyes of the face on the image side of the iris optical imaging unit; (cPXi, cPYi) is the image side XY pixel resolution of the iris optical imaging unit; (PXe, PYe) is the XY coordinates of the center pixel position of both eyes of the face on the image side of the face optical imaging unit; (cPXe, cPYe) is the image side XY pixel resolution of the face optical imaging unit.

[0030] Furthermore, by limiting the local area of ​​the iris image, the dynamic synchronous update frequency of the execution association state between the optical mapping unit and the local area configuration unit is controlled.

[0031] Further, when the iris optical imaging unit collects the local area iris image according to the local area of ​​the iris image synchronized by the local area configuration unit, the local area iris image is an image collected within an area centered on the coordinates of the pixel position (PXi, PYi) and surrounded by the pixel range (PXroi, PYroi);

[0032] Where: PXroi, PYroi are the XY pixel ranges of the local area of ​​the iris image;

[0033] cPXi / 16<=PXroi<=cPXi / 2, cPYi / 16<=PYroi<=cPYi / 2

[0034] ROI=RECT(left, top, right, bottom)

[0035] =RECT(PXi-PXroi / 2, PYi-PYroi / 2, PXi+PXroi / 2, PYi+PYroi / 2);

[0036] or

[0037] ROI=RECT(left,top,width,height)

[0038] =RECT(PXi-PXroi / 2, PYi-PYroi / 2, PXroi, PYroi).

[0039] Further, when collecting the local area iris image according to the pixel position (PXi, PYi) and the pixel range (PXroi, PYroi), by using the physical simulation local array image output (X_ADD_START, Y_ADD_START, X_ADD_END, Y_ADD_END) corresponding to the imaging pixel array path of the iris optical imaging unit, or the corresponding digital offset local array image output (X_CROP_OFFSET, Y_CROP_OFFSET, X_CROP_WIDTH, Y_CROP_HEIGHT);

[0040] in:

[0041] X_ADD_START=left=PXi-PXroi / 2,

[0042] Y_ADD_START=top=PYi-PYroi / 2,

[0043] X_ADD_END=right=PXi+PXroi / 2,

[0044] Y_ADD_END=bottom=PYi+PYroi / 2;

[0045] or

[0046] X_CROP_OFFSET=left=PXi-PXroi / 2,

[0047] Y_CROP_OFFSET=top=PYi-PYroi / 2,

[0048] X_CROP_WIDTH=width=PXroi,

[0049] Y_CROP_HEIGHT=height=PYroi.

[0050] Furthermore, it also includes an iris image AF automatic focusing unit, and the image acquisition processing unit is connected to the iris image AF automatic focusing unit and controls the iris image AF automatic focusing unit.

[0051] Furthermore, it also includes an iris image AF automatic focusing unit, and the image acquisition processing unit is connected to the iris image AF automatic focusing unit and controls the iris image AF automatic focusing unit;

[0052] The iris image AF automatic focusing unit is used to automatically focus the local area iris image according to the local area iris image corresponding to the iris optical imaging unit to obtain a focused local area iris image.

[0053] Furthermore, the iris image automatic focusing AF unit adopts the imaging lens driving parameter control of the iris optical imaging unit.

[0054] Furthermore, the AF automatic focusing parameter control depends on the imaging lens driving type and is achieved by adjusting the imaging optical image distance or adjusting the imaging optical focal length / diopter focal power.

[0055] Furthermore, the imaging lens driving parameter control of the iris optical imaging unit includes:

[0056] The number of steps 2*K+1 and the step length STEP parameter, K is the number of steps;

[0057] Achieve continuous automatic focusing in one direction within the predetermined focusing range FR = (2*K+1)*STEP;

[0058] in:

[0059] STEP=2*FNO*SOC

[0060] FNO is the aperture parameter of the iris optical imaging unit;

[0061] SOC is the minimum resolution parameter of the physical spot of the iris optical imaging unit.

[0062] Furthermore, the imaging lens driving parameter control method of the iris optical imaging unit includes:

[0063] Step S1, calculating PD, where PD is the phase difference corresponding to the local area of ​​the iris image;

[0064] Step S2, calculating the corresponding focus position relative distance FP according to the phase difference obtained in step S1, the calculation formula is:

[0065] FP = PD * CC;

[0066] Wherein, CC is the conversion coefficient between the phase difference and the corresponding relative distance FP of the focus position (i.e., the defocus distance, the relative distance from the focus position), i.e., the defocus rate, which is realized by fitting the corresponding linear or nonlinear relationship.

[0067] Step S3, according to the focus position relative distance FP obtained in step S2, driving the PDAF type driver to execute the imaging lens driving parameters of the iris optical imaging unit, driving the corresponding focus position relative distance FP, that is, directly driving STEP*K step positions;

[0068] Step S4, iterating step S1-step S3, terminating at PD<=EP, EP=STEP / CC, EP is a predetermined phase difference focusing error, that is, terminating at a single step length, K<=1.

[0069] A method for collecting face and iris images, comprising: a face optical imaging unit, a human eye detection unit, an optical mapping unit, a local area configuration unit, an iris optical imaging unit, and an image collection and processing unit; the image collection and processing unit is respectively connected to the face optical imaging unit, the human eye detection unit, the optical mapping unit, the local area configuration unit, and the iris optical imaging unit, and controls the face optical imaging unit, the human eye detection unit, the optical mapping unit, the local area configuration unit, and the iris optical imaging unit;

[0070] The specific steps are:

[0071] Step 1: The image acquisition processing unit controls the face optical imaging unit to acquire a face image;

[0072] Step 2: The image acquisition processing unit controls the eye detection unit to detect the center pixel positions of both eyes of the face from the face image acquired in step 1;

[0073] Step 3: the image acquisition processing unit controls the optical mapping unit to transform the optical geometric mapping relationship of the face optical imaging unit in response to the object-image optical imaging unit according to the central pixel positions of the eyes of the face detected in step 2;

[0074] Step 4: the image acquisition processing unit synchronously and dynamically configures the local area of ​​the iris image with a fixed pixel resolution corresponding to the iris optical imaging unit according to the object-image optical geometric mapping relationship obtained by the transformation in step 3;

[0075] Step 5: The image acquisition processing unit controls the iris optical imaging unit to acquire the iris image of the local area according to the local area of ​​the iris image obtained by the synchronous and dynamic configuration in step 4.

[0076] Furthermore, it also includes an iris image AF automatic focusing unit, and the image acquisition processing unit is connected to the iris image AF automatic focusing unit and controls the iris image AF automatic focusing unit;

[0077] After step five, step six is ​​also provided;

[0078] Step six: the image acquisition and processing unit controls automatic focusing of the local area iris image according to the local area iris image acquired in step five to obtain a focused local area iris image.

[0079] The present invention has the following improvements over the prior art:

[0080] In the present invention, by matching the optical mapping unit and the local area configuration unit, the optical mapping unit can transform the optical geometric mapping relationship of the face optical imaging unit in response to the object image corresponding to the iris optical imaging unit according to the central pixel position of both eyes of the face; the local area configuration unit then synchronously and dynamically configures the local area of ​​the iris image with a fixed pixel resolution corresponding to the iris optical imaging unit according to the optical geometric mapping relationship of the object image; under the action of the optical mapping unit and the local area configuration unit, the face image collected by the face optical imaging unit and the local area iris image collected by the iris optical imaging unit can be associated and mapped. When the face state corresponding to the face image changes, it is only necessary to collect the face image again. A new local area iris image is directly generated according to the re-collected face image and in combination with the association relationship between the previous face image and the local area iris image. It is only necessary to collect the local area iris image, which greatly reduces the image data volume. Therefore, the present invention can meet the frame rate requirements of real-time image processing under high-pixel resolution image data acquisition and transmission, realize high-speed and accurate transmission, can realize high-speed and accurate processing of high-pixel resolution images, improve image calculation and processing speed performance, and can realize high-speed and accurate automatic focusing to acquire high-quality iris images.

[0081] It should be understood that the foregoing general description and the following detailed description are exemplary illustrations and explanations and should not be used as limitations on the content of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0083] Figure 1 The present invention is a logic principle diagram of the unit components of the device for collecting face and iris images;

[0084] Figure 2 The present invention is a logic principle diagram of the image acquisition processing unit of the device for acquiring face and iris images. DETAILED DESCRIPTION

[0085] By referring to the exemplary embodiments, the objects and functions of the present invention and the methods for achieving these objects and functions will be explained. However, the present invention is not limited to the exemplary embodiments disclosed below; it can be implemented in different forms. The essence of the description is only to help those skilled in the relevant art to comprehensively understand the specific details of the present invention.

[0086] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0087] Example 1

[0088] like Figure 1 As shown, the embodiment provides a device for collecting face and iris images, including:

[0089] Face optical imaging unit, iris optical imaging unit, eye detection unit, optical mapping unit, local area configuration unit, iris image AF automatic focusing unit and image acquisition processing unit.

[0090] The human eye detection unit, the optical mapping unit, and the local area configuration unit constitute an optical imaging association unit.

[0091] The face optical imaging unit outputs a global area face image.

[0092] The optical imaging association unit is used to establish a synchronous association relationship between the output of the global-local area image between the predetermined face optical imaging unit and the iris optical imaging unit.

[0093] The iris optical imaging unit outputs a corresponding local area iris image according to the synchronous association relationship;

[0094] Furthermore,

[0095] The face optical imaging unit is used to realize the corresponding global area face image acquisition.

[0096] The eye detection unit is used to detect the central pixel positions of the eyes of the corresponding face according to the global area face image corresponding to the face optical imaging unit.

[0097] The optical mapping unit is used to realize the transformation of the optical geometric mapping relationship of the face optical imaging unit in response to the object-image optical imaging unit corresponding to the iris optical imaging unit according to the central pixel positions of the two eyes of the face corresponding to the eye detection unit.

[0098] The local area configuration unit is used to realize synchronous dynamic configuration of the local area ROI of the iris image with a fixed pixel resolution corresponding to the iris optical imaging unit according to the optical geometric mapping relationship corresponding to the optical mapping unit.

[0099] The iris optical imaging unit is used to realize iris image acquisition of the iris image local area ROI with a fixed pixel resolution corresponding to the local area configuration unit.

[0100] The iris image AF automatic focusing unit is used to implement AF automatic focusing processing of the iris image local area ROI corresponding to the iris image local area ROI corresponding to the iris optical imaging unit.

[0101] The image acquisition and processing unit is respectively connected to the face optical imaging unit, the eye detection unit, the optical mapping unit, the local area configuration unit, and the iris optical imaging unit, and controls the face optical imaging unit, the eye detection unit, the optical mapping unit, the local area configuration unit, and the iris optical imaging unit to achieve synchronous image acquisition, image processing calculation, and control of each unit.

[0102] The imaging field of view FOV of the face optical imaging unit is greater than or equal to the imaging field of view FOV of the iris optical imaging unit.

[0103] The human eye detection unit is used to realize the detection of the center pixel positions of both eyes of the human face in the global area human face image of the human face optical imaging unit.

[0104] The present invention uses the currently known convolutional neural network (CNN) cascade model based on deep learning to reliably and accurately realize face area detection and face eye center pixel position detection output coordinates (PXe, PYe).

[0105] The optical mapping unit is used to implement the object-image optical geometric mapping relationship transformation OMT of the face optical imaging unit in response to the iris optical imaging unit according to the center pixel positions of both eyes of the face of the face / eye detection unit, (PXe, PYe): -> (PXi, PYi).

[0106] PXi-cPXi / 2

[0107] =Fi / (Zi-Fi)*(Xi) / PSi

[0108] =Fi / (Zi-Fi)*(Xe-Xoffset) / PSi

[0109] =Fi / (Zi-Fi)*[(Ze-Fe) / Fe*(PXe-cPXe / 2)*PSe-Xoffset)] / PSi.

[0110] PYi-cPYi / 2

[0111] =Fi / (Zi-Fi)*(Yi) / PSi

[0112] =Fi / (Zi-Fi)*(Ye-Yoffset) / PSi

[0113] =Fi / (Zi-Fi)*[(Ze-Fe) / Fe*(PYe-cPYe / 2)*PSe-Yoffset)] / Psi.

[0114] In particular, when the condition Ze>>Zoffset, Fi, Fe is met,

[0115] (PXi-cPXi / 2)+Fi / (Zi-Fi)*Xoffset / PSi=(PXe-cPXe / 2)*Fi*PSe / (Fe*PSi)

[0116] (PYi-cPYi / 2)+Fi / (Zi-Fi)*Yoffset / PSi=(PYe-cPYe / 2)*Fi*PSe / (Fe*PSi)

[0117] In particular, when the conditions Ze>>Xoffset, Yoffset, Zoffset, Fi, Fe are met, the important basic characteristics of the present invention are the inherent object-image optical geometric mapping relationship transformation factor OMTF:

[0118] (PXi-cPXi / 2) / (PXe-cPXe / 2)=

[0119] (PYi-cPYi / 2) / (PYe-cPYe / 2)=

[0120] Fi*PSe / (Fe*PSi)=OMTF

[0121] Where: (PXe, PYe) is the XY coordinates of the center pixel position of both eyes of the face on the image side of the face optical imaging unit, pixel.

[0122] (cPXe, cPYe) is the image-side XY pixel resolution of the face optical imaging unit, pixel.

[0123] PSe is the unit pixel resolution of the face optical imaging unit, um / pixel.

[0124] Fe is the imaging optical focal length of the face optical imaging unit, mm.

[0125] Ze is the imaging optical object distance of the face optical imaging unit, mm.

[0126] (Xe, Ye) is the XY position coordinate of the face optical imaging unit in object space, mm.

[0127] (PXi, PYi) is the XY coordinates of the center pixel position of both eyes of the face on the image side of the iris optical imaging unit, pixel.

[0128] (cPXi, cPYi) is the image-side XY pixel resolution of the iris optical imaging unit, pixel.

[0129] PSi is the unit pixel resolution of the iris optical imaging unit, um / pixel.

[0130] Fi is the imaging optical focal length of the iris optical imaging unit, mm.

[0131] Zi is the imaging optical object distance of the iris optical imaging unit, mm.

[0132] (Xi, Yi) is the XY coordinate of the object space position of the iris optical imaging unit, mm.

[0133] (Xoffset, Yoffset, Zoffset)

[0134] Xoffset=Xe-Xi, Yoffset=Ye-Yi, Zoffset=Ze-Zi is the XYZ coordinate offset of the object space position of the face optical imaging unit relative to the iris optical imaging unit, in mm.

[0135] The inherent object-image optical geometry mapping transformation factor OMTF means no correlation with external imaging conditions.

[0136] The local area configuration unit of the present invention is used to generate a local area ROI of the iris image with a corresponding fixed pixel resolution in the iris optical imaging unit according to the optical geometric mapping relationship of the optical mapping unit to achieve synchronous dynamic configuration.

[0137] The present invention specifically configures a local area ROI of an iris image with a fixed pixel resolution, with the pixel position (PXi, PYi) coordinates as the center, and the local area ROI pixel range of the iris image is (PXroi, PYroi),

[0138] Where: PXroi, PYroi is the XY pixel range of the local area ROI of the iris image,

[0139] cPXi / 16<=PXroi<=cPXi / 2, cPYi / 16<=PYroi<=cPYi / 2

[0140] ROI=RECT(left, top, right, bottom)

[0141] =RECT(PXi-PXroi / 2, PYi-PYroi / 2, PXi+PXroi / 2, PYi+PYroi / 2);

[0142] Or equivalently ROI = RECT(left, top, width, height)

[0143] =RECT(PXi-PXroi / 2, PYi-PYroi / 2, PXroi, PYroi).

[0144] The iris optical imaging unit of the specific embodiment of the present invention realizes iris image acquisition of the central pixel position (PXi, PYi) and pixel range (PXroi, PYroi) of the local area ROI of the iris image with a fixed pixel resolution corresponding to the central pixel position (PXi, PYi) and pixel range (PXroi, PYroi) of the local area ROI of the iris image with a fixed pixel resolution corresponding to the above-mentioned ROI configuration unit.

[0145] Furthermore, in a specific embodiment of the present invention, the iris image acquisition of the central pixel position (PXi, PYi) and the pixel range (PXroi, PYroi) of the local area ROI of the iris image is limited by setting the imaging pixel array path (sensor imaging pixel array passpath) of the iris optical imaging unit corresponding to the physical analog local (analog address) array image output (X_ADD_START, Y_ADD_START, X_ADD_END, Y_ADD_END), or the corresponding digital offset local (digital offset) array image output (X_CROP_OFFSET, Y_CROP_OFFSET, X_CROP_WIDTH, Y_CROP_HEIGHT), wherein

[0146] X_ADD_START=left=PXi-PXroi / 2,

[0147] Y_ADD_START=top=PYi-PYroi / 2,

[0148] X_ADD_END=right=PXi+PXroi / 2,

[0149] Y_ADD_END=bottom=PYi+PYroi / 2

[0150] or

[0151] X_CROP_OFFSET=left=PXi-PXroi / 2,

[0152] Y_CROP_OFFSET=top=PYi-PYroi / 2,

[0153] X_CROP_WIDTH=width=PXroi,

[0154] Y_CROP_HEIGHT=height=PYroi.

[0155] Furthermore, the specific embodiment of the present invention allows the error accuracy of the face / eye detection unit's detection of the center pixel position of both eyes of the face to vary by limiting the appropriate local area ROI of the iris image. More importantly, it allows the optical mapping unit to reduce the dynamic synchronization update of the execution association state between the local area configuration units. The important basic characteristics of the present invention are based on this to optimize the image acquisition and processing performance and execution efficiency, and improve the recognition rate and recognition speed.

[0156] Specific description,

[0157] The specific embodiment of the present invention maintains the current association state between the optical mapping unit and the local area configuration unit according to the current real-time dynamic position (PXe, PYe) satisfying the ROI condition limited to the local area of ​​the iris image, and avoids frequent execution of synchronous update of the association state.

[0158] According to a specific embodiment of the present invention, the current real-time dynamic position (PXe, PYe) does not satisfy the condition limited to the local area ROI of the iris image, and the associated states between the optical mapping unit and the local area configuration unit are synchronously updated.

[0159] Furthermore, the above-mentioned iris image local area ROI limiting conditions of the present invention are converted into the predetermined mutually overlapping unit fixed partition limiting conditions of the image space of the face optical imaging unit, which are specifically as follows:

[0160] -PXroi / 2<=OMTF*(PXe-PXeo)<=PXroi / 2

[0161] -PYroi / 2<=OMTF*(PYe-PYeo)<=PYroi / 2

[0162] (PXe, PYe) is the XY coordinates of the center pixel positions of both eyes of the image side face of the current real-time dynamic face optical imaging unit.

[0163] The predetermined overlapping unit fixed partitions have (PXeo, PYeo) as the partition center position and (PXezone, PYezone) as the partition range.

[0164] (PXeo, PYeo) = (cPXi / OMTF*n / N, cPYi / OMTF*m / M)

[0165] =(PXroi / OMTF*n,PYroi / OMTF*m)

[0166] n=[1,N-1]

[0167] m=[1,M-1]

[0168] N, M are the numbers of predetermined unit fixed partitions.

[0169] N=cPXi / PXroi

[0170] M=cPYi / PYroi

[0171] PXezone=PXroi / OMTF, PYezone=PYroi / OMTF

[0172] It can be understood that in essence, the above mathematical transformations satisfy the limiting conditions. Furthermore, the mathematical relationship between point elements and sets is:

[0173] (PXe,PYe)∈{(PXroi / OMTF*n±PXezone / 2,PYroi / OMTF*m±PYezone / 2)}

[0174] Specifically, by judging whether the (PXe, PYe) coordinates (point elements) belong to the corresponding unit fixed partition (n, m) in the predetermined (N, M) total unit fixed partitions (point element sets), the corresponding unit fixed partition (n, m) association status is dynamically and synchronously updated.

[0175] or, equivalently,

[0176] The above-mentioned mathematical transformation that satisfies the limiting conditions, further, the mathematical position space relationship is:

[0177] PXroi / OMTF*n-PXezone / 2<=PXe<=PXroi / OMTF*n+PXezone / 2

[0178] PYroi / OMTF*m-PYezone / 2<=PYe<=PYroi / OMTF*m+PYezone / 2,

[0179] Specifically, by determining that the (PXe, PYe) coordinates (point position) are positioned at the corresponding unit fixed partition (n, m) in the predetermined (N, M) total unit fixed partitions (spatial positions), the corresponding unit fixed partition (n, m) association status is dynamically and synchronously updated.

[0180] The present invention allows controlling the dynamic synchronization update frequency of the execution association state between the optical mapping unit and the local area configuration unit by limiting the appropriate pixel range (PXroi, PYroi) of the local area ROI of the iris image.

[0181] An important basic characteristic of the present invention is that by limiting the pixel range of the local area ROI (PXroi, PYroi) of the iris image, the amount of image data transmission is reduced. The higher the frame rate achieved under the same transmission bandwidth, the higher the image acquisition speed. In addition, the reduction in the amount of image data reduces the image processing calculation amount and system resources, thereby improving the image processing speed performance.

[0182] In a preferred example of the present invention, the original pixel resolution of the iris optical imaging system is 64 Mpixels. By determining a suitable dynamic configuration, the local area ROI of the iris image with a fixed pixel resolution is limited to the coordinates of the pixel position (PXi, PYi) as the center, and the pixel range of the local area ROI of the iris image with a fixed pixel resolution is PXroi=cPXi / 4, PYroi=cPYi / 4, that is, the pixel range of the local area ROI of the iris image with a fixed pixel resolution is limited to 4 Mpixels.

[0183] Reducing pixel resolution means further, high frame rates such as 60fps, 120fps, increasing the image acquisition frame rate to achieve faster and more accurate image acquisition and processing, faster iris image AF autofocus speed, and improved recognition rate and speed.

[0184] An important basic characteristic of the present invention is that a local area configuration unit is used to achieve synchronous dynamic configuration with a fixed pixel resolution, which can ensure that the resolution of the image data sending end of the iris optical imaging unit and the image data receiving end of the image acquisition and processing unit remain fixed and the same without adjusting the image resolution transmission mode. There is no delay and no configuration switching changes between different resolution modes during dynamic synchronous configuration.

[0185] In a specific embodiment of the present invention, the image acquisition processing unit is connected to the above-mentioned units to realize synchronous image acquisition and image processing calculation. In a specific embodiment, the image acquisition processing unit further preferably comprises an ISP / NPU / CPU component, wherein the ISP is used for image acquisition, the NPU is used for acquisition image calculation, and the CPU is used for acquisition image processing. As shown in FIG2 , a logical schematic diagram of the image acquisition processing unit in a specific embodiment of the present invention is shown. Figure 2 The parallel pipeline execution architecture of the ISP / NPU / CPU components is shown in the figure. Based on this, the image acquisition and computing processing performance and execution efficiency are optimized, and the recognition rate and speed are improved.

[0186] The ISP component further includes dual MIPI CSI for respectively interfacing the image data (sensor RAW data) input of the face optical imaging unit and the iris optical imaging unit.

[0187] Based on the input RAW image data, ISP completes the predetermined image acquisition including AE / AF, contrast / sharpness enhancement, denoise and other internal functions.

[0188] The NPU component is used to perform deep learning-based convolutional neural network (CNN) cascade model calculations based on image data collected by the ISP component, such as implementing the face area detection and face eye center pixel position detection output coordinates (PXe, PYe) of the above-mentioned eye detection unit, face / iris detection segmentation feature extraction and other neural network model algorithm executions.

[0189] The CPU component is used to execute feedback processing control of corresponding associated units according to the image data results of the image calculation of the NPU component, such as the image data feedback processing control of the corresponding associated execution optical mapping unit, the local area configuration unit, and the iris image AF autofocus unit.

[0190] In a specific embodiment of the present invention, the image acquisition and processing unit adopts an image cycle parallel synchronous logic timing working mode. The image cycle parallel synchronous logic timing working mode adopts parallel and synchronous execution of the image calculation cycle (TCn+1) and the image processing cycle (TPn) within the current image acquisition cycle (TAn+2).

[0191] According to the above-mentioned timing relationship requirements of the parallel synchronous working mode, the timing relationship of the current cycle of the present invention satisfies:

[0192] TAn+2>=TCn+1>=TPn

[0193] The time relationship requirements corresponding to the parallel synchronous working mode are as follows:

[0194] TA>=TC>=TP

[0195] The image acquisition time (TA) is greater than or equal to the image calculation time (TC) and the image processing time (TP) is greater than or equal to the image processing time.

[0196] The iris image autofocus AF unit adopts the imaging lens drive parameter control of the iris optical imaging unit; when controlling the parameters, the AF autofocus parameter control depends on the imaging lens drive type, which is achieved by adjusting the imaging optical image distance or adjusting the imaging optical focal length / diopter focal length.

[0197] The imaging lens driving parameter control of the iris optical imaging unit includes:

[0198] Step number 2*K+1 and step length (unit step length) STEP parameter, K is the number of steps;

[0199] Achieve continuous autofocus in one direction within the focusing range FR = (2*K+1)*STEP, such as K = 4,

[0200] in:

[0201] Step size STEP = β2*DOF or STEP = [β / (1+β)]2*DOF, depending on the imaging lens driving type, such as adjusting the imaging optical image distance type STEP = β2*DOF, such as adjusting the imaging optical focal length / diopter focal power type STEP = [β / (1+β)]2*DOF;

[0202] DOF=2*FNO*SOC*(1+β) / β2,

[0203] in:

[0204] β is the lateral optical magnification of the iris optical imaging unit;

[0205] DOF is the depth of field of the iris optical imaging unit;

[0206] FNO is the aperture parameter of the iris optical imaging unit;

[0207] SOC is the minimum resolution parameter of the physical spot of the iris optical imaging unit;

[0208] A very important feature of the present invention is that, in particular, when β<<1, the autofocus satisfies STEP=2*FNO*SOC regardless of any imaging lens drive type. The important feature of this method is that for a given iris optical imaging system, the step length has an inherent object-image optical constant and is independent of the imaging lens drive type.

[0209] The iris image automatic focusing AF unit of the present invention uses the imaging lens driving parameter control of the iris optical imaging unit to generate 2*K+1 step numbers, and the unit step STEP=2*FNO*SOC step length, so as to achieve the object depth of field range of focusing accuracy +-DOF / 2 within the predetermined focusing range FR, and at the same time control the step number to be minimized. The single-direction continuous driving control automatic focusing method avoids the traditional image focusing quality maximum value search two-way repeated iteration, and can ensure that the automatic focusing control process can be executed and completed within 0.1s.

[0210] Furthermore, the automatic focusing AF unit of the present invention can also use different types of imaging lens drivers to execute the imaging lens driving parameters of the iris optical imaging unit, such as PDAF type drivers that adjust the imaging optical image distance or focal length. Depends on the characteristics of the PDAF type driver, its essence is to directly respond to the corresponding focus position relative distance through the image side phase difference, such as a central motor, a closed-loop motor, a memory metal motor, a piezoelectric motor, a MEMS motor, and other linear or nonlinear motors.

[0211] The AF automatic focusing unit method based on PDAF of the present invention includes executing:

[0212] Step S1. First, calculate PD(ROI), where PD is the phase difference corresponding to the ROI area, which depends on different phase pixel detection types such as left-right shield pixel, left-right micro-lens pixel, left-right Dual Pixel, 2*2 / 8PD omnidirectional phase pixel, etc.

[0213] Step S2. Calculate the corresponding focus position relative distance FP according to the corresponding phase difference,

[0214] FP=PD*CC=STEP*K,K=FP / STEP=CC*PD / STEP

[0215] Step length (unit step length) STEP parameter, STEP = 2*FNO*SOC

[0216] CC is the conversion coefficient between the phase difference and the corresponding relative distance FP from the focus position (i.e., the defocus distance, the relative distance from the focus position), i.e., the defocus rate. The defocus rate is achieved by fitting the corresponding linear or nonlinear relationship.

[0217] Step S3. According to the corresponding focus position relative distance FP, the PDAF type driver is driven to execute the imaging lens driving parameters of the iris optical imaging unit, and the corresponding focus position relative distance FP is directly driven. That is, STEP*K step positions are directly driven;

[0218] Step S4. Iterate step S1 to step S3, and terminate at PD<=EP, EP=STEP / CC, EP is a predetermined phase difference focusing error, that is, terminate at a single step length, K<=1.

[0219] The present invention is attributed to the PDAF type driver directly corresponding to the focus position parameter drive. The above method of the present invention can realize the AF automatic focusing within 3 iteration cycles, and the focus position relative distance parameter corresponding to the direct drive is executed no more than 3 times, completing the object depth of field range that meets the focus accuracy +-DOF / 2, and can ensure that the automatic focusing control process can be completed within 0.03s.

[0220] Example 2

[0221] The present embodiment provides a method for collecting face and iris images, comprising: a face optical imaging unit, a human eye detection unit, an optical mapping unit, a local area configuration unit, an iris optical imaging unit, an iris image AF automatic focusing unit, and an image collection and processing unit; the image collection and processing unit is respectively connected to the face optical imaging unit, the human eye detection unit, the optical mapping unit, the local area configuration unit, the iris optical imaging unit, and the iris image AF automatic focusing unit, and controls the face optical imaging unit, the human eye detection unit, the optical mapping unit, the local area configuration unit, the iris optical imaging unit, and the iris image AF automatic focusing unit.

[0222] Step 1. The image acquisition processing unit controls the face optical imaging unit to acquire the corresponding global area face image FaceImage;

[0223] Step 2. The image acquisition processing unit controls the eye detection unit to detect the center pixel position of the eyes of the corresponding face according to the corresponding global area face image FaceImage;

[0224] Step 3. The image acquisition processing unit controls the optical mapping unit to transform the global area face image FaceImage in response to the object-image optical geometric mapping relationship corresponding to the iris image IrisImage according to the corresponding center pixel position of the two eyes of the face;

[0225] Step 4. The image acquisition processing unit controls the local area configuration unit to execute the synchronous dynamic configuration of the local area ROI of the iris image with a corresponding fixed pixel resolution according to the corresponding object-image optical geometric mapping relationship;

[0226] Step 5. The image acquisition processing unit controls the iris optical imaging unit to acquire the iris image IrisImage of the local area ROI of the iris image with a corresponding fixed pixel resolution according to the local area ROI of the iris image with a corresponding fixed pixel resolution;

[0227] Step 6. The image acquisition processing unit controls the iris image AF automatic focusing unit to perform iris image AF automatic focusing according to the corresponding iris image local area ROI iris image IrisImage with a fixed pixel resolution.

[0228] The human eye detection unit is used to realize the detection of the center pixel positions of both eyes of the human face in the global area human face image of the human face optical imaging unit.

[0229] The present invention uses the currently known convolutional neural network (CNN) cascade model based on deep learning to reliably and accurately realize face area detection and face eye center pixel position detection output coordinates (PXe, PYe).

[0230] The optical mapping unit is used to implement the object-image optical geometric mapping relationship transformation OMT of the face optical imaging unit in response to the iris optical imaging unit according to the center pixel positions of both eyes of the face of the face / eye detection unit, (PXe, PYe): -> (PXi, PYi).

[0231] PXi-cPXi / 2

[0232] =Fi / (Zi-Fi)*(Xi) / PSi

[0233] =Fi / (Zi-Fi)*(Xe-Xoffset) / PSi

[0234] =Fi / (Zi-Fi)*[(Ze-Fe) / Fe*(PXe-cPXe / 2)*PSe-Xoffset)] / PSi.

[0235] PYi-cPYi / 2

[0236] =Fi / (Zi-Fi)*(Yi) / PSi

[0237] =Fi / (Zi-Fi)*(Ye-Yoffset) / PSi

[0238] =Fi / (Zi-Fi)*[(Ze-Fe) / Fe*(PYe-cPYe / 2)*PSe-Yoffset)] / PSi.

[0239] In particular, when the condition Ze>>Zoffset, Fi, Fe is met,

[0240] (PXi-cPXi / 2)+Fi / (Zi-Fi)*Xoffset / PSi=(PXe-cPXe / 2)*Fi*PSe / (Fe*PSi)

[0241] (PYi-cPYi / 2)+Fi / (Zi-Fi)*Yoffset / PSi=(PYe-cPYe / 2)*Fi*PSe / (Fe*PSi)

[0242] In particular, when the conditions Ze>>Xoffset, Yoffset, Zoffset, Fi, Fe are met, the important basic characteristics of the present invention are the inherent object-image optical geometric mapping relationship transformation factor OMTF:

[0243] (PXi-cPXi / 2) / (PXe-cPXe / 2)=

[0244] (PYi-cPYi / 2) / (PYe-cPYe / 2)=

[0245] Fi*PSe / (Fe*PSi)=OMTF

[0246] Where: (PXe, PYe) is the XY coordinates of the center pixel position of both eyes of the face on the image side of the face optical imaging unit, pixel.

[0247] (cPXe, cPYe) is the image-side XY pixel resolution of the face optical imaging unit, pixel.

[0248] PSe is the unit pixel resolution of the face optical imaging unit, um / pixel.

[0249] Fe is the imaging optical focal length of the face optical imaging unit, mm.

[0250] Ze is the imaging optical object distance of the face optical imaging unit, mm.

[0251] (Xe, Ye) is the XY position coordinate of the face optical imaging unit in object space, mm.

[0252] (PXi, PYi) is the XY coordinates of the center pixel position of both eyes of the face on the image side of the iris optical imaging unit, pixel.

[0253] (cPXi, cPYi) is the image-side XY pixel resolution of the iris optical imaging unit, pixel.

[0254] PSi is the unit pixel resolution of the iris optical imaging unit, um / pixel.

[0255] Fi is the imaging optical focal length of the iris optical imaging unit, mm.

[0256] Zi is the imaging optical object distance of the iris optical imaging unit, mm.

[0257] (Xi, Yi) is the XY coordinate of the object space position of the iris optical imaging unit, mm.

[0258] (Xoffset, Yoffset, Zoffset)

[0259] Xoffset=Xe-Xi, Yoffset=Ye-Yi, Zoffset=Ze-Zi is the XYZ coordinate offset of the object space position of the face optical imaging unit relative to the iris optical imaging unit, in mm.

[0260] The inherent object-image optical geometry mapping transformation factor OMTF means no correlation with external imaging conditions.

[0261] The local area configuration unit of the present invention is used to generate a local area ROI of the iris image with a corresponding fixed pixel resolution in the iris optical imaging unit according to the optical geometric mapping relationship of the optical mapping unit to achieve synchronous dynamic configuration.

[0262] The present invention specifically configures a local area ROI of an iris image with a fixed pixel resolution, with the pixel position (PXi, PYi) coordinates as the center, and the local area ROI pixel range of the iris image is (PXroi, PYroi),

[0263] Where: PXroi, PYroi is the XY pixel range of the local area ROI of the iris image,

[0264] cPXi / 16<=PXroi<=cPXi / 2, cPYi / 16<=PYroi<=cPYi / 2

[0265] ROI=RECT(left, top, right, bottom)

[0266] =RECT(PXi-PXroi / 2, PYi-PYroi / 2, PXi+PXroi / 2, PYi+PYroi / 2);

[0267] Or equivalently ROI = RECT(left, top, width, height)

[0268] =RECT(PXi-PXroi / 2, PYi-PYroi / 2, PXroi, PYroi).

[0269] The iris optical imaging unit of the specific embodiment of the present invention realizes iris image acquisition of the central pixel position (PXi, PYi) and pixel range (PXroi, PYroi) of the local area ROI of the iris image with a fixed pixel resolution corresponding to the central pixel position (PXi, PYi) and pixel range (PXroi, PYroi) of the local area ROI of the iris image with a fixed pixel resolution corresponding to the above-mentioned ROI configuration unit.

[0270] Furthermore, in a specific embodiment of the present invention, the iris image acquisition of the central pixel position (PXi, PYi) and the pixel range (PXroi, PYroi) of the local area ROI of the iris image is limited by setting the imaging pixel array path (sensor imaging pixel array passpath) of the iris optical imaging unit corresponding to the physical analog local (analog address) array image output (X_ADD_START, Y_ADD_START, X_ADD_END, Y_ADD_END), or the corresponding digital offset local (digital offset) array image output (X_CROP_OFFSET, Y_CROP_OFFSET, X_CROP_WIDTH, Y_CROP_HEIGHT), wherein

[0271] X_ADD_START=left=PXi-PXroi / 2,

[0272] Y_ADD_START=top=PYi-PYroi / 2,

[0273] X_ADD_END=right=PXi+PXroi / 2,

[0274] Y_ADD_END=bottom=PYi+PYroi / 2

[0275] or

[0276] X_CROP_OFFSET=left=PXi-PXroi / 2,

[0277] Y_CROP_OFFSET=top=PYi-PYroi / 2,

[0278] X_CROP_WIDTH=width=PXroi,

[0279] Y_CROP_HEIGHT=height=PYroi.

[0280] Furthermore, the specific embodiment of the present invention allows the error accuracy of the face / eye detection unit's detection of the center pixel position of both eyes of the face to vary by limiting the appropriate local area ROI of the iris image. More importantly, it allows the optical mapping unit to reduce the dynamic synchronization update of the execution association state between the local area configuration units. The important basic characteristics of the present invention are based on this to optimize the image acquisition and processing performance and execution efficiency, and improve the recognition rate and recognition speed.

[0281] Specific description,

[0282] The specific embodiment of the present invention maintains the current association state between the optical mapping unit and the local area configuration unit according to the current real-time dynamic position (PXe, PYe) satisfying the ROI condition limited to the local area of ​​the iris image, and avoids frequent execution of synchronous update of the association state.

[0283] According to a specific embodiment of the present invention, the current real-time dynamic position (PXe, PYe) does not satisfy the condition limited to the local area ROI of the iris image, and the associated states between the optical mapping unit and the local area configuration unit are synchronously updated.

[0284] Furthermore, the above-mentioned iris image local area ROI limiting conditions of the present invention are converted into the predetermined mutually overlapping unit fixed partition limiting conditions of the image space of the face optical imaging unit, which are specifically as follows:

[0285] -PXroi / 2<=OMTF*(PXe-PXeo)<=PXroi / 2

[0286] -PYroi / 2<=OMTF*(PYe-PYeo)<=PYroi / 2

[0287] (PXe, PYe) is the XY coordinates of the center pixel positions of both eyes of the image side face of the current real-time dynamic face optical imaging unit.

[0288] The predetermined overlapping unit fixed partitions have (PXeo, PYeo) as the partition center position and (PXezone, PYezone) as the partition range.

[0289] (PXeo, PYeo) = (cPXi / OMTF*n / N, cPYi / OMTF*m / M)

[0290] =(PXroi / OMTF*n,PYroi / OMTF*m)

[0291] n=[1,N-1]

[0292] m=[1,M-1]

[0293] N, M are the number of fixed partitions in the unit

[0294] N=cPXi / PXroi

[0295] M=cPYi / PYroi

[0296] PXezone=PXroi / OMTF, PYezone=PYroi / OMTF

[0297] It can be understood that in essence, the above mathematical transformations satisfy the limiting conditions. Furthermore, the mathematical relationship between point elements and sets is:

[0298] (PXe,PYe)∈{(PXroi / OMTF*n±PXezone / 2,PYroi / OMTF*m±PYezone / 2)}

[0299] Specifically, by judging whether the (PXe, PYe) coordinates (point elements) belong to the corresponding unit fixed partition (n, m) in the predetermined (N, M) total unit fixed partitions (point element sets), the corresponding unit fixed partition (n, m) association status is dynamically and synchronously updated.

[0300] or, equivalently,

[0301] The above-mentioned mathematical transformation that satisfies the limiting conditions, further, the mathematical position space relationship is:

[0302] PXroi / OMTF*n-PXezone / 2<=PXe<=PXroi / OMTF*n+PXezone / 2

[0303] PYroi / OMTF*m-PYezone / 2<=PYe<=PYroi / OMTF*m+PYezone / 2,

[0304] Specifically, by determining that the (PXe, PYe) coordinates (point position) are positioned at the corresponding unit fixed partition (n, m) in the predetermined (N, M) total unit fixed partitions (spatial positions), the corresponding unit fixed partition (n, m) association status is dynamically and synchronously updated.

[0305] The present invention allows controlling the dynamic synchronization update frequency of the execution association state between the optical mapping unit and the local area configuration unit by limiting the appropriate pixel range (PXroi, PYroi) of the local area ROI of the iris image.

[0306] An important basic characteristic of the present invention is that by limiting the pixel range of the local area ROI (PXroi, PYroi) of the iris image, the amount of image data transmission is reduced. The higher the frame rate achieved under the same transmission bandwidth, the higher the image acquisition speed. In addition, the reduction in the amount of image data reduces the image processing calculation amount and system resources, thereby improving the image processing speed performance.

[0307] In a preferred example of the present invention, the original pixel resolution of the iris optical imaging system is 64 Mpixels. By determining a suitable dynamic configuration, the local area ROI of the iris image with a fixed pixel resolution is limited to the coordinates of the pixel position (PXi, PYi) as the center, and the pixel range of the local area ROI of the iris image with a fixed pixel resolution is PXroi=cPXi / 4, PYroi=cPYi / 4, that is, the pixel range of the local area ROI of the iris image with a fixed pixel resolution is limited to 4 Mpixels.

[0308] Reducing pixel resolution means further, high frame rates such as 60fps, 120fps, increasing the image acquisition frame rate to achieve faster and more accurate image acquisition and processing, faster iris image AF autofocus speed, and improved recognition rate and speed.

[0309] An important basic characteristic of the present invention is that a local area configuration unit is used to achieve synchronous dynamic configuration with a fixed pixel resolution, which can ensure that the resolution of the image data sending end of the iris optical imaging unit and the image data receiving end of the image acquisition and processing unit remain fixed and the same without adjusting the image resolution transmission mode. There is no delay and no configuration switching changes between different resolution modes during dynamic synchronous configuration.

[0310] In a specific embodiment of the present invention, the image acquisition processing unit is connected to the above-mentioned units to realize synchronous image acquisition and image processing calculation. In a specific embodiment, the image acquisition processing unit further preferably comprises an ISP / NPU / CPU component, wherein the ISP is used for image acquisition, the NPU is used for acquisition image calculation, and the CPU is used for acquisition image processing. As shown in FIG2 , a logical schematic diagram of the image acquisition processing unit in a specific embodiment of the present invention is shown. Figure 2 The parallel pipeline execution architecture of the ISP / NPU / CPU components is shown in the figure. Based on this, the image acquisition and computing processing performance and execution efficiency are optimized, and the recognition rate and speed are improved.

[0311] The ISP component further includes dual MIPI CSI for respectively interfacing the image data (sensor RAW data) input of the face optical imaging unit and the iris optical imaging unit.

[0312] Based on the input RAW image data, ISP completes the predetermined image acquisition including AE / AF, contrast / sharpness enhancement, denoise and other internal functions.

[0313] The NPU component is used to perform deep learning-based convolutional neural network (CNN) cascade model calculations based on image data collected by the ISP component, such as implementing the face area detection and face eye center pixel position detection output coordinates (PXe, PYe) of the above-mentioned eye detection unit, face / iris detection segmentation feature extraction and other neural network model algorithm executions.

[0314] The CPU component is used to execute feedback processing control of corresponding associated units according to the image data results of the image calculation of the NPU component, such as the image data feedback processing control of the corresponding associated execution optical mapping unit, the local area configuration unit, and the iris image AF autofocus unit.

[0315] In a specific embodiment of the present invention, the image acquisition and processing unit adopts an image cycle parallel synchronous logic timing working mode. The image cycle parallel synchronous logic timing working mode adopts parallel and synchronous execution of the image calculation cycle (TCn+1) and the image processing cycle (TPn) within the current image acquisition cycle (TAn+2).

[0316] According to the above-mentioned timing relationship requirements of the parallel synchronous working mode, the timing relationship of the current cycle of the present invention satisfies:

[0317] TAn+2>=TCn+1>=TPn

[0318] The time relationship requirements corresponding to the parallel synchronous working mode are as follows:

[0319] TA>=TC>=TP

[0320] The image acquisition time (TA) is greater than or equal to the image calculation time (TC) and the image processing time (TP) is greater than or equal to the image processing time.

[0321] The iris image autofocus AF unit adopts the imaging lens drive parameter control of the iris optical imaging unit; when controlling the parameters, the AF autofocus parameter control depends on the imaging lens drive type, which is achieved by adjusting the imaging optical image distance or adjusting the imaging optical focal length / diopter focal length.

[0322] The imaging lens driving parameter control of the iris optical imaging unit includes:

[0323] Step number 2*K+1 and step length (unit step length) STEP parameter, K is the number of steps;

[0324] Achieve continuous autofocus in one direction within the focusing range FR = (2*K+1)*STEP, such as K = 4,

[0325] in:

[0326] Step size STEP = β2*DOF or STEP = [β / (1+β)]2*DOF, depending on the imaging lens driving type, such as adjusting the imaging optical image distance type STEP = β2*DOF, such as adjusting the imaging optical focal length / diopter focal power type STEP = [β / (1+β)]2*DOF;

[0327] DOF=2*FNO*SOC*(1+β) / β2,

[0328] in:

[0329] β is the lateral optical magnification of the iris optical imaging unit;

[0330] DOF is the depth of field of the iris optical imaging unit;

[0331] FNO is the aperture parameter of the iris optical imaging unit;

[0332] SOC is the minimum resolution parameter of the physical spot of the iris optical imaging unit;

[0333] A very important feature of the present invention is that, in particular, when β<<1, the autofocus satisfies STEP=2*FNO*SOC regardless of any imaging lens drive type. The important feature of this method is that for a given iris optical imaging system, the step length has an inherent object-image optical constant and is independent of the imaging lens drive type.

[0334] The iris image automatic focusing AF unit of the present invention uses the imaging lens driving parameter control of the iris optical imaging unit to generate 2*K+1 step numbers, and the unit step STEP=2*FNO*SOC step length, so as to achieve the object depth of field range of focusing accuracy +-DOF / 2 within the predetermined focusing range FR, and at the same time control the step number to be minimized. The single-direction continuous driving control automatic focusing method avoids the traditional image focusing quality maximum value search two-way repeated iteration, and can ensure that the automatic focusing control process can be executed and completed within 0.1s.

[0335] Furthermore, the automatic focusing AF unit of the present invention can also use different types of imaging lens drivers to execute the imaging lens driving parameters of the iris optical imaging unit, such as PDAF type drivers that adjust the imaging optical image distance or focal length. Depends on the characteristics of the PDAF type driver, its essence is to directly respond to the corresponding focus position relative distance through the image side phase difference, such as a central motor, a closed-loop motor, a memory metal motor, a piezoelectric motor, a MEMS motor, and other linear or nonlinear motors.

[0336] The AF automatic focusing unit method based on PDAF of the present invention includes executing:

[0337] Step S1. First, calculate PD(ROI), where PD is the phase difference corresponding to the ROI area, which depends on different phase pixel detection types such as left-right shield pixel, left-right micro-lens pixel, left-right Dual Pixel, 2*2 / 8PD omnidirectional phase pixel, etc.

[0338] Step S2. Calculate the corresponding focus position relative distance FP according to the corresponding phase difference,

[0339] FP=PD*CC=STEP*K,K=FP / STEP=CC*PD / STEP

[0340] Step length (unit step length) STEP parameter, STEP = 2*FNO*SOC

[0341] CC is the conversion coefficient between the phase difference and the corresponding relative distance FP from the focus position (i.e., the defocus distance, the relative distance from the focus position), i.e., the defocus rate. The defocus rate is achieved by fitting the corresponding linear or nonlinear relationship.

[0342] Step S3: according to the corresponding focus position relative distance FP, the PDAF type driver is driven to execute the imaging lens driving parameters of the iris optical imaging unit, and the corresponding focus position relative distance FP is directly driven, that is, STEP*K step positions are directly driven.

[0343] Step S4. Iterate step S1 to step S3, and terminate at PD<=EP, EP=STEP / CC, EP is a predetermined phase difference focusing error, that is, terminate at a single step length, K<=1.

[0344] The present invention is attributed to the PDAF type driver directly corresponding to the focus position parameter drive. The above method of the present invention can realize the AF automatic focusing within 3 iteration cycles, and the focus position relative distance parameter corresponding to the direct drive is executed no more than 3 times, completing the object depth of field range that meets the focus accuracy +-DOF / 2, and can ensure that the automatic focusing control process can be completed within 0.03s.

[0345] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here, such as parameter adjustment, component change, step replacement, etc. The obvious changes or modifications derived therefrom are still within the protection scope of the invention.

Claims

1. A face and iris recognition image acquisition and processing device, characterized in that: include: An image acquisition processing unit, wherein the image acquisition processing unit acquires images in an image period parallel synchronous logic timing working mode, wherein the image period parallel synchronous logic timing working mode executes the image calculation cycle TCn+1 and the image processing cycle TPn in parallel and synchronously within the current image acquisition cycle TAn+2; Among them, the timing relationship of the current cycle satisfies: TAn+2>=TCn+1>=TPn; Time relationship meets: TA>=TC>=TP; TA is the image acquisition time, TC is the image calculation time, and TP is the image processing time.

2. The device according to claim 1, characterized in that The image acquisition processing unit comprises: ISP component, NPU component and CPU component; The ISP component is used for image acquisition, the NPU component is used for image calculation, and the CPU component is used for image processing; The ISP component, NPU component and CPU component workflow execute a parallel pipeline architecture; The ISP components are interfaced with the image data input of the face optical imaging unit and the iris optical imaging unit respectively; The ISP component is used to complete the predetermined face and iris image acquisition function according to the input image data; The NPU component is used to perform deep learning-based cascade model calculation according to the image data collected by the ISP component, and the deep learning-based cascade model includes a neural network algorithm model for face / iris detection and segmentation feature extraction; The CPU component is used to perform feedback processing control of the corresponding association unit according to the image data result of the image calculation of the NPU component.

3. The device according to claim 1, characterized in that The image acquisition processing unit is connected to the iris image AF automatic focusing unit and controls the iris image AF automatic focusing unit.

4. The device according to claim 3, characterized in that The iris image automatic focusing AF unit adopts the imaging lens driving parameter control of the iris image acquisition unit to achieve or adjust the imaging optical focal length / diopter focal length by adjusting the imaging optical image distance.

5. The device according to claim 4, characterized in that The iris image automatic focusing AF unit adopts the imaging lens driving parameter control of the iris image acquisition unit, including: The step number 2*K+1 and the unit step length STEP parameters realize continuous automatic focusing in one direction within the focusing range FR=(2*K+1)*STEP; Where: K is the step number parameter; STEP=2*FNO*SOC, where FNO is the aperture parameter of the iris optical imaging unit; and SOC is the minimum resolution parameter of the physical light spot of the iris optical imaging unit.

6. The device according to claim 5, characterized in that The iris image automatic focusing AF unit adopts the imaging lens driving parameter control of the iris image acquisition unit, including performing the following steps based on the PDAF type driver: Step S1, calculating the phase difference PD corresponding to the local area ROI of the iris image; Step S2, calculating the corresponding focus position relative distance FP according to the phase difference; Step S3, driving the PDAF type driver to execute the imaging lens driving parameters of the iris optical imaging unit according to the relative distance FP, driving the corresponding focus position relative distance FP, and driving STEP*K step positions; Step S4, iterate step S1 to step S3 until PD<=EP is satisfied; Wherein, EP is the predetermined phase difference focusing error, K is the number of steps, and STEP is the unit step length.

7. The device according to claim 6, characterized in that The phase difference PD corresponding to the ROI area is realized according to different phase pixel detection types of omnidirectional phase pixels.

8. The device according to claim 6, characterized in that The phase difference calculation corresponding to the focus position relative distance FP is realized by the following formula: FP = PD * CC = STEP * K; Among them, K = FP / STEP = CC*PD / STEP; STEP = 2 * FNO * SOC; CC is the defocus ratio of the phase difference and the corresponding focus position relative distance FP, and the defocus ratio is achieved by fitting the actual corresponding linear or nonlinear relationship.

9. The device according to claim 6, characterized in that The PDAF type actuator responds directly to the corresponding focus position relative distance via the image-side phase difference.

10. A method for collecting and processing facial iris recognition images, characterized in that: Using the device as described in any one of claims 1 to 9, image acquisition and processing for face and iris recognition is implemented.

Images