A method for amplifying periorbital samples and its application

Abstract

The present invention discloses a method for amplifying periorbital samples, comprising the following steps: dividing a sample image into a periorbital region, an iris region, and a pupil region; calculating the coordinate mapping relationship between the amplified image and the sample image such that the radius ratio of the iris region and the pupil region in the amplified image is m / 1; performing an interpolation operation on the sample image to calculate the pixel value of each pixel in the amplified image; filling the information-deficient regions in the iris region and the pupil region of the amplified image respectively to obtain an enhanced amplified image; adding the enhanced amplified image to the periorbital sample library to participate in the training of the periorbital recognition model. The present invention also discloses an application of the method for amplifying periorbital samples, using the trained periorbital recognition model to classify the collected periorbital images. The present invention effectively eliminates the periorbital recognition errors caused by the inconsistent sizes of the pupils and irises in the periorbital images collected by the same person under different scene conditions, and improves the effect of periorbital recognition achieved by deep learning.

Description

Technical Field

[0001] The present invention belongs to the technical field of graphic recognition, and particularly relates to a method for amplifying periorbital samples and its application. Background Art

[0002] The development of modern society has put forward higher requirements for the accuracy, security and usability of human identity recognition. At present, traditional photo recognition can no longer meet the requirements of the times, and biometric recognition has gradually replaced photo recognition as the mainstream identity recognition method. Biometric recognition technology refers to using the inherent physiological or behavioral characteristics of the human body for identity recognition. Because biometric characteristics have advantages such as universality, uniqueness, stability, and convenience, and are difficult to forge and imitate, they have made great contributions in many fields.

[0003] Among them, among many biometric characteristics, periorbital characteristics have great advantages because they are stable, unique, non-invasive within a certain period of time, and have random detail and texture characteristics. At the same time, periorbital recognition technology also has the advantage of non-contact acquisition, making it have a broad market prospect and scientific research value. The existing periorbital detection is realized based on deep learning technology, that is, by training a large number of periorbital samples on a deep neural network, so that the trained deep neural network has the ability to detect and accurately classify a certain type of periorbital image.

[0004] For the training samples and detection samples of periorbital recognition, due to the diversity and complexity of the acquisition scenarios and conditions, there are problems such as different pupil sizes and iris sizes in the samples. Even for the same person, due to different sampling environments and conditions, samples with different pupil and iris sizes will be collected.

[0005] In the periorbital recognition model composed of a deep neural network, this situation where periorbital samples belong to the same person but have different or even very different iris and pupil sizes will become an interference item for recognition, affecting the accuracy of the periorbital recognition model. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for amplifying periorbital samples and its application in view of the deficiencies in the above-mentioned prior art. The structure is simple and reasonably designed. The pupil radius is adjusted according to the radius ratio of the pupil area and the iris area to obtain an amplified image; the weight value in the bilinear interpolation method is changed from distance to the square of the distance, and the pixel value of the pixel point is obtained by using the bilinear interpolation method; the area with missing information in the amplified image is filled with information to obtain an enhanced amplified image, which can effectively eliminate the periorbital recognition errors caused by the inconsistent pupil and iris sizes of the periorbital images collected by the same person under different scene conditions, and improve the learning effect of periorbital recognition using deep learning technology.

[0007] To solve the above technical problems, the technical solution adopted by the present invention is: a method for amplifying periorbital samples, characterized by including the following steps:

[0008] Step 1: Extract the pre-stored sample images from the periorbital sample library, and divide the sample images into periorbital regions, iris regions, and pupil regions;

[0009] Step 2: Amplify the image region normalization: Calculate the coordinate mapping relationship between the amplified image and the sample image, so that the radius ratio of the iris region and the pupil region in the amplified image is m / 1, obtain the coordinates of the iris region and the pupil region in the amplified image, and the coordinates of the iris region and the pupil region form a coordinate point set (X, Y), where m is a positive integer;

[0010] Step 3: Perform interpolation operation on the sample image to calculate the pixel value of each pixel in the amplified image: Where (X t , Y t ) represents the t-th coordinate in the coordinate point set (X, Y) in the amplified image, f(X t , Y t ) represents the pixel value of the coordinate (X t , Y t ) in the amplified image, and the floating-point coordinates of (X t , Y t ) mapped to the sample image coordinate system are denoted as (x' t , y' t ), (x αt , y αt ) represents the α-th integer coordinate point closest to the floating-point coordinates (x' t , y' t ) in the sample image coordinate system, ω a represents the weight, α = 1, 2, 3, 4, represents the distance between the α-th integer coordinate point (x αt , y αt ) and the floating-point coordinates (x' t, y′ t ), Representation and Related functions;

[0011] Step 4: Determine whether there is an information missing area in the iris area. If not, proceed to step 6. If so, proceed to step 5.

[0012] Step 5, filling the information missing area of ​​the augmented image: filling the information missing area of ​​the iris area in the augmented image to obtain an enhanced augmented image;

[0013] Step 6: Add the enhanced amplified image to the periocular sample library to participate in the training of the periocular recognition model.

[0014] The above method for amplifying periocular samples is characterized in that: in step 2,

[0015] The above method for amplifying periocular samples is characterized in that: in step 2,

[0016] The above-mentioned method for amplifying periocular samples is characterized in that: the specific method of step 5 is:

[0017] Step 501, dividing the augmented image into an eye periphery area, an iris occlusion area and an iris known area;

[0018] Step 502, corroding the iris occlusion area: using a morphological algorithm, corroding the iris occlusion area from the first edge to the second edge, corroding z units in each round, and the pixels of the corroded iris occlusion area are assigned black;

[0019] Step 503: Fill the iris occlusion area: According to the formula Calculate the pixel value f(p,q) of the point in the corroded iris occlusion area, where (p,q) represents the pixel point in the corroded iris occlusion area, (p v ,q v ) represents the pixel point located in the known iris area and adjacent to (p, q), 1≤v≤u, u is a positive integer not less than 2, the pixel value f(p,q) is assigned to the pixel point (p,q), and the pixel point (p,q) is added to the known iris area;

[0020] Step 504: Repeat steps 502 and 503 until the iris occlusion area is completely eroded and filled.

[0021] The above-mentioned method for amplifying periocular samples is characterized in that the corrosion is performed by horizontal corrosion or vertical corrosion.

[0022] The above-mentioned method for amplifying periorbital samples is characterized in that: in the step 3, wherein, d1 represents the distance between (x′ t , y′ t ) and (x 1t , y 1t ), d2 represents the distance between (x′ t , y′ t ) and (x 2t , y 2t ), d3 represents the distance between (x′ t , y′ t ) and (x 3t , y 3t ), d4 represents the distance between (x′ t , y′ t ) and (x 4t , y 4t ).

[0023] The above-mentioned method for amplifying periorbital samples is characterized in that: the sample image has eyelid marking coordinates, iris marking coordinates and pupil marking coordinates. According to the eyelid marking coordinates, an eyelid curve is fitted;

[0024] According to the pupil marking coordinates, a pupil curve is fitted;

[0025] According to the iris marking coordinates, an iris curve is fitted;

[0026] The sample image is divided into a periorbital area, a pupil area and an iris area according to the eyelid curve, the pupil curve and the iris curve.

[0027] An application of a method for amplifying periorbital samples is characterized in that samples in a periorbital sample library are amplified to obtain enhanced amplified images, the enhanced amplified images are added to the periorbital sample library, and based on the samples in the periorbital sample library, a periorbital recognition model composed of a deep neural network is trained to obtain a trained periorbital recognition model, and the trained periorbital recognition model is used to classify the collected periorbital images for identity recognition.

[0028] The present invention has the following advantages compared with the prior art:

[0029] 1. The structure of the present invention is simple, reasonably designed, and convenient to implement and use.

[0030] 2. The present invention changes the weight in the bilinear interpolation method in the prior art from distance to the square of distance, and uses the square of the distance between integer coordinate points as the weight ω a, by combining the pixel values of four integer coordinate points and using the bilinear interpolation method to calculate the interpolation of the floating-point coordinates at the coordinates (X, Y) in the amplified image, this interpolation is the pixel value of the coordinates (X, Y) in the amplified image, so that the interpolation has stronger non-linear characteristics, and thus more sharpness of the original image texture is retained during the magnification of the iris region.

[0031] 3. The present invention determines the pupil radius in the amplified image according to the radius ratio of the pupil region and the iris region, calculates the coordinate mapping relationship between the coordinate points in the iris region of the amplified image and the coordinate points in the iris region of the sample image according to the change of the pupil radius, and maps each coordinate point in the pupil region of the amplified image to the coordinate point on the pupil region of the sample image, which can effectively eliminate the eye perimeter recognition errors caused by the inconsistent pupil and iris sizes in the eye perimeter images collected by the same person under different scene conditions, and improve the learning effect of eye perimeter recognition using deep learning technology.

[0032] 4. The present invention fills the regions with missing information in the amplified image, can well restore the structural features of the image, make the restoration result more natural, improve the quality of the enhanced amplified image, enhance the recognition efficiency of the eye perimeter recognition model, and improve the reliability of the eye perimeter recognition model.

[0033] In summary, the present invention has a simple structure and reasonable design. It adjusts the pupil radius according to the radius ratio of the pupil region and the iris region to obtain an amplified image; changes the weight in the bilinear interpolation method from distance to the square of the distance, and uses the bilinear interpolation method to obtain the pixel of the pixel point; fills the regions with missing information in the amplified image to obtain an enhanced amplified image, effectively eliminating the eye perimeter recognition errors caused by the inconsistent pupil and iris sizes in the eye perimeter images collected by the same person under different scene conditions, and improving the learning effect of eye perimeter recognition using deep learning technology.

[0034] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0035] Figure 1 is the flowchart of the method of the present invention.

[0036] Figure 2 is the comparison diagram of the normalized amplified image and the sample image of the present invention.

[0037] Figure 3 is the comparison diagram of the amplified image before and after filling of the present invention.

[0038] Figure 4 is the ROC performance curve obtained by training the eye perimeter recognition model with the eye perimeter sample library using the amplified image with enhancement added and the amplified image without enhancement respectively. Detailed implementation manners

[0039] The method of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

[0040] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary implementation manners according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above accompanying drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the implementation manners of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0043] For the sake of description, spatial relative terms such as "above", "over", "on the upper surface", "above" can be used herein to describe the spatial positional relationship between a device or feature shown in the figure and other devices or features. It should be understood that the spatial relative terms are intended to include different orientations in use or operation in addition to the orientation described in the figure of the device. For example, if the device in the figure is inverted, the device described as "above other devices or structures" or "over other devices or structures" will then be positioned as "below other devices or structures" or "under other devices or structures". Thus, the exemplary term "above" can include both the orientation of "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and corresponding interpretations are made for the spatial relative descriptions used herein.

[0044] Embodiment 1

[0045] Embodiment 1 of the present application provides a method for amplifying periorbital samples, which is executed by an electronic device. The electronic device can be a server or a terminal device. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a mobile terminal such as a mobile phone, a laptop, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Computer), a PMP (Portable Multimedia Player), a vehicle terminal (such as a vehicle navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, which are not limited in the embodiments of the present application.

[0046] As Figure 1 shown, Embodiment 1 of the present application specifically includes the following steps:

[0047] Step 1: Extract the pre-stored sample images from the periorbital sample library, and divide the sample images into a periorbital region, an iris region, and a pupil region.

[0048] Among them, the sample image is a periorbital sample image, which includes a periorbital region, an iris region, and a pupil region. The periorbital region is the area around the eye, from the inner iris region to the temple, from the upper orbital margin to the upper orbital rim, from the lower orbital margin to the lower orbital rim, and the middle part of this area.

[0049] The periorbital sample library includes multiple collected sample images of the periorbital region. In actual use, some sample images are called from the periorbital sample library, and professional ophthalmologists mark the eyelid key points, iris key points, and pupil key points on the sample images to obtain the eyelid marking coordinates, iris marking coordinates, and pupil marking coordinates.

[0050] Assume that the eyelid is a quadratic curve, the iris edge is a closed circle, and the pupil edge is a closed circle. According to the fitting of the marking coordinates, an eyelid curve, an iris curve, and a pupil curve can be formed, and the sample image can be divided into a periorbital region, a pupil region, and an iris region according to the eyelid curve, pupil curve, and iris curve.

[0051] Step 2: Amplification image region normalization: Calculate the coordinate mapping relationship between the amplified image and the sample image, so that the radius ratio of the iris region and the pupil region in the amplified image is m / 1, obtain the coordinates of the iris region and the pupil region in the amplified image, and the coordinates of the iris region and the pupil region form a coordinate point set (X, Y), where m is a positive integer; obtain the coordinate point set (X, Y) of the amplified image, (X, Y) = {(X 0i , Y 0i ), (X 1j , Y 1j ), (X 0i, Y 0i ) represents the coordinates within the pupil region in the amplified image, (X 1j , Y 1j ) represents the coordinates within the iris region in the amplified image.

[0052] In a possible embodiment, where R represents the iris radius of the sample image, r1 represents the pupil radius of the sample image, r3 represents a preset pupil radius, d represents the distance between (X 1j , Y 1j ) and the center of the pupil region on the amplified image, (x 0i , y 0i ) represents the i-th coordinate of the pupil region in the sample image, (X 0i , Y 0i ) represents the coordinate corresponding to (x 0i , y 0i ) within the pupil region in the amplified image, (x 1j , y 1j ) represents the j-th coordinate of the iris region in the sample image, (X 1j , Y 1j ) represents the coordinate corresponding to (x 1j , y 1j ) within the iris region in the amplified image.

[0053] In a possible embodiment, r3 represents the pupil radius in the amplified image, r2 represents the iris radius in the amplified image, and it is set to be half of the iris radius R of the current sample image to obtain the ideal pupil radius size of the current sample image. Then, according to the difference between the pupil radius r1 of the sample image and the ideal pupil radius R / 2, the size of the pupil region is adjusted, and the size of the iris region is adjusted adaptively to obtain the coordinate mapping relationship between the sample image and the amplified image, thereby obtaining the amplified image.

[0054] In actual use, when the total area of the pupil + iris in the pupil remains unchanged, if the pupil region is enlarged according to the pupil radius being one-third of the iris radius, the area occupied by the iris region will necessarily shrink accordingly. Similarly, if the pupil region is required to shrink, the area occupied by the iris region will necessarily enlarge accordingly.

[0055] As Figure 2 shown, Figure 2 a is the sample image, Figure 2 in a, the pupil radius is much larger than half of the iris radius, and the sample image does not belong to an ideal sample image. For Figure 2 a, the method in step 2 is used for amplification to obtain Figure 2 b, Figure 2 b is the amplified image, throughFigure 2 b It can be clearly seen that the area occupied by the pupil region decreases, and the area occupied by the iris region increases. Figure 2 b In b, the pupil radius is equal to one - half of the iris radius.

[0056] In a possible embodiment, the coordinate mapping relationship between the coordinate points of the pupil region of the amplified image and the coordinate points of the pupil region of the sample image is That is Through this formula, each coordinate point of the pupil region of the amplified image can be mapped to the coordinate point on the pupil region of the sample image.

[0057] In a possible embodiment, the coordinate mapping relationship between the coordinate points of the iris region of the amplified image and the coordinate points of the iris region of the sample image is That is d represents the distance between (X 1j , Y 1j ) on the amplified image and the center of the pupil region, and D represents the distance between (x 1j , y 1j ) on the sample image and the center of the pupil region. By the formula The change in the pupil radius is obtained, and then the coordinate mapping relationship between the coordinate points of the iris region of the amplified image and the coordinate points of the iris region of the sample image is calculated according to the change in the pupil radius.

[0058] Step 3: Perform interpolation operation on the sample image to calculate the pixel value of each pixel of the amplified image: Where (X t , Y t ) represents the t - th coordinate in the coordinate point set (X, Y) in the amplified image, f(X t , Y t ) represents the pixel value of the coordinate (X t , Y t ) in the amplified image. The floating - point coordinates of (X t , Y t ) mapped to the sample image coordinate system are denoted as (x′ t , y′ t ). (x αt , y αt ) represents the α - th integer coordinate point closest to the floating - point coordinates (x′ t , y′ t ) in the sample image coordinate system. ω a represents the weight, α = 1, 2, 3, 4, represents the distance from the α - th integer coordinate point (x αt , yαt ) The squared value of the distance from the floating-point coordinates (x′ t , y′ t ), which represents a function related to . .

[0059] In a possible embodiment, in step 3, where d1 represents the distance between (x′ t , y′ t ) and (x 1t , y 1t ), d2 represents the distance between (x′ t , y′ t ) and (x 2t , y 2t ), d3 represents the distance between (x′ t , y′ t ) and (x 3t , y 3t ), d4 represents the distance between (x′ t , y′ t ) and (x 4t , y 4t ).

[0060] It should be noted that the floating-point coordinates obtained by mapping (X, Y) to the sample image coordinate system are denoted as (x', y'). Generally, the floating-point coordinates (x', y') are not integers, and non-integer coordinates cannot be used in discrete data such as images. Therefore, by finding the four integer coordinate points closest to the floating-point coordinates (x', y'), the pixel value of the coordinate (X, Y) in the amplified image is calculated.

[0061] For example, if the corresponding coordinate values of (x', y') are (2.5, 4.5), then the four integer coordinate points (x α , y α ) closest to it are (2, 4), (2, 5), (3, 4), and (3, 5).

[0062] In a possible embodiment, the weight ω is calculated based on the squared value of the distance between the α-th integer coordinate point and the floating-point coordinates (x', y'). a Combined with the pixel values of the four integer coordinate points, the interpolation of the floating-point coordinates of the coordinate (X, Y) in the amplified image is calculated using the bilinear interpolation method, and this interpolation is the pixel value of the coordinate (X, Y) in the amplified image.

[0063] The present application changes the weight in the bilinear interpolation method in the prior art from distance to the square of the distance, which can be regarded as a further change of the bilinear interpolation method, so that the interpolation has a stronger nonlinear characteristic, thereby retaining more of the sharpness of the original image texture during the enlargement of the iris area.

[0064] Step 4: Determine whether there is an information missing area in the iris region. If not, proceed to step 6; if so, proceed to step 5.

[0065] The method for determining whether there is an information missing area in the iris region does not belong to the innovation of the present application. In a possible embodiment, preferably, the method for determining whether there is an information missing area in the iris region can be: first, the iris image in the augmented image is extracted by positioning, and then the iris image in the augmented image is decomposed into 4 layers of multi-resolution. By counting the number of points with large amplitudes of the detail component at the resolution and comparing it with a preset threshold, it can be determined whether the iris image has an eyelid occlusion problem.

[0066] Step 5, filling the information missing area of ​​the augmented image: filling the information missing area of ​​the iris area in the augmented image to obtain an enhanced augmented image.

[0067] After normalizing the size of the pupil and iris areas in the sample image, there may be areas with missing information that need to be filled and repaired. This is because in the commonly collected periocular samples, many periocular samples have the iris or even the pupil partially obscured by the eyelids. Since the augmented image needs to adjust the size of the pupil and iris areas, if the iris area is partially obscured, if the iris area needs to be enlarged, then the edge of the obtained augmented image will have a small part that happens to fall on the area obscured by the eyelids in the original sample image, so that some parts of the information in the augmented image cannot find a reference point in the sample image, resulting in information loss.

[0068] like Figure 3 As shown, Figure 3 a is a sample image. The upper and lower parts of the iris area are covered by the eyelids. Figure 3 b is the sample image after corrosion, the area in the iris region that is blocked by the eyelid is assigned to other pixels. Figure 3 c is a sample image after restoration and filling, that is, an enhanced augmented image, in which the area in the iris region obscured by the eyelid is assigned a pixel interpolation value of the adjacent pixels.

[0069] In a possible embodiment, the specific method of step 5 is:

[0070] Step 501, dividing the augmented image into an eye periphery area, an iris obstruction area and an iris known area;

[0071] Step 502, Erode the iris occluded area: Using the morphological algorithm, erode the iris occluded area from the first edge to the second edge direction, eroding z units in each round, and the pixel points of the eroded iris occluded area are assigned black.

[0072] Erosion is one of the most basic morphological algorithm operations, which can eliminate the boundary points of the image and make the image shrink inward along the boundary.

[0073] In a possible embodiment, the main function used for image erosion is erode(). The erosion operation processes each pixel x in the iris occluded area as follows: Take a template, place the pixel x at the first edge, and according to the size of the template, traverse all other pixels covered by the template and modify the value of pixel x. The template is a 1×1 unit pixel point.

[0074] Step 503, Fill the iris occluded area: According to the formula Calculate the pixel value f(p,q) of the points in the eroded iris occluded area, where (p,q) represents the pixel points in the eroded iris occluded area, (p v , q v ) represents the pixel points in the known iris area adjacent to (p,q), 1≤v≤u, u is a positive integer not less than 2, assign the pixel value f(p,q) to the pixel point (p,q), and add the pixel point (p,q) to the known iris area.

[0075] In Step 502, for each round of eroded unknown iris pixel points, take the average of the accumulated pixel values of its adjacent known iris pixel points as its own pixel value and set its status to known iris. After several rounds of filling, all unknown pixels are filled with pixel values.

[0076] Step 504, Repeat Step 502 and Step 503 until the iris occluded area is completely eroded and filled.

[0077] Filling the areas with missing information in the amplified image can well restore the structural features of the image, make the repair result more natural, improve the quality of the enhanced amplified image, enhance the recognition efficiency of the periorbital recognition model, and improve the reliability of the periorbital recognition model.

[0078] In a possible embodiment, the erosion adopts horizontal erosion or vertical erosion.

[0079] Step 6: Add the enhanced amplified image to the periorbital sample library to participate in the training of the periorbital recognition model.

[0080] The enhanced amplified images generated by the method of the present application can be used as a special periorbital training sample, which together with the sample images pre-stored in the periorbital sample library constitutes a training set, thus enriching the diversity of periorbital samples and training the periorbital recognition model. Through actual tests, the present invention can effectively eliminate the periorbital recognition errors caused by the inconsistent pupil and iris sizes of periorbital images collected by the same person under different scene conditions, and improve the effect of deep learning for periorbital recognition.

[0081] ROC curve comparison is a common way to compare the performance of recognition models. The higher the ROC performance curve, the better the performance of the recognition model.

[0082] As Figure 4 shown, Curve 1 is the ROC performance curve of the trained periorbital recognition model obtained by adding enhanced amplified images to participate in training. Curve 2 is the ROC performance curve of the trained periorbital recognition model obtained by only using sample images for training.

[0083] All the ROC curves of Curve 1 are ahead of those of Curve 2. That is to say, the method of the present invention for normalizing and enhancing sample images and jointly training with the original sample images has significantly better effects than only using the original sample images for training.

[0084] Before the periorbital image detection by the periorbital recognition model composed of a deep neural network, the periorbital image is first processed by the amplification method of the present application to eliminate the interference caused by the difference in iris and pupil sizes, improve the recognition efficiency of the periorbital recognition model, and enhance the reliability of the periorbital recognition model.

[0085] Embodiment 2

[0086] This embodiment provides an application of a periorbital sample amplification method. The sample images in the periorbital sample library are amplified to obtain enhanced amplified images, the enhanced amplified images are added to the periorbital sample library, and based on the samples in the periorbital sample library, a periorbital recognition model composed of a deep neural network is trained to obtain a trained periorbital recognition model, and the trained periorbital recognition model is used to classify the periorbital images collected in the application scenario for identity recognition.

[0087] Specifically, first, a plurality of sample images are obtained to establish a periorbital sample library. The sample images include pictures and labels corresponding to each picture; in a possible implementation manner, the label includes identity information, and the identity information includes but is not limited to any one of employee number, name, and ID (Identity document, identity identification number).

[0088] Then, the sample images in the periorbital sample library are amplified to obtain enhanced amplified images, and the enhanced amplified images are added to the periorbital sample library.

[0089] Input the samples in the periorbital sample library into the periorbital recognition model to be trained, and obtain the predicted labels corresponding to each sample. Among them, the periorbital recognition model to be trained includes: a periorbital feature extraction network to be trained, a fully connected layer to be trained, and a classifier to be trained. Among them, the periorbital feature extraction network to be trained is used to extract the periorbital features of the sample pictures. Calculate the loss value between the predicted label corresponding to the sample and its own label by using a preset loss function. Iteratively train the periorbital recognition model to be trained according to the loss value until the loss value is less than the preset loss threshold, and obtain the finally trained periorbital recognition model. Among them, the finally trained periorbital recognition model includes: a periorbital feature extraction network, a fully connected layer, and a classifier.

[0090] Collect the periorbital images of the application scenario, and input the periorbital images into the trained periorbital recognition model. The trained periorbital recognition model outputs the predicted labels corresponding to the periorbital images for identity recognition.

[0091] The application embodiment does not limit the acquisition scenario of the periorbital images, which can be scenarios where periorbital features can be extracted, such as at the entrance of a community, at the entrance of a laboratory, at a bank window, at the entrance of a factory, etc. For example, when the application scenario is at the entrance of a community, a camera device is placed at the position of the entrance of the community in advance, and the camera device can at least capture the periorbital images of people.

[0092] The amplification of the sample images in this application mainly adjusts the sizes of the iris region and the pupil region to achieve regional normalization. The features of the iris region and the pupil region can be used as the basis for personnel identity recognition. The features of the iris region and the pupil region can be obtained by any one of computer vision, deep learning algorithms, and manual annotation. The application embodiment does not limit the method of using the iris region and the pupil region for identity recognition.

[0093] Preferably, the method of using periorbital features for identity recognition can be: determining the identity information corresponding to the iris region features from the corresponding relationship according to the iris region features, and / or determining the identity information corresponding to the pupil region features from the corresponding relationship according to the pupil region features.

[0094] Among them, the corresponding relationship is the corresponding relationship between the iris region features and the identity information corresponding to the iris region features, and / or the corresponding relationship between the pupil region features and the identity information corresponding to the pupil region features.

[0095] Specifically, the corresponding relationship between the iris region features and the identity information, and / or the corresponding relationship between the pupil region features and the identity information are pre-stored in the electronic device. After the iris region features and / or the pupil region features are extracted, based on the iris region features and / or the pupil region features through the corresponding relationship, determine the identity information corresponding to the iris region features and / or the pupil region features.

[0096] Embodiment 2 of this application is executed by an electronic device, which can be a server or a terminal device. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a mobile terminal such as a mobile phone, a laptop, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Computer), a PMP (Portable Multimedia Player), a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, and this application embodiment does not make any restrictions here.

[0097] As described above, the above are only embodiments of the present invention and do not impose any limitations on the present invention. Any simple modifications, changes, and equivalent structural changes made to the above embodiments according to the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for amplifying periorbital samples, characterized in that: The following steps are included: Step 1: extracting a pre-stored sample image from the eye periphery sample library, and dividing the sample image into an eye periphery area, an iris area, and a pupil area; Step 2, normalization of the augmented image area: Calculate the coordinate mapping relationship between the augmented image and the sample image, so that the radius ratio of the iris area and the pupil area in the augmented image is m / 1, obtain the coordinates of the iris area and the pupil area in the augmented image, the coordinates of the iris area and the pupil area constitute a coordinate point set (X, Y), and m is a positive integer; Step 3: Perform interpolation operation on the sample image to calculate the pixel value of each pixel of the augmented image: where (X t , Y t ) represents the t-th coordinate in the coordinate point set (X, Y) in the amplified image, f(X t , Y t ) represents the pixel value of the coordinate (X t , Y t ) in the amplified image, (X t , Y t ) mapped to the floating-point coordinates in the sample image coordinate system is denoted as (x′ t , y′ t ), (x αt , y αt ) represents the α-th integer coordinate point closest to the floating-point coordinates (x′ t , y′ t ) in the sample image coordinate system, ω a represents the weight, α = 1, 2, 3, 4, represents the square of the distance between the α-th integer coordinate point (x αt , y αt ) and the floating-point coordinates (x′ t , y′ t ), represents a function related to ; Step 4: Determine whether there is an information missing area in the iris area. If not, proceed to step 6. If so, proceed to step 5. Step 5, filling the information missing area of ​​the augmented image: filling the information missing area of ​​the iris area in the augmented image to obtain an enhanced augmented image; Step 6: Add the enhanced amplified image to the periocular sample library to participate in the training of the periocular recognition model; In step 2, (X, Y) = {(X 0i , Y 0i ), (X 1j , Y 1j )}, where (x 0i , y 0i ) represents the i-th coordinate of the pupil region in the sample image, (X 0i , Y 0i ) represents the coordinate corresponding to (x 0i , y 0i ) in the pupil region of the amplified image, (x 1j , y 1j ) represents the j-th coordinate of the iris region in the sample image, (X 1j , Y 1j ) represents the coordinate corresponding to (x 1j , y 1j ) in the iris region of the amplified image, R represents the iris radius of the sample image, r1 represents the pupil radius of the sample image, r3 represents the preset pupil radius, and d represents the distance between (X 1j , Y 1j ) and the center of the pupil region on the amplified image; D represents the distance between (x 1j , y 1j ) on the sample image and the center of the pupil region; In step 5, Step 501, dividing the augmented image into an eye periphery area, an iris occlusion area and an iris known area; Step 502, corroding the iris occlusion area: using a morphological algorithm, corroding the iris occlusion area from the first edge to the second edge, corroding z units in each round, and the pixels of the corroded iris occlusion area are assigned black; Step 503, filling the iris occlusion area: According to the formula calculate the pixel value f(p, q) of the points in the eroded iris occlusion area, where (p, q) represents the pixel points in the eroded iris occlusion area, and (p v , q v ) represents the pixel points located in the known iris area and adjacent to (p, q), 1 ≤ v ≤ u, and u is a positive integer not less than 2. Assign the pixel value f(p, q) to the pixel point (p, q), and add the pixel point (p, q) to the known iris area; Step 504: Repeat steps 502 and 503 until the iris occlusion area is completely eroded and filled.

2. The periorbital sample amplification method according to claim 1, wherein: The etching is performed by horizontal etching or vertical etching.

3. The periorbital sample amplification method according to claim 1, wherein: In step 3, where d1 represents the distance between (x′ t , y′ t ) and (x 1t , y 1t ), d2 represents the distance between (x′ t , y′ t ) and (x 2t , y 2t ), d3 represents the distance between (x′ t , y′ t ) and (x 3t , y 3t ), d4 represents the distance between (x′ t , y′ t ) and (x 4t , y 4t ).

4. The periorbital sample amplification method according to claim 1, wherein: The sample image has eyelid marking coordinates, iris marking coordinates and pupil marking coordinates, and an eyelid curve is fitted according to the eyelid marking coordinates; Fit the pupil curve according to the pupil marking coordinates; Fitting the iris curve according to the iris marker coordinates; The sample image is divided into the eye area, pupil area and iris area according to the eyelid curve, pupil curve and iris curve.

5. Use of the periorbital sample amplification method according to claim 1, characterized in that The sample images in the periocular sample library are amplified to obtain enhanced amplified images, and the enhanced amplified images are added to the periocular sample library. According to the samples in the periocular sample library, a periocular recognition model based on a deep neural network is trained to obtain a trained periocular recognition model. The trained periocular recognition model is used to classify the collected periocular images for identity recognition.

Images