Iris occlusion analysis method and device, computer equipment and storage medium

Abstract

The invention relates to an iris occlusion analysis method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: performing key point prediction on an eye image of a target object based on a key point prediction algorithm to obtain a plurality of iris boundary key points and a plurality of eyelid boundary key points; performing contour fitting on each iris boundary key point according to an iris contour shape condition to obtain a predicted iris region; determining an eyelid boundary formed by the eyelid boundary key points; and carrying out iris shielding analysis on the target object based on a relative position relationship between the predicted iris region and a region formed by the eyelid boundary to obtain an iris shielding analysis result. By adopting the method, the iris occlusion analysis result can be quickly and accurately obtained.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an iris occlusion analysis method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] The iris is the annular area between the black pupil and the white sclera on the surface of the human eye. Each iris contains unique characteristics. The characteristic information of the iris in the eye can be used to achieve the purpose of identity authentication, which is suitable for identity authentication technology in information security.

[0003] However, in actual application, in iris recognition scenarios, due to differences in user eye size, there will be different degrees of eyelid occlusion, etc., and it is necessary to judge the occlusion of the iris by the eyelid to ensure the iris recognition effect. The existing algorithm for judging iris occlusion is mainly through semantic segmentation of the eyelid and iris, and then judging iris occlusion. However, due to the presence of eyelid shadows, eyelash interference, etc., the semantic segmentation accuracy is low and the processing speed is slow. Summary of the invention

[0004] Based on this, it is necessary to provide an iris occlusion analysis method, apparatus, computer device, computer-readable storage medium and computer program product that can ensure the accuracy and efficiency of iris occlusion analysis results in response to the above technical problems.

[0005] In a first aspect, the present application provides an iris occlusion analysis method. The method comprises:

[0006] Based on the key point prediction algorithm, key points of the eye image of the target object are predicted to obtain multiple iris boundary key points and multiple eyelid boundary key points;

[0007] According to the iris contour shape condition, contour fitting is performed on each of the iris boundary key points to obtain a predicted iris area;

[0008] Determine the eyelid boundary formed by each of the eyelid boundary key points;

[0009] Based on the relative positional relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result.

[0010] In a second aspect, the present application further provides an iris occlusion analysis device. The device comprises:

[0011] A key point prediction module is used to predict key points of the eye image of the target object based on a key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points;

[0012] A contour fitting module, used to perform contour fitting on each of the iris boundary key points according to the iris contour shape conditions to obtain a predicted iris area;

[0013] A boundary determination module, used for determining the eyelid boundary formed by each of the eyelid boundary key points;

[0014] The occlusion analysis module is used to perform iris occlusion analysis on the target object based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary to obtain an iris occlusion analysis result.

[0015] In some embodiments, the device further comprises:

[0016] An image acquisition module, used to acquire an eye image including the eye of the target object collected according to a preset size;

[0017] The key point prediction module is also used to perform preliminary key point recognition on the eye image of the target object based on the key point prediction algorithm to determine the initial key points; crop the eye image according to the distribution of the initial key points in the eye image to obtain a target image in which the key point distribution meets the distribution conditions; and perform key point prediction on the target image based on the key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points.

[0018] In some embodiments, the key point prediction process is implemented by a key point prediction model; the training process of the key point prediction model includes:

[0019] Sample images of eyes in different occlusion states are obtained, wherein the sample images are marked with iris boundary key points and eyelid boundary key points, and the distribution of the marked iris boundary key points in the sample images meets the iris contour shape conditions; the marked eyelid boundary key points are used to characterize the eyelid boundary in the sample images; an initial deep neural network model is trained based on the sample images until the model training stop conditions are met, so as to obtain a key point prediction model for key point prediction of eye images.

[0020] In some embodiments, the eyelid boundary includes an upper eyelid boundary and a lower eyelid boundary;

[0021] The eyelid boundary key points include the eye corner key points located at the intersection of the upper eyelid boundary and the lower eyelid boundary, the upper eyelid key points located on the upper eyelid boundary, and the lower eyelid key points located on the lower eyelid boundary; the number of the upper eyelid key points is greater than the number of the lower eyelid key points.

[0022] In some of the embodiments, the boundary determination module is also used to identify the eye corner key point, the upper eyelid key point and the lower eyelid key point from each of the eyelid boundary key points; using the eye corner key point as the endpoint, connecting each of the upper eyelid key points through a first line to obtain the upper eyelid boundary; using the eye corner key point as the endpoint, connecting each of the lower eyelid key points through a second line to obtain the lower eyelid boundary; and determining the eyelid boundary based on the upper eyelid boundary and the lower eyelid boundary.

[0023] In some of the embodiments, the key point prediction module is further used to perform key point prediction on the eye image of the target object based on the key point prediction algorithm to obtain a plurality of pupil boundary key points, a plurality of iris outer boundary key points and a plurality of eyelid boundary key points;

[0024] The contour fitting module is also used to perform iris outer contour fitting on each of the iris outer boundary key points based on the distribution of each of the iris outer boundary key points and according to the iris outer contour shape condition to obtain a predicted iris outer boundary; perform pupil contour fitting on each of the pupil boundary key points based on the distribution of each of the pupil boundary key points and according to the pupil contour shape condition to obtain a predicted pupil boundary; and determine a predicted iris area according to the predicted iris outer boundary and the predicted pupil boundary.

[0025] In some embodiments, the iris outer contour shape condition is an elliptical contour; the pupil contour shape condition is a circular contour;

[0026] The contour fitting module is further used to respectively determine the elliptical area formed by the outer boundary of the predicted iris and the circular area formed by the outer boundary of the predicted iris; and determine the area in the elliptical area that does not overlap with the circular area as the predicted iris area.

[0027] In some of the embodiments, the occlusion analysis module is further used to screen out target pixel points within the area formed by the eyelid boundary for each pixel point within the predicted iris area according to the coordinates of each pixel point; determine the iris occlusion ratio of the target object based on the ratio of the number of target pixel points to the total number of pixel points within the predicted iris area; perform iris occlusion analysis based on the iris occlusion ratio and a maximum tolerable iris occlusion ratio threshold to obtain an iris occlusion analysis result.

[0028] In some of the embodiments, the device also includes an iris recognition module, which is used to extract iris features in the iris area when the iris occlusion analysis result is that the iris occlusion ratio is less than or equal to a maximum tolerable iris occlusion ratio threshold; based on the iris features, perform iris recognition processing on the target object to obtain a recognition result.

[0029] In some of the embodiments, the device further comprises a message generating module, configured to generate a reminder message for the target object when the iris occlusion analysis result indicates that the iris occlusion ratio is greater than a maximum tolerable iris occlusion ratio threshold.

[0030] In some of the embodiments, the message generation module is further used to determine the type of reminder message based on the current scene of the target object when the iris occlusion analysis result is that the iris occlusion ratio is greater than the maximum tolerable iris occlusion ratio threshold; and generate a reminder message for the target object based on the reminder message type.

[0031] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0032] Based on the key point prediction algorithm, key points of the eye image of the target object are predicted to obtain multiple iris boundary key points and multiple eyelid boundary key points;

[0033] According to the iris contour shape condition, contour fitting is performed on each of the iris boundary key points to obtain a predicted iris area;

[0034] Determine the eyelid boundary formed by each of the eyelid boundary key points;

[0035] Based on the relative positional relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Based on the key point prediction algorithm, key points of the eye image of the target object are predicted to obtain multiple iris boundary key points and multiple eyelid boundary key points;

[0038] According to the iris contour shape condition, contour fitting is performed on each of the iris boundary key points to obtain a predicted iris area;

[0039] Determine the eyelid boundary formed by each of the eyelid boundary key points;

[0040] Based on the relative positional relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result.

[0041] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0042] Based on the key point prediction algorithm, key points of the eye image of the target object are predicted to obtain multiple iris boundary key points and multiple eyelid boundary key points;

[0043] According to the iris contour shape condition, contour fitting is performed on each of the iris boundary key points to obtain a predicted iris area;

[0044] Determine the eyelid boundary formed by each of the eyelid boundary key points;

[0045] Based on the relative positional relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result.

[0046] The above-mentioned iris occlusion analysis method, device, computer equipment, storage medium and computer program product can perform key point prediction on the eye image of the target object through the key point prediction algorithm, and can directly predict multiple iris boundary key points and multiple eyelid boundary key points, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each iris boundary key point according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A diagram showing an application environment of an iris occlusion analysis method in an embodiment;

[0048] Figure 2 is a schematic diagram of a flow chart of an iris occlusion analysis method in one embodiment;

[0049] Figure 3 is a schematic diagram of various key points in an iris occlusion analysis method in one embodiment;

[0050] Figure 4 A schematic diagram of a boundary determined based on key points in one embodiment;

[0051] Figure 5 is a schematic diagram of an embodiment in which the iris is not blocked by the eyelid;

[0052] Figure 6 A schematic diagram of an embodiment in which the iris is partially blocked by the eyelid;

[0053] Figure 7 A schematic diagram of the structure of a key point prediction model in one embodiment;

[0054] Figure 8 A schematic diagram of a regression process of a key point prediction model in one embodiment;

[0055] Fig. 9 is a schematic diagram of a flow chart of an iris occlusion analysis method in one embodiment;

[0056] Fig.10 is a structural block diagram of an iris occlusion analysis device in one embodiment;

[0057] Fig.11 is an internal structure diagram of a computer device in one embodiment;

[0058] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0061] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0062] Computer Vision Technology (CV) Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0063] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0064] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0065] The solution provided in the embodiment of the present application involves technologies such as artificial intelligence computer vision, which locates the eye key points of the target object in the image, determines the predicted iris area and eyelid boundary in the eye image, so as to perform iris occlusion analysis on the target object, obtain the iris occlusion analysis result, and determine whether iris recognition can be performed or whether the target object needs to be reminded to open his eyes and re-capture and analyze the image.

[0066] The iris occlusion analysis method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The server 104 performs key point prediction on the eye image of the target object based on the key point prediction algorithm, obtains multiple iris boundary key points and multiple eyelid boundary key points, and performs contour fitting on each iris boundary key point according to the iris contour shape condition to obtain a predicted iris area, determines the eyelid boundary formed by each eyelid boundary key point, and performs iris occlusion analysis on the target object based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary to obtain an iris occlusion analysis result.

[0067] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers. The client of the target application can be installed in the terminal 102. The target application can be any application that can provide image processing functions. Typically, the application is an image processing application. This type of application provides the function of analyzing the content of the input image. Of course, in addition to image processing applications, image processing services can also be provided in other types of applications, such as news applications, shopping applications, social applications, interactive entertainment applications, browser applications, content sharing applications, virtual reality (VR) applications, augmented reality (AR) applications, etc., which are not limited in the embodiments of the present application. In addition, different applications may process different types of images and have different corresponding functions, which can be pre-configured according to actual needs, and the present application embodiment does not limit this. Optionally, the terminal device 102 runs a client of the above application.

[0068] In one example, the key point prediction process is performed in the key point prediction model. The key point prediction model runs on a computer device, that is, the method provided by the present application, and the execution subject of each step can be a computer device, and the computer device can be any electronic device with data storage and processing capabilities. For example, the computer device can be Figure 1 The server 104 in may be Figure 1 The terminal device 102 in the example may also be another device other than the terminal 102 and the server 104.

[0069] As society's requirements for privacy continue to increase, iris recognition has a wider application prospect in practical application scenarios such as payment and identity authentication. In one embodiment, taking the terminal as a VR device as an example, the target object wears the VR device, and the VR device can collect the eye image of the target object, upload the eye image to the server, and the server predicts the key points of the eye image of the target object based on the key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points. According to the iris contour shape condition, the contour of each iris boundary key point is fitted to obtain a predicted iris area, and the eyelid boundary formed by each eyelid boundary key point is determined. Based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary, the target object is subjected to iris occlusion analysis to obtain an iris occlusion analysis result. When the iris occlusion analysis result is that the occlusion ratio exceeds the maximum tolerable iris occlusion ratio threshold, the iris occlusion analysis result is sent to the VR device, so that the VR device sends a prompt message to the target object to open his eyes so as to re-collect the eye image for analysis. When the iris occlusion analysis result shows that the occlusion ratio is less than or equal to the maximum tolerable iris occlusion ratio threshold, the server extracts iris features in the iris area, performs iris recognition processing on the target object based on the iris features, and obtains an iris recognition result.

[0070] In another embodiment, the iris occlusion analysis method can also be applied to traffic scenes, educational scenes, etc. For example, in the process of driving in a traffic scene, the driver is subjected to iris occlusion analysis by the above iris occlusion analysis method, and the result of the iris occlusion analysis is used to determine whether the driver is driving fatigued, and if the driver is found to be driving fatigued, a prompt message is sent to the driver in a timely manner. For another example, in the process of class in an educational scene, the student is subjected to iris occlusion analysis by the above iris occlusion analysis method, and the result of the iris occlusion analysis is used to determine whether the student is dozing off in class, and if the student is found to be dozing off, a prompt message is sent to the student in a timely manner.

[0071] In some embodiments, the VR device is integrated with an algorithm for performing iris occlusion analysis and has the computing power for performing iris occlusion analysis. After acquiring the eye image of the target object, the VR device performs key point prediction on the eye image of the target object based on the key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points. According to the iris contour shape condition, each iris boundary key point is contour fitted to obtain a predicted iris area, and the eyelid boundary formed by each eyelid boundary key point is determined. Based on the relative positional relationship between the predicted iris area and the area formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result. When the iris occlusion analysis result is that the occlusion ratio exceeds the maximum tolerable iris occlusion ratio threshold, a prompt message to open the eyes is sent to the target object so as to re-collect the eye image for analysis. When the iris occlusion analysis result is that the occlusion ratio is less than or equal to the maximum tolerable iris occlusion ratio threshold, the server extracts the iris features in the iris area, and performs iris recognition processing on the target object based on the iris features to obtain an iris recognition result.

[0072] In one embodiment, Figure 2 As shown, an iris occlusion analysis method is provided, and the method is applied to a computer device as an example for explanation. The computer device may be Figure 1 The server in the embodiment may also be a terminal with data processing capability, or may be implemented through interaction between the server and the terminal. The method comprises the following steps:

[0073] Step 202: perform key point prediction on the eye image of the target object based on a key point prediction algorithm to obtain a plurality of iris boundary key points and a plurality of eyelid boundary key points.

[0074] Among them, the key point prediction algorithm is an algorithm for predicting key points of an image. Key point prediction can be performed using a deep network prediction, which can be to extract features using CNN, and then use a fully connected layer to directly numerically regress the coordinates of the key points, for example, the DeepPose human pose estimation network (Human Pose Estimation via Deep Neural Networks) or the MTCNN network (Multi-task Cascaded Convolutional Networks), or it can be a heat map prediction method, extracting the point in the channel where the pixel response is greater than a certain threshold and the response is the largest, and the coordinates of the point are the coordinates of the key point. In one embodiment, taking the key point prediction algorithm as the regression-based pose estimation algorithm DeepPose as an example, through the DeepPose algorithm, key point prediction can be performed on the eye image of the target object to obtain multiple iris boundary key points and multiple eyelid boundary key points.

[0075] The prediction process of the iris boundary key points and the eyelid boundary key points can be implemented by a neural network model that can implement a key point prediction algorithm. The neural network model can be trained based on sample eye images with key point annotations. The types of key points that can be predicted by the key point prediction algorithm can be determined based on the categories corresponding to the samples with key point annotations.

[0076] Iris boundary key points refer to the key points used to characterize the boundary of the area where the iris is located in the eye. The iris is the annular area between the black pupil and the white sclera on the surface of the human eye. Specifically, the iris boundary key points include the key points of the iris outer boundary and the key points of the pupil boundary. The iris outer boundary is the boundary between the iris and the white sclera, and the pupil boundary is the boundary between the iris and the black pupil.

[0077] Eyelid boundary key points refer to key points used to characterize eyelid boundary of an eye. Eyelid of an eye includes upper eyelid and lower eyelid, and eyelid boundary key points may include upper eyelid boundary key points and lower eyelid boundary key points. The number of upper eyelid boundary key points and lower eyelid boundary key points may be the same or different.

[0078] In the process of predicting key points of an eye image, the number of iris boundary key points and eyelid boundary key points that need to be predicted is multiple. Specifically, the number of iris boundary key points and eyelid boundary key points can be preset. For example, the number of iris boundary key points can be set to X, and the number of eyelid boundary key points can be set to Y.

[0079] In a specific application, since the pupil itself is a circular area, and in the iris recognition scenario, the pupil will be guided to become smaller through environmental adjustment, and the imaging distortion is not large at a small angle, the number of key points of the pupil boundary can be more than 3, for example, the number of key points of the pupil boundary is set to 4. The iris area is larger than the pupil, and the iris area is more susceptible to occlusion by the eyelids. The number of key points of the outer boundary of the iris can be greater than the number of key points of the pupil boundary. For example, the number of key points of the outer boundary of the iris can be set to 8. Since the upper part of the eyelid boundary has a larger curve than the lower part, the number of key points of the upper eyelid boundary can be greater than the number of key points of the lower eyelid boundary.

[0080] In an application embodiment, if Figure 3 As shown, the number of pupil boundary key points is 4, the number of iris outer boundary key points is 8, and the number of eyelid boundary key points is 6, wherein the eyelid boundary key points include 2 eye corner key points, 1 lower eyelid boundary key point and 3 upper eyelid boundary key points.

[0081] Step 204: according to the iris contour shape condition, contour fitting is performed on each iris boundary key point to obtain a predicted iris area.

[0082] The iris contour shape condition is a condition for defining the boundary contour of the fitted iris region. The iris contour shape condition may include an iris outer boundary shape condition and an iris inner boundary shape condition, and the iris inner boundary shape condition is the pupil boundary shape condition. The iris outer boundary shape condition and the pupil boundary shape condition may be defined as the same boundary shape or different boundary shapes.

[0083] Contour fitting refers to the process of fitting a contour boundary line that meets the iris contour shape conditions based on the distribution of iris boundary key points. Different contour fitting algorithms can be used to achieve different iris contour shape conditions. For example, a circular contour boundary line can be obtained by fitting a circular contour boundary line through a circular fitting algorithm, and an elliptical contour boundary line can be obtained by fitting an ellipse fitting algorithm.

[0084] The iris contour shape condition can be determined according to the actual eye image acquisition angle and acquisition scene. For eye images acquired from the front view, the iris contour shape condition can be set to a circular contour. For eye images acquired from multiple angles, the iris outer boundary shape condition in the iris contour shape condition can be set to an elliptical contour, and the pupil boundary shape condition in the iris contour shape condition can be set to a circular contour. The predicted iris area includes the iris area displayed in the eye image and the iris area blocked by the eyelid.

[0085] In one embodiment, after acquiring the eye image of the target object, the computer device performs key point prediction on the eye image based on a key point prediction algorithm, predicts multiple iris boundary key points, and marks or records the coordinate positions of the multiple iris boundary key points in the eye image. For the multiple iris boundary key points, the computer device first obtains pre-set iris contour shape conditions, performs contour fitting on each iris boundary key point according to the iris contour shape conditions, and determines the predicted iris area based on the fitting results.

[0086] Step 206: determine the eyelid boundary formed by each eyelid boundary key point.

[0087] The eyelid boundary key point refers to the key point in the eye image that represents the position of the eye edge. The eyelid boundary is determined by each eyelid boundary key point. The eyelid boundary can be obtained by curve fitting of each eyelid boundary key point, or by connecting the key points directly based on the coordinates of each eyelid boundary key point.

[0088] Specifically, the eyelid boundary key points can be divided into the eye corner key points and the eyelid middle key points. In the process of determining the eyelid boundary, the eye corner key points can be used as endpoints to fit the eyelid middle key points, so as to obtain the eyelid boundary. The eyelid middle key points can specifically include the upper eyelid boundary key points and the lower eyelid boundary key points. In the process of determining the eyelid boundary, the eye corner key points can be used as endpoints to fit the upper eyelid boundary key points and the lower eyelid boundary key points, respectively, so as to obtain the upper eyelid boundary and the lower eyelid boundary, and the eyelid boundary is formed based on the upper eyelid boundary and the lower eyelid boundary.

[0089] In one of the embodiments, after acquiring the eye image of the target object, the computer device predicts key points of the eye image based on a key point prediction algorithm. While predicting multiple iris boundary key points, the computer device can also predict multiple eyelid boundary key points, and mark the multiple eyelid boundary key points in the eye image or record their coordinate positions. For the multiple eyelid boundary key points, the computer device processes each eyelid boundary key point by connecting key points or fitting key points, and obtains the eyelid boundary formed by each eyelid boundary key point.

[0090] In an application embodiment, if Figure 4 As shown, the predicted key points include 4 pupil boundary key points, 8 iris outer boundary key points, and 6 eyelid boundary key points. The eyelid boundary key points include 2 eye corner key points, 1 lower eyelid boundary key point, and 3 upper eyelid boundary key points. The computer device fits a circular pupil boundary through the 4 pupil boundary key points, and fits an elliptical iris outer boundary through the 8 iris outer boundary key points. Based on the fitted iris outer boundary and the fitted pupil boundary, the predicted iris area is determined. Further, the computer device obtains the upper eyelid boundary by connecting the 2 eye corner key points and the 3 upper eyelid boundary key points, and obtains the lower eyelid boundary by connecting the 2 eye corner key points and the 1 lower eyelid boundary key point, and then determines the eyelid boundary based on the upper eyelid boundary and the lower eyelid boundary.

[0091] Step 208: Based on the relative position relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result.

[0092] The area formed by the eyelid boundary is the inner area surrounded by the eyelid boundary, and at least a part of it overlaps with the predicted iris area. Figure 5 As shown in , when the target object is collecting eye images, the eyes are in a completely open state, and the area formed by the eyelid boundary includes the predicted iris area. Figure 6As shown, when the target object is collecting eye images, the eyes are in a half-open state, and the area formed by the eyelid boundary includes a part of the predicted iris area, and the other part of the predicted iris area is covered by the eyelid.

[0093] Iris occlusion analysis refers to the analysis of the degree of occlusion of the iris by the eyelid. The degree of iris occlusion can be analyzed by setting an occlusion ratio threshold, or can be determined by whether the eyelid occludes the pupil area.

[0094] In some of the embodiments, the computer device can determine the occlusion ratio of the eyelid to the iris based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary, and then compare the occlusion ratio with a set occlusion ratio threshold to determine the degree of occlusion of the iris by the eyelid to obtain an iris occlusion analysis result.

[0095] In some other embodiments, the computer device can also determine whether the eyelid blocks the pupil based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary. If the eyelid blocks the pupil, it indicates that the degree of occlusion of the iris by the eyelid is high; if the eyelid does not block the pupil, it indicates that the degree of occlusion of the iris by the eyelid is low.

[0096] Furthermore, the iris occlusion analysis in the present application is to obtain the iris information of the target object for identity recognition, and the eye image of the target object includes the iris information. In order to protect the iris information of the target object and the identity recognition result obtained based on the iris information, data encryption can be used when transmitting data between the device that collects the eye image of the target object and the device that performs iris recognition.

[0097] Among them, when the iris occlusion analysis process is implemented by the server, the data sent by the terminal to the server is an encrypted eye image, wherein the encryption of the eye image can be implemented by image segmentation and reorganization, key encryption, etc. Image segmentation and reorganization refers to the process of segmenting the eye image into multiple image blocks according to certain rules and recombining them. In the process of transmitting the eye image, the segmented and reorganized image is transmitted. After the server obtains the segmented and reorganized image, the reverse process of image segmentation and reorganization can be performed based on the terminal to restore the multiple image blocks to the eye image, and then perform subsequent processing to avoid the iris information contained in the eye image of the target object from being leaked during the transmission process. Furthermore, after the server performs identity recognition on the iris information in the eye graphic, if the application scenario needs to feedback specific identity information to the terminal, in the process of transmitting the identity information, it can be encrypted by a key to avoid the leakage of the identity information corresponding to the eye image and the threat to the identity information security of the target object.

[0098] When the iris occlusion analysis process is completed by the terminal, the terminal can perform iris recognition when the iris occlusion analysis result represents that the eye image can be used for iris recognition, and the extracted iris information is sent to the server for identity recognition. The encryption method can be data block reorganization and key encryption.

[0099] The above-mentioned iris occlusion analysis method performs key point prediction on the eye image of the target object through the key point prediction algorithm, and can directly predict multiple iris boundary key points and multiple eyelid boundary key points, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each iris boundary key point according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained.

[0100] In one of the embodiments, the iris occlusion analysis method further includes: acquiring an eye image including an eye of the target object collected according to a preset size.

[0101] Furthermore, based on the key point prediction algorithm, key point prediction is performed on the eye image of the target object to obtain multiple iris boundary key points and multiple eyelid boundary key points, including:

[0102] Based on the key point prediction algorithm, the key points of the target object's eye image are preliminarily identified to determine the initial key points; according to the distribution of the initial key points in the eye image, the eye image is cropped to obtain a target image with key point distribution that meets the distribution conditions; based on the key point prediction algorithm, the key points of the target image are predicted to obtain multiple iris boundary key points and multiple eyelid boundary key points.

[0103] Among them, the preset size can be an acquisition parameter set by the image acquisition device. The same acquisition parameters are used for different objects in different scenes, and the acquired images with the same size are obtained. However, since the acquisition distance between the target object and the image acquisition device may change with the change of the acquisition scene, and different target objects may also have different acquisition distances between the target object and the image acquisition device due to their own individual differences. Therefore, when performing image processing, the key point prediction algorithm adopts the idea of ​​iteration to realize the prediction of key points. In the first stage of the key point prediction algorithm, after a series of convolution processing is performed on the image input of a specific size through the key point prediction algorithm, the final predicted key point coordinates (xi, yi) are obtained through two fully connected layers. Since the input target size is uncertain, the input image of a specific size will cause an overly large image to have errors in the final key point prediction due to scaling. In the second stage of the key point prediction algorithm, the idea of ​​the first stage is continued to be reused, and the area around the predicted key point position is cropped and enlarged to continue to make further more accurate predictions, thereby improving the final prediction accuracy.

[0104] Specifically, in the first stage, the key point prediction algorithm used is the DeepPose algorithm. Its core idea is to transform the key point detection algorithm into a purely mathematical prediction problem. By manually annotating a large number of sample images of eye key points in various states and learning the sample images through DNN (Deep Neural Networks), a more general end-to-end key point prediction can be achieved.

[0105] In a specific application, a computer device acquires an eye image acquired by an image acquisition device for a target object, and the eye image includes an eye region of the target object. The process of acquiring the eye image includes the computer device sending an acquisition instruction to the image acquisition device to acquire the eye image including the eye of the target object acquired by the image acquisition device according to a preset size.

[0106] In other embodiments, the eye image acquisition process may also be that the image acquisition device actively triggers the eye image acquisition of the target object when detecting that a specific condition is met, and sends the acquired eye image to the computer device. The image acquisition device may be a device that communicates with the computer device through a network, or may be a device that comes with the computer device.

[0107] In some embodiments, taking VR scenes as an example, the image acquisition device may be an infrared image acquisition device, which acquires infrared light imaging using an infrared sensor. Since the iris has a special absorption characteristic for infrared, and in a VR scene, the interference of VR color imaging is avoided, and the use of infrared images to acquire iris information has better anti-interference and recognition effects.

[0108] After acquiring an eye image of a preset size, the computer device first performs preliminary identification of key points of the eye image of the target object based on a key point prediction algorithm to determine the initial key points. Then, the computer device crops the eye image according to the distribution data of the initial key points in the eye image to obtain a target image in which the distribution of key points meets the preset distribution conditions. Among them, the distribution of key points meets the preset distribution conditions, which means that the proportion of the eye part in the eye image in the target image reaches a set ratio threshold. After obtaining the target image, the computer device can perform key point prediction on the target image based on the key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points.

[0109] Among them, Figure 7 As shown, according to the distribution of the initial key points in the eye image, the eye image is cropped to obtain a target image in which the key point distribution meets the distribution conditions. This effect can be achieved through multiple cropping, each cropping is performed according to the set ratio, and the key points of the cropped image are re-predicted until the key point distribution in the cropped image meets the distribution conditions. The image is used as the target image to predict the iris boundary key points and the eyelid boundary key points.

[0110] In this embodiment, by looping key point prediction and image cropping through a key point prediction algorithm, the key point prediction results can be optimized so that the predicted iris boundary key points and eyelid boundary key points have higher accuracy.

[0111] In some embodiments, the key point prediction process is implemented by a key point prediction model. The training process of the key point prediction model includes: obtaining sample images of the eyes in different occlusion states, the sample images are annotated with iris boundary key points and eyelid boundary key points, and the distribution of the annotated iris boundary key points in the sample images meets the iris contour shape conditions; the annotated eyelid boundary key points are used to characterize the eyelid boundary in the sample image; based on the sample images, the initial deep neural network model is trained until the model training stop condition is met, and a key point prediction model for key point prediction of the eye image is obtained.

[0112] Among them, the key point prediction model can be obtained by training the initial deep neural network model with sample images carrying key point annotations. The annotation method of the key points in the sample image will affect the prediction effect of the trained key point prediction model. Therefore, when annotating the sample image, it is necessary to ensure that the distribution of the annotated iris boundary key points in the sample image meets the iris contour shape conditions, and the annotated eyelid boundary key points are located on the eyelid boundary, so that the annotated eyelid boundary key points can accurately characterize the eyelid boundary in the sample image. Among them, the annotation method of the key points in the sample image can be implemented separately based on the type of key points. For example, the iris boundary key points can be selected on the iris contour with a specific shape in the sample image, and the eyelid boundary key points can be directly selected on the eyelid boundary in the sample image.

[0113] The initial deep neural network model can be a DNN network model, wherein the structure of the DNN network model is as follows Figure 8 As shown, it includes multiple convolutional layers and two fully connected layers. After the fully connected layer processes and obtains the predicted target points, the current image will be further cropped based on the predicted target points to obtain a new image and the DNN network model will re-predict the target points until the target image with key point distribution that meets the distribution conditions is obtained. Then, the DNN network model performs the last key point prediction process and outputs the prediction result. During the training process, the DNN network model uses supervised training based on annotated key points, and uses the key point prediction model obtained through training to perform key point prediction processing on the eye image of the target object.

[0114] In this embodiment, by using a deep neural network model as the model architecture and using sample images labeled with various key points as training samples, the trained key point prediction model can have accurate and efficient key point recognition and prediction capabilities, thereby improving the prediction efficiency of key points in eye images.

[0115] In some embodiments, the eyelid boundary includes an upper eyelid boundary and a lower eyelid boundary; the eyelid boundary key points include a corner of the eye key point located at the intersection of the upper eyelid boundary and the lower eyelid boundary, an upper eyelid key point located on the upper eyelid boundary, and a lower eyelid key point located on the lower eyelid boundary; the number of upper eyelid key points is greater than the number of lower eyelid key points.

[0116] The upper eyelid boundary refers to the boundary line representing the boundary position of the upper eyelid of the target object in the eye image, and the lower eyelid boundary refers to the boundary line representing the boundary position of the lower eyelid of the target object in the eye image. The intersection of the upper eyelid and the lower eyelid is the corner of the eye, and the key point marked at the corner of the eye is the corner of the eye key point. The key point marked on the upper eyelid boundary is the upper eyelid key point, and the key point marked on the lower eyelid boundary is the lower eyelid key point. Since the curvature of the upper eyelid boundary is generally greater than that of the lower eyelid boundary, when marking the eyelid boundary key points, the number of marked upper eyelid key points can be greater than the number of lower eyelid key points.

[0117] In this embodiment, by limiting the position and number of key points on the eyelid boundary and marking the key points on the eyelid boundary, the marked key points in the sample image can more accurately characterize the specific position of the eyelid, thereby enabling the key point prediction model trained based on the sample image marked with key points to more accurately predict the key points.

[0118] In some embodiments, after performing key point prediction on the eye image of the target object and obtaining the eyelid boundary key points, it is necessary to determine the eyelid boundary in the eye image based on the eyelid boundary key points. As described above, the eyelid boundary can be realized by curve fitting and key point connection. The following specifically describes the specific implementation method of determining the eyelid boundary by key point connection.

[0119] Specifically, the eyelid boundary formed by each eyelid boundary key point is determined, including:

[0120] From each eyelid boundary key point, identify the eye corner key point, upper eyelid key point and lower eyelid key point; with the eye corner key point as the endpoint, connect each upper eyelid key point through a first line to obtain the upper eyelid boundary; with the eye corner key point as the endpoint, connect each lower eyelid key point through a second line to obtain the lower eyelid boundary; based on the upper eyelid boundary and the lower eyelid boundary, determine the eyelid boundary.

[0121] Among them, the eyelid boundary key points can be divided into three categories according to the position relationship, namely, the eye corner key points, the upper eyelid key points and the lower eyelid key points. Among them, the eye corner key points represent the position of the eye corner in the eye image, and are the endpoints of the upper eyelid boundary and the lower eyelid boundary. Therefore, the computer device can use the eye corner key point as the endpoint, connect the upper eyelid key points through the first line to obtain the upper eyelid boundary; use the eye corner key point as the endpoint, connect the lower eyelid key points through the second line to obtain the lower eyelid boundary, so that the eyelid boundary can be determined based on the upper eyelid boundary and the lower eyelid boundary.

[0122] In other embodiments, the eyelid boundary formed by each eyelid boundary key point may be determined by sequentially connecting each eyelid boundary key point in a clockwise direction or a counterclockwise direction to form a convex polygon, and the convex polygon is the eyelid boundary.

[0123] In this embodiment, the eyelid boundary is determined by connecting key points, which is simple to process and can quickly determine the position of the eyelid boundary, thereby improving data processing speed.

[0124] In some embodiments, key point prediction is performed on the eye image of the target object based on a key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points, including: key point prediction is performed on the eye image of the target object based on a key point prediction algorithm to obtain multiple pupil boundary key points, multiple iris outer boundary key points and multiple eyelid boundary key points.

[0125] Furthermore, according to the iris contour shape condition, contour fitting is performed on each iris boundary key point to obtain a predicted iris area, including:

[0126] Based on the distribution of each iris outer boundary key point, according to the iris outer contour shape condition, the iris outer contour is fitted for each iris outer boundary key point to obtain the predicted iris outer boundary; based on the distribution of each pupil boundary key point, according to the pupil contour shape condition, the pupil contour is fitted for each pupil boundary key point to obtain the predicted pupil boundary; according to the predicted iris outer boundary and the predicted pupil boundary, the predicted iris area is determined.

[0127] The iris is the annular area between the black pupil and the white sclera on the surface of the human eye. According to their positional relationship, the iris boundary key points can be divided into two categories, namely, the pupil boundary key points and the iris outer boundary key points. The pupil boundary key points are the key points located on the boundary line between the pupil and the iris, and the iris outer boundary key points are the key points located on the boundary line between the iris and the sclera.

[0128] In the process of performing contour fitting on each iris boundary key point, the computer device can perform contour fitting on the pupil boundary key point and the iris outer boundary key point respectively. The contour fitting of the pupil boundary key point and the iris outer boundary key point can be performed synchronously or asynchronously, which can be determined according to the allocation of data processing resources.

[0129] Specifically, in the iris outer contour fitting process, the computer device performs iris outer contour fitting on each iris outer boundary key point based on the distribution of each iris outer boundary key point and according to the iris outer contour shape condition to obtain a predicted iris outer boundary. In the pupil contour fitting process, the computer device performs pupil contour fitting on each pupil boundary key point based on the distribution of each pupil boundary key point and according to the pupil contour shape condition to obtain a predicted pupil boundary. After obtaining the predicted iris outer boundary and the predicted pupil boundary, the computer device can determine the area between the predicted iris outer boundary and the predicted pupil boundary as the predicted iris area according to the positional relationship between the predicted iris outer boundary and the predicted pupil boundary.

[0130] In this embodiment, by performing contour fitting on the pupil boundary key points and the iris outer boundary key points respectively, the predicted iris outer boundary and the predicted pupil boundary are obtained, so that the iris boundary key points at different positions can be used to achieve different iris boundary predictions, thereby improving the position accuracy of the predicted iris area.

[0131] In some embodiments, the iris outer contour shape condition is an elliptical contour; the pupil contour shape condition is a circular contour. Further, according to the predicted iris outer boundary and the predicted pupil boundary, the predicted iris area is determined, including:

[0132] The elliptical area formed by the outer boundary of the predicted iris and the circular area formed by the outer boundary of the predicted iris are determined respectively; and the area in the elliptical area that does not overlap with the circular area is determined as the predicted iris area.

[0133] In the actual application process, in order to avoid blocking the sight line of the target object, the eye image of the target object can be collected from the side and other angles. In this case, due to angle distortion, the iris in the eye image of the target object appears elliptical. Therefore, the iris outer boundary shape condition in the iris contour shape condition can be set to an elliptical contour, and the pupil boundary shape condition in the iris contour shape condition can be set to a circular contour.

[0134] In the process of fitting the iris outer contour, the computer device fits the iris outer contour to each iris outer boundary key point according to the elliptical contour based on the distribution of each iris outer boundary key point, and obtains an elliptical predicted iris outer boundary. In the process of fitting the pupil contour, the computer device fits the pupil contour to each pupil boundary key point according to the circular contour based on the distribution of each pupil boundary key point, and obtains a circular predicted pupil boundary. After obtaining the elliptical area formed by the predicted iris outer boundary and the circular area formed by the predicted pupil boundary, the computer device can determine the area in the elliptical area that does not overlap with the circular area as the predicted iris area.

[0135] In this embodiment, by considering the influence of angle distortion on the outer boundary of the iris, the outer boundary shape condition of the iris in the iris contour shape condition can be set as an elliptical contour, which can better achieve contour fitting of key points of the outer boundary of the iris, and improve the accuracy of the description of the outer boundary of the pupil by the predicted outer boundary of the iris obtained by fitting. At the same time, considering that the pupil area is small and the influence of distortion is not great, in order to simplify the fitting process in the pupil fitting process, the pupil boundary shape condition in the iris contour shape condition can be set as a circular contour, which can effectively improve the fitting speed of the pupil boundary.

[0136] In some embodiments, based on the relative position relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result, including:

[0137] For each pixel in the predicted iris area, the target pixel points in the area formed by the eyelid boundary are screened out according to the coordinates of each pixel point; the iris occlusion ratio of the target object is determined based on the ratio of the number of target pixels to the total number of pixels in the predicted iris area; iris occlusion analysis is performed based on the iris occlusion ratio and the maximum tolerable iris occlusion ratio threshold to obtain the iris occlusion analysis result.

[0138] In the process of iris occlusion analysis, the computer device uses the target pixel ratio calculation method to ensure the accuracy of the iris occlusion analysis results. Specifically, the iris occlusion ratio of the target object is essentially the ratio of the target pixel points in the area formed by the eyelid boundary in the predicted iris area to the total pixel points in the predicted iris area. The larger the ratio of the number of target pixels to the total number of pixels in the predicted iris area, the less the iris is occluded, and the smaller the ratio of the number of target pixels to the total number of pixels in the predicted iris area, the more the iris is occluded.

[0139] The maximum tolerable iris occlusion ratio threshold refers to the threshold set because the degree of iris occlusion affects the subsequent processing process. For example, when the target object wearing the VR device is subjected to eye image acquisition and iris occlusion analysis to obtain the iris occlusion analysis result, when the iris occlusion analysis result is that the occlusion ratio exceeds the maximum tolerable iris occlusion ratio threshold, the iris occlusion analysis result is sent to the VR device so that the VR device sends a prompt message to the target object to open his eyes so as to re-acquire the eye image for analysis. For another example, when the iris occlusion analysis result is that the occlusion ratio is less than or equal to the maximum tolerable iris occlusion ratio threshold, the server extracts the iris features in the iris area, and based on the iris features, performs iris recognition processing on the target object to obtain the iris recognition result.

[0140] Further, in some embodiments, when the iris occlusion analysis result is that the iris occlusion ratio is less than or equal to the maximum tolerable iris occlusion ratio threshold, iris features in the iris area are extracted; based on the iris features, iris recognition processing is performed on the target object to obtain a recognition result. In other embodiments, when the iris occlusion analysis result is that the iris occlusion ratio is greater than the maximum tolerable iris occlusion ratio threshold, a reminder message is generated for the target object.

[0141] Specifically, when the iris occlusion analysis result is that the iris occlusion ratio is greater than the maximum tolerable iris occlusion ratio threshold, a reminder message for the target object is generated, including:

[0142] When the iris occlusion analysis result is that the iris occlusion ratio is greater than the maximum tolerable iris occlusion ratio threshold, the reminder message type is determined based on the current scene of the target object; and a reminder message for the target object is generated based on the reminder message type.

[0143] For example, the iris occlusion analysis method can be applied to traffic scenes, education scenes, etc. For example, during the driving process of the driver in the traffic scene, the iris occlusion analysis method is used to perform iris occlusion analysis on the driver, and the iris occlusion analysis result is used to determine whether the driver is driving fatigued, and if the driver is found to be driving fatigued, a prompt message is sent to the driver in time.

[0144] For another example, during a class in an educational scenario, the iris occlusion analysis method is used to perform iris occlusion analysis on students, and the results of the iris occlusion analysis are used to determine whether the students are dozing off in class. If the students are found to be dozing off, prompt messages are sent to the students in a timely manner.

[0145] In some embodiments, an iris occlusion analysis method is also provided. Fig. 9 As shown, the method includes:

[0146] Step 902: Acquire sample images of eyes in different occlusion states, wherein the sample images are annotated with iris outer boundary key points, pupil boundary key points, and eyelid boundary key points.

[0147] Step 904: Train the initial deep neural network model based on the sample image until the model training stop condition is met, and obtain a key point prediction model for key point prediction of the eye image.

[0148] The sample image is a sample data that carries eye key point annotations and is used for key point prediction model. The annotation method of the key points in the sample image will affect the prediction effect of the trained key point prediction model. Therefore, when annotating the sample image, it is necessary to ensure that the distribution of the annotated iris boundary key points in the sample image meets the iris contour shape conditions, and the annotated eyelid boundary key points are located on the eyelid boundary, so that the annotated eyelid boundary key points can accurately characterize the eyelid boundary in the sample image. The annotation method of the key points in the sample image can be implemented separately based on the type of key points. For example, the iris boundary key points can be selected on the iris contour with a specific shape in the sample image, and the eyelid boundary key points can be directly selected on the eyelid boundary in the sample image.

[0149] The initial deep neural network model may be a DNN network model, wherein the DNN network model includes multiple convolutional layers and two fully connected layers. After the fully connected layer processes and obtains the predicted target points, the current image is further cropped based on the predicted target points to obtain a new image and the DNN network model re-performs the target point prediction process until a target image with key point distribution that meets the distribution conditions is obtained, and then the DNN network model performs the last key point prediction process and outputs the prediction result. During the training process, the DNN network model uses supervised training based on annotated key points, and performs key point prediction processing on the eye image of the target object with the key point prediction model obtained through training.

[0150] Step 906: Acquire an eye image including the eyes of the target object collected according to a preset size.

[0151] The acquisition of the eye image includes the computer device sending an acquisition instruction to the image acquisition device, and acquiring the eye image containing the eye of the target object acquired by the image acquisition device according to the preset size. In other embodiments, the acquisition of the eye image can also be that the image acquisition device actively triggers the acquisition of the eye image for the target object when detecting that a specific condition is met, and sends the acquired eye image to the computer device. Among them, the image acquisition device can be a device that communicates with the computer device through a network, or it can be a device that comes with the computer device. In some embodiments, for example, in a VR scene, the image acquisition device can be an infrared image acquisition device, and the infrared image acquisition device uses an infrared sensor to collect infrared light imaging. Since the iris has a relatively special absorption characteristic for infrared, and at the same time avoids the interference of VR color imaging in the VR scene, the use of infrared images to collect iris information has better anti-interference and recognition effects.

[0152] Step 908 , performing preliminary key point recognition on the eye image of the target object based on the key point prediction algorithm used by the key point prediction model to determine initial key points.

[0153] Step 910, cropping the eye image according to the distribution of the initial key points in the eye image, to obtain a target image in which the distribution of key points meets the distribution conditions.

[0154] Step 912: perform key point prediction on the eye image of the target object based on a key point prediction algorithm to obtain a plurality of pupil boundary key points, a plurality of iris outer boundary key points, and a plurality of eyelid boundary key points.

[0155] According to the distribution of the initial key points in the eye image, the eye image is cropped to obtain a target image whose key point distribution meets the distribution conditions. This effect can be achieved through multiple cropping, each cropping is performed according to the set ratio, and the cropped image is re-predicted for key points until the key point distribution in the cropped image meets the distribution conditions. The image is used as the target image to predict the pupil boundary key points, the iris outer boundary key points, and the eyelid boundary key points. By looping the key point prediction and image cropping by the key point prediction algorithm, the key point prediction results can be optimized so that the predicted iris boundary key points and eyelid boundary key points have higher accuracy.

[0156] Step 914: Based on the distribution of each iris outer boundary key point, the iris outer boundary key point is fitted according to the ellipse contour to obtain a predicted iris outer boundary.

[0157] Step 916: Based on the distribution of each pupil boundary key point, pupil contour fitting is performed on each pupil boundary key point according to a circular contour to obtain a predicted pupil boundary.

[0158] Step 918, respectively determine the elliptical area formed by the outer boundary of the predicted iris and the circular area formed by the outer boundary of the predicted iris, and determine the area in the elliptical area that does not overlap with the circular area as the predicted iris area.

[0159] In actual application, in order to avoid blocking the sight line of the target object, the eye image of the target object can be collected from the side and other angles. In this case, due to angle distortion, the iris in the eye image of the target object appears elliptical. Therefore, the iris outer boundary shape condition in the iris contour shape condition can be set to an elliptical contour, and the pupil boundary shape condition in the iris contour shape condition can be set to a circular contour.

[0160] In the process of fitting the iris outer contour, the computer device fits the iris outer contour to each iris outer boundary key point according to the elliptical contour based on the distribution of each iris outer boundary key point, and obtains an elliptical predicted iris outer boundary. In the process of fitting the pupil contour, the computer device fits the pupil contour to each pupil boundary key point according to the circular contour based on the distribution of each pupil boundary key point, and obtains a circular predicted pupil boundary. After obtaining the elliptical area formed by the predicted iris outer boundary and the circular area formed by the predicted pupil boundary, the computer device can determine the area in the elliptical area that does not overlap with the circular area as the predicted iris area.

[0161] Step 920, identifying the eye corner key points, upper eyelid key points and lower eyelid key points from each eyelid boundary key points.

[0162] Step 922, with the eye corner key point as the endpoint, connect the upper eyelid key points through the first line to obtain the upper eyelid boundary, with the eye corner key point as the endpoint, connect the lower eyelid key points through the second line to obtain the lower eyelid boundary.

[0163] Step 924, determining the eyelid boundary based on the upper eyelid boundary and the lower eyelid boundary.

[0164] Among them, the eyelid boundary key points can be divided into three categories according to the position relationship, namely, the eye corner key points, the upper eyelid key points and the lower eyelid key points. Among them, the eye corner key points represent the position of the eye corner in the eye image, and are the endpoints of the upper eyelid boundary and the lower eyelid boundary. Therefore, the computer device can use the eye corner key points as the endpoints, connect the upper eyelid key points through the first line, and obtain the upper eyelid boundary; use the eye corner key points as the endpoints, connect the lower eyelid key points through the second line, and obtain the lower eyelid boundary, so that the eyelid boundary can be determined based on the upper eyelid boundary and the lower eyelid boundary. In other embodiments, the method of determining the eyelid boundary formed by the eyelid boundary key points can also be to connect the eyelid boundary key points in sequence in a clockwise direction or a counterclockwise direction into a convex polygon, and the convex polygon is the eyelid boundary.

[0165] Step 926: for each pixel point in the predicted iris area, according to the coordinates of each pixel point, select the target pixel point in the area formed by the eyelid boundary.

[0166] Step 928 , determining the iris occlusion ratio of the target object based on the ratio of the number of target pixels to the total number of pixels in the predicted iris area.

[0167] Step 930: Perform iris occlusion analysis based on the iris occlusion ratio and the maximum tolerable iris occlusion ratio threshold to obtain an iris occlusion analysis result.

[0168] Step 932: when the iris occlusion analysis result shows that the iris occlusion ratio is less than or equal to the maximum tolerable iris occlusion ratio threshold, extract the iris features in the iris area, and perform iris recognition processing on the target object based on the iris features to obtain a recognition result.

[0169] Step 934, when the iris occlusion analysis result is that the iris occlusion ratio is greater than the maximum tolerable iris occlusion ratio threshold, determine the reminder message type based on the current scene of the target object, and generate a reminder message for the target object based on the reminder message type.

[0170] In the process of iris occlusion analysis, the computer device uses the target pixel ratio calculation method to ensure the accuracy of the iris occlusion analysis results. Specifically, the iris occlusion ratio of the target object is essentially the ratio of the target pixel points in the area formed by the eyelid boundary in the predicted iris area to the total pixel points in the predicted iris area. The larger the ratio of the number of target pixels to the total number of pixels in the predicted iris area, the less the iris is occluded, and the smaller the ratio of the number of target pixels to the total number of pixels in the predicted iris area, the more the iris is occluded.

[0171] The maximum tolerable iris occlusion ratio threshold refers to the threshold set because the degree of iris occlusion affects the subsequent processing process. For example, when the target object wearing the VR device is subjected to eye image acquisition and iris occlusion analysis to obtain the iris occlusion analysis result, when the iris occlusion analysis result is that the occlusion ratio exceeds the maximum tolerable iris occlusion ratio threshold, the iris occlusion analysis result is sent to the VR device so that the VR device sends a prompt message to the target object to open his eyes so as to re-acquire the eye image for analysis. For another example, when the iris occlusion analysis result is that the occlusion ratio is less than or equal to the maximum tolerable iris occlusion ratio threshold, the server extracts the iris features in the iris area, performs iris recognition processing on the target object based on the iris features, and obtains the iris recognition result.

[0172] The above-mentioned iris occlusion analysis method, device, computer equipment, storage medium and computer program product can perform key point prediction on the eye image of the target object through the key point prediction algorithm, and can directly predict multiple iris boundary key points and multiple eyelid boundary key points, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each iris boundary key point according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained.

[0173] The present application also provides an application scenario of VR iris recognition, which applies the above-mentioned iris occlusion analysis method. Specifically, the application of the iris occlusion analysis method in this application scenario is as follows:

[0174] In the iris recognition scenario, due to the difference in the size of users' eyes, there will be different degrees of eyelid occlusion, etc. Based on this, relevant algorithms are needed to determine the occlusion of the iris by the eyelid, so as to ensure the iris recognition effect. The existing algorithm for judging iris occlusion is mainly to perform semantic segmentation on the eyelid and iris, and then judge the iris occlusion. Since this type of method is based on semantic segmentation, it has high requirements on the accuracy and speed of the segmentation algorithm. In the VR recognition scenario, the speed is poor and real-time processing cannot be achieved. At the same time, based on semantic segmentation, only the content area of ​​iris and eyelid can be obtained, and the proportion of iris occlusion cannot be determined. Due to the difference in camera imaging distance (near is large and far is small), although the size of the iris area is obtained, the proportion of occlusion cannot be evaluated, so it is impossible to determine whether the recognition algorithm supports it. At the same time, iris recognition in VR scenarios will lead to insufficient semantic segmentation accuracy due to the presence of eyelid shadows, eyelash interference, etc.

[0175] In this application, after the user wears the VR device, the iris recognition camera will continuously collect photos of the user's eyes, and use the algorithm to determine the user's iris occlusion and give corresponding prompts. If the occlusion ratio is large, the user will be reminded to open the glasses or adjust the display content to guide the user to open the glasses. Specifically, the computer device can predict the eyelid boundary key points and iris boundary key points through the DeepPose algorithm, and then determine the iris occlusion ratio based on the relationship between the eyelid key points and the pupil and iris positions.

[0176] Specifically, this plan consists of three parts:

[0177] 1. Key point calculation based on DeepPose's pupil boundary key points, iris outer boundary key points, and eyelid boundary key points

[0178] 2. Based on the pupil boundary key points, iris outer boundary key points, and eyelid boundary key points, determine the iris area formed by the pupil boundary and the iris outer boundary, and accurately calculate the iris occlusion ratio based on the eyelid boundary

[0179] 3. The optimal logic determines whether the occlusion ratio meets the requirements and gives a normal prompt.

[0180] Furthermore, the calculation of iris occlusion requires knowing the position information of the iris itself and the position information of the eyelid, so as to correctly calculate the occlusion ratio and evaluate whether the algorithm supports this degree of occlusion. Regarding the position information of the iris area, key points can be predicted through DeepPose, and the predicted iris area can be obtained based on the predicted key points.

[0181] The idea of ​​the DeepPose algorithm is to turn the key point detection algorithm into a purely mathematical prediction problem without considering the anthropomorphic problems under complex postures. By manually annotating a large amount of human eye key point data under various postures and learning the sample data through the DNN convolutional neural network, a more general end-to-end eye key point detection algorithm can be achieved.

[0182] Among them, in the first stage, the DNN convolutional neural network can perform a series of convolutions on the image input of a set size, and finally obtain the predicted key points (xi, yi) through two fully connected layers. At the same time, the optimization idea of ​​the algorithm is that since the input target size is uncertain, and the network itself only receives input images of fixed size, it will cause the overly large image to cause errors in the final target key point prediction due to scaling. Based on the second stage of this algorithm, the idea of ​​the first stage is continued to be reused, but the area around the estimated key point position is cropped and enlarged to continue further and more accurate predictions, thereby improving the final prediction accuracy. The DeepPose algorithm is a DNN network fitting of nonlinear regression problems, and the end-to-end return network prediction results.

[0183] In one of the embodiments. The eye image of the target object is predicted by DeepPose and returns 19 key points of the eye area, including 4 estimated key points for eyelid key points, 8 estimated key points for points around the iris, and 7 estimated key points for points around the pupil. Specifically, since the pupil itself is a circular area, and in the iris recognition scenario, the pupil will be guided to become smaller through environmental adjustment, and the imaging distortion is not large at small angles, only 4 key points are used for the pupil key points. The iris area is larger than the pupil, and the upper right is more easily blocked by the eyelid, so 8 key points are used for the iris area for alignment. Similarly, since the upper part of the eyelid area has a larger curve than the lower part, the upper part has more key points and 7 key points are used for alignment.

[0184] Based on the key points of the iris boundary, since the pupil area and the iris area are circular areas, they will become elliptical areas due to distortion such as angles. However, considering that the pupil area is small, the distortion has little effect. At the same time, too many registration points will lead to high errors. Therefore, for the pupil area, we use 4 points to determine a circle to mark the pupil area. For the iris edge, 8 points are used with the fitEllipse method of opencv to perform ellipse matching to obtain the location information of the iris area. For the eyelid boundary, due to its irregular shape, a simple line method is used to calibrate the eyelid boundary.

[0185] Specifically, the computer device fits a circular pupil boundary through 4 pupil boundary key points, and fits an elliptical iris boundary through 8 iris boundary key points. Based on the fitted iris boundary and the fitted pupil boundary, the predicted iris region is determined. Further, the computer device obtains an upper eyelid boundary by connecting 2 eye corner key points and 3 upper eyelid boundary key points, obtains a lower eyelid boundary by connecting 2 eye corner key points and 1 lower eyelid boundary key point, and then determines the eyelid boundary based on the upper eyelid boundary and the lower eyelid boundary.

[0186] The iris area can be determined by the known pupil boundary and iris outer boundary. The computer device can accurately determine the iris occlusion ratio by judging whether each pixel in the iris area is within the area formed by the upper eyelid boundary and the lower eyelid boundary.

[0187] The iris recognition algorithm makes a judgment based on the iris occlusion ratio. If the occlusion ratio is greater than the maximum ratio tolerated by the iris recognition algorithm, the user is reminded to open their eyes. Otherwise, if the iris recognition algorithm requirements are met, the next step of iris recognition and other processing is performed based on the iris recognition algorithm.

[0188] In this embodiment, since the implementation of this solution mainly relies on the DeepPose algorithm to estimate the key points of the iris, the algorithm speed is relatively fast. For the iris recognition scene under VR, it can give a correct iris occlusion judgment while ensuring the recognition speed, overcoming the problem that the traditional semantic segmentation-based algorithm is slow and cannot correctly calculate the occlusion ratio.

[0189] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0190] Based on the same inventive concept, the embodiment of the present application also provides an iris occlusion analysis device for implementing the iris occlusion analysis method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of one or more iris occlusion analysis device embodiments provided below can refer to the limitations of the iris occlusion analysis method above, and will not be repeated here.

[0191] In one embodiment, Fig.10 As shown, an iris occlusion analysis device is provided, comprising: a key point prediction module 1002, a contour fitting module 1004, a boundary determination module 1006 and an occlusion analysis module 1008, wherein:

[0192] A key point prediction module 1002 is used to perform key point prediction on the eye image of the target object based on a key point prediction algorithm to obtain a plurality of iris boundary key points and a plurality of eyelid boundary key points;

[0193] The contour fitting module 1004 is used to perform contour fitting on each of the iris boundary key points according to the iris contour shape condition to obtain a predicted iris area;

[0194] A boundary determination module 1006, used to determine the eyelid boundary formed by the eyelid boundary key points;

[0195] The occlusion analysis module 1008 is used to perform iris occlusion analysis on the target object based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary to obtain an iris occlusion analysis result.

[0196] In some embodiments, the device further comprises:

[0197] An image acquisition module, used to acquire an eye image including the eye of the target object collected according to a preset size;

[0198] The key point prediction module is also used to perform preliminary key point recognition on the eye image of the target object based on the key point prediction algorithm to determine the initial key points; crop the eye image according to the distribution of the initial key points in the eye image to obtain a target image in which the key point distribution meets the distribution conditions; and perform key point prediction on the target image based on the key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points.

[0199] In some embodiments, the key point prediction process is implemented by a key point prediction model; the training process of the key point prediction model includes:

[0200] Sample images of eyes in different occlusion states are obtained, wherein the sample images are marked with iris boundary key points and eyelid boundary key points, and the distribution of the marked iris boundary key points in the sample images meets the iris contour shape conditions; the marked eyelid boundary key points are used to characterize the eyelid boundary in the sample images; an initial deep neural network model is trained based on the sample images until the model training stop conditions are met, so as to obtain a key point prediction model for key point prediction of eye images.

[0201] In some embodiments, the eyelid boundary includes an upper eyelid boundary and a lower eyelid boundary; the eyelid boundary key points include a corner of the eye key point located at the intersection of the upper eyelid boundary and the lower eyelid boundary, an upper eyelid key point located on the upper eyelid boundary, and a lower eyelid key point located on the lower eyelid boundary; the number of the upper eyelid key points is greater than the number of the lower eyelid key points.

[0202] In some of the embodiments, the boundary determination module is also used to identify the eye corner key point, the upper eyelid key point and the lower eyelid key point from each of the eyelid boundary key points; using the eye corner key point as the endpoint, connecting each of the upper eyelid key points through a first line to obtain the upper eyelid boundary; using the eye corner key point as the endpoint, connecting each of the lower eyelid key points through a second line to obtain the lower eyelid boundary; and determining the eyelid boundary based on the upper eyelid boundary and the lower eyelid boundary.

[0203] In some of the embodiments, the key point prediction module is further used to perform key point prediction on the eye image of the target object based on the key point prediction algorithm to obtain a plurality of pupil boundary key points, a plurality of iris outer boundary key points and a plurality of eyelid boundary key points;

[0204] The contour fitting module is also used to perform iris outer contour fitting on each of the iris outer boundary key points based on the distribution of each of the iris outer boundary key points and according to the iris outer contour shape condition to obtain a predicted iris outer boundary; perform pupil contour fitting on each of the pupil boundary key points based on the distribution of each of the pupil boundary key points and according to the pupil contour shape condition to obtain a predicted pupil boundary; and determine a predicted iris area according to the predicted iris outer boundary and the predicted pupil boundary.

[0205] In some embodiments, the iris outer contour shape condition is an elliptical contour; the pupil contour shape condition is a circular contour;

[0206] The contour fitting module is further used to respectively determine the elliptical area formed by the outer boundary of the predicted iris and the circular area formed by the outer boundary of the predicted iris; and determine the area in the elliptical area that does not overlap with the circular area as the predicted iris area.

[0207] In some of the embodiments, the occlusion analysis module is further used to screen out target pixel points within the area formed by the eyelid boundary for each pixel point within the predicted iris area according to the coordinates of each pixel point; determine the iris occlusion ratio of the target object based on the ratio of the number of target pixel points to the total number of pixel points within the predicted iris area; perform iris occlusion analysis based on the iris occlusion ratio and a maximum tolerable iris occlusion ratio threshold to obtain an iris occlusion analysis result.

[0208] In some of the embodiments, the device also includes an iris recognition module, which is used to extract iris features in the iris area when the iris occlusion analysis result is that the iris occlusion ratio is less than or equal to a maximum tolerable iris occlusion ratio threshold; based on the iris features, perform iris recognition processing on the target object to obtain a recognition result.

[0209] In some of the embodiments, the device further comprises a message generating module, configured to generate a reminder message for the target object when the iris occlusion analysis result indicates that the iris occlusion ratio is greater than a maximum tolerable iris occlusion ratio threshold.

[0210] In some of the embodiments, the message generation module is further used to determine the type of reminder message based on the current scene of the target object when the iris occlusion analysis result is that the iris occlusion ratio is greater than the maximum tolerable iris occlusion ratio threshold; and generate a reminder message for the target object based on the reminder message type.

[0211] The above-mentioned iris occlusion analysis device performs key point prediction on the eye image of the target object through the key point prediction algorithm, and can directly predict multiple iris boundary key points and multiple eyelid boundary key points, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each of the iris boundary key points according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained.

[0212] Each module in the above iris occlusion analysis device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0213] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an iris occlusion analysis method is implemented.

[0214] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an iris occlusion analysis method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0215] Those skilled in the art will understand that Fig.11 and Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0216] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0217] The above-mentioned computer device performs key point prediction on the eye image of the target object through the key point prediction algorithm, and can directly predict multiple iris boundary key points and multiple eyelid boundary key points, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each of the iris boundary key points according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained.

[0218] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0219] The above-mentioned computer-readable storage medium can directly predict multiple iris boundary key points and multiple eyelid boundary key points by performing key point prediction on the eye image of the target object through the key point prediction algorithm, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each of the iris boundary key points according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained.

[0220] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0221] The above-mentioned computer program product performs key point prediction on the eye image of the target object through the key point prediction algorithm, and can directly predict multiple iris boundary key points and multiple eyelid boundary key points, and can quickly predict the key points in the eye image, and then can determine the predicted iris area and eyelid boundary based on the key points, wherein the predicted iris area is not only the iris displayed in the eye image, but is obtained by contour fitting of each of the iris boundary key points according to the iris contour shape condition, and can quickly and conveniently determine the predicted iris area, and the eyelid boundary is composed of eyelid boundary key points, which can accurately express the eyelid position in the eye image, so that based on the relative position relationship between the predicted iris area and the area composed of the eyelid boundary, the target object can be subjected to iris occlusion analysis, and the iris occlusion analysis result can be quickly and accurately obtained.

[0222] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0224] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An iris occlusion analysis method, characterized in that: The method comprises: Based on the key point prediction algorithm, key points of the eye image of the target object are predicted to obtain multiple iris boundary key points and multiple eyelid boundary key points; According to the iris contour shape condition, contour fitting is performed on each of the iris boundary key points to obtain a predicted iris area; Determine the eyelid boundary formed by each of the eyelid boundary key points; Based on the relative positional relationship between the predicted iris region and the region formed by the eyelid boundary, an iris occlusion analysis is performed on the target object to obtain an iris occlusion analysis result.

2. The method according to claim 1, characterized in that The method further comprises: Acquire an eye image including the eye of the target object collected according to a preset size; The method of performing key point prediction on the eye image of the target object based on the key point prediction algorithm to obtain a plurality of iris boundary key points and a plurality of eyelid boundary key points includes: Performing preliminary key point recognition on the eye image of the target object based on the key point prediction algorithm to determine initial key points; According to the distribution of the initial key points in the eye image, the eye image is cropped to obtain a target image in which the distribution of key points meets the distribution conditions; Key point prediction is performed on the target image based on the key point prediction algorithm to obtain a plurality of iris boundary key points and a plurality of eyelid boundary key points.

3. The method according to claim 1, characterized in that The key point prediction process is implemented by a key point prediction model; the training process of the key point prediction model includes: Acquire sample images of eyes in different occlusion states, wherein the sample images are annotated with iris boundary key points and eyelid boundary key points, and distribution of the annotated iris boundary key points in the sample images conforms to iris contour shape conditions; the annotated eyelid boundary key points are used to characterize the eyelid boundary in the sample images; The initial deep neural network model is trained based on the sample image until the model training stop condition is met, thereby obtaining a key point prediction model for predicting key points of the eye image.

4. The method according to claim 3, characterized in that The eyelid boundary includes an upper eyelid boundary and a lower eyelid boundary; The eyelid boundary key points include a corner of the eye key point located at the intersection of the upper eyelid boundary and the lower eyelid boundary, an upper eyelid key point located on the upper eyelid boundary, and a lower eyelid key point located on the lower eyelid boundary; The number of the upper eyelid key points is greater than the number of the lower eyelid key points.

5. The method according to claim 1, characterized in that The step of determining the eyelid boundary formed by the eyelid boundary key points comprises: From each of the eyelid boundary key points, identify the eye corner key point, the upper eyelid key point and the lower eyelid key point; Taking the eye corner key point as an endpoint, connecting the upper eyelid key points through a first connecting line to obtain an upper eyelid boundary; Taking the eye corner key point as an endpoint, connecting the lower eyelid key points through a second connecting line to obtain a lower eyelid boundary; An eyelid boundary is determined based on the upper eyelid boundary and the lower eyelid boundary.

6. The method according to claim 1, characterized in that The method of performing key point prediction on the eye image of the target object based on the key point prediction algorithm to obtain a plurality of iris boundary key points and a plurality of eyelid boundary key points includes: Based on the key point prediction algorithm, key points of the eye image of the target object are predicted to obtain multiple pupil boundary key points, multiple iris outer boundary key points and multiple eyelid boundary key points; According to the iris contour shape condition, contour fitting is performed on each of the iris boundary key points to obtain a predicted iris area, including: Based on the distribution of each of the iris outer boundary key points and in accordance with the iris outer contour shape condition, the iris outer contour is fitted to each of the iris outer boundary key points to obtain a predicted iris outer boundary; Based on the distribution of each pupil boundary key point and in accordance with the pupil contour shape condition, pupil contour fitting is performed on each pupil boundary key point to obtain a predicted pupil boundary; A predicted iris region is determined according to the predicted iris outer boundary and the predicted pupil boundary.

7. The method according to claim 6, characterized in that The shape condition of the iris outer contour is an elliptical contour; the shape condition of the pupil contour is a circular contour; Determining the predicted iris area according to the predicted iris outer boundary and the predicted pupil boundary includes: respectively determining an elliptical area formed by the outer boundary of the predicted iris and a circular area formed by the outer boundary of the predicted iris; An area in the elliptical area that does not overlap with the circular area is determined as a predicted iris area.

8. The method according to any one of claims 1 to 7, characterized in that The step of performing iris occlusion analysis on the target object based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary to obtain an iris occlusion analysis result includes: For each pixel point in the predicted iris area, according to the coordinates of each pixel point, a target pixel point in the area formed by the eyelid boundary is screened out; Determining an iris occlusion ratio of the target object based on a ratio of the number of target pixels to the total number of pixels in the predicted iris region; An iris occlusion analysis is performed based on the iris occlusion ratio and a maximum tolerable iris occlusion ratio threshold to obtain an iris occlusion analysis result.

9. The method according to claim 8, characterized in that The method further comprises: When the iris occlusion analysis result is that the iris occlusion ratio is less than or equal to a maximum tolerable iris occlusion ratio threshold, extracting iris features in the iris region; Based on the iris features, iris recognition processing is performed on the target object to obtain a recognition result.

10. The method according to claim 8, characterized in that The method further comprises: When the iris occlusion analysis result is that the iris occlusion ratio is greater than a maximum tolerable iris occlusion ratio threshold, a reminder message for the target object is generated.

11. The method according to claim 10, characterized in that When the iris occlusion analysis result is that the iris occlusion ratio is greater than a maximum tolerable iris occlusion ratio threshold, generating a reminder message for the target object includes: When the iris occlusion analysis result is that the iris occlusion ratio is greater than a maximum tolerable iris occlusion ratio threshold, determining a reminder message type based on a current scene of the target object; Based on the reminder message type, a reminder message for the target object is generated.

12. An iris occlusion analysis device, characterized in that: The device comprises: A key point prediction module is used to predict key points of the eye image of the target object based on a key point prediction algorithm to obtain multiple iris boundary key points and multiple eyelid boundary key points; A contour fitting module, used to perform contour fitting on each of the iris boundary key points according to the iris contour shape conditions to obtain a predicted iris area; A boundary determination module, used for determining the eyelid boundary formed by each of the eyelid boundary key points; The occlusion analysis module is used to perform iris occlusion analysis on the target object based on the relative position relationship between the predicted iris area and the area formed by the eyelid boundary to obtain an iris occlusion analysis result.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images