Pupil and iris positioning method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a pupil and iris positioning method and device, electronic equipment and storage medium. The method comprises: obtaining a target eye image; inputting the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, wherein the target detection network model is pre-trained based on sample data, and the sample data comprises: sample eye images labeled with the regions where the pupil and iris are located; determining a target iris region from each candidate iris region, wherein the size ratio of the target iris region to the corresponding candidate pupil region is within a preset range; determining the target iris region as the region where the iris is located in the target eye image, and determining the candidate pupil region corresponding to the target iris region as the region where the pupil is located in the target eye image. The embodiments can improve the efficiency and accuracy of determining the regions where the pupil and iris are located in the target eye image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent algorithm, in particular to a pupil and iris positioning method and device, electronic equipment and storage medium. BACKGROUND

[0002] When determining the position of the iris in the image, a pixel segmentation can be used to determine the region of the iris based on a segmentation algorithm of deep learning. However, the segmentation algorithm model based on deep learning is usually large and has high complexity, and consumes a lot of resources of the algorithm running environment, so that the efficiency of determining the region where the iris is located by the segmentation algorithm model is low. SUMMARY

[0003] The embodiments of the present application aim to provide a pupil and iris positioning method and device, electronic equipment and storage medium to improve the efficiency of determining the region where the pupil and iris are located. The specific technical solutions are as follows:

[0004] In the first aspect of the present application, a pupil and iris positioning method is provided, which comprises:

[0005] obtaining a target eye image;

[0006] inputting the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, wherein the target detection network model is obtained by pre-training based on sample data, and the sample data comprises sample eye images labeled with the regions where the pupil and iris are located;

[0007] determining a target iris region from each candidate iris region, wherein the size ratio of the target iris region to the corresponding candidate pupil region is within a preset range;

[0008] determining the target iris region as the region where the iris is located in the target eye image, and determining the candidate pupil region corresponding to the target iris region as the region where the pupil is located in the target eye image.

[0009] In a possible embodiment, the inputting the target eye image into the pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region comprises:

[0010] inputting the target eye image into the pre-trained target detection network model to obtain a plurality of detection iris regions output by the target detection network model and a detection pupil region corresponding to each detection iris region;

[0011] The detected iris regions and / or the detected pupil regions are de-duplicated to obtain a plurality of candidate iris regions and corresponding candidate pupil regions.

[0012] In one possible implementation, the de-duplicating the detected iris regions and / or the detected pupil regions to obtain a plurality of candidate iris regions and corresponding candidate pupil regions includes:

[0013] A target detected iris region is determined from a set of iris regions, wherein the set of iris regions initially includes the detected iris regions;

[0014] A duplicate iris region is determined from the set of iris regions that overlaps with the target detected iris region to a degree greater than a preset overlap threshold;

[0015] If a preset loop condition is not satisfied, the set of iris regions excluding the duplicate iris region and the target detected iris region is taken as a new set of iris regions, and the step of determining the target detected iris region from the set of iris regions is executed again;

[0016] If the preset loop condition is satisfied, the detected iris regions that are not determined as the duplicate iris region are determined as candidate iris regions to obtain a plurality of candidate iris regions and corresponding candidate pupil regions;

[0017] and / or

[0018] A target detected pupil region is determined from a set of pupil regions, wherein the set of pupil regions initially includes the detected pupil regions;

[0019] A duplicate pupil region is determined from the set of pupil regions that overlaps with the target detected pupil region to a degree greater than a preset overlap threshold;

[0020] If a preset loop condition is not satisfied, the set of pupil regions excluding the duplicate pupil region and the target detected pupil region is taken as a new set of pupil regions, and the step of determining the target detected pupil region from the set of pupil regions is executed again;

[0021] If the preset loop condition is satisfied, the detected pupil regions that are not determined as the duplicate pupil region are determined as candidate pupil regions to obtain a plurality of candidate pupil regions and corresponding candidate iris regions.

[0022] In a possible embodiment, the target detection network model is trained based on sample data by the following steps in advance, comprising:

[0023] inputting the sample data into the detection network model to be trained to obtain a plurality of regression regions and a classification result corresponding to each regression region, the classification result being used to represent an iris or a pupil;

[0024] determining a region overlap degree between each regression region and a region labeled in the sample data, and a category difference between a category represented by the classification result corresponding to each regression region and a category labeled in the sample data for the regression region;

[0025] determining a model loss according to the overlap degree and the category difference, the model loss being negatively correlated with the overlap degree and positively correlated with the category difference;

[0026] adjusting network parameters of the detection network model to be trained based on the model loss to obtain the target detection network model.

[0027] In a possible embodiment, the method further comprises:

[0028] determining a larger value between a length and a width of the target iris region as a first maximum size;

[0029] determining a center and / or a radius of an iris in the target eye image according to a position of the target iris region and the first maximum size;

[0030] determining a larger value between a length and a width of the target pupil region as a second maximum size, the target pupil region being the candidate pupil region corresponding to the target iris region;

[0031] determining a center and / or a radius of a pupil in the target eye image according to a position of the target pupil region and the second maximum size.

[0032] In a second aspect of the present application, a pupil and iris positioning device is provided, the device comprising:

[0033] an image acquisition module, configured to acquire a target eye image;

[0034] a target detection module, configured to input the target eye image into a target detection network model trained in advance to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, wherein the target detection network model is trained in advance based on sample data, and the sample data comprises a sample eye image with regions where a pupil and an iris are located labeled;

[0035] a target iris region determination module configured to determine a target iris region from each of the candidate iris regions, the target iris region having a size ratio to a corresponding candidate pupil region within a preset range;

[0036] an iris pupil region determination module configured to determine the target iris region as a region in which an iris is located in the target eye image, and determine the candidate pupil region corresponding to the target iris region as a region in which a pupil is located in the target eye image.

[0037] In a possible embodiment, the target detection module comprises:

[0038] a detection region output unit configured to input the target eye image into a pre-trained target detection network model to obtain a plurality of detection iris regions and a detection pupil region corresponding to each of the detection iris regions output by the target detection network model;

[0039] a detection region deduplication unit configured to deduplicate each of the detection iris regions and / or each of the detection pupil regions to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions.

[0040] In a possible embodiment, the detection region deduplication unit deduplicates each of the detection iris regions and / or each of the detection pupil regions to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions, comprising:

[0041] determining a target detection iris region from a set of iris regions, the set of iris regions initially comprising each of the detection iris regions, the target detection iris region having a confidence degree satisfying a preset confidence condition;

[0042] determining a duplicate iris region in the set of iris regions, the duplicate iris region having an overlap degree greater than a preset overlap threshold with the target detection iris region;

[0043] if a preset loop condition is not satisfied, taking the set of iris regions from which the duplicate iris region and the target detection iris region are removed as a new set of iris regions, and returning to the step of determining the target detection iris region from the set of iris regions, the target detection iris region having the confidence degree satisfying the preset confidence condition;

[0044] if the preset loop condition is satisfied, determining the detection iris region that is not determined as the duplicate iris region as a candidate iris region to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions;

[0045] and / or

[0046] determining a target detected pupil region from the pupil region set, wherein the pupil region set initially comprises each of the detected pupil regions;

[0047] determining a duplicate pupil region from the pupil region set, wherein the duplicate pupil region overlaps with the target detected pupil region to a degree greater than a preset overlap threshold;

[0048] if the preset loop condition is not met, taking the pupil region set after removing the duplicate pupil region and the target detected pupil region as a new pupil region set, and returning to the step of determining the target detected pupil region from the pupil region set;

[0049] if the preset loop condition is met, determining the detected pupil region that is not determined as the duplicate pupil region as a candidate pupil region, to obtain a plurality of candidate pupil regions and a candidate iris region corresponding to each of the candidate pupil regions.

[0050] In a possible embodiment, the apparatus further includes:

[0051] a model training module configured to train the target detection network model based on sample data by using the following steps:

[0052] inputting the sample data into a detection network model to be trained to obtain a plurality of regression regions and a classification result corresponding to each of the regression regions, wherein the classification result is used to represent an iris or a pupil;

[0053] determining a region overlap degree between each of the regression regions and a region labeled in the sample data, and a category difference between a category represented by the classification result corresponding to each of the regression regions and a category labeled in the sample data for the regression region;

[0054] determining a model loss based on the region overlap degree and the category difference, wherein the model loss is negatively correlated with the region overlap degree and positively correlated with the category difference;

[0055] adjusting a network parameter of the detection network model to be trained based on the model loss to obtain a target detection network model.

[0056] In a possible embodiment, the apparatus further includes:

[0057] a first maximum size determination module configured to determine a larger value between a length and a width of the target iris region as a first maximum size;

[0058] an iris center and / or radius determination module configured to determine a center and / or a radius of an iris in the target eye image based on a position of the target iris region and the first maximum size.

[0059] a second maximum size determination module configured to determine a larger one of a length and a width of the target pupil region as a second maximum size, the target pupil region being the candidate pupil region corresponding to the target iris region;

[0060] a pupil center and / or radius determination module configured to determine a center and / or a radius of a pupil in the target eye image according to a position of the target pupil region and the second maximum size.

[0061] In a third aspect of the present application, an electronic device is provided, comprising:

[0062] a memory configured to store a computer program;

[0063] a processor configured to execute the computer program stored in the memory to implement any of the pupil and iris positioning methods described above.

[0064] In a fourth aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement any of the pupil and iris positioning methods described above.

[0065] The embodiments of the present application have the following beneficial effects:

[0066] The pupil and iris positioning method, device, electronic device and storage medium provided by the embodiments of the present application, by detecting the target eye image through the target detection network model, the target detection network model being obtained by pre-training based on the sample eye image with the region where the pupil and iris are located labeled, so as to obtain multiple candidate iris regions of the target eye image and the candidate pupil region corresponding to each candidate iris region, and determine the target iris region from each candidate iris region, the size ratio of the target iris region and the corresponding candidate pupil region being within a preset range, so as to determine the target iris region as the region where the iris is located in the target eye image, determine the candidate pupil region corresponding to the target iris region as the region where the pupil is located in the target eye image, and obtain the region where the pupil and iris are located in the target eye image. Since the complexity of the target detection network model is lower than that of the pixel segmentation algorithm model, compared with the pixel segmentation algorithm model, the target detection network model has lower complexity, controllable size and faster running speed, thereby improving the efficiency of determining the region where the pupil and iris are located in the target eye image. Moreover, the size ratio relationship between the iris region and the pupil region is fully considered, so as to screen each candidate iris region and the candidate pupil region corresponding to the candidate iris region, so that the region where the pupil and iris are located in the target eye image is more accurate.

[0067] Of course, implementing any product or method of the present application does not necessarily require achieving all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0069] Figure 1 A first flowchart of the pupil and iris positioning method provided by the present application;

[0070] Figure 2 A flowchart of the target detection network model training method provided by the present application;

[0071] Figure 3a A schematic diagram of the sample eye image used when training the target detection network model provided by the present application;

[0072] Figure 3b A schematic diagram of the overlap degree calculation parameter provided by the present application;

[0073] Figure 3c A schematic diagram of the target detection network model output result provided by the present application;

[0074] Figure 3d Another schematic diagram of the target detection network model output result provided by the present application;

[0075] Figure 4 A second flowchart of the pupil and iris positioning method provided by the present application;

[0076] Figure 5 A third flowchart of the pupil and iris positioning method provided by the present application;

[0077] Figure 6 An application flowchart of the pupil and iris positioning method provided by the present application;

[0078] Figure 7 A structural schematic diagram of the pupil and iris positioning device provided by the present application;

[0079] Figure 8 A structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0080] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application are within the scope of protection of the present application.

[0081] To improve the efficiency of determining the region where the pupil and the iris are located, the present application provides a pupil and iris positioning method, device, electronic equipment and storage medium, which are described in detail below.

[0082] The present application provides a pupil and iris positioning method, as shown in the method comprises: Figure 1

[0083] S101, obtaining a target eye image.

[0084] S102, inputting the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region.

[0085] The target detection network model is obtained by pre-training based on sample data, and the sample data includes sample eye images labeled with the regions where the pupil and the iris are located.

[0086] S103, determining a target iris region from each candidate iris region, wherein the size ratio of the target iris region to the corresponding candidate pupil region is within a preset range.

[0087] S104, determining the target iris region as the region where the iris is located in the target eye image, and determining the candidate pupil region corresponding to the target iris region as the region where the pupil is located in the target eye image.

[0088] ​In this embodiment, the target eye image is detected by a target detection network model. The target detection network model is trained based on sample eye images with the pupil and the iris regions labeled in advance. Thus, multiple candidate iris regions and the corresponding candidate pupil regions of the target eye image are obtained. The target iris region is determined from the candidate iris regions, which has a size ratio with the corresponding candidate pupil region within a preset range. Thus, the target iris region is determined as the region where the iris is located in the target eye image, and the corresponding candidate pupil region is determined as the region where the pupil is located in the target eye image. The regions where the pupil and the iris are located in the target eye image are obtained. Since the complexity of the target detection network model is lower than that of the pixel segmentation algorithm model, the target detection network model has lower complexity, is controllable in size, and runs faster than the pixel segmentation algorithm model. Thus, the efficiency of determining the regions where the pupil and the iris are located in the target eye image is improved. Moreover, the size ratio relationship between the iris region and the pupil region is fully considered. Thus, the candidate iris regions and the corresponding candidate pupil regions are screened, and the regions where the pupil and the iris are located in the target eye image are more accurate.

[0089] The foregoing S101-S104 will be described below.

[0090] In S101, the original eye image can be acquired as the target eye image. The original eye image can also be preprocessed to obtain the target eye image after being acquired.

[0091] The preprocessing of the original eye image can include scaling the original eye image to a specified size by using a letter_box function (adaptive picture scaling function). The original eye image can be adaptively added with as few black edges as possible, and the original eye image is scaled to a size that can be processed by the target detection network model while the information of the original eye image is well preserved.

[0092] In S102, the candidate iris region and the candidate pupil region represent the positions of the iris and the pupil in the target eye image. The regions can have various forms of representation, such as a circular frame according to the shape of the iris or the pupil, a circumscribed square frame of the iris and the pupil, and the like, which are not limited in the present application.

[0093] Since the pupil and the iris are in one-to-one correspondence in the sample eye image in the sample data, the target detection model can learn how to determine the regions where the pupil and the iris are located and the correspondence between the regions where the pupil and the iris are located during the training of the target detection model using the sample data. Therefore, when the target eye image is input into the target detection network model, a plurality of candidate iris regions and a plurality of candidate pupil regions can be obtained, and each candidate iris region has a corresponding candidate pupil region.

[0094] In S103, it can be understood that, due to the structure of the human body, the size of the iris is often positively correlated with the size of the pupil, and there is a certain proportional relationship, so the size ratio between the iris region and the corresponding pupil region is within a certain range. The size ratio of the obtained candidate iris region and the corresponding candidate pupil region that is not within this range can be considered as an inaccurate result. Therefore, the obtained plurality of candidate iris regions and the corresponding candidate pupil regions of each candidate iris region can be screened by using a preset size ratio range of the iris region and the pupil region to obtain a more accurate target iris region and a candidate pupil region corresponding to the target iris region.

[0095] The preset size ratio range of the iris region and the pupil region can be set according to past experience. For example, the preset range can be the ratio range of the pupil region to the iris region, which can be set to 0.2-0.9 or 0.3-0.8. The present application does not make any limitation on this.

[0096] In S104, since the target iris region obtained by screening is relatively accurate, the candidate pupil region corresponding to the target iris region is also relatively accurate. Therefore, the target iris region can be determined as the region where the iris is located in the target eye image, and the candidate pupil region corresponding to the target iris region can be determined as the region where the pupil is located in the target eye image.

[0097] The following will exemplarily illustrate how to pre-train the target detection network model based on the sample data. In one possible embodiment, the sample data can be used to pre-train the detection network model for a preset number of times. For each trained detection network model, the accuracy of the trained detection network model can be calculated using a test set, and each trained detection network model corresponds to an accuracy. The trained detection network model corresponding to the maximum accuracy can be selected as the target detection network model.

[0098] In another possible embodiment, the target detection network model can also be pre-trained based on the sample data by using the following steps, as shown in Figure 2 The steps include:

[0099] S201, input the sample data into the detection network model to be trained, and obtain multiple regression regions and the classification results corresponding to each regression region.

[0100] The classification result is used to represent the iris or pupil.

[0101] S202, determine the degree of overlap between each regression region and the region labeled in the sample data, and the difference between the category represented by the classification result corresponding to each regression region and the category labeled in the regression region in the sample data.

[0102] S203, determine the model loss based on the degree of overlap and class difference. The model loss is negatively correlated with the degree of overlap and positively correlated with the class difference.

[0103] S204. Adjust the network parameters of the target detection network model to be trained based on the model loss to obtain the target detection network model.

[0104] By using this embodiment, the degree of overlap and class difference can be calculated from the multiple regression regions output by training and the classification results and sample data corresponding to each regression region. The model loss can be determined based on the negative correlation between the degree of overlap and the model loss, and the positive correlation between the class difference and the model loss. This allows the network parameters of the detection network model to be trained to be adjusted based on the model loss, thereby making the obtained target detection network model more accurate.

[0105] The following will explain S201-S204 respectively, wherein:

[0106] In S201, the sample data is generated from the original sample database. Since the iris and pupil can be approximated as perfect circles, the shape of the iris and pupil bounding boxes can be set to squares circumscribed by the iris and pupil. Correct bounding boxes help improve the model training effect. These bounding boxes are the annotations of the regions containing the iris and pupil in the sample eye image. The annotation information is the detection box I_rect (Iris_rectangle, iris rectangle) for the outer circle of the iris and the detection box P_rect (Pupil_rectangle, pupil rectangle) for the inner circle of the pupil. The generated sample eye images with annotations of the pupil and iris regions are input into the detection network model to be trained. The sample eye images with annotations of the pupil and iris regions can be used as follows: Figure 3a As shown. Figure 3a The large circle in the diagram represents the outer circle of the iris, the small circle represents the inner circle of the pupil, the large rectangle represents the detection frame for the outer circle of the iris, and the small rectangle represents the detection frame for the inner circle of the pupil.

[0107] Before the sample data is input into the detection network model to be trained, the sample data can be pre-processed. The sample eye images can be enhanced by using a Mosaic data enhancement method. Four sample eye images are processed by at least one of random scaling, random cropping, and random arrangement, and are spliced on a mosaic canvas to obtain a new sample eye image, thereby enriching the detection background and increasing the batch size. Specifically, random scaling refers to randomly scaling the four sample eye images to any size, random cropping refers to randomly cropping the four sample eye images and retaining the cropped parts, and random arrangement refers to randomly arranging the four sample eye images on a mosaic canvas. After processing the four sample eye images, the images are spliced on a mosaic canvas to obtain a spliced image. If the spliced image has an area that exceeds the mosaic canvas, the spliced image is cropped to retain only the part of the spliced image that does not exceed the mosaic canvas as a new sample eye image.

[0108] For example, assuming that the four sample eye images are processed and spliced to obtain a new sample eye image by using random scaling and random cropping, the four sample eye images need to be processed by random scaling and random cropping, respectively. The four processed sample eye images are spliced on a mosaic canvas according to a preset arrangement to obtain a spliced image. Only the part of the spliced image that does not exceed the mosaic canvas is retained as a new sample eye image.

[0109] For example, assuming that the four sample eye images are processed and spliced to obtain a new sample eye image by using random scaling and random arrangement, the four sample eye images need to be processed by random scaling. The four processed sample eye images are randomly arranged on a mosaic canvas and spliced to obtain a spliced image. Only the part of the spliced image that does not exceed the mosaic canvas is retained as a new sample eye image.

[0110] The letter_box function is used to scale the sample eye images to a specified size, so that the letter_box function can adaptively add as little black border as possible to the sample eye images, reduce information redundancy, and improve the model training speed. When the sample eye images are pre-processed as described above, the label box information in the sample eye images is also pre-processed as described above.

[0111] The to-be-detected network model can be a yolov5 model. Before model training by the yolov5 model, the sample eye image set can be clustered according to the scale of the detection frame by a k-means clustering algorithm to obtain N initial anchor frame information of different scales. The value of N can be different according to different application scenarios. For example, N can be 3, 4 or another integer. For the convenience of description, only the case of N being 3 is exemplarily described below. The principle is the same for other values of N, which is not described here. During the training of the to-be-detected network model, the to-be-detected network model can output a detection frame on the basis of the initial anchor frame, and then compare it with the real frame, calculate the difference between the two, and then update the network parameters in reverse. Therefore, the initial anchor frame information can be set to speed up the model convergence speed.

[0112] The width-height ratio of all the output detection frames of the current three scales and the initial anchor frame information is calculated, and compared with the pre-set scale threshold. If the larger value of the width-height ratio of the detection frame and the initial anchor frame is less than the pre-set scale threshold, the detection frame is considered to be a positive sample, otherwise it is considered to be a negative sample. That is, when the larger value of the width-height ratio of the detection frame and the initial anchor frame is greater than or equal to the pre-set scale threshold, the detection frame is considered to be a negative sample. Since the detection frame is placed in an SxS grid of the same size as the sample eye image when selecting a positive sample, the detection frame corresponding to the middle points of the two grids closest to the middle point of the positive sample detection frame can also be considered as a positive sample. Therefore, each real frame has at least 3 detection frames on each scale for training, and at least 9 detection frames on the three scales are allocated. The scale threshold can be set according to past experience or actual needs. In specific applications, the scale threshold can be set to 2.

[0113] The preprocessed sample data is input into the to-be-detected network model configured according to the above operation for training to obtain an output result, i.e., a plurality of regression regions and the classification result of each regression region as an iris or a pupil. The output result can be as shown in Figure 3c and Figure 3d . Figure 3c The large circle in represents the area of the iris in the sample eye image, and the small circle represents the area of the pupil in the sample eye image. Figure 3d The large circle in represents the area of the iris in the sample eye image, and the small circle represents the area of the pupil in the sample eye image.

[0114] In S202, when calculating the area overlap degree between each regression area and the area labeled by the sample data, taking the area as a rectangular detection box for example, the regression area is recorded as detection box A, and the area labeled by the sample data is recorded as detection box B, then calculating the area overlap degree between each regression area and the area labeled by the sample data is equivalent to calculating the overlap degree between detection box A and detection box B.

[0115] In a possible implementation, the overlap degree between detection box A and detection box B can be calculated by using an IOU calculation method, that is, calculated according to formula (1):

[0116]

[0117] wherein, S A∩B represents the intersection area of detection box A and detection box B, S A∪B represents the combined area of detection box A and detection box B.

[0118] In a possible implementation, the overlap degree between detection box A and detection box B can also be calculated by using a CIOU calculation method, that is, calculated according to formula (2):

[0119]

[0120] wherein, IOU can be calculated by the aforementioned formula (1), b represents the center point of detection box A, b gt represents the center point of detection box B, p(b, b gt ) represents the Euclidean distance between the center point of detection box A and the center point of detection box B, c represents the length of the diagonal of the minimum circumscribed rectangle of detection box A and detection box B, a is used to balance the proportional consistency between detection box A and detection box B, and v represents a weight coefficient.

[0121] As shown in FIG. 1, Figure 3b the two solid-line boxes in FIG. 1 are detection box A and detection box B respectively, the dashed-line box is the minimum circumscribed rectangle of detection box A and detection box B, Figure 3b d in FIG. 1 represents the Euclidean distance p(b, b gt ) between the center point of detection box A and the center point of detection box B, Figure 3b c in FIG. 1 represents the length of the diagonal of the minimum circumscribed rectangle of detection box A and detection box B. Figure 3b v can be calculated by formula (3):

[0122]

[0123]

[0124] wherein, w gt represents the width of detection box B, and h​gt w represents the height of detection box B, w represents the width of detection box A, and h represents the height of detection box A.

[0125] α can be calculated using formula (4):

[0126]

[0127] Among them, IOU can be calculated by the aforementioned formula (1), and v can be calculated by the aforementioned formula (3).

[0128] This implementation method calculates the overlap between detection boxes A and B using CIOU. Compared to IOU, it adds information on the center point distance and the aspect ratio of the detection boxes, resulting in better training effects for iris and pupil detection and improving the convergence speed and accuracy of the model.

[0129] When calculating the class difference between the class represented by the classification result corresponding to each regression region and the class labeled in the regression region of the sample data, BCE loss (binary cross-entropy loss) can be used.

[0130] For example, the class difference between the classification result corresponding to a certain regression region and the class labeled in the regression region in the sample data can be calculated using formula (5):

[0131] loss p,y=-y×log(p)-(1-y)log(1-p)…(5)

[0132] Where p represents the category represented by the classification result corresponding to a certain regression region, y represents the category labeled in the regression region in the sample data, and loss(p, y) represents the category difference between the category represented by the classification result corresponding to a certain regression region and the category labeled in the regression region in the sample data.

[0133] In S203, it can be understood that the higher the degree of overlap between the regression regions and the labeled regions in the sample data, the more similar the regression regions are to the labeled regions in the sample data, and the more accurate the regression regions output by the model, i.e., the smaller the model loss. Conversely, the smaller the difference between the category represented by the classification result corresponding to each regression region and the category labeled in the regression region in the sample data, the more accurate the category represented by the classification result corresponding to the regression region, and the more accurate the classification result corresponding to the regression region output by the model, i.e., the smaller the model loss. Therefore, the model loss is negatively correlated with the degree of overlap and positively correlated with the class difference.

[0134] In S204, the network parameters of the to-be-trained detection network model are adjusted according to the model loss. When the number of adjustments reaches a preset number, the to-be-trained detection network model after the adjustment is considered as the target detection network model. When the number of adjustments does not reach the preset number, the sample data is input into the to-be-trained detection network model after the adjustment of the network parameters, and a plurality of regression regions and classification results corresponding to each regression region are obtained. The target detection network model can also be determined according to the model loss. When the model loss no longer decreases with the adjustment of the parameters, the to-be-trained detection network model after the adjustment is considered as the target detection network model. When the model loss is still decreasing, the sample data is input into the to-be-trained detection network model after the adjustment of the network parameters, and a plurality of regression regions and classification results corresponding to each regression region are obtained.

[0135] In a possible embodiment, as shown in Figure 4 The foregoing step S102 includes:

[0136] S1021, inputting the target eye image into a pre-trained target detection network model to obtain a plurality of detected iris regions and a detected pupil region corresponding to each detected iris region output by the target detection network model.

[0137] S1022, removing duplicates from each detected iris region and / or each detected pupil region to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region.

[0138] In this embodiment, the output results of the target detection network model are removed to remove duplicate detected iris regions and detected pupil regions in the output results, thereby obtaining a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, reducing the number of candidate iris regions and candidate pupil regions, and facilitating subsequent screening of the candidate iris regions and candidate pupil regions according to a preset size ratio range of the iris region and the pupil region, thereby improving the efficiency of determining the regions of the iris and the pupil in the target eye image.

[0139] The foregoing S1021-S1022 will be described below.

[0140] In S1021, the target eye image is input into a pre-trained target detection network model to obtain an output result of the target detection network model, which is a plurality of detected iris regions and a detected pupil region corresponding to each detected iris region. The output result of the target detection network model can be as shown in Figure 3c and Figure 3d

[0141] ​In S1022, according to the training process of the target detection network model, the target detection network model is trained based on the yolov5 model, therefore, the detection regions output by the target detection network model will have three detection regions of different scales, and there will be at least 3 detection regions on each scale, therefore, there will be repetitive output results in the multiple detection iris regions output by the target detection network model and the detection pupil regions corresponding to each detection iris region, and it is necessary to remove the repetitive detection iris regions and the detection pupil regions corresponding to each detection iris region, so as to reduce the number of candidate iris regions and candidate pupil regions, and facilitate the subsequent determination of the target iris region and the target pupil region.

[0142] The possible implementation of S1022 will be exemplarily described below, please refer to the following mode 1-mode 3.

[0143] Mode 1: determining a target detection iris region that satisfies a pre-set confidence condition from the iris region set, wherein the iris region set initially includes each detection iris region.

[0144] Specifically, according to the confidence of each detection iris region, the detection iris region that satisfies the pre-set confidence condition is taken as the target detection iris region. The pre-set confidence condition can be that the confidence is greater than or equal to a pre-set confidence threshold, that is, when the confidence of the detection iris region is greater than or equal to the pre-set confidence threshold, the detection iris region is taken as the target detection iris region. The pre-set confidence threshold can be set according to past experience or actual demand, and in specific application, the confidence threshold can be set to 0.5.

[0145] Determining a repeated iris region that overlaps with the target detection iris region to a degree greater than a pre-set overlap threshold in the iris region set.

[0146] Wherein, when calculating the overlap degree between the detection iris region in the iris region set and the target detection iris region, taking a rectangular detection box as an example, the detection iris region in the iris region set is recorded as detection box A, and the target detection iris region is recorded as detection box B, then calculating the overlap degree between the detection iris region in the iris region set and the target detection iris region is equivalent to calculating the overlap degree between detection box A and detection box B.

[0147] The overlap degree between detection box A and detection box B can be calculated according to the foregoing formula (1) or formula (2)-formula (4), which will not be described here.

[0148] It can be understood that the greater the degree of overlap between the detected iris region and the target detected iris region, the more similar the detected iris region and the target detected iris region, that is, it can be considered that the detected iris region and the target detected iris region are repeated, and therefore the detected iris region with a degree of overlap greater than a preset overlap threshold with the target detected iris region can be regarded as a repeated iris region. The preset overlap threshold can be set according to past experience or actual demand, and in specific applications, the preset overlap threshold can be set to 0.5.

[0149] If the preset loop condition is not met, the iris region set after removing the repeated iris region and the target detected iris region is taken as a new iris region set, and the step of determining the target detected iris region with a confidence degree satisfying the preset confidence condition from the iris region set is returned to be executed; if the preset loop condition is met, the detected iris region that is not determined as the repeated iris region is determined as a candidate iris region, and a plurality of candidate iris regions and candidate pupil regions corresponding to each candidate iris region are obtained.

[0150] The preset loop condition can refer to a preset loop number, when the loop number does not reach the preset loop number, that is, the preset loop condition is not met, the repeated iris region is removed from the iris region set to obtain a new iris region set, and the step of determining the target detected iris region with a confidence degree satisfying the preset confidence condition from the iris region set is returned to be executed; when the loop number reaches the preset loop number, that is, the preset loop condition is met, the detected iris region that is not determined as the repeated iris region is determined as the candidate iris region, that is, the repeated detected iris region is removed, and the detected iris region is de-duplicated, and the detected pupil region corresponding to the detected iris region that is not determined as the repeated iris region is determined as the candidate pupil region.

[0151] The preset loop condition can also refer to that the number of determined repeated iris regions is greater than or equal to a preset number of repeated iris regions, when the number of determined repeated iris regions is less than the preset number of repeated iris regions, that is, the preset loop condition is not met, the repeated iris region is removed from the iris region set to obtain a new iris region set, and the step of determining the target detected iris region with a confidence degree satisfying the preset confidence condition from the iris region set is returned to be executed; when the number of determined repeated iris regions is greater than or equal to the preset number of repeated iris regions, that is, the preset loop condition is met, the detected iris region that is not determined as the repeated iris region is determined as the candidate iris region, that is, the repeated detected iris region is removed, and the detected iris region is de-duplicated, and the detected pupil region corresponding to the detected iris region that is not determined as the repeated iris region is determined as the candidate pupil region.

[0152] With this embodiment, the target detection iris region with higher reliability can be determined according to the confidence of each detection iris region, and the detection iris regions are continuously screened according to the overlapping degree between the detection iris regions in the iris region set and the target detection iris region until the preset loop condition is met, so as to obtain multiple candidate iris regions and the candidate pupil region corresponding to each candidate iris region. The multiple detection iris regions are screened through the confidence and the overlapping degree, so as to remove the repeated detection iris regions, reduce the number of candidate iris regions, and also reduce the number of candidate pupil regions, facilitating the subsequent determination of the target iris region and the target pupil region, and improving the efficiency of determining the region where the iris and the pupil in the target eye image are located.

[0153] Mode 2: determining a target detection pupil region with confidence satisfying a preset confidence condition from the pupil region set, wherein the pupil region set initially includes each detection pupil region.

[0154] Specifically, according to the confidence of each detection pupil region, the detection pupil region satisfying the preset confidence condition is taken as the target detection pupil region. The preset confidence condition can be that the confidence is greater than or equal to a preset confidence threshold, that is, when the confidence of the detection pupil region is greater than or equal to the preset confidence threshold, the detection pupil region is taken as the target detection pupil region. The preset confidence threshold can be set according to past experience or actual demand, and in specific applications, the confidence threshold can be set to 0.5.

[0155] In the pupil region set, a repeated pupil region with an overlapping degree greater than a preset overlapping threshold with the target detection pupil region is determined.

[0156] In the calculation of the overlapping degree between the detection pupil region in the pupil region set and the target detection pupil region, for example, taking the region as a rectangular detection frame, the detection pupil region in the pupil region set is recorded as detection frame A, and the target detection pupil region is recorded as detection frame B. Then, the calculation of the overlapping degree between the detection pupil region in the pupil region set and the target detection pupil region is equivalent to the calculation of the overlapping degree between detection frame A and detection frame B.

[0157] The overlapping degree between detection frame A and detection frame B can be calculated according to the aforementioned formula (1) or formula (2)-(4), which will not be repeated here.

[0158] It can be understood that the greater the degree of overlap between the detected pupil region and the target detected pupil region, the more similar the detected pupil region and the target detected pupil region, that is, it can be considered that the detected pupil region and the target detected pupil region are repeated, and therefore the detected pupil region with a degree of overlap greater than a preset overlap threshold with the target detected pupil region can be regarded as a repeated pupil region. The preset overlap threshold can be set according to past experience or actual demand, and in specific applications, the preset overlap threshold can be set to 0.5.

[0159] If the preset loop condition is not met, the pupil region set after removing the repeated pupil region and the target detected pupil region is taken as a new pupil region set, and the step of determining the target detected pupil region with a confidence degree satisfying the preset confidence condition from the pupil region set is executed again; if the preset loop condition is met, the detected pupil region that is not determined as the repeated pupil region is determined as a candidate pupil region, and a plurality of candidate pupil regions and candidate iris regions corresponding to each candidate pupil region are obtained.

[0160] The preset loop condition can refer to a preset loop number. When the loop number does not reach the preset loop number, that is, the preset loop condition is not met, the repeated pupil region is removed from the pupil region set to obtain a new pupil region set, and the step of determining the target detected pupil region with a confidence degree satisfying the preset confidence condition from the pupil region set is executed again; when the loop number reaches the preset loop number, that is, the preset loop condition is met, the detected pupil region that is not determined as the repeated pupil region is determined as the candidate pupil region, that is, the repeated detected pupil region is removed, and the detected pupil region is de-duplicated, and the detected iris region corresponding to the detected pupil region that is not determined as the repeated pupil region is determined as the candidate iris region.

[0161] The preset loop condition can also refer to that the number of determined repeated pupil regions is greater than or equal to a preset number of repeated pupil regions. When the number of determined repeated pupil regions is less than the preset number of repeated pupil regions, that is, the preset loop condition is not met, the repeated pupil region is removed from the pupil region set to obtain a new pupil region set, and the step of determining the target detected pupil region with a confidence degree satisfying the preset confidence condition from the pupil region set is executed again; when the number of determined repeated pupil regions is greater than or equal to the preset number of repeated pupil regions, that is, the preset loop condition is met, the detected pupil region that is not determined as the repeated pupil region is determined as the candidate pupil region, that is, the repeated detected pupil region is removed, and the detected pupil region is de-duplicated, and the detected iris region corresponding to the detected pupil region that is not determined as the repeated pupil region is determined as the candidate iris region.

[0162] According to the embodiment, the confidence of each detected pupil region can be used to determine a target detected pupil region with higher reliability, and the detected pupil regions in the set of pupil regions are continuously screened according to the overlap between the detected pupil regions and the target detected pupil region until a preset loop condition is met, to obtain multiple candidate pupil regions and candidate iris regions corresponding to each candidate pupil region. The multiple detected pupil regions are screened according to the confidence and the overlap, so that the repeated detected pupil regions are removed, the number of candidate pupil regions is reduced, and the number of candidate iris regions is also reduced, facilitating subsequent determination of the target iris region and the target pupil region, and improving the efficiency of determining the regions where the iris and the pupil are located in the target eye image.

[0163] Mode 3: According to the embodiments of the preceding Mode 1 and Mode 2, each detected iris region and each detected pupil region is de-duplicated to obtain multiple candidate iris regions and multiple candidate pupil regions. If a certain candidate iris region has no corresponding candidate pupil region, or a certain candidate pupil region has no corresponding candidate iris region, the candidate iris region or the candidate pupil region is removed, so that the multiple candidate iris regions and the multiple candidate pupil regions correspond to each other.

[0164] According to the embodiment, each detected iris region and each detected pupil region can be de-duplicated at the same time, which can better remove the repeated detected iris regions and the repeated detected pupil regions, ensure that each candidate iris region and each candidate pupil region obtained do not have repeated regions, and thus reduce the number of candidate pupil regions and candidate iris regions, facilitate subsequent determination of the target iris region and the target pupil region, and improve the efficiency of determining the regions where the iris and the pupil are located in the target eye image.

[0165] Specifically, in the implementation of step S1022, any one of Mode 1 or Mode 2 can be used to de-duplicate each detected iris region or each detected pupil region to obtain multiple candidate iris regions and candidate pupil regions corresponding to each candidate iris region. Alternatively, the solution of Mode 3 can be used, i.e., the solutions of Mode 1 and Mode 2 are implemented at the same time, so that each detected iris region and each detected pupil region is de-duplicated to obtain multiple candidate iris regions and candidate pupil regions corresponding to each candidate iris region.

[0166] In a possible embodiment, the aforementioned sample data can further include: the center and / or radius of the iris and the center and / or radius of the pupil. In the sample data making process, in addition to marking the area where the pupil and the iris are located, a label can be added to each sample eye image, and the content of the label is the center and / or radius of the iris and the center and / or radius of the pupil. The center of the pupil and the iris can be obtained by the intersection point of the diagonals of the pupil marking box and the iris marking box, and half of the side length of the iris marking box can be taken as the radius of the iris, and half of the side length of the pupil marking box can be taken as the radius of the pupil. Moreover, in the pre-processing of the sample eye image, the label is smoothed to reduce the overfitting of the model. Thus, when the target eye image is input into the target detection network model trained, the center and / or radius of the iris and the center and / or radius of the pupil in the target eye image can also be obtained.

[0167] However, when the pupil or the iris is located at the edge of the target eye image, the pupil or the iris in the target eye image is incomplete, and the center of the pupil or the iris can not be in the target eye image. When the target eye image is input into the target detection network model, the candidate iris region and the candidate pupil region obtained can be in the form of a rectangular detection box. At this time, if the intersection point of the diagonals of the rectangular detection box is taken as the center of the pupil or the iris, the center of the pupil or the iris falls in the target eye image, thereby causing the center of the detected pupil or iris to be inaccurate. Moreover, when the pupil or the iris is located at the edge of the target eye image, the pupil or the iris in the target eye image can be smaller than the complete pupil or iris. At this time, if half of the length of any side of the rectangular detection box is taken as the radius of the pupil or the iris, the detected radius of the pupil or the iris can be smaller than the actual radius of the pupil or the iris, thereby causing the detected radius of the pupil or the iris to be inaccurate. Therefore, when the pupil or the iris is located at the edge of the eye image, the center and / or radius of the pupil and the iris obtained according to the above method are not accurate enough. Based on this, in a possible embodiment, as shown in Figure 5 the pupil and iris positioning method further includes:

[0168] S501, determining the larger value of the length and the width of the target iris region as a first maximum size.

[0169] S502, determining the center and / or radius of the iris and the center and / or radius of the pupil in the target eye image according to the position of the target iris region and the first maximum size.

[0170] In a possible implementation, half of the first maximum size can be taken as the radius of the iris in the target eye image, and a circle with the half of the first maximum size as the radius is made according to the position of the target iris region, so that the center of the iris in the target eye image is located on the perpendicular line of the longest side of the target iris region, and the intersection point of the perpendicular line and the longest side of the target iris region is the midpoint of the longest side of the target iris region, thereby obtaining the center of the iris in the target eye image.

[0171] In another possible implementation, the smaller one of the length and the width of the target iris region can be determined as the first minimum size, and the sum of half of the first maximum size and the first minimum size can be taken as the radius of the iris in the target eye image, and a circle with the sum of half of the first maximum size and the first minimum size as the radius is made according to the position of the target iris region, so that the center of the iris in the target eye image is located on the perpendicular line of the longest side of the target iris region, and the intersection point of the perpendicular line and the longest side of the target iris region is the midpoint of the longest side of the target iris region, thereby obtaining the center of the iris in the target eye image.

[0172] S503, the larger one of the length and the width of the target pupil region is determined as the second maximum size.

[0173] The target pupil region is a candidate pupil region corresponding to the target iris region.

[0174] S504, the center and / or the radius of the pupil in the target eye image are determined according to the position of the target pupil region and the second maximum size.

[0175] In a possible implementation, half of the second maximum size can be taken as the radius of the pupil in the target eye image, and a circle with half of the second maximum size as the radius is made according to the position of the target pupil region, so that the center of the pupil in the target eye image is located on the perpendicular line of the longest side of the target pupil region, and the intersection point of the perpendicular line and the longest side of the target pupil region is the midpoint of the longest side of the target pupil region, thereby obtaining the center of the pupil in the target eye image.

[0176] In another possible implementation, the smaller one of the length and the width of the target pupil region can be determined as the second minimum size, and the sum of half of the second maximum size and the second minimum size can be taken as the radius of the pupil in the target eye image, and a circle with the sum of half of the second maximum size and the second minimum size as the radius is made according to the position of the target pupil region, so that the center of the pupil in the target eye image is located on the perpendicular line of the longest side of the target pupil region, and the intersection point of the perpendicular line and the longest side of the target pupil region is the midpoint of the longest side of the target pupil region, thereby obtaining the center of the pupil in the target eye image.

[0177] By using this embodiment, the center and / or radius of the iris and the center and / or radius of the pupil in the target eye image can be determined based on the maximum value of the position, length, and width of the target iris region and the target pupil region, respectively. This allows for a more accurate calculation of the center and / or radius of the pupil and iris in the target eye image, resulting in richer and more accurate positional information of the iris and pupil in the obtained target eye image.

[0178] In practical applications, such as Figure 6 As shown, Figure 6 This is a schematic diagram of an application process for the pupil and iris localization method provided in this application. It is divided into two parts: a training process and an inference process. The training process is used to train the target detection network model, and the inference process is used to use the target detection network model to detect the input iris image, obtain the output result, and perform deduplication and correction on the output result to obtain a more accurate detection result.

[0179] The specific training process is as follows: input an iris image, which corresponds to the aforementioned sample eye image.

[0180] Image preprocessing involves preprocessing the input iris image, i.e., preprocessing the sample eye image. Specifically, image preprocessing can be performed according to the preprocessing method described in S201 above.

[0181] The preprocessed sample eye images are input into the target detection network model to be trained for forward inference, yielding the forward inference output. Specifically, the preprocessed sample eye images can be input into the YOLOv5 model to be trained, obtaining the forward inference output, which includes classification and regression results. The regression results represent the regions where the pupil and iris are located in the sample eye image, while the classification results indicate whether a given detection box contains the pupil or the iris.

[0182] Calculate the loss on the forward feed and validate the current model on the test set. Specifically, calculate the loss on the forward feed as described in S202 above, and test the current model on the test set to obtain the test results, which is the accuracy of the current model on the validation set.

[0183] Determine whether the current model meets the accuracy requirement on the validation set or whether the loss is no longer decreasing. If yes, that is, the current model meets the accuracy requirement on the validation set or the loss is no longer decreasing, then stop training, save the model weights, and use the current model as the object detection network model. If no, that is, the current model does not meet the accuracy requirement on the validation set or the loss is decreasing, then return to the step of inputting the model forward and outputting the forward result, and continue training the model.

[0184] The specific reasoning process is: inputting an iris image, the iris image corresponding to the original eye image in the foregoing S101.

[0185] Image preprocessing is performed on the original eye image to obtain a target eye image. Specifically, the original eye image can be preprocessed according to the preprocessing method in the foregoing S101 to obtain the target eye image.

[0186] The target eye image is input into a target detection network model for forward reasoning to obtain a forward result output by the target detection network model after forward reasoning. Specifically, the target eye image is input into the target detection network model to obtain the forward result, which is a plurality of detected iris regions and a detected pupil region corresponding to each detected iris region output by the target detection model.

[0187] The reasoning result is post-processed, including NMS (non-maximum suppression algorithm), calculation and correction of iris and pupil center radii. Specifically, the reasoning result is a plurality of detected iris regions and a detected pupil region corresponding to each detected iris region output by the target detection model. Each detected iris region and each detected pupil region can be de-duplicated according to the method in the foregoing S1022 to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region. The target iris region whose size ratio to the corresponding candidate pupil region is within a preset range is determined from each candidate iris region according to the method in the foregoing S103-S104, so as to obtain the region where the pupil and the iris are located in the target eye image. The iris, the pupil center and the radius in the target eye image are calculated and corrected according to the method in the foregoing S501-S504, so as to obtain the iris, the pupil center and the radius in the target eye image, and output the center and the radius of the iris and the pupil.

[0188] In this embodiment, by inputting the target eye image into the target detection network model, the reasoning result is obtained, and the reasoning result is post-processed, so that the position information of the iris and the pupil, i.e., the region where the iris and the pupil are located and the center and the radius of the iris and the pupil, can be quickly and accurately obtained from the target eye image, and the influence of noise such as eyelashes, glasses and eyelids is small. Moreover, since the position information of the iris and the pupil includes the annotation of the region where the iris and the pupil are located, the obtained position information of the iris and the pupil can be directly used as the output of the algorithm for iris normalization, eye positioning and gaze tracking, and other algorithm processing is not required. Compared with other iris positioning schemes, the scheme of this embodiment is more efficient.

[0189] Corresponding to the foregoing pupil and iris positioning method, the present application also provides a pupil and iris positioning device, as shown in Figure 7 , which comprises:

[0190] The image acquisition module 701 is configured to acquire a target eye image.

[0191] The target detection module 702 is configured to input the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, wherein the target detection network model is obtained by pre-training based on sample data, and the sample data includes sample eye images with regions of pupils and irises labeled.

[0192] The target iris region determination module 703 is configured to determine, from the candidate iris regions, a target iris region whose size ratio to the corresponding candidate pupil region is within a preset range.

[0193] The iris-pupil region determination module 704 is configured to determine the target iris region as a region of an iris in the target eye image, and determine the candidate pupil region corresponding to the target iris region as a region of a pupil in the target eye image.

[0194] In a possible embodiment, the target detection module includes:

[0195] The detection region output unit is configured to input the target eye image into a pre-trained target detection network model to obtain a plurality of detection iris regions output by the target detection network model and a detection pupil region corresponding to each detection iris region.

[0196] The detection region deduplication unit is configured to deduplicate the detection iris regions and / or the detection pupil regions to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region.

[0197] In a possible embodiment, the detection region deduplication unit deduplicates the detection iris regions and / or the detection pupil regions to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, including:

[0198] Determining, from the iris region set, a target detection iris region whose confidence degree satisfies a preset confidence condition, wherein the iris region set initially includes the detection iris regions;

[0199] Determining, in the iris region set, a repeated iris region whose overlap degree with the target detection iris region is greater than a preset overlap threshold;

[0200] If the preset loop condition is not satisfied, taking the iris region set after the repeated iris regions and the target detection iris region are removed as a new iris region set, and returning to the step of determining, from the iris region set, a target detection iris region whose confidence degree satisfies a preset confidence condition;

[0201] If the preset loop condition is met, the detected iris region that is not determined as the repeated iris region is determined as a candidate iris region, to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region;

[0202] and / or

[0203] The target detected pupil region with the confidence satisfying the preset confidence condition is determined from the pupil region set, wherein the pupil region set initially includes the detected pupil regions;

[0204] The repeated pupil region overlapping with the target detected pupil region to a degree greater than a preset overlap threshold is determined in the pupil region set;

[0205] If the preset loop condition is not met, the pupil region set with the repeated pupil region and the target detected pupil region removed is taken as a new pupil region set, and the step of determining the target detected pupil region with the confidence satisfying the preset confidence condition from the pupil region set is returned to be executed;

[0206] If the preset loop condition is met, the detected pupil region that is not determined as the repeated pupil region is determined as a candidate pupil region, to obtain a plurality of candidate pupil regions and a candidate iris region corresponding to each candidate pupil region.

[0207] In a possible embodiment, the apparatus further includes:

[0208] The model training module is configured to train the target detection network model based on sample data by using the following steps in advance:

[0209] The sample data is input into the detection network model to be trained, to obtain a plurality of regression regions and a classification result corresponding to each regression region, the classification result being used to represent an iris or a pupil;

[0210] The region overlap degree between each regression region and a region labeled in the sample data, and the category difference between the category represented by the classification result corresponding to each regression region and the category labeled in the regression region in the sample data are determined;

[0211] The model loss is determined according to the overlap degree and the category difference, the model loss being negatively correlated with the overlap degree and positively correlated with the category difference;

[0212] The network parameters of the detection network model to be trained are adjusted based on the model loss, to obtain the target detection network model.

[0213] In a possible embodiment, the apparatus further includes:

[0214] The first maximum size determination module is configured to determine the larger one of the length and the width of the target iris region as the first maximum size;

[0215] an iris center and / or radius determination module configured to determine a center and / or a radius of the iris in the target eye image according to the position of the target iris region and the first maximum size;

[0216] a second maximum size determination module configured to determine a larger value between a length and a width of the target pupil region as the second maximum size, the target pupil region being a candidate pupil region corresponding to the target iris region;

[0217] a pupil center and / or radius determination module configured to determine a center and / or a radius of the pupil in the target eye image according to the position of the target pupil region and the second maximum size.

[0218] The embodiments of the present application also provide an electronic device, as shown in the accompanying drawings, comprising: Figure 8

[0219] a memory 801 configured to store a computer program;

[0220] a processor 802 configured to execute the program stored in the memory 801 to implement the following steps:

[0221] obtaining a target eye image;

[0222] inputting the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each candidate iris region, wherein the target detection network model is obtained by pre-training based on sample data, and the sample data comprises sample eye images with pupil and iris regions labeled;

[0223] determining a target iris region from the candidate iris regions, the target iris region having a size ratio to the corresponding candidate pupil region within a preset range;

[0224] determining the target iris region as a region where the iris in the target eye image is located, and determining the candidate pupil region corresponding to the target iris region as a region where the pupil in the target eye image is located.

[0225] The electronic device can further comprise a communication bus and / or a communication interface, and the processor 802, the communication interface and the memory 801 can communicate with each other through the communication bus.

[0226] ​The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0227] The communication interface is used for communication between the above electronic device and other devices.

[0228] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0229] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0230] In another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of any of the above pupil and iris positioning methods are implemented.

[0231] In another embodiment provided in the present application, a computer program product containing instructions is also provided, and when the computer program product is run on a computer, the computer is caused to execute any of the pupil and iris positioning methods in the above embodiments.

[0232] In the embodiments described above, all or some of the steps can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, all or some of the steps can be implemented in the form of one or more computer programs or program elements. The computer programs reside (at least temporarily) in a memory of a computer during execution. The memory can be a RAM memory, a flash memory, a ROM memory, an EPROM memory, or any other suitable memory. The memory can be integral to or separate from the computer. The computer programs can be written in any suitable programming language, such as C, C++, Java, Visual Basic, etc. The computer programs can be written in assembly or machine language, if desired. The computer programs can be distributed over network coupled file servers, or can be distributed by any other suitable means.

[0233] It is to be understood that the terminology "first", "second" and the like used throughout this specification is merely intended to differentiate one entity or operation from another, and does not imply or suggest any actual relationship or order between such entities or operations. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "comprises", "comprising", or any other variations thereof, do not exclude the presence of additional elements in a process, method, article, or apparatus that comprises the elements listed.

[0234] Each of the embodiments described in the specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device, electronic device, and computer-readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0235] The above merely provides the preferred embodiment of the present application, and not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of pupil and iris positioning, characterized by, The method comprises: obtaining a target eye image; inputting the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions, wherein the target detection network model is pre-trained based on sample data, and the sample data comprises sample eye images labeled with regions where pupils and irises are located; determining, from each of the candidate iris regions, a target iris region whose size ratio to the corresponding candidate pupil region is within a preset range; determining the target iris region as a region where an iris is located in the target eye image, and determining the candidate pupil region corresponding to the target iris region as a region where a pupil is located in the target eye image; the step of inputting the target eye image into the pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions comprises: inputting the target eye image into the pre-trained target detection network model to obtain a plurality of detected iris regions output by the target detection network model and a detected pupil region corresponding to each of the detected iris regions; performing deduplication on each of the detected iris regions and / or each of the detected pupil regions to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions, and / or a plurality of candidate pupil regions and a candidate iris region corresponding to each of the candidate pupil regions; the method further comprises: determining a larger value between a length and a width of the target iris region as a first maximum dimension; determining a center and / or a radius of an iris in the target eye image according to a position of the target iris region and the first maximum dimension; determining a larger value between a length and a width of the target pupil region as a second maximum dimension, the target pupil region being the candidate pupil region corresponding to the target iris region; determining a center and / or a radius of a pupil in the target eye image according to a position of the target pupil region and the second maximum dimension; the target detection network model is pre-trained based on sample data by using the following steps, which comprise: inputting the sample data into a to-be-trained detection network model to obtain a plurality of regression regions and a classification result corresponding to each of the regression regions, the classification result being used to represent an iris or a pupil; determining a region overlap degree between each of the regression regions and a region labeled in the sample data, and a category difference between a category represented by the classification result corresponding to each of the regression regions and a category labeled in the sample data for the regression region; determining a model loss according to the overlap degree and the category difference, the model loss being negatively correlated with the overlap degree and positively correlated with the category difference; adjusting network parameters of the to-be-trained detection network model based on the model loss to obtain a target detection network model.

2. The method of claim 1, wherein, the step of performing deduplication on each of the detected iris regions and / or each of the detected pupil regions comprises: determining a target detection iris region from the iris region set, wherein the iris region set initially comprises each of the detection iris regions; determining a duplicate iris region from the iris region set, wherein the duplicate iris region overlaps with the target detection iris region to a degree greater than a preset overlap threshold; if the preset loop condition is not met, taking the iris region set without the duplicate iris region and the target detection iris region as a new iris region set, and returning to execute the step of determining the target detection iris region from the iris region set; if the preset loop condition is met, determining the detection iris region that is not determined as the duplicate iris region as a candidate iris region, obtaining a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions; and / or determining a target detection pupil region from the pupil region set, wherein the pupil region set initially comprises each of the detection pupil regions; determining a duplicate pupil region from the pupil region set, wherein the duplicate pupil region overlaps with the target detection pupil region to a degree greater than a preset overlap threshold; if the preset loop condition is not met, taking the pupil region set without the duplicate pupil region and the target detection pupil region as a new pupil region set, and returning to execute the step of determining the target detection pupil region from the pupil region set; if the preset loop condition is met, determining the detection pupil region that is not determined as the duplicate pupil region as a candidate pupil region, obtaining a plurality of candidate pupil regions and a candidate iris region corresponding to each of the candidate pupil regions.

3. A pupil and iris positioning device, characterized in that The device comprises: an image acquisition module configured to acquire a target eye image; a target detection module configured to input the target eye image into a pre-trained target detection network model to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions, wherein the target detection network model is pre-trained based on sample data, and the sample data comprises sample eye images with regions of pupils and irises labeled; a target iris region determination module configured to determine a target iris region from each of the candidate iris regions, wherein a size ratio of the target iris region to the corresponding candidate pupil region is within a preset range; an iris-pupil region determination module configured to determine the target iris region as a region of an iris in the target eye image, and determine the candidate pupil region corresponding to the target iris region as a region of a pupil in the target eye image; the target detection module comprises: a detection region output unit configured to input the target eye image into a pre-trained target detection network model to obtain a plurality of detection iris regions output by the target detection network model and a detection pupil region corresponding to each of the detection iris regions; The detection region deduplication unit is configured to deduplicate each of the detection iris region and / or each of the detection pupil region to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions, and / or a plurality of candidate pupil regions and a candidate iris region corresponding to each of the candidate pupil regions. The device further comprises: The first maximum size determination module is configured to determine a larger value between a length and a width of the target iris region as a first maximum size. The iris center and / or radius determination module is configured to determine a center and / or a radius of an iris in the target eye image according to a position of the target iris region and the first maximum size. The second maximum size determination module is configured to determine a larger value between a length and a width of the target pupil region as a second maximum size, the target pupil region being the candidate pupil region corresponding to the target iris region. The pupil center and / or radius determination module is configured to determine a center and / or a radius of a pupil in the target eye image according to a position of the target pupil region and the second maximum size. The device further comprises: The model training module is configured to pre-train the target detection network model based on sample data by using the following steps: The sample data is input into a detection network model to be trained to obtain a plurality of regression regions and a classification result corresponding to each of the regression regions, the classification result being used to represent an iris or a pupil. An area overlap degree between each of the regression regions and an area labeled in the sample data and a category difference between a category represented by each of the classification results and a category labeled in the sample data is determined. A model loss is determined according to the overlap degree and the category difference, the model loss being negatively correlated with the overlap degree and positively correlated with the category difference. Network parameters of the detection network model to be trained are adjusted based on the model loss to obtain the target detection network model.

4. The apparatus of claim 3, wherein, The detection region deduplication unit is configured to deduplicate each of the detection iris region and / or each of the detection pupil region to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions, and / or a plurality of candidate pupil regions and a candidate iris region corresponding to each of the candidate pupil regions, and the detection region deduplication unit comprises: A target detection iris region with a confidence satisfying a preset confidence condition is determined from a set of iris regions, wherein the set of iris regions initially comprises each of the detection iris regions. A duplicate iris region with an overlap degree greater than a preset overlap threshold with the target detection iris region is determined in the set of iris regions. If a preset loop condition is not satisfied, the set of iris regions from which the duplicate iris region and the target detection iris region are removed is taken as a new set of iris regions, and the step of determining the target detection iris region from the set of iris regions is performed again. If the preset loop condition is satisfied, the detection iris region that is not determined as the duplicate iris region is determined as a candidate iris region to obtain a plurality of candidate iris regions and a candidate pupil region corresponding to each of the candidate iris regions. and / or determining a target detected pupil region from a set of pupil regions, wherein the set of pupil regions initially comprises each of the detected pupil regions, and wherein the target detected pupil region has a confidence value satisfying a preset confidence condition; determining a duplicate pupil region from the set of pupil regions, wherein the duplicate pupil region has an overlap degree with the target detected pupil region greater than a preset overlap threshold; if a preset loop condition is not satisfied, taking the set of pupil regions excluding the duplicate pupil region and the target detected pupil region as a new set of pupil regions, and returning to the step of determining a target detected pupil region from a set of pupil regions; if the preset loop condition is satisfied, determining the detected pupil region not determined as the duplicate pupil region as a candidate pupil region, and obtaining a plurality of candidate pupil regions and a candidate iris region corresponding to each of the candidate pupil regions.

5. An electronic device, comprising: comprising: a memory for storing a computer program; a processor for executing the program stored in the memory, and implementing the method of any one of claims 1-2.

6. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to implement the method of any one of claims 1-2.

Images