Pupil center positioning method and device and electronic equipment

Abstract

The invention provides a pupil center positioning method and device and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps: obtaining a target pupil image; performing gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filtering image; taking the MASK image of the target pupil image as a mask to execute mask operation to obtain a pupil position candidate image, filtering each circular arc contour in the pupil position candidate image to obtain a candidate contour, fitting the circle center of the candidate contour to obtain a circle center-containing set, filtering the circle centers in the circle center set to obtain a target circle center, and obtaining the target circle center. And determining the target circle center as the pupil center. According to the pupil center positioning method and device and the electronic equipment, multiple filtering modes are adopted in the whole positioning process, the pupil center can be accurately obtained, and meanwhile high accuracy can still be achieved under the conditions of poor image contrast, shielding, deformation and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pupil center positioning method, device and electronic equipment. Background Art

[0002] Eye center localization is a challenging problem faced by many computer vision applications. The effects usually arise from the variability of the scene, e.g., a wide range of shapes, lighting conditions, viewpoints, and occlusions.

[0003] Nowadays, there is a growing interest in deep neural networks, and thus, deep neural network based eye localizers are widely used for eye center localization.

[0004] However, the application of deep neural networks requires a large amount of training data. Therefore, obtaining high-quality training data to train eye locators is a difficult and challenging task.

[0005] Typically, training data requires AI labels to mark eye data, which are often subject to human errors and inevitably involve large amounts of labeling of eye data. An important issue is the reliance on labeled data, which is expensive to acquire and prone to errors, thus affecting the accuracy of eye positioning and tracking. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a method, device and electronic device for locating the center of pupil, which can improve the accuracy of eye positioning and tracking, thereby alleviating the above-mentioned technical problems.

[0007] In a first aspect, an embodiment of the present invention provides a method for locating a pupil center, the method comprising: acquiring a target pupil image; performing gradient calculation on the target pupil image according to a preconfigured gradient direction to obtain a gradient filter map corresponding to the target pupil image; using the MASK map of the target pupil image as a mask, performing a mask operation on the gradient filter map to obtain a pupil position candidate map, wherein the pupil position candidate map contains at least one arc contour; filtering each of the arc contours in the pupil position candidate map to obtain at least one candidate contour, and fitting the center of each candidate contour to obtain a center set containing each candidate contour; filtering the centers of circles in the center set according to a preconfigured filtering principle to obtain a target center of circle, and determining the target center of circle as the pupil center.

[0008] In combination with the first aspect, an embodiment of the present invention provides a first possible implementation of the first aspect, wherein the above-mentioned step of performing gradient calculation on the target pupil image according to a preconfigured gradient direction to obtain a gradient filter map corresponding to the target pupil image includes: performing gradient processing on the target pupil image according to a preconfigured gradient direction to obtain a first gradient map and a second gradient map corresponding to the target; wherein the number of gradient processing times of the second gradient map is greater than that of the first gradient map; and performing threshold filtering processing on the second gradient map based on the first gradient map to obtain a gradient filter map corresponding to the target pupil image.

[0009] In combination with the first possible implementation of the first aspect, an embodiment of the present invention provides a second possible implementation of the first aspect, wherein the above-mentioned first gradient map is a gradient map obtained by a single gradient processing, and the second gradient map is a gradient map obtained by a secondary gradient processing; based on the first gradient map, the second gradient map is subjected to threshold filtering processing to obtain the gradient filter map corresponding to the target pupil image, comprising: based on each pixel point in the first gradient map, the pixel points at the corresponding position in the second gradient map are filtered according to a preset pixel threshold to obtain the gradient filter map corresponding to the target pupil image.

[0010] In combination with the first aspect, an embodiment of the present invention provides a third possible implementation of the first aspect, wherein the above-mentioned step of filtering each of the arc contours in the pupil position candidate map to obtain at least one candidate contour includes: extracting each of the arc contours contained in the pupil position candidate map, and the coordinate information of the key points of each of the arc contours; determining whether the coordinate information of the key points meets the pre-set coordinate filtering conditions; if so, determining the corresponding arc contour as the candidate contour.

[0011] In combination with the third possible implementation of the first aspect, an embodiment of the present invention provides a fourth possible implementation of the first aspect, wherein the key points of the above-mentioned arc contour include the upper left point, the middle point and the upper right point of the arc; the step of judging whether the coordinate information of the key point meets the preset coordinate filtering condition includes: determining the minimum circumscribed rectangle of the arc contour and the center point of the minimum circumscribed rectangle based on the upper left point, the middle point and the upper right point; judging whether the coordinate information of the upper left point, the middle point, the upper right point and the center point all meet the coordinate filtering condition; if so, determining that the coordinate information of the key point meets the preset coordinate filtering condition; wherein the coordinate filtering condition includes: the distance between the upper left point and the upper right point is greater than a preset distance threshold; the horizontal coordinate of the upper left point is less than the horizontal coordinate of the center point; the horizontal coordinate of the upper right point is greater than the horizontal coordinate of the center point; the vertical coordinate of the upper left point is greater than the vertical coordinate of the center point; the vertical coordinate of the upper right point is greater than the vertical coordinate of the center point; the vertical coordinate of the middle point is less than the vertical coordinate of the center point;

[0012] In combination with the first aspect, an embodiment of the present invention provides a fifth possible implementation of the first aspect, wherein the step of filtering the centers of circles in the center set according to a preconfigured filtering principle to obtain a target center of circle includes: judging based on the target pupil image whether there is a target bright spot within a preset range corresponding to the center of circle, wherein the target bright spot is determined based on the color parameters of the target pupil image; if so, determining that the center of circle is the target center of circle.

[0013] In combination with the first aspect, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein the target pupil image is iteratively filtered according to a pre-configured multi-scale dilated convolution kernel until the number of iterations reaches a preset threshold number, thereby obtaining a MASK map of the target pupil image.

[0014] In a second aspect, an embodiment of the present invention further provides a pupil center positioning device, the device comprising: an acquisition module, used to acquire a target pupil image; a gradient calculation module, used to perform gradient calculation on the target pupil image according to a preconfigured gradient direction, and obtain a gradient filter map corresponding to the target pupil image; a mask calculation module, used to perform a mask operation on the gradient filter map using the MASK map of the target pupil image as a mask, and obtain a pupil position candidate map, wherein the pupil position candidate map contains at least one arc contour; a filtering module, used to filter each of the arc contours in the pupil position candidate map, obtain at least one candidate contour, and fit the center of each candidate contour to obtain a center set containing each candidate contour; a positioning module, used to filter the centers of circles in the center set according to a preconfigured filtering principle, obtain a target center of circle, and determine the target center of circle as the pupil center.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the steps of the method described in the first aspect.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] A pupil center positioning method, device and electronic device provided by an embodiment of the present invention can obtain a target pupil image; perform gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filter image corresponding to the target pupil image; use the MASK image of the target pupil image as a mask, perform a mask operation on the gradient filter image to obtain a pupil position candidate image, then filter each arc contour in the pupil position candidate image to obtain at least one candidate contour, and fit the center of each candidate contour to obtain a center set containing each candidate contour; finally, filter the centers of circles in the center set according to a pre-configured filtering principle to obtain a target center, and determine the target center as the pupil center. The entire positioning process adopts a variety of filtering methods, which can accurately obtain the pupil center, and at the same time, can still have high accuracy under conditions of poor image contrast, occlusion, deformation, etc.

[0019] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of a method for locating the pupil center provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a target pupil image provided by an embodiment of the present invention;

[0024] Figure 3 A MASK diagram provided by an embodiment of the present invention;

[0025] Figure 4 A flowchart of another method for locating the pupil center provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a first gradient map provided by an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of a gradient filtering graph provided by an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of a pupil position candidate map provided by an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of an arc profile provided by an embodiment of the present invention;

[0030] Fig. 9 A schematic diagram of the structure of a pupil center positioning device provided by an embodiment of the present invention;

[0031] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] Currently, in the relevant technologies, traditional human eye positioning methods, circular corner point detection, circular template matching and other algorithms can all realize the detection of pupil center. However, these algorithms are not satisfactory in terms of time efficiency or accuracy in scene adaptability. Therefore, there is currently no efficient and accurate method to obtain the pupil center.

[0034] Based on this, a pupil center positioning method, device and electronic device provided in an embodiment of the present invention can improve the accuracy of eye positioning and tracking, thereby alleviating the above-mentioned technical problems.

[0035] To facilitate understanding of this embodiment, a method for locating the pupil center disclosed in an embodiment of the present invention is first introduced in detail.

[0036] In a possible implementation, the embodiment of the present invention provides a method for locating the pupil center, such as Figure 1 A flow chart of a method for locating the pupil center is shown, the method comprising the following steps:

[0037] Step S102, obtaining a target pupil image;

[0038] The target pupil image in the embodiment of the present invention is an image containing the pupil, which is usually obtained by preprocessing the original image.

[0039] In actual use, the preprocessing process can be used to remove abnormal points in the original image and adjust the contrast to obtain a relatively stable pupil image.

[0040] Specifically, an image containing only the pupil part can be cut out from the original image, and then preprocessed, and the preprocessing process in the embodiment of the present invention is mainly a process of normalizing the image. For example, the image is normalized to the minimum and maximum values, wherein the maximum and minimum values ​​can be obtained based on the histogram of the original image, for example, the pixel values ​​at the cumulative number of pixels at both ends of the histogram of 0.05 and 0.95 are respectively used as the minimum and maximum values, and then the image containing the pupil part is normalized to obtain the target pupil image in the embodiment of the present invention.

[0041] For ease of understanding, Figure 2A schematic diagram of a target pupil image is shown, wherein: Figure 2 (a) in the figure is the pupil image before normalization. Figure 2 (b) in FIG. 1 is the pupil image after normalization, that is, the target pupil image in the embodiment of the present invention.

[0042] Step S104, performing gradient calculation on the target pupil image according to the pre-configured gradient direction to obtain a gradient filter image corresponding to the target pupil image;

[0043] Among them, in the embodiment of the present invention, the structural characteristics from light to dark from the cornea to the pupil are taken into consideration. Therefore, the target pupil image is convolved with a gradient matrix along the Y axis from top to bottom or from bottom to top to obtain a gradient filter map.

[0044] Step S106, using the MASK image of the target pupil image as a mask, performing a mask operation on the gradient filter image to obtain a pupil position candidate image;

[0045] Wherein, the pupil position candidate map in the embodiment of the present invention includes at least one arc contour;

[0046] Specifically, in this step, the process of mask operation is to remove the noise area, that is, to remove the noise points contained in the gradient filter image, and then obtain the pupil position candidate image.

[0047] Step S108, filtering each arc contour in the pupil position candidate map to obtain at least one candidate contour, and fitting the center of each candidate contour to obtain a set of centers including each candidate contour;

[0048] Step S110, filtering the circle centers in the circle center set according to a pre-configured filtering principle to obtain a target circle center, and determining the target circle center as the pupil center.

[0049] In actual use, considering the facial structure features of the eyes and the areas around the eyes, after the calculations in the above steps, multiple tracks or contours may appear, and the pupil contour will also be included. Therefore, by filtering the arc contour, a portion of the non-pupil contour can be filtered out. The above process of filtering the center of the circle further determines the validity of the center of the circle, thereby making the final target center of the circle more accurate.

[0050] Therefore, a pupil center positioning method provided by an embodiment of the present invention can obtain a target pupil image; perform gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filter image corresponding to the target pupil image; use the MASK image of the target pupil image as a mask, perform a mask operation on the gradient filter image to obtain a pupil position candidate image, then filter each arc contour in the pupil position candidate image to obtain at least one candidate contour, and fit the center of each candidate contour to obtain a center set containing each candidate contour; finally, filter the centers of circles in the center set according to a pre-configured filtering principle to obtain a target center, and determine the target center as the pupil center. The entire positioning process adopts a variety of filtering methods, which can accurately obtain the pupil center, and at the same time, it can still have high accuracy under conditions of poor image contrast, occlusion, deformation, etc.

[0051] In actual use, the MASK image used in the above step S106 is usually obtained by performing a convolution operation on the target pupil image obtained after the normalization process. Specifically, in an embodiment of the present invention, the target pupil image can be iteratively filtered according to a pre-configured multi-scale dilated convolution kernel until the number of iterations reaches a preset number threshold, thereby obtaining the MASK image of the target pupil image.

[0052] In the specific implementation, the target pupil image is filtered each time using a multi-scale dilated convolution kernel, and the output after each filtering is used as the input of the next iteration until the number of iterations reaches a preset threshold, and finally a MASK image containing only the flat area of ​​the image is obtained.

[0053] The multi-scale dilated convolution kernel used in the embodiment of the present invention can be expressed as:

[0054]

[0055]

[0056] Among them, kernel is a multi-scale dilated convolution kernel, and level is the number of iterations;

[0057] b is the gradient kernel, usually expressed as b = {0.0625, 0.25, 0.375, 0.25, 0.0625}0; step is the step size, usually expressed as step = pow(2, level-1)-1, pow represents the function used to calculate the power of the value, len is the length and width of the multi-scale hole convolution kernel, expressed as: len = 5 + step * 4, ii is the subscript of b.

[0058] further, Figure 3A MASK image obtained after the multi-scale dilated convolution kernel filtering process is shown, and the MASK image is used for the mask operation in the above step S106.

[0059] Furthermore, the gradient calculation process in the above step S104 in the embodiment of the present invention includes a secondary gradient calculation and a threshold filtering process, thereby obtaining a final gradient filtering graph.

[0060] For ease of understanding, Figure 1 On the basis of Figure 4 FIG. 1 is a flowchart of another method for locating the pupil center, and further describes the gradient calculation process and the subsequent arc contour filtering process. Figure 4 As shown, the following steps are included:

[0061] Step S402, obtaining a target pupil image;

[0062] The target pupil image in the embodiment of the present invention is an image containing the pupil, which is usually obtained by preprocessing the original image.

[0063] Step S404, performing gradient processing on the target pupil image according to a pre-configured gradient direction to obtain a first gradient map and a second gradient map corresponding to the target;

[0064] The second gradient map has a greater number of gradient processing times than the first gradient map;

[0065] Step S406, performing threshold filtering on the second gradient image based on the first gradient image to obtain a gradient filtering image corresponding to the target pupil image;

[0066] In specific implementation, in an embodiment of the present invention, the first gradient map is a gradient map obtained by a single gradient process, and the second gradient map is a gradient map obtained by a secondary gradient process; it should be understood that the number of gradient processes of the first gradient map and the second gradient map can also be set according to actual usage, and the embodiment of the present invention does not limit this.

[0067] Furthermore, for the first gradient map obtained by the above-mentioned gradient processing, a matrix convolution calculation can usually be performed on the target pupil image along the Y-axis from top to bottom or from bottom to top gradient direction, wherein in the embodiment of the present invention, the gradient direction from bottom to top is taken as an example for explanation.

[0068] Moreover, in the embodiment of the present invention, the gradient matrix used in the gradient processing is expressed as

[0069] or,

[0070] Further, for ease of understanding, Figure 5A schematic diagram of a first gradient map is also shown, and Figure 6 A schematic diagram of a gradient filtering graph is shown.

[0071] in, Figure 6 Yes Figure 5 The first gradient image shown is subjected to secondary gradient processing to obtain a second gradient image, and the second gradient image is subjected to threshold filtering processing based on the first gradient image to obtain a gradient filtered image.

[0072] Specifically, in an embodiment of the present invention, when the second gradient map is threshold filtered based on the first gradient map, the pixel points at the corresponding position in the second gradient map are filtered according to a preset pixel threshold based on each pixel point in the first gradient map to obtain a gradient filter map corresponding to the target pupil image.

[0073] Among them, the process of the last secondary filtration can be expressed as the following formula:

[0074]

[0075] in, They are Figure 5 The first gradient map and Figure 6 In the gradient filter graph shown, each coordinate (i, j) corresponds to the pixel value of the position, th is the pixel threshold, which is used to find the threshold of the edge of the gradient filter graph. col represents the column in the image, corresponding to Figure 5 or Figure 6 The column of pixels in the middle, row means Figure 5 or Figure 6 The row of pixels in .

[0076] Furthermore, for the above gradient filter image, it is necessary to further perform the following masking operation to remove the noise area.

[0077] Step S408, using the MASK image of the target pupil image as a mask, performing a mask operation on the gradient filter image to obtain a pupil position candidate image;

[0078] Among them, the MASK diagram at this time is the above Figure 3 In actual use, in the above step S408, when performing mask operation, it is necessary to perform a "logical AND" operation on each pixel point in the MASK image and the gradient filter image. For example, when the pixel value (or RGB value) of a certain pixel point in the MASK image and the gradient filter image is greater than the preset threshold, the pixel point is retained, otherwise the pixel point is removed.

[0079] After the processing of step S408, the noise area can be removed to obtain a pupil position candidate map. Figure 7 A schematic diagram of a pupil position candidate map is shown, Figure 6 and Figure 7 It can be seen that Figure 7 The pupil position candidate map shown effectively removes the noise area.

[0080] Step S410, extracting each arc contour contained in the pupil position candidate map, and the coordinate information of the key points of each arc contour;

[0081] Step S412, determining whether the coordinate information of the key point meets the preset coordinate filtering condition; if so, determining the corresponding arc contour as a candidate contour;

[0082] In specific implementation, the key points of the arc contour in the embodiment of the present invention include the upper left point, the middle point and the upper right point of the arc; therefore, in the above step S410, when extracting the arc contour and the coordinate information of the key points of the arc contour, the upper left point p of each arc contour can be extracted. l (x,y), middle point p m (x,y), upper right point p r (x, y), and determine the minimum bounding rectangle of the arc contour based on the upper left point, the middle point and the upper right point, as well as the center point of the minimum bounding rectangle. The coordinate information of the center point of the minimum bounding rectangle is represented by p c (x,y).

[0083] For ease of understanding, Figure 8 shows a schematic diagram of an arc profile, and, Figure 8 Also shown in FIG. 1 is the minimum circumscribed rectangle of the arc contour, as well as the key points and the center point of the minimum circumscribed rectangle, i.e., the upper left point p l (x,y), middle point p m (x,y), upper right point p r (x,y) and the center point p c (x,y).

[0084] Further, in the above step S412, when judging whether the coordinate information of the key point meets the preset coordinate filtering conditions, it is necessary to judge whether the coordinate information of the upper left point, middle point, upper right point and center point all meet the coordinate filtering conditions; if so, it is determined that the coordinate information of the key point meets the preset coordinate filtering conditions; if one does not meet the coordinate filtering conditions, the arc contour is discarded.

[0085] The coordinate filtering conditions in the embodiment of the present invention include:

[0086] The distance between the upper left point and the upper right point is greater than the preset distance threshold;

[0087] The horizontal coordinate of the upper left point is smaller than that of the center point;

[0088] The horizontal coordinate of the upper right point is greater than that of the center point;

[0089] The ordinate of the upper left point is greater than the ordinate of the center point;

[0090] The ordinate of the upper right point is greater than the ordinate of the center point;

[0091] The ordinate of the middle point is smaller than the ordinate of the center point.

[0092] Corresponding to the upper left point p l (x,y), middle point p m (x,y), upper right point p r (x,y) and the center point p c (x, y) coordinate information, the above coordinate filtering condition can be expressed as:

[0093]

[0094] Among them, index l 、index r The upper left point p l (x,y), upper right point p r (x, y) is the subscript on the contour, and th0 is the set minimum arc length of the pupil, that is, the distance between the upper left point and the upper right point is greater than the preset minimum arc length of the pupil. Respectively represent the horizontal coordinates of the coordinate information of the upper left point, upper right point, middle point and center point, Respectively represent the vertical coordinates in the coordinate information of the upper left point, upper right point, middle point and center point.

[0095] Furthermore, based on the above coordinate filtering conditions, an upward concave contour is finally obtained, and the upward concave contour is the candidate contour in the embodiment of the present invention.

[0096] Step S414, fitting the center of each candidate contour to obtain a set of centers including each candidate contour;

[0097] Step S416, filtering the circle centers in the circle center set according to a pre-configured filtering principle to obtain a target circle center, and determining the target circle center as the pupil center.

[0098] In actual use, refer to Figure 2 As can be seen from Figure (b) in the figure, in the target pupil image, the position of the pupil center is a highlight point. Therefore, in the above step S416, when filtering the center of the circle in the center set, it can be determined based on the target pupil image whether there is a target bright spot within the preset range corresponding to the center of the circle, wherein the target bright spot is determined based on the color parameters of the target pupil image; if so, the center of the circle is determined to be the target center of the circle, and then the target center of the circle is determined to be the pupil center.

[0099] In specific implementation, the preset range corresponding to the center of the circle can be configured according to the radius r of the pupil, and the brightness threshold of the target bright spot can be set, such as the pixel value (or RGB value greater than 200), and then it is determined whether there is a bright spot in the preset range corresponding to the center of the circle, that is, a point greater than the brightness threshold of the target bright spot. If so, it means that the center of the circle is valid and is further determined as the center of the pupil. In addition, based on the filtering conditions of each of the above steps, the method for locating the center of the pupil provided in the embodiment of the present invention can ensure that an accurate center of the pupil is obtained, and still has high accuracy in the case of poor image contrast, occlusion, deformation, etc.

[0100] In summary, in the embodiment of the present invention, a MASK image of the smooth area corresponding to the target pupil image is generated, and then the pupil edge is detected by gradient processing, and the noise area is filtered out using the MASK image as a mask. Next, arc filtering detection is performed on the arc contour contained in the pupil position candidate image after filtering by the MASK image, and the center position of the circle is located. Finally, the correct pupil center is determined in combination with the highlight feature of the pupil center of the original image. The whole process efficiently solves various problems caused by incomplete pupil shooting, occlusion, large brightness changes, and inconsistent contrast, while ensuring that the pupil center detection is highly accurate and real-time.

[0101] Further, based on the above embodiment, the embodiment of the present invention also provides a pupil center positioning device, such as Fig. 9 A schematic diagram of the structure of a pupil center positioning device is shown, the device comprising:

[0102] An acquisition module 90, used for acquiring a target pupil image;

[0103] A gradient calculation module 92, used to perform gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filter image corresponding to the target pupil image;

[0104] A mask calculation module 94 is used to perform a mask operation on the gradient filter image using the MASK image of the target pupil image as a mask to obtain a pupil position candidate image, wherein the pupil position candidate image includes at least one arc contour;

[0105] A filtering module 96, configured to filter each of the arc contours in the pupil position candidate map to obtain at least one candidate contour, and fit the center of each candidate contour to obtain a center set containing each candidate contour;

[0106] The positioning module 98 is used to filter the centers of circles in the center set according to a pre-configured filtering principle to obtain a target center of circle, and determine the target center of circle as the pupil center.

[0107] The pupil center positioning device provided in the embodiment of the present invention has the same technical features as the pupil center positioning method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0108] Furthermore, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0109] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are executed.

[0110] Furthermore, an embodiment of the present invention also provides a structural diagram of an electronic device, such as Fig.10 As shown, it is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 101 and a memory 100, the memory 100 stores computer executable instructions that can be executed by the processor 101, and the processor 101 executes the computer executable instructions to implement the above method.

[0111] exist Fig.10 In the illustrated embodiment, the electronic device further includes a bus 102 and a communication interface 103 , wherein the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .

[0112] Among them, the memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 102 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 102 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0113] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 101 or the instruction in the form of software. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 101 reads the information in the memory and completes the above method in combination with its hardware.

[0114] The pupil center positioning method, device and computer program product of the electronic device provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0116] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0117] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0118] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0119] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for locating the pupil center, characterized in that: The method comprises: Acquire a target pupil image; Performing gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filter image corresponding to the target pupil image; Using the MASK image of the target pupil image as a mask, performing a mask operation on the gradient filter image to obtain a pupil position candidate image, wherein the pupil position candidate image includes at least one arc contour; Filtering each of the arc contours in the pupil position candidate map to obtain at least one candidate contour, and fitting the center of each candidate contour to obtain a center set containing each candidate contour; The centers of circles in the circle center set are filtered according to a pre-configured filtering principle to obtain a target center of circle, and the target center of circle is determined as the pupil center.

2. The method according to claim 1, characterized in that The step of performing gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filter map corresponding to the target pupil image comprises: Performing gradient processing on the target pupil image according to a pre-configured gradient direction to obtain a first gradient map and a second gradient map corresponding to the target; wherein the number of gradient processing times of the second gradient map is greater than that of the first gradient map; The second gradient map is subjected to threshold filtering based on the first gradient map to obtain a gradient filtering map corresponding to the target pupil image.

3. The method according to claim 2, characterized in that The first gradient map is a gradient map obtained by a primary gradient process, and the second gradient map is a gradient map obtained by a secondary gradient process; The step of performing threshold filtering on the second gradient map based on the first gradient map to obtain a gradient filtering map corresponding to the target pupil image includes: Based on each pixel point in the first gradient map, the pixel points at the corresponding position in the second gradient map are filtered according to a preset pixel threshold to obtain a gradient filter map corresponding to the target pupil image.

4. The method according to claim 1, characterized in that: The step of filtering each of the arc contours in the pupil position candidate map to obtain at least one candidate contour comprises: Extracting each of the circular arc contours contained in the pupil position candidate map, and coordinate information of key points of each of the circular arc contours; Determine whether the coordinate information of the key point meets the preset coordinate filtering condition; If yes, the corresponding arc contour is determined as the candidate contour.

5. The method according to claim 4, characterized in that The key points of the arc contour include the upper left point, the middle point and the upper right point of the arc; The step of determining whether the coordinate information of the key point meets a preset coordinate filtering condition comprises: Determine the minimum circumscribed rectangle of the arc contour and the center point of the minimum circumscribed rectangle based on the upper left point, the middle point and the upper right point; Determine whether the coordinate information of the upper left point, the middle point, the upper right point and the center point all meet the coordinate filtering condition; If yes, determining that the coordinate information of the key point satisfies a preset coordinate filtering condition; The coordinate filtering conditions include: The distance between the upper left point and the upper right point is greater than a preset distance threshold; The horizontal coordinate of the upper left point is smaller than the horizontal coordinate of the center point; The horizontal coordinate of the upper right point is greater than the horizontal coordinate of the center point; The ordinate of the upper left point is greater than the ordinate of the center point; The ordinate of the upper right point is greater than the ordinate of the center point; The longitudinal coordinate of the intermediate point is smaller than the longitudinal coordinate of the central point.

6. The method according to claim 1, characterized in that The step of filtering the centers of circles in the circle center set according to a pre-configured filtering principle to obtain a target circle center includes: Determining whether there is a target bright spot within a preset range corresponding to the center of the circle based on the target pupil image, wherein the target bright spot is determined based on a color parameter of the target pupil image; If yes, the center of the circle is determined to be the target center of the circle.

7. The method according to claim 1, characterized in that The method further comprises: The target pupil image is iteratively filtered according to a pre-configured multi-scale dilated convolution kernel until the number of iterations reaches a preset threshold, thereby obtaining a MASK image of the target pupil image.

8. A pupil center positioning device, characterized in that: The device comprises: An acquisition module, used for acquiring a target pupil image; A gradient calculation module, used to perform gradient calculation on the target pupil image according to a pre-configured gradient direction to obtain a gradient filter image corresponding to the target pupil image; A mask calculation module, used for performing a mask operation on the gradient filter image using the MASK image of the target pupil image as a mask to obtain a pupil position candidate image, wherein the pupil position candidate image includes at least one arc contour; A filtering module, used for filtering each of the arc contours in the pupil position candidate map to obtain at least one candidate contour, and fitting the center of each candidate contour to obtain a center set containing each candidate contour; The positioning module is used to filter the centers of circles in the circle center set according to a pre-configured filtering principle to obtain a target center of circle, and determine the target center of circle as the pupil center.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are executed.

Images