Centrally localized pupil recognition methods, systems, media, products, and terminals

Abstract

This application provides a pupil recognition method, system, medium, product, and terminal based on center localization. It performs a horizontal gradient transformation operation on distortion-corrected human eye image data to generate eye gradient amplitude image data. Through gradient transformation, it extracts edge features of the pupil and iris regions, effectively separating the pupil edge from complex eye structures such as eyelid occlusion, eyelash projection, and iris pigment spots / crypts, avoiding misidentification of these interferences as pupil edges and improving the accuracy of pupil edge feature recognition. Combining the distortion-corrected human eye image data and eye gradient amplitude image data, it performs a pupil template matching operation based on the Hough circle detection algorithm. By leveraging the edge features of the gradient amplitude image, it enhances the matching accuracy between the pupil edge and the template, effectively ensuring the continuity and integrity of pupil contour recognition, further improving the reliability and accuracy of pupil measurement results.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and particularly to a pupil recognition method, system, medium, product and terminal based on center positioning. Background Art

[0002] With the continuous development of refractive surgery technology, the industry has gradually entered the era of "personalization and precision". Surgical planning is no longer limited to single refractive correction, but has developed into a systematic project covering multiple parameters such as wavefront aberration, corneal shape, and pupil characteristics. In corneal refractive surgery, the femtosecond laser system has become an important surgical tool in current clinical applications due to its core advantages of high precision and minimally invasive.

[0003] To achieve precise surgical planning, the preoperative biometric devices supporting the femtosecond laser system need to comprehensively collect relevant parameters of the patient's eye, including a set of data such as corneal curvature, corneal thickness, anterior chamber depth, axial length, and pupil parameters. Among them, the measurement of pupil diameter is particularly crucial, and its measurement result directly determines the rationality of setting the effective optical zone of the surgery. If the surgical optical zone is smaller than the pupil diameter in the dark adaptation state, it is very likely to cause abnormal visual quality problems such as night glare and halos in patients after surgery, seriously affecting the surgical effect and the patient's postoperative visual experience.

[0004] Currently, the methods for obtaining pupil diameter clinically mainly rely on infrared video pupillometers or pupil measurement modules integrated in devices such as corneal topographers and wavefront aberrometers. The corresponding measurement algorithms mostly use simple region growing and threshold segmentation algorithms. In the ideal situation of good imaging quality and clear and complete pupil contour, they can meet the basic measurement requirements. However, in actual clinical applications, affected by various factors such as equipment and eye physiological structure, the existing measurement algorithms and means face many technical challenges, and the measurement accuracy is difficult to be effectively guaranteed, which is mainly reflected in the following aspects:

[0005] (1) Optical distortion interference: In clinical measurements, in order to obtain a larger eye field of view, wide-angle lenses are often used, and camera lenses (especially wide-angle lenses) are prone to introduce barrel or pincushion distortion. This distortion will directly distort the geometric shape of the pupil, thereby causing deviation in pupil diameter measurement.

[0006] (2) Structural interference: The complexity of the eye's own structure will interfere with the measurement. For example, situations such as partial occlusion of the pupil by the eyelid, the projection interference of long eyelashes, and the presence of non-uniform textures (such as pigment spots and crypts) in the iris region may all be misjudged as the pupil edge by the measurement algorithm, thus affecting the measurement accuracy.

[0007] (3) Reflection interference: The Purkinje reflection spot formed on the corneal surface will destroy the continuity of the pupil area imaging, resulting in errors in pupil contour recognition and affecting the accuracy of the measurement result. Summary of the Invention

[0008] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a pupil recognition method, system, medium, product and terminal based on center positioning, so as to solve the technical problem of inaccurate pupil diameter measurement in the prior art.

[0009] To achieve the above and other related objectives, a first aspect of this application provides a pupil recognition method based on center localization, comprising: acquiring distorted human eye image data; performing a horizontal gradient transformation operation based on the distorted human eye image data to generate eye gradient amplitude image data; and performing a pupil template matching operation based on the distorted human eye image data and the eye gradient amplitude image data, and using the Hough circle detection algorithm, to generate target pupil diameter data.

[0010] In some embodiments of the first aspect of this application, the distorted human eye image data includes a plurality of original pixels; wherein, the method of performing a horizontal gradient transformation operation based on the distorted human eye image data to generate eye gradient amplitude image data includes: performing a horizontal gradient transformation operation based on the distorted human eye image data and using a horizontal gradient operator to generate horizontal gradient data for each original pixel; constructing an eye image gradient matrix based on the horizontal gradient data of each original pixel; and performing a normalization operation on the eye image gradient matrix to generate eye gradient amplitude image data.

[0011] In some embodiments of the first aspect of this application, the method of generating target pupil diameter data by performing a pupil template matching operation based on the distorted human eye image data and the eye gradient amplitude image data, and based on the Hough circle detection algorithm, includes: performing a pupil center positioning operation based on the distorted human eye image data to generate effective pupil center position data; constructing multiple effective pupil edge template data based on the effective pupil center position data and a preset pupil physiological diameter threshold range; and performing a pupil template matching operation based on each of the effective pupil edge template data and the eye gradient amplitude image data to generate target pupil diameter data.

[0012] In some embodiments of the first aspect of this application, the method of performing a pupil center localization operation to generate effective pupil center position data based on the distorted human eye image data includes: performing an edge detection operation on the distorted human eye image data to generate binary image data of eye edge detection; performing a convolution operation on the distorted human eye image data to generate neighborhood gradient data of each original pixel; locking multiple non-zero pixels and the gradient direction of each non-zero pixel in the binary image data of eye edge detection based on the neighborhood gradient data of each original pixel; and performing a pupil center localization operation based on each non-zero pixel and the gradient direction of each non-zero pixel in the binary image data of eye edge detection to generate effective pupil center position data.

[0013] In some embodiments of the first aspect of this application, the method of performing a pupil center localization operation to generate valid pupil center position data based on each non-zero pixel in the binary image data of eye edge detection and the gradient direction of each non-zero pixel includes: constructing and initializing a circle center accumulator space; traversing each non-zero pixel in the binary image data of eye edge detection, and performing a line segment drawing operation based on the gradient direction of each non-zero pixel to lock multiple candidate pupil center position data, and generating voting data of the circle center accumulator space at each candidate pupil center position data; and performing a statistical sorting operation based on the voting data of the circle center accumulator space at each candidate pupil center position data to generate valid pupil center position data.

[0014] In some embodiments of the first aspect of this application, the method of generating target pupil diameter data by performing a pupil template matching operation based on each of the effective pupil edge template data and the eye gradient amplitude image data includes: performing a similarity calculation operation based on each of the effective pupil edge template data and the eye gradient amplitude image data to generate convolution response data between each of the effective pupil edge template data and the eye gradient amplitude image data; generating global response map data based on the convolution response data between each of the effective pupil edge template data and the eye gradient amplitude image data; and performing a pupil template matching operation based on the global response map data to generate target pupil diameter data.

[0015] To achieve the above and other related objectives, a second aspect of this application provides a pupil recognition system based on center positioning, comprising: a data acquisition module for acquiring distorted human eye image data; a horizontal gradient transformation module for performing a horizontal gradient transformation operation based on the distorted human eye image data to generate eye gradient amplitude image data; and a pupil diameter calculation module for performing a pupil template matching operation based on the distorted human eye image data and the eye gradient amplitude image data, and using the Hough circle detection algorithm, to generate target pupil diameter data.

[0016] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the center-based pupil recognition method as described above.

[0017] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the center-based pupil recognition method as described above.

[0018] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the center-based pupil recognition method as described above.

[0019] As described above, the pupil recognition method, system, medium, product, and terminal based on center positioning of this application have the following beneficial effects:

[0020] (1) Effectively eliminate optical distortion interference: The image acquisition operation is performed using the pupil camera that has been optically distorted to obtain the original human eye image containing optical distortion. The optical distortion calibration parameters are read from the control unit of the pupil camera, and the original human eye image is rectified to obtain the distortion-corrected human eye image data. This eliminates the distortion of the pupil geometry caused by the barrel and pincushion distortion introduced by the wide-angle lens in clinical measurement, reduces the measurement deviation caused by optical distortion interference, and thus improves the accuracy of pupil diameter measurement.

[0021] (2) Reduce interference from eye structure: Based on the distortion-corrected human eye image data, a horizontal gradient transformation operation is performed to generate eye gradient amplitude image data. The edge features of the pupil and iris regions are extracted through gradient transformation, and the pupil edge is effectively separated from complex eye structures such as eyelid occlusion, eyelash projection, iris pigment spots / crypts. This solves the problem of easily misjudging the above structural interference as the pupil edge, improves the accuracy of pupil edge feature recognition, and provides feature basis for subsequent template matching.

[0022] (3) Suppressing interference from reflected light spots: Combining the distorted human eye image data and the eye gradient amplitude image data, the pupil template matching operation is performed based on the Hough circle detection algorithm. The edge features of the eye gradient amplitude image data are fully utilized to enhance the matching accuracy between the pupil edge and the template. Since the reflected light spots have no effective gradient edge features, they cannot cause decisive interference to the template matching results. This effectively ensures the continuity and integrity of pupil contour recognition and further improves the reliability and accuracy of pupil measurement results. Attached Figure Description

[0023] Figure 1 The diagram shown is a flowchart of a pupil recognition method based on center positioning in one embodiment of this application.

[0024] Figure 2 The diagram shown is a flowchart illustrating the execution of a horizontal gradient transformation operation in one embodiment of this application.

[0025] Figure 3 The diagram shown is a flowchart of the Hough circle detection algorithm in one embodiment of this application.

[0026] Figure 4 The diagram shows a flowchart of the pupil center positioning operation in one embodiment of this application.

[0027] Figure 5 The diagram shows a flowchart of the pupil template matching operation performed in one embodiment of this application.

[0028] Figure 6 The diagram shown is a schematic block diagram of a pupil recognition system based on center positioning in one embodiment of this application.

[0029] Figure 7 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0030] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0031] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0032] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0033] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0034] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0035] <1> Canny: An edge detection operator used to accurately extract edge features from images.

[0036] <2> Sobel is a gradient calculation operator based on first-order derivatives, often used to extract edge gradient features in the horizontal and vertical directions of an image.

[0037] <3> Prewitt: A first-order differential gradient operator used to extract gradient information from image edges.

[0038] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 A flowchart illustrating the pupil recognition method based on center localization in this embodiment of the invention is shown. The pupil recognition method based on center localization in this embodiment mainly includes the following steps:

[0039] S101: Acquire the image data of the human eye after distortion correction.

[0040] In this embodiment, the method for acquiring distortion-corrected human eye image data includes:

[0041] (1) Use a pupil camera that has been optically distorted to perform image acquisition operations to obtain a raw human eye image containing optical distortion.

[0042] (2) Read the optical distortion calibration parameters from the control unit of the pupil camera, and perform a correction operation on the original human eye image based on the preset mathematical model to obtain the distorted human eye image data.

[0043] In this embodiment, after optical distortion correction of the pupil camera, the optical distortion calibration parameters of the pupil camera are obtained and stored in the pupil camera's control unit. Using the optically distorted pupil camera to acquire original human eye images, the control unit automatically calls the stored optical distortion calibration parameters and performs geometric correction of the pixel coordinates of the captured original human eye images using a preset mathematical model, thereby obtaining distorted human eye image data. This restores the distorted image to a distortion-free image consistent with the true physical state, thus eliminating barrel distortion, pincushion distortion, and tangential distortion caused by the lens.

[0044] In this embodiment, the method for optical distortion correction of the pupil camera includes:

[0045] (1) Based on the network fixture with preset parameters, use a pupil camera to collect the fixture distortion image.

[0046] In this embodiment, a network fixture with preset parameters is retrieved and placed within the field of view of the pupil camera, ensuring that the network fixture is parallel to the pupil camera's shooting plane and meets the calibration conditions. The pupil camera to be calibrated is used to photograph the network fixture, obtaining an image of the fixture containing optical distortion, i.e., the fixture distortion image.

[0047] (2) Perform a recognition operation on the tooling distortion image to extract the distortion coordinate data of each grid point.

[0048] In this embodiment, the network fixture includes multiple grid points. A recognition operation is performed on the distorted image of the fixture to extract the pixel coordinate data of each grid point in the distorted image, i.e., the distortion coordinate data of each grid point.

[0049] (3) Based on the distortion coordinate data of each grid point and the real physical coordinate data of each grid point, and based on the preset mathematical model, the optical distortion calibration parameters of the pupil camera are obtained.

[0050] In this embodiment, the actual physical coordinate data and distortion coordinate data of each grid point are substituted into a preset mathematical model. By fitting and solving the two sets of coordinate data, the optical distortion calibration parameters of the pupil camera are obtained. The optical distortion calibration parameters include, but are not limited to, radial distortion parameters, tangential distortion parameters, and intrinsic parameters. These optical distortion calibration parameters are then stored in the control unit of the pupil camera.

[0051] In this embodiment, the preset mathematical model is a camera optical distortion model that includes radial distortion and tangential distortion. Its model structure is fixed, and its parameters change with the distortion state of the camera lens.

[0052] In this embodiment, taking radial distortion as an example, the preset mathematical model includes:

[0053] Formula (1)

[0054] in, This represents the distorted coordinate data of grid points in the horizontal direction; x represents the distorted coordinate data of the grid point in the vertical direction; y represents the true physical coordinate data of the grid point in the horizontal direction; r² = x² + y² represents the square of the distance from the grid point to the image center. Indicates the first radial distortion coefficient; Indicates the second radial distortion coefficient; This represents the third radial distortion coefficient.

[0055] In this embodiment, k1, k2, and k3 are radial distortion coefficients, which are the core parameters that need to be solved for calibration. Among them, k1 usually dominates and is used to correct the main "barrel distortion" or "pincushion distortion". k2 and k3 are used to correct higher-order distortions.

[0056] In this embodiment, a network fixture with preset parameters is retrieved, and the pupil camera to be calibrated is used to capture an image of the fixture to obtain a distorted image of the fixture. The pixel coordinate data of each grid point in the distorted image of the fixture is extracted, that is, the distorted coordinate data of each grid point, which is the value in formula (1). , Since the actual physical coordinates of each grid point are known, i.e., x, y, and r in formula (I) are known quantities, generally each (x, y) corresponds to one ( , Based on the distorted coordinate data of each grid point and the actual physical coordinate data of each grid point, k1, k2, and k3 are obtained using formula (I).

[0057] In this embodiment, it should be noted that Formula (I) is merely one embodiment of the present invention, primarily used for explanation and illustration, and not for limiting the scope of protection of the present invention. In practical applications, camera distortion is diverse. The above explanation only takes radial distortion as an example. The form of distortion may be quite complex, and the formula is not limited to Formula (I). Multiple distortions can be unified into a specific mathematical formula, requiring specific formulas to be set according to a specific camera. However, the overall correction process remains unchanged: using a network tool with known parameters, establishing the coordinate correspondence between the distorted points and the true points, and then solving for the distortion coefficients.

[0058] S102: Based on the distorted human eye image data, perform a horizontal gradient transformation operation to generate eye gradient amplitude image data.

[0059] In this embodiment, as Figure 2 The diagram illustrates a flowchart of the horizontal gradient transformation operation performed in an embodiment of the present invention. The distorted human eye image data includes multiple original pixels; wherein, the method of performing a horizontal gradient transformation operation based on the distorted human eye image data to generate eye gradient amplitude image data includes:

[0060] S1021: Based on the distorted and corrected human eye image data, a horizontal gradient transformation operation is performed using the horizontal gradient operator to generate horizontal gradient data for each original pixel.

[0061] In this embodiment, a horizontal gradient transformation operation is performed on the distortion-corrected human eye image data to calculate the horizontal gradient data of each original pixel. A horizontal gradient operator (such as the Sobel operator or the Prewitt operator) is used for convolution to generate the horizontal gradient data of each original pixel. The calculation formula is as follows:

[0062] Formula (II)

[0063] in, This indicates the original pixels in the human eye image data that have undergone distortion correction. The horizontal gradient data; i represents the horizontal neighborhood deviation of the original pixel; I represents the distortion-corrected human eye image data; This represents the coordinate data of the original pixel in the horizontal direction; This represents the coordinate data of the original pixel in the vertical direction; This represents the convolution kernel used to calculate the gradient in the horizontal direction.

[0064] In this embodiment, to calculate the horizontal gradient data of the current original pixel, it is necessary to select a certain number of neighboring pixels in the horizontal direction of the current original pixel to perform convolution operations. Here, i represents the current original pixel. The deviation from the neighborhood in the horizontal direction. Since only the horizontal gradient data of the original pixel needs to be calculated, when performing convolution calculation using the above formula (II), only pixels in the same row are selected, and the vertical neighborhood is not involved. For example, when i=-1, it represents the original pixel adjacent to the left of the current original pixel. When i=0, it represents the current original pixel itself. i=1 represents the original pixel to the right of the current original pixel. . The convolution kernel is a one-dimensional vector, for example, Therefore, according to formula (ii), it is actually the current original pixel point. The gray value of the original pixel on the right is subtracted from the gray value of the original pixel on the left, and the result is the horizontal gradient data of the current original pixel.

[0065] S1022: Construct an eye image gradient matrix based on the horizontal gradient data of each of the original pixels.

[0066] In this embodiment, the horizontal gradient data of each original pixel is arranged according to the image coordinates to construct the eye image gradient matrix. Gradient matrix of eye image It is a matrix of the same size as the distorted human eye image data, the gradient matrix of the eye image. The element value at each position corresponds to the horizontal gradient data of the original pixel, representing the intensity of the grayscale change of that original pixel in the horizontal direction:

[0067] (1) When the element value is positive, it indicates that the original pixel changes from dark to bright from left to right (e.g., the left edge of the pupil: dark pupil → bright iris).

[0068] (2) When the element value is negative, it indicates that the original pixel changes from bright to dark from left to right (e.g., the right edge of the pupil: bright iris → dark pupil).

[0069] (3) The absolute value of the element indicates the degree of gray change of the original pixel in the horizontal direction, that is, the sharpness of the eye edge. The larger the absolute value, the higher the probability that the position is the eye edge.

[0070] S1023: Perform a normalization process on the eye image gradient matrix to generate eye gradient amplitude image data.

[0071] In this embodiment, the gradient matrix of the eye image is... The absolute value is taken or normalization is performed, i.e., standardization, to generate eye gradient amplitude image data. In this eye gradient amplitude image data, the left and right boundaries of the pupil will appear as two bright curves. Since there is a clear gray-level step feature between the pupil region and the iris region (the gray value of the pupil region is usually lower than that of the iris region), the boundary point between the two corresponds to the location of the maximum gray-level gradient. Therefore, the peak pixels in the eye gradient amplitude image data can be identified as candidate points for the horizontal boundary between the pupil and the iris.

[0072] It is worth noting that for distorted human eye image data, the horizontal gradient operator is used to perform a horizontal gradient transformation operation, which extracts only the gray-scale changes in the horizontal direction of the image and enhances edge features. This can effectively filter out irrelevant edge interference in the vertical or diagonal direction, such as eyelid edges, eyelashes, and light spots, and retain only the significant left and right horizontal boundaries between the pupil and iris. While reducing the amount of computation, it improves the matching degree with the geometric features of the pupil, thereby improving the stability and accuracy of pupil localization and diameter measurement.

[0073] S103: Based on the distorted human eye image data and the eye gradient amplitude image data, and using the Hough circle detection algorithm, perform a pupil template matching operation to generate target pupil diameter data.

[0074] In this embodiment, as Figure 3 The diagram illustrates the flowchart of the Hough circle detection algorithm in an embodiment of the present invention. The method for generating target pupil diameter data by performing pupil template matching based on the distorted human eye image data and the eye gradient amplitude image data, and using the Hough circle detection algorithm, includes:

[0075] S1031: Based on the distorted human eye image data, perform a pupil center localization operation to generate valid pupil center position data.

[0076] In this embodiment, as Figure 4 The diagram illustrates a flowchart of the pupil center localization operation in an embodiment of the present invention. The method for generating valid pupil center position data by performing the pupil center localization operation based on the distortion-corrected human eye image data includes:

[0077] S10311: Perform edge detection operation on the distortion-corrected human eye image data to generate binary image data of eye edge detection.

[0078] In this embodiment, the Canny operator is used to perform edge detection on the distortion-corrected human eye image data to obtain binary image data of eye edge detection. Pixels in the binary image data of eye edge detection are divided into edge pixels and non-edge pixels. Edge pixels are non-zero pixels with a pixel value of 1, while non-edge pixels are zero pixels with a pixel value of 0.

[0079] S10312: Perform a convolution operation on the distorted human eye image data to generate neighborhood gradient data for each original pixel.

[0080] In this embodiment, the Sobel operator is used to perform convolution operations on the distorted human eye image data in the horizontal and vertical directions to obtain the gradient magnitude and gradient direction information corresponding to each original pixel, thereby forming the neighborhood gradient data of each original pixel.

[0081] S10313: Based on the neighborhood gradient data of each of the original pixels, lock multiple non-zero pixels in the binary image data of eye edge detection and the gradient direction of each non-zero pixel.

[0082] In this embodiment, edge pixels with a pixel value of 1 in the binary image data of eye edge detection are determined as non-zero pixels. Then, based on the gradient information recorded in the neighborhood gradient data, the gradient direction corresponding to each non-zero pixel is matched to provide a directional basis for subsequent pupil center localization.

[0083] S10314: Based on each non-zero pixel in the binary image data of the eye edge detection and the gradient direction of each non-zero pixel, perform a pupil center localization operation to generate valid pupil center position data. The method includes:

[0084] (1) Construct and initialize the space of the centered accumulator.

[0085] In this embodiment, a centered accumulator space N(a, b) is constructed and initialized, with all N(a, b) values ​​set to 0. The centered accumulator space N(a, b) represents the number of times coordinates (a, b) are voted as the pupil center; initializing it to 0 indicates that all coordinate points have not yet been voted on. Here, 'a' represents the horizontal dimension coordinate of the centered accumulator space, its value range corresponding to the horizontal pixel coordinate range of the distorted human eye image data, used to represent the horizontal pixel coordinate of the pupil center to be detected. 'b' represents the vertical dimension coordinate of the centered accumulator space, its value range corresponding to the vertical pixel coordinate range of the distorted human eye image data, used to represent the vertical pixel coordinate of the pupil center to be detected.

[0086] (2) Traverse each non-zero pixel in the binary image data of eye edge detection, and perform line segment drawing operation based on the gradient direction of each non-zero pixel to lock multiple candidate pupil center position data, and generate voting data of the circle center accumulator space at each candidate pupil center position data.

[0087] In this embodiment, all non-zero pixels in the binary image data of eye edge detection are traversed. For each non-zero pixel, a line segment is drawn within the pixel radius corresponding to a preset normal pupil diameter threshold based on its gradient direction (the normal direction perpendicular to the edge tangent at that non-zero pixel) to lock multiple candidate pupil center position data. For each candidate pupil center position data, if its coordinates are within the valid value range of the circle center accumulator space N(a, b), the voting data at that candidate pupil center position data in the circle center accumulator space is incremented by 1 to complete the voting operation of the non-zero pixel on the candidate pupil center position data.

[0088] In this embodiment, for example, the size of the distortion-corrected human eye image data is 640 pixels × 480 pixels. The constructed circle-centered accumulator space N(a, b) matches the image size: the value of a ranges from 0 to 639, and the value of b ranges from 0 to 479. A non-zero pixel point with coordinates (100, 200) is found. Along its gradient direction, a line segment is drawn within a range of 20 to 40 pixels from this point. All points on this line segment, such as (100, 180), (100, 170), and (100, 160), are candidate pupil center position data for this non-zero pixel point. It is checked whether these candidate pupil center position data are within the 640×480 circle-centered accumulator space. If they are, the voting data of positions N(100, 180), N(100, 170), and N(100, 160) are each incremented by 1. If another non-zero pixel (102, 200) also has candidate pupil center position data containing (100, 170), then the voting data of N(100, 170) is incremented by 1 again.

[0089] (3) Based on the voting data of the center accumulator space at each candidate pupil center position data, perform a statistical sorting operation to generate valid pupil center position data.

[0090] In this embodiment, the voting data of each candidate pupil center position data in the center accumulator space are statistically analyzed and sorted, and the candidate pupil center position data with the largest voting data is selected as the valid pupil center position data.

[0091] It is worth noting that by performing edge detection on the distorted human eye image data, binary image data of eye edge detection is generated. Combined with the neighborhood gradient data of each original pixel, voting statistics are performed in the circle accumulator space to complete the pupil center localization. By making full use of the near-circular structural feature of the pupil, it can effectively distinguish the real edge of the pupil from interference information such as eyelids, eyelashes, light spots, and false edges. It has a strong suppression effect on noise and non-target edges, making the pupil center localization more accurate and the anti-interference ability stronger. It provides a reliable initial position for subsequent pupil diameter detection and improves the overall pupil recognition accuracy and reliability.

[0092] S1032: Based on the effective pupil center position data and combined with the preset pupil physiological diameter threshold range, construct multiple effective pupil edge template data.

[0093] In this embodiment, based on the effective pupil center position data and combined with a preset pupil physiological diameter threshold range, a series of effective pupil edge template data with different diameters are pre-constructed. Specifically, with the effective pupil center position data (a0, b0) as the center, multiple gradient radius values ​​r are selected within the preset pupil physiological diameter threshold range. i , respectively with each r i Draw a circular / elliptical outline with a radius, and each circular / elliptical outline corresponds to a valid pupil edge template data.

[0094] S1033: Based on each of the effective pupil edge template data and the eye gradient amplitude image data, perform a pupil template matching operation to generate target pupil diameter data.

[0095] In this embodiment, as Figure 5 The diagram illustrates a flowchart of the pupil template matching operation in an embodiment of the present invention. Based on the valid pupil edge template data and the eye gradient amplitude image data, the pupil template matching operation is performed to generate target pupil diameter data, including:

[0096] S10331: Based on each of the effective pupil edge template data and the eye gradient amplitude image data, perform a similarity calculation operation to generate convolutional response data between each of the effective pupil edge template data and the eye gradient amplitude image data.

[0097] In this embodiment, effective pupil edge template data of various diameters are globally traversed across the eye gradient amplitude image data. For each traversal position in the eye gradient amplitude image data, the convolution value between the current effective pupil edge template data and the eye gradient amplitude image data at that traversal position is calculated. This convolution value characterizes the degree of match between the edge shape of the eye gradient amplitude image data and the current effective pupil edge template data at that traversal position; a higher convolution value indicates a higher degree of matching. The convolution response values ​​of the effective pupil edge template data at all traversal positions in the eye gradient amplitude image data are integrated to generate convolution response data between the effective pupil edge template data and the eye gradient amplitude image data.

[0098] S10332: Generate global response map data based on the convolution response data between each of the effective pupil edge template data and the eye gradient amplitude image data.

[0099] In this embodiment, the convolutional response data between all valid pupil edge template data and eye gradient amplitude image data are integrated to construct a global response atlas with dimensions consistent with the eye gradient amplitude image data. The global response atlas records the matching response magnitude of each valid pupil edge template data with all sliding positions. A higher response value indicates a higher degree of matching between that position as the pupil center and the diameter of that valid pupil edge template data as the pupil diameter.

[0100] S10333: Based on the global response map data, perform a pupil template matching operation to generate target pupil diameter data.

[0101] In this embodiment, the global response map data is traversed, a pupil template matching operation is performed, the highest global response value in the global response map data is found, the image coordinates corresponding to the highest global response value are used as the final pupil center position data, and the diameter of the effective pupil edge template data corresponding to the highest global response value is used as the optimal pupil diameter, thereby generating the target pupil diameter data.

[0102] It is worth noting that by globally sliding the effective pupil edge template data of various diameters across the eye gradient amplitude image data, the convolution value between each effective pupil edge template data and the eye gradient amplitude image data at each sliding position is calculated. This fully utilizes the edge features of the eye gradient amplitude image data, enhancing the matching accuracy between the pupil edge and the template. Furthermore, by integrating all convolutional response data to generate a global response map, and combining the highest global response value to select the optimal template diameter, the detection errors caused by a single template or fixed threshold can be effectively avoided. This approach adapts to pupil size changes under different physiological states and effectively suppresses noise, false edges, and interference from eyelids and eyelashes, ensuring the reliability and consistency of the target pupil diameter data.

[0103] Furthermore, existing technologies typically employ a combination of gradient calculation and ellipse fitting for pupil localization. However, when outliers such as eyelids, eyelashes, and light spots are mixed in with the edge points, the fitting results can shift due to the interference of these outliers, leading to instability in the pupil center and contour detection results. This invention, however, uses a gradient direction voting algorithm combined with a Hough circle detection algorithm in accumulator space to achieve pupil center localization. It determines the target pupil diameter data by matching multiple valid pupil edge template data with eye gradient amplitude image data. This effectively suppresses interference from noise and outliers such as eyelids, eyelashes, light spots, and false edges, resulting in more accurate pupil center localization and ensuring the reliability and consistency of the target pupil diameter data.

[0104] In summary, the pupil recognition method based on center localization of the present invention has the following advantages:

[0105] (1) Effectively eliminate optical distortion interference: The image acquisition operation is performed using the pupil camera that has been optically distorted to obtain the original human eye image containing optical distortion. The optical distortion calibration parameters are read from the control unit of the pupil camera, and the original human eye image is rectified to obtain the distortion-corrected human eye image data. This eliminates the distortion of the pupil geometry caused by the barrel and pincushion distortion introduced by the wide-angle lens in clinical measurement, reduces the measurement deviation caused by optical distortion interference, and thus improves the accuracy of pupil diameter measurement.

[0106] (2) Reduce interference from eye structure: Based on the distortion-corrected human eye image data, a horizontal gradient transformation operation is performed to generate eye gradient amplitude image data. The edge features of the pupil and iris regions are extracted through gradient transformation, and the pupil edge is effectively separated from complex eye structures such as eyelid occlusion, eyelash projection, iris pigment spots / crypts. This solves the problem of easily misjudging the above structural interference as the pupil edge, improves the accuracy of pupil edge feature recognition, and provides feature basis for subsequent template matching.

[0107] (3) Suppressing interference from reflected light spots: Combining the distorted human eye image data and the eye gradient amplitude image data, the pupil template matching operation is performed based on the Hough circle detection algorithm. The edge features of the eye gradient amplitude image data are fully utilized to enhance the matching accuracy between the pupil edge and the template. Since the reflected light spots have no effective gradient edge features, they cannot cause decisive interference to the template matching results. This effectively ensures the continuity and integrity of pupil contour recognition and further improves the reliability and accuracy of pupil measurement results.

[0108] Figure 6 This is a schematic block diagram of a pupil recognition system based on center positioning provided in an embodiment of this application. Figure 6 As shown, the pupil recognition system 600 based on center positioning includes:

[0109] The data acquisition module 601 is used to acquire the image data of the human eye after distortion correction.

[0110] The horizontal gradient transformation module 602 is used to perform a horizontal gradient transformation operation based on the distorted and corrected human eye image data to generate eye gradient amplitude image data.

[0111] The pupil diameter calculation module 603 is used to generate target pupil diameter data by performing a pupil template matching operation based on the distorted human eye image data and the eye gradient amplitude image data, and on the Hough circle detection algorithm.

[0112] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0113] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0114] Figure 7 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 7As shown, an electronic terminal includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the pupil recognition method based on center positioning as described above. The electronic terminal 700 includes at least one processor 701, a memory 702, at least one network interface 703, and a user interface 705. The various components in the electronic terminal 700 are coupled together via a bus system 704. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 7 The general will label all buses as bus systems.

[0115] The user interface 705 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0116] It is understood that memory 702 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0117] In this embodiment of the invention, the memory 702 is used to store various types of data to support the operation of the electronic terminal 700. Examples of this data include: any executable program for operation on the electronic terminal 700, such as the operating system 7021 and application program 7022; the operating system 7021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 7022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The pupil recognition method based on center positioning provided in this embodiment of the invention can be included in the application program 7022.

[0118] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in software form. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 701 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0119] In an exemplary embodiment, the electronic terminal 700 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0120] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any of the embodiments above.

[0121] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, the computer implements the method as described in any of the embodiments above.

[0122] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0123] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0124] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0128] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0129] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] In summary, this application provides a pupil recognition method, system, medium, product, and terminal based on center localization. It performs a horizontal gradient transformation operation on distortion-corrected human eye image data to generate eye gradient amplitude image data. Through gradient transformation, it extracts edge features of the pupil and iris regions, effectively separating the pupil edge from complex eye structures such as eyelid occlusion, eyelash projection, and iris pigment spots / crypts, avoiding misidentification of these interferences as pupil edges and improving the accuracy of pupil edge feature recognition. Combining the distortion-corrected human eye image data and eye gradient amplitude image data, it performs a pupil template matching operation based on the Hough circle detection algorithm. By leveraging the edge features of the gradient amplitude image, it enhances the matching accuracy between the pupil edge and the template, effectively ensuring the continuity and integrity of pupil contour recognition and further improving the reliability and accuracy of pupil measurement results. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial applicability.

[0131] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A pupil recognition method based on center localization, characterized in that, include: Acquire distortion-corrected human eye image data; Based on the distortion-corrected human eye image data, a horizontal gradient transformation operation is performed to generate eye gradient amplitude image data; The distortion-corrected human eye image data includes multiple original pixels; Based on the distorted human eye image data and the eye gradient amplitude image data, and using the Hough circle detection algorithm, a pupil template matching operation is performed to generate target pupil diameter data. The methods include: Based on the distorted human eye image data, a pupil center localization operation is performed to generate valid pupil center position data; Based on the effective pupil center position data and combined with the preset pupil physiological diameter threshold range, multiple effective pupil edge template data are constructed. Based on the valid pupil edge template data and the eye gradient amplitude image data, a pupil template matching operation is performed to generate target pupil diameter data; the method includes: Based on each of the effective pupil edge template data and the eye gradient amplitude image data, a similarity calculation operation is performed to generate convolutional response data between each of the effective pupil edge template data and the eye gradient amplitude image data. Global response map data is generated based on the convolution response data between each of the effective pupil edge template data and the eye gradient amplitude image data. Based on the global response map data, a pupil template matching operation is performed to generate target pupil diameter data.

2. The pupil recognition method based on center localization according to claim 1, characterized in that, The method of performing a horizontal gradient transformation operation to generate eye gradient amplitude image data based on the distortion-corrected human eye image data includes: Based on the distorted human eye image data, a horizontal gradient transformation operation is performed using the horizontal gradient operator to generate horizontal gradient data for each original pixel. Based on the horizontal gradient data of each of the original pixels, construct the eye image gradient matrix; The gradient matrix of the eye image is subjected to a normalization process to generate eye gradient amplitude image data.

3. The pupil recognition method based on center localization according to claim 1, characterized in that, The methods for generating valid pupil center position data by performing pupil center localization operations based on the distortion-corrected human eye image data include: An edge detection operation is performed on the distortion-corrected human eye image data to generate binary image data of eye edge detection; A convolution operation is performed on the distortion-corrected human eye image data to generate neighborhood gradient data for each original pixel. Based on the neighborhood gradient data of each original pixel, multiple non-zero pixels in the binary image data of eye edge detection and the gradient direction of each non-zero pixel are locked. Based on each non-zero pixel in the binary image data of eye edge detection and the gradient direction of each non-zero pixel, a pupil center localization operation is performed to generate valid pupil center position data.

4. The pupil recognition method based on center localization according to claim 3, characterized in that, The method for generating valid pupil center position data by performing pupil center localization based on each non-zero pixel in the binary image data of eye edge detection and the gradient direction of each non-zero pixel includes: Construct and initialize the space for the centered accumulator; Iterate through each non-zero pixel in the binary image data of eye edge detection, and perform line segment drawing operation based on the gradient direction of each non-zero pixel to lock multiple candidate pupil center position data, and generate voting data of the circle center accumulator space at each candidate pupil center position data. Based on the voting data of the center accumulator space at each candidate pupil center position data, a statistical sorting operation is performed to generate valid pupil center position data.

5. A pupil recognition system based on center positioning, characterized in that, include: The data acquisition module is used to acquire distortion-corrected human eye image data; The horizontal gradient transformation module is used to perform a horizontal gradient transformation operation based on the distorted and corrected human eye image data to generate eye gradient amplitude image data; the distorted and corrected human eye image data includes multiple original pixels. The pupil diameter calculation module is used to generate target pupil diameter data by performing a pupil template matching operation based on the distorted human eye image data and the eye gradient amplitude image data, and the Hough circle detection algorithm. The methods include: Based on the distorted human eye image data, a pupil center localization operation is performed to generate valid pupil center position data; Based on the effective pupil center position data and combined with the preset pupil physiological diameter threshold range, multiple effective pupil edge template data are constructed. Based on the valid pupil edge template data and the eye gradient amplitude image data, a pupil template matching operation is performed to generate target pupil diameter data; the method includes: Based on each of the effective pupil edge template data and the eye gradient amplitude image data, a similarity calculation operation is performed to generate convolutional response data between each of the effective pupil edge template data and the eye gradient amplitude image data. Global response map data is generated based on the convolution response data between each of the effective pupil edge template data and the eye gradient amplitude image data. Based on the global response map data, a pupil template matching operation is performed to generate target pupil diameter data.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pupil recognition method based on center positioning as described in any one of claims 1 to 4.

7. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, enables the computer to implement the pupil recognition method based on center positioning as described in any one of claims 1 to 4.

8. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the pupil recognition method based on center positioning as described in any one of claims 1 to 4.

Images