A multi-screen two-dimensional gaze point positioning method based on polarization imaging

Abstract

The present invention provides a multi-screen two-dimensional gaze point positioning method based on polarization imaging, which is used to estimate the gaze point of human eyes in a multi-screen display scene, and realize the estimation of the two-dimensional gaze point of human eyes across screens. The present invention utilizes the polarization characteristics of multi-screen bright spots on the human eye iris reflection plane, collects images of four polarization angles, and calculates the corresponding polarization degree image and polarization angle image to realize the fusion positioning of the pupil center and the multi-screen bright spot area. At the same time, the present invention constructs the pupil center-multi-screen bright spot center feature vector, establishes a relationship model between the feature vector and the multi-screen gaze plane, selects the multi-screen gaze plane, and then uses the homography normalization method on a single screen to complete the gaze point calibration and positioning. The present invention does not add additional artificial auxiliary light sources, and only uses the screen display in the scene as a light source. It has the characteristics of high precision, strong robustness, and user-friendliness, and can realize the two-dimensional gaze point positioning without head constraints.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a multi-screen two-dimensional gaze point positioning method based on polarization imaging. Background Art

[0002] Existing methods for locating the gaze point of the human eye usually use pupil corneal reflection technology to extract the position of the pupil center and the position of the Purkinje spots reflected by the light source, which requires the assistance of an additional near-infrared light source. However, long-term radiation from near-infrared light sources can make users feel tired. Moreover, the Purkinje spots reflected by the light source are only more obvious in the pupil iris area, and are not easy to detect in the sclera area, which affects the detection results. In addition, the existing methods for locating the gaze point are all aimed at single-screen displays and fail when switching between multiple screens. Under the above premise, when the existing methods for locating the gaze point of the human eye are used, the user's head movement range is extremely limited, and a chin rest or head rest is needed to fix the head to limit the Purkinje spots from significantly exceeding the iris area in order to obtain better detection results. Summary of the invention

[0003] The present invention provides a multi-screen two-dimensional gaze point positioning method based on polarization imaging, which uses the display as an auxiliary light source and realizes unconstrained two-dimensional gaze point positioning of the head through gaze point calibration of multiple screen displays. This solves the problem of extremely limited head movement range of the user, and can accurately estimate the gaze point of the human eye in the multi-screen display scene.

[0004] The technical means adopted by the present invention are as follows:

[0005] A multi-screen two-dimensional gaze point positioning method based on polarization imaging is implemented based on a multi-screen gaze point positioning system, wherein the multi-screen gaze point positioning system comprises: a single polarization camera and a plurality of LCD screen displays, wherein the single polarization camera and the plurality of LCD screen displays are arranged on the same side of a human eye;

[0006] The method comprises the following steps:

[0007] The images of the human eye region at four polarization angles are obtained by a polarization camera, the Stokes parameters are solved according to the images of the human eye region at four angles, and the polarization degree information and polarization angle information of the human eye region are obtained by calculation, thereby obtaining the polarization degree image and polarization angle image of the human eye region;

[0008] On the human eye region images at the four polarization angles, statistically based threshold segmentation is performed respectively, and the pupil region is subjected to fusion morphological processing, so as to locate the pupil center position;

[0009] On the polarization degree image of the human eye area, adaptive threshold segmentation is performed to obtain a binary image of the bright spot area of ​​multiple screens; on the corresponding polarization angle image, the binary image of the bright spot area of ​​multiple screens is used as a mask to locate the bright spot area of ​​multiple screens, and the centroid and area of ​​the bright spot area of ​​different screens are calculated respectively;

[0010] Construct the vector from the through-hole center to the centroid of the bright spot area on different screens, establish a multi-screen regression model, and realize the plane selection of the gaze screen;

[0011] Look at each single screen, calculate the area of ​​the bright spot region of the corresponding screen, judge the gaze credibility of the screen, use corner point detection to locate the four corner points of the bright spot area of ​​the screen, fit the imaging of the bright spot area with a rectangle, and use dynamic points to achieve smooth trailing dynamic calibration of a single screen.

[0012] Further, the human eye region images at four polarization angles are obtained by using a polarization camera, including using the polarization camera to obtain four polarization angles of 0 ° , 45 ° , 90 ° and 135 ° Polarization image of the human eye area: I(0 ° )、I(45 ° )、I(90 ° ) and I(135 ° );

[0013] The Stokes parameters are:

[0014] S0=(0 ° )+(90 ° )

[0015] S1=(0 ° )-I(90 ° )

[0016] S2=(45 ° )-I(135 ° )

[0017] Among them, I(0 ° ) indicates that the polarization angle is 0 ° Polarized image of the human eye area, I(45 ° ) indicates that the polarization angle is 45 ° Polarized image of the human eye area, I(90 ° ) indicates that the polarization angle is 90 ° Polarized image of the human eye area, I(135 ° ) indicates that the polarization angle is 135 ° Polarized image of the human eye area.

[0018] Furthermore, the polarization degree image is obtained according to the following calculation:

[0019]

[0020] Among them, I D represents the polarization degree image, and S0, S1, and S2 are Stokes parameters.

[0021] The polarization angle image is obtained according to the following calculation:

[0022]

[0023] Among them, I A Represents the polarization angle image.

[0024] Furthermore, on the human eye region images at the four polarization angles, statistically based threshold segmentation is performed respectively, and the pupil region is subjected to fusion morphological processing, so as to locate the pupil center position, including:

[0025] On the human eye region images at four polarization angles, a statistical threshold segmentation method is used to select the grayscale value corresponding to the first 10% of the cumulative distribution of the grayscale histogram as the threshold, and the grayscale image is converted into a binary image;

[0026] Morphological processing is used to preliminarily remove invalid small areas in binary images;

[0027] Adding the binary images of the human eye region images at the four polarization angles obtained to obtain the pupil region to be estimated;

[0028] The pupil area is corrected by using an ellipse fitting algorithm, and the pupil center position is defined as the position with the minimum cost to reach all positions in the pupil area. The pupil center position in the pupil area is solved using this minimum cost constraint.

[0029] Furthermore, adaptive threshold segmentation is performed on the polarization degree image of the human eye area to obtain a binary image of the bright spot area of ​​multiple screens, and the binary image of the bright spot area of ​​multiple screens is used as a mask on the corresponding polarization angle image to locate the bright spot area of ​​multiple screens, and the centroid and area of ​​the bright spot area of ​​different screens are calculated respectively, including:

[0030] Using the Otsu method on the polarization degree image of the human eye area, adaptive threshold segmentation is performed to convert the polarization degree image into a binary image;

[0031] Morphological processing is used to preliminarily remove invalid small areas in binary images;

[0032] Performing regional clustering segmentation on the polarization angle map of the human eye area, retaining regional blocks with a pixel number greater than 5×5, and using the polarization degree binarization map as a mask to locate the multi-screen bright spot area in the polarization angle image;

[0033] Calculate the centroid and area of ​​different screen bright spot areas respectively.

[0034] Furthermore, a vector from the through hole center to the centroid of the bright spot area of ​​different screens is constructed, and a multi-screen regression model is established to realize the plane selection of the gaze screen, including:

[0035] Taking the pupil center as the starting point, connecting the pupil center and the bright spot centers of different screens, and establishing the pupil center-multi-screen bright spot center feature vector in the imaging plane coordinate system;

[0036] The human eye looks at different screen planes in turn, marking the feature vectors, screen bright spot areas and index screen numbers of different screen planes;

[0037] Partial least squares regression is used to establish the relationship model between the feature vector and the fixation plane.

[0038] Furthermore, a vector from the through hole center to the centroid of the bright spot area of ​​different screens is constructed, and a multi-screen regression model is established to realize the plane selection of the gaze screen, including:

[0039] Take the center points of the bright spots on multiple screens as vertices, connect and construct one or more triangles, and take all the internal angle values ​​of the triangles as feature vectors;

[0040] The relationship model between the feature vector and the fixation plane is established using the nearest neighbor regression method.

[0041] Furthermore, each single screen is observed, the bright spot area of ​​the corresponding screen is calculated, the gaze credibility of the screen is determined, the four corner points of the bright spot area of ​​the screen are located by using corner point detection, and the imaging of the bright spot area is fitted with a rectangle, and the smooth trailing dynamic calibration of a single screen is realized by using dynamic points, including:

[0042] The human eye looks at a single screen to be calibrated, calculates the area of ​​the bright spot region of the corresponding screen, and determines the area reliability. The area reliability is calculated based on the area of ​​the bright spot region currently solved divided by the area of ​​the bright spot region of the corresponding single screen marked in the multi-screen gaze plane selection stage. If the area reliability of the bright spot region is greater than 0.9, the current screen does not need to be calibrated again. If the area reliability of the bright spot region is less than or equal to 0.9, it is considered that the current head posture rotation is greatly different from the marked head posture, and the screen needs to be calibrated;

[0043] When calibration is required, the four corner points of the current screen bright spot area are located using the corner point detection method, and the screen bright spot area is fitted using the rectangle fitting algorithm to determine the final four corner point positions of the screen bright spot area;

[0044] According to the two mapping relationships from the imaging plane to the corneal reflection plane and from the corneal reflection plane to the screen plane, the corresponding homography matrix is ​​calculated to realize the projection mapping of the gaze point from the imaging plane to the corneal reflection plane and then to the screen plane.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] 1. The present invention provides a multi-screen two-dimensional gaze point positioning method based on polarization imaging, which is used to estimate the gaze point of human eyes in a multi-screen display scene, and realize the estimation of the two-dimensional gaze point of human eyes across screens. The present invention uses the polarization characteristics of multi-screen bright spots on the human eye iris reflection plane to collect images at four polarization angles, and calculates the corresponding polarization degree image and polarization angle image to realize the fusion positioning of the pupil center and the multi-screen bright spot area.

[0047] 2. The present invention constructs pupil center-multi-screen bright spot center feature vector, establishes a relationship model between the feature vector and the multi-screen gaze plane, selects the multi-screen gaze plane, and then uses the homography normalization method on a single screen to complete the gaze point calibration and positioning.

[0048] 3. The present invention does not add additional artificial auxiliary light sources, but only uses the screen display in the scene as a light source. It has the characteristics of high precision, strong robustness, and user-friendliness, and can realize unconstrained two-dimensional gaze point positioning of the head. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 The present invention is a flow chart of a multi-screen two-dimensional gaze point positioning method based on polarization imaging.

[0051] Figure 2 This is a schematic diagram of the imaging principle of a multi-screen gaze point positioning system in an embodiment of the present invention.

[0052] Figure 3 Schematic diagram of multi-screen polarization imaging in an embodiment of the present invention.

[0053] Figure 4The present invention is a flowchart of performing two-dimensional gaze point positioning on multiple screens in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0055] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] Considering that the LCD display itself contains a polarizing filter, the light it emits is polarized light. The present invention proposes a multi-screen two-dimensional gaze point positioning method and device based on polarization imaging, using the display as an auxiliary light source. The polarized bright spot reflected by the display is highly robust compared to the infrared light spot and will not cause secondary damage to the human eye. In view of the differences in the angles of the polarizing filters in different screen displays, the bright spots reflected by different screen displays can be easily distinguished in the image of the human eye area captured by the polarization camera. By calibrating the gaze points of multiple screen displays, unconstrained two-dimensional gaze point positioning of the head can be achieved. Specifically, if Figure 1 As shown, the present invention provides a multi-screen gaze point positioning system, the multi-screen gaze point positioning system includes: a single polarization camera and a plurality of LCD screen displays, the single polarization camera and the plurality of LCD screen displays are arranged on the same side of the human eye, the system structure is as shown Figure 2 The method comprises the following steps:

[0057] S1. Obtain images of the human eye region at four polarization angles through a polarization camera, solve Stokes parameters based on the images of the human eye region at four angles, and obtain polarization degree information and polarization angle information of the human eye region through calculation, thereby obtaining a polarization degree image and a polarization angle image of the human eye region.

[0058] This step is mainly used to obtain image information. The LCD adjusts the screen backlight through a linear polarization filter so that the display emits polarized light.

[0059] Specifically, a single polarization camera and several LCD displays are used to form a gaze point positioning system, and the polarization camera is used to obtain four polarization angles 0 ° , 45 ° , 90 ° , 135 ° Polarization image of the human eye area: I(0 ° )、I(45 ° )、I(90 ° )、I(135 ° ), where the polarization angle refers to the angle between the polarization direction that can be transmitted and the vertical direction. Linear polarizers are used to transmit linear polarized light in a certain direction, while polarized light orthogonal to this direction will be absorbed or deflected. Given the differences in the angles of polarizing filters in different screen displays, the bright spots reflected by different screen displays can be easily distinguished in the images of the human eye area captured by the polarization camera. When a polarization camera is used to capture polarized images of the human eye area, the polarized light reflection of the display can be observed. The imaging principle is as follows: Figure 3 As shown, after the screen display emits light, it passes through the polarizer with the corresponding polarization angle, and the polarization camera captures the image through the polarizer with the corresponding polarization angle. The intensity of the bright spot on the corneal surface in the polarized image of the human eye area can be adjusted by the polarization filter on the camera end. The polarization angle value of the bright spot area on the corneal surface is almost equal to the polarization angle value of the light emitted by the display. The LCD display can be a desktop display, an industrial computer display, a portable display, etc. Common screen placement positions include: multiple screens in parallel horizontally, multiple screens in parallel vertically, and screens in parallel horizontally and vertically.

[0060] Furthermore, the Stokes parameter is obtained according to the following calculation: S0 = I(0 ° )+I(90 ° ), S1=I(0 ° )-I(90 ° ), S2=I(45 ° )-I(135 ° ). The polarization state information that can be calculated by the Stokes formula includes the degree of polarization and the polarization angle. D and polarization angle diagram I A The calculation method is as follows:

[0061]

[0062]

[0063] The value of the polarization angle depends on the normal vector of the object surface.

[0064] S2. Performing statistically-based threshold segmentation on the human eye area images at the four polarization angles, and performing fusion morphological processing on the pupil area, so as to locate the center of the pupil.

[0065] The present invention utilizes the polarization characteristics of multi-screen bright spots on the human eye iris reflection plane, and realizes the fusion positioning of the pupil center through the collected four polarization angle images, and the calculated polarization degree image and polarization angle image. In order to reduce the influence of random noise during image acquisition, a 5×5 Gaussian filter with a standard deviation of 2 pixels is used to preprocess the image.

[0066] On the images of four polarization angles, a statistical threshold segmentation method is used to select the grayscale value corresponding to the first 10% of the cumulative distribution of the grayscale histogram as the threshold, and the grayscale image is converted into a binary image. Morphological processing (dilation, erosion, etc.) is used to preliminarily remove invalid small areas. The binary images of the four polarization angles are added to obtain the pupil area to be estimated, and the pupil area is corrected using an ellipse fitting algorithm. The pupil center position is defined as the position with the minimum cost to reach all positions in the pupil area. Using this minimum cost constraint, the pupil center position in the pupil area is solved.

[0067] S3. Perform adaptive threshold segmentation on the polarization degree image of the human eye area to obtain a binary image of the bright spot area of ​​multiple screens. On the corresponding polarization angle image, use the binary image of the bright spot area of ​​multiple screens as a mask to locate the bright spot area of ​​multiple screens, and calculate the centroid and area of ​​the bright spot area of ​​different screens respectively.

[0068] Specifically, the Otsu method is used to perform adaptive threshold segmentation on the polarization degree map, and the polarization degree map is converted into a binary image. Morphological processing (dilation, erosion, etc.) is used to preliminarily remove invalid small areas. On the polarization angle map, regional clustering segmentation is performed, and regional blocks with a number of pixels greater than 5×5 are retained. The polarization degree binary map is used as a mask to locate the multi-screen bright spot areas in the polarization angle image. The centroid and area of ​​different screen bright spot areas are calculated respectively.

[0069] S4. Construct a vector from the center of the through hole to the centroid of the bright spot area of ​​different screens, establish a multi-screen regression model, and realize the plane selection of the gaze screen.

[0070] This step is mainly used to realize multi-screen gaze plane selection and model training. Specifically, the present invention constructs pupil center-multi-screen bright spot center feature vector, establishes the relationship model between feature vector and multi-screen gaze plane, selects multi-screen gaze plane, and then locates the gaze point on a single screen. The present invention does not consider the gaze point position outside the gaze screen, uses multi-screen gaze plane selection, switches the gaze plane, and realizes cross-screen gaze point positioning.

[0071] In one embodiment, for the head chin rest fixation, the pupil center is taken as the starting point, the pupil center and the bright spot centers of different screens are connected, and the pupil center-multi-screen bright spot center feature vector in the imaging plane coordinate system is established. The human eye sequentially fixates on different screen planes, and the feature vectors, screen bright spot area, and index screen number of different screen planes are marked. Partial least squares regression is used to establish a relationship model between the feature vector and the fixation plane.

[0072] In another embodiment, when the head posture rotates greatly, the center points of the bright spots on multiple screens are used as vertices to connect and construct one or more triangles, and all the internal angle values ​​of the triangles are used as feature vectors. The nearest neighbor regression method is used to establish a relationship model between the feature vector and the gaze plane. Figure 3 As shown in the figure, on the polarization camera imaging plane, capture the bright spot area of ​​multiple screens, calculate the center of each bright spot area of ​​the screen, and connect and construct triangles with the center points of the bright spots of multiple screens as vertices. If there are more than three bright spot areas of multiple screens, use Delaunay triangulation to construct multiple triangular areas. Take all the internal angle values ​​of the triangle as feature vectors, and use the nearest neighbor regression method to establish a relationship model between the feature vector and the gaze plane.

[0073] S5. Look at each single screen, calculate the area of ​​the bright spot region of the corresponding screen, determine the look credibility of the screen, use corner point detection to locate the four corner points of the bright spot region of the screen, fit the imaging of the bright spot region with a rectangle, and use dynamic points to achieve smooth trailing dynamic calibration of a single screen.

[0074] This step is mainly used to achieve single-screen gaze calibration and single-screen gaze estimation. Specifically:

[0075] The present invention performs single screen gaze calibration on all screen planes, mainly by performing homography mapping on corresponding screen corner points. The specific method is as follows:

[0076] First, the human eye looks at the single screen to be calibrated, calculates the area of ​​the bright spot region of the corresponding screen, and determines the area credibility, that is, the bright spot region area currently solved is divided by the bright spot region area of ​​the corresponding single screen marked in the multi-screen gaze plane selection stage. The four corner points of the bright spot region of the current screen are located using the corner point detection method, and the screen bright spot region is fitted with the rectangle fitting algorithm to determine the final four corner point positions of the screen bright spot region.

[0077] At this time, the homography normalization method is used to complete the line of sight calibration. According to the two mapping relationships from the imaging plane to the corneal reflection plane and from the corneal reflection plane to the screen plane, the corresponding homography matrix is ​​calculated to realize the projection mapping of the gaze point from the imaging plane to the corneal reflection plane and then to the screen plane.

[0078] When all single screens have completed gaze calibration, gaze estimation can be performed. During the gaze estimation stage, the system detects and locates the pupil center and the bright spot area of ​​the human eye. When the human eye is fixated on a certain screen, the gaze estimation system first selects and identifies the multi-screen gaze planes to determine which plane is being gazed; then the credibility of the bright spot area of ​​the current screen is calculated. If the credibility of the bright spot area is greater than 0.9, the current screen does not need to be calibrated again. If the credibility of the bright spot area is less than or equal to 0.9, it is considered that the current head posture rotation is significantly different from the marked head posture, and the screen needs to be calibrated. The execution process of line of sight calibration and line of sight estimation through the scheme provided in this embodiment is as follows: Figure 4 shown.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-screen two-dimensional gaze point positioning method based on polarization imaging, characterized in that: The multi-screen gaze point positioning system is implemented, and the multi-screen gaze point positioning system includes: a single polarization camera and a display of a plurality of LCD screens, and the single polarization camera and the display of the plurality of LCD screens are arranged on the same side of the human eye; The method comprises the following steps: The images of the human eye region at four polarization angles are obtained by a polarization camera, the Stokes parameters are solved according to the images of the human eye region at four angles, and the polarization degree information and polarization angle information of the human eye region are obtained by calculation, thereby obtaining the polarization degree image and polarization angle image of the human eye region; On the human eye region images at the four polarization angles, statistically based threshold segmentation is performed respectively, and the pupil region is subjected to fusion morphological processing, so as to locate the pupil center position; On the polarization degree image of the human eye area, adaptive threshold segmentation is performed to obtain a binary image of the bright spot area of ​​multiple screens; on the corresponding polarization angle image, the binary image of the bright spot area of ​​multiple screens is used as a mask to locate the bright spot area of ​​multiple screens, and the centroid and area of ​​the bright spot area of ​​different screens are calculated respectively; Construct the vector from the pupil center to the centroid of the bright spot area on different screens, establish a multi-screen regression model, and realize the plane selection of the gaze screen; Look at each single screen, calculate the area of ​​the bright spot region of the corresponding screen, judge the gaze credibility of the screen, use corner point detection to locate the four corner points of the bright spot area of ​​the screen, fit the imaging of the bright spot area with a rectangle, and use dynamic points to achieve smooth trailing dynamic calibration of a single screen.

2. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 1, characterized in that: Acquiring images of the human eye region at four polarization angles by a polarization camera, including acquiring polarization images of the human eye region at four polarization angles of 0°, 45°, 90°, and 135° by using a polarization camera: I(0°), I(45°), I(90°), and I(135°); The Stokes parameters are: S0=I(0°)+I(90°) S1=I(0°)-I(90°) S2=I(45°)-I(135°) Among them, I(0°) represents the polarization image of the human eye area with a polarization angle of 0°, and I(45°) represents the polarization image of the human eye area with a polarization angle of 45°. ° Polarized image of the human eye area, I(90 ° ) indicates that the polarization angle is 90 ° Polarized image of the human eye area, I(135 ° ) indicates that the polarization angle is 135 ° Polarized image of the human eye area.

3. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 2, characterized in that: The polarization degree image is obtained according to the following calculation: Among them, I D represents the polarization degree image, S0, S1, S2 are Stokes parameters; The polarization angle image is obtained according to the following calculation: Among them, I A Represents the polarization angle image.

4. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 1, characterized in that: On the human eye region images at the four polarization angles, statistically based threshold segmentation is performed respectively, and the pupil region is subjected to fusion morphological processing, so as to locate the pupil center position, including: On the human eye region images at four polarization angles, a statistical threshold segmentation method is used to select the grayscale value corresponding to the first 10% of the cumulative distribution of the grayscale histogram as the threshold, and the grayscale image is converted into a binary image; Morphological processing is used to preliminarily remove invalid small areas in binary images; Adding the binary images of the human eye region images at the four polarization angles obtained to obtain the pupil region to be estimated; The pupil area is corrected by using an ellipse fitting algorithm, and the pupil center position is defined as the position with the minimum cost to reach all positions in the pupil area. The pupil center position in the pupil area is solved using this minimum cost constraint.

5. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 1, characterized in that: On the polarization degree image of the human eye area, adaptive threshold segmentation is performed to obtain a binary image of the bright spot area of ​​multiple screens, and on the corresponding polarization angle image, the binary image of the bright spot area of ​​multiple screens is used as a mask to locate the bright spot area of ​​multiple screens, and the centroid and area of ​​the bright spot area of ​​different screens are calculated respectively, including: Using the Otsu method on the polarization degree image of the human eye area, adaptive threshold segmentation is performed to convert the polarization degree image into a binary image; Morphological processing is used to preliminarily remove invalid small areas in binary images; Performing regional clustering segmentation on the polarization angle map of the human eye area, retaining regional blocks with a pixel number greater than 5×5, and using the polarization degree binarization map as a mask to locate the multi-screen bright spot area in the polarization angle image; Calculate the centroid and area of ​​different screen bright spot areas respectively.

6. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 1, characterized in that: Construct the vector from the center of the through hole to the centroid of the bright spot area of ​​different screens, establish a multi-screen regression model, and realize the plane selection of the gaze screen, including: Taking the pupil center as the starting point, connecting the pupil center and the bright spot centers of different screens, and establishing the pupil center-multi-screen bright spot center feature vector in the imaging plane coordinate system; The human eye looks at different screen planes in turn, marking the feature vectors, screen bright spot areas and index screen numbers of different screen planes; Partial least squares regression is used to establish the relationship model between the feature vector and the fixation plane.

7. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 1, characterized in that: Construct the vector from the center of the through hole to the centroid of the bright spot area of ​​different screens, establish a multi-screen regression model, and realize the plane selection of the gaze screen, including: Take the center points of the bright spots on multiple screens as vertices, connect and construct one or more triangles, and take all the internal angle values ​​of the triangles as feature vectors; The relationship model between the feature vector and the fixation plane is established using the nearest neighbor regression method.

8. The multi-screen two-dimensional gaze point positioning method based on polarization imaging according to claim 1, characterized in that: Look at each single screen, calculate the bright spot area of ​​the corresponding screen, determine the look credibility of the screen, locate the four corner points of the bright spot area of ​​the screen using corner point detection, fit the imaging of the bright spot area with a rectangle, and use dynamic points to achieve smooth trailing dynamic calibration of a single screen, including: The human eye looks at a single screen to be calibrated, calculates the area of ​​the bright spot region of the corresponding screen, and determines the area reliability. The area reliability is calculated based on the area of ​​the bright spot region currently solved divided by the area of ​​the bright spot region of the corresponding single screen marked in the multi-screen gaze plane selection stage. If the area reliability of the bright spot region is greater than 0.9, the current screen does not need to be calibrated again. If the area reliability of the bright spot region is less than or equal to 0.9, it is considered that the current head posture rotation is greatly different from the marked head posture, and the screen needs to be calibrated; When calibration is required, the four corner points of the current screen bright spot area are located using the corner point detection method, and the screen bright spot area is fitted using the rectangle fitting algorithm to determine the final four corner point positions of the screen bright spot area; According to the two mapping relationships from the imaging plane to the corneal reflection plane and from the corneal reflection plane to the screen plane, the corresponding homography matrix is ​​calculated to realize the projection mapping of the gaze point from the imaging plane to the corneal reflection plane and then to the screen plane.

Images