Interfering light spot screening method, system and device, and storage medium

Abstract

The present application provides an interfering light spot screening method, system and device, and a storage medium. The method comprises: obtaining iris contour point coordinates on the basis of iris contour point position detection, and then using the iris contour point coordinates to perform iris contour drawing and color filling on a preset image so as to obtain a reference image; and using first light spot position coordinates obtained by means of light spot position detection to extract pixel colors from the reference image, comparing and recognizing the extracted pixel colors, and deleting corresponding measured values outside an iris contour from a first light spot position array, thereby effectively improving the accuracy of light spot position detection. In the embodiments of the present application, an interfering light spot formed by an opposite light source can be recognized and deleted in light spot position detection to obtain a correct light spot position detection result. Then, light spot numbering is performed on the correct light spot position detection result, and the accuracy of light spot numbering is also improved.

Description

Interference light spot screening method, system, device and storage medium

[0001] Cross-references of related documents

[0002] This application claims priority to the Chinese patent application No. 2024112675533, filed on September 11, 2024, and entitled "Interference light spot screening method, system, device and storage medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] Embodiments of the present application relate to the technical field of gaze tracking, and in particular to an interference light spot screening method, system, device and storage medium. BACKGROUND

[0004] Gaze tracking technology is a technology for measuring the gaze point of a human eye and the degree of movement relative to the head. The gaze tracking method using the corneal reflection method needs to obtain correct light spot position information and its corresponding relationship with the light source. In order to obtain the above information and the corresponding relationship, the eye map of the human eye is usually collected to detect the light spot position and the detected light spots are numbered. When the eye map of the human eye is collected, interference light spots are often formed due to the influence of the opposite light source.

[0005] For example, a predetermined number of light sources are arranged near the optical lenses of the left eye and the right eye of a head-mounted device, an eye map is collected for a single target human eye (left eye or right eye), and a target human eye eye map is obtained. The opposite light source refers to the light source arranged near the optical lens of the other eye. In the existing light spot position detection and light spot numbering implementation, the result is directly obtained for the collected target human eye eye map, without considering the problem of interference light spots formed by the opposite light source. SUMMARY

[0006] To this end, the present application provides an interference light spot screening method, system, device and storage medium to solve the technical problem that the existing light spot position detection and light spot numbering implementation do not consider the problem of interference light spots formed by the opposite light source.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] According to a first aspect of the present application, the embodiments of the present application provide an interference light spot screening method, which comprises:

[0009] Performing light spot position detection on the first original eye map to obtain a first light spot position array;

[0010] Performing iris contour point position detection on the second original eye map to obtain a preset number of iris contour point coordinates;

[0011] drawing the iris contour on a preset image with the same size as the first original eye map using each of the iris contour point coordinates; the preset image has a first preset color;

[0012] filling an internal region of the drawn iris contour with a second preset color to obtain a reference image;

[0013] extracting pixel colors from the reference image according to each of the first spot position coordinates in the first spot position array;

[0014] identifying the extracted pixel colors corresponding to each of the first spot position coordinates;

[0015] if the extracted pixel color is the first preset color, the corresponding spot is not within the iris contour, and the corresponding first spot position coordinate is deleted;

[0016] if the extracted pixel color is the second preset color, the corresponding spot is within the iris contour, and the corresponding first spot position coordinate is retained as a second spot position coordinate;

[0017] obtaining a second spot position array as a to-be-numbered spot position array using all the second spot position coordinates.

[0018] According to a second aspect of the present application, there is provided an interference spot screening system for performing the steps of an interference spot screening method as described above, the system comprising:

[0019] a first detection unit configured to perform spot position detection on a first original eye map to obtain a first spot position array;

[0020] a second detection unit configured to perform iris contour point position detection on a second original eye map to obtain a preset number of iris contour point coordinates;

[0021] an exclusion unit configured to perform the following steps:

[0022] drawing the iris contour on a preset image with the same size as the first original eye map using each of the iris contour point coordinates; the preset image has a first preset color;

[0023] filling an internal region of the drawn iris contour with a second preset color to obtain a reference image;

[0024] extracting pixel colors from the reference image according to each of the first spot position coordinates in the first spot position array;

[0025] identifying the extracted pixel colors corresponding to each of the first spot position coordinates;

[0026] If the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinate is deleted;

[0027] If the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinate is reserved as a second light spot position coordinate;

[0028] The second light spot position array is obtained as the to-be-numbered light spot position array by using all the second light spot position coordinates.

[0029] According to a third aspect of the present application, an interference light spot screening device is provided, the device comprising: a processor and a memory;

[0030] The memory is configured to store one or more program instructions;

[0031] The processor is configured to execute the one or more program instructions to perform the steps of the interference light spot screening method.

[0032] According to a fourth aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the interference light spot screening method.

[0033] Compared with the prior art, the interference light spot screening method, system, device and storage medium provided by the embodiments of the present application can effectively improve the accuracy of light spot position detection. The embodiments of the present application can identify and delete the interference light spots formed by the opposite light source in the light spot position detection, and obtain correct light spot position detection results. In this way, the correct light spot position detection results are numbered, and the accuracy of light spot numbering is also improved. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only exemplary, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0035] The structures, proportions, sizes, etc. shown in the specification are only used to cooperate with the content disclosed in the specification, to be understood and read by those skilled in the art, and are not used to limit the conditions that can be implemented by the application, so they do not have technical significance. Any modification of the structure, change of the proportion relationship or adjustment of the size, without affecting the effect and purpose that can be achieved by the application, should still fall within the scope of the technology disclosed by the application.

[0036] Fig. 1 is a schematic diagram of the logical structure of an interference spot screening system according to an embodiment of the present application;

[0037] Fig. 2 is a schematic diagram of the flow of an interference spot screening method according to an embodiment of the present application;

[0038] Fig. 3 is a schematic diagram of the flow of an interference spot screening method according to another embodiment of the present application;

[0039] Fig. 4 is a schematic diagram of the flow of establishing an eye diagram horizontal line coordinate system according to an embodiment of the present application;

[0040] Fig. 5 is a schematic diagram of the flow of an interference spot screening method according to another embodiment of the present application;

[0041] Fig. 6 is a schematic diagram of the flow of an interference spot screening method according to another embodiment of the present application;

[0042] Fig. 7 is a schematic diagram of the flow of an interference spot screening method according to another embodiment of the present application. DETAILED DESCRIPTION

[0043] The embodiments of the present application are described below by specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the specification. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0044] The purpose of the present application is to effectively exclude the interference spot formed by the opposite light source in the process of spot position detection and spot numbering, obtain correct spot position detection results and numbering, and ensure accuracy.

[0045] In order to solve the above technical problems, as shown in Fig. 1, the present application provides an interference spot screening system, which is used to realize interference spot screening.

[0046] Specifically, the interference spot screening system provided by the present application comprises a first detection unit 1, a second detection unit 2 and an exclusion unit 3.

[0047] Further, the first detection unit 1 is configured to perform speckle position detection on the first original eye diagram to obtain a first speckle position array; and the second detection unit 2 is configured to perform iris contour point position detection on the second original eye diagram to obtain a preset number of iris contour point coordinates.

[0048] Further, the exclusion unit 3 is configured to perform the following steps: drawing an iris contour on a preset image with the same size as the first original eye diagram by using each of the iris contour point coordinates, the preset image having a first preset color; filling an internal region of the drawn iris contour with a second preset color to obtain a reference image; extracting pixel colors of the reference image according to each of the first speckle position coordinates in the first speckle position array; identifying the extracted pixel colors corresponding to each of the first speckle position coordinates; if the extracted pixel color is the first preset color, the corresponding speckle is not within the iris contour, and the corresponding first speckle position coordinate is deleted; if the extracted pixel color is the second preset color, the corresponding speckle is within the iris contour, and the corresponding first speckle position coordinate is kept as a second speckle position coordinate; and obtaining a second speckle position array as a to-be-numbered speckle position array by using all the second speckle position coordinates.

[0049] Compared with the prior art, the interference speckle screening system provided by the embodiment of the present application is based on iris contour point position detection to obtain iris contour point coordinates, and then draws an iris contour on a preset image by using the iris contour point coordinates and performs color filling to obtain a reference image; each of the first speckle position coordinates obtained by speckle position detection is used to extract pixel colors of the reference image, and the extracted pixel colors are compared and identified to delete the detection values corresponding to the outside of the iris contour from the first speckle position array, thereby effectively improving the accuracy of speckle position detection. In the speckle position detection, the interference speckle formed by the opposite light source can be identified and deleted, and a correct speckle position detection result can be obtained. Therefore, the correct speckle position detection result is numbered, and the accuracy of speckle numbering is also improved.

[0050] Corresponding to the above disclosed interference speckle screening system, the embodiment of the present application further discloses an interference speckle screening method. The interference speckle screening method disclosed in the embodiment of the present application is described in detail below in combination with the above described interference speckle screening system.

[0051] As shown in FIG. 2, the specific steps of the interference speckle screening method provided by the embodiment of the present application are described in detail below.

[0052] The interference speckle screening method provided by the embodiment of the present application is used to realize interference speckle screening.

[0053] The first speckle position array is obtained by performing speckle position detection on the first original eye diagram by the first detection unit 1.

[0054] In the embodiments of the present application, the first original eye diagram is an eye grayscale image captured by a camera. For example, the target person can wear a head-mounted device, and a predetermined number of light sources are arranged near the left and right eye optical lenses of the head-mounted device. Assuming that the predetermined number is 6 or 8, the 6 or 8 light sources are distributed in a circle around the left eye optical lens, and a camera is located directly or obliquely below the left eye optical lens, close to the cheek. Similarly, the 6 or 8 light sources are distributed in a circle around the right eye optical lens, and a camera is located directly or obliquely below the right eye optical lens, close to the cheek. The eye diagram of a single target person's eye (left eye or right eye) is collected, and the first original eye diagram described above can be obtained.

[0055] The above steps specifically include: performing adaptive threshold segmentation on the eye diagram to be detected to obtain a first eye diagram; performing erosion and dilation operations on the first eye diagram to obtain a second eye diagram; performing speckle contour detection on the second eye diagram to obtain a speckle contour set; determining the first center point position coordinates of the corresponding speckles according to the first moments of the speckle contours; and taking the first center point position coordinates of the speckles as the corresponding first speckle position coordinates to obtain the first speckle position array.

[0056] As described above, in the embodiments of the present application, the first original eye diagram described above can be directly used as the eye diagram to be detected, and the adaptive threshold segmentation is realized by an adaptive threshold function (adaptiveThreshold() function). The mean value or Gaussian mean value method is used to identify the bright spots in the eye diagram to be detected to obtain the first eye diagram img_threshold. The adaptiveThreshold() function in OpenCV is a special threshold processing method, which is used to convert a grayscale image or a color image into a binary image. It can adaptively determine the threshold value of each pixel according to the local grayscale characteristics of the neighborhood of each pixel, so it can better binarize the image with uneven illumination than the traditional global threshold method.

[0057] Further, the first eye image img_threshold is eroded and dilated, mainly to eliminate the influence of small dots and noise, and the image after the operation is a second eye image img_dilation. Erosion and dilation are the two most basic morphological operations. Morphological operations are a series of image processing operations based on shape. OpenCV provides a convenient function for morphological transformation of images, one of the main functions of which is to eliminate noise. Erosion and dilation are both for the white part (highlighted part) of the image. The erosion operation is a "field expansion" of the highlighted part of the image, and the effect image has a larger highlighted area than the original image. The dilation operation is that the highlighted area in the original image is eaten away, and the effect image has a smaller highlighted area than the original image.

[0058] Next, in the embodiment of the present application, the findContours() function is used to complete the spot contour detection of the second eye image img_dilation, find the contour, and each contour corresponds to a spot. The findContours() function in C++ OpenCV is used to detect the contour in a binary image. It can extract continuous regions in the image to form a set of spot contours. This function is very suitable for computer vision tasks such as object detection and image segmentation.

[0059] Then, the moments() function is used to determine the center point of each spot according to the first moment of each spot contour, that is, the first center point position coordinates (pixel coordinates) of each spot. The moments() function in OpenCV is a kind of contour feature function, which is used to calculate the moment of an image. In this way, a first spot position array storing a plurality of first center point position coordinates is obtained.

[0060] The second detection unit 2 detects the iris contour point position of the second original eye image to obtain a preset number of iris contour point coordinates.

[0061] In the embodiment of the present application, the first original eye image and the second original eye image are the same original eye image, or the first original eye image and the second original eye image are different original eye images obtained under the same acquisition condition. The same acquisition condition includes the same eye image acquisition device, the same target eye, the same target eye fixation point, the same target eye position, and the same arrangement position of each light source in the light source group.

[0062] Further, the preset number can be 8, i.e., 8 iris contour point coordinates are obtained by performing iris contour point position detection on the second original eye diagram.

[0063] The iris contour is drawn on the preset image by the exclusion unit 3 using each iris contour point coordinate.

[0064] For the preset image, on one hand, the preset image has the same size as the first original eye diagram, and on the other hand, the preset image has a first preset color.

[0065] Further, in the embodiment of the present application, the iris contour is drawn on the preset image by ellipse fitting using the preset number of iris contour point coordinates. The ellipse fitting of the contour means that the shape of the contour is approximated by an ellipse. When the major axis and the minor axis of the ellipse are equal, it is a circle. The basic idea of ellipse fitting is that for a set of sample points on a given plane, an ellipse is found that is as close as possible to these sample points. That is, a set of data in the image is fitted with an ellipse equation as a model, so that a certain ellipse equation can meet these data as much as possible, and each parameter of the ellipse equation is solved. In OpenCV, three functions are provided to realize ellipse fitting, and the three functions are fitEllipse() function, fitEllipseAMS() function and fitEllipseDirect() function.

[0066] Next, the exclusion unit 3 performs the following steps: filling the internal region of the drawn iris contour as a second preset color to obtain a reference image; extracting pixel colors from the reference image according to each first glint position coordinate in the first glint position array; and identifying the extracted pixel colors corresponding to each first glint position coordinate.

[0067] In this way, if the extracted pixel color is the first preset color, the corresponding glint is not in the iris contour, and the corresponding first glint position coordinate is deleted. If the extracted pixel color is the second preset color, the corresponding glint is in the iris contour, and the corresponding first glint position coordinate is retained as a second glint position coordinate. The exclusion unit 3 obtains the second glint position array as the to-be-numbered glint position array using all the second glint position coordinates.

[0068] In the embodiments of the present application, for example, the first preset color is black, the second preset color is white, and the preset image is a full black image with the same size as the first original eye diagram. In this way, the inside of the iris contour drawn on a full black image with the same size as the first original eye diagram is filled with white, and the reference image after filling the inside of the iris contour with white is denoted as img cornea0. After identifying the pixel color corresponding to each first light spot position coordinate according to the above steps, if the extracted pixel color is white, it is considered that the light spot is within the iris contour, and if the extracted pixel color is black, it is considered that the light spot is not within the iris contour, that is, the light spot is outside the iris contour. The above steps ensure that the second light spot position coordinates in the second light spot position array are all within the ellipse-fitted iris contour.

[0069] Compared with the prior art, the interference light spot screening method provided by the embodiments of the present application detects the iris contour point coordinates based on the iris contour point positions, then draws an iris contour on a preset image and fills the iris contour with color by using the iris contour point coordinates, to obtain a reference image; extracts the pixel color of the reference image by using each first light spot position coordinate obtained by light spot position detection, and identifies the extracted pixel color by comparison, to delete the detection values corresponding to the outside of the iris contour from the first light spot position array, thereby effectively improving the accuracy of light spot position detection. The embodiments of the present application can identify and delete the interference light spot formed by the opposite light source in light spot position detection, to obtain a correct light spot position detection result. In this way, the correct light spot position detection result is numbered, and the accuracy of light spot numbering is also improved.

[0070] Preferably, the step of detecting the light spot position of the first original eye diagram by the first detection unit 1 further includes: determining the first pupil center position coordinate in the first original eye diagram; setting the first horizontal and vertical cutting pixel value ranges based on the iris size in the first original eye diagram; and cutting the first original eye diagram to obtain a to-be-detected eye diagram by taking the first pupil center position coordinate and the first horizontal and vertical cutting pixel value ranges as first preset conditions. The iris region in the first original eye diagram is contained in the to-be-detected eye diagram.

[0071] Correspondingly, in the specific steps of the above-mentioned interference spot screening method disclosed in the embodiments of the present application, after filling the internal region of the drawn iris contour with the second preset color, the method further comprises: determining the second pupil center position coordinates in the second original eye map; setting the second transverse and longitudinal cutting pixel value range based on the iris size in the second original eye map; and cutting the preset image filled with the second preset color to obtain a reference image, taking the second pupil center position coordinates and the second transverse and longitudinal cutting pixel value range as the second preset condition. The iris region in the second original eye map is contained in the reference image, and the size of the reference image after cutting according to the second preset condition is the same as the size of the to-be-detected eye map after cutting according to the first preset condition.

[0072] As described above, the first original eye map and the preset image filled with the second preset color are cut in the same way, and the reference image is cut to an image with the same size as the to-be-detected eye map obtained after cutting. At this time, the pupil center is still the center point of the eye map after cutting, and the reference image after cutting is denoted as img_cornea. In this way, the image region outside the iris contour is cut before the pixel color is extracted, which has no effect on the accuracy of the result, and at the same time, the data processing amount of the pixel color extraction and identification steps is reduced, and the processing efficiency is greatly improved.

[0073] In the embodiments of the present application, if the first original eye map is cut according to the above-mentioned first preset condition to obtain the to-be-detected eye map, after obtaining the to-be-numbered spot position array, each coordinate value in the to-be-numbered spot position array needs to be restored to the corresponding coordinate value in the first original eye map according to the first preset condition.

[0074] Referring to FIG. 3, the interference spot screening method disclosed in the embodiments of the present application further comprises the following steps: determining, by the exclusion unit 3, whether the number of second spot position coordinates in the second spot position array is greater than or equal to a first preset threshold. Specifically, the first preset threshold in the above-mentioned step can be 3.

[0075] If the number of second spot position coordinates in the second spot position array is greater than or equal to the first preset threshold, first, the exclusion unit 3 establishes a spot coordinate system based on each second spot position coordinate.

[0076] Further, in the embodiments of the present application, in the process of establishing the above-mentioned light spot coordinate system, the centroid position of each second light spot position coordinate is taken as the origin position. Specifically, the centroid position coordinate is calculated based on each second light spot position coordinate, that is, the centroid position horizontal coordinate is the average of each second light spot position horizontal coordinate, and the centroid position vertical coordinate is the average of each second light spot position vertical coordinate. The centroid position coordinate is taken as the origin of the above-mentioned light spot coordinate system, and the positive direction of the X-axis of the light spot coordinate system and the positive direction of the Y-axis of the light spot coordinate system are determined, for example, the right direction is taken as the positive direction of the X-axis of the light spot coordinate system, and the downward direction is taken as the positive direction of the Y-axis of the light spot coordinate system, to obtain the above-mentioned light spot coordinate system.

[0077] Then, the exclusion unit 3 sorts each second light spot position coordinate in the second light spot position array according to a first preset rule to obtain a third light spot position array.

[0078] In the embodiments of the present application, preferably, the above-mentioned first preset rule is a clockwise sorting rule. Specifically, the above-mentioned step specifically includes: obtaining a corresponding to-be-screened light spot vector by using each second light spot position coordinate and the origin coordinate of the light spot coordinate system, obtaining the X-axis positive direction included angle θ x between each to-be-screened light spot vector and the positive direction of the X-axis of the light spot coordinate system according to the tangent function tan(x / y), and converting each X-axis positive direction included angle θ x into a Y-axis positive direction included angle θ y between the corresponding to-be-screened light spot vector and the positive direction of the Y-axis of the light spot coordinate system according to the arctangent function arctan(y / x). If there is a second light spot position coordinate, which is the i-th (i≤n1, n1 is the total number of second light spot position coordinates in the second light spot position array) in the second light spot position array, when the Y-axis positive direction included angle θ y (i) corresponding to the second light spot position coordinate is 0°, then the second light spot position coordinate is in the positive direction of the Y-axis of the light spot coordinate system; when the Y-axis positive direction included angle θ y (i) corresponding to the second light spot position coordinate is 90°, then the second light spot position coordinate is in the negative direction of the X-axis of the light spot coordinate system; when the Y-axis positive direction included angle θ y (i) corresponding to the second light spot position coordinate is 180°, then the second light spot position coordinate is in the negative direction of the Y-axis of the light spot coordinate system; when the Y-axis positive direction included angle θ y (i) corresponding to the second light spot position coordinate is 270°, then the second light spot position coordinate is in the positive direction of the X-axis of the light spot coordinate system. In this way, based on the above-mentioned clockwise sorting rule, each corresponding second light spot position coordinate is sorted in ascending order of each Y-axis positive direction included angle θ y . A new third light spot position array and a new Y-axis positive direction included angle θ yn array are obtained. The third light spot position array has the characteristic of clockwise sorting.

[0079] In addition, the first preset rule can also be a counterclockwise ordering rule. In this way, based on the clockwise ordering rule, the first vector included angle θ y The corresponding second light spot position coordinates are sorted in descending order to obtain a new third light spot position array and a new Y-axis positive direction included angle θ yn Array. At this time, the third light spot position array has the characteristics of counterclockwise ordering.

[0080] Then, the exclusion unit 3 detects whether the first vector included angle corresponding to the adjacent two third light spot position coordinates in the third light spot position array is greater than or equal to a second preset threshold value in order of sorting.

[0081] If the first vector included angle is greater than or equal to the second preset threshold value, the corresponding adjacent two third light spot position coordinates are determined as a first light spot position pair without interference light spots; if the first vector included angle is less than the second preset threshold value, the corresponding adjacent two third light spot position coordinates are determined as a second light spot position pair with interference light spots.

[0082] In the embodiment of the application, the second preset threshold value is denoted as θmax, and the above step is described below with θmax set to 5°. If there is a third light spot position coordinate, which is the i-th (i < n2, n2 is the total number of second light spot position coordinates in the second light spot position array) in the third light spot position array, the first vector included angle is calculated according to the following formula: θ yd (i) = | θ yn (i+1) - θ yn (i)|

[0083] Wherein, θ yd (i) is the first vector included angle corresponding to the i-th third light spot position coordinate and the i+1-th third light spot position coordinate in the third light spot position array, θ yn (i) is the Y-axis positive direction included angle corresponding to the i-th third light spot position coordinate in the third light spot position array, θ yn (i+1) is the Y-axis positive direction included angle corresponding to the i+1-th third light spot position coordinate in the third light spot position array.

[0084] If there is θ yd (i) is greater than or equal to 5°, the corresponding adjacent two third light spot position coordinates are determined as a first light spot position pair without interference light spots; if there is θ yd (i) is less than 5°, it is considered that interference light spot position coordinates are mixed by the opposite light source, and the corresponding adjacent two third light spot position coordinates are determined as a second light spot position pair with interference light spots.

[0085] Then, the eye diagram horizontal coordinate system is established by the exclusion unit 3, and the coordinates of the two third spot positions of each second spot position pair are determined to be the transformation coordinates of the two first spot positions in the eye diagram horizontal coordinate system.

[0086] In this embodiment of the application, referring to Figure 4, the above steps specifically include: establishing a three-dimensional world coordinate system based on the target face; detecting the first left eye center position coordinate and the first right eye center position coordinate of the target face in the three-dimensional world coordinate system; transforming the first left eye center position coordinate and the first right eye center position coordinate to the camera coordinate system of the first original eye map to obtain the corresponding second left eye center position coordinate and second right eye center position coordinate; using the second left eye center position coordinate and the second right eye center position coordinate, calculating the slope of the horizontal line of the eye map, the horizontal line of the eye map being the line connecting the projection of the left eye center and the right eye center of the target face in the camera coordinate system; selecting any point from the camera coordinate system as the origin, determining the coordinate axis direction by the slope of the horizontal line of the eye map, and establishing the horizontal line coordinate system of the eye map.

[0087] As mentioned above, establishing the horizontal coordinate system of the eye diagram requires calculating the slope of the line formed by the horizontal line connecting the two eyes in the eye diagram within the three-dimensional world coordinate system. The three-dimensional world coordinate system has its origin (0,0,0) at the bridge of the nose of the target face. The X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the z-axis represents the direction perpendicular to the plane formed by the X and Y axes. The coordinates of the center positions of the first left eye and the first right eye can be represented as E... left and E right For example, E left =(-31.5,0,0), E right = (31.5, 0, 0). The eye diagram horizontal coordinate system is established assuming the user is wearing the head-mounted display normally. Typically, the line connecting the three-dimensional coordinates of the left and right eye centers is parallel to the line connecting the optical centers of the left and right physical lenses of the head-mounted display, with only a translational relationship between the two (ignoring line lengths). The first left and right eye center coordinates are transformed into the camera coordinate system and then into image pixel coordinates to obtain the corresponding second left and right eye center coordinates, denoted as P. left and P right The coordinates of the center position of the second left eye are P. left The coordinates of the center position of the second right eye P right The line connecting them is denoted as P. l P r That is, the coordinates E of the center position of the first left eye in the three-dimensional world coordinate system. left and the coordinates of the center position of the first right eye E right The line E between them l Er a line segment projected onto the first original eye diagram.

[0088] In the above use scenario layout of the embodiments of the present application, the second left eye center position coordinate P left is (X_Pleft, Y_Pleft), the second right eye center position coordinate Pright is (X_Pright, Y_Pright), and the slope k of the line PlPr between the second left eye center position coordinate Pleft and the second right eye center position coordinate Pright is calculated according to the following formula: k = |Y_P right -Y_P left | / |X_P right -X_P left |

[0089] In this way, the line E left E right between the first left eye center position coordinate E l E r in the three-dimensional world coordinate system can be represented as a line segment with a slope k in the camera coordinate system of the first original eye diagram. Based on the camera coordinate system of the first original eye diagram, a new eye diagram horizontal line coordinate system is established, which can select any point in the camera coordinate system of the first original eye diagram as the origin, for example, the eye diagram horizontal line coordinate system selects the following points as the origin: the center point of the first original eye diagram, the pupil center position of the first original eye diagram, the origin (0, 0) of the first original eye diagram pixel coordinate system, and a certain point on the line P left P right between the second left eye center position coordinate P l P r . The eye diagram horizontal line takes the line segment y = kx + b with a slope k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0090] Then, the exclusion unit 3 compares the sizes of the corresponding two first light spot position conversion coordinates, and identifies the third light spot position coordinate corresponding to the interference light spot according to a second preset rule.

[0091] In the embodiments of the present application, the above-mentioned second preset rule includes: identifying the third light spot position coordinate corresponding to the maximum value / minimum value of the two first light spot position conversion coordinates as the third light spot position coordinate corresponding to the interference light spot.

[0092] Specifically, in the embodiments of the present application, according to the different establishment manners of the above-mentioned various coordinate systems, the above-mentioned second preset rule has multiple judgment rules, for example, the third spot position coordinates corresponding to the maximum / minimum value of the horizontal coordinate x in the two first spot position conversion coordinates are identified as the third spot position coordinates corresponding to the interference spot.

[0093] The following only lists an example in the above-mentioned specific use scenario. As described above, taking the first original eye pattern collected for the left eye as an example, according to the establishment manner of the above-mentioned various coordinate systems, the positive direction of the X-axis of the eye pattern horizontal line coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . By converting the two third spot position coordinates of each second spot position pair in which the interference spot exists into the representation in the new eye pattern horizontal line coordinate system, the most likely interference spot formed by the opposite light source can be judged according to the x value of the corresponding two first spot position conversion coordinates in the new eye pattern horizontal line coordinate system. Taking the first original eye pattern collected for the left eye as an example, the specific rule is that the greater the x value of the corresponding first spot position conversion coordinate, the closer to the other eye (right eye), and the third spot position coordinate corresponding to the maximum value of the x value of the corresponding two first spot position conversion coordinates is the most likely interference spot formed by the opposite light source, which is identified as the third spot position coordinate corresponding to the interference spot.

[0094] In addition, the following is an example in the above-mentioned specific use scenario. As described above, taking the first original eye pattern collected for the right eye as an example, according to the establishment manner of the above-mentioned various coordinate systems, the positive direction of the X-axis of the eye pattern horizontal line coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . By converting the two third spot position coordinates of each second spot position pair in which the interference spot exists into the representation in the new eye pattern horizontal line coordinate system, the most likely interference spot formed by the opposite light source can be judged according to the x value of the corresponding two first spot position conversion coordinates in the new eye pattern horizontal line coordinate system. Taking the first original eye pattern collected for the right eye as an example, the specific rule is that the smaller the x value of the corresponding first spot position conversion coordinate, the closer to the other eye (left eye), and the third spot position coordinate corresponding to the minimum value of the x value of the corresponding two first spot position conversion coordinates is the most likely interference spot formed by the opposite light source, which is identified as the third spot position coordinate corresponding to the interference spot.

[0095] Then, the third spot position coordinates corresponding to all the identified interference spots are deleted from the third spot position array by the excluding unit 3 to obtain a fourth spot position array as a to-be-numbered spot position array.

[0096] In addition, if the number of second spot position coordinates in the second spot position array is less than the first preset threshold, the second spot position array is directly stored as the fourth spot position array as the to-be-numbered spot position array.

[0097] After the interference spots are screened by using the above-described second preset rule, there can still be interference spots formed by the opposite light source. Generally, the interference spots formed by the opposite light source not only have the features described in the above-described second preset rule, but also have the features that the distance between a spot and an interference spot and the distance between interference spots are small. In the embodiments of the present application, whether there is still an interference spot can be further determined according to the distance between each spot and other spots in the fourth spot position array.

[0098] Referring to FIG. 5, the interference spot screening method disclosed in the embodiments of the present application further includes the following steps: the excluding unit 3 calculates a first distance between any two fourth spot position coordinates in the fourth spot position array. Then, the excluding unit 3 determines whether the first distance is less than or equal to a third preset threshold. If the first distance is greater than the third preset threshold, the corresponding two fourth spot position coordinates are determined as a third spot position pair without interference spots; if the first distance is less than or equal to the third preset threshold, the corresponding two fourth spot position coordinates are determined as a fourth spot position pair with interference spots.

[0099] Generally, taking a certain actual use scene as an example, the minimum value of the distance between the correct eye pattern spot coordinates formed in the scene is greater than 40 pixels. If the opposite light source forms several interference spots on the eye pattern, generally, the maximum value of the distance between these interference spots is about 18 pixels. Therefore, in the embodiments of the present application, the third preset threshold can be set to 18 pixels. In this way, whether there is an interference spot is determined by judging whether there is a first distance between two fourth spot position coordinates in the fourth spot position array is less than or equal to 18 pixels.

[0100] If the fourth spot position pair with interference spots is identified by the above-described third preset threshold, similarly, referring to FIG. 5, the excluding unit 3 establishes an eye pattern horizontal line coordinate system to determine two second spot position conversion coordinates corresponding to the two fourth spot position coordinates of each fourth spot position pair in the eye pattern horizontal line coordinate system.

[0101] In the embodiment of the present application, with reference to FIG. 4, the above steps specifically include: establishing a three-dimensional world coordinate system based on the target face; detecting a first left eye center position coordinate and a first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate into a camera coordinate system of a first original eye map to obtain corresponding second left eye center position coordinates and second right eye center position coordinates; calculating a slope of an eye map horizontal line using the second left eye center position coordinates and the second right eye center position coordinates, the eye map horizontal line being a projection connecting line of the left eye center and the right eye center of the target face in the camera coordinate system; and selecting an arbitrary point in the camera coordinate system as an origin and determining the coordinate axis direction with the slope of the eye map horizontal line to establish an eye map horizontal line coordinate system.

[0102] As described above, the establishment of the eye map horizontal line coordinate system requires calculation of the slope of the line formed by the horizontal line of the connecting line of the two eyes in the eye map in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the bridge of the nose of the target face as the origin (0, 0, 0), the X axis represents the horizontal direction, the Y axis represents the vertical direction, and the z axis represents the vertical direction of the plane formed by the X axis and the Y axis. The first left eye center position coordinate and the first right eye center position coordinate can be represented as E left and E right , respectively. For example, E left = (-31.5, 0, 0) and E right = (31.5, 0, 0). The establishment of the eye map horizontal line coordinate system is based on the assumption that the user normally wears the head-mounted device, and generally, the connecting line between the three-dimensional position coordinates of the left eye center and the three-dimensional position coordinates of the right eye center is parallel to the connecting line between the optical centers of the physical lenses of the left eye and the right eye of the head-mounted device, and the former only has a translation relationship (ignoring the length of the line segment) with the latter. The first left eye center position coordinate and the first right eye center position coordinate are converted into the camera coordinate system and into image pixel coordinates to obtain the corresponding second left eye center position coordinates and second right eye center position coordinates, which are denoted as P left and P right , respectively. The connecting line between the second left eye center position coordinates P left and the second right eye center position coordinates P right is denoted as P l P r , that is, the connecting line E left E right between the first left eye center position coordinate E l and the first right eye center position coordinate E r in the three-dimensional world coordinate system is projected onto the line segment of the first original eye map.

[0103] In the above use scene layout of the embodiment of the present application, the second left eye center position coordinates P leftGiven (X_Pleft, Y_Pleft) and the center position of the second left eye, Pright, also (X_Pright, Y_Pright), calculate the slope kk of the line PlPr connecting the center positions of the second left eye (Pleft) and the second right eye (Pright) using the following formula: |Y_Pleft|Pright ... right -Y_P left | / |X_P right -X_P left |

[0104] Thus, the coordinates E of the center position of the first left eye in the three-dimensional world coordinate system left and the coordinates of the center position of the first right eye E right The line E between them l E r In the camera coordinate system of the first original eye image, this can be represented as a line segment with a slope of k. Based on the camera coordinate system of the first original eye image, a new horizontal eye image coordinate system is established. This horizontal eye image coordinate system can select any point in the camera coordinate system of the first original eye image as its origin. For example, the horizontal eye image coordinate system can select the following points as its origin: the center point of the first original eye image, the pupil center position of the first original eye image, the origin (0,0) of the pixel coordinate system of the first original eye image, and the coordinates of the second left eye center position P. left The coordinates of the center position of the second right eye P right The line connecting P l P r A point on the horizon. The horizontal line of the eye diagram has the line segment y = kx + b with a downward slope of k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0105] The elimination unit 3 compares the magnitudes of the coordinate transformations of the two corresponding second light spot positions and identifies the coordinates of the fourth light spot position corresponding to the interfering light spot according to the second preset rule.

[0106] In this embodiment of the application, the second preset rule includes: identifying the fourth spot position coordinates corresponding to the maximum / minimum value among the two second spot position transformation coordinates as the fourth spot position coordinates corresponding to the interference spot.

[0107] Specifically, in this embodiment, depending on the different ways of establishing the coordinate systems, the second preset rule has multiple judgment rules. For example, the fourth spot position coordinate corresponding to the maximum / minimum value of the x-coordinate in the two second spot position transformation coordinates is identified as the fourth spot position coordinate corresponding to the interfering spot. Alternatively, the fourth spot position coordinate corresponding to the maximum / minimum value of the y-coordinate in the two second spot position transformation coordinates is identified as the fourth spot position coordinate corresponding to the interfering spot.

[0108] The following is only an example of the above-mentioned specific use scenario. As described above, taking the first original eye diagram collected for the left eye as an example, according to the establishment mode of each coordinate system, the positive direction of the X-axis of the eye diagram horizontal line coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . By converting the two fourth light spot position coordinates of each fourth light spot position pair in which the interference light spot exists into the new eye diagram horizontal line coordinate system, it can be determined that the interference light spot most likely formed by the opposite light source according to the x value of the corresponding two second light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected for the left eye as an example, the specific rule is that the larger the x value of the corresponding first light spot position conversion coordinate, the closer to the other eye (right eye), and the minimum value of the x value of the corresponding two second light spot position conversion coordinates corresponds to the fourth light spot position coordinate most likely formed by the interference light spot of the opposite light source, which is identified as the fourth light spot position coordinate corresponding to the interference light spot.

[0109] In addition, the above-mentioned specific use scenario. As described above, taking the first original eye diagram collected for the right eye as an example, according to the establishment mode of each coordinate system, the positive direction of the X-axis of the eye diagram horizontal line coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . By converting the two fourth light spot position coordinates of each fourth light spot position pair in which the interference light spot exists into the new eye diagram horizontal line coordinate system, it can be determined that the interference light spot most likely formed by the opposite light source according to the x value of the corresponding two second light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected for the right eye as an example, the specific rule is that the smaller the x value of the corresponding first light spot position conversion coordinate, the closer to the other eye (left eye), and the minimum value of the x value of the corresponding two second light spot position conversion coordinates corresponds to the fourth light spot position coordinate most likely formed by the interference light spot of the opposite light source, which is identified as the fourth light spot position coordinate corresponding to the interference light spot.

[0110] Then, the fourth light spot position coordinates corresponding to all the identified interference light spots are deleted from the fourth light spot position array by the exclusion unit 3, and the fifth light spot position array is obtained as the to-be-numbered light spot position array.

[0111] Referring to FIG. 6, the interference spot screening method disclosed in the embodiment of the present application further includes the following steps: the exclusion unit 3 sequentially traverses each fifth spot position coordinate in the fifth spot position array and the second distance between the remaining each fifth spot position coordinate to obtain a second distance array corresponding to each fifth spot position coordinate according to the sorting order. The exclusion unit 3 obtains the corresponding second distance minimum value and the second distance maximum value from each second distance array. And calculates the first extreme value ratio of each second distance maximum value divided by the corresponding second distance minimum value.

[0112] Then, the exclusion unit 3 determines whether each first extreme value ratio is greater than a fourth preset threshold value. If the first extreme value ratio is greater than the fourth preset threshold value, the corresponding fifth spot position coordinate is determined as the first abnormal spot position coordinate. If the first extreme value ratio is less than or equal to the fourth preset threshold value, the corresponding fifth spot position coordinate is determined as the first normal spot position coordinate.

[0113] In the embodiment of the present application, in a certain scenario, the above step of determining whether there is still an interference spot according to the first extreme value ratio is described in detail by taking the fourth preset threshold value of 3.5 as an example. The second distance between each fifth spot position coordinate in the fifth spot position array and the remaining each fifth spot position coordinate is sequentially traversed and calculated, for example, the second distance between the i-th fifth spot position coordinate and the remaining each fifth spot position coordinate is calculated, where i is an integer greater than or equal to 1 and less than or equal to n3, n3 is the total number of fifth spot position coordinates in the fifth spot position array, the second distance array corresponding to the i-th fifth spot position coordinate is denoted as Length, the second distance maximum value in the second distance array is denoted as l_max, the second distance minimum value in the second distance array is denoted as l_min, and the first extreme value ratio of the second distance maximum value l_max divided by the second distance minimum value l_min is denoted as l ratio . As described above, if the first extreme value ratio l ratio is greater than 3.5, the corresponding fifth spot position coordinate is determined as the first abnormal spot position coordinate. For example, the second distance maximum value l_max corresponds to the second distance between the i-th fifth spot position coordinate and the j-th fifth spot position coordinate, and the second distance minimum value l_min corresponds to the second distance between the i-th fifth spot position coordinate and the k-th fifth spot position coordinate. It can be considered that the i-th fifth spot position coordinate may be abnormal and is determined as the first abnormal spot position coordinate.

[0114] Then, the exclusion unit 3 generates a first abnormal light spot position array by acquiring all the first abnormal light spot position coordinates, and judges whether the number of the first abnormal light spot position coordinates in the first abnormal light spot position array is greater than a fifth preset threshold. If the number of the first abnormal light spot position coordinates in the first abnormal light spot position array is greater than the fifth preset threshold, the exclusion unit 3 establishes an eye diagram horizontal line coordinate system, and determines the third light spot position conversion coordinates corresponding to each first abnormal light spot position coordinate in the eye diagram horizontal line coordinate system.

[0115] In the embodiment of the present application, the fifth preset threshold can be set to 2. If the number of the first abnormal light spot position coordinates in the first abnormal light spot position array is greater than 2, the fifth light spot position coordinates corresponding to the interference light spot are further identified by establishing the eye diagram horizontal line coordinate system and are deleted from the fifth light spot position array. After deleting the fifth light spot position coordinates corresponding to the interference light spot, the above steps of judging the first abnormal light spot position coordinates by the first extreme value ratio are repeatedly performed until the number of the first abnormal light spot position coordinates is less than or equal to 2.

[0116] In the embodiment of the present application, with reference to FIG. 4, the step of establishing the eye diagram horizontal line coordinate system specifically includes: establishing a three-dimensional world coordinate system based on the target face; detecting the first left eye center position coordinate and the first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate to the camera coordinate system of the first original eye diagram to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate; calculating the slope of the eye diagram horizontal line by using the second left eye center position coordinate and the second right eye center position coordinate, the eye diagram horizontal line being the projection connecting line of the left eye center and the right eye center of the target face in the camera coordinate system; selecting any point as the origin in the camera coordinate system, and determining the coordinate axis direction by the slope of the eye diagram horizontal line to establish the eye diagram horizontal line coordinate system.

[0117] As described above, the establishment of the eye diagram horizontal line coordinate system requires calculating the slope of the line formed by the horizontal line of the connecting line of the two eyes in the eye diagram in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the nose bridge of the target face as the origin (0, 0, 0), the X axis represents the horizontal direction, the Y axis represents the vertical direction, and the z axis represents the vertical direction of the plane formed by the X axis and the Y axis. The first left eye center position coordinate and the first right eye center position coordinate can be represented as E left and E right , respectively. For example, E left = (-31.5, 0, 0), and E right= (31.5, 0, 0). The eye diagram horizontal coordinate system is established assuming the user is wearing the head-mounted display normally. Typically, the line connecting the three-dimensional coordinates of the left and right eye centers is parallel to the line connecting the optical centers of the left and right physical lenses of the head-mounted display, with only a translational relationship between the two (ignoring line lengths). The first left and right eye center coordinates are transformed into the camera coordinate system and then into image pixel coordinates to obtain the corresponding second left and right eye center coordinates, denoted as P. left and P right The coordinates of the center position of the second left eye are P. left The coordinates of the center position of the second right eye P right The line connecting them is denoted as P. l P r That is, the coordinates E of the center position of the first left eye in the three-dimensional world coordinate system. left and the coordinates of the center position of the first right eye E right The line E between them l E r Line segments projected onto the first original eye diagram.

[0118] In the above-described usage scenario layout of this application embodiment, the coordinates P of the center position of the second left eye are... left Given (X_Pleft, Y_Pleft) and the center position Pright of the second right eye, calculate the slope kk of the line PlPr connecting the center positions Pleft and Pright of the second left and second right eyes using the following formula: kk = |Y_Pleft|Pright ... right -Y_P left | / |X_P right -X_P left |

[0119] Thus, the coordinates E of the center position of the first left eye in the three-dimensional world coordinate system left and the coordinates of the center position of the first right eye E right The line E between them l E r In the camera coordinate system of the first original eye image, this can be represented as a line segment with a slope of k. Based on the camera coordinate system of the first original eye image, a new horizontal eye image coordinate system is established. This horizontal eye image coordinate system can select any point in the camera coordinate system of the first original eye image as its origin. For example, the horizontal eye image coordinate system can select the following points as its origin: the center point of the first original eye image, the pupil center position of the first original eye image, the origin (0,0) of the pixel coordinate system of the first original eye image, and the coordinates of the second left eye center position P. left The coordinates of the center position of the second right eye P rightthe connection P between the points P l P r a certain point on the horizontal line of the eye diagram. The horizontal line of the eye diagram is a line segment y=kx+b with a slope k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram), where the value of b depends on the origin of the new horizontal line of the eye diagram coordinate system.

[0120] Then, the exclusion unit 3 compares the sizes of the third spot position conversion coordinates, and identifies the fifth spot position coordinates corresponding to the interference spots according to a second preset rule. The fifth spot position coordinates corresponding to the interference spots are deleted from the fifth spot position array, and the process is repeated to perform the second distance traversal calculation on the currently retained fifth spot position coordinates.

[0121] In the embodiments of the present application, the second preset rule includes: identifying the fifth spot position coordinates corresponding to the maximum / minimum values of the third spot position conversion coordinates as the fifth spot position coordinates corresponding to the interference spots.

[0122] Specifically, in the embodiments of the present application, according to the different establishment methods of the above-mentioned coordinate systems, the second preset rule has multiple judgment rules, for example, identifying the fifth spot position coordinates corresponding to the maximum / minimum values of the x coordinates of the two third spot position conversion coordinates as the fifth spot position coordinates corresponding to the interference spots. Or identifying the fifth spot position coordinates corresponding to the maximum / minimum values of the y coordinates of the two third spot position conversion coordinates as the fifth spot position coordinates corresponding to the interference spots.

[0123] The following only lists an example in a specific use scenario. As described above, taking the first original eye diagram collected for the left eye as an example, according to the establishment method of the above-mentioned coordinate systems, the positive direction of the X axis of the horizontal line of the eye diagram coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . Converting the first abnormal spot position coordinates with interference spots into the new horizontal line of the eye diagram coordinate system can determine the most likely interference spots formed by the opposite light source according to the x value of the corresponding third spot position conversion coordinates in the new horizontal line of the eye diagram coordinate system. Taking the first original eye diagram collected for the left eye as an example, the specific rule is that the larger the x value of the corresponding third spot position conversion coordinates, the closer to the other eye (right eye), and the fifth spot position coordinate corresponding to the maximum x value of the corresponding third spot position conversion coordinates is the most likely interference spot formed by the opposite light source, which is identified as the fifth spot position coordinate corresponding to the interference spot.

[0124] In addition, the above example in the specific use scenario. As described above, taking the first original eye diagram collected for the right eye as an example, according to the establishment mode of each coordinate system, the positive direction of the X-axis of the eye diagram horizontal line coordinate system is the direction of the vector from the second right eye center coordinate P right to the second left eye center coordinate P left . Converting each first abnormal light spot position coordinate where the interference light spot exists into a new eye diagram horizontal line coordinate system representation can determine the most likely interference light spot formed by the opposite light source according to the x value of the corresponding third light spot position conversion coordinate in the new eye diagram horizontal line coordinate system representation. Taking the first original eye diagram collected for the left eye as an example, the specific rule is that the smaller the x value of the corresponding third light spot position conversion coordinate, the closer to the other eye (left eye). The minimum value of the x value of the corresponding third light spot position conversion coordinate corresponds to the fifth light spot position coordinate that is most likely to be an interference light spot formed by the opposite light source, and the fifth light spot position coordinate is identified as the fifth light spot position coordinate corresponding to the interference light spot.

[0125] If the number of first abnormal light spot position coordinates in the first abnormal light spot position array is less than or equal to the fifth preset threshold, the exclusion unit 3 takes the current retained each fifth light spot position coordinate as the sixth light spot position coordinate, and obtains the sixth light spot position array as the to-be-numbered light spot position array.

[0126] Referring to FIG. 7, the interference light spot screening method disclosed in the embodiment of the present application further includes the following steps: the exclusion unit 3 takes each sixth light spot position coordinate in the sixth light spot position array as a vertex in turn according to the sorting order; and iteratively calculates the edge included angle formed by the connection line between the vertex and any other two sixth light spot position coordinates in the sixth light spot position array to obtain the corresponding edge included angle set.

[0127] The exclusion unit 3 determines whether there is an edge included angle that meets the second preset condition in each edge included angle set. If there is an edge included angle that meets the second preset condition in the current edge included angle set, the sixth light spot position coordinate corresponding to the vertex is determined as the second abnormal light spot position coordinate; if there is no edge included angle that meets the second preset condition in the current edge included angle set, the sixth light spot position coordinate corresponding to the vertex is determined as the second normal light spot position coordinate.

[0128] In the embodiment of the present application, the above-mentioned second preset condition includes that the edge included angle is less than the fifth preset threshold or the edge included angle is greater than the sixth preset threshold. Specifically, the fifth preset threshold is denoted as Angle_max, and Angle_max can be set to 5°, and the sixth preset threshold is set to 180°-Angle_max.

[0129] Then, the second abnormal light spot position array is generated by the exclusion unit 3 based on all the second abnormal light spot position coordinates, and an eye diagram horizontal line coordinate system is established to determine the fourth light spot position conversion coordinates corresponding to each second abnormal light spot position coordinate in the second abnormal light spot position array in the eye diagram horizontal line coordinate system.

[0130] In the embodiment of the present application, with reference to FIG. 4, the above steps specifically include: establishing a three-dimensional world coordinate system based on the target face; detecting a first left eye center position coordinate and a first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate into a camera coordinate system of a first original eye diagram to obtain corresponding second left eye center position coordinates and second right eye center position coordinates; calculating a slope of an eye diagram horizontal line by using the second left eye center position coordinates and the second right eye center position coordinates, the eye diagram horizontal line being a projection connecting line of the left eye center and the right eye center of the target face in the camera coordinate system; selecting an arbitrary point as an origin in the camera coordinate system, and determining the coordinate axis direction by the slope of the eye diagram horizontal line to establish an eye diagram horizontal line coordinate system.

[0131] As described above, the establishment of the eye diagram horizontal line coordinate system requires calculation of the slope of the line formed by the horizontal line of the connecting line of the two eyes in the eye diagram in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the nose bridge of the target face as the origin (0, 0, 0), the X axis represents the horizontal direction, the Y axis represents the vertical direction, and the z axis represents the vertical direction of the plane formed by the X axis and the Y axis. The first left eye center position coordinate and the first right eye center position coordinate can be represented as E left and E right , respectively. For example, E left = (-31.5, 0, 0) and E right = (31.5, 0, 0). The establishment of the eye diagram horizontal line coordinate system is based on the assumption that the user normally wears the head-mounted device. Generally, the connecting line between the three-dimensional position coordinates of the left eye center and the three-dimensional position coordinates of the right eye center is parallel to the connecting line between the optical centers of the physical lenses of the left eye and the right eye of the head-mounted device, and the former only has a translation relationship (ignoring the length of the line segment) with the latter. The first left eye center position coordinate and the first right eye center position coordinate are converted into the camera coordinate system and into image pixel coordinates to obtain the corresponding second left eye center position coordinates and second right eye center position coordinates, which are denoted as P left and P right , respectively. The connecting line between the second left eye center position coordinates P left and the second right eye center position coordinates P right is denoted as P l P r , that is, the first left eye center position coordinate E left and the first right eye center position coordinate Eright the line segment between the first left eye center position coordinate E l E r the line segment between the first left eye center position coordinate E

[0132] In the use scenario layout of the embodiments of the present application, the second left eye center position coordinate Pleft is (X_Pleft, Y_Pleft), the second right eye center position coordinate Pright is (X_Pright, Y_Pright), and the slope k of the line PlPr between the second left eye center position coordinate Pleft and the second right eye center position coordinate Pright is calculated according to the following formula: k = |Y_P right -Y_P left | / |X_P right -X_P left |

[0133] In this way, the first left eye center position coordinate E left and the first right eye center position coordinate E right between the first left eye center position coordinate E l E r can be represented as a line segment with a slope k in the camera coordinate system of the first original eye diagram. Based on the camera coordinate system of the first original eye diagram, a new eye diagram horizontal line coordinate system is established, which can select any point in the camera coordinate system of the first original eye diagram as the origin, for example, the eye diagram horizontal line coordinate system selects the following point as the origin: the center point of the first original eye diagram, the pupil center position of the first original eye diagram, the origin (0, 0) of the first original eye diagram pixel coordinate system, or a certain point on the line P left P right between the second left eye center position coordinate P l P r . The eye diagram horizontal line takes the line segment y = kx + b with a slope k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0134] The exclusion unit 3 compares the sizes of the fourth light spot position conversion coordinates, and identifies the sixth light spot position coordinates corresponding to the interference light spots according to the second preset rule; and deletes the identified sixth light spot position coordinates corresponding to the interference light spots from the sixth light spot position array to obtain a seventh light spot position array as a to-be-numbered light spot position array.

[0135] In the embodiments of the present application, the second preset rule includes: identifying the sixth light spot position coordinates corresponding to the maximum value / minimum value in the fourth light spot position conversion coordinates as the sixth light spot position coordinates corresponding to the interference light spots.

[0136] Specifically, in the embodiments of the present application, according to the different establishment modes of the above-mentioned various coordinate systems, the above-mentioned second preset rule has multiple judgment rules, for example, the sixth light spot position coordinates corresponding to the maximum / minimum value of the horizontal coordinate x in the two fourth light spot position conversion coordinates are identified as the sixth light spot position coordinates corresponding to the interference light spot.

[0137] The following only lists an example in the above-mentioned specific use scenario. As described above, taking the first original eye pattern collected for the left eye as an example, according to the establishment mode of the above-mentioned various coordinate systems, the positive direction of the X-axis of the eye pattern horizontal line coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . By converting the various second abnormal light spot position coordinates in which the interference light spot exists into the representation in the new eye pattern horizontal line coordinate system, the most likely interference light spot formed by the opposite light source can be judged according to the x value of the corresponding fourth light spot position conversion coordinate in the new eye pattern horizontal line coordinate system. Taking the first original eye pattern collected for the left eye as an example, the specific rule is that the larger the x value of the corresponding fourth light spot position conversion coordinate, the closer to the other eye (right eye), and the sixth light spot position coordinate corresponding to the maximum value of the x value of the corresponding fourth light spot position conversion coordinate is the most likely interference light spot formed by the opposite light source, which is identified as the sixth light spot position coordinate corresponding to the interference light spot.

[0138] In addition, the following is an example in the above-mentioned specific use scenario. As described above, taking the first original eye pattern collected for the right eye as an example, according to the establishment mode of the above-mentioned various coordinate systems, the positive direction of the X-axis of the eye pattern horizontal line coordinate system is the direction of the vector from the second right eye center position coordinate P right to the second left eye center position coordinate P left . By converting the various second abnormal light spot position coordinates in which the interference light spot exists into the representation in the new eye pattern horizontal line coordinate system, the most likely interference light spot formed by the opposite light source can be judged according to the x value of the corresponding fourth light spot position conversion coordinate in the new eye pattern horizontal line coordinate system. Taking the first original eye pattern collected for the left eye as an example, the specific rule is that the smaller the x value of the corresponding fourth light spot position conversion coordinate, the closer to the other eye (left eye), and the sixth light spot position coordinate corresponding to the minimum value of the x value of the corresponding fourth light spot position conversion coordinate is the most likely interference light spot formed by the opposite light source, which is identified as the sixth light spot position coordinate corresponding to the interference light spot.

[0139] In addition, the embodiment of the present application further provides a device for screening interference light spots, the device comprising: a processor and a memory; the memory is used for storing one or more program instructions; the processor is used for running the one or more program instructions to execute the steps of the method for screening interference light spots.

[0140] In addition, the embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the method for screening interference light spots.

[0141] In the embodiment of the present application, the processor can be an integrated circuit chip with a signal processing capability. The processor can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0142] The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed by using a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any combination thereof. The steps of the methods disclosed in the embodiments of the present application can be directly embodied as: hardware code executed in a processor, or a combination of hardware and software modules in the processor. The software modules can be located in a storage medium such as random access memory (RAM), flash memory, read only memory (ROM), programmable read only memory (PROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), registers, or other mature storage mediums in the art. The processor reads information in the storage medium and combines the information with hardware to execute the steps of the above methods.

[0143] The storage medium can be a memory, for example, a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0144] The non-volatile memory can be a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory.

[0145] The volatile memory can be a Random Access Memory (RAM), which is used as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The RAM optionally can be a non-volatile cache, which also is used as internal memory for the processor.

[0146] The storage media described in the embodiments of the present application are intended to include, but are not limited to these and any other suitable types of memory.

[0147] Those skilled in the art should be aware that, in one or more examples described above, the functions described in the embodiments of the present application can be implemented in combination of hardware and software. When the software is applied, the corresponding functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on the computer readable medium. The computer readable medium includes a computer storage medium and a communication medium, wherein the communication medium includes any medium that facilitates the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0148] Although the present application has been described in detail with general description and specific embodiments above, some modifications or improvements can be made on the basis of the present application, which is obvious to those skilled in the art. Therefore, these modifications or improvements made on the basis of not deviating from the spirit of the present application, are within the scope of the present application.

Claims

1. A method of interference light spot screening, characterized by, The method comprises: Performing spot position detection on a first original eye diagram to obtain a first spot position array; Performing iris contour point position detection on a second original eye diagram to obtain a preset number of iris contour point coordinates; Drawing an iris contour on a preset image of the same size as the first original eye diagram using each iris contour point coordinate; the preset image has a first preset color; Filling the internal region of the drawn iris contour with a second preset color to obtain a reference image; Extracting pixel colors from the reference image according to each first spot position coordinate in the first spot position array; Identifying the pixel colors extracted corresponding to each first spot position coordinate; If the extracted pixel color is the first preset color, the corresponding spot is not within the iris contour, and the corresponding first spot position coordinate is deleted; If the extracted pixel color is the second preset color, the corresponding spot is within the iris contour, and the corresponding first spot position coordinate is retained as a second spot position coordinate; Using all the second spot position coordinates, a second spot position array is obtained as a to-be-numbered spot position array.

2. A method of interference spot screening as claimed in claim 1, characterized in that, Performing spot position detection on a first original eye diagram to obtain a first spot position array, comprising: Performing adaptive threshold segmentation on the to-be-detected eye diagram to obtain a first eye diagram; Performing erosion and dilation operations on the first eye diagram to obtain a second eye diagram; Performing spot contour detection on the second eye diagram to obtain a spot contour set; Determining the first center point position coordinates of the corresponding spots according to the first moments of each spot contour; Taking the first center point position coordinates of each spot as the corresponding first spot position coordinates to obtain the first spot position array.

3. A method of interference spot screening as claimed in claim 2, characterized in that, Performing spot position detection on a first original eye diagram to obtain a first spot position array, further comprising: Determining the first pupil center position coordinates in the first original eye diagram; Setting the first horizontal and vertical clipping pixel value ranges based on the iris size in the first original eye diagram; Clipping the first original eye diagram using the first pupil center position coordinates, the first horizontal and vertical clipping pixel value ranges as first preset conditions to obtain a to-be-detected eye diagram; The iris region in the first original eye diagram is included in the to-be-detected eye diagram.

4. A method of interference spot screening as claimed in claim 3, characterized in that, After filling the internal region of the drawn iris contour with the second preset color, the method further comprises: Determining the second pupil center position coordinates in the second original eye diagram; Setting the second horizontal and vertical clipping pixel value ranges based on the iris size in the second original eye diagram; Clipping the preset image filled with the second preset color using the second pupil center position coordinates, the second horizontal and vertical clipping pixel value ranges as second preset conditions to obtain the reference image; The iris region in the second original eye diagram is included in the reference image, and the size of the reference image clipped according to the second preset conditions is the same as the size of the to-be-detected eye diagram clipped according to the first preset conditions.

5. A method of interferometric scatterer screening as claimed in claim 4, wherein, The first original eye diagram and the second original eye diagram are the same original eye diagram, or the first original eye diagram and the second original eye diagram are different original eye diagrams obtained under the same acquisition condition, and the same acquisition condition includes that an eye diagram acquisition device is the same, a target human eye collected is the same, a target human eye gaze point is the same, a target human eye position collected is the same, and arrangement positions of each light source in a light source group are the same.

6. A method of interferometric scatterometry according to any one of claims 1 to 5, wherein, The method further comprises: determining whether the number of second light spot position coordinates in the second light spot position array is greater than or equal to a first preset threshold value; if the number of second light spot position coordinates in the second light spot position array is greater than or equal to the first preset threshold value, establishing a light spot coordinate system based on each second light spot position coordinate; sorting each second light spot position coordinate in the second light spot position array according to a first preset rule to obtain a third light spot position array; sequentially detecting, according to the sorting order, whether a first vector included angle corresponding to two adjacent third light spot position coordinates in the third light spot position array is greater than or equal to a second preset threshold value; if the first vector included angle is greater than or equal to the second preset threshold value, determining that the corresponding two adjacent third light spot position coordinates are a first light spot position pair in which no interference light spot exists; if the first vector included angle is less than the second preset threshold value, determining that the corresponding two adjacent third light spot position coordinates are a second light spot position pair in which an interference light spot exists; establishing an eye diagram horizontal line coordinate system to determine two first light spot position conversion coordinates corresponding to the two third light spot position coordinates of each second light spot position pair in the eye diagram horizontal line coordinate system; comparing the sizes of the two first light spot position conversion coordinates, and identifying a third light spot position coordinate corresponding to an interference light spot according to a second preset rule; deleting all identified third light spot position coordinates corresponding to interference light spots from the third light spot position array to obtain a fourth light spot position array as a to-be-numbered light spot position array.

7. A method of interferometric scatterer screening as claimed in claim 6, wherein, The method further comprises: if the number of second light spot position coordinates in the second light spot position array is less than the first preset threshold value, directly storing the second light spot position array as the fourth light spot position array as the to-be-numbered light spot position array.

8. A method of interferometric scatterer screening as claimed in claim 7, wherein, The origin position of the light spot coordinate system is a centroid position of each second light spot position coordinate.

9. A method of interferometric scatterer screening as claimed in claim 7, wherein, The first preset rule is a clockwise sorting rule or a counterclockwise sorting rule.

10. A method of interferometric scatterer screening as claimed in claim 7, wherein, Establishing an eye diagram horizontal line coordinate system comprises: establishing a three-dimensional world coordinate system based on a target human face; detecting a first left eye center position coordinate and a first right eye center position coordinate of the target human face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate to a camera coordinate system of the first original eye diagram to obtain a corresponding second left eye center position coordinate and a second right eye center position coordinate; calculating a slope of an eye diagram horizontal line by using the second left eye center position coordinate and the second right eye center position coordinate, the eye diagram horizontal line being a projection connecting line of a left eye center and a right eye center of the target human face in the camera coordinate system; Select any point in the camera coordinate system as the origin, and determine the coordinate axis direction according to the slope of the eye diagram horizontal line to establish the eye diagram horizontal line coordinate system.

11. A method of interferometric scatterer screening as claimed in any one of claims 7 to 10, wherein, The method further comprises: calculating a first distance between any two fourth spot position coordinates in the fourth spot position array; determining whether the first distance is less than or equal to a third preset threshold value; if the first distance is greater than the third preset threshold value, determining the corresponding two fourth spot position coordinates as a third spot position pair without an interfering spot; if the first distance is less than or equal to the third preset threshold value, determining the corresponding two fourth spot position coordinates as a fourth spot position pair with an interfering spot; establishing an eye diagram horizontal line coordinate system to determine two second spot position conversion coordinates corresponding to the two fourth spot position coordinates of each fourth spot position pair in the eye diagram horizontal line coordinate system; comparing the sizes of the corresponding two second spot position conversion coordinates, and identifying the fourth spot position coordinates corresponding to the interfering spot according to the second preset rule; deleting all identified fourth spot position coordinates corresponding to the interfering spot from the fourth spot position array to obtain a fifth spot position array as a to-be-numbered spot position array.

12. A method of interferometric patch screening as claimed in claim 11, wherein, The method further comprises: in the sorting order, sequentially calculating a second distance between each fifth spot position coordinate in the fifth spot position array and the rest of the fifth spot position coordinates to obtain a second distance array corresponding to each fifth spot position coordinate; obtaining a corresponding second distance minimum value and a second distance maximum value from each second distance array; calculating a first extreme value ratio of each second distance maximum value divided by the corresponding second distance minimum value; determining whether each first extreme value ratio is greater than a fourth preset threshold value; if the first extreme value ratio is greater than the fourth preset threshold value, determining the corresponding fifth spot position coordinate as a first abnormal spot position coordinate; if the first extreme value ratio is less than or equal to the fourth preset threshold value, determining the corresponding fifth spot position coordinate as a first normal spot position coordinate; generating a first abnormal spot position array by obtaining all first abnormal spot position coordinates; determining whether the number of first abnormal spot position coordinates in the first abnormal spot position array is greater than a fifth preset threshold value; if the number of first abnormal spot position coordinates in the first abnormal spot position array is greater than the fifth preset threshold value, establishing an eye diagram horizontal line coordinate system to determine third spot position conversion coordinates corresponding to each first abnormal spot position coordinate in the eye diagram horizontal line coordinate system; comparing the sizes of the third spot position conversion coordinates, and identifying the fifth spot position coordinates corresponding to the interfering spot according to the second preset rule; deleting all identified fifth spot position coordinates corresponding to the interfering spot from the fifth spot position array, and looping back to the second distance calculation of the currently retained fifth spot position coordinates; If the number of first abnormal light spot position coordinates in the first abnormal light spot position array is less than or equal to a fifth preset threshold, each fifth light spot position coordinate currently reserved is taken as a sixth light spot position coordinate, and a sixth light spot position array is obtained as the to-be-numbered light spot position array.

13. A method of interferometric patch screening as claimed in claim 12, wherein, The method further comprises: in the order of the sequence, each sixth light spot position coordinate in the sixth light spot position array is taken as a vertex in turn; angles between the vertex and any two sixth light spot position coordinates in the sixth light spot position array are calculated to obtain a corresponding set of edge angles; whether there is an edge angle in each set of edge angles that meets a second preset condition is determined; if there is an edge angle in the current set of edge angles that meets the second preset condition, the sixth light spot position coordinate of the corresponding vertex is determined as a second abnormal light spot position coordinate; if there is no edge angle in the current set of edge angles that meets the second preset condition, the sixth light spot position coordinate of the corresponding vertex is determined as a second normal light spot position coordinate; all second abnormal light spot position coordinates are obtained to generate a second abnormal light spot position array; an eye diagram horizontal line coordinate system is established, and fourth light spot position conversion coordinates of each second abnormal light spot position coordinate in the second abnormal light spot position array in the eye diagram horizontal line coordinate system are determined; sizes of each fourth light spot position conversion coordinate are compared, and a sixth light spot position coordinate corresponding to an interference light spot is identified according to the second preset rule; the identified sixth light spot position coordinate corresponding to the interference light spot is deleted from the sixth light spot position array to obtain a seventh light spot position array as the to-be-numbered light spot position array.

14. A method of interferometric scatterer screening as claimed in claim 13, wherein, The second preset condition comprises that the edge angle is less than a fifth preset threshold or the edge angle is greater than a sixth preset threshold.

15. A method of interferometric patch screening as claimed in claim 12, wherein, The second preset rule comprises: a third light spot position coordinate corresponding to the maximum / minimum of the two first light spot position conversion coordinates is identified as a third light spot position coordinate corresponding to the interference light spot; a fourth light spot position coordinate corresponding to the maximum / minimum of the two second light spot position conversion coordinates is identified as a fourth light spot position coordinate corresponding to the interference light spot; a fifth light spot position coordinate corresponding to the maximum / minimum of each third light spot position conversion coordinate is identified as a fifth light spot position coordinate corresponding to the interference light spot; a sixth light spot position coordinate corresponding to the maximum / minimum of each fourth light spot position conversion coordinate is identified as a sixth light spot position coordinate corresponding to the interference light spot.

16. An interferometric patch screening system characterized by, A system for performing the steps of a method for screening an interference light spot as claimed in any one of claims 1 to 15, the system comprising: a first detection unit for performing light spot position detection on a first original eye diagram to obtain a first light spot position array; a second detection unit for performing iris contour point position detection on a second original eye diagram to obtain a preset number of iris contour point coordinates; an exclusion unit for performing the following steps: using each iris contour point coordinate, an iris contour is drawn on a preset image of the same size as the first original eye diagram; the preset image has a first preset color; an internal region of the drawn iris contour is filled with a second preset color to obtain a reference image; and a system for performing the steps of a method for screening an interference light spot as claimed in any one of claims 1 to 15, the system comprising: a first detection unit for performing light spot position detection on a first original eye diagram to obtain a first light spot position array; a second detection unit for performing iris contour point position detection on a second original eye diagram to obtain a preset number of iris contour point coordinates; an exclusion unit for performing the following steps: using each iris contour point coordinate, an iris contour is drawn on a preset image of the same size as the first original eye diagram; the preset image has a first preset color; an internal region of the drawn iris contour is filled with a second preset color to obtain a reference image; and extracting pixel colors from the reference image according to each first light spot position coordinate in the first light spot position array; identifying the extracted pixel colors corresponding to each first light spot position coordinate; if the extracted pixel color is a first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinate is deleted; if the extracted pixel color is a second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinate is reserved as a second light spot position coordinate; obtaining a second light spot position array as a to-be-numbered light spot position array by using all the second light spot position coordinates.

17. An interferometric patch screening apparatus, characterized in that, The device comprises a processor and a memory; The memory is used to store one or more program instructions; The processor is used to run one or more program instructions to execute the steps of the interference light spot screening method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer program stored on the computer readable storage medium is executed by the processor to implement the steps of the interference light spot screening method according to any one of claims 1 to 15.

Images