Interference spot screening method, system, device and storage medium

Abstract

The present application provides an interference spot screening method, system, device and storage medium. Based on the iris contour point position detection, the iris contour point coordinates are obtained. Then, the iris contour point coordinates are used to draw the iris contour and fill the color on the preset image to obtain a reference image. The pixel color of the reference image is extracted using the first light spot position coordinates obtained by the light spot position detection. By comparing and identifying the extracted pixel colors, the detection value corresponding to the iris contour is deleted from the first light spot position array, which effectively improves the accuracy of the light spot position detection. The embodiment of the present application can identify and delete the interference light spot formed by the opposite light source in the light spot position detection, and obtain the correct light spot position detection result. In this way, the correct light spot position detection result is spot numbered, and the accuracy of the light spot numbering is also improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of line of sight tracking technology, and specifically to an interference light spot screening method, system, device and storage medium. Background Art

[0002] Gaze tracking technology is a technology that measures the gaze point of human eyes and the degree of movement relative to the head. The gaze tracking method using corneal reflection method needs to obtain the correct light spot position information and its correspondence with the light source. In order to obtain the above information and correspondence, the human eye diagram is usually collected to detect the light spot position and the detected light spots are numbered. When collecting the human eye diagram, it is often affected by the opposite light source to form an interference light spot.

[0003] For example, a predetermined number of light sources are arranged near the left and right optical lenses of the head-mounted display device, and an eye diagram of a single target human eye (left eye or right eye) is collected, that is, the target human eye diagram is obtained, and the contralateral light source refers to the light source arranged near the optical lens of the other eye. In the existing light spot position detection and light spot numbering implementation, the results are mostly obtained directly from the collected target human eye eye diagram, without considering the problem of interference light spots formed by the contralateral light source. Summary of the invention

[0004] To this end, the present application provides an interference light spot screening method, system, device and storage medium to solve the technical problem that the interference light spot formed by the opposite side light source is not considered in the existing light spot position detection and light spot numbering implementation.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] According to the first aspect of the present application, an embodiment of the present application provides an interference spot screening method, the method comprising:

[0007] Performing light spot position detection on the first original eye image to obtain a first light spot position array;

[0008] Performing iris contour point position detection on the second original eye image to obtain a preset number of iris contour point coordinates;

[0009] Using the coordinates of each iris contour point, drawing an iris contour on a preset image of the same size as the first original eye image; the preset image has a first preset color;

[0010] Filling the inner area of ​​the drawn iris outline with a second preset color to obtain a reference image;

[0011] Extracting pixel colors from the reference image according to the coordinates of each first light spot position in the first light spot position array;

[0012] Identify the extracted pixel colors corresponding to the coordinates of each first light spot position;

[0013] If the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinates are deleted;

[0014] If the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinates are retained as the second light spot position coordinates;

[0015] By using all the second light spot position coordinates, a second light spot position array is obtained as the light spot position array to be numbered.

[0016] According to a second aspect of the present application, an interference spot screening system is provided, which is used to perform the steps in the interference spot screening method described above, and the system comprises:

[0017] A first detection unit, used for performing light spot position detection on the first original eye diagram to obtain a first light spot position array;

[0018] A second detection unit is used to perform iris contour point position detection on the second original eye image to obtain a preset number of iris contour point coordinates;

[0019] Excludes cells that perform the following steps:

[0020] Using the coordinates of each iris contour point, drawing an iris contour on a preset image of the same size as the first original eye image; the preset image has a first preset color;

[0021] Filling the inner area of ​​the drawn iris outline with a second preset color to obtain a reference image;

[0022] Extracting pixel colors from the reference image according to the coordinates of each first light spot position in the first light spot position array;

[0023] Identify the extracted pixel colors corresponding to the coordinates of each first light spot position;

[0024] If the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinates are deleted;

[0025] If the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinates are retained as the second light spot position coordinates;

[0026] By using all the second light spot position coordinates, a second light spot position array is obtained as the light spot position array to be numbered.

[0027] According to a third aspect of the present application, an interference spot screening device is provided, the device comprising: a processor and a memory;

[0028] The memory is used to store one or more program instructions;

[0029] The processor is used to run one or more program instructions to execute the steps of the interference spot screening method as described above.

[0030] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the interference spot screening method described above are implemented.

[0031] Compared with the prior art, the interference spot screening method, system, device and storage medium provided by the embodiment of the present application obtain the iris contour point coordinates based on the iris contour point position detection, and then use the iris contour point coordinates to draw the iris contour and fill the color on the preset image to obtain a reference image; use the first light spot position coordinates obtained by the light spot position detection to extract the pixel color of the reference image, and compare and identify the extracted pixel colors, and delete the detection value corresponding to the iris contour from the first light spot position array, which effectively improves the accuracy of the light spot position detection. The embodiment of the present application can identify and delete the interference light spot formed by the opposite light source in the light spot position detection, and obtain the correct light spot position detection result. In this way, the correct light spot position detection result is spot numbered, and the accuracy of the light spot numbering is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0033] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0034] Figure 1 A schematic diagram of the logical structure of an interference spot screening system provided by one embodiment of the present application;

[0035] Figure 2 A schematic diagram of a flow chart of an interference spot screening method provided in one embodiment of the present application;

[0036] Figure 3 A schematic flow chart of an interference spot screening method provided in another embodiment of the present application;

[0037] Figure 4 A schematic diagram of a process for establishing an eye diagram horizontal line coordinate system provided in an embodiment of the present application;

[0038] Figure 5 A schematic flow chart of an interference spot screening method provided in another embodiment of the present application;

[0039] Figure 6 A schematic flow chart of an interference spot screening method provided in another embodiment of the present application;

[0040] Figure 7 A schematic flow chart of an interference spot screening method provided in accordance with another embodiment of the present application. DETAILED DESCRIPTION

[0041] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of this application is to effectively eliminate the interference light spots formed by the opposite side light source during the process of light spot position detection and light spot numbering, obtain the correct light spot position detection results and number them, and ensure accuracy.

[0043] In order to solve the above technical problems, Figure 1 As shown, an embodiment of the present application provides an interference light spot screening system, which is used to implement interference light spot screening.

[0044] Specifically, an interference light spot screening system provided in an embodiment of the present application includes: a first detection unit 1, a second detection unit 2, and an elimination unit 3.

[0045] Furthermore, the first detection unit 1 is used to perform light spot position detection on the first original eye image to obtain a first light spot position array; the second detection unit 2 is used to perform iris contour point position detection on the second original eye image to obtain a preset number of iris contour point coordinates.

[0046] Further, the exclusion unit 3 is used to perform the following steps: using the coordinates of each iris contour point to draw an iris contour on a preset image of the same size as the first original eye diagram; the preset image has a first preset color; filling the inner area of ​​the drawn iris contour with a second preset color to obtain a reference image; extracting pixel colors from the reference image according to each first light spot position coordinate in the first light spot position array; identifying each first light spot position coordinate corresponding to the extracted pixel color; if the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinate is deleted; if the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinate is retained as the second light spot position coordinate; using all the second light spot position coordinates, a second light spot position array is obtained as the light spot position array to be numbered.

[0047] Compared with the prior art, the interference spot screening system provided by the embodiment of the present application obtains the coordinates of the iris contour points based on the iris contour point position detection, and then obtains the reference image by drawing the iris contour and filling the color on the preset image using the iris contour point coordinates; extracts the pixel color of the reference image using the coordinates of each first spot position obtained by the spot position detection, and deletes the detection value corresponding to the iris contour from the first spot position array by comparing and identifying the extracted pixel colors, thereby effectively improving the accuracy of the spot position detection. The embodiment of the present application can identify and delete the interference spot formed by the opposite light source in the spot position detection, and obtain the correct spot position detection result. In this way, the correct spot position detection result is spot numbered, and the accuracy of the spot numbering is also improved.

[0048] Corresponding to the above-disclosed interference light spot screening system, the present application embodiment further discloses an interference light spot screening method. The following describes in detail the interference light spot screening method disclosed in the present application embodiment in combination with the above-described interference light spot screening system.

[0049] like Figure 2 As shown, the specific steps of an interference spot screening method provided in an embodiment of the present application are described in detail below.

[0050] An interference light spot screening method provided in an embodiment of the present application is used to implement interference light spot screening.

[0051] The first detection unit 1 performs light spot position detection on the first original eye diagram to obtain a first light spot position array.

[0052] In an embodiment of the present application, the first original eye image is a grayscale image of the eye captured by a camera. For example, the target person can wear a head-mounted display device, and a predetermined number of light sources are arranged near the left eye and right eye optical lenses of the head-mounted display device. If the predetermined number can be 6 or 8, then 6 or 8 light sources are distributed in a circle around the left eye optical lens, and a camera is located directly below or diagonally below the left eye optical lens, and the camera is close to the cheek. Similarly, 6 or 8 light sources are distributed in a circle around the right eye optical lens, and a camera is located directly below or diagonally below the right eye optical lens, and the camera is close to the cheek. By collecting an eye image of a single target human eye (left eye or right eye), the above-mentioned first original eye image can be obtained.

[0053] The above steps specifically include: performing adaptive threshold segmentation on the eye image to be detected to obtain a first eye image; performing corrosion and expansion operations on the first eye image to obtain a second eye image; performing spot contour detection on the second eye image to obtain a spot contour set; determining the first center point position coordinates of the corresponding light spot according to the first-order moment of each light spot contour; using the first center point position coordinates of each light spot as the corresponding first light spot position coordinates to obtain a first light spot position array.

[0054] As described above, in the embodiment of the present application, the above-mentioned first original eye image can be directly used as the eye image to be detected, and the adaptive threshold segmentation is realized by an adaptive threshold function (adaptiveThreshold() function), and the bright spot in the eye image to be detected is identified by the mean or Gaussian mean method to obtain the first eye image img_threshold. The adaptiveThreshold() function in OpenCV is a special threshold processing method for converting a grayscale image or a color image into a binary image. It can adaptively determine the threshold of each pixel according to the local grayscale characteristics of each pixel neighborhood, so for images with uneven illumination, it can perform binarization better than the traditional global threshold method.

[0055] Furthermore, the first eye image img_threshold is subjected to erode and dilate operations, mainly to eliminate the influence of small dots and noise. The image after the operation is the second eye image img_dilation. Erode and dilate operations are the two most basic morphological operations. Morphological operations are a series of image processing operations based on shapes. OpenCV provides convenient functions for morphological transformation of images, one of its main functions is to eliminate noise. Erode and dilate operations are both for the white part (highlight part) in the image. Erode operation is to "expand the field" of the highlight part of the image, and the effect image has a larger highlight area than the original image. Dilate operation is to erode the highlight area in the original image, and the effect image has a smaller highlight area than the original image.

[0056] Next, in the embodiment of the present application, the findContours() function is used to complete the spot contour detection of the second eye image img_dilation, and the contours are found, and each contour corresponds to a spot. The findContours() function in C++ OpenCV is used to detect contours in a binary image. It can extract continuous areas in an image to form a set of spot contours. This function is very suitable for computer vision tasks such as target detection and image segmentation.

[0057] Then, the moments() function is used to determine the center point of the corresponding light spot according to the first-order moment of each light spot contour, that is, the first center point position coordinates (pixel coordinates) of each light spot. The moments() function in OpenCV is one of the contour feature functions, which is used to calculate the moment of the image. In this way, a set of first light spot position arrays storing several first center point position coordinates is obtained.

[0058] The second detection unit 2 performs iris contour point position detection on the second original eye image to obtain a preset number of iris contour point coordinates.

[0059] In an embodiment of the present application, the first original eye diagram and the second original eye diagram are the same original eye diagram, or the first original eye diagram and the second original eye diagram are different original eye diagrams obtained under the same acquisition conditions, and the same acquisition conditions include: the eye diagram acquisition equipment is the same, the target human eyes for acquisition are the same, the target human eye gaze points are the same, the acquisition target human eye positions are the same, and the arrangement positions of the light sources in the light source group are the same.

[0060] Furthermore, the preset number may be 8, that is, by detecting the positions of iris contour points on the second original eye image, 8 iris contour point coordinates are obtained. The iris contour point coordinates are the pixel coordinates of the iris contour points on the second original eye image.

[0061] The exclusion unit 3 draws the iris contour on the preset image using the coordinates of each iris contour point.

[0062] For the preset image, on one hand, the preset image has the same size as the first original eye pattern, and on the other hand, the preset image has a first preset color.

[0063] Furthermore, in an embodiment of the present application, the iris contour is drawn on a preset image by ellipse fitting using the above-mentioned preset number of iris contour point coordinates. Ellipse fitting of a contour refers to using an ellipse to approximate the shape of the contour. When the major axis and minor axis of the ellipse are equal, it is a circle. The basic idea of ​​ellipse fitting is: for a set of sample points on a given plane, find an ellipse that is as close to these sample points as possible. In other words, a set of data in the image is fitted with an ellipse equation as a model, so that a certain ellipse equation satisfies these data as much as possible, and the various parameters of the ellipse equation are calculated. Three functions are provided in OpenCV to implement ellipse fitting, and the above three functions are fitEllipse() function, fitEllipseAMS() function, and fitEllipseDirect() function.

[0064] Next, the exclusion unit 3 performs the following steps: fills the inner area of ​​the drawn iris contour with a second preset color to obtain a reference image; extracts pixel colors from the reference image according to the coordinates of each first light spot position in the first light spot position array; and identifies the extracted pixel colors corresponding to the coordinates of each first light spot position.

[0065] In this way, if the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinates are deleted. If the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinates are retained as the second light spot position coordinates. The exclusion unit 3 uses all the second light spot position coordinates to obtain the second light spot position array as the light spot position array to be numbered.

[0066] In the embodiment of the present application, for example, the first preset color is black, the second preset color is specifically white, and the preset image is a completely black image of the same size as the first original eye diagram. In this way, on a completely black image of the same size as the first original eye diagram, the inner area of ​​the drawn iris contour is filled with white, and the reference image after the inner area of ​​the iris contour is filled with white is recorded as img_cornea0. After identifying the extracted pixel colors corresponding to the coordinates of each first light spot position according to the above steps, if the extracted pixel color is white, it is considered that the light spot is within the iris contour, and if the extracted pixel color is black, it is considered that the light spot is not within the iris contour, and the light spot is outside the iris contour. The above steps ensure that the second light spot position coordinates in the second light spot position array are all within the iris contour of the ellipse fitting.

[0067] Compared with the prior art, the interference spot screening method provided by the embodiment of the present application obtains the coordinates of the iris contour points based on the iris contour point position detection, and then obtains the reference image by drawing the iris contour and filling the color on the preset image using the iris contour point coordinates; extracts the pixel color of the reference image using the coordinates of each first spot position obtained by the spot position detection, and deletes the detection value corresponding to the iris contour from the first spot position array by comparing and identifying the extracted pixel colors, thereby effectively improving the accuracy of the spot position detection. The embodiment of the present application can identify and delete the interference spot formed by the opposite light source in the spot position detection, and obtain the correct spot position detection result. In this way, the correct spot position detection result is spot numbered, and the accuracy of the spot numbering is also improved.

[0068] Preferably, the step of performing spot position detection on the first original eye image by the first detection unit 1 further includes: determining the coordinates of the first pupil center position in the first original eye image; setting the first horizontal and vertical cropping pixel value ranges based on the iris size in the first original eye image; and cropping the first original eye image with the first pupil center position coordinates and the first horizontal and vertical cropping pixel value ranges as the first preset conditions to obtain the eye image to be detected. The iris area in the first original eye image is included in the eye image to be detected.

[0069] Correspondingly, in the specific steps of the above-mentioned interference spot screening method disclosed in the embodiment of the present application, after filling the inner area of ​​the drawn iris outline with the second preset color, it also includes: determining the coordinates of the second pupil center position in the second original eye diagram; setting the second horizontal and vertical cropping pixel value range based on the iris size in the second original eye diagram; using the second pupil center position coordinates and the second horizontal and vertical cropping pixel value range as the second preset conditions, cropping the preset image filled with the second preset color to obtain a reference image. The iris area in the second original eye diagram is included in the reference image, and the size of the reference image cropped according to the second preset condition is the same as the size of the eye diagram to be detected cropped according to the first preset condition.

[0070] As described above, the first original eye image and the preset image filled with the second preset color are cropped in the same manner, and the reference image is cropped to an image with the same size as the eye image to be detected after cropping. At this time, the center of the pupil is still the center point of the cropped eye image, and the cropped reference image is recorded as img_cornea. In this way, before extracting pixel color recognition, the image area outside the iris contour is cropped, which has no effect on the accuracy of the result, and at the same time reduces the amount of data processed in the pixel color extraction and recognition steps, greatly improving the processing efficiency.

[0071] In an embodiment of the present application, if the first original eye diagram is cropped according to the above-mentioned first preset condition to obtain the eye diagram to be detected, after obtaining the array of light spot positions to be numbered, it is necessary to restore each coordinate value in the array of light spot positions to be numbered to the corresponding coordinate value in the first original eye diagram according to the first preset condition.

[0072] refer to Figure 3 , an interference spot screening method disclosed in the embodiment of the present application further includes the following steps: the exclusion unit 3 determines whether the number of second light spot position coordinates in the second light spot position array is greater than or equal to a first preset threshold. Specifically, the first preset threshold in the above step can be 3.

[0073] If the number of second light spot position coordinates in the second light spot position array is greater than or equal to the first preset threshold, the exclusion unit 3 first establishes a light spot coordinate system based on each second light spot position coordinate.

[0074] Further, in the embodiment of the present application, in the process of establishing the above-mentioned spot coordinate system, the centroid position of each second spot position coordinate is used as the origin position. Specifically, the centroid position coordinate is calculated based on each second spot position coordinate, that is, the centroid position horizontal coordinate is the average value of the horizontal coordinates of each second spot position, and the centroid position vertical coordinate is the average value of the vertical coordinates of each second spot position. With the centroid position coordinate as the origin of the above-mentioned spot coordinate system, the positive direction of the X-axis of the spot coordinate system and the positive direction of the Y-axis of the spot coordinate system are determined. For example, the right is the positive direction of the X-axis of the above-mentioned spot coordinate system, and the downward is the positive direction of the Y-axis of the above-mentioned spot coordinate system, and the above-mentioned spot coordinate system is obtained.

[0075] Then, the exclusion unit 3 sorts the coordinates of the second light spot positions in the second light spot position array according to the first preset rule to obtain a third light spot position array.

[0076] In the embodiment of the present application, preferably, the first preset rule is a clockwise sorting rule. Specifically, the above steps specifically include: using the coordinates of each second spot position and the origin coordinates of the spot coordinate system to obtain the corresponding spot vector to be screened, and obtaining the positive X-axis angle θ between each spot vector to be screened and the positive X-axis direction of the spot coordinate system. x According to the inverse tangent function arctan(y / x), the positive angles θ of each X axis are x Converted into the positive Y-axis angle θ between the corresponding spot vector to be screened and the positive Y-axis direction of the spot coordinate system y If there is a second light spot position coordinate, the second light spot position coordinate is the i-th in the second light spot position array (i≤ n1, n1 is the total number of second light spot position coordinates in the second light spot position array), when the Y-axis positive angle θ corresponding to the second light spot position coordinate y When (i) is 0°, the second light spot position coordinate is in the positive direction of the Y axis of the light spot coordinate system; when the positive Y axis angle θ corresponding to the second light spot position coordinate is y (i) is 90°, the second spot position coordinate is in the negative direction of the X axis of the spot coordinate system; when the second spot position coordinate corresponds to the positive Y axis angle θ y (i) is 180°, the second light spot position coordinate is in the negative direction of the Y axis of the light spot coordinate system; when the second light spot position coordinate corresponds to the positive Y axis angle θ y (i) When the angle θ is 270°, the second spot position coordinate is in the positive direction of the X axis of the spot coordinate system. y Sort the corresponding second light spot position coordinates in ascending order to obtain a new third light spot position array and a new Y-axis positive angle θ ynArray. The third spot position array has the characteristic of clockwise sorting.

[0077] In addition, the above first preset rule can also be a counterclockwise sorting rule. In this way, based on the above clockwise sorting rule, according to each positive Y-axis included angle θ y Sort the corresponding second spot position coordinates in descending order to obtain a new third spot position array and a new positive Y-axis included angle θ yn Array. At this time, the third spot position array has the characteristic of counterclockwise sorting.

[0078] Then, the exclusion unit 3 sequentially detects whether the included angle of the first vectors corresponding to two adjacent third spot position coordinates in the third spot position array is greater than or equal to a second preset threshold according to the sorting order.

[0079] If the included angle of the first vectors is greater than or equal to the second preset threshold, determine the corresponding two adjacent third spot position coordinates as the first spot position pair without interference spots; if the included angle of the first vectors is less than the second preset threshold, determine the corresponding two adjacent third spot position coordinates as the second spot position pair with interference spots.

[0080] In the embodiment of the present application, the above second preset threshold is denoted as θmax. The following uses θmax set to 5° to illustrate the above steps. If there is a third spot position coordinate, and this third spot position coordinate is the i-th in the third spot position array (i < n2, where n2 is the total number of second spot position coordinates in the second spot position array), calculate the included angle of the first vectors according to the following formula:

[0081]

[0082] where θ yd (i) is the included angle of the first vectors corresponding to the i-th and the (i + 1)-th third spot position coordinates in the third spot position array, and θ yn (i) is the positive Y-axis included angle corresponding to the i-th third spot position coordinate in the third spot position array, and θ yn (i + 1) is the positive Y-axis included angle corresponding to the (i + 1)-th third spot position coordinate in the third spot position array.

[0083] If there is θ yd (i) greater than or equal to 5°, determine the corresponding two adjacent third spot position coordinates as the first spot position pair without interference spots; if there is θ yd (i) less than 5°, it is considered that the opposite light source is mixed in to form an interference spot position coordinate, and the corresponding two adjacent third spot position coordinates are determined as the second spot position pair with interference spots.

[0084] Then, the exclusion unit 3 establishes an eye diagram horizontal line coordinate system, and determines two first light spot position conversion coordinates corresponding to two third light spot position coordinates of each second light spot position pair in the eye diagram horizontal line coordinate system.

[0085] In the examples of this application, refer to Figure 4 The above steps specifically include: establishing a three-dimensional world coordinate system based on the target face; detecting the first left eye center position coordinate and the first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate to the camera coordinate system of the first original eye diagram to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate; using the second left eye center position coordinate and the second right eye center position coordinate, calculating the slope of the eye diagram horizontal line, the eye diagram horizontal line is the projection line connecting the left eye center and the right eye center of the target face in the camera coordinate system; selecting any point from the camera coordinate system as the origin, determining the coordinate axis direction by the slope of the eye diagram horizontal line, and establishing the eye diagram horizontal line coordinate system.

[0086] As mentioned above, the establishment of the eye diagram horizontal line coordinate system requires calculating the slope of the line formed by the horizontal line connecting the two eyes in the eye diagram in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the bridge of the target face's nose as the origin (0,0,0), the X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the Z-axis represents the vertical direction of the plane formed by the X-axis and the Y-axis. The first left eye center position coordinate and the first right eye center position coordinate can be expressed as E left and E right For example, E left =(-31.5,0,0),E right =(31.5,0,0). The eye diagram horizontal line coordinate system is established on the assumption that the user wears the head display device normally. Usually, the line between the left eye center 3D position coordinate and the right eye center 3D position coordinate is parallel to the line between the optical center of the left eye physical lens and the optical center of the right eye physical lens of the head display device, and the former and the latter only have a translation relationship (ignore the length of the line segment). The first left eye center position coordinate and the first right eye center position coordinate are respectively converted to the camera coordinate system, and then converted to image pixel coordinates to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate, which are recorded as P respectively. left and P right The second left eye center position coordinate P left and the second right eye center position coordinate P right The line between them is denoted as P l P r , that is, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line betweenl E r Line segment projected onto the first original eye diagram.

[0087] Under the above usage scenario layout of the embodiment of the present application, the second left eye center position coordinate P left =X_P left , Y_P left ), the second right eye center position coordinate P right =X_P right , Y_P right ), calculate the second left eye center position coordinate P according to the following formula left and the second right eye center position coordinate P right The line P between l P r The slope k is:

[0088]

[0089] Thus, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r , which can be represented as a line segment with a slope of k in the camera coordinate system of the first original eye diagram. Based on the camera coordinate system of the first original eye diagram, a new eye diagram horizontal line coordinate system is established. The eye diagram horizontal line coordinate system can select any point in the camera coordinate system of the first original eye diagram as the origin. For example, the eye diagram horizontal line coordinate system selects the following points as the origin: the center point of the first original eye diagram, the center position of the pupil of the first original eye diagram, the origin (0,0) of the pixel coordinate system of the first original eye diagram, the center position coordinate P of the second left eye left and the second right eye center position coordinate P right The line P between l P r The eye diagram horizontal line takes the line segment y=kx+b with a slope of k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0090] Then, the exclusion unit 3 compares the sizes of the corresponding two first light spot position conversion coordinates, and identifies the third light spot position coordinates corresponding to the interference light spot according to the second preset rule.

[0091] In the embodiment of the present application, the second preset rule includes: identifying the third light spot position coordinates corresponding to the maximum / minimum value of the two first light spot position conversion coordinates as the third light spot position coordinates corresponding to the interference light spot.

[0092] Specifically, in the embodiment of the present application, according to different ways of establishing the above-mentioned coordinate systems, the above-mentioned second preset rule has multiple judgment rules, for example, the third light spot position coordinate corresponding to the maximum / minimum value of the horizontal coordinate x in the two first light spot position conversion coordinates is identified as the third light spot position coordinate corresponding to the interference light spot. Or the third light spot position coordinate corresponding to the maximum / minimum value of the vertical coordinate y in the two first light spot position conversion coordinates is identified as the third light spot position coordinate corresponding to the interference light spot.

[0093] The following is only an example of the above-mentioned specific usage scenario. As mentioned above, taking the first original eye image acquired from the left eye as an example, according to the establishment method of each coordinate system mentioned above, the positive direction of the X-axis of the eye image horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P left The direction of the vector. The two third light spot position coordinates of each second light spot position pair with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spot most likely formed by the opposite side light source can be judged according to the x value of the corresponding two first light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the left eye as an example, the specific rule is that the larger the x value of the corresponding first light spot position conversion coordinate is, the closer it is to the other eye (right eye), and the third light spot position coordinate corresponding to the maximum value of the x value of the corresponding two first light spot position conversion coordinates is most likely to be the interference light spot formed by the opposite side light source, and it is identified as the third light spot position coordinate corresponding to the interference light spot.

[0094] In addition, the examples in the above specific usage scenarios. As mentioned above, taking the first original eye diagram collected from the right eye as an example, according to the establishment methods of the above coordinate systems, the positive direction of the X-axis of the eye diagram horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P left The direction of the vector. The two third light spot position coordinates of each second light spot position pair with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spot most likely formed by the opposite light source can be judged according to the x value of the corresponding two first light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the right eye as an example, the specific rule is that the smaller the x value of the corresponding first light spot position conversion coordinate is, the closer it is to the other eye (left eye), and the third light spot position coordinate corresponding to the minimum x value of the corresponding two first light spot position conversion coordinates is most likely to be the interference light spot formed by the opposite light source, and it is identified as the third light spot position coordinate corresponding to the interference light spot.

[0095] Then, the exclusion unit 3 deletes the third light spot position coordinates corresponding to all the identified interfering light spots from the third light spot position array, and obtains a fourth light spot position array as the light spot position array to be numbered.

[0096] In addition, if the number of second light spot position coordinates in the second light spot position array is less than the first preset threshold, the second light spot position array is directly stored as the fourth light spot position array as the light spot position array to be numbered.

[0097] After screening the interference light spots by using the second preset rule, there may still be interference light spots formed by the opposite side light source. Usually, the interference light spots formed by the opposite side light source not only have the characteristics described in the second preset rule, but also have the characteristics of small distances between the light spots and the interference light spots, and small distances between the interference light spots. In the embodiment of the present application, it is also possible to further determine whether there are still interference light spots based on the distances between each light spot and other light spots in the fourth light spot position array.

[0098] refer to Figure 5 , an interference spot screening method disclosed in the embodiment of the present application also includes the following steps: the exclusion unit 3 calculates the first distance between any two fourth spot position coordinates in the fourth spot position array. Then, the exclusion unit 3 determines whether the first distance is less than or equal to the third preset threshold. If the first distance is greater than the third preset threshold, the corresponding two fourth spot position coordinates are determined as the third spot position pair without interference spots; if the first distance is less than or equal to the third preset threshold, the corresponding two fourth spot position coordinates are determined as the fourth spot position pair with interference spots.

[0099] Generally, taking a certain actual use scenario as an example, the minimum value of the distance between the coordinates of the correct eye pattern light spots formed in the scenario is greater than 40 pixels. If the opposite light source forms a number of interference light spots on the eye pattern, the maximum value of the distance between these interference light spots is usually about 18 pixels. Therefore, in the embodiment of the present application, the third preset threshold can be set to 18 pixels. In this way, by judging whether there are two fourth light spot position coordinates in the fourth light spot position array whose first distance is less than or equal to 18 pixels, it is judged whether there is an interference light spot.

[0100] If the fourth light spot position pair with an interfering light spot is identified by the third preset threshold, similarly, refer to Figure 5 The exclusion unit 3 establishes an eye diagram horizontal line coordinate system, and determines two second light spot position conversion coordinates corresponding to two fourth light spot position coordinates of each fourth light spot position pair in the eye diagram horizontal line coordinate system.

[0101] In the embodiment of the present invention, reference Figure 4The above steps specifically include: establishing a three-dimensional world coordinate system based on the target face; detecting the first left eye center position coordinate and the first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate to the camera coordinate system of the first original eye diagram to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate; using the second left eye center position coordinate and the second right eye center position coordinate, calculating the slope of the eye diagram horizontal line, the eye diagram horizontal line is the projection line connecting the left eye center and the right eye center of the target face in the camera coordinate system; selecting any point from the camera coordinate system as the origin, determining the coordinate axis direction by the slope of the eye diagram horizontal line, and establishing the eye diagram horizontal line coordinate system.

[0102] As mentioned above, the establishment of the eye diagram horizontal line coordinate system requires calculating the slope of the line formed by the horizontal line connecting the two eyes in the eye diagram in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the bridge of the target face's nose as the origin (0,0,0), the X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the Z-axis represents the vertical direction of the plane formed by the X-axis and the Y-axis. The first left eye center position coordinate and the first right eye center position coordinate can be expressed as E left and E right For example, E left =(-31.5,0,0),E right =(31.5,0,0). The eye diagram horizontal line coordinate system is established on the assumption that the user wears the head display device normally. Usually, the line between the left eye center 3D position coordinate and the right eye center 3D position coordinate is parallel to the line between the optical center of the left eye physical lens and the optical center of the right eye physical lens of the head display device, and the former and the latter only have a translation relationship (ignore the length of the line segment). The first left eye center position coordinate and the first right eye center position coordinate are respectively converted to the camera coordinate system, and then converted to image pixel coordinates to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate, which are recorded as P respectively. left and P right The second left eye center position coordinate P left and the second right eye center position coordinate P right The line between them is denoted as P l P r , that is, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r Line segment projected onto the first original eye diagram.

[0103] Under the above usage scenario layout of the embodiment of the present invention, the second left eye center position coordinate P left =X_P left, Y_P left ), the second left eye center position coordinate P right =X_P right , Y_P right ), calculate the second left eye center position coordinate P according to the following formula left and the second right eye center position coordinate P right The line P between l P r The slope k is:

[0104]

[0105] Thus, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r , which can be represented as a line segment with a slope of k in the camera coordinate system of the first original eye diagram. Based on the camera coordinate system of the first original eye diagram, a new eye diagram horizontal line coordinate system is established. The eye diagram horizontal line coordinate system can select any point in the camera coordinate system of the first original eye diagram as the origin. For example, the eye diagram horizontal line coordinate system selects the following points as the origin: the center point of the first original eye diagram, the center position of the pupil of the first original eye diagram, the origin (0,0) of the pixel coordinate system of the first original eye diagram, the center position coordinate P of the second left eye left and the second right eye center position coordinate P right The line P between l P r The eye diagram horizontal line takes the line segment y=kx+b with a slope of k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0106] The elimination unit 3 compares the sizes of the corresponding two second light spot position conversion coordinates, and identifies the fourth light spot position coordinates corresponding to the interfering light spot according to the second preset rule.

[0107] In the embodiment of the present application, the second preset rule includes: identifying the fourth light spot position coordinates corresponding to the maximum / minimum value of the two second light spot position conversion coordinates as the fourth light spot position coordinates corresponding to the interference light spot.

[0108] Specifically, in the embodiment of the present application, according to different ways of establishing the above-mentioned coordinate systems, the above-mentioned second preset rule has multiple judgment rules, for example, the fourth light spot position coordinate corresponding to the maximum / minimum value of the horizontal coordinate x in the two second light spot position conversion coordinates is identified as the fourth light spot position coordinate corresponding to the interference light spot. Or the fourth light spot position coordinate corresponding to the maximum / minimum value of the vertical coordinate y in the two second light spot position conversion coordinates is identified as the fourth light spot position coordinate corresponding to the interference light spot.

[0109] The following is only an example of the above-mentioned specific usage scenario. As mentioned above, taking the first original eye image acquired from the left eye as an example, according to the establishment method of each coordinate system mentioned above, the positive direction of the X-axis of the eye image horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P left The direction of the vector. The two fourth light spot position coordinates of each fourth light spot position pair with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spot most likely formed by the opposite side light source can be judged according to the x value of the corresponding two second light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the left eye as an example, the specific rule is that the larger the x value of the corresponding second light spot position conversion coordinate is, the closer it is to the other eye (right eye), and the fourth light spot position coordinate corresponding to the maximum value of the x value of the corresponding two second light spot position conversion coordinates is most likely to be the interference light spot formed by the opposite side light source, and it is identified as the fourth light spot position coordinate corresponding to the interference light spot.

[0110] In addition, the examples in the above specific usage scenarios. As mentioned above, taking the first original eye diagram collected from the right eye as an example, according to the establishment methods of the above coordinate systems, the positive direction of the X-axis of the eye diagram horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P left The direction of the vector. The two fourth light spot position coordinates of each fourth light spot position pair with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spot most likely formed by the opposite side light source can be judged according to the x value of the corresponding two second light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the right eye as an example, the specific rule is that the smaller the x value of the corresponding first light spot position conversion coordinate is, the closer it is to the other eye (left eye), and the fourth light spot position coordinate corresponding to the minimum x value of the corresponding two second light spot position conversion coordinates is most likely to be the interference light spot formed by the opposite side light source, and it is identified as the fourth light spot position coordinate corresponding to the interference light spot.

[0111] Next, the exclusion unit 3 deletes the fourth light spot position coordinates corresponding to all identified interfering light spots from the fourth light spot position array, and obtains a fifth light spot position array as the light spot position array to be numbered.

[0112] refer to Figure 6 , an interference spot screening method disclosed in the embodiment of the present application further includes the following steps: the exclusion unit 3 sequentially traverses and calculates the second distance between each fifth spot position coordinate and the remaining fifth spot position coordinates in the fifth spot position array in accordance with the sorting order, and obtains the second distance array corresponding to each fifth spot position coordinate. The exclusion unit 3 obtains the corresponding second distance minimum value and second distance maximum value from each second distance array. And calculates the first extreme value ratio of each second distance maximum value divided by the corresponding second distance minimum value.

[0113] Next, the exclusion unit 3 determines whether each first extreme value ratio is greater than the fourth preset threshold. If the first extreme value ratio is greater than the fourth preset threshold, the corresponding fifth light spot position coordinate is determined as the first abnormal light spot position coordinate. If the first extreme value ratio is less than or equal to the fourth preset threshold, the corresponding fifth light spot position coordinate is determined as the first normal light spot position coordinate.

[0114] In an embodiment of the present application, in a certain scenario, taking the fourth preset threshold value set to 3.5 as an example, the above-mentioned step of determining whether there are still interfering light spots according to the first extreme value ratio is described in detail. The second distances between each fifth light spot position coordinate and the remaining fifth light spot position coordinates in the fifth light spot position array are sequentially traversed and calculated. For example, when calculating the second distance between the i-th fifth light spot position coordinate and the remaining fifth light spot position coordinates, where i is an integer greater than or equal to 1 and less than or equal to n3, and n3 is the total number of fifth light spot position coordinates in the fifth light spot position array, the second distance array corresponding to the i-th fifth light spot position coordinate is recorded as Length. If the second distance maximum value in the second distance array is recorded as l_max, and the second distance minimum value in the second distance array is recorded as l_min, the first extreme value ratio of the second distance maximum value l_max divided by the second distance minimum value l_min is recorded as l ratio As mentioned above, if the first extreme value ratio l ratio If the value of the second distance maximum l_max corresponds to the second distance between the ith fifth light spot position coordinate and the jth fifth light spot position coordinate, and the second distance minimum l_min corresponds to the second distance between the ith fifth light spot position coordinate and the kth fifth light spot position coordinate, then it can be considered that the ith fifth light spot position coordinate may be abnormal and is determined as the first abnormal light spot position coordinate.

[0115] Next, the exclusion unit 3 obtains all the first abnormal light spot position coordinates to generate a first abnormal light spot position array, and determines whether the number of the first abnormal light spot position coordinates in the first abnormal light spot position array is greater than the fifth preset threshold. If the number of the first abnormal light spot position coordinates in the first abnormal light spot position array is greater than the fifth preset threshold, the exclusion unit 3 establishes an eye diagram horizontal line coordinate system, and determines the third light spot position conversion coordinates corresponding to each first abnormal light spot position coordinate in the eye diagram horizontal line coordinate system.

[0116] In the embodiment of the present application, the fifth preset threshold value can be set to 2. If the number of the first abnormal light spot position coordinates in the first abnormal light spot position array is greater than 2, the fifth light spot position coordinate corresponding to the interfering light spot is further identified by establishing the eye diagram horizontal line coordinate system and deleted from the fifth light spot position array. After deleting the fifth light spot position coordinate corresponding to the interfering light spot, the above step of determining the first abnormal light spot position coordinate by the first extreme value ratio is repeated until the number of the first abnormal light spot position coordinates is less than or equal to 2.

[0117] In the examples of this application, refer to Figure 4 The step of establishing the eye diagram horizontal line coordinate system specifically includes: establishing a three-dimensional world coordinate system based on the target face; detecting the first left eye center position coordinate and the first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate to the camera coordinate system of the first original eye diagram to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate; using the second left eye center position coordinate and the second right eye center position coordinate, calculating the slope of the eye diagram horizontal line, the eye diagram horizontal line is the projection line connecting the left eye center and the right eye center of the target face in the camera coordinate system; selecting any point from the camera coordinate system as the origin, determining the coordinate axis direction by the slope of the eye diagram horizontal line, and establishing the eye diagram horizontal line coordinate system.

[0118] As mentioned above, the establishment of the eye diagram horizontal line coordinate system requires calculating the slope of the line formed by the horizontal line connecting the two eyes in the eye diagram in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the bridge of the target face's nose as the origin (0,0,0), the X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the Z-axis represents the vertical direction of the plane formed by the X-axis and the Y-axis. The first left eye center position coordinate and the first right eye center position coordinate can be expressed as E left and E right For example, E left =(-31.5,0,0),E right=(31.5,0,0). The eye diagram horizontal line coordinate system is established on the assumption that the user wears the head display device normally. Usually, the line between the left eye center 3D position coordinate and the right eye center 3D position coordinate is parallel to the line between the optical center of the left eye physical lens and the optical center of the right eye physical lens of the head display device, and the former and the latter only have a translation relationship (ignore the length of the line segment). The first left eye center position coordinate and the first right eye center position coordinate are respectively converted to the camera coordinate system, and then converted to image pixel coordinates to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate, which are recorded as P respectively. left and P right The second left eye center position coordinate P left and the second right eye center position coordinate P right The line between them is denoted as P l P r , that is, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r Line segment projected onto the first original eye diagram.

[0119] Under the above usage scenario layout of the embodiment of the present application, the second left eye center position coordinate P left =X_P left , Y_P left ), the second right eye center position coordinate P right =X_P right , Y_P right ), calculate the second left eye center position coordinate P according to the following formula left and the second right eye center position coordinate P right The line P between l P r The slope k is:

[0120]

[0121] Thus, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r, which can be represented as a line segment with a slope of k in the camera coordinate system of the first original eye diagram. Based on the camera coordinate system of the first original eye diagram, a new eye diagram horizontal line coordinate system is established. The eye diagram horizontal line coordinate system can select any point in the camera coordinate system of the first original eye diagram as the origin. For example, the eye diagram horizontal line coordinate system selects the following points as the origin: the center point of the first original eye diagram, the center position of the pupil of the first original eye diagram, the origin (0,0) of the pixel coordinate system of the first original eye diagram, the center position coordinate P of the second left eye left and the second right eye center position coordinate P right The line P between l P r The eye diagram horizontal line takes the line segment y=kx+b with a slope of k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0122] Next, the exclusion unit 3 compares the sizes of the converted coordinates of the third light spot positions, and identifies the fifth light spot position coordinates corresponding to the interfering light spot according to the second preset rule. The fifth light spot position coordinates corresponding to all the identified interfering light spots are deleted from the fifth light spot position array, and the second distance traversal calculation is performed again for all the fifth light spot position coordinates currently retained.

[0123] In the embodiment of the present application, the second preset rule includes: identifying the fifth light spot position coordinates corresponding to the maximum / minimum value in each third light spot position conversion coordinate as the fifth light spot position coordinates corresponding to the interference light spot.

[0124] Specifically, in the embodiment of the present application, according to different ways of establishing the above-mentioned coordinate systems, the above-mentioned second preset rule has multiple judgment rules, for example, the fifth light spot position coordinate corresponding to the maximum / minimum value of the horizontal coordinate x in the two third light spot position conversion coordinates is identified as the fifth light spot position coordinate corresponding to the interference light spot. Or the fifth light spot position coordinate corresponding to the maximum / minimum value of the vertical coordinate y in the two third light spot position conversion coordinates is identified as the fifth light spot position coordinate corresponding to the interference light spot.

[0125] The following is only an example of the above-mentioned specific usage scenario. As mentioned above, taking the first original eye image acquired from the left eye as an example, according to the establishment method of each coordinate system mentioned above, the positive direction of the X-axis of the eye image horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P leftThe direction of the vector. The coordinates of the first abnormal light spots with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spots most likely to be formed by the opposite light source can be judged according to the x-values ​​of the corresponding third light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the left eye as an example, the specific rule is that the larger the x-value of the corresponding third light spot position conversion coordinate is, the closer it is to the other eye (right eye). The fifth light spot position coordinate corresponding to the maximum value of the x-value of each corresponding third light spot position conversion coordinate is most likely to be the interference light spot formed by the opposite light source, and it is identified as the fifth light spot position coordinate corresponding to the interference light spot.

[0126] In addition, the examples in the above specific usage scenarios. As mentioned above, taking the first original eye image collected from the right eye as an example, according to the establishment methods of the above coordinate systems, the positive direction of the X-axis of the eye image horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P left The direction of the vector. The coordinates of the first abnormal light spots with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spots most likely to be formed by the opposite light source can be judged according to the x-values ​​of the corresponding third light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the left eye as an example, the specific rule is that the smaller the x-value of the corresponding third light spot position conversion coordinate is, the closer it is to the other eye (left eye), and the fifth light spot position coordinate corresponding to the minimum x-value of the corresponding third light spot position conversion coordinate is most likely to be the interference light spot formed by the opposite light source, and it is identified as the fifth light spot position coordinate corresponding to the interference light spot.

[0127] If the number of first abnormal light spot position coordinates in the first abnormal light spot position array is less than or equal to the fifth preset threshold, the exclusion unit 3 uses the currently retained fifth light spot position coordinates as the sixth light spot position coordinates to obtain the sixth light spot position array as the light spot position array to be numbered.

[0128] refer to Figure 7 , an interference spot screening method disclosed in an embodiment of the present application also includes the following steps: the exclusion unit 3 takes each sixth light spot position coordinate in the sixth light spot position array as a vertex in sequence according to the sorting order; and traverses and calculates the side angles formed by the lines connecting any two remaining sixth light spot position coordinates in the sixth light spot position array and the vertices to obtain a corresponding side angle set.

[0129] The exclusion unit 3 determines whether there is a side angle satisfying the second preset condition in each side angle set. If there is a side angle satisfying the second preset condition in the current side angle set, the sixth light spot position coordinate of the corresponding vertex is determined as the second abnormal light spot position coordinate; if there is no side angle satisfying the second preset condition in the current side angle set, the sixth light spot position coordinate of the corresponding vertex is determined as the second normal light spot position coordinate.

[0130] In the embodiment of the present application, the second preset condition includes: the side angle is less than the fifth preset threshold or the side angle is greater than the sixth preset threshold. Specifically, the fifth preset threshold is Angle_max, Angle_max can be set to 5°, and the sixth preset threshold is correspondingly set to 180°-Angle_max.

[0131] Next, the exclusion unit 3 obtains all the second abnormal light spot position coordinates to generate a second abnormal light spot position array; and establishes an eye diagram horizontal line coordinate system to determine the fourth light spot position conversion coordinates corresponding to each second abnormal light spot position coordinate in the second abnormal light spot position array in the eye diagram horizontal line coordinate system.

[0132] In the examples of this application, refer to Figure 4 The above steps specifically include: establishing a three-dimensional world coordinate system based on the target face; detecting the first left eye center position coordinate and the first right eye center position coordinate of the target face in the three-dimensional world coordinate system; converting the first left eye center position coordinate and the first right eye center position coordinate to the camera coordinate system of the first original eye diagram to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate; using the second left eye center position coordinate and the second right eye center position coordinate, calculating the slope of the eye diagram horizontal line, the eye diagram horizontal line is the projection line connecting the left eye center and the right eye center of the target face in the camera coordinate system; selecting any point from the camera coordinate system as the origin, determining the coordinate axis direction by the slope of the eye diagram horizontal line, and establishing the eye diagram horizontal line coordinate system.

[0133] As mentioned above, the establishment of the eye diagram horizontal line coordinate system requires calculating the slope of the line formed by the horizontal line connecting the two eyes in the eye diagram in the three-dimensional world coordinate system. The three-dimensional world coordinate system takes the bridge of the target face's nose as the origin (0,0,0), the X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the Z-axis represents the vertical direction of the plane formed by the X-axis and the Y-axis. The first left eye center position coordinate and the first right eye center position coordinate can be expressed as E left and E right For example, E left =(-31.5,0,0),E right=(31.5,0,0). The eye diagram horizontal line coordinate system is established on the assumption that the user wears the head display device normally. Usually, the line between the left eye center 3D position coordinate and the right eye center 3D position coordinate is parallel to the line between the optical center of the left eye physical lens and the optical center of the right eye physical lens of the head display device, and the former and the latter only have a translation relationship (ignore the length of the line segment). The first left eye center position coordinate and the first right eye center position coordinate are respectively converted to the camera coordinate system, and then converted to image pixel coordinates to obtain the corresponding second left eye center position coordinate and the second right eye center position coordinate, which are recorded as P respectively. left and P right The second left eye center position coordinate P left and the second right eye center position coordinate P right The line between them is denoted as P l P r , that is, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r Line segment projected onto the first original eye diagram.

[0134] Under the above usage scenario layout of the embodiment of the present application, the second left eye center position coordinate P left =X_P left , Y_P left ), the second right eye center position coordinate P right =X_P right , Y_P right ), calculate the second left eye center position coordinate P according to the following formula left and the second right eye center position coordinate P right The line P between l P r The slope k is:

[0135]

[0136] Thus, the first left eye center position coordinate E in the three-dimensional world coordinate system left and the first right eye center position coordinate E right The line between l E r, which can be represented as a line segment with a slope of k in the camera coordinate system of the first original eye diagram. Based on the camera coordinate system of the first original eye diagram, a new eye diagram horizontal line coordinate system is established. The eye diagram horizontal line coordinate system can select any point in the camera coordinate system of the first original eye diagram as the origin. For example, the eye diagram horizontal line coordinate system selects the following points as the origin: the center point of the first original eye diagram, the center position of the pupil of the first original eye diagram, the origin (0,0) of the pixel coordinate system of the first original eye diagram, the center position coordinate P of the second left eye left and the second right eye center position coordinate P right The line P between l P r The eye diagram horizontal line takes the line segment y=kx+b with a slope of k in the original eye diagram coordinate system (the camera coordinate system of the first original eye diagram) as the X-axis direction, where the value of b depends on the origin of the new eye diagram horizontal line coordinate system.

[0137] The exclusion unit 3 compares the sizes of the converted coordinates of each fourth light spot position, and identifies the sixth light spot position coordinate corresponding to the interfering light spot according to the second preset rule; and deletes the sixth light spot position coordinate corresponding to the identified interfering light spot from the sixth light spot position array, to obtain the seventh light spot position array as the light spot position array to be numbered.

[0138] In the embodiment of the present application, the second preset rule includes: identifying the sixth light spot position coordinates corresponding to the maximum / minimum value in each fourth light spot position conversion coordinate as the sixth light spot position coordinates corresponding to the interference light spot.

[0139] Specifically, in the embodiment of the present application, according to different ways of establishing the above-mentioned coordinate systems, the above-mentioned second preset rule has multiple judgment rules, for example, the sixth light spot position coordinate corresponding to the maximum / minimum value of the horizontal coordinate x in the two fourth light spot position conversion coordinates is identified as the sixth light spot position coordinate corresponding to the interference light spot. Or the sixth light spot position coordinate corresponding to the maximum / minimum value of the vertical coordinate y in the two fourth light spot position conversion coordinates is identified as the sixth light spot position coordinate corresponding to the interference light spot.

[0140] The following is only an example of the above-mentioned specific usage scenario. As mentioned above, taking the first original eye image acquired from the left eye as an example, according to the establishment method of each coordinate system mentioned above, the positive direction of the X-axis of the eye image horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P leftThe direction of the vector. The coordinates of the second abnormal light spot positions with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spot most likely to be formed by the opposite light source can be judged according to the x value of each corresponding fourth light spot position conversion coordinate in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the left eye as an example, the specific rule is that the larger the x value of the corresponding fourth light spot position conversion coordinate is, the closer it is to the other eye (right eye), and the sixth light spot position coordinate corresponding to the maximum value of the x value of each corresponding fourth light spot position conversion coordinate is most likely to be an interference light spot formed by the opposite light source, and it is identified as the sixth light spot position coordinate corresponding to the interference light spot.

[0141] In addition, the examples in the above specific usage scenarios. As mentioned above, taking the first original eye image collected from the right eye as an example, according to the establishment methods of the above coordinate systems, the positive direction of the X-axis of the eye image horizontal line coordinate system is from the second right eye center position coordinate P right To the second left eye center position coordinate P left The direction of the vector. The coordinates of the second abnormal light spot positions with interference light spots are converted into the new eye diagram horizontal line coordinate system, and the interference light spot most likely to be formed by the opposite side light source can be judged according to the x value of the corresponding fourth light spot position conversion coordinates in the new eye diagram horizontal line coordinate system. Taking the first original eye diagram collected from the left eye as an example, the specific rule is that the smaller the x value of the corresponding fourth light spot position conversion coordinate is, the closer it is to the other eye (left eye), and the sixth light spot position coordinate corresponding to the minimum x value of each corresponding fourth light spot position conversion coordinate is most likely to be an interference light spot formed by the opposite side light source, and it is identified as the sixth light spot position coordinate corresponding to the interference light spot.

[0142] In addition, an embodiment of the present application also provides an interference light spot screening device, which includes: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of an interference light spot screening method as described above.

[0143] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the interference spot screening method described above are implemented.

[0144] In the embodiment of the present application, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0145] The methods, steps and logic diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0146] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0147] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0148] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM).

[0149] The storage media described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memory.

[0150] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0151] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for screening interference spots, characterized in that: The method comprises: Performing light spot position detection on the first original eye image to obtain a first light spot position array; Performing iris contour point position detection on the second original eye image to obtain a preset number of iris contour point coordinates; Using the coordinates of each iris contour point, drawing an iris contour on a preset image of the same size as the first original eye image; the preset image has a first preset color; Filling the inner area of ​​the drawn iris outline with a second preset color to obtain a reference image; Extracting pixel colors from the reference image according to the coordinates of each first light spot position in the first light spot position array; Identify the extracted pixel colors corresponding to the coordinates of each first light spot position; If the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinates are deleted; If the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinates are retained as the second light spot position coordinates; By using all the second light spot position coordinates, a second light spot position array is obtained as the light spot position array to be numbered.

2. The interference spot screening method according to claim 1, characterized in that: Performing light spot position detection on the first original eye diagram to obtain a first light spot position array includes: Perform adaptive threshold segmentation on the eye diagram to be detected to obtain the first eye diagram; Performing corrosion and expansion operations on the first eye image to obtain a second eye image; Performing spot profile detection on the second eye image to obtain a spot profile set; Determine the first center point position coordinates of the corresponding light spot according to the first-order moment of each light spot profile; The first center point position coordinates of each light spot are used as the corresponding first light spot position coordinates to obtain a first light spot position array.

3. The interference spot screening method according to claim 2, characterized in that: Performing light spot position detection on the first original eye diagram to obtain a first light spot position array also includes: Determine the coordinates of the first pupil center position in the first original eye image; Based on the iris size in the first original eye image, setting a first horizontal and vertical cropping pixel value range; Using the first pupil center position coordinates and the first horizontal and vertical cropping pixel value ranges as first preset conditions, cropping the first original eye image to obtain an eye image to be detected; The iris region in the first original eye image is included in the eye image to be detected.

4. The interference spot screening method according to claim 3, characterized in that: After filling the inner area of ​​the drawn iris outline with a second preset color, the method further includes: Determine the coordinates of the second pupil center position in the second original eye image; Based on the iris size in the second original eye image, setting a second horizontal and vertical cropping pixel value range; Using the second pupil center position coordinates and second horizontal and vertical cropping pixel value ranges as second preset conditions, cropping the preset image filled with the second preset color to obtain the reference image; The iris region in the second original eye image is included in the reference image, and the size of the reference image cropped according to the second preset condition is the same as the size of the eye image to be detected cropped according to the first preset condition.

5. The interference spot screening method according to claim 4, characterized in that: The first original eye diagram and the second original eye diagram are the same original eye diagram, or the first original eye diagram and the second original eye diagram are different original eye diagrams obtained under the same acquisition conditions, and the same acquisition conditions include: the eye diagram acquisition equipment is the same, the target human eyes for acquisition are the same, the target human eye gaze points are the same, the acquisition target human eye positions are the same, and the arrangement positions of the light sources in the light source group are the same.

6. A method for screening interference spots according to any one of claims 1 to 5, characterized in that: The method further comprises: Determining whether the number of second light spot position coordinates in the second light spot position array is greater than or equal to a first preset threshold; If the number of second light spot position coordinates in the second light spot position array is greater than or equal to a first preset threshold, establishing a light spot coordinate system based on each second light spot position coordinate; According to a first preset rule, sorting the coordinates of each second light spot position in the second light spot position array to obtain a third light spot position array; In order of sorting, it is detected in sequence whether the first vector angle corresponding to two adjacent third light spot position coordinates in the third light spot position array is greater than or equal to a second preset threshold; If the first vector angle is greater than or equal to a second preset threshold, the corresponding two adjacent third light spot position coordinates are determined as a first light spot position pair without an interfering light spot; If the angle between the first vectors is smaller than a second preset threshold, the corresponding two adjacent third light spot position coordinates are determined as a second light spot position pair with an interfering light spot; Establishing an eye diagram horizontal line coordinate system, and determining two first light spot position conversion coordinates corresponding to two third light spot position coordinates of each second light spot position pair in the eye diagram horizontal line coordinate system; Compare the sizes of the corresponding two first light spot position conversion coordinates, and identify the third light spot position coordinates corresponding to the interference light spot according to the second preset rule; The third light spot position coordinates corresponding to all identified interfering light spots are deleted from the third light spot position array to obtain a fourth light spot position array as the light spot position array to be numbered.

7. The interference spot screening method according to claim 6, characterized in that: The method further comprises: If the number of second light spot position coordinates in the second light spot position array is less than the first preset threshold, the second light spot position array is directly stored as a fourth light spot position array as the light spot position array to be numbered.

8. The interference spot screening method according to claim 7, characterized in that: The origin position of the light spot coordinate system is the centroid position of each second light spot position coordinate.

9. The interference spot screening method according to claim 7, characterized in that: The first preset rule is a clockwise sorting rule or a counterclockwise sorting rule.

10. The interference spot screening method according to claim 7, characterized in that: Establish the eye diagram horizontal line coordinate system, including: Establish a three-dimensional world coordinate system based on the target face; Detecting a first left eye center position coordinate and a first right eye center position coordinate of the target face in the three-dimensional world coordinate system; Convert the first left eye center position coordinate and the first right eye center position coordinate into the camera coordinate system of the first original eye image to obtain corresponding second left eye center position coordinate and second right eye center position coordinate; Using the second left eye center position coordinates and the second right eye center position coordinates, calculating the slope of an eye diagram horizontal line, where the eye diagram horizontal line is a projection line connecting the left eye center and the right eye center of the target face in the camera coordinate system; An arbitrary point is selected from the camera coordinate system as the origin, the direction of the coordinate axis is determined by the slope of the eye diagram horizontal line, and the eye diagram horizontal line coordinate system is established.

11. A method for screening interference spots according to any one of claims 7 to 10, characterized in that: The method further comprises: Calculating a first distance between any two fourth light spot position coordinates in the fourth light spot position array; Determining whether the first distance is less than or equal to a third preset threshold; If the first distance is greater than a third preset threshold, the corresponding two fourth light spot position coordinates are determined as a third light spot position pair without an interfering light spot; If the first distance is less than or equal to a third preset threshold, the corresponding two fourth light spot position coordinates are determined as a fourth light spot position pair with an interfering light spot; Establishing an eye diagram horizontal line coordinate system, and determining two second light spot position conversion coordinates corresponding to two fourth light spot position coordinates of each fourth light spot position pair in the eye diagram horizontal line coordinate system; Comparing the sizes of the corresponding two second light spot position conversion coordinates, identifying the fourth light spot position coordinates corresponding to the interference light spot according to the second preset rule; The fourth light spot position coordinates corresponding to all identified interfering light spots are deleted from the fourth light spot position array to obtain a fifth light spot position array as the light spot position array to be numbered.

12. The interference spot screening method according to claim 11, characterized in that: The method further comprises: In order of sorting, the second distances between each fifth light spot position coordinate and the remaining fifth light spot position coordinates in the fifth light spot position array are traversed and calculated in sequence to obtain a second distance array corresponding to each fifth light spot position coordinate; Obtain the corresponding second distance minimum value and second distance maximum value from each second distance array; Calculate a first extreme value ratio of each second distance maximum value divided by the corresponding second distance minimum value; Determine whether each first extreme value ratio is greater than a fourth preset threshold; If the first extreme value ratio is greater than a fourth preset threshold, the corresponding fifth light spot position coordinate is determined as the first abnormal light spot position coordinate; If the first extreme value ratio is less than or equal to the fourth preset threshold, the corresponding fifth light spot position coordinate is determined as the first normal light spot position coordinate; Acquire all first abnormal light spot position coordinates to generate a first abnormal light spot position array; Determining whether the number of first abnormal light spot position coordinates in the first abnormal light spot position array is greater than a fifth preset threshold; If the number of first abnormal light spot position coordinates in the first abnormal light spot position array is greater than a fifth preset threshold, an eye diagram horizontal line coordinate system is established to determine the third light spot position conversion coordinates corresponding to each first abnormal light spot position coordinate in the eye diagram horizontal line coordinate system; Comparing the sizes of the converted coordinates of the positions of the third light spots, and identifying the position coordinates of the fifth light spot corresponding to the interfering light spot according to the second preset rule; Deleting the fifth light spot position coordinates corresponding to all identified interference light spots from the fifth light spot position array, and looping to recalculate the second distance for all currently retained fifth light spot position coordinates; If the number of first abnormal spot position coordinates in the first abnormal spot position array is less than or equal to the fifth preset threshold, the currently reserved fifth spot position coordinates are used as sixth spot position coordinates to obtain the sixth spot position array as the spot position array to be numbered.

13. The interference spot screening method according to claim 12, characterized in that: The method further comprises: In order of sorting, each sixth light spot position coordinate in the sixth light spot position array is used as a vertex; Traversing and calculating the side angles formed by the lines connecting the coordinates of any two remaining sixth light spot positions in the sixth light spot position array and the vertices to obtain a corresponding side angle set; Determine whether there is an edge angle that meets a second preset condition in each edge angle set; If there is an edge angle satisfying the second preset condition in the current edge angle set, the sixth light spot position coordinate of the corresponding vertex is determined as the second abnormal light spot position coordinate; If there is no side angle satisfying the second preset condition in the current side angle set, the sixth light spot position coordinates of the corresponding vertex are determined as the second normal light spot position coordinates; Acquire all the second abnormal light spot position coordinates to generate a second abnormal light spot position array; Establishing an eye diagram horizontal line coordinate system, and determining fourth light spot position conversion coordinates corresponding to each second abnormal light spot position coordinate in the second abnormal light spot position array in the eye diagram horizontal line coordinate system; Comparing the sizes of the converted coordinates of the positions of the fourth light spots, and identifying the position coordinates of the sixth light spot corresponding to the interfering light spot according to the second preset rule; The sixth light spot position coordinate corresponding to the identified interfering light spot is deleted from the sixth light spot position array to obtain a seventh light spot position array as the light spot position array to be numbered.

14. The interference spot screening method according to claim 13, characterized in that: The second preset condition includes: the side angle is smaller than a fifth preset threshold or the side angle is larger than a sixth preset threshold.

15. The interference spot screening method according to claim 12, characterized in that: The second preset rule includes: Identify the third light spot position coordinates corresponding to the maximum / minimum value of the two first light spot position conversion coordinates as the third light spot position coordinates corresponding to the interference light spot; Identify the fourth light spot position coordinates corresponding to the maximum / minimum value of the two second light spot position conversion coordinates as the fourth light spot position coordinates corresponding to the interference light spot; Identify the fifth light spot position coordinates corresponding to the maximum / minimum values ​​in each third light spot position conversion coordinate as the fifth light spot position coordinates corresponding to the interference light spot; The sixth light spot position coordinates corresponding to the maximum / minimum values ​​in the fourth light spot position conversion coordinates are identified as the sixth light spot position coordinates corresponding to the interference light spot.

16. An interference spot screening system, characterized in that: The system is used to perform the steps in the interference spot screening method according to any one of claims 1 to 15, comprising: A first detection unit, used for performing light spot position detection on the first original eye diagram to obtain a first light spot position array; A second detection unit is used to perform iris contour point position detection on the second original eye image to obtain a preset number of iris contour point coordinates; Excludes cells that perform the following steps: Using the coordinates of each iris contour point, drawing an iris contour on a preset image of the same size as the first original eye image; the preset image has a first preset color; Filling the inner area of ​​the drawn iris outline with a second preset color to obtain a reference image; Extracting pixel colors from the reference image according to the coordinates of each first light spot position in the first light spot position array; Identify the extracted pixel colors corresponding to the coordinates of each first light spot position; If the extracted pixel color is the first preset color, the corresponding light spot is not within the iris contour, and the corresponding first light spot position coordinates are deleted; If the extracted pixel color is the second preset color, the corresponding light spot is within the iris contour, and the corresponding first light spot position coordinates are retained as the second light spot position coordinates; By using all the second light spot position coordinates, a second light spot position array is obtained as the light spot position array to be numbered.

17. An interference spot screening device, characterized in that: The device comprises: a processor and a memory; The memory is used to store one or more program instructions; The processor is used to run one or more program instructions to execute the steps of an interference spot screening method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interference spot screening method according to any one of claims 1 to 15 are implemented.

Images