Pupil distance detection method, storage medium and electronic equipment

Abstract

The invention provides an interpupillary distance detection method, a storage medium and electronic equipment, after a left view image and a right view image are acquired, a first left eye pupil position and a first right eye pupil position in the left view image are analyzed, and a second left eye pupil position and a second right eye pupil position in the right view image are analyzed. And determining the parallax of the left-eye camera and the right-eye camera. And determining the shooting depths of the left eye pupil and the right eye pupil. And finally converting the positions of the two pupils in the plane image into an expression mode under a world coordinate system through coordinate system conversion. In a world coordinate system, the accurate pupil distance can be obtained by directly using the coordinates of the two pupil positions to carry out distance calculation. According to the scheme, the position representation mode of the left eye pupil and the position representation mode of the right eye pupil are finally achieved in the coordinate representation mode under the same world coordinate system, the mode that the pupil distance is calculated by restoring the real positions of the two pupils is adopted, the obtained pupil distance detection result is more consistent with the actual situation, and the obtained lens matching result is more consistent with the requirement.

Description

Technical Field

[0001] The present application relates to the technical field of eyeglass fitting equipment, and in particular to a pupil distance detection method, a storage medium and an electronic device. Background Art

[0002] In short, the pupil distance is the horizontal distance between the two points of the left and right pupil centers, which is a key parameter required for fitting glasses. The distance between the centers of the two lenses (optical center distance) should match the pupil distance of the wearer. Otherwise, even if the degree is correct, wearing glasses will cause discomfort, and in severe cases, it will affect vision.

[0003] The optometry instruments in the prior art have the function of measuring pupil distance. After the chin of the measured object is placed on the chin rest of the optometry instrument, the facial image of the measured object is captured by a binocular camera. One camera obtains a left view image and the other camera obtains a right view image. The pupil positions in the left view image and the right view image are calculated, and the average result is used as the actual pupil position of the measured object, and the pupil distance is calculated accordingly. However, in actual applications, the shooting results of the two cameras in the binocular camera will have errors to a large extent. For example, due to the different influences of the shooting environment lighting and the different distortion of the camera lenses on the two cameras, there will be certain deviations between the left view image and the right view image itself. The pupil position obtained in each image may also have the same error. On this basis, the pupil position calculated by simply using the averaging method has a large error, and the accuracy of the pupil distance obtained is also low, which affects the glasses fitting results. Summary of the invention

[0004] The technical problem to be solved by the present application is that the pupil distance detection method in the prior art is not accurate enough, and thus a pupil distance detection method, a storage medium and an electronic device are provided.

[0005] In a first aspect, the technical solution of the present application provides a method for detecting pupil distance, comprising:

[0006] Acquire a left-view image, parse the left-view image to obtain a first left-eye pupil position and a first right-eye pupil position, wherein the first left-eye pupil position and the first right-eye pupil position are represented by pixel coordinates in a left-view image coordinate system; acquire a right-view image, parse the right-view image to obtain a second left-eye pupil position and a second right-eye pupil position, wherein the second left-eye pupil position and the second right-eye pupil position are represented by pixel coordinates in a right-view image coordinate system;

[0007] Calculating the parallax of the left-eye camera and the right-eye camera according to the distance between the first left-eye pupil position and the left imaging plane of the left-view image, the distance between the second left-eye pupil position and the left imaging plane of the right-view image, the distance between the first right-eye pupil position and the right imaging plane of the left-view image, and the distance between the second right-eye pupil position and the right imaging plane of the right-view image;

[0008] Determine a left eye pupil depth and a right eye pupil depth according to the parallax and the camera focal length, wherein the left eye pupil depth and the right eye pupil depth respectively represent vertical distances between the left eye pupil and the right eye pupil and a straight line where the left eye camera and the right eye camera are located;

[0009] Determine the three-dimensional coordinates of the left eye pupil in the camera coordinate system according to the left eye pupil depth, in combination with the first left eye pupil position and the second left eye pupil position; determine the three-dimensional coordinates of the right eye pupil in the camera coordinate system according to the right eye pupil depth, in combination with the first right eye pupil position and the second right eye pupil position;

[0010] According to the mapping relationship between the world coordinate system and the camera coordinate system, the three-dimensional coordinates of the left eye pupil are converted into the world coordinates of the left eye pupil, and the three-dimensional coordinates of the right eye pupil are converted into the world coordinates of the right eye pupil;

[0011] The pupil distance is calculated according to the left eye pupil world coordinates and the right eye pupil world coordinates.

[0012] In some embodiments, the pupil distance detection method, wherein the acquiring a left-view image, parsing the left-view image to obtain a first left-eye pupil position and a first right-eye pupil position, and the acquiring a right-view image, parsing the right-view image to obtain a second left-eye pupil position and a second right-eye pupil position, comprises:

[0013] Acquire a face image in an image captured by a camera, wherein the camera is a left camera and a right camera;

[0014] Determine a left eye area and a right eye area in the face image;

[0015] Preprocessing the left eye region and the right eye region to obtain a left eye region of interest corresponding to the left eye pupil and a right eye region of interest corresponding to the right eye pupil;

[0016] Finding a plurality of left intra-pupil area contours in the left-eye region of interest and a plurality of right intra-pupil area contours in the right-eye region of interest;

[0017] Fitting a minimum circumscribed circle of the left area to the contours of several left pupil areas, and if the radius of the minimum circumscribed circle of the left area is within a set range, taking the center of the minimum circumscribed circle of the left area as the left eye pupil position; when the camera is a left-eye camera, the left eye pupil position is the first left eye pupil position, and when the camera is a right-eye camera, the left eye pupil position is the second left eye pupil position;

[0018] Fitting the right area minimum circumscribed circle of several right pupil area contours, if the radius of the right area minimum circumscribed circle is within the set radius range, the center of the right area minimum circumscribed circle is used as the right eye pupil position; when the camera is a left-eye camera, the right eye pupil position is the first right eye pupil position, when the camera is a right-eye camera, the right eye pupil position is the second right eye pupil position.

[0019] 3. The pupil distance detection method according to claim 2, characterized in that the left eye area and the right eye area are preprocessed to obtain a left eye interest area corresponding to the left eye pupil and a right eye interest area corresponding to the right eye pupil, wherein the preprocessing comprises:

[0020] Gaussian blur processing to obtain a Gaussian blurred image;

[0021] Performing binarization conversion on the Gaussian blurred image to obtain a binarized image;

[0022] A morphological operation is performed on the binary image, wherein the morphological operation includes first performing a closing operation and then performing an opening operation.

[0023] In some embodiments of the pupil distance detection method, the set radius range is determined by:

[0024] Get the number of pixels contained in the left eye region of interest corresponding to the left eye pupil and the right eye region of interest corresponding to the right eye pupil;

[0025] Determine the radius base according to the focal length of the camera and the number of pixels;

[0026] The set radius range is determined according to the allowable error coefficient and the radius base.

[0027] Pupillary distance detection methods described in some schemes:

[0028] The parallax between the left eye camera and the right eye camera is calculated according to the distance between the first left eye pupil position and the left imaging plane of the left view image, the distance between the second left eye pupil position and the left imaging plane of the right view image, the distance between the first right eye pupil position and the right imaging plane of the left view image, and the distance between the second right eye pupil position and the right imaging plane of the right view image:

[0029] The distance between the first left eye pupil position and the left imaging plane of the left view image is represented by X LL , the distance between the second left eye pupil position and the left imaging plane of the right-view image is expressed as X RL ; The parallax d of the binocular camera for the left pupil L It is calculated as follows: d L =|X LL -X RL |;

[0030] The distance between the second left eye pupil position and the right imaging plane of the left-view image is represented by X LR , the distance between the second left eye pupil position and the right imaging plane of the right-view image is expressed as X RR ; The parallax d of the binocular camera for the right pupil R It is calculated as follows: d R =|X LR -X RR |;

[0031] According to the parallax and the focal length of the camera, the left eye pupil depth and the right eye pupil depth are determined: the left eye pupil depth is: Z L =b×f / d L , the right eye pupil depth is: Z R =b×f / d R , where b represents the distance between the center points of the left camera and the right camera, and f represents the focal length of the camera.

[0032] In some embodiments, the pupil distance detection method includes determining the three-dimensional coordinates of the left eye pupil in the camera coordinate system according to the left eye pupil depth, in combination with the first left eye pupil position and the second left eye pupil position; and determining the three-dimensional coordinates of the right eye pupil in the camera coordinate system according to the right eye pupil depth, in combination with the first right eye pupil position and the second right eye pupil position, including:

[0033] Obtain a first connection line between the projection center point of the left camera and the first left eye pupil position, and obtain a second connection line between the projection center point of the right camera and the second left eye pupil position;

[0034] The intersection point of the extended lines of the first connecting line and the second connecting line is the position of the left eye pupil in the camera coordinate system, and the three-dimensional coordinates of the left eye pupil are determined based on the intersection point;

[0035] Obtain a third connecting line between the projection center point of the right camera and the first right eye pupil position, and obtain a fourth connecting line between the projection center point of the right camera and the second right eye pupil position;

[0036] The intersection point of the extended lines of the third connecting line and the fourth connecting line is the position of the right eye pupil in the camera coordinate system, and the three-dimensional coordinates of the right eye pupil are determined based on this.

[0037] In some embodiments, the pupil distance detection method, according to the mapping relationship between the world coordinate system and the camera coordinate system, converting the left eye pupil three-dimensional coordinates into the left eye pupil world coordinates, and converting the right eye pupil three-dimensional coordinates into the right eye pupil world coordinates, comprises:

[0038] The mapping relationship between the world coordinate system and the camera coordinate system is represented by the following transformation matrix:

[0039]

[0040] Among them, (X w , Y w , Z w ) is the coordinate representation in the world coordinate system, (X c , Y c , Z c ) is the coordinate representation in the camera coordinate system, R represents the rotation matrix, and T represents the translation matrix.

[0041] In a second aspect, the technical solution of the present application provides a computer-readable storage medium, in which program information is stored. After a computer reads the program information, the computer executes the steps of the pupil distance detection method described in any one of the technical solutions of the first aspect.

[0042] In a third aspect, the technical solution of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the pupil distance detection method described in any technical solution of the first aspect are implemented.

[0043] In a fourth aspect, the technical solution of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the pupil distance detection method described in any technical solution of the first aspect.

[0044] Compared with the prior art, the above technical solution provided by this application has the following technical effects:

[0045] The pupil distance detection method, storage medium and electronic device provided by the present application, after acquiring the left-view image and the right-view image, parse the left eye pupil position and the right eye pupil position in the left-view image as the first left eye pupil position and the first right eye pupil position, and parse the left eye pupil position and the right eye pupil position in the right-view image as the second left eye pupil position and the second right eye pupil position. The imaging principle of the camera is used to determine the parallax of the left-eye camera and the right-eye camera. On the premise that the camera parallax and focal length have been clarified, the shooting depth of the left eye pupil and the right eye pupil can be determined by combining the positions of the two pupils in the left-view image and the right-view image. Because the image captured by the camera is planar, in order to be able to calculate the accurate pupil distance, the two pupil positions in the plane image are transformed through the coordinate system and finally converted into the representation in the world coordinate system. In the world coordinate system, the accurate pupil distance can be obtained by directly using the coordinates of the two pupil positions for distance calculation. Through the above technical solution of the present application, the images captured by the left-view camera and the right-view camera are processed and transformed in coordinate system mapping, and finally the position representation of the left eye pupil and the right eye pupil is realized in the same coordinate representation under the world coordinate system. Compared with the method of simply using the image shooting results to calculate the mean in the prior art, the solution of the present application eliminates the shooting error caused by the camera itself by restoring the real position of the two pupils and then calculating the pupil distance. The obtained pupil distance detection result is more consistent with the actual situation, and the glasses fitting result obtained based on this is also more in line with the needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a pupil distance detection method according to an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the optical path of a binocular camera imaging process according to an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the mapping relationship between the camera coordinate system and the world coordinate system according to an embodiment of the present application;

[0049] Figure 4 A flowchart of a pupil recognition process according to an embodiment of the present application;

[0050] Figure 5 A flow chart of the steps for determining a radius range for setting a pupil radius according to an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the hardware connection relationship of an electronic device that performs a pupil distance detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] The specific implementation of the present application is further described below with reference to the accompanying drawings.

[0053] It is easy to understand that according to the technical solution of the present application, without changing the essential spirit of the present application, a variety of structural modes and implementation modes that can be replaced by those skilled in the art can be replaced with each other. Therefore, the following specific implementation modes and drawings are only exemplary descriptions of the technical solution of the present application, and should not be regarded as the entirety of the present application or as a limitation or restriction on the technical solution of the application.

[0054] The directional terms such as up, down, left, right, front, back, front, back, top, bottom, etc. mentioned or may be mentioned in this specification are defined relative to the structures shown in the drawings. They are relative concepts and may change accordingly according to different positions and different usage conditions. Therefore, these or other directional terms should not be interpreted as restrictive terms.

[0055] This embodiment provides a pupil distance detection method, which is applied to vision screening equipment such as optometry instruments. The screening equipment has a binocular camera configuration, such as Figure 1 As shown, the method includes:

[0056] S10: Acquire a left-view image, parse the left-view image to obtain a first left-eye pupil position and a first right-eye pupil position, wherein the first left-eye pupil position and the first right-eye pupil position are represented by pixel coordinates in a left-view image coordinate system; acquire a right-view image, parse the right-view image to obtain a second left-eye pupil position and a second right-eye pupil position, wherein the second left-eye pupil position and the second right-eye pupil position are represented by pixel coordinates in a right-view image coordinate system.

[0057] Specifically, the left view image is obtained by taking a picture by a left eye camera, and the right view image is obtained by taking a picture by a right eye camera. The positions of the two pupils in each image are determined according to the analysis results of the left view image and the right view image. The image analysis method can identify the pupil position by determining the pixel information. Usually, the color of the pupil is different from the color of the surrounding area, and its brightness is also different. Therefore, the pupil position can be determined by determining the chromaticity and brightness of the pixel. In this step, the pupil position is represented by the pixel coordinates in the image coordinate system. An image can be regarded as an array of M rows and N columns. Each value in the array is the brightness of the point. A rectangular coordinate system u and v is established in the upper left corner of the image. The coordinates (u, v) of each pixel point are the row and column of the pixel in the array. Therefore, in the left view image and the right view image, the pupil position can be represented by the corresponding pixel coordinates (u, v) in the pixel coordinate system. In order to describe the specific position of an object in three-dimensional space, it is necessary to establish a coordinate system related to the actual physical unit and convert the pixel coordinate system into the image coordinate system. The image coordinate system uses a point in the image as the coordinate origin, and its x-axis and y-axis are parallel to the u-axis and v-axis of the pixel coordinate system respectively. The mapping relationship between any pixel on the image and the two coordinate systems is:

[0058]

[0059] As mentioned above, u, v are pixel coordinates in the pixel coordinate system, x, y are coordinates in the image coordinate system, du, dv are the actual physical distances of a single pixel, and x0, y0 are the offsets in the image coordinate system. Converting the above equation into a homogeneous equation is:

[0060]

[0061] That is, as long as the pixel coordinates (u, v) of any point are determined, its pixel coordinates (x, y) in the image coordinate system can be determined.

[0062] S20: Calculate the parallax of the left-eye camera and the right-eye camera according to the distance between the first left-eye pupil position and the left imaging plane of the left-view image, the distance between the second left-eye pupil position and the left imaging plane of the right-view image, the distance between the first right-eye pupil position and the right imaging plane of the left-view image, and the distance between the second right-eye pupil position and the right imaging plane of the right-view image.

[0063] like Figure 2 As shown, point P is set to be any point in the actual space, O L and O R Respectively represent the center of the left camera and the center of the right camera. L and P RThey are respectively the imaging point of point P on the imaging plane L of the left camera and the imaging point of the left pupil P on the imaging plane L of the right camera, and the imaging point P L The distance between the left imaging surface of the left viewing image is X L , imaging point P R The distance X between the left imaging plane of the right-view image R . b represents the distance between the center of the left camera and the projection center of the right camera. F represents the focal length of the camera. According to the principle of rectilinear propagation of light, point P is the intersection of the line connecting the projection center point and the imaging point of the left camera and the right camera. The distance from point P to the center plane of the binocular camera projection is also its depth. Then the parallax of point P in the left camera and the right camera is obtained as follows: d = |x L -x R In practical applications, the left eye pupil and the right eye pupil are respectively used as the P point, and the above principle can be used to determine the parallax between the left eye camera and the right eye camera corresponding to the left eye pupil and the right eye pupil respectively.

[0064] S30: Determine the left eye pupil depth and the right eye pupil depth according to the parallax and the camera focal length, wherein the left eye pupil depth and the right eye pupil depth respectively represent the vertical distance between the left eye pupil and the right eye pupil and the straight line where the left eye camera and the right eye camera are located.

[0065] Also combined Figure 2 The schematic diagram shown in the figure shows the imaging point P in the left eye view. L and right camera P R The distance P between L P R for:

[0066]

[0067] According to the theory of similar triangles, we can conclude that:

[0068]

[0069] The distance Z from point P to the center plane of the projection, i.e. its depth, is calculated as follows:

[0070]

[0071] From the above formula, it can be determined that the parallax is inversely proportional to the distance from any point in the three-dimensional space to the center plane of the binocular camera projection. Based on the same principle, after determining the parallax of the pupil, the pupil depth can be determined.

[0072] Specifically, the distance between the first left eye pupil position and the left imaging plane of the left-view image is expressed as X LL, the distance between the second left eye pupil position and the left imaging plane of the right-view image is expressed as X RL ; The parallax d of the binocular camera for the left pupil L It is calculated as follows: d L =|X LL -X RL |; The distance between the second left eye pupil position and the right imaging plane of the left-view image is represented by X LR , the distance between the second left eye pupil position and the right imaging plane of the right-view image is expressed as X RR ; The parallax d of the binocular camera for the right pupil R It is calculated as follows: d R =|X LR -X RR |; According to the parallax and camera focal length, determine the left eye pupil depth and the right eye pupil depth: the left eye pupil depth is: Z L =b×f / d L , the right eye pupil depth is: Z R =b×f / d R .

[0073] S40: Determine the three-dimensional coordinates of the left eye pupil in the camera coordinate system according to the left eye pupil depth, in combination with the first left eye pupil position and the second left eye pupil position; determine the three-dimensional coordinates of the right eye pupil in the camera coordinate system according to the right eye pupil depth, in combination with the first right eye pupil position and the second right eye pupil position.

[0074] As mentioned above, the first left eye pupil position and the second left eye pupil position, the first right eye pupil position and the second right eye pupil position are pixel coordinates in the image coordinate system. Although the image coordinate system establishes a physical relationship between the image and the real world, it is only a two-dimensional relationship. In order to calculate the accurate pupil distance, it is necessary to establish a three-dimensional coordinate system related to the three-dimensional world, that is, the camera coordinate system. In essence, the image coordinate system is a plane of the camera coordinate system. The x-axis and y-axis of the camera coordinate system are parallel to the coordinate axes corresponding to the image coordinate system. The distance between the two coordinate systems is the focal length f of the camera. The camera coordinate system uses the optical center of the camera as the coordinate origin, and the optical axis is the z-axis. Figure 2 Taking point P as an example, its coordinates in the camera coordinate system are expressed as (Xc, Yc, -Z). The mapping relationship expression used when converting the image coordinate system to the camera coordinate system is:

[0075]

[0076] Therefore, as long as the coordinates of any point in the image coordinate system are determined, the coordinates in the camera coordinate system can be converted using the above mapping relationship expression.

[0077] Specifically in this step, a first connecting line between the projection center point of the left camera and the first left eye pupil position is obtained, and a second connecting line between the projection center point of the right camera and the second left eye pupil position is obtained; the intersection of the extended lines of the first connecting line and the second connecting line is the position of the left eye pupil in the camera coordinate system, and the three-dimensional coordinates of the left eye pupil are determined accordingly; a third connecting line between the projection center point of the right camera and the first right eye pupil position is obtained, and a fourth connecting line between the projection center point of the right camera and the second right eye pupil position is obtained; the intersection of the extended lines of the third connecting line and the fourth connecting line is the position of the right eye pupil in the camera coordinate system, and the three-dimensional coordinates of the right eye pupil are determined accordingly.

[0078] S50: According to the mapping relationship between the world coordinate system and the camera coordinate system, the three-dimensional coordinates of the left eye pupil are converted into the world coordinates of the left eye pupil, and the three-dimensional coordinates of the right eye pupil are converted into the world coordinates of the right eye pupil.

[0079] The camera coordinate system is a three-dimensional coordinate system that describes the real world with the camera as the center. In the real space, there are countless coordinate systems that can describe the three-dimensional space. In order to determine the specific positions of the two pupils in the three-dimensional space, it is necessary to determine a unique reference coordinate system to describe the space, which is the world coordinate system, in which the coordinates are represented by (Xw, Yw, Zw). Figure 3 As shown, the coordinate system in the form of a dotted line is the world coordinate system, and the coordinate system in the form of a solid line is the camera coordinate system. The mapping relationship between the world coordinate system and the camera coordinate system can be determined by the following matrix transformation method:

[0080]

[0081] It can be transformed into:

[0082]

[0083] Where: R is the 3×3 unit matrix, which represents the rotation operation of the coordinate system; T is the translation matrix, which represents the translation amount of the coordinate system; 0 represents the three-dimensional zero vector, 0 T Represents the transpose of the three-dimensional zero vector.

[0084] According to the above matrix transformation method, as long as the coordinates of the two pupils in the camera coordinate system are determined, the coordinates of the two pupils in the world coordinate system can be determined.

[0085] The distance b between the center of the left camera and the projection center of the right camera, the focal length f of the camera, the rotation matrix R, and the translation matrix T in the above formulas can be obtained by performing a calibration test on the camera.

[0086] S60: Calculate the pupil distance according to the left eye pupil world coordinates and the right eye pupil world coordinates.

[0087] After determining the positions of the left and right pupils in the world coordinate system, the distance calculation formula can be used to directly solve the pupil distance. Specifically,

[0088] Among them, the left eye pupil world coordinate system is expressed as (X WL , Y WL , Z WL ), the world coordinate system of the right pupil is expressed as (X WR , Y WR , Z WR ).

[0089] The solution provided in the above embodiment of the present application, after obtaining the left view image and the right view image, the left eye pupil position and the right eye pupil position in the left view image are parsed as the first left eye pupil position and the first right eye pupil position, and the left eye pupil position and the right eye pupil position in the right view image are parsed as the second left eye pupil position and the second right eye pupil position. The parallax of the left eye camera and the right eye camera is determined by using the camera imaging principle. On the premise that the camera parallax and focal length have been clarified, the shooting depth of the left eye pupil and the right eye pupil can be determined by combining the positions of the two pupils in the left view image and the right view image. Because the image captured by the camera is planar, in order to be able to calculate the accurate pupil distance, the two pupil positions in the plane image are finally converted into a representation in the world coordinate system through the transformation of the coordinate system. In the world coordinate system, the accurate pupil distance can be obtained by directly using the coordinates of the two pupil positions for distance calculation. Through the above technical solution of the present application, the images captured by the left-view camera and the right-view camera are processed and transformed in coordinate system mapping, and finally the position representation of the left eye pupil and the right eye pupil is realized in the same coordinate representation under the world coordinate system. Compared with the method of simply using the image shooting results to calculate the mean in the prior art, the solution of the present application eliminates the shooting error caused by the camera itself by restoring the real position of the two pupils and then calculating the pupil distance. The obtained pupil distance detection result is more consistent with the actual situation, and the glasses fitting result obtained based on this is also more in line with the needs.

[0090] like Figure 4As shown, in the above scheme, the step S10 of acquiring a left-view image, parsing the left-view image to obtain a first left-eye pupil position and a first right-eye pupil position, and the step of acquiring a right-view image, parsing the right-view image to obtain a second left-eye pupil position and a second right-eye pupil position include:

[0091] S101: Acquire a face image in an image captured by a camera, wherein the camera is a left-eye camera and a right-eye camera.

[0092] Specifically, the left view image is obtained by taking a picture with the left camera, and the right view image is obtained by taking a picture with the right camera. The face area can be identified by image processing technology. It can be understood that if the face information cannot be identified, it is necessary to retake the picture. In addition, in this step, the shooting is carried out under an infrared light source, because infrared light does not belong to visible light and has little stimulation to the human eye. When shooting, there will be no effects such as pupil contraction caused by the light source stimulating the human eye.

[0093] S102: Determine a left eye region and a right eye region in the face image.

[0094] Specifically, after the left eye and the right eye in the face image are identified, the left eye region and the right eye region are framed. The eye region can be identified by an existing machine learning method or an image processing method.

[0095] S103: Preprocessing the left eye region and the right eye region to obtain a left eye region of interest corresponding to the left eye pupil and a right eye region of interest corresponding to the right eye pupil.

[0096] The left eye interest region corresponding to the left eye pupil and the right eye interest region corresponding to the right eye pupil are multiple small regions in the left eye pupil and multiple small regions in the right eye pupil region. Each small region is a part of the pupil.

[0097] S104: searching for a plurality of left intra-pupil area contours in the left-eye region of interest and a plurality of right intra-pupil area contours in the right-eye region of interest.

[0098] The outer contour of each region is determined by contour searching. For each region, the pixel values ​​of the pixels inside and outside the region are different. Based on this, the boundary where the pixel value changes can be found, and then the contour can be determined.

[0099] S105: Fitting the minimum circumscribed circle of the left area to the contours of several left pupil areas. If the radius of the minimum circumscribed circle of the left area is within a set range, the center of the minimum circumscribed circle of the left area is used as the left eye pupil position; when the camera is a left-eye camera, the left eye pupil position is the first left eye pupil position, and when the camera is a right-eye camera, the left eye pupil position is the second left eye pupil position.

[0100] The minimum circumscribed circle is used to simulate the pupil, because the radius of the pupil has a certain range, and the radius of the minimum circumscribed circle needs to match the actual radius of the pupil. Therefore, in this step, when the radius of the minimum circumscribed circle of the left area is within the set range, it is used as the simulated left pupil contour, and its center or center of the circle is used as the left eye pupil position.

[0101] S106: Fitting the right area minimum circumscribed circle for several right pupil area contours, and if the radius of the right area minimum circumscribed circle is within a set radius range, taking the center of the right area minimum circumscribed circle as the right eye pupil position; when the camera is a left-eye camera, the right eye pupil position is the first right eye pupil position, and when the camera is a right-eye camera, the right eye pupil position is the second right eye pupil position.

[0102] Similar to step S105, when the radius of the minimum circumscribed circle of the right area is within the set range, it is used as the simulated right pupil contour, and its center or the center of the circle is used as the right eye pupil position.

[0103] Through the solution of the present application, several areas contained in the left eye pupil and the right eye pupil can be determined using mature image processing methods, and then the center of the pupil is determined as the pupil position using the center of the minimum circumscribed circle. The above method is used for the first left eye pupil position detection and the first right eye pupil position detection in the left eye view, and the second left eye pupil position detection and the second right eye pupil position detection in the right eye view. This solution has the advantages of small amount of calculation and can quickly determine the pupil position.

[0104] Further preferably, the set radius range is determined by the following method: obtaining the number of pixels contained in the left eye interest area corresponding to the left eye pupil and the right eye interest area corresponding to the right eye pupil; determining the radius base according to the focal length of the camera and the number of pixels; determining the set radius range according to the allowable error coefficient and the radius base. That is, the higher the camera resolution, the more pixels are contained in the captured image, and the smaller the distance between pixels is. The distance between pixels is determined, and then the number of rows and columns of pixels covered by the pupil is determined, and then the size of the pupil area can be determined. This method is simple to calculate and has high accuracy, which can improve the efficiency of pupil recognition.

[0105] Furthermore, in the above scheme S103, the left eye region and the right eye region are preprocessed to obtain a left eye interest region corresponding to the left eye pupil and a right eye interest region corresponding to the right eye pupil, such as Figure 5 As shown, the preprocessing includes:

[0106] S1031: Gaussian blur processing to obtain a Gaussian blurred image.

[0107] Gaussian blur processing is Gaussian smoothing, which is used to reduce image noise and reduce the level of detail. The Gaussian blur process of an image is to convolve the image with a normal distribution.

[0108] S1032: Binarize the Gaussian blurred image to obtain a binary image.

[0109] The binary conversion of an image is to set the grayscale value of the pixels on the image to 0 or 255, which means that the entire image will appear to be only black and white.

[0110] S1033: performing a morphological operation on the binary image, wherein the morphological operation includes first performing a closing operation and then performing an opening operation.

[0111] Opening and closing operations are basic morphological noise elimination modules in image processing, that is, after the opening and closing operations in this step, the edges can be smoothed and the noise can be reduced.

[0112] This solution can simplify the difficulty of image processing through Gaussian blur processing and binarization conversion. Finally, closing and opening operations are used to enhance image contrast and clarity, which can improve the accuracy of pupil recognition.

[0113] An embodiment of the present application provides a computer-readable storage medium, in which program information is stored. After a computer reads the program information, the computer executes the steps of the pupil distance detection method described in any one of the above method embodiments.

[0114] An embodiment of the present application provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the pupil distance detection method described in any one of the schemes in the above method embodiments are implemented.

[0115] The present application also provides an electronic device, such as Figure 6As shown, the electronic device includes at least one processor 61 and at least one memory 62, at least one of the memories 62 stores program information, and at least one of the processors 61 reads the program information and executes the pupil distance detection method described in any one of the above method embodiments. The device may also include: an input device 63 and an output device 64. The processor 61, the memory 62, the input device 63 and the output device 64 can be connected in communication. The memory 62, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor 61 executes various functional applications and data processing by running the non-volatile software programs, instructions and modules stored in the memory 62, that is, the pupil distance detection method provided by any of the above schemes is realized. The memory 62 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the pupil distance detection method, etc. In addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 62 may optionally include a memory remotely arranged relative to the processor 61, and these remote memories may be connected to a device for executing the pupil distance detection method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The input device 63 may receive input user clicks, and generate signal inputs related to user settings and function control of the pupil distance detection method. The output device 64 may include a display device such as a display screen. When the one or more modules are stored in the memory 62, when they are run by the one or more processors 61, the pupil distance detection method in any of the above-mentioned method embodiments is executed.

[0116] As needed, the above technical solutions can be combined to achieve the best technical effect.

[0117] The above are only the principles and preferred embodiments of the present application. It should be noted that, for ordinary technicians in this field, on the basis of the principles of the present application, several other modifications can be made, which should also be considered as the protection scope of the present application.

Claims

1. A method for detecting pupil distance, characterized in that: include: Acquire a left-view image, and analyze the left-view image to obtain a first left-eye pupil position and a first right-eye pupil position, wherein the first left-eye pupil position and the first right-eye pupil position are represented by pixel coordinates in a left-view image coordinate system; Acquire a right-view image, and analyze the right-view image to obtain a second left-eye pupil position and a second right-eye pupil position, wherein the second left-eye pupil position and the second right-eye pupil position are represented by pixel coordinates in a right-view image coordinate system; Calculating the parallax of the left-eye camera and the right-eye camera according to the distance between the first left-eye pupil position and the left imaging plane of the left-view image, the distance between the second left-eye pupil position and the left imaging plane of the right-view image, the distance between the first right-eye pupil position and the right imaging plane of the left-view image, and the distance between the second right-eye pupil position and the right imaging plane of the right-view image; Determine a left eye pupil depth and a right eye pupil depth according to the parallax and the camera focal length, wherein the left eye pupil depth and the right eye pupil depth respectively represent vertical distances between the left eye pupil and the right eye pupil and a straight line where the left eye camera and the right eye camera are located; Determine the left eye pupil three-dimensional coordinates of the left eye pupil in the camera coordinate system according to the left eye pupil depth and in combination with the first left eye pupil position and the second left eye pupil position; Determine the three-dimensional coordinates of the right eye pupil in the camera coordinate system according to the right eye pupil depth and in combination with the first right eye pupil position and the second right eye pupil position; According to the mapping relationship between the world coordinate system and the camera coordinate system, the three-dimensional coordinates of the left eye pupil are converted into the world coordinates of the left eye pupil, and the three-dimensional coordinates of the right eye pupil are converted into the world coordinates of the right eye pupil; The pupil distance is calculated according to the left eye pupil world coordinates and the right eye pupil world coordinates.

2. The pupil distance detection method according to claim 1, characterized in that: The acquiring of the left-view image, parsing the left-view image to obtain a first left-eye pupil position and a first right-eye pupil position, and the acquiring of the right-view image, parsing the right-view image to obtain a second left-eye pupil position and a second right-eye pupil position, include: Acquire a face image in an image captured by a camera, wherein the camera is a left camera and a right camera; Determine a left eye area and a right eye area in the face image; Preprocessing the left eye region and the right eye region to obtain a left eye region of interest corresponding to the left eye pupil and a right eye region of interest corresponding to the right eye pupil; Finding a plurality of left intra-pupil area contours in the left-eye region of interest and a plurality of right intra-pupil area contours in the right-eye region of interest; Fitting a minimum circumscribed circle of the left area to the contours of several left pupil areas, and if the radius of the minimum circumscribed circle of the left area is within a set range, taking the center of the minimum circumscribed circle of the left area as the left eye pupil position; when the camera is a left-eye camera, the left eye pupil position is the first left eye pupil position, and when the camera is a right-eye camera, the left eye pupil position is the second left eye pupil position; Fitting the right area minimum circumscribed circle of several right pupil area contours, if the radius of the right area minimum circumscribed circle is within the set radius range, the center of the right area minimum circumscribed circle is used as the right eye pupil position; when the camera is a left-eye camera, the right eye pupil position is the first right eye pupil position, when the camera is a right-eye camera, the right eye pupil position is the second right eye pupil position.

3. The pupil distance detection method according to claim 2, characterized in that: The left eye region and the right eye region are preprocessed to obtain a left eye region of interest corresponding to the left eye pupil and a right eye region of interest corresponding to the right eye pupil, wherein the preprocessing includes: Gaussian blur processing to obtain a Gaussian blurred image; Performing binarization conversion on the Gaussian blurred image to obtain a binarized image; A morphological operation is performed on the binary image, wherein the morphological operation includes first performing a closing operation and then performing an opening operation.

4. The pupil distance detection method according to claim 2, characterized in that: The set radius range is determined by: Get the number of pixels contained in the left eye region of interest corresponding to the left eye pupil and the right eye region of interest corresponding to the right eye pupil; Determine the radius base according to the focal length of the camera and the number of pixels; The set radius range is determined according to the allowable error coefficient and the radius base.

5. The pupil distance detection method according to any one of claims 1 to 4, characterized in that: The parallax between the left eye camera and the right eye camera is calculated according to the distance between the first left eye pupil position and the left imaging plane of the left view image, the distance between the second left eye pupil position and the left imaging plane of the right view image, the distance between the first right eye pupil position and the right imaging plane of the left view image, and the distance between the second right eye pupil position and the right imaging plane of the right view image: The distance between the first left eye pupil position and the left imaging plane of the left view image is represented by X LL , the distance between the second left eye pupil position and the left imaging plane of the right-view image is expressed as X RL ; The parallax d of the binocular camera for the left pupil L It is calculated as follows: d L =|X LL -X RL |; The distance between the second left eye pupil position and the right imaging plane of the left view image is expressed as X LR , the distance between the second left eye pupil position and the right imaging plane of the right-view image is expressed as X RR ; The parallax d of the binocular camera for the right pupil R It is calculated as follows: d R =|X LR -X RR |; According to the parallax and the focal length of the camera, the left eye pupil depth and the right eye pupil depth are determined: the left eye pupil depth is: Z L =b×f / d L , the right eye pupil depth is: Z R =b×f / d R , where b represents the distance between the center points of the left camera and the right camera, and f represents the focal length of the camera.

6. The pupil distance detection method according to claim 5, characterized in that: Determining the left eye pupil three-dimensional coordinates of the left eye pupil in the camera coordinate system according to the left eye pupil depth and in combination with the first left eye pupil position and the second left eye pupil position; According to the right eye pupil depth, combined with the first right eye pupil position and the second right eye pupil position, determining the right eye pupil three-dimensional coordinates in the camera coordinate system, including: Obtain a first connection line between the projection center point of the left camera and the first left eye pupil position, and obtain a second connection line between the projection center point of the right camera and the second left eye pupil position; The intersection point of the extended lines of the first connecting line and the second connecting line is the position of the left eye pupil in the camera coordinate system, and the three-dimensional coordinates of the left eye pupil are determined based on the intersection point; Obtain a third connecting line between the projection center point of the right camera and the first right eye pupil position, and obtain a fourth connecting line between the projection center point of the right camera and the second right eye pupil position; The intersection point of the extended lines of the third connecting line and the fourth connecting line is the position of the right eye pupil in the camera coordinate system, and the three-dimensional coordinates of the right eye pupil are determined based on this.

7. The pupil distance detection method according to claim 6, characterized in that: The step of converting the left eye pupil three-dimensional coordinates into the left eye pupil world coordinates and converting the right eye pupil three-dimensional coordinates into the right eye pupil world coordinates according to the mapping relationship between the world coordinate system and the camera coordinate system comprises: The mapping relationship between the world coordinate system and the camera coordinate system is represented by the following transformation matrix: Among them, (X w , Y w , Z w ) is the coordinate representation in the world coordinate system, (X c , Y c , Z c ) is the coordinate representation in the camera coordinate system, R represents the rotation matrix, and T represents the translation matrix.

8. A computer-readable storage medium, characterized in that: The storage medium stores program information, and the computer reads the program information and executes the steps of the pupil distance detection method according to any one of claims 1 to 7.

9. A computer program product, characterized in that The method comprises a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the pupil distance detection method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the pupil distance detection method according to any one of claims 1 to 7.

Images