A method of pupil positioning and related apparatus

Abstract

The application discloses a pupil positioning method and a related device, specifically, a first pupil image and parameter information of a left camera and a right camera are acquired; the first pupil image is corrected based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction; coordinate information of a center point of the second pupil image is acquired; and the center point of the second pupil image is adjusted to a center position of the left camera and the right camera and is in the same plane with the left camera and the right camera based on the coordinate information of the center point of the second pupil image. In the pupil positioning method provided by the application, manual operation of an ophthalmic detection device to align the pupil is no longer needed, the shortcomings of slow speed and poor accuracy caused by manual operation of the ophthalmic detection device to align the pupil are solved, the processing speed and the processing accuracy are greatly improved, and the pupil can be accurately aligned every time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a pupil positioning method and related device. BACKGROUND

[0002] At present, with the rapid development of image processing technology, image processing technology begins to be widely applied in medical instruments. Among them, the most important thing for ophthalmic detection equipment when checking the eyes of the measured person is to control the ophthalmic detection equipment to align with the pupil of the measured person, and also to ensure that it can accurately align with the pupil of the measured person every time.

[0003] However, when the traditional ophthalmic detection equipment checks the eyes of the measured person, it usually manually controls the ophthalmic detection equipment to align with the pupil of the measured person. Since the eyes of the measured person are difficult to keep still for a long time, the manual operation of the ophthalmic detection equipment to align with the pupil of the measured person is slow and inefficient, and the manual operation of the ophthalmic detection equipment cannot ensure that it can accurately align with the pupil of the measured person every time, which greatly reduces the detection accuracy. SUMMARY

[0004] In view of the above problems, the present application provides a pupil positioning method and related device to realize efficient, accurate and rapid adjustment of the first pupil of the measured person to the M point, ensure the consistency of the distance and angle of each alignment of the first pupil, and ensure the consistency of the shooting result. The specific scheme is as follows:

[0005] A pupil positioning method, comprising:

[0006] obtaining a first pupil image and parameter information of a left camera and a right camera;

[0007] performing image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction;

[0008] obtaining coordinate information of a center point of the second pupil image;

[0009] adjusting the center point of the second pupil image to the center position of the left camera and the right camera and to the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image.

[0010] Optionally, the parameter information of the left camera and the right camera includes intrinsic matrix, distortion parameters of the left camera and the right camera, and relative rotation matrix and translation matrix of the left camera and the right camera.

[0011] Optionally, performing image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction comprises:

[0012] Based on the parameter information of the left camera and the right camera, a rectification rotation matrix, a projection matrix and a re-projection matrix are obtained through a stereo rectification function in OpenCV;

[0013] A calibration mapping parameter is obtained through a distortion rectification function in OpenCV.

[0014] Finally, the first pupil image obtained by the left camera and the right camera is calibrated through a re-mapping function in OpenCV to obtain a second pupil image after rectification.

[0015] Optionally, the coordinate information of the center point of the second pupil image comprises:

[0016] The target recognition algorithm is used to perform target recognition on the second pupil image to obtain coordinate information of the center point of the second pupil image in the left camera and the right camera.

[0017] Optionally, based on the coordinate information of the center point of the second pupil image, the center point of the second pupil image is adjusted to the center position of the left camera and the right camera and is in the same plane as the left camera and the right camera, comprising:

[0018] Coordinate information of the center position of the left camera and the right camera and height coordinate information of the same plane in which the left camera and the right camera are located are obtained.

[0019] Coordinate information of the center point of the second pupil image and height coordinate information of the plane in which the second pupil image is located are obtained.

[0020] Based on the coordinate information of the center position of the left camera and the right camera and the coordinate information of the center point of the second pupil image, a first difference value is calculated.

[0021] Based on the height coordinate information of the same plane in which the left camera and the right camera are located and the height coordinate information of the plane in which the second pupil image is located, a second difference value is calculated.

[0022] According to the first difference value and the second difference value, the center point of the second pupil image is adjusted to the center position of the left camera and the right camera and is in the same plane as the left camera and the right camera.

[0023] A pupil positioning device comprises:

[0024] A first acquisition unit is configured to acquire a first pupil image and parameter information of a left camera and a right camera.

[0025] A rectification unit is configured to perform image rectification on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after rectification.

[0026] A second acquisition unit is configured to acquire coordinate information of a center point of the second pupil image.

[0027] The adjusting unit adjusts the center point of the second pupil image to the center positions of the left camera and the right camera and to the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image.

[0028] Optionally, the correcting unit is specifically used for:

[0029] The correcting unit obtains a rectified rotation matrix, a projection matrix and a re-projection matrix through a stereo rectification function in OpenCV based on the parameter information of the left camera and the right camera.

[0030] The correcting unit further obtains a calibration mapping parameter through a distortion rectification function in OpenCV.

[0031] Finally, the correcting unit calibrates the first pupil image obtained by the left camera and the right camera through a remapping function in OpenCV to obtain a rectified second pupil image.

[0032] Optionally, the adjusting unit is specifically used for:

[0033] The adjusting unit obtains coordinate information of the center positions of the left camera and the right camera and height coordinate information of the same plane where the left camera and the right camera are located.

[0034] The adjusting unit obtains coordinate information of the center point of the second pupil image and height coordinate information of the plane where the second pupil image is located.

[0035] The adjusting unit calculates a first difference value based on the coordinate information of the center positions of the left camera and the right camera and the coordinate information of the center point of the second pupil image.

[0036] The adjusting unit calculates a second difference value based on the height coordinate information of the same plane where the left camera and the right camera are located and the height coordinate information of the plane where the center point of the second pupil image is located.

[0037] The adjusting unit adjusts the center point of the second pupil image to the center positions of the left camera and the right camera and to the same plane as the left camera and the right camera according to the first difference value and the second difference value.

[0038] An electronic device comprising at least one processor and a memory connected to the processor, wherein:

[0039] The memory is used to store computer programs or instructions.

[0040] The processor is used to execute the computer programs or instructions to enable the electronic device to implement the pupil positioning method as described above.

[0041] A computer storage medium, the storage medium carries one or more computer program instructions, when one or more computer programs are executed by an electronic device, the electronic device can realize the pupil positioning method as described above.

[0042] By the above technical solution, in the pupil positioning method provided by the application, the first pupil image and the parameter information of the left camera and the right camera are obtained; the first pupil image is corrected based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction; the coordinate information of the center point of the second pupil image is obtained; and the center point of the second pupil image is adjusted to the center position of the left camera and the right camera and is in the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image. In the pupil positioning method provided by the application, manual operation of the ophthalmic detection device to align the pupil is no longer required, and the slow speed and poor accuracy caused by manual operation of the ophthalmic detection device to align the pupil are solved, the processing speed and accuracy are greatly improved, and the pupil can be accurately aligned each time. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0044] Figure 1 A flowchart of a pupil positioning method provided by an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the left camera and the right camera correcting the first pupil image provided by an embodiment of the present application;

[0046] Figure 3 A principle schematic diagram of a pupil positioning method provided by an embodiment of the present application;

[0047] Figure 4 A schematic diagram of adjusting the position of the center point of the second pupil image provided by an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a pupil positioning device provided by an embodiment of the present application;

[0049] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described in the specification of the present application combined with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to be limiting to the present application. As used in the specification and appended claims of the present application, the singular forms “a,” “an,” and “the” are intended to include the plural forms, e.g., “one or more,” unless the context clearly indicates otherwise.

[0051] In the present specification, the phrase “one embodiment” or “some embodiments” etc. means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the phrases “in one embodiment”, “in some embodiments”, “in other some embodiments”, “in further some embodiments” etc. appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean “one or more but not all embodiments”, unless otherwise specifically emphasized. The terms “comprise”, “include”, “have” and their conjugates mean “including but not limited to”, unless otherwise specifically emphasized.

[0052] The plurality referred to in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the terms “first”, “second” etc. are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0053] The conventional ophthalmic detection equipment needs to perform accurate three-axis positioning and alignment operations of the X-axis, Y-axis and Z-axis of the pupil of the measured person when in use. However, most of the ophthalmic detection equipment on the market needs manual operation of the detection equipment to align the pupil of the measured person. Since the eyes of the measured person are difficult to keep still for a long time, manual operation of the detection equipment to align the pupil of the measured person has the disadvantages of slow alignment speed and low alignment accuracy, and manual operation of the detection equipment has large measurement error and cannot guarantee accurate alignment of the pupil every time.

[0054] In order to solve the above problems, the present application provides a pupil positioning method and related device, which will be described in detail in combination with specific embodiments.

[0055] Embodiment one

[0056] Figure 1 A flowchart of a pupil positioning method provided by the embodiments of the present application is shown.

[0057] As shown in Figure 1 A pupil positioning method provided by the embodiments of the present application specifically includes the following steps:

[0058] S10. Obtain the first pupil image and the parameter information of the left and right cameras.

[0059] The detection device with a pupil localization method provided in this application embodiment is equipped with two cameras, a left camera and a right camera. The left and right cameras are used to capture images of the subject's pupils to obtain pupil images. Since the subject has two pupils, the pupil localization method provided in this application embodiment processes the subject's left and right pupils separately. Specifically, the left and right cameras in the detection device with a pupil localization method provided in this application embodiment first acquire the subject's left pupil image, and after processing the subject's left pupil image, they continue to acquire the subject's right pupil image. The process of processing the left pupil image is the same as the process of processing the right pupil image.

[0060] It should be noted that the detection device with pupil localization method provided in this application embodiment is also equipped with an illumination module, which can use a 940nm infrared LED for illumination. Furthermore, the detection device with pupil localization method provided in this application embodiment can collect parameter information from the left and right cameras. Specifically, the parameter information of the left and right cameras may include: intrinsic parameter matrices of the left and right cameras, distortion parameters, and rotation and translation matrices of the right camera relative to the left camera, etc.

[0061] S20. Based on the parameter information of the left and right cameras, perform image correction on the first pupil image to obtain the corrected second pupil image.

[0062] In this embodiment of the application, the first pupil image obtained in step S10 is corrected based on the camera parameter information obtained in step S10 to obtain the corrected second pupil image.

[0063] It should be noted that the first pupil image captured by the left and right cameras in the detection device with pupil localization method provided in this application embodiment can be either the left pupil image or the right pupil image of the subject. Because the left and right cameras in the detection device with pupil localization method provided in this application embodiment have different positions, the first pupil images obtained when the left and right cameras capture images of the subject's left pupil often differ. Therefore, image correction is required for the first pupil images captured by the left and right cameras to obtain a corrected second pupil image.

[0064] like Figure 2 As shown, Figure 2 The image shown is a corrected second pupil image obtained by correcting the first pupil image using the left and right cameras provided in an embodiment of this application.

[0065] Figure 2 Fig. 1 (a) shows a first pupil image captured by the left camera, Figure 2 Fig. 1 (b) shows a first pupil image captured by the right camera, Figure 2 Fig. 1 (c) shows a second pupil image captured by the left camera after image rectification, Figure 2 Fig. 1 (d) shows a second pupil image captured by the right camera after image rectification.

[0066] By comparing Figure 2 Fig. 1 (a) and (c), it can be seen that the pupil in the first pupil image of Fig. 1 (a) is elliptical, while the pupil in the second pupil image of Fig. 1 (c) is circular. Figure 2 The pupil in the first pupil image of Fig. 1 (a) is elliptical, while the pupil in the second pupil image of Fig. 1 (c) is circular. Figure 2 The pupil in the first pupil image of Fig. 1 (a) is elliptical, while the pupil in the second pupil image of Fig. 1 (c) is circular. Figure 2 By comparing Figure 2 Fig. 1 (b) and (d), it can be seen that the pupil in the first pupil image of Fig. 1 (b) is elliptical, while the pupil in the second pupil image of Fig. 1 (d) is circular. Figure 2 The pupil in the first pupil image of Fig. 1 (b) is elliptical, while the pupil in the second pupil image of Fig. 1 (d) is circular.

[0067] Figure 2 Fig. 1 (e) shows the second pupil images captured by the left and right cameras after image rectification, which are in the same plane and have horizontally aligned pupils.

[0068] Specifically, based on the intrinsic matrix, distortion parameters, and rotation matrix and translation matrix of the right camera relative to the left camera of the left and right cameras, a rectification rotation matrix R, a projection matrix P, and a re-projection matrix Q are obtained through a stereo rectification function in OpenCV, a calibration mapping parameter is obtained through a distortion rectification function in OpenCV, and finally the first pupil images captured by the left and right cameras are calibrated through a re-mapping function in OpenCV to obtain the second pupil images in the same plane in the left and right cameras.

[0069] S30, obtaining coordinate information of a center point of the second pupil image.

[0070] Specifically, the target recognition algorithm is used to perform target recognition on the second pupil image obtained after image rectification, and the coordinate information of the second pupil image in the left and right cameras can be obtained.

[0071] Figure 3 Fig. 2 shows a principle diagram of pupil positioning provided by an embodiment of the present application.

[0072] From Figure 3As can be seen, the distance between the left camera and the right camera is T, and the vertical distance of the left camera and the right camera from the X axis is denoted as f.

[0073] Suppose that P is an arbitrary point in the coordinate system, and the coordinates of P are P (X, Y, Z), wherein the coordinate of P in the X axis direction relative to the left camera is denoted as x l , and the coordinate of P in the X axis direction relative to the right camera is denoted as x r According to the similar triangle theorem, the following formula can be obtained,

[0074]

[0075]

[0076]

[0077]

[0078] From formula 1 and formula 2, it can be obtained that,

[0079]

[0080] From formula 3 and formula 5, it can be obtained that,

[0081]

[0082] From formula 1 and formula 5, it can be obtained that,

[0083]

[0084] In order to simplify the expression of the above formula, another parallax d = x l -x r is introduced into formula 5, formula 6 and formula 7, and the following can be obtained,

[0085]

[0086]

[0087]

[0088] Thus, the coordinate information of P can be obtained, and it should be noted that, since P is an arbitrary point in the coordinate system, the coordinate information of P is only used for formula derivation, that is, the specific coordinate information of an arbitrary coordinate point in the coordinate system can be obtained through formula derivation.

[0089] S40, the center point of the second pupil image is adjusted to the center position of the left camera and the right camera and is in the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image.

[0090] In the actual testing process, based on the above step S30, the actual test coordinates of the second pupil image can be obtained as P(X). 测 Y 测 ), where the coordinates of point P relative to the left camera are (X... l Y l The coordinates of point P relative to the right camera are (X... r Y r Then, from formulas 8 and 9, we can obtain...

[0091]

[0092]

[0093] Assume the coordinates of point M are M(X) 目 Y 目 ),from Figure 3 As can be seen, point M is located at the center of the left and right cameras, and from the Y-axis direction, point M is on the same plane as the left and right cameras. That is, the Y-axis coordinate of point M is the same as the Y-axis coordinate of the plane containing the left and right cameras, meaning point M is on the same plane as the plane containing the left and right cameras. The Z-axis direction can be understood as the depth information of the image. Specifically, the coordinates of point M relative to the left camera are (X... t Y t Therefore, we can obtain the following from formulas 8 and 9:

[0094]

[0095]

[0096] Based on the actual test coordinates of the center point of the second pupil image, P(X) 测 Y 测 The coordinates of point M are M(X) and M(X). 目 Y 目 This allows us to determine the distance the center point of the second pupil image needs to move along the X-axis.

[0097] See Figure 4 , Figure 4 This illustration shows a schematic diagram of adjusting the position of the center point of the second pupil image according to an embodiment of this application.

[0098] like Figure 4 As shown in the figure, the circle represents the imaging plane of the second pupil image, the horizontal dashed line represents the Y-axis direction, and the vertical dashed line represents the X-axis direction. The Y-axis direction is... Figure 3The information in the middle can be understood as height coordinates. The intersection of the dashed lines in the middle of each circle represents the position coordinates of the aforementioned point M. The "+" in the figure represents the position information of the center point of the second pupil image. The pupil positioning method provided in this application embodiment controls the three-axis drive module mounted on the ophthalmic detection device. The three-axis drive module can control the movement of the X-axis, Y-axis, and Z-axis, so that the position of the center point of the second pupil image is finally moved to the position of point M.

[0099] It should be noted that the pupil positioning method provided in this application does not control the movement of the center point of the second pupil image, but rather controls the movement of the X-axis, Y-axis and Z-axis through a three-axis drive module, thereby moving the position of point M to the position of the center point of the second pupil image, thus achieving accurate alignment with the subject's pupil every time.

[0100] Specifically, the distance that needs to be moved along the X-axis is defined as X. 移 Then X 移 =X 测 -X 目 You can get

[0101]

[0102] Since the center point of the second pupil image may be located to the left or right of point M, it is necessary to determine the direction of movement of the X-axis based on the relative position of the center point of the second pupil image and point M.

[0103] like Figure 4 As shown in Figures c, f, and i, point M is located to the left of the center point of the second pupil image, i.e., X. l >X t Therefore, it is necessary to control the X-axis movement to the right in the three-axis drive module.

[0104]

[0105] like Figure 4 As shown in figures b, e, and h, point M and the center point of the second pupil image are located on the X-axis, i.e., X... l =X t Therefore, the X-axis in the three-axis drive module does not need to move, i.e., X 移 =0.

[0106] like Figure 4 As shown in figures a, d, and g, point M is located to the right of the center point of the second pupil image, i.e., X. l <X t X 移 <0, therefore it is necessary to control the X-axis movement to the left in the three-axis drive module.

[0107]

[0108] Similarly, define the distance that needs to be moved in the Y-axis direction as Y. 移 Then Y 移 =Y 测 -Y 目 You can get

[0109]

[0110] Since the center point of the second pupil image may be located above or below point M, the direction of movement of the Y-axis needs to be determined based on the relative position of the center point of the second pupil image and point M.

[0111] like Figure 4 As shown in figures a, b, and c, point M is located below the center point of the second pupil image, i.e., Y. l >Y t Therefore, it is necessary to control the upward movement of the Y-axis in the three-axis drive module.

[0112]

[0113] like Figure 4 As shown in figures d, e, and f, the center point of the second pupil image is located on the Y-axis at point M, i.e., Y... l =Y t Therefore, the Y-axis drive in the three-axis drive module does not need to move, i.e., Y... 移 =0.

[0114] like Figure 4 As shown in the g, h, and i diagrams, point M is located above the center point of the second pupil image, i.e., Y. l <Y t Therefore, it is necessary to control the downward movement of the Y-axis in the three-axis drive module.

[0115]

[0116] It should be noted that in the pupil positioning method provided in this application embodiment, the X-axis direction can be understood as the coordinate information of the horizontal extension of the left and right cameras, the Y-axis direction can be understood as the height information of the plane where the left and right cameras are located, and the Z-axis direction can be understood as the depth information of the image. Therefore, to move the center point of the second pupil image to point M, the movement is mainly in the X-axis and Y-axis directions.

[0117] It should be noted that, since the Z-axis direction represents the depth information of the center point of the second pupil image, as can be seen from Formula 7, the depth information of the center point of the second pupil image can be calculated based on the coordinate information of the center point of the second pupil image. Therefore, the second pupil image can be moved to the depth information required in this embodiment based on the calculated depth information of the center point of the second pupil image.

[0118] Embodiment Two

[0119] Figure 5 A schematic diagram of a pupil positioning device provided by an embodiment of the present application is shown.

[0120] As shown in Figure 5 , the pupil positioning device comprises:

[0121] A first acquisition unit 501 configured to acquire a first pupil image and parameter information of a left camera and a right camera;

[0122] A correction unit 502 configured to perform image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction;

[0123] A second acquisition unit 503 configured to acquire coordinate information of a center point of the second pupil image;

[0124] An adjustment unit 504 configured to adjust the center point of the second pupil image to a center position of the left camera and the right camera and to the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image.

[0125] The pupil positioning device provided by the embodiment of the present application acquires the first pupil image and the parameter information of the left camera and the right camera, performs image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain the second pupil image after correction, acquires the coordinate information of the center point of the second pupil image, and adjusts the center point of the second pupil image to the center position of the left camera and the right camera and to the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image. In the pupil positioning method provided by the present application, manual operation of the ophthalmic detection device to align the pupil is no longer required, thereby solving the shortcomings of slow speed and poor accuracy caused by manual operation of the ophthalmic detection device to align the pupil, greatly improving the processing speed and accuracy, and ensuring accurate alignment of the pupil each time.

[0126] In an embodiment of the present application, based on the foregoing scheme, the correction unit 502 is specifically configured to:

[0127] based on the parameter information of the left camera and the right camera, obtain a correction rotation matrix, a projection matrix, and a re-projection matrix through a stereo correction function in OpenCV;

[0128] then obtain a calibration mapping parameter through a distortion correction function in OpenCV;

[0129] finally, calibrate the first pupil image acquired by the left camera and the right camera through a re-mapping function in OpenCV to obtain the second pupil image after correction.

[0130] In one embodiment of the present application, based on the foregoing scheme, the adjusting unit 504 is specifically configured to:

[0131] obtain coordinate information of the center positions of the left and right cameras and height coordinate information of the same plane where the left and right cameras are located;

[0132] obtain coordinate information of the center point of the second pupil image and height coordinate information of the plane where the second pupil image is located;

[0133] based on the coordinate information of the center positions of the left and right cameras and the coordinate information of the center point of the second pupil image, a first difference value is calculated;

[0134] based on the height coordinate information of the same plane where the left and right cameras are located and the height coordinate information of the plane where the center point of the second pupil image is located, a second difference value is calculated;

[0135] adjust the center point of the second pupil image to the center positions of the left and right cameras and to the same plane where the left and right cameras are located according to the first difference value and the second difference value.

[0136] Embodiment Three

[0137] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0138] Reference Figure 6 As shown in the figure, a structural schematic diagram suitable for implementing an electronic device in an embodiment of the present application is shown. The electronic device in the embodiment of the present application can include, but is not limited to, a fixed terminal such as a notebook computer, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a desktop computer, and the like. Figure 6 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0139] As Figure 6 shown, the electronic device can include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. In the state that the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0140] Generally, the following devices can be connected to the I / O interface 605: input device(s) 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output device(s) 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage device(s) 608 including, for example, a memory card, a hard disk, and the like; and communication device(s) 609. The communication device(s) 609 can allow the electronic device to communicate wirelessly or through a wired connection with other devices to exchange data. Although Figure 6 Electronic devices having various devices are illustrated, but it is understood that all of the illustrated devices are not required. More or fewer devices can alternatively be implemented.

[0141] Embodiment Four

[0142] The embodiment of the present application provides a computer readable storage medium applied to an electronic device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device can implement the pupil positioning method in the embodiment one.

[0143] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transmit a program for use by or in connection with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, an optical fiber, an RF (radio frequency) or the like, or any suitable combination of the above.

[0144] Although preferred embodiments of the application have been described, those skilled in the art will recognize that additional modifications and changes can be made thereto without departing from the scope of the application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the application.

[0145] Finally, it should be noted that, in the description of the application, relative terms such as first and second, and the like, can be used solely to distinguish one from another without necessarily implying any actual relationship or order between or among such entities or operations. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional elements of that element that contain the same function.

Claims

1. A method of pupil positioning, characterized in that, The method comprises the following steps: acquiring a first pupil image and parameter information of a left camera and a right camera; performing image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction; acquiring coordinate information of a center point of the second pupil image; adjusting the center point of the second pupil image to the center positions of the left camera and the right camera and to the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image, which comprises acquiring coordinate information of the center positions of the left camera and the right camera and height coordinate information of the same plane where the left camera and the right camera are located; acquiring the coordinate information of the center point of the second pupil image and the height coordinate information of the plane where the center point of the second pupil image is located; calculating a first difference value based on the coordinate information of the center positions of the left camera and the right camera and the coordinate information of the center point of the second pupil image; calculating a second difference value based on the height coordinate information of the same plane where the left camera and the right camera are located and the height coordinate information of the plane where the center point of the second pupil image is located; and adjusting the center point of the second pupil image to the center positions of the left camera and the right camera and to the same plane as the left camera and the right camera according to the first difference value and the second difference value.

2. The method of pupil localization according to claim 1, wherein, The parameter information of the left camera and the right camera comprises intrinsic matrix, distortion parameters of the left camera and the right camera, and relative rotation matrix and translation matrix of the left camera and the right camera.

3. The method of pupil localization according to claim 1, wherein, The method of performing image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction comprises the following steps: obtaining a rectification rotation matrix, a projection matrix and a re-projection matrix through a stereo rectification function in OpenCV based on the parameter information of the left camera and the right camera; obtaining calibration mapping parameters through a distortion rectification function in OpenCV; and finally calibrating the first pupil image acquired by the left camera and the right camera through a re-mapping function in OpenCV to obtain a second pupil image after correction.

4. The method of pupil localization according to claim 1, wherein, The method of acquiring coordinate information of the center point of the second pupil image comprises the following steps: performing target recognition on the second pupil image based on a target recognition algorithm to obtain coordinate information of the center point of the second pupil image in the left camera and the right camera respectively.

5. An apparatus for pupil positioning, characterized by The method comprises the following steps: a first acquisition unit is configured to acquire a first pupil image and parameter information of a left camera and a right camera; a correction unit is configured to perform image correction on the first pupil image based on the parameter information of the left camera and the right camera to obtain a second pupil image after correction; a second acquisition unit is configured to acquire coordinate information of a center point of the second pupil image; an adjustment unit is configured to adjust the center point of the second pupil image to the center positions of the left camera and the right camera and to the same plane as the left camera and the right camera based on the coordinate information of the center point of the second pupil image. The adjusting unit is specifically used for: obtaining coordinate information of center positions of the left camera and the right camera and height coordinate information of the same plane where the left camera and the right camera are located; obtaining coordinate information of a center point of the second pupil image and height coordinate information of a plane where the center point of the second pupil image is located; calculating a first difference value based on the coordinate information of the center positions of the left camera and the right camera and the coordinate information of the center point of the second pupil image; calculating a second difference value based on the height coordinate information of the same plane where the left camera and the right camera are located and the height coordinate information of the plane where the center point of the second pupil image is located; and adjusting the center point of the second pupil image to the center positions of the left camera and the right camera and to the same plane as the left camera and the right camera according to the first difference value and the second difference value.

6. The pupil localization device of claim 5, wherein, The correcting unit is specifically used for: obtaining a correction rotation matrix, a projection matrix and a re-projection matrix based on the parameter information of the left camera and the right camera through a stereo correction function in OpenCV; obtaining calibration mapping parameters through a distortion correction function in OpenCV; and finally calibrating the first pupil image obtained by the left camera and the right camera through a re-mapping function in OpenCV to obtain a corrected second pupil image.

7. An electronic device, comprising: The electronic device comprises at least one processor and a memory connected with the processor, wherein: the memory is used for storing computer programs or instructions; the processor is used for executing the computer programs or instructions, so that the electronic device can implement the pupil positioning method in any one of claims 1 to 4.

8. A computer storage medium, characterized in that The storage medium carries one or more computer program instructions, which can enable the electronic device to implement the pupil positioning method in any one of claims 1 to 4 when the one or more computer programs are executed by the electronic device.

Images