Pupil diameter measuring method and system based on infrared binocular camera, equipment and medium

Abstract

The invention provides a pupil diameter measuring method and system based on an infrared binocular camera, equipment and a medium. The method comprises the following steps: acquiring human eye images shot by the infrared binocular camera at the same time; matching the position difference of pupils from the shot human eye image; and the pupil diameter is calculated through a binocular stereo imaging relation. Through binocular 4K infrared imaging, the position difference of the pupils is matched from the left image and the right image through the neural network model, and the diameter of the pupils is directly calculated through the binocular stereo imaging relation. Compared with a traditional monocular camera, binocular imaging can adapt to distance-variable tests, and a tester does not need to be close to a special eyeshade of equipment and the like and can measure the diameter of the pupil only by looking at the camera equipment within an effective distance at a glance.

Description

Technical Field

[0001] The present invention relates to the technical field of pupil diameter measurement, and in particular to a pupil diameter measurement method based on an infrared binocular camera, a system, a device and a medium. Background Art

[0002] The sensitivity of the human eye to light changes can be used to determine a person's mental state, vital signs, etc. The sensitivity of the human eye to light can be determined by the changes in the pupil to light. The diameter of the pupil can be measured through the image. Conventional measurement equipment is similar to an industrial camera. It takes a picture of the human eye at a fixed distance and calculates the size of the pupil by the ratio of the shooting distance and focal length and the area of ​​the pupil in the image. In this case, when the distance between the human eye and the camera changes, it cannot be measured accurately, resulting in misjudgment. Summary of the invention

[0003] In order to achieve the above-mentioned purpose and other advantages of the present invention, the first purpose of the present invention is to provide a pupil diameter measurement method based on an infrared binocular camera, comprising the following steps:

[0004] Acquire human eye images taken simultaneously by infrared binocular cameras;

[0005] Match the position difference of the pupil from the captured human eye image;

[0006] The pupil diameter is calculated through binocular stereo imaging relationship.

[0007] Furthermore, the step of matching the position difference of the pupil from the captured human eye image includes:

[0008] Find the pupil image from the captured human eye image;

[0009] Fitting a pupil circle in the pupil image;

[0010] Based on the fitted pupil circle, the number of diameter pixels of the pupil imaged in the image and the number of pixels from the center of the pupil fitting circle to the center axis of the left and right cameras are calculated.

[0011] Furthermore, the step of finding the pupil image from the captured human eye image includes:

[0012] The pupil image is found from the captured human eye image using the YOLO neural network model.

[0013] Furthermore, the calculation formula of the pupil diameter is:

[0014]

[0015] Among them, D is the pupil diameter, C is the number of diameter pixels of the pupil imaged in the image, the calculation of C selects the number of pixels occupied by the pupil in the left or right camera or takes the average number of pixels occupied by the left and right cameras, B is the lateral distance between the centers of the two camera lenses, N1 is the number of pixels horizontally from the left lens center axis at the imaging point P1 of the left camera, N2 is the number of pixels horizontally from the right lens center axis at the imaging point P2 of the right camera, P1 and P2 are the images of the pupil center point P on the sensor target surfaces of the left and right cameras respectively.

[0016] The second object of the present invention is to provide a pupil diameter measurement system based on an infrared binocular camera to implement the above method, comprising two infrared illuminators, two cameras, a fixed bracket, and a main controller, wherein the infrared illuminators and the cameras are installed on the fixed bracket, the two infrared illuminators are arranged on both sides of the two cameras, the two cameras are used to simultaneously capture human eye images, and the main controller is used to match the position difference of the pupil from the captured human eye images, and calculate the pupil diameter through the binocular stereo imaging relationship.

[0017] Furthermore, the infrared illuminator adopts an infrared illuminator with a wavelength of 940nm.

[0018] Furthermore, the diameter of the infrared illuminator is 4 mm, and the diameter of the camera is 14 mm.

[0019] Furthermore, the lateral distance between the centers of the two camera lenses is 28 mm.

[0020] A third object of the present invention is to provide a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0021] A fourth object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0023] The present invention uses binocular 4K infrared imaging, matches the pupil position difference from the left and right images through a neural network model, and directly calculates the pupil diameter through the binocular stereo imaging relationship. Compared with traditional monocular cameras, binocular imaging can adapt to tests with variable distances. The tester does not need to get close to the device's special eye mask, etc., and only needs to look directly at the camera device within an effective distance to measure the pupil diameter.

[0024] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation of the present invention is given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 Schematic diagram of pupil diameter measurement system based on infrared binocular camera;

[0027] Figure 2 This is the principle diagram of pupil diameter measurement based on infrared binocular camera;

[0028] Figure 3 This is a flow chart of the pupil diameter measurement method based on an infrared binocular camera;

[0029] Figure 4 A flow chart for matching the position difference of the pupil from the captured human eye image;

[0030] Figure 5 It is a schematic diagram of computer equipment;

[0031] Figure 6 A schematic diagram of a computer-readable storage medium. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. It should be noted that, under the premise of no conflict, the embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0033] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0034] The figure numbers in this application are only used to distinguish the various steps in the scheme, and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0036] Traditional devices for measuring pupil diameter usually require a fixed distance between the eye and the camera. When the distance changes, the measurement cannot be accurate. The tester must be very cooperative and familiar with the test process to be able to measure accurately.

[0037] The infrared binocular pupil diameter measurement system provided by the present invention does not need to fix the distance between the human eye and the camera. As long as the camera is facing the eye within the effective range, the pupil diameter can be accurately measured. Compared with traditional measuring equipment, it is quite convenient to use. The camera can also be installed in equipment with different eye mask distances without adaptation. The production process does not need to strictly control the distance between the eye mask and the camera.

[0038] Example 1

[0039] A pupil diameter measurement system based on infrared binocular camera, such as Figure 1 As shown, it includes two infrared illuminators, two cameras, a fixed bracket, and a main controller. The infrared illuminators and the cameras are installed on the fixed bracket. The two infrared illuminators are arranged on both sides of the two cameras. The two cameras are used to simultaneously capture human eye images. The main controller is used to match the position difference of the pupil from the captured human eye images and calculate the pupil diameter through the binocular stereo imaging relationship.

[0040] The two infrared illuminators are Figure 1 The first infrared illuminator 1 and the second infrared illuminator 4 are respectively infrared illuminators with a wavelength of 940 nm, and r is the diameter of the infrared illuminator. Furthermore, the diameter of the infrared illuminator is 4 mm.

[0041] The two cameras are Figure 1 In the first camera 2 and the second camera 3, R is the diameter of the lens, and B is the distance between the centers of the camera lenses. Further, the diameter of the camera is 14 mm; the lateral distance between the centers of the two camera lenses is 28 mm.

[0042] Optionally, Figure 1 The fixing bracket 5 in the embodiment adopts a black metal fixing bracket, and the camera and the infrared lamp are both installed on the metal fixing bracket.

[0043] The measurement principle of the pupil diameter measurement system based on infrared binocular camera is as follows: Figure 2 shown. Figure 2In the figure, P is the test point, P1 and P2 are the images of the pupil center point P on the target surfaces of the left and right camera sensors respectively, X1 is the distance from P1 to the central axis of the left camera lens, X2 is the distance from P2 to the central axis of the right camera lens, F is the focal length of the camera, L is the longitudinal distance from P to the center of the lens, A1 is the lateral distance from P to the central axis of the first lens, A2 is the lateral distance from P to the central axis of the second lens, and B is the lateral distance between the centers of the two camera lenses.

[0044]

[0045] Assume that the length and width of the camera sensor pixel are U, the number of pixels horizontally from P1 to the central axis of the left lens is N1, the number of pixels horizontally from P2 to the central axis of the right lens is N2, the actual horizontal distance of the object corresponding to a single pixel is T, the pupil diameter is D, and the number of diameter pixels of the pupil imaged in the image is C. It can be deduced that:

[0046]

[0047] The specific process of measuring pupil diameter by the pupil diameter measurement system based on infrared binocular camera is as follows:

[0048] Turn on the first infrared illuminator 1 and the second infrared illuminator 4, and the first camera 2 and the second camera 3 simultaneously capture the human eye image;

[0049] Use the YOLO neural network model to find the pupil image from the captured human eye image;

[0050] Optionally, the YOLO neural network model is trained and designed based on a deep convolutional neural network (CNN). The YOLO neural network model divides the input image into multiple grids, each grid is responsible for predicting the existence, category and position of the target in the area, which enables YOLO to detect multiple targets at the same time and effectively handle overlap and occlusion problems between targets.

[0051] Fit the pupil circle, calculate the number of pixels C of the pupil diameter and the number of pixels N1 and N2 from the center of the pupil fitting circle to the center axis of the left and right cameras;

[0052] The number of pupil diameter pixels C may be calculated by selecting the number of pixels occupied by the pupil on the left or right camera, or taking the average number of pixels occupied by the left and right cameras.

[0053] Optionally, the pupil center and radius are first pre-determined to intercept a small image of the target area, and then the edge of the target area is extracted using a least squares parabola fitting.

[0054] The pupil diameter is calculated using formula (1).

[0055] The pupil diameter measurement system based on an infrared binocular camera provided by the present invention can effectively solve the problem of inaccurate measurement values ​​caused by changes in the measurement distance through binocular 4K infrared imaging. The system can adapt to devices with different eye mask distances without calibrating the test distance. The tester can also measure accurately without having to lean on the eye mask.

[0056] Example 2

[0057] A pupil diameter measurement method based on an infrared binocular camera is provided, based on the above system. For a detailed description of the system, reference can be made to the corresponding description in the above system embodiment, which will not be repeated here. Figure 3 As shown, the method comprises the following steps:

[0058] S1, obtaining human eye images taken simultaneously by infrared binocular cameras;

[0059] Specifically, the first infrared illuminator 1 and the second infrared illuminator 4 are turned on, and the first camera 2 and the second camera 3 simultaneously capture images of human eyes.

[0060] S2, matching the pupil position difference from the captured human eye image;

[0061] The measurement principle of the pupil diameter measurement system based on infrared binocular camera is as follows: Figure 2 shown. Figure 2 In the figure, P is the test point, P1 and P2 are the images of the pupil center point P on the target surfaces of the left and right camera sensors respectively, X1 is the distance from P1 to the central axis of the left camera lens, X2 is the distance from P2 to the central axis of the right camera lens, F is the focal length of the camera, L is the longitudinal distance from P to the center of the lens, A1 is the lateral distance from P to the central axis of the first lens, A2 is the lateral distance from P to the central axis of the second lens, and B is the lateral distance between the centers of the two camera lenses.

[0062]

[0063] Assume that the length and width of the camera sensor pixel are U, the number of pixels horizontally from P1 to the central axis of the left lens is N1, the number of pixels horizontally from P2 to the central axis of the right lens is N2, the actual horizontal distance of the object corresponding to a single pixel is T, the pupil diameter is D, and the number of diameter pixels of the pupil imaged in the image is C. It can be deduced that:

[0064]

[0065] In some embodiments, Figure 4 As shown, the step of matching the position difference of the pupil from the captured human eye image includes:

[0066] S21, finding a pupil image from the captured human eye image;

[0067] Specifically, the step of finding the pupil image from the captured human eye image includes:

[0068] The pupil image is found from the captured human eye image using the YOLO neural network model.

[0069] Optionally, the YOLO neural network model is trained and designed based on a deep convolutional neural network (CNN). The YOLO neural network model divides the input image into multiple grids, each grid is responsible for predicting the existence, category and position of the target in the area, which enables YOLO to detect multiple targets at the same time and effectively handle overlap and occlusion problems between targets.

[0070] S22, fitting a pupil circle in the pupil image;

[0071] Optionally, the pupil center and radius are first pre-determined to intercept a small image of the target area, and then the edge of the target area is extracted using a least squares parabola fitting.

[0072] S23. Based on the fitted pupil circle, calculate the number of diameter pixels C of the pupil imaged in the image and the number of pixels N1 and N2 from the center of the pupil fitting circle to the central axis of the left and right cameras.

[0073] The number of pupil diameter pixels C may be calculated by selecting the number of pixels occupied by the pupil on the left or right camera, or taking the average number of pixels occupied by the left and right cameras.

[0074] S3. Calculate the pupil diameter through binocular stereo imaging relationship. Specifically, the pupil diameter is calculated as follows:

[0075]

[0076] Among them, D is the pupil diameter, C is the number of diameter pixels of the pupil imaged in the image, B is the lateral distance between the centers of the two camera lenses, N1 is the number of pixels from the pupil center in the horizontal direction from the imaging point P1 of the left camera to the central axis of the left lens, N2 is the number of pixels from the pupil center in the horizontal direction from the imaging point P2 of the right camera to the central axis of the right lens, and P1 and P2 are the images of the pupil center P on the sensor target surfaces of the left and right cameras respectively.

[0077] Example 3

[0078] A computer device 600, such as Figure 5 As shown, it includes a memory 610, a processor 620, and a computer program 630 stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a pupil diameter measurement method based on an infrared binocular camera are implemented. For a detailed description of the method, reference may be made to the corresponding description in the above method embodiment, which will not be repeated here.

[0079] Example 4

[0080] A computer readable storage medium such as Figure 6 As shown, a computer program is stored thereon, and when the computer program is executed by the processor, the steps of a pupil diameter measurement method based on an infrared binocular camera are implemented. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiment, and no further description is given here.

[0081] Example 5

[0082] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of a pupil diameter measurement method based on an infrared binocular camera. For a detailed description of the method, reference may be made to the corresponding description in the above method embodiment, which will not be repeated here.

[0083] The number of devices and processing scales described here are used to simplify the description of the present invention. Applications, modifications and variations of the present invention will be obvious to those skilled in the art.

[0084] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and implementation modes. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

[0085] The apparatus, computer device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, computer device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, computer device and non-volatile computer storage medium will not be repeated here.

[0086] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software units for implementing the method and structures within the hardware component.

[0087] The systems, devices or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions in various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or more software and / or hardware.

[0088] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0089] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0093] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program units. Generally, program units include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program units may be located in local and remote computer storage media, including storage devices.

[0094] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0095] The above description is only an embodiment of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A pupil diameter measurement method based on an infrared binocular camera, characterized in that: The following steps are involved: Acquire human eye images taken simultaneously by infrared binocular cameras; Match the position difference of the pupil from the captured human eye image; The pupil diameter is calculated through binocular stereo imaging relationship.

2. A pupil diameter measurement method based on an infrared binocular camera as claimed in claim 1, characterized in that: The step of matching the position difference of the pupil from the captured human eye image comprises: Find the pupil image from the captured human eye image; Fitting a pupil circle in the pupil image; Based on the fitted pupil circle, the number of diameter pixels of the pupil imaged in the image and the number of pixels from the center of the pupil fitting circle to the center axis of the left and right cameras are calculated.

3. A pupil diameter measurement method based on an infrared binocular camera as claimed in claim 2, characterized in that: The step of finding the pupil image from the captured human eye image comprises: The pupil image is found from the captured human eye image using the YOLO neural network model.

4. A pupil diameter measurement method based on an infrared binocular camera as claimed in claim 2, characterized in that: The calculation formula of the pupil diameter is: Among them, D is the pupil diameter, C is the number of diameter pixels of the pupil imaged in the image, the calculation of C selects the number of pixels occupied by the pupil in the left or right camera or takes the average number of pixels occupied by the left and right cameras, B is the lateral distance between the centers of the two camera lenses, N1 is the number of pixels horizontally from the left lens center axis at the imaging point P1 of the left camera, N2 is the number of pixels horizontally from the right lens center axis at the imaging point P2 of the right camera, P1 and P2 are the images of the pupil center point P on the sensor target surfaces of the left and right cameras respectively.

5. A pupil diameter measurement system based on an infrared binocular camera, implementing the method according to any one of claims 1 to 4, characterized in that: It includes two infrared illuminators, two cameras, a fixed bracket, and a main controller. The infrared illuminators and the cameras are installed on the fixed bracket. The two infrared illuminators are arranged on both sides of the two cameras. The two cameras are used to simultaneously capture human eye images. The main controller is used to match the position difference of the pupils from the captured human eye images and calculate the pupil diameter through the binocular stereo imaging relationship.

6. A pupil diameter measurement system based on an infrared binocular camera as claimed in claim 5, characterized in that: The infrared illuminator adopts an infrared illuminator with a wavelength of 940nm.

7. A pupil diameter measurement system based on an infrared binocular camera as claimed in claim 6, characterized in that: The diameter of the infrared illuminator is 4 mm, and the diameter of the camera is 14 mm.

8. A pupil diameter measurement system based on an infrared binocular camera as claimed in claim 7, characterized in that: The lateral distance between the centers of the two camera lenses is 28 mm.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images