Wearable pupil monitoring device and method

Abstract

The present invention proposes a wearable pupil monitoring device and method based on a flexible near-infrared point array detector. The device and method do not require an infrared camera and lens. The flexible near-infrared detector point array closely fits the eyelid surface to accurately detect the near-infrared light signal transmitted by the closed pupil. At the same time, dynamic closed-eye pupil monitoring is achieved by establishing a closed eyelid pupil image processing model and using a genetic algorithm as an optimization algorithm. The device and method not only provide users with a convenient and lightweight way to dynamically monitor pupil size in the closed-eye state, but also improve the accuracy and reliability of pupil monitoring in the closed-eye state.

Description

Technical Field

[0001] The present invention relates to the field of pupil monitoring technology, and in particular to a wearable pupil monitoring device and a method for determining the pupil size of a human eye in a closed eyelid state. The device and method achieve dynamic and accurate monitoring of pupil size in a closed eye state by using a flexible near-infrared detector point array placed close to the eyelid without using an infrared camera and a lens. Background Art

[0002] The monitoring of pupil changes plays a vital role in clinical nursing. Its monitoring results are not only an important indicator for evaluating the functional state of the patient's nervous system, but also a key basis for predicting and diagnosing a variety of potential diseases. Especially in the fields of neurosurgery, neurology, and intensive care, the real-time monitoring of pupil size and its light reflex is of irreplaceable importance to the patient's life safety. In the absence of drugs and sensitive light stimulation, the diameter of a normal adult pupil is between 2-6 mm under natural light. When the pupil is dilated (diameter greater than 6 mm), it may indicate increased intracranial pressure, midbrain damage, or the use of certain drugs (such as atropine); and when the pupil is constricted (diameter less than 2 mm), it may indicate brainstem damage, organophosphorus pesticide poisoning, or the use of certain drugs (such as morphine). By measuring the size of the pupil, doctors can preliminarily assess whether the patient has cerebral ischemia and hypoxia, craniocerebral injury, midbrain lesions, etc., providing effective assistance for the judgment and evaluation of the condition. In particular, pupil monitoring is of special significance for patients in a coma. Since their consciousness is impaired and their eyes are often closed, real-time pupil monitoring becomes particularly difficult.

[0003] In clinical practice, traditional methods such as using a pupil ruler to estimate pupil size through naked eye observation often rely on the experience of medical staff and have large subjective errors. Although the pupil measurement instrument based on the infrared imaging principle has high integration and accuracy, its operation is also relatively complicated, requiring the operator to have corresponding professional knowledge and skills. In addition, these traditional measurement methods are mostly static and difficult to reflect the dynamic changes of the pupil in a complex environment in real time, especially the inability to monitor the pupil size in the closed eye state, and thus unable to timely reflect the physiological condition of comatose patients.

[0004] The patent document (publication number CN116823746A) proposes a method that preprocesses the original human eye image to extract the pupil image, and uses a preset pupil feature extraction model to extract image, convolution and depth features respectively, fuses these features to construct pupil features, and then predicts the pupil size based on the features through a pupil size prediction model. However, this method requires that the user must be in an open-eyed state to extract the original human eye image, and then perform subsequent processing procedures such as pupil image extraction, feature extraction and pupil size prediction. This requirement limits the application of this method in closed-eye scenarios, because when the user closes his eyes, due to the occlusion of the eyelids, a clear image of the pupil cannot be directly obtained, and thus subsequent pupil size calculation cannot be performed. Therefore, this method cannot realize pupil size calculation in the closed-eye state, and fails to provide an effective solution for closed-eye scenarios. Summary of the invention

[0005] Based on this, the purpose of the present invention is to provide a wearable pupil monitoring device and method, which does not use an infrared camera and lens, and realizes accurate monitoring of the closed eye pupil state through a flexible near-infrared detector point array placed close to the eyelid.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A wearable pupil monitoring device and a device for collecting and transmitting output signals of a wearable pupil monitoring device having a flexible near-infrared detector point array, comprising a main frame, a flexible near-infrared detector point array, a near-infrared light source, a black and white grid calibration object of known size, a data acquisition module, a control and processing module, and a wireless transmission module.

[0008] The frame of the main frame is made of nylon (PA) material with high mechanical strength and toughness, and the eye mask and nose pads that contact the user's eyes are made of silicone material with good elasticity and chemical corrosion resistance. The temples in the frame are thickened, and the power module and near-infrared light source for the entire device are integrated inside.

[0009] The flexible near-infrared detector array is controlled by a program according to the instructions issued by the terminal to provide the detector array N ports V in clock order. in (AI0—AIn) drive signal, output signal V out(AO0—AOn) is collected after being amplified by the signal amplification circuit. The basic structure of the detector unit in the flexible near-infrared photodetector array is a silicon-based photodiode, which is used to receive optical signals. In particular, the number of detectors in the flexible near-infrared photodetector array is N×N, and array data scanning is achieved through N signal write ports and N signal read ports. The flexible near-infrared photodetector array is integrated on a flexible substrate (such as polyimide material or polydimethylsiloxane material, etc.) to protect the internal structure.

[0010] The near-infrared light source is used to emit near-infrared light to the user's eyes. The wavelength range of the near-infrared light source is 750-900nm, ensuring that it can effectively and safely penetrate the eye tissue and transmit from the pupil. The near-infrared light source type can be a light emitting diode (LED) and a laser diode (LD) near-infrared light source, etc. The specific safe power range should be determined comprehensively based on factors such as the laser's wavelength, pulse duration, beam diameter, and application scenarios, and should comply with the provisions of the International Electrotechnical Commission (IEC) or other relevant safety standards.

[0011] The device of the wearable pupil monitoring device with flexible near-infrared detector point array output signal is characterized in that the black and white grid calibration object (1×1cm 2 ) is attached to the eyelid for calibration. After calibration, it can be taken out without changing the position of the flexible near-infrared detector point array of the wearable pupil monitoring device relative to the eyelid, and the relative position of the near-infrared light source of the wearable pupil monitoring device and the corner of the eye.

[0012] The near-infrared light source is wrapped by an external sleeve to reduce the impact of reflected light caused by light leakage on the fidelity of the collected image. The near-infrared light source and the external sleeve are placed on the thickened part of the temple of the frame. Considering that the near-infrared light source should effectively penetrate the eye tissue, including the skin at the corner of the eye, the eyeball and the eyelid, the near-infrared light source should be close to the corner of the eye, and the emission direction of the near-infrared light source can be appropriately adjusted to ensure light emission from the pupil.

[0013] The flexible near-infrared detector point array includes a plurality of near-infrared detectors arranged in an array and closely fitting the user's eyelids, and is used to receive and detect near-infrared light signals transmitted from the user's pupils, and convert the light signals into voltage signals in an array scanning manner.

[0014] The data acquisition module and the control and processing module are connected to the near-infrared light source and the flexible near-infrared detector point array, and are used to control the light frequency and intensity of the near-infrared light source according to the terminal instructions received by the wireless transmission module, and to collect and process the light signals captured by the flexible near-infrared detector point array.

[0015] The wearable pupil monitoring device includes a power module and a main frame worn on the user's head or eyes. The power module is used to provide power to the near-infrared light source, the flexible near-infrared detector array, the data acquisition module, the control and processing module, and the wireless transmission module; the main frame ensures that the device is stably fixed near the user's eyes and fits the eyelids to improve the accuracy of pupil status monitoring.

[0016] A wearable pupil monitoring method. Specifically, the wearable pupil monitoring device needs to have an image calibration function to achieve accurate measurement of pupil size. The specific implementation method is to adjust the position of the near-infrared light source so that it penetrates the eye tissue and transmits from the pupil, attach a black and white grid calibration object with a known size to the eyelid, use an image processing algorithm to perform contour recognition on the collected image, and calculate the proportional coefficient between the number of pixels contained in the unit black and white grid in the collected image and the actual size, that is, the number of pixels corresponding to each millimeter of length.

[0017] In particular, in the eyes-closed state, the light signal transmitted from the eyelid can be approximated as the convolution result of the pupil image and the Gaussian distribution function. This method establishes a correct image processing model and uses a genetic algorithm as the closed-eye pupil size optimization algorithm to complete the fitting optimization of the collected closed-eyelid image to correct the corresponding pupil light intensity distribution, thereby improving the accuracy and fidelity of the calculation.

[0018] The closed-eye pupil monitoring method used in the present invention is as follows:

[0019] Preferably, taking the laser diode near-infrared light source used in the present invention as an example, the laser diode near-infrared light source uses a VCSEL near-infrared light source (central wavelength is about 808nm, driving voltage is in the range of 1.6V to 2.0V, hereinafter referred to as a VCSEL light source), which has a high transmittance to human eye tissue and is highly safe. The VCSEL light source is placed near the corner of the human eye. By slightly adjusting the position of the VCSEL light source, an infrared image with high contrast can be collected from the closed eyelid, which can be regarded as the convolution result of the pupil image and the Gaussian distribution function:

[0020]

[0021]

[0022]

[0023] Among them, I0 is the normalized light intensity value transmitted to the pupil, a and b are the length of the major semi-axis and the minor semi-axis of the elliptical pupil respectively, (x, y) is the coordinate information of the closed eyelid image, ε is the standard deviation of the Gaussian distribution function, I1 is the light intensity distribution of the pupil image, I2 is the standard Gaussian distribution function, and I3 is the convolution result of I1 and I2.

[0024] The collected closed eyelid image can be approximated as the convolution result of the pupil image and the Gaussian distribution function. The mathematical model closest to the closed eyelid image can be obtained by continuous iterative optimization using a genetic algorithm. The mathematical model should contain the following optimal parameters: I0, a, b, ε. Finally, the light intensity distribution function I1(x, y) at the pupil under the closed eyelid is obtained, and the calibration parameters are used to obtain the accurate long-axis and short-axis sizes of the pupil.

[0025] The present invention proposes a wearable pupil monitoring device and method by establishing a closed eyelid pupil image processing model and using a genetic algorithm as an optimization algorithm, and the beneficial effects are as follows:

[0026] 1) Through collaborative control with wearable devices, data collection, processing and data visualization can be integrated;

[0027] 2) This method does not require the use of infrared cameras and lenses and supports closed eyelid pupil status monitoring;

[0028] 3) After the image processing model and the optimization algorithm are modeled, the optimization process of the method is completed by an external device or mobile application that supports wireless data transmission, which increases the real-time performance and flexibility of the device application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the wearable pupil monitoring device described in Example 1, wherein 1 is the frame of the device main body frame, and 2 is the temple structure of the wearable device main body frame.

[0030] Figure 2 This is a conceptual diagram of the monitoring device worn by the user. 3 is the power supply module and 4 is the near-infrared light source.

[0031] Figure 3 In Example 1, Figure 1 The components of the main frame mirror frame are shown, where 5 is a PCB mainboard module and 6 is a near-infrared detector array.

[0032] Figure 4 For Example 1 Figure 3 The components of the PCB mainboard module shown are described, where 7 is a data acquisition module, 8 is a control and processing module, and 9 is a wireless transmission module.

[0033] Figure 5 For Example 1 Figure 3 Detail of the flexible near-infrared detector array shown.

[0034] Figure 6 This is a flow chart of the operation of the wearable pupil monitoring device described in Example 1.

[0035] Figure 7 The result is a pupil image acquisition under closed eyelids.

[0036] Figure 8 The pupil size fitting result under closed eyelids. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings and Example 1.

[0038] Embodiment 1:

[0039] In this embodiment, an accurate size calibration step is first performed, using a chessboard calibration method to achieve accurate calibration between the image size and the actual size. Specifically, a black and white grid calibration object with known size (in this example, the size is 1×1 cm) is attached to the eyelid. 2 ), use image processing algorithms to perform contour recognition on the collected images, and calculate the proportional coefficient between the number of pixels contained in the unit black and white grid in the collected image and the actual size, that is, the number of pixels corresponding to each millimeter length is approximately 0.77.

[0040] Next, to ensure the accuracy of the measurement, Figure 1 2. Wearable device main frame, temples and Figure 2 The near-infrared light source in the wearable monitoring device glasses is adjusted to ensure that the end face of the light source sleeve is completely in line with the corner of the eye to avoid light leakage. The effect of the user wearing the monitoring device is as follows Figure 2 In particular, Figure 3 As shown in Figure 1, the size of the 6 flexible near-infrared detector array must be larger than the eyeball and fit closely with the user's eyelids. At the terminal, we control the flickering of the near-infrared light source and the Figure 5 The scanning operation of the flexible near-infrared detector array (15×15 array in this example) is controlled by the program to give the detector drive signals in clock order. The detector voltage response signal is as follows: Figure 3 5PCB motherboard module collection and processing. Specifically, Figure 4 The data acquisition module 7 collects and converts the analog signals into digital signals, and the control and processing module 8 then processes these signals to distinguish the images of the eyelids or the calibration objects, and transmits the relevant data to the terminal for further processing through the wireless transmission module 9.

[0041] In the terminal processing stage, follow the Figure 6The logic block diagram shown first images and stores the data, and then grayscales the resulting image. The closed-eye image formed by the pupil transmitted light is collected, and the intensity distribution image is generated based on the transmitted brightness at the pupil. Within the system, the initial value I0 of the normalized light intensity transmitted to the pupil is preset as 100, the pupil size 2a=2.1mm, 2b=2.3mm, and the Gaussian kernel standard deviation ε=3.31 as initial parameters. Through Gaussian convolution, the pupil image in the closed-eye state is simulated, and the mean square error of the image is calculated to determine the iteration threshold. If the value is less than the preset threshold δ th = 0.01, the simulation is considered successful, and the actual size of the pupil in the closed eye state is output. If the mean square error value does not meet the conditions, we use the genetic algorithm to reassign I0, 2a, 2b and ε, and repeat the above calculation process until the conditions are met. The closed eyelid image captured is as follows Figure 7 As shown in Figure 1, the optimal values ​​of the parameters are I0 = 121, 2a = 4.36 mm, 2b = 4.21 mm and ε = 6.74. The final fitting results are shown in Figure 1. Figure 8 As shown (taking the long semi-axis fitting of the elliptical pupil as an example), it can be seen from the figure that the convolution result after optimization by the genetic algorithm is consistent with the light intensity distribution of the closed eyelid image collected in the experiment, which verifies the beneficial effects of the present invention from an experimental perspective.

[0042] Obviously, the above-mentioned embodiments of the present invention are only examples for more clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Any modification, equivalent substitution and improvement within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A wearable pupil monitoring device, characterized in that: include: A main frame, wherein the main frame encapsulates various components of the wearable device, mainly including a frame and temples; A flexible near-infrared photodetector dot array is placed in the frame of the main body frame; A near-infrared light source is placed in the thickened portion of the temple of the main frame; A black and white grid calibration object of known size; A data acquisition module, which is placed in the lens frame of the main frame and connected to the output end of the flexible near-infrared photoelectric detector point array, is used to collect the output signal of the photoelectric detector and convert it into a digital signal for subsequent processing; A control and processing module is placed in the frame of the main frame and connected to the data acquisition module. On the one hand, it processes the data received from the data acquisition module according to the instructions sent by the terminal. The data processing includes amplification, filtering, and A / D conversion to ensure the accuracy and availability of the data. On the other hand, it is used to control the overall operation process of the system, including scheduling data information collection and wireless transmission. The information includes picture information and video information; A wireless transmission module, which is placed in the frame of the main frame and connected to the control and processing module, is used to upload the digital signal converted by the data acquisition module to the terminal, receive the setting instruction sent by the terminal and send it to the control and processing module. The wireless transmission module adopts Bluetooth or Wi-Fi technology, depending on the application requirements and communication distance; The closed eye pupil image formed by pupil transmission light is calculated as the convolution result of the pupil image and the Gaussian distribution function: Where I0 is the normalized light intensity value transmitted to the pupil, a and b are the length of the major and minor axes of the elliptical pupil, respectively, (x, y) is the coordinate information of the closed eyelid image, ε is the standard deviation of the Gaussian distribution function, I1 is the light intensity distribution of the pupil image, I2 is the standard Gaussian distribution function, and I3 is the convolution result of I1 and I2; The closed eye pupil image formed by the pupil transmitted light is the convolution result of the pupil image and the Gaussian distribution function. The mathematical model closest to the closed eyelid image is obtained by iterative optimization using a genetic algorithm. The mathematical model should include the following optimal parameters: I0, a, b, ε. Finally, the light intensity distribution function I1 (x, y) at the pupil under the closed eyelid is obtained, and the proportional coefficient is used to obtain the accurate long axis size and short axis size of the pupil.

2. The wearable pupil monitoring device according to claim 1, characterized in that: The frame in the main frame is made of nylon (PA) material with high mechanical strength and toughness, the eye mask and nose pad that contact the user's eyes are made of silicone material with good elasticity and chemical corrosion resistance, the temples in the frame are thickened, and the power module and near-infrared light source for the entire device are integrated inside.

3. The wearable pupil monitoring device according to claim 1, characterized in that: The flexible near-infrared photoelectric detector array is controlled by a program according to the instructions issued by the terminal to provide the N ports V of the detector array in clock order. in (AI0—AIn) drive signal, output signal V out (AO0—AOn) is collected after being amplified by the signal amplification circuit. The basic structure of the detector unit in the flexible near-infrared photodetector array is a silicon-based photodiode, which is used to receive light signals. The number of detectors in the flexible near-infrared photodetector array is N×N, and array data scanning is achieved through N signal write ports and N signal read ports. The flexible near-infrared photodetector array is integrated on a flexible substrate to protect the internal structure.

4. The wearable pupil monitoring device according to claim 1, characterized in that: The wavelength range of the near-infrared light source is 750-900nm, ensuring that it can effectively and safely penetrate the eye tissue and transmit from the pupil. The type of the near-infrared light source is a light emitting diode (LED) near-infrared light source or a laser diode (LD) near-infrared light source. The safe power range should be determined comprehensively based on the wavelength, pulse duration, beam diameter and application scenario of the laser, and should comply with the provisions of the International Electrotechnical Commission (IEC) safety standards.

5. The wearable pupil monitoring device according to claim 1, characterized in that: The black and white grid calibration object of known size is attached to the eyelid for calibration. After calibration, it is taken out without changing the position of the flexible near-infrared photodetector point array of the wearable pupil monitoring device relative to the eyelid, and the relative position of the near-infrared light source of the wearable pupil monitoring device and the corner of the eye.

6. The wearable pupil monitoring device according to claim 1, characterized in that: The near-infrared light source is wrapped by an external sleeve to reduce the impact of reflected light caused by light leakage on the fidelity of the collected image. The near-infrared light source and the external sleeve are placed on the thickened part of the temple of the frame. Considering that the near-infrared light source should effectively penetrate the eye tissue, the external sleeve should be close to the corner of the eye, and the emission direction of the near-infrared light source is appropriately adjusted to ensure light emission from the pupil.

7. The wearable pupil monitoring device according to claim 1, characterized in that: The data acquisition module further includes a timestamp unit for acquiring time data, and the picture information and the video information respectively include timestamp data.

8. A wearable pupil monitoring method, characterized in that: The method comprises: Adjust the position of the near-infrared light source so that it penetrates the eye tissue and is transmitted from the pupil; Capturing a black-and-white grid calibration object image of known size formed by pupil transmission light and a closed-eye pupil image through a flexible near-infrared photodetector dot array of a wearable pupil monitoring device, wherein calibration of the black-and-white grid calibration object image of known size formed by pupil transmission light involves graying the image and calculating a proportionality coefficient between the number of pixels contained in a unit black-and-white grid in the image and the actual size; Sending a closed-eye pupil image formed by the pupil transmission light to a terminal, so as to calibrate the size ratio and calculate the pupil size using the captured image; receiving and displaying, on the terminal, a closed-eye pupil image formed by the pupil transmitted light; The closed eye pupil image formed by pupil transmission light is calculated as the convolution result of the pupil image and the Gaussian distribution function: Where I0 is the normalized light intensity value transmitted to the pupil, a and b are the length of the major and minor axes of the elliptical pupil, respectively, (x, y) is the coordinate information of the closed eyelid image, ε is the standard deviation of the Gaussian distribution function, I1 is the light intensity distribution of the pupil image, I2 is the standard Gaussian distribution function, and I3 is the convolution result of I1 and I2; The closed eye pupil image formed by the pupil transmitted light is the convolution result of the pupil image and the Gaussian distribution function. The mathematical model closest to the closed eyelid image is obtained by iterative optimization using a genetic algorithm. The mathematical model should include the following optimal parameters: I0, a, b, ε. Finally, the light intensity distribution function I1 (x, y) at the pupil under the closed eyelid is obtained. The proportional coefficient is used to obtain the accurate major and minor axis sizes of the pupil.

9. The wearable pupil monitoring method according to claim 8, characterized in that: The accurate pupil size obtained according to the proportionality coefficient should include the closed-eye pupil image formed by the pupil transmitted light and the pupil size.

Images