Pupil light reflex auxiliary diagnosis detector based on smart phone and detection method

Abstract

The invention discloses a pupil light reflex auxiliary diagnosis detector based on a smart phone and a detection method. According to the pupil light reflex autism auxiliary diagnosis detector based on the smart phone and the detection method, combined optimization of stimulation light source spectrums is achieved through intelligent program control multi-wavelength light stimulation of a mobile phone OLED screen, and single-shot majority synchronous image analysis based on the smart phone is designed. The auxiliary diagnosis kit is convenient to use, quick in auxiliary diagnosis result, easy in technology implementation, low in cost, convenient to popularize and friendly to users, and the convenience of auxiliary diagnosis and detection of the PLR of the autism children is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of eye movement detection, and particularly relates to a pupil light reflection auxiliary diagnosis detection instrument based on a smart phone and a detection method. BACKGROUND

[0002] Autism Spectrum Disorder (ASD, simply autism) is a group of neurodevelopmental disorders that occur in early childhood, with social communication disorders, narrow interests, and repetitive stereotyped behaviors as the main characteristics. Autism usually occurs in infancy, and there is currently no effective treatment drug. Early detection, early diagnosis and early intervention can improve the symptoms and prognosis of children to varying degrees. Eye tracking technology has matured after a long development process. The current popular eye tracking technology is mainly based on "non-invasive" technology of eye image analysis. Its basic principle is: a light beam and a camera are aimed at the eye of the subject, and the direction of the subject's gaze is inferred through light and back-end analysis. The camera records the interaction process. The current eye tracker mostly uses pupil-corneal reflection spot recognition technology. The feature it uses that remains unchanged during eye movement is the Purkinje spot on the outer surface of the cornea, which is a bright spot on the cornea produced by the reflection of light entering the pupil on the outer surface of the cornea. Since the position of the eye tracker (including the camera) is fixed, the position of the light source is also fixed, and the center of the eye is fixed (assuming the eye is spherical and the head is not moving), the absolute position of the Purkinje spot does not change with the rotation of the eye. But its position relative to the pupil and the eye is constantly changing - for example, when you stare at the camera, the Purkinje spot is between your pupils; when your eye is raised, the Purkinje spot is below your pupil. In this way, as long as the positions of the pupil and the Purkinje spot on the eye image are located in real time, the corneal reflection vector can be calculated, and the user's line of sight direction can be estimated using a geometric model. The eye tracking device in the related technology has the following shortcomings: the light source is generally independent of the eye tracker and is set externally, which has the disadvantages of large volume, inconvenience to carry, complex structure, and inability to detect at any time and any place; the wavelength of the illuminating light is single, and different subjects have different pupil response sensitivities to different wavelengths, which can easily cause test fatigue of the subjects; the light produced by the light source in the device can easily cause glare, which can harm the eyes of children; due to the long distance between the light source and the eye during detection, it is easy to be disturbed by external light, and the requirements for the use environment are also relatively high. There is no autism PLR early auxiliary diagnosis product based on a smart phone on the market. Smart phones have increasingly powerful computing capabilities, and smart phones with dual-core processors and independent display units plus high-definition cameras have become popular. Smart phones have become smart mobile Internet terminals with powerful processing capabilities, and their expansion functions have far exceeded people's imagination. Pupil recognition requires high processing hardware and imaging equipment. Previously, it had to rely on computers and high-precision camera systems. With the rapid development of smart phones, smart phones also have the processing capability to process pupil images in real time, and the development of 5G networks has created superior conditions for the timely communication between smart phones and remote information centers. SUMMARY

[0003] To solve at least one of the above technical problems, the application provides a smartphone-based pupil light reflection autism auxiliary diagnosis detector and a detection method. The purpose of the application is achieved by the following technical solutions: on the one hand, a smartphone-based pupil light reflection autism auxiliary diagnosis detector is provided, which comprises: a light source regulation and AI color classification extraction system based on a mobile phone screen, a light source regulation system for automatic control of multi-wavelength OLED light stimulation is realized through evolutionary calculation and swarm intelligence algorithm. The color of light stimulation is accurately classified by using fuzzy technology to realize AI color classification extraction, and intelligent data classification is provided for the pupil light reflection in the later stage; a pupil diameter dynamic data acquisition module based on a mobile phone front camera simulates the use of a mobile phone front camera to collect the pupil images of the left and right eyes of an autistic child, simultaneously performs image segmentation through AI control, and uses edge threshold detection algorithm, Hough transform, etc. to analyze the bimodal histogram, and processes the pupil images to obtain pupil dynamic data.

[0004] As a further improvement, the smartphone-based pupil light reflection autism auxiliary diagnosis detector comprises a 64 million pixel camera (f / 3.5 aperture, OIS optical image stabilization), supports 3.5 times optical zoom (3.5 times zoom is an approximate value, and the lens focal lengths are 13 mm, 23 mm, 26 mm, and 90 mm), and 100 times digital zoom; as a further improvement, the maximum delay of the front camera of the smartphone-based pupil light reflection autism auxiliary diagnosis detector is 8.5 milliseconds, and the data processing delay is less than 10 milliseconds. As a further improvement, the technical index of the mobile phone OLED screen light stimulation is 1.07 billion colors, P3 wide color gamut, supports a maximum refresh rate of 120 Hz, 1440 Hz high-frequency PWM dimming, 300 Hz sampling rate, FHD+ 2700x1228 pixels, pixel density 450 ppi, and output light wavelength range: 400-980 nm; as a further improvement, the sampling frequency of the pupil diameter dynamic data acquisition module is greater than or equal to 240 FPS; as a further improvement, the pupil dynamic data of the pupil diameter dynamic data acquisition module is pupil size accuracy ±0.05 mm.

[0005] The smartphone-based pupil light reflection autism auxiliary diagnosis detector provided by the application comprises: a light source regulation and AI color classification extraction system based on a mobile phone screen, which is arranged on the mobile phone screen, and the light emitting direction of the multi-wavelength OLED light stimulation is towards the eye side; and a pupil diameter dynamic data acquisition module based on a mobile phone front camera, which is arranged on the front camera of the smartphone, and the viewfinder direction of the image acquisition module is towards the eye side.

[0006] The integrated and integral autism auxiliary diagnosis and detection instrument based on the smartphone is convenient, fast, low in cost and easy to popularize. Since the light source is arranged on the screen of the smartphone, the detection is less likely to be disturbed by external light and the requirement for the use environment is also lower, which is friendly to the use environment. The rich multi-wavelength light source irradiation reduces the test fatigue of the testees, improves the pupil response sensitivity of different testees to different wavelengths, improves the convenience of the PLR detection of special children and improves the detection efficiency.

[0007] In another aspect, a detection method using the PLR auxiliary diagnosis and detection system based on the smartphone is provided, which comprises the following steps: S1, the screen of the PLR auxiliary diagnosis and detection instrument based on the smartphone faces the face of the testee; S2, the pupil diameter dynamic data acquisition module based on the front camera of the smartphone focuses on the pupil and tracks and follows the rotation of the pupil, and the high-frame-rate pupil image is collected, so that the light source control and color classification module controls the light compensation of the pupils; S3, the multi-wavelength OLED light is emitted from the OLED screen of the smartphone to stimulate the pupils, the spectrum distribution and the light intensity distribution of the OLED screen are used to combine and optimize the output light, so that the multi-wavelength OLED light emits light rays with different wavelengths in color and illumination, so as to achieve the effect of arousing the pupil light stimulation reflex response; S4, the high-frame-rate high-definition continuous images of the pupils under the light stimulation reflex response are collected through the pupil image acquisition module; S5, the main control unit of the smartphone processes the high-frame-rate high-definition continuous images of the pupils, extracts the dynamic parameters of the pupil images, and obtains the pupil reflection curve; S6, the obtained results are transmitted to the autism children information cloud database through the image storage module and the wireless communication module, the pupil light reflection data under the light stimulation are compared with the autism children pupil database, the gradient boosting tree algorithm is used to classify the pupil light reflection big data set, and the predicted autism diagnosis result is obtained; and S7, the auxiliary judgment result is output through the mobile phone display unit.

[0008] As a further improvement, in step S2, the near-infrared light with a wavelength of 940nm is continuously emitted from the screen of the smartphone through the light source control and color classification module to illuminate the pupil. In step S3, the multi-wavelength OLED light randomly selects different wavelengths of flash light in the range of 0-1000ms to stimulate the pupil.

[0009] As a further improvement, in step S4, including pupil image acquisition and pupil light stimulation reflex image acquisition, the pupil image acquisition first acquires a baseline pupil image for 1 second, then the multi-wavelength OLED light source emits an automatically adjustable flash light in the range of 0-1000 ms, and then continues to acquire the pupil image for 2 seconds to capture the entire pupil constriction and recovery process; the pupil light stimulation reflex image acquisition calculates the corneal reflection vector and pupil change size data by locating the positions of the pupil and Purkinje spot on the eye image.

[0010] As a further improvement, in step S5, in one 4-second detection, the image acquisition rate from each eye is 240Hz, the image size is 192 pixels x 192 pixels, and the resolution is 12 bits.

[0011] The detection method based on the above-mentioned smartphone-based pupil light reflection autism auxiliary diagnosis detector provided by the present application should have the same or corresponding beneficial effects as the detector, so further description is not given. Many specific details are described in the above description in order to fully understand the present application, but the present application can also be implemented in other ways different from those described herein, therefore, it cannot be understood as limiting the scope of protection of the present application.

[0012] In summary, although the above-mentioned preferred mode is listed, it should be noted that although those skilled in the art can make various changes and modifications, unless such changes and modifications deviate from the scope of the present application, they should be included in the protection scope of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0013] The present application is further illustrated by the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the following drawings.

[0014] Figure 1 is a use schematic diagram of a smartphone-based pupil light reflection auxiliary diagnosis detector.

[0015] Figure 2 is a main module diagram of a smartphone-based pupil light reflection auxiliary diagnosis detector.

Claims

1. A smartphone-based pupil photoreflectance assisted diagnostic detection instrument, characterized in that, include: The smartphone-based PLR autism auxiliary diagnostic detection module is mainly divided into two modules: (1) a light source control and AI color classification extraction system based on the mobile phone screen, which is set on the mobile phone screen with the light source facing the eyes; (2) a pupil diameter dynamic data acquisition module based on the front camera of the mobile phone. The viewing direction of the pupil diameter dynamic image acquisition system is facing the eyes. The smartphone-based image data storage module is used to store the image data acquired by the image acquisition system; the smartphone-based wireless communication module is used to receive external control commands and output image data.

2. A smartphone-based pupil photoreflectance assisted diagnosis detector, characterized in that: The module for light source control and AI color classification extraction based on a mobile phone screen, through in-depth research on the light reflection response of the sclera, iris, and pupil, utilizes the mobile phone's intelligent programmable OLED to analyze the spectral distribution and light intensity distribution of the OLED screen and optimize its output light combination. It achieves an automatic light source control system for multi-wavelength OLED light stimulation through evolutionary computation and swarm intelligence algorithms.

3. A smartphone-based pupillary light reflex assisted diagnostic detector, characterized in that, The pupil diameter dynamic data acquisition module based on the front camera of a mobile phone uses the front camera of a mobile phone to collect pupil images of the left and right eyes of children with autism. At the same time, it uses AI control to perform image segmentation, and uses edge threshold detection algorithm, Hough transform and other methods to analyze the bimodal histogram to process the pupil images to obtain dynamic pupil data.

4. A smartphone-based pupil photoreflectance assisted diagnostic detector, characterized in that, The pupil light reflection data under light stimulation is synchronously acquired and transmitted to the autism children's information cloud database through the image storage module and wireless communication module.

5. A detection method using the smartphone-based pupil photoreflectance auxiliary diagnostic detector according to any one of claims 1 to 4, characterized in that, The process includes the following steps: S1. Facing the screen of the smartphone-based PLR-assisted diagnostic testing device towards the subject's face; S2. Focusing the pupil on the viewfinder of the pupil diameter dynamic data acquisition module based on the smartphone's front camera, tracking and following the pupil's rotation, and acquiring pupil images at a high frame rate, allowing the light source control and color classification module to supplement the illumination of both pupils; S3. Controlling the smartphone's OLED screen to emit multi-wavelength OLED light to stimulate the pupils through the light source control and color classification module, optimizing the output light by combining the spectral distribution and light intensity distribution of the OLED screen, so that the multi-wavelength OLED light emits light of different colors and illuminance wavelengths to achieve the effect of eliciting pupil light stimulation and reflection response; S4. Acquiring high-definition continuous images of the pupil at multiple frame rates of light stimulation and reflection response through the pupil image acquisition module. S5. The main control unit of the smartphone processes high-definition continuous images of the pupil at multiple frame rates, and obtains the pupil reflex curve by extracting dynamic parameters of the pupil image. S6. The obtained results are transmitted to the autism children's information cloud database through the image storage module and wireless communication module. The data is compared with the autism children's pupil database, and the large dataset of pupil reflexes is classified using the gradient boosting tree algorithm to obtain the predicted autism diagnosis result. S7. Output auxiliary judgment results through the mobile phone display unit.

6. The detection method of claim 5, wherein: In step S2, the pupil is continuously illuminated by the near-infrared light with a wavelength of 940 nm emitted by the light source and the color classification module mobile phone screen. In step S3, the pupil is stimulated by the multi-wavelength OLED light randomly selected to emit different wavelengths of flash light in the range of 0-1000 ms.

7. The method of claim 6, wherein: In step S4, the pupil image acquisition and the pupil light stimulation reflection image acquisition are included. The pupil image acquisition first acquires a baseline pupil image for 1 second, then the multi-wavelength OLED light source emits flash light in the range of 0-1000 ms, and then the pupil image acquisition continues for 2 seconds to capture the entire pupil constriction and recovery process. The pupil light stimulation reflection image acquisition calculates the corneal reflection vector and the pupil change size data by locating the positions of the pupil and the Purkinje spot on the eye image.

8. The detection method of claim 7, wherein: In step S5, in one 4-second detection, the image acquisition rate is 240 Hz, the image size is 192 pixels x 192 pixels, and the resolution is 12 bits from each eye.

Images