Visual perception analysis method and system based on contact lens integrated retina monitoring

Abstract

The invention discloses a visual perception analysis method and system based on contact lens integrated retina monitoring, and relates to the technical field of visual perception, and the method comprises the steps: arranging a miniature image sensor array on a contact lens right facing an eyeball; the retina at the bottom of the eyeball is shot and collected through the miniature image sensor array, and a retina image is obtained; and processing the retina image by using a deep learning algorithm to complete a visual perception task. The miniature image sensor array is integrated in the contact lens, physiological signals and image information of the retina are accurately acquired in real time, the acquired data are deeply analyzed by utilizing an advanced machine learning algorithm and a deep learning model, and different visual perception modes are accurately identified by learning and training a large amount of sample data, so that the visual perception effect of the contact lens is improved. Therefore, comprehensive and deep understanding of visual perception is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of visual perception, and more specifically to a visual perception analysis method and system based on integrated retinal monitoring of contact lenses. Background Art

[0002] In the field of visual perception research, traditional visual detection methods have many limitations. Previous vision examinations mainly focused on simple visual acuity tests, such as logarithmic visual acuity charts or international standard visual acuity charts. These methods can only provide a preliminary assessment of visual acuity, such as the ability to recognize letters or symbols at a specific distance, but are far from sufficient for mining in-depth information of visual perception.

[0003] With the development of technology, although there are some eye detection devices, such as optical coherence tomography (OCT), which can perform high-precision imaging of the retinal structure, these devices are large in size, expensive, and the examination process requires patients to cooperate with professionals in a specific medical environment, making it impossible to achieve daily and dynamic visual perception monitoring.

[0004] In addition, people's research on visual perception increasingly focuses on its actual performance in daily life scenarios. For example, how visual perception changes under different lighting conditions (strong light, weak light, natural light, etc.), different motion states (walking, running, head turning, etc.), and different emotional states (nervous, relaxed, excited, etc.), and it is difficult for traditional methods to obtain comprehensive visual perception data in these complex scenarios.

[0005] In the context of the continuous development of smart wearable devices, contact lenses, as a device that closely fits the eyes, have great potential application value. It can continuously stay close to the eyes in a relatively non-invasive manner and is more likely to obtain continuous and real visual perception-related data compared to traditional devices. However, currently, the technology of using contact lenses for integrated retinal monitoring and conducting visual perception analysis based on this is not yet mature, and there is a lack of such an efficient, accurate, and visual perception analysis method and system that can adapt to multiple scenarios in the market.

[0006] Therefore, how to provide a visual perception analysis method and system based on integrated retinal monitoring of contact lenses, use advanced machine learning algorithms and deep learning models to deeply analyze the collected data, accurately identify different visual perception patterns, and thus achieve a comprehensive and in-depth understanding of visual perception is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a visual perception analysis method and system based on integrated retinal monitoring of contact lenses. A micro-image sensor array is integrated in the contact lenses to collect physiological signals and image information of the retina in real time and accurately. Advanced machine learning algorithms and deep learning models are used to deeply analyze the collected data. Through learning and training on a large number of sample data, different visual perception patterns are accurately identified, so as to achieve a comprehensive and in-depth understanding of visual perception.

[0008] To achieve the above object, the present invention adopts the following technical solutions: A visual perception analysis method based on integrated retinal monitoring of contact lenses, including:

[0009] Setting the micro-image sensor array on the contact lens facing the eyeball;

[0010] Taking pictures of the retina at the bottom of the eyeball through the micro-image sensor array to obtain a retinal image;

[0011] Using a deep learning algorithm to process the retinal image to complete the visual perception task.

[0012] Preferably, after obtaining the retinal image, it further includes preprocessing the retinal image.

[0013] Preferably, using a deep learning algorithm to process the retinal image includes:

[0014] Extracting LBP-HF texture features from the retinal image;

[0015] Inputting the extracted LBP-HF texture feature matrix into a convolutional neural network for training;

[0016] Constructing a retinal feature extraction model based on the convolutional neural network;

[0017] The convolutional neural network uses multiple convolutional layers to further extract features from the retinal image;

[0018] Performing visual perception analysis according to the feature extraction results.

[0019] Preferably, extracting LBP-HF texture features from the retinal image includes:

[0020] Calculating the original mode LBP value of the whole retinal image;

[0021] On the basis of the original mode LBP value, fusing the rotation invariant LBP mode and the equivalent Uniform LBP mode to obtain a rotation invariant equivalent mode operator;

[0022] Make a histogram for rotation-invariant equivalence, count the histogram and perform discrete Fourier transform; calculate the multi-scale local binary pattern Fourier histogram feature, that is, the LBP-HF texture feature.

[0023] Preferably, the convolutional neural network includes a convolutional layer C1, a pooling layer S1, a convolutional layer C2, a pooling layer S2, a convolutional layer C3, a pooling layer S3, a convolutional layer C4, a fully connected layer F1, and a softmax layer cascaded in sequence.

[0024] Preferably, before the micro-image sensor array captures an image of the retina at the bottom of the eyeball, it further includes: integrating a micro-optical lens in front of the micro-image sensor array to focus the light;

[0025] Dynamically adjust the exposure time of the micro-image sensor according to the intensity of the reflected light from the retina;

[0026] Set the retina image acquisition frequency according to the eyeball movement speed and the dynamic range of the retina image change.

[0027] Preferably, a visual perception analysis system based on an integrated contact lens retina monitor includes: a micro-image sensor array module disposed on a contact lens facing the eyeball;

[0028] The shooting angle of the micro-image sensor array module is synchronized with the eyeball angle;

[0029] The micro-image sensor array module is used to capture an image of the retina at the bottom of the eyeball to obtain a retina image;

[0030] A transmission module, respectively connected to the image processing module and the micro-image sensor array module, is used to transmit the retina image obtained by the micro-image sensor array module to the image processing module;

[0031] The image processing module is used to process the retina image by using a deep learning algorithm to complete the visual perception task.

[0032] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a visual perception analysis method and system based on integrated retinal monitoring of contact lenses, including: arranging a micro-image sensor array on a contact lens facing the eyeball; capturing an image of the retina at the bottom of the eyeball through the micro-image sensor array to obtain a retinal image; and processing the retinal image using a deep learning algorithm to complete a visual perception task. The present invention integrates a micro-image sensor array in a contact lens to collect physiological signals and image information of the retina in real time and accurately, and uses advanced machine learning algorithms and deep learning models to deeply analyze the collected data. Through learning and training on a large number of sample data, different visual perception patterns are accurately identified, so as to achieve a comprehensive and in-depth understanding of visual perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0034] Figure 1 FIG. is a schematic flow chart of a visual perception analysis method based on integrated retinal monitoring of contact lenses provided by the present invention.

[0035] Figure 2 FIG. is a schematic structural diagram of a visual perception analysis system based on integrated retinal monitoring of contact lenses provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0037] An embodiment of the present invention discloses a visual perception analysis method based on integrated retinal monitoring of contact lenses, as Figure 1 shown, including:

[0038] Arranging a micro-image sensor array on a contact lens facing the eyeball; the micro-image sensor array is manufactured by micro-nano processing technology, composed of micro-image sensors with various functions, and has biocompatibility and will not cause adverse reactions to the eyes;

[0039] The retina at the bottom of the eyeball is photographed and collected through the micro image sensor array to obtain a retina image; during the collection process, the viewing angle of the micro image sensor array is kept in real-time synchronization with the viewing angle of the eyeball to ensure that the collected image covers the area where the eyeball is looking.

[0040] A deep learning algorithm is used to process the retina image to complete the visual perception task. The deep learning algorithm has been trained with a large number of retina image samples and can identify various features in the image, such as the morphology of retinal blood vessels and signs of nerve cell activity.

[0041] Specifically, after obtaining the retina image, it also includes preprocessing the retina image. The preprocessing includes operations such as grayscale processing, image denoising, and contrast enhancement to improve the image quality and facilitate the subsequent processing by the deep learning algorithm.

[0042] Specifically, for image denoising, an adaptive median filtering algorithm is used. Its core idea is to dynamically adjust the size and parameters of the filter according to the statistical characteristics of local pixels. For each pixel, first calculate the median of its neighborhood. If the difference between the central pixel value and the median exceeds a certain threshold, then this pixel is determined as a noise point and replaced with the neighborhood median. At the same time, combined with bilateral filtering technology, the neighborhood pixels are weighted and averaged according to the spatial distance and gray similarity of the pixels, retaining the edge information of the image while removing noise and avoiding damaging the boundary details of the retinal blood vessels and lesion areas; for contrast enhancement, an adaptive histogram equalization method is adopted. The image is divided into multiple sub-regions, and histogram equalization operations are performed on each sub-region respectively. This can avoid the over-enhancement problem that may be caused by global histogram equalization, making the details of different regions clearer. Especially in the case of uneven light and color distribution in the retina, it can balance the contrast of the entire image. By introducing gamma correction technology and adjusting the gamma value (for example, γ>1 can enhance the details of the bright part, γ<1 can enhance the details of the dark part), the contrast of the image is further optimized to make the overall visual effect of the image more in line with the requirements of subsequent feature extraction.

[0043] Specifically, using a deep learning algorithm to process the retina image includes:

[0044] Extracting the LBP-HF texture features of the retina image;

[0045] Inputting the extracted LBP-HF texture feature matrix into a convolutional neural network for training;

[0046] Normalize the LBP-HF texture feature matrix to an interval to ensure the consistency and stability of the input data; during the training process, divide the dataset into a training set, a validation set, and a test set, and the ratio can be 7:1:2. Use the mini-batch gradient descent method, select a small batch of samples each time for gradient calculation and parameter update to improve the training efficiency. At the same time, use the momentum method to optimize the gradient descent process, avoid getting stuck in local optima, and accelerate the convergence speed.

[0047] Construct a retinal feature extraction model based on a convolutional neural network;

[0048] The convolutional neural network uses multiple convolutional layers to further extract features from the retinal image;

[0049] Conduct visual perception analysis based on the feature extraction results.

[0050] Specifically, the extraction of LBP-HF texture features from the retinal image includes:

[0051] Calculate the original pattern LBP value of the entire retinal image;

[0052] Specifically, for each pixel in the retinal image, taking it as the center, select a circular neighborhood with a radius of R, use the gray value of this pixel as the threshold to compare the pixels in the neighborhood, and obtain a binary code. Arrange the codes in the neighborhood in a clockwise or counterclockwise order to form a binary number, which is the original pattern LBP value of this pixel. To improve the calculation efficiency, use the integral image technique to pre-calculate the sum of pixels in the upper left corner area of each pixel to quickly calculate the sum of pixels in the neighborhood and accelerate the calculation process of the LBP value.

[0053] On the basis of the original pattern LBP value, fuse the rotation invariant LBP pattern and the equivalent Uniform LBP pattern to obtain a rotation invariant equivalent pattern operator;

[0054] Specifically, the rotation invariant LBP pattern makes the original LBP value undergo circular shifting and takes the minimum value as the rotation invariant pattern to ensure that the texture features remain unchanged when the image rotates; the equivalent Uniform LBP pattern classifies the LBP values with the same number of jumps (the number of times from 0 to 1 or from 1 to 0) into one category to reduce the dimension of the feature space. Combine the rotation invariant LBP pattern and the equivalent Uniform LBP pattern to obtain a rotation invariant equivalent pattern operator, which has both rotation invariance and can reduce the feature dimension for subsequent processing.

[0055] Make a histogram of the rotation invariant equivalent, count the histogram and perform a discrete Fourier transform; calculate the multi-scale local binary pattern Fourier histogram feature, that is, the LBP-HF texture feature.

[0056] Generate a histogram for rotation-invariant equivalent pattern operators, and count the occurrence frequency of each pattern; perform a discrete Fourier transform on the histogram to convert the time-domain features into frequency-domain features, highlighting the high-frequency information. In the frequency domain, the high-frequency part reflects the details and local variations of the texture, while the low-frequency part reflects the overall structure of the texture; in order to obtain multi-scale features, perform multiple LBP calculations using neighborhoods with different radii R and different numbers of sampling points P, and combine the frequency-domain features obtained at different scales to form multi-scale local binary pattern Fourier histogram features (LBP-HF) to more comprehensively describe the texture features of retinal images.

[0057] Specifically, the convolutional neural network includes a convolutional layer C1, a pooling layer S1, a convolutional layer C2, a pooling layer S2, a convolutional layer C3, a pooling layer S3, a convolutional layer C4, a fully connected layer F1, and a softmax layer cascaded in sequence.

[0058] Specifically as follows: Layer C1: The input is an image block of 64×64, using 32 convolutional kernels of size 5×5 with a stride of 1, and obtaining 32 feature maps of 64×64 by padding with all 0s;

[0059] Layer S2: Using the method of maximum operation, the filter size used is 2×2 with a stride of 2, and outputting 32 feature maps of size 32×32;

[0060] Layer C2: Using 64 convolutional kernels of size 5×5 with a stride of 1, without using padding with all 0s, and obtaining 64 feature maps of 28×28;

[0061] Layer S2: Using the method of maximum operation, the filter size used is 2×2 with a stride of 2, and outputting 64 feature maps of size 14×14;

[0062] Layer C3: Using 128 convolutional kernels of size 5×5 with a stride of 1, without using padding with all 0s, and obtaining 128 feature maps of 10×10;

[0063] Layer S3: Using the method of maximum operation, the filter size used is 2×2 with a stride of 2, and outputting 128 feature maps of size 5×5;

[0064] Layer C4: Using 256 convolutional kernels of size 5×5, 256 feature maps of 1×1;

[0065] F1: The number of input nodes is 256, and the number of output nodes is 3, denoted as {y0, y1, y3}, representing {blood vessels, macula area, lesion area} in sequence;

[0066] Softmax layer: Convert the output of F1 into a probability distribution.

[0067] Based on the probability distribution output by the softmax layer, determine the probabilities that the retinal image belongs to the blood vessel area, the macular area, or the lesion area. For the lesion area, further analyze the lesion type and severity.

[0068] Generate a visual feature map using the feature extraction results, map the intensities of different features onto the image, and assist the doctor in intuitively understanding the state of the retina. For example, display the distribution and intensity of blood vessel features and lesion features through color coding or grayscale mapping.

[0069] Specifically, before capturing images of the retina at the bottom of the eyeball through the micro-image sensor array, it further includes: integrating a micro-optical lens in front of the micro-image sensor array to focus the light; making the retinal image clearly fall on the sensor array, and using a liquid crystal spatial light modulator (SLM) or a microelectromechanical system (MEMS) deformable mirror to measure and correct the eye's aberration in real time to ensure high-resolution acquisition of the retinal image.

[0070] Dynamically adjust the exposure time of the micro-image sensor according to the intensity of the reflected light from the retina; adopt an automatic exposure algorithm, analyze the initial signal intensity collected by the sensor, and adjust the exposure parameters in real time to obtain a clear retinal image.

[0071] Set the retinal image acquisition frequency according to the eyeball movement speed and the dynamic range of retinal image changes. For normal eyeball movement, collect 30 - 60 frames of images per second to ensure that the dynamic changes of the retina can be captured.

[0072] In a specific embodiment of the present invention, a visual perception analysis system based on a contact lens integrated retinal monitor, as Figure 2 shown, includes: a micro-image sensor array module disposed on a contact lens facing the eyeball;

[0073] The shooting angle of the micro-image sensor array module is synchronized with the eyeball angle;

[0074] The micro-image sensor array module is used to capture images of the retina at the bottom of the eyeball to obtain retinal images;

[0075] A transmission module, respectively connected to the image processing module and the micro-image sensor array module, is used to transmit the retinal image obtained by the micro-image sensor array module to the image processing module;

[0076] The image processing module is used to process the retinal image using a deep learning algorithm to complete the visual perception task.

[0077] In a specific embodiment of the present invention, in the medical field, the early diagnosis and precise monitoring of retinal diseases have always been important research directions. This embodiment aims to demonstrate the feasibility and effectiveness of a visual perception analysis method based on integrated retinal monitoring with contact lenses in practical applications. By constructing a complete system, the acquisition, processing, and analysis of retinal images are realized to assist doctors in disease diagnosis. A micro-nano processing technology is used to manufacture a micro-image sensor array, which is precisely set on a contact lens facing the eyeball. The array consists of micro-image sensors with various functions and has good biocompatibility to ensure that no adverse reactions will occur to the eyes during long-term wearing. To achieve real-time synchronization with the eyeball's perspective, a high-precision eyeball movement tracking component is integrated, which can quickly and accurately sense the rotation direction and angle changes of the eyeball and accordingly adjust the shooting angle of the micro-image sensor array to ensure that the collected images always cover the area where the eyeball is gazing. The specific implementation steps are as follows:

[0078] Retinal image acquisition:

[0079] The patient wears a contact lens integrated with a micro-image sensor array. Before wearing, the doctor will clean and examine the patient's eyes to ensure the safety and comfort of wearing.

[0080] When the patient starts normal activities, the micro-image sensor array starts to work. First, the micro-optical lens focuses the incoming light so that the retinal image is clearly formed on the sensor array. At the same time, the liquid crystal spatial light modulator (SLM) or the microelectromechanical system (MEMS) deformable mirror starts to work to measure and correct the eye's aberration in real time to ensure that the collected retinal images have high resolution.

[0081] According to the intensity of the retinal reflected light, the micro-image sensor automatically adjusts the exposure time. Through the built-in automatic exposure algorithm, the initial signal intensity collected by the sensor is analyzed, and the exposure parameters are adjusted in real time to obtain clear retinal images. In addition, according to the eyeball movement speed and the dynamic range of retinal image changes, the image acquisition frequency is set to 50 frames per second (within the normal eyeball movement range) to ensure that the dynamic changes of the retina can be captured.

[0082] Image transmission:

[0083] The micro-image sensor array module wirelessly sends the collected retinal images to the image processing module through the transmission module. During the transmission process, the data is packed into a specific format and encrypted to prevent data leakage.

[0084] Image preprocessing:

[0085] After the image processing module receives the image, it first performs grayscale processing to convert the color retinal image into a grayscale image, simplifying the subsequent processing flow.

[0086] Next, an adaptive median filtering algorithm is used for image denoising. For each pixel, the median of its neighborhood is calculated. If the difference between the central pixel value and the median exceeds a certain threshold, the pixel is determined to be a noise point and replaced with the neighborhood median. At the same time, combined with bilateral filtering technology, the neighborhood pixels are weighted and averaged according to the spatial distance and gray similarity of the pixels, retaining the edge information of the image while removing noise and avoiding damaging the boundary details of the retinal blood vessels and lesion areas.

[0087] Then, an adaptive histogram equalization method is used to enhance the contrast. The image is divided into multiple sub-regions, and histogram equalization operations are performed on each sub-region respectively to avoid the over-enhancement problem that may be caused by global histogram equalization, making the details of different regions clearer. The gamma correction technology is introduced. By adjusting the gamma value (for example, setting it to 1.2 to enhance the details of the bright part), the contrast of the image is further optimized, making the overall visual effect of the image more in line with the requirements of subsequent feature extraction.

[0088] Feature extraction:

[0089] Extract the LBP-HF texture features of the preprocessed retinal image.

[0090] Calculate the original pattern LBP value of the entire retina image. For each pixel in the retina image, taking it as the center, a circular neighborhood with a radius of R = 3 is selected. Using the gray value of this pixel as the threshold, the pixels in the neighborhood are compared to obtain a binary code. The codes in the neighborhood are arranged in a clockwise order to form a binary number, which is the original pattern LBP value of this pixel. To improve the calculation efficiency, integral image technology is adopted to pre-calculate the sum of pixels in the upper left corner area of each pixel, so as to quickly calculate the sum of pixels in the neighborhood and accelerate the calculation process of the LBP value.

[0091] Based on the original pattern LBP value, the rotation invariant equivalent pattern operator is obtained by fusing the rotation invariant LBP pattern and the equivalent Uniform LBP pattern. The rotation invariant LBP pattern ensures that the texture features remain unchanged when the image rotates by performing circular shifting on the original LBP value and taking the minimum value as the rotation invariant pattern; the equivalent Uniform LBP pattern classifies the LBP values with the same number of jumps (the number of times from 0 to 1 or from 1 to 0) into one category to reduce the dimension of the feature space. Combining the rotation invariant LBP pattern and the equivalent Uniform LBP pattern, the rotation invariant equivalent pattern operator is obtained. This operator has both rotation invariance and can reduce the feature dimension, facilitating subsequent processing.

[0092] Generate a histogram for the rotation-invariant equivalent pattern operator and count the occurrence frequency of each pattern. Perform a discrete Fourier transform on the histogram to convert the time-domain features into frequency-domain features and highlight the high-frequency information. To obtain multi-scale features, perform multiple LBP calculations using neighborhoods with different radii R (such as 2, 3, 4) and different numbers of sampling points P (such as 8, 16, 24), and combine the frequency-domain features obtained at different scales to form the multi-scale local binary pattern Fourier histogram feature (LBP-HF) to more comprehensively describe the texture features of the retinal image.

[0093] Normalize the extracted LBP-HF texture feature matrix to the interval [0, 1] to ensure the consistency and stability of the input data.

[0094] Model training and analysis:

[0095] In the training stage, divide a large number of labeled retinal image datasets into a training set (70%), a validation set (10%), and a test set (20%). Adopt the mini-batch gradient descent method, select a small batch of samples (such as 32 samples) each time for gradient calculation and parameter update to improve the training efficiency. At the same time, use the momentum method (with the momentum coefficient set to 0.9) to optimize the gradient descent process, avoid falling into local optimal solutions, and accelerate the convergence speed.

[0096] Construct a retinal feature extraction model based on a convolutional neural network. This convolutional neural network includes a convolutional layer C1, a pooling layer S1, a convolutional layer C2, a pooling layer S2, a convolutional layer C3, a pooling layer S3, a convolutional layer C4, a fully connected layer F1, and a softmax layer cascaded in sequence.

[0097] According to the probability distribution output by the softmax layer, judge the probabilities of the retinal image belonging to the blood vessel area, the macula area, or the lesion area. For the lesion area, further analyze the lesion type and severity. If the probability of the lesion area exceeds 0.5 and the specific feature pattern has a high degree of matching with the retinal lesion characteristics of known diseases, initially judge it as a lesion, and evaluate the severity of the lesion according to the intensity and distribution of the features.

[0098] Result display and application:

[0099] Generate a visual feature map using the feature extraction results, map the intensities of different features onto the image to assist doctors in intuitively understanding the state of the retina. Display the distribution and intensity of blood vessel features and lesion features through color coding (such as red for blood vessels, green for the macula area, and yellow for the lesion area) or gray-scale mapping.

[0100] Based on the system analysis results and the visual feature map, combined with the patient's medical history and other clinical examination data, doctors conduct a comprehensive diagnosis and formulate corresponding treatment plans.

[0101] The embodiments of the present invention can achieve efficient and accurate acquisition and processing of retinal images, provide detailed retinal status information for doctors, assist doctors in early disease diagnosis and condition monitoring, and are expected to improve the diagnostic accuracy and treatment effect of retinal diseases and improve the quality of life of patients. At the same time, the contact lens integrated design of the system has the advantages of convenient wearing and non-invasiveness, and is more acceptable to patients.

[0102] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description of the method part.

[0103] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visual perception analysis method based on contact lens integrated retinal monitoring, characterized in that: include: Placing a micro-image sensor array on a contact lens facing the eyeball; The retina at the bottom of the eyeball is photographed and collected by the micro-image sensor array to obtain a retinal image; A deep learning algorithm is used to process the retinal image to complete the visual perception task.

2. The visual perception analysis method based on contact lens integrated retinal monitoring according to claim 1, characterized in that: After acquiring the retinal image, the method further includes preprocessing the retinal image.

3. The visual perception analysis method based on contact lens integrated retinal monitoring according to claim 1, characterized in that: The retinal image is processed using a deep learning algorithm, including: Extract LBP-HF texture features from retinal images; The extracted LBP-HF texture feature matrix is ​​input into the convolutional neural network for training; Construct a retinal feature extraction model based on convolutional neural network; Convolutional neural networks use multiple convolutional layers to further extract features from retinal images; Perform visual perception analysis based on the feature extraction results.

4. The visual perception analysis method based on contact lens integrated retinal monitoring according to claim 3 is characterized in that: Extract LBP-HF texture features from retinal images, including: Calculate the original pattern LBP value of the global retinal image; Based on the original mode LBP value, the rotation-invariant LBP mode and the equivalent Uniform LBP mode are fused to obtain the rotation-invariant equivalent mode operator; The rotation invariant equivalent is histogrammed, the histogram is statistically analyzed and discrete Fourier transform is performed; the multi-scale local binary pattern Fourier histogram feature, namely LBP-HF texture feature, is calculated.

5. The visual perception analysis method based on contact lens integrated retinal monitoring according to claim 3, characterized in that: The convolutional neural network includes a convolutional layer C1, a pooling layer S1, a convolutional layer C2, a pooling layer S2, a convolutional layer C3, a pooling layer S3, a convolutional layer C4, a fully connected layer F1, and a softmax layer which are cascaded in sequence.

6. The visual perception analysis method based on contact lens integrated retinal monitoring according to claim 1, characterized in that: Before the micro-image sensor array is used to capture the retina at the bottom of the eyeball, the method further includes: integrating a micro-optical lens in front of the micro-image sensor array to focus the light; Dynamically adjust the exposure time of the micro-image sensor according to the intensity of the light reflected from the retina; The retinal image acquisition frequency was set according to the eye movement speed and the dynamic range of retinal image changes.

7. A visual perception analysis system based on contact lens integrated retinal monitoring, applying the visual perception analysis method based on contact lens integrated retinal monitoring according to any one of claims 1 to 6, characterized in that: It includes: a micro-image sensor array module, which is arranged on a contact lens facing the eyeball; The shooting angle of view of the micro-image sensor array module is synchronized with the eyeball angle of view; The micro-image sensor array module is used to capture the retina at the bottom of the eyeball and obtain a retinal image; A transmission module, connected to the image processing module and the micro-image sensor array module, respectively, and used for transmitting the retinal image acquired by the micro-image sensor array module to the image processing module; The image processing module is used to process the retinal image using a deep learning algorithm to complete the visual perception task.

Images