Intelligent glasses and intelligent monitoring system and method for eye using behaviors of intelligent glasses

Through smart glasses and their eye behavior intelligent monitoring system, eye data is collected and analyzed in real time, reminders and recommendations are provided, and the problem of inability to effectively monitor and correct eye habits in the existing technology is solved, and the use of eyes for children and adolescents is realized. Healthy and efficient supervision and management.

CN119094552AInactive Publication Date: 2024-12-06SEAWAY TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411140591.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing smart glasses cannot effectively monitor and correct the eye use habits of children and adolescents, especially users with poor autonomy, and cannot achieve long-term effective eye use health supervision and management.

Method used

A smart glasses and their eye behavior intelligent monitoring system was designed, including user-side glasses monitoring components, user-side smart devices and cloud servers. Through pattern analysis processing modules, sensor modules and communication modules, they collect and analyze eye data in real time. And through local reminder and remote reminder mechanisms, we provide recommendations for eye use behavior to realize monitors' supervision and management of eye use health data.

Benefits of technology

Real-time reminder and monitoring of the user's eye health status, enhance the monitor's supervision ability, improve the management efficiency of eye health data, and make the system more efficient, flexible and effective.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094552A_ABST
    Figure CN119094552A_ABST
Patent Text Reader

Abstract

The invention provides intelligent glasses and an intelligent eye using behavior monitoring system and method thereof, and the method comprises the steps: obtaining eye using data at a user side, carrying out the mode analysis of the eye using data, and uploading the obtained eye using data and / or a mode analysis result to a cloud server through a user side intelligent device or directly; local reminding and / or eye using behavior recommendation are / is carried out on the user according to the mode analysis result; receiving configuration information of the cloud server and / or the user side intelligent equipment on the component parameters; a mode analysis result of the cloud server side is obtained at the monitoring side, and a user is remotely called or remotely reminded through user side intelligent equipment according to the mode analysis result; and remote parameter setting is carried out on the user side glasses monitoring assembly, and the parameters are uploaded to the cloud server. Omnibearing combination is effectively carried out, the eye using condition of a glasses wearer is supervised, meanwhile, the eye using environment of the glasses wearer is monitored, and omni-bearing, safe and efficient monitoring is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vision monitoring technology, and in particular to a pair of smart glasses and a system and method for intelligently monitoring eye behavior thereof. Background Art

[0002] As a common device, glasses have been integrated into people's lives and helped many people adjust their vision. However, their function is single and they can only allow users to see clearly, but cannot relieve or correct users' eye fatigue and eye habits. The development of smart glasses provides an effective solution to the above problems.

[0003] Existing smart glasses usually have a variety of sensors and prompt devices on the glasses, such as light sensors, distance sensors and posture sensors, etc., which are used to collect eye data of the wearer and process the eye data. When the eye data exceeds the normal range, such as in weak light, when reading or writing, the eyes are too close to the book, or when the posture is not right, the vibration motor on the glasses will give a vibration warning to prompt the wearer to improve the eye environment or correct the eye posture, thereby correcting the user's eye habits and reducing the user's eye fatigue.

[0004] Since the users of smart glasses are mainly children and teenagers, the self-awareness of this user group is relatively poor, and the effect of status correction by only giving warnings is poor; since the functions of smart glasses with sensors and prompt devices are relatively simple, the effect cannot meet the expectations, and the monitors cannot monitor and correct their wrong postures in real time, nor can they understand their eye conditions in time and conduct long-term monitoring and analysis of their eye conditions, which cannot meet the real monitoring needs of smart glasses for the user group.

[0005] After searching, we found:

[0006] The Chinese invention patent application "A Youth Vision Monitoring System" with publication number CN111583602A includes a processor, a distance measurement module, a light detection module, a voice module, a timing module, a sitting posture correction module, and a memory; the distance measurement module is connected to the processor, and sends out ultrasonic waves through the distance measurement module, and automatically detects the returned ultrasonic waves, detects the time from the ultrasonic wave to the return, calculates the distance of the user's eyes, and detects whether the user is at a safe distance. The youth vision monitoring system will remind the child through voice for reading distances that are too close, too dark light, reading behavior that lasts too long, and bad sitting postures, and remind the child to correct them in real time to form good eye habits. For the child's eye habits during the day, parents can view the eye data report through their mobile phones to help parents effectively guide their children and teach them to develop good eye habits. However, the system only monitors and voice reminds reading distances, too dark light, reading behavior that lasts too long, and bad sitting postures. The effect is not obvious for children with poor autonomy, and it is difficult for parents, school doctors, teachers and other guardians to effectively and systematically supervise and understand the child's eye condition in the long term. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a smart pair of glasses and an intelligent monitoring system and method for eye behavior thereof.

[0008] According to one aspect of the present invention, there is provided an intelligent eye behavior monitoring system, comprising: a user-side glasses monitoring component, a user-side intelligent device, a cloud server, and a plurality of monitoring intelligent terminal devices; wherein:

[0009] The user-side glasses monitoring component is used to obtain the user's eye data, perform pattern analysis on the eye data, and upload the obtained eye data and / or pattern analysis results to the cloud server through the user-side smart device or directly in a manner set by the cloud server; provide local reminders and / or eye behavior recommendations to the user based on the pattern analysis results; and receive configuration information on component parameters from the cloud server and / or the user-side smart device;

[0010] The user-side smart device is used to serve as a communication intermediate end between the user-side glasses monitoring component and the cloud server, to set local reminder parameters for the user-side glasses monitoring component, and / or to receive remote calls and / or reminders from the monitor's smart terminal device;

[0011] The cloud server is used to establish a communication connection with the user-side smart device and then communicate with the user-side glasses monitoring component, or directly communicate with the user-side glasses monitoring component to achieve data interaction with the user-side glasses monitoring component;

[0012] The monitor's smart terminal device is used to obtain the pattern analysis results of the cloud server, and to remotely call the user or remotely remind the user through the user-side smart device according to the pattern analysis results.

[0013] Preferably, the user-side glasses monitoring component includes: a mode analysis and processing module, a power management module, a communication module, a local reminder module and a sensor module; wherein:

[0014] The pattern analysis processing module is connected to the power management module, the communication module, the local reminder module and the sensor module respectively, and is used to obtain the data collected by the sensor module and perform data analysis to obtain a pattern analysis result;

[0015] The power management module is used to perform battery charging management, low-power sleep management, and current and voltage management of different modules;

[0016] The communication module is used to interact with the user-side smart device and / or the cloud server;

[0017] The local reminder module is used to remind the user locally according to the pattern analysis result;

[0018] The sensor module includes: a distance sensor, a light intensity sensor, a 9-axis sensor and / or a temperature and humidity sensor, which is used to obtain relevant sensor data as eye data.

[0019] Preferably, the pattern analysis and processing module comprises: a control module, a receiving module, a conversion module, an analysis module and a storage module; wherein:

[0020] The control module is used to control the execution behaviors of the receiving module, the conversion module and the analysis module;

[0021] The receiving module is used to receive various sensor data collected by the sensor component;

[0022] The conversion module is used to perform data fusion and conversion on the various sensor data;

[0023] The analysis module is used to perform pattern analysis on the converted sensor data and obtain irregular eye behavior according to the pattern analysis results;

[0024] The storage module is used to locally store various sensor data acquired by the receiving module;

[0025] Preferably, the communication module includes a Bluetooth module and a wireless communication module; wherein the Bluetooth module is used for synchronous data interaction between the user-end glasses monitoring component and the user-end smart device; and the wireless communication module is used for data interaction between the user-end glasses monitoring component and the cloud server.

[0026] Preferably, the user-side glasses monitoring component also includes: a voice recognition module, which is used to monitor the audio data stream through a set voice keyword recognition algorithm, and after monitoring the specified keyword, send alarm data to the cloud server through the user-side smart device or directly send alarm data to the user-side smart device.

[0027] Preferably, the analysis module is used to perform pattern analysis on each converted sensor data, and obtain irregular eye behavior according to the pattern analysis result, including:

[0028] Provide a pre-trained IPSO-LSTM model to extract features from the converted sensor data and obtain irregular eye behavior;

[0029] Provide a pre-trained Apriori algorithm model to perform behavioral analysis on the converted sensor data to obtain the user's eye behavior habits;

[0030] The thresholds of various sensors are set, and the converted sensor data are compared with the corresponding thresholds to obtain the irregular eye behavior.

[0031] Preferably, providing a pre-trained Apriori algorithm model includes:

[0032] Set the minimum weighted support threshold W m suph and minimum weighted confidence threshold W m conf;

[0033] Scan a set of data item sets in the eye database D of the cloud server, calculate the weighted support Wsuph of the data item set, and generate a candidate item set C1; wherein the weighted support Wsuph is:

[0034] Wsuph(X)=max{w1,w2,...,w n}*Wsup(X)

[0035] Where X is the data item set, max{w1, w2, ..., w n} is the maximum horizontal weight item of the data item set X, Wsup(X) is the support; the eye database D includes relevant data in the public data set and eye data generated in the actual use process;

[0036] Determine whether the weighted support of the candidate item set C1 is greater than or equal to the minimum weighted support threshold W m suph; if yes, generate the frequent item set L1; if no, prune the candidate item set C1;

[0037] Connect and iterate the frequent item set L1 to generate a candidate item set C2;

[0038] Determine whether the weighted support of the candidate item set C2 is greater than or equal to the minimum weighted support threshold W m suph; if yes, generate the frequent item set L2; if no, prune the candidate item set C2;

[0039] And so on, until all the data in the database are scanned, a strongly associated frequent item set is obtained, the confidence of the strongly associated frequent item set rules is calculated, the factors leading to eye health problems are determined according to the size of the confidence, the eye behavior habits are obtained, and relevant eye behavior recommendations are given.

[0040] According to a second aspect of the present invention, there is provided a pair of smart glasses, which use any one of the above-mentioned intelligent eye behavior monitoring systems of the present invention to obtain the eye data of the user, and remind the user of the eye behavior, and / or make eye behavior recommendations; wherein:

[0041] The user-side glasses monitoring component of the eye behavior intelligent monitoring system is installed on the user's glasses through a detachable glasses clip.

[0042] According to a third aspect of the present invention, there is provided a method for intelligently monitoring eye behavior, which uses the intelligent eye behavior monitoring system described in any one of the above-mentioned embodiments of the present invention to obtain the user's eye data, and to provide eye behavior reminders and / or eye behavior recommendations to the user.

[0043] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0044] The smart glasses and the intelligent eye behavior monitoring system and method provided by the present invention can not only provide real-time reminders of the user's eye health status, but also enable the monitor to effectively supervise and manage the user's eye health data, and monitor the user's safety while efficiently monitoring the eye health status. The entire system is more efficient, flexible and effective.

[0045] The smart glasses and their eye behavior intelligent monitoring system and method provided by the present invention adopt a steady-state Kalman filtering method to solve the problem of excessive user data and consumption of a large amount of computing resources. A pre-calculated fixed gain matrix is ​​used, and there is no need to update the gain matrix at each step, which greatly reduces the amount of calculation and facilitates subsequent operations.

[0046] The smart glasses and the smart eye behavior monitoring system and method provided by the present invention adopt a pre-trained Apriori algorithm model to solve the shortcoming that the algorithm will generate too many candidate sets, providing convenience for subsequent calculations.

[0047] The smart glasses and the smart eye behavior monitoring system and method provided by the present invention provide a solution to the loss of equipment. The communication mode between the smart glasses and the cloud platform can be switched through the cloud platform. When the user is not wearing the watch, the watch is out of power or the watch is lost, the user can directly switch the communication mode of the glasses to a single-channel data communication mode for direct data interaction with the cloud platform through the operation on the terminal device side. Connecting the glasses through the terminal device ensures that user data will not be lost. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0049] Figure 1 The figure is a schematic diagram of the working principle of the intelligent monitoring system for eye behavior in a specific application example of the present invention.

[0050] Figure 2 The figure is a training flow chart of the Apriori algorithm model in a specific application example of the present invention.

[0051] Figure 3 The figure is a calculation flow chart of a steady-state Kalman filtering method in a specific application example of the present invention. DETAILED DESCRIPTION

[0052] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0053] Smart glasses, as eye monitoring devices, have begun to be widely used among children and adolescents. However, existing smart glasses only observe and prompt the user's eye data by setting sensors and prompt devices on the glasses, and there is a problem that the monitor cannot monitor the child's eye health problems in a timely and effective manner.

[0054] In response to the above problems, an embodiment of the present invention provides an intelligent monitoring system for eye health data. While satisfying the functions of ordinary smart glasses, the system can also enable monitors (such as doctors, teachers, parents, etc.) in addition to the wearers of the glasses to monitor and manage the eye health data of the wearers in real time, systematically and effectively.

[0055] Specifically, the eye health data intelligent monitoring system provided in this embodiment may include: a user-side glasses monitoring component, a user-side intelligent device, a cloud server, and several monitoring intelligent terminal devices. Among them:

[0056] The user-side glasses monitoring component is used to obtain the user's eye data, perform pattern analysis on the eye data, and upload the obtained eye data and / or pattern analysis results to the cloud server through the user-side smart device or directly in the manner set by the cloud server; provide local reminders and / or eye behavior recommendations to the user based on the pattern analysis results; and receive configuration information on component parameters from the cloud server and / or the user-side smart device. In this embodiment, the user-side glasses monitoring component uploads data to the cloud server by communicating with the user's smart device, or it can directly upload data to the cloud server; therefore, in this embodiment, the user-side glasses monitoring component is simultaneously provided with components that support communication with the user's smart device and components that communicate with the cloud server. Moreover, in a specific embodiment under a specific application scenario, the unbinding between glasses, user smart devices, and cloud servers is achieved by the user's glasses monitoring component directly communicating with the cloud server, thereby making the application scenario more extensive and flexible.

[0057] The user-side smart device is used as a communication intermediate between the user-side glasses monitoring component and the cloud server to set local reminder parameters for the user-side glasses monitoring component and / or receive remote calls and / or reminders from the monitor's smart terminal device.

[0058] The cloud server is used to establish a communication connection with the user-side smart device and then communicate with the user-side glasses monitoring component, or directly communicate with the user-side glasses monitoring component to achieve data interaction with the user-side glasses monitoring component.

[0059] The monitor's smart terminal device is used to obtain the pattern analysis results from the cloud server, and remotely call the user through the user's smart device based on the pattern analysis results, or remotely remind the user.

[0060] In order to more comprehensively monitor the user's eye conditions, in some preferred embodiments, the user-side glasses monitoring component may further include: a mode analysis and processing module, a power management module, a communication module, a local reminder module and a sensor module; wherein:

[0061] The mode analysis processing module is connected to the power management module, the communication module, the local reminder module and the sensor module respectively, and is used to obtain the data collected by the sensor module and perform data analysis to obtain the mode analysis result;

[0062] Power management module, used for battery charging management, low-power sleep management and current and voltage management of different modules;

[0063] Communication module, used for data interaction with user-side smart devices and / or cloud servers;

[0064] A local reminder module is used to remind users locally based on the pattern analysis results;

[0065] The sensor module includes: a distance sensor, a light intensity sensor, a 9-axis sensor and / or a temperature and humidity sensor, which is used to obtain relevant sensor data, that is, to obtain eye data.

[0066] In some preferred embodiments, the above-mentioned pattern analysis and processing module may further include: a control module, a receiving module, a conversion module, an analysis module and a storage module; wherein:

[0067] A control module, used to control the execution behaviors of the receiving module, the conversion module and the analysis module;

[0068] A receiving module, used for receiving various sensor data collected by the sensor component;

[0069] The conversion module is used to fuse and convert the data of various sensors;

[0070] An analysis module is used to perform pattern analysis on the converted sensor data and obtain irregular eye behavior based on the pattern analysis results;

[0071] A storage module, used for locally storing various sensor data acquired by the receiving module;

[0072] In some preferred embodiments, the above-mentioned communication module may further include: a Bluetooth module and a wireless communication module; wherein the Bluetooth module is used for synchronous data interaction between the user-side glasses monitoring component and the user-side smart device; the wireless communication module is used for data interaction between the user-side glasses monitoring component and the cloud server, thereby realizing a single-channel data communication mode.

[0073] In some preferred embodiments, the above-mentioned local reminder module may further include: a vibration component and an audio-visual component. The pattern analysis and processing module triggers the vibration component and / or the audio-visual component to act according to the irregular eye behavior, and makes local reminders to the user; wherein the reminder parameters of the vibration component and / or the audio-visual component are set through the user-side smart device.

[0074] In some preferred embodiments, the sensor module may further include: a sensing switch, which controls the on and off of the user-side glasses monitoring component according to the user's behavior.

[0075] In some preferred implementations, the power management module may further include: a power indicator light.

[0076] In some preferred embodiments, the above-mentioned user-end glasses monitoring component may further include: a voice recognition module, which is used to monitor the audio data stream through a set voice keyword recognition algorithm, and after monitoring the specified keyword, send alarm data to the cloud server through the user-end smart device or directly send alarm data to the user-end smart device.

[0077] In some preferred embodiments, the analysis module is used to perform pattern analysis on the converted sensor data, and obtain the irregular eye behavior according to the pattern analysis results, and may further include:

[0078] IPSO-LSTM module, which is used to provide a pre-trained IPSO-LSTM model to extract features from the converted sensor data and obtain irregular eye behavior;

[0079] Apriori algorithm module, which is used to provide a pre-trained Apriori algorithm model to perform behavioral analysis on the converted sensor data to obtain the user's eye behavior habits;

[0080] The threshold judgment module is used to set the thresholds of various sensors, and compare the converted sensor data with the corresponding thresholds to obtain irregular eye behavior.

[0081] Due to the difficulty of fusing various types of sensor data, most algorithms require a lot of computing resources and are difficult to implement in hardware or embedded systems. In order to solve the above problems, the steady-state Kalman filter method uses a pre-calculated fixed gain matrix and does not need to update the gain matrix at each step, which greatly reduces the amount of calculation and is suitable for real-time processing. The algorithm is relatively simple and easy to implement in hardware or embedded systems. Based on this, in some preferred embodiments, the above-mentioned conversion module is used to perform data fusion and conversion on various sensor data, and can further include:

[0082] The steady-state Kalman filter method is used to fuse data from different sensors;

[0083] Use the steady-state Kalman filter method to directly calculate the optimal variance and optimal Kalman gain at each moment, so that the optimal estimate under steady-state conditions can be achieved at each moment to obtain the fused sensor data;

[0084] The fused sensor data is preprocessed by means of conversion.

[0085] Further preferably, the specific operation method of fusing data from different sensors using the steady-state Kalman filtering method is as follows:

[0086] Initialization: First, define the state variables and initialize the state estimate and error covariance matrix of each sensor.

[0087] Sensor data collection: Collect data from each sensor at each time step. This data will serve as the input of the Kalman filter;

[0088] Prediction step: predict the next state and error covariance based on the previous state and the system model. This involves using the state transition model to estimate the current state and projecting the error covariance into the future.

[0089] Measurement Update: For each sensor, calculate the Kalman gain, which determines how much the prediction should be adjusted based on the new measurement. The Kalman gain is calculated from the measurement noise covariance and the prediction error covariance. Update the state estimate using the new measurement and the Kalman gain. This step uses the actual measurement to adjust the predicted state to correct for any bias. Update the error covariance matrix to reflect the reduced uncertainty after the measurement update.

[0090] Sensor Data Fusion: Combine the updated estimates from all sensors to produce a single fused estimate. This can be achieved by weighting the estimates based on their respective accuracy or covariance. A steady-state Kalman filter ensures that the fusion process is optimized to minimize the overall estimation error.

[0091] Steady-state considerations: Ensure that at each time step, the system reaches a steady state such that the Kalman gain and error covariance converge to constant values. This ensures the stability and reliability of the system, providing accurate estimates.

[0092] By following these steps, the method integrates data from multiple sensors to produce a robust and accurate estimate of the state variables, leveraging the strengths of each sensor and mitigating their respective weaknesses.

[0093] The fused sensor data is preprocessed by means of conversion.

[0094] Further preferably, regarding the conversion, the following technical means can be used:

[0095] Wavelet Transform:

[0096] Purpose: To analyze data at different frequency levels and capture both time and frequency information.

[0097] How to use: Good for detecting changes or anomalies in sensor data, helps in denoising and extracting features that are difficult to see in the time domain.

[0098] Fourier Transform:

[0099] Purpose: Convert data from time domain to frequency domain.

[0100] How to use: Helps identify the main frequencies in sensor data and filter out noise.

[0101] Principal Component Analysis (PCA):

[0102] Purpose: Reduce the dimensionality of the data while retaining the most important variance.

[0103] How to use: Helps to compress data and identify the main components that explain the variation in sensor data.

[0104] Normalization and Standardization:

[0105] Purpose: To rescale data onto a common scale without distorting differences in numerical ranges.

[0106] How to use: Ensure comparability of sensor data and prevent data from any single sensor from dominating due to scale differences.

[0107] Data Smoothing Techniques:

[0108] Purpose: To remove noise and make data smoother so that trends can be analyzed more clearly.

[0109] How to use: Techniques like moving average or Gaussian smoothing can be applied to sensor data to reduce random noise.

[0110] The main purpose of these transformations is to preprocess the sensor data to improve its quality and relevance for subsequent analysis. By transforming the data, it is easier to identify patterns, remove noise, and extract meaningful features, ultimately achieving more accurate and reliable sensor fusion results.

[0111] In order to solve complex data classification and time series prediction problems, in some preferred embodiments, the IPSO-LSTM module provides a pre-trained IPSO-LSTM model, which may further include:

[0112] Training samples are constructed based on the data in the storage module, and a three-layer neural network is constructed for each type of sample data (i.e., each type of composition), and the training samples are trained through the neural network; the data in the storage module include reading posture, reading distance, reading time and / or light intensity; the training samples are combined vectors composed of grayscale features, HOG features, LBP features, and Haar features of the image; the first hidden layer in the three-layer neural network learns edge features, the second hidden layer learns shape features composed of edge features, and the third hidden layer learns pattern features composed of shapes, and finally a judgment result is classified as to whether the pattern satisfies this type of composition.

[0113] Furthermore, different types of image combinations are learned and recognized through a three-layer neural network structure. Specifically, in this context, each type of composition represents a certain combination of image features, including grayscale features, HOG features, LBP features, and Haar features. Through the training of the neural network, these features are combined into a whole to represent and classify specific types of images or patterns. Among them: each feature is obtained by preprocessing and feature extraction of sensor data. The specific steps are as follows:

[0114] Grayscale features: directly extracted from the image by converting the color image into a grayscale image to obtain the grayscale value of each pixel.

[0115] HOG feature (Histogram of Oriented Gradients): describes the local shape characteristics of the image by calculating the gradient direction histogram of the local area in the image.

[0116] LBP features (Local Binary Patterns): Generate binary patterns by comparing the grayscale values ​​of each pixel with the pixels in its neighborhood to reflect the texture information of the image.

[0117] Haar features: Extract features of edges, lines, and other simple shapes of an image by applying Haar filters.

[0118] These features can be extracted from the raw sensor data through corresponding image processing algorithms and then input into the neural network for training and learning.

[0119] The judgment result refers to the frequent item sets and their support generated by the Apriori algorithm model, which are used to determine whether the mined patterns or rules meet user needs or set standards. These results are used for further analysis and decision-making.

[0120] The traditional Apriori algorithm has the problem of scanning the database too many times, generating a large number of candidate sets, and significantly increasing the operation time. In order to effectively process a large amount of data and extract valuable association rules, this step uses a pre-trained Apriori algorithm model. Based on this, in some preferred embodiments, the above-mentioned Apriori algorithm module provides a pre-trained Apriori algorithm model, and can further include:

[0121] Set the minimum weighted support threshold W m suph and minimum weighted confidence threshold W m conf;

[0122] Scan a set of data items in the eye database D of the cloud server, calculate the weighted support Wsuph of the data items, and generate the candidate item set C1; where the weighted support Wsuph is:

[0123] Wsuph(X)=max{w1,w2,...,w n}*Wsup(X)

[0124] Where X is the data item set, max{w1, w2, ..., w n} is the maximum horizontal weight item of data item set X, and Wsup(X) is the support; database D is a comprehensive dataset composed of the dataset used by the pre-trained model during training and the dataset collected by users during later use. It contains information in the public dataset and also contains the eye data generated by users during actual use.

[0125] Determine whether the weighted support of the candidate item set C1 is greater than or equal to the minimum weighted support threshold W m suph; if yes, generate frequent item set L1; if no, prune candidate item set C1;

[0126] Connect and iterate the frequent item set L1 to generate the candidate item set C2;

[0127] Determine whether the weighted support of candidate item set C2 is greater than or equal to the minimum weighted support threshold W m suph; if yes, generate frequent item set L2; if no, prune candidate item set C2;

[0128] And so on, until all the data in the database are scanned, and strongly associated frequent item sets are obtained, the confidence of the strongly associated frequent item set rules is calculated, and the factors leading to eye health problems are determined according to the size of the confidence, the eye behavior habits are obtained, and relevant eye behavior recommendations are given.

[0129] Furthermore, in the Apriori algorithm, the frequent item set L1 is self-connected to generate the candidate item set C2, and then the support of each candidate item in C2 is calculated to select the item set that meets the minimum support threshold as the new frequent item set L2. The specific steps are as follows:

[0130] Perform self-connection on the frequent item set L1 to generate the candidate item set C2;

[0131] Calculate the support for each candidate item set C2;

[0132] Item sets whose support is greater than or equal to the set threshold are selected as new frequent item sets L2.

[0133] Furthermore, confidence is calculated by the support of frequent itemsets, which indicates the probability of an item set appearing under given conditions. The calculation formula is: Confidence (A→B) = Support (A∪B) / Support (A)

[0134] Among them, support (A∪B) is the number of data items that contain both item A and item B, and support (A) is the number of data items that contain item A.

[0135] Furthermore, the judgment process includes calculating the support and confidence of each candidate item set and comparing it with the preset minimum weighted confidence threshold Wmconf. If the confidence of a candidate item set is greater than or equal to Wmconf, it is considered that the item set meets the conditions and can be used as a strong association rule. This setting is used to screen out important and meaningful association rules and avoid interference from irrelevant or noise data.

[0136] The result obtained by setting a threshold is usually to filter frequent itemsets based on a predefined support or confidence threshold. The judgment result here is to gradually filter out frequent itemsets that meet the conditions by calculating and comparing the support of multiple candidate itemsets. The process is more dynamic and detailed, and can better discover hidden patterns and associations.

[0137] In some preferred implementations, the cloud server and / or the user-side smart device corresponds to different working states according to the preset user eye health state, and sets the configuration information of the component parameters.

[0138] In some preferred implementations, a first application module is arranged on the user-side smart device, and the first application module is used to:

[0139] Synchronously receive eye data and / or pattern analysis results transmitted by the user-side glasses monitoring component, and upload them to the cloud server;

[0140] Set local reminder parameters for the user-side glasses monitoring component.

[0141] In some preferred implementations, a second application module is arranged on the monitor's smart terminal device, and the second application module is used to:

[0142] Send data requests to the cloud server and receive relevant data sent by the cloud server;

[0143] Remotely set parameters for the user-side glasses monitoring component and send them to the cloud server;

[0144] Bind the user's smart device;

[0145] In the absence of the user-side smart device, the data interaction between the user-side glasses monitoring component and the cloud server is switched to a single-channel data communication mode.

[0146] In some preferred implementations, the user-side smart device is a smart phone watch.

[0147] In some preferred implementations, the monitor's intelligent terminal device is a smart phone or a tablet computer.

[0148] The technical solution and its working principle provided by the above embodiment of the present invention are further described in detail below in conjunction with a specific application example.

[0149] like Figure 1 As shown, in this specific application example, the user-side smart device adopts a smart phone watch, and the monitor's smart terminal device adopts a smart phone or tablet computer. By clipping the user-side glasses monitoring component on the ordinary glasses used by the user, it is possible to implement eye behavior supervision of users (such as teenagers, children, etc.) by monitors of different roles (such as parents, teachers, school doctors, etc.), wherein the number of monitors' smart terminal devices can be one or more, and there is no limitation on this; an independent, detachable monitoring component is set on the glasses, and the monitoring component integrates a pattern analysis and processing module, a power management module, a communication module, a local reminder module (such as a vibration motor, an audio-visual component, etc.) and a sensor component. The pattern analysis and processing module is respectively connected to the power management module, the communication module, the vibration motor, and the sensor component for collecting sensor data and performing data processing; the above-mentioned sensor component includes a distance sensor, a light intensity sensor, a 9-axis sensor, a temperature and humidity sensor, and a contact sensing switch.

[0150] The sensor module is used to collect sensor data and send the sensor data to the pattern analysis and processing module. The pattern analysis and processing module uploads the processing results to the corresponding application module on the smart watch, and the data is further uploaded to the cloud server through the corresponding application module on the smart watch. When the cloud server receives the data request from the monitor's smart terminal device, it provides the relevant data to the monitor's smart terminal device for presentation. The corresponding application module is installed on the monitor's smart terminal device.

[0151] The intelligent eye health data monitoring system provided by this specific application example is aimed at children, teenagers and other groups who cannot control their eye behavior in real time. It can better supervise the users' eye use and change their habits. Through long-term and systematic monitoring, it assists scientific eye use and avoids problems caused by eye habits such as myopia.

[0152] In this specific application example, the monitoring component can automatically select different working states according to the user's eye status, including work / study time, whether the eyes are used more at close range or at long distance, myopia, astigmatism, hyperopia, etc. After the user sets the eye status, it automatically adjusts to the corresponding working state according to the eye status. Different working states have different sensor data collection frequencies, sensor working numbers, etc.; users can also choose the working state by themselves. For example, it includes eye use time in different modes, indoor and outdoor activity time, sunlight intake, exercise step count statistics, eye use distance, cervical spine posture, etc.

[0153] Different working states are automatically selected according to the user's eye state, and a model training method is provided.

[0154] In this specific application example, the Apriori algorithm is used to mine the eye behavior of teenagers, such as the eye habits of users in different time periods, and to find out which combinations of behaviors may cause vision problems. By identifying frequent eye usage patterns, the system can better monitor the user's unhealthy eye usage patterns and make health recommendations. The core idea of ​​the Apriori algorithm is to find the frequent item level, extract association rules from the frequent item set, and use an iterative method to gradually increase the size of the item set to find all frequent item sets. Then select a working mode suitable for the user.

[0155] The Apriori algorithm is a classic association analysis algorithm used to discover frequent item sets and association rules in data sets. It is a method based on the generation of frequent item sets, which generates candidate item sets through iteration, and uses the support of candidate item sets to screen out frequent item sets. The traditional Apriori algorithm uses the same support when performing data mining, and believes that the risk factors in the database are independent of each other and have no difference in importance, but in fact there is a primary and secondary distinction between risk factors. The traditional Apriori algorithm can only obtain association results by scanning the risk factors with high frequency in the database. It is difficult to reflect the impact of risk factors with low frequency but high importance on association rules. For this reason, weight data is introduced to improve the support into weighted support. The present invention uses a horizontal weighted method to improve the support. The comprehensive weight is the result obtained by combining subjective and objective weights. The data is more scientific and more in line with the actual situation. The weighted support formula is as follows:

[0156] Wsuph(X)=max{w1,w2,...,w n}*Wsup(X)

[0157] Among them, X is the item set, max{w1, w2, ..., w n} is the maximum horizontal weight of X. Wsuph(X) is the weighted support, and Wsup(X) is the support.

[0158] The improved algorithm is different from the traditional Apriori algorithm process, which is reflected in the two steps of connection and pruning. The definition of horizontal weighting makes the pruning step slightly different from the traditional algorithm, and can only be judged by the unique horizontal weighted support. The specific weighted Apriori algorithm process is as follows Figure 2 shown.

[0159] Figure 3 In, W m suph is the minimum weighted support, W m conf is the minimum weighted confidence. Database D is the eye data of teenagers in the smart glasses cloud server.

[0160] First, a candidate item set (Ck) is generated. A candidate item set of size k is generated from a frequent item set of size k-1. If the first k-2 items of two frequent item sets of size k-1 are the same, they can be merged to generate a candidate item set of size k. Then, pruning is performed to remove candidate item sets whose subsets are not frequent item sets. The database is scanned and the support (Support) of each candidate item set is calculated. Candidate item sets whose support is greater than or equal to the minimum support threshold are retained to obtain frequent item sets. Association rules are generated from frequent item sets, that is, the correlation between various behaviors of teenagers in the process of using their eyes and the resulting vision health problems. For each frequent item set, all possible non-empty subsets are generated, and then the confidence (Confidence) of the rule is calculated. According to the size of the confidence, the factors that lead to the vision health problems of teenagers are determined, that is, irregular eye use behaviors, such as too close distance, incorrect posture, too long time, etc.

[0161] At the same time, due to the differences in gender, age, and living environment among young people, a single calculation result cannot give accurate mode recommendation options. By using the large-scale data set D of the cloud server and the new data set generated by new users wearing smart glasses, we can continuously combine and iterate to obtain more accurate association rules. As the user base continues to increase, the cloud server data will further expand, allowing smart glasses to give intelligent mode recommendations for different youth user groups, better protecting the health of teenagers' eyes.

[0162] After the monitor (such as a teacher or parent) detects the user's bad eye habits on the smart terminal, such as long-term close-up use of the eyes, poor lighting, maintaining the same posture for more than a certain time, and other predetermined conditions, the monitor can use the smart watch to remotely call the user or remotely remind the user, thereby effectively supervising the user's eye habits.

[0163] The pattern analysis and processing module is used to realize the recognition of various eye activity patterns, which includes a control module, a receiving module, a conversion module, an analysis module and a storage module; the control module is a single-chip microcomputer, which is used for the execution behavior control of the smart glasses; priority scheduling is used to manage the state conversion and task scheduling between each module. The receiving module is used to obtain the information detected by the sensor group; high-precision sensors and real-time data acquisition technology are used. Data synchronization: to ensure the time synchronization of multi-sensor data, timestamps or synchronization signals can be used. The conversion module is used to convert and read the information detected by the sensor group for the control module to select and issue instructions; for example, wavelet transform is used in the present invention to process and analyze eye movement signals. The analysis module is used to analyze and predict the information detected by the sensor group; the LSTM algorithm is used to predict the detected information, and then the feature data is analyzed to identify eye patterns such as fatigue. In order to solve the shortcomings of the traditional LSTM algorithm, the particle swarm optimization improved algorithm IPSO is selected in some preferred embodiments of the present invention to construct an IPSO-LSTM prediction model. The storage module is used to store the information detected by the sensor group; the data samples in the storage module are used as training samples, and a three-layer neural network is constructed for each type of composition. The training samples are trained through the neural network. The data samples include reading posture, reading distance, reading time, and light intensity; the training samples are combined vectors composed of the grayscale features, HOG features, LBP features, and Haar features of the image; the first hidden layer in the three-layer neural network learns the features of the edge, the second hidden layer learns the features of the shape composed of the edge, and the third hidden layer learns the features of the pattern composed of the shape, and finally classifies the judgment result of whether the pattern meets this composition.

[0164] The power management module is used for battery charging management, low-power sleep management, and current and voltage management of different modules; the power management module includes a power module and a detection module; the power module is used to provide power to the smart glasses; the detection module is used to obtain the usage mode of the smart glasses, and then select one of the battery charging management, low-power sleep management, and current and voltage management of different modules through the mode analysis and processing module.

[0165] A distance sensor is used to accurately measure the reading distance to detect eye habits. A miniature, three-axis measurement, high-sensitivity vibration sensor is selected, and a design method for an ultra-compact fiber Bragg grating three-axis vibration sensor with an interlaced combination of discrete components is proposed.

[0166] The light intensity sensor is used to collect the ambient light intensity to analyze the light intensity.

[0167] The vibration motor is used to provide real-time warnings and provide alarm vibrations when the posture is incorrect, the eyes are used too close, or the eyes are used for too long.

[0168] 9-axis sensor, which is used for acceleration and gyroscope to capture human motion to detect reading posture.

[0169] Temperature and humidity sensors are used to detect the temperature and humidity of the environment and to conduct comprehensive analysis of indoor and outdoor scenes.

[0170] In this specific application example, a Kalman filter is used to fuse data from different sensors to provide a smooth and accurate estimation result. Among them, a steady-state Kalman filter method is used to directly calculate the optimal variance and optimal Kalman gain at each moment, so that the optimal estimation under steady-state conditions can be achieved at each moment. Specifically, Figure 3 As shown:

[0171] Step 1: Filter calculation loop

[0172] Step 1.1: State prediction

[0173] According to the state estimation of the previous time step k-1 Predict the state at the current time step k:

[0174]

[0175] Among them, a k|k-1 It is the state transfer coefficient, which represents the transfer relationship of the state from time step k-1 to time step k.

[0176] Step 1.2: Error covariance prediction

[0177] Predict the error covariance P of the current time step k k|k-1 , which is the variance of the estimation error:

[0178]

[0179] Among them, P k-1 is the error covariance of the previous time step k-1, Q k-1 is the process noise covariance, which represents the uncertainty in the prediction process.

[0180] Step 1.3: Measure residuals

[0181] Calculate the measurement residual Z at the current time step k k , that is, the actual measured value z k Difference from predicted measurement:

[0182]

[0183] Among them, z k is the actual measurement at time step k.

[0184] Step 1.4: Kalman Gain

[0185] Calculate the Kalman gain K k,stable , used to adjust the forecast status:

[0186]

[0187] Among them, R k is the measurement noise covariance, T is the constant parameter of the Kalman filter, P stable is the steady-state error covariance.

[0188] Step 1.5: Status Update

[0189] Update the state estimate at the current time step k using the Kalman gain

[0190]

[0191] Step 1.6: Error covariance update

[0192] Update the error covariance P of the current time step k k :

[0193] P k =(1-K k,stable ) k|y-1

[0194] Step 2, gain calculation loop

[0195] Step 2.1: Steady-state error covariance calculation

[0196] Calculate the steady-state error covariance P stable , used to calculate the Kalman gain:

[0197]

[0198] Step 2.2: Steady-state Kalman gain calculation

[0199] Calculate the steady-state Kalman gain K using the steady-state error covariance stable :

[0200]

[0201] The above steady-state Kalman filter method:

[0202] In the prediction stage: the state and error are predicted using the state transfer equation based on the state and error covariance of the previous time step.

[0203] In the update phase: the difference between the actual measured value and the predicted value is used to calculate the measurement residual and the predicted state is corrected using the Kalman gain.

[0204] Calculation in steady state: Ensure the filter has optimal performance in steady state by calculating steady state error covariance and steady state Kalman gain.

[0205] At each time step, these two loops work alternately to ensure the accuracy of sensor data fusion and state estimation, achieving real-time optimal estimation effect.

[0206] Among them, a and a k|k-1 are model parameters, all of which are constants, P k|k-1 is the prediction variance at time k, Q k-1 is the system noise variance, R k is the measurement noise variance, K k,stable is the steady-state gain of the gyroscope at time k, Z k is the observation value input to the filter calculation loop. k is the process noise, and T is the state transfer matrix.

[0207] The communication module includes a Bluetooth module, which is used for data interaction between smart glasses and smart watches.

[0208] In this specific application example, the smart glasses collect sensor data such as acceleration, distance, and light intensity every t1 time, and take the average value of these data every t2 time; when in a Bluetooth connected state with a smart watch, the smart glasses will transmit the historical data collected every t1 time to the watch APP; in a non-connected state, the smart glasses will save the sensor data collected every t1 time locally in the glasses, and upload it after waiting for the Bluetooth connection to be successful.

[0209] The smart glasses also have a battery indicator light, which is an LED light. Intermittent flashing of the LED light indicates low battery, periodic flashing of the LED light indicates charging, and a long-lasting LED light indicates charging is complete.

[0210] The smart glasses are turned on in the following manner: touching the contact sensing switch for 3 seconds, the smart glasses are automatically turned on, the light is on and the vibrator vibrates once for 300ms, and the recording and monitoring of eye behavior begins. In a preferred embodiment of the present invention, after the smart glasses detect that the user is wearing the glasses, the smart glasses are automatically turned on and start recording and monitoring eye behavior.

[0211] The smart glasses can be turned off in the following ways: if the wearer takes off the glasses and remains still for 3 minutes, the glasses will automatically turn off, and they will also automatically turn off when connected to a charger. If the battery level is less than 9%, the vibrator will vibrate for 600ms and the glasses will automatically turn off and enter storage mode to protect the battery.

[0212] When the smart glasses are in working (detection) state, the wearer presses and holds the touch contact sensor switch for 3 seconds, the light turns on for 600ms, the vibrator vibrates for 600ms, and then enters sleep mode. After entering sleep mode, the smart glasses do not perform alarm actions, but record eye behavior normally, and send the manual forced sleep behavior record to the watch APP.

[0213] In this specific application example, the vibration intensity of the smart glasses is set to three levels and can be controlled through the watch APP. The three parameters of distance alarm threshold, head angle alarm threshold, and eye use time alarm threshold can be set through the watch APP.

[0214] The configuration parameters can be rewritten remotely through the cloud server; it should be noted that the above is for illustration only and does not represent the entire scope of protection of the present invention. For glasses wearers of different age groups, different eye use environments, etc., the above parameters can be adjusted to adapt to different users and eye use environments.

[0215] In this specific application instance, an application module (such as a WeChat applet module) is installed on the monitor's smart terminal device. Through this application module, the following general functions are implemented: user registration: enter name, mobile phone number, and password to register, and verify with a mobile phone verification code; forgotten password operation: reset password via SMS verification code; account login and logout: mobile phone number, login, password, and verification code confirmation; logout: log out of the current account. The homepage content of the system's WeChat applet or APP includes: watch device list: device identification, bound user, device status, and battery level; glasses device list: device identification, bound user, device status, and battery level; watch latest event: the device's most recent alarm information, recording playback, and alarm keywords; glasses latest event: eye error statistics, average light intensity, average distance, and attention assessment (for the day); map: displays the watch device location (GPS positioning), device status, and child's name; watch data details: 1. Alarm record (trigger reason and recording); 2. Voice call record; vision profile: view personal vision profile, including eye axis, vision and other data (vision information: naked eye vision, vision with glasses, spherical lens, cylindrical lens, axis position, corneal curvature, hyperopia, and eye axis length. Main symptoms: squinting, rubbing eyes, blinking, tilting head, squinting, frowning, etc.).

[0216] Details of eye usage data: 1. Today’s eye usage data, including: today’s usage time, number of times the light is too dark, number of times the distance is too close, number of incorrect postures, time and average distance of the distance being too close, average weight borne by the cervical spine, average angle of tilting the head, average angle of lowering the head, etc. 2. Historical eye usage data: date screening, comprehensive score, usage time, average distance, average angle, average light, etc. 3. Course data (attention data, light environment data).

[0217] By collecting user eye data and uploading it to the cloud server for calculation and training, as the user uses smart glasses for more time, the monitoring system will train an eye usage pattern that is more suitable for the user and better monitor the eye health of young users. Through the health detection system worn on the glasses, the user's eye data is obtained and recognized using deep learning models such as convolutional neural networks (CNN).

[0218] The general settings on the watch side in this solution include: Glasses alarm setting list: keywords (text input keywords, such as: dirty words / beat you to death / would you dare to do it next time, etc.), eye use alarm threshold, bound device, contact, phone. Watch SOS settings, set emergency contacts, phone, set SOS switch.

[0219] The above embodiment of the present invention proposes a smart pair of glasses and a smart monitoring system and method for eye behavior, which have multiple functions to improve the eye health and safety of users. Among them:

[0220] Smart glasses can detect the user's eye distance, cervical spine angle, ambient light, and study time, and can automatically enter sleep mode according to the set alarm interval. At the same time, these data will be automatically synchronized to a small program via Bluetooth, including angle, distance, light and other information. The application (small program) can also synchronize the alarm settings to the glasses.

[0221] Smart watches are used in conjunction with smart glasses to synchronize data via Bluetooth. The watch can not only receive data from the glasses, but also transmit alarm settings back to the glasses. The watch uploads eye data and location data to the cloud server via the 4G network and obtains the configuration data of the server. This function is not limited to glasses. The watch can also record and manage the user's eye behavior data.

[0222] The cloud server management system is powerful and can manage and search for glasses and watches by user name, mobile phone number, MAC address, etc. The information of users and wearers can be registered and maintained through the mini program, and can be searched in various ways. The system will record the alarm data of glasses and watches in detail, including warning information such as dark light and wrong posture.

[0223] Every week, the cloud server management system will push the user's eye behavior analysis report to the associated WeChat public account, so that users can understand their own eye conditions in a timely manner. At the same time, users can calculate daily attention and light data through the set course schedule. The system analyzes and updates the data in the early morning every day, providing comparison and analysis of historical data.

[0224] The user's vision file can also be maintained through the system for secondary maintenance, supporting filtering and viewing according to fields such as region, school, and grade. The system automatically saves the filtering settings for application the next time you visit. These data can be displayed in various forms of charts and can also be classified according to the user's eye habits to predict the risk of myopia.

[0225] The cloud server management system also supports voice call configuration with the call server, and has a permission system to control the access rights of different users. In short, the invention provides a comprehensive and efficient eye monitoring and environmental safety monitoring solution through the combination of smart glasses, smart watches, cloud servers and guardians' smart terminal devices. Through these technical means, all-round monitoring of glasses wearers is achieved.

[0226] All matters not covered in the above embodiments of the present invention are well known in the art.

[0227] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. An intelligent eye behavior monitoring system, characterized in that: include: User-side glasses monitoring components, user-side smart devices, cloud servers and several monitoring smart terminal devices; among which: The user-side glasses monitoring component is used to obtain the user's eye data, perform pattern analysis on the eye data, and upload the obtained eye data and / or pattern analysis results to the cloud server through the user-side smart device or directly; provide local reminders and / or eye behavior recommendations to the user based on the pattern analysis results; and receive configuration information of component parameters from the cloud server and / or the user-side smart device; The user-side smart device is used to serve as a communication intermediate end between the user-side glasses monitoring component and the cloud server, to set local reminder parameters for the user-side glasses monitoring component, and / or to receive remote calls and / or reminders from the monitor's smart terminal device; The cloud server is used to establish a communication connection with the user-side smart device and then communicate with the user-side glasses monitoring component, or directly communicate with the user-side glasses monitoring component to achieve data interaction with the user-side glasses monitoring component; The monitor's smart terminal device is used to obtain the pattern analysis results of the cloud server, and remotely call the user through the user-side smart device according to the pattern analysis results, or remotely remind the user; remotely set parameters for the user-side glasses monitoring component and upload them to the cloud server.

2. The intelligent eye behavior monitoring system according to claim 1, characterized in that: The user-side glasses monitoring component includes: a mode analysis and processing module, a power management module, a communication module, a local reminder module and a sensor module; wherein: The pattern analysis processing module is connected to the power management module, the communication module, the local reminder module and the sensor module respectively, and is used to obtain the data collected by the sensor module and perform data analysis to obtain a pattern analysis result; The power management module is used to perform battery charging management, low-power sleep management, and current and voltage management of different modules; The communication module is used to interact with the user-side smart device and / or the cloud server; The local reminder module is used to remind the user locally according to the pattern analysis result; The sensor module includes: a distance sensor, a light intensity sensor, a 9-axis sensor and / or a temperature and humidity sensor, which is used to obtain relevant sensor data as eye data; The user-side glasses monitoring component further includes any one or more of the following: - The pattern analysis and processing module includes: a control module, a receiving module, a conversion module, an analysis module and a storage module; wherein: The control module is used to control the execution behaviors of the receiving module, the conversion module and the analysis module; The receiving module is used to receive various sensor data collected by the sensor component; The conversion module is used to perform data fusion and conversion on the various sensor data; The analysis module is used to perform pattern analysis on the converted sensor data and obtain irregular eye behavior according to the pattern analysis results; The storage module is used to locally store various sensor data acquired by the receiving module; - The communication module includes a Bluetooth module and a wireless communication module; wherein the Bluetooth module is used for synchronous data interaction between the user-side glasses monitoring component and the user-side smart device; the wireless communication module is used for realizing data interaction between the user-side glasses monitoring component and the cloud server; - the local reminder module includes a vibration component and an acoustic and optical component, and the pattern analysis and processing module triggers the vibration component and / or the acoustic and optical component to act according to the irregular eye behavior, and locally reminds the user; wherein the reminder parameters of the vibration component and / or the acoustic and optical component are set by the user-side smart device; - the sensor module further includes: a sensing switch, the sensing switch controls the user-side glasses monitoring component to be turned on and off according to the user's behavior; -The power management module also includes: a power indicator light.

3. The intelligent eye behavior monitoring system according to claim 1 or 2, characterized in that: The user-side glasses monitoring component also includes: a voice recognition module, which is used to monitor the audio data stream through a set voice keyword recognition algorithm, and after monitoring the specified keyword, send alarm data to the cloud server through the user-side smart device or directly send alarm data to the user-side smart device.

4. The intelligent eye behavior monitoring system according to claim 3, characterized in that: Also includes any one or more of the following: - The analysis module is used to perform pattern analysis on the converted sensor data and obtain irregular eye behavior according to the pattern analysis results, including: Provide a pre-trained IPSO-LSTM model to extract features from the converted sensor data and obtain irregular eye behavior; Provide a pre-trained Apriori algorithm model to perform behavioral analysis on the converted sensor data to obtain the user's eye behavior habits; Set the thresholds of each sensor, and compare the converted sensor data with the corresponding thresholds to obtain irregular eye behavior; -The conversion module is used to perform data fusion and conversion on the various sensor data, including: The steady-state Kalman filter method is used to fuse data from different sensors; A steady-state Kalman filtering method is used to directly calculate the optimal variance and optimal Kalman gain at each moment, so that the optimal estimation under steady-state conditions can be achieved at each moment to obtain fused sensor data; the fused sensor data is preprocessed by a conversion method.

5. The intelligent eye behavior monitoring system according to claim 4, characterized in that: Also includes any one or more of the following: -Providing a pre-trained IPSO-LSTM model, including: Building training samples based on the sensor data in the storage module, constructing a three-layer neural network for each type of sample, and training the training samples through the neural network; the sensor data in the storage module includes: reading posture, reading distance, reading time and / or light intensity; the training sample is a combination vector composed of grayscale features, HOG features, LBP features, and Haar features of the image; the first hidden layer in the three-layer neural network is used to learn edge features, the second hidden layer is used to learn shape features composed of edge features, and the third hidden layer is used to learn pattern features composed of shape features, and finally classifying the judgment results of the corresponding types of samples; -Providing a pre-trained Apriori algorithm model, including: Set the minimum weighted support threshold W m suph and minimum weighted confidence threshold W m conf; Scan a set of data item sets in the eye database D of the cloud server, calculate the weighted support Wsuph of the data item set, and generate a candidate item set C1; wherein the weighted support Wsuph is: Wsuph(X)=max{w1,w2,…,w n }*Wsup(X) Where X is the data item set, max{w1, w2, …, w n } is the maximum horizontal weight item of the data item set X, Wsup(X) is the support; the eye database D includes relevant data in the public data set and eye data generated in the actual use process; Determine whether the weighted support of the candidate item set C1 is greater than or equal to the minimum weighted support threshold W m suph; if yes, generate the frequent item set L1; if no, prune the candidate item set C1; Connect and iterate the frequent item set L1 to generate a candidate item set C2; Determine whether the weighted support of the candidate item set C2 is greater than or equal to the minimum weighted support threshold W m suph; if yes, generate the frequent item set L2; if no, prune the candidate item set C2; And so on, until all the data in the database are scanned, a strongly associated frequent item set is obtained, the confidence of the strongly associated frequent item set rules is calculated, the factors leading to eye health problems are determined according to the size of the confidence, the eye behavior habits are obtained, and relevant eye behavior recommendations are given.

6. The intelligent eye behavior monitoring system according to claim 1, characterized in that: The cloud server and / or the user-side smart device corresponds to different working states according to the preset health states of the user's glasses, and sets the configuration information of the component parameters.

7. The intelligent eye behavior monitoring system according to claim 1, characterized in that: Also includes any one or more of the following: - A first application module is arranged on the user-side smart device, and the first application module is used to: Synchronously receiving eye data and / or pattern analysis results transmitted by the user-side glasses monitoring component, and uploading them to the cloud server; Setting local reminder parameters for the user-side glasses monitoring component; - A second application module is arranged on the monitor's smart terminal device, and the second application module is used to: Sending a data request to the cloud server and receiving relevant data sent by the cloud server; Remotely setting parameters of the user-side glasses monitoring component and sending the parameters to the cloud server; Binding the user-side smart device; In the absence of the user-side smart device, the data interaction single-channel data communication mode between the user-side glasses monitoring component and the cloud server is switched.

8. The intelligent eye behavior monitoring system according to claims 1-5, characterized in that: The user-side smart device is a smart phone watch; and / or the monitor's smart terminal device is a smart phone or a tablet computer.

9. A smart glasses, characterized in that: The eye behavior intelligent monitoring system according to any one of claims 1 to 8 is used to obtain the user's eye data, and to remind the user of eye behavior and / or make eye behavior recommendations; wherein: The user-side glasses monitoring component of the eye behavior intelligent monitoring system is installed on the user's glasses via a detachable glasses clip.

10. A method for intelligently monitoring eye behavior, characterized in that: The intelligent eye behavior monitoring system described in any one of claims 1 to 8 is used to obtain the user's eye usage data, and to provide eye behavior reminders and / or eye behavior recommendations to the user.

Citation Information

Patent Citations

  • Teenager vision monitoring system

    CN111583602A

  • Intelligent myopia prevention and control equipment, prevention and control system and prevention and control method

    CN109523726A

  • Intelligent glasses capable of analyzing eye using habits of children

    CN111243742A

  • Artificial intelligence system of wearable device for preventing and controlling myopia

    CN111803076A

  • Smart toothbrush control circuit, smart toothbrush, and tooth-brushing behavior monitoring method

    US20210153989A1

Cited By

  • Myopia early warning method and system for children and teenagers

    CN121330750A