Early warning method, device and system of intelligent glasses

Abstract

The invention discloses an early warning method, device and system for intelligent glasses, and the method comprises the steps: obtaining an eyeball image of a current user, carrying out the preprocessing of the collected eyeball image of the user, and extracting image features; inputting the image features into a state detection model, and obtaining the current state of the user through training and analysis of the state detection model; when the current state of the user is determined to be the fatigue state, early warning information is generated, early warning is given to the user through a microphone, and a massage device is activated to massage the user based on the fatigue state. The accuracy and practicability of the user fatigue monitoring system in the intelligent glasses scene are improved, the working or driving safety of the user is improved, and the accident risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of smart glasses, and more specifically, to an early warning method, device and system for smart glasses. Background Art

[0002] Smart glasses will play an increasingly important role in future technological development; virtual reality technologies, such as the development of AR / VR, will also enrich the functions of smart glasses. In terms of interaction, smart glasses mainly have technologies such as voice control, gesture recognition and eye tracking, which provide users with interactive experience. When users are wearing smart glasses and driving in a tired state, accidents are prone to occur; therefore, how to identify user fatigue and remind users is a technical problem that needs to be solved urgently. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes an early warning method, device and system for smart glasses, which aims to solve the technical problem in the prior art that users wearing smart glasses are prone to fatigue and may cause accidents.

[0004] A first aspect of the present invention provides an early warning method, comprising:

[0005] Obtain the current user's eyeball image, pre-process the collected user's eyeball image, and extract image features;

[0006] Inputting the image features into a state detection model, and obtaining the current state of the user through training and analysis by the state detection model;

[0007] When it is determined that the current state of the user is a fatigue state, an early warning message is generated, the user is warned through a microphone, and the massage device is activated to massage the user based on the fatigue state.

[0008] In this solution, the state detection model is constructed using a neural network, specifically:

[0009] Acquire a user eyeball image dataset, preprocess the eyeball image samples in the user eyeball image dataset, extract image features of the user eyeball image samples, classify the eyeball image samples, and mark them according to the user's awake state and tired state;

[0010] The eyeball image samples of each state category in the user eyeball image dataset are divided into a training dataset and a test dataset in the same proportion, and the image features and SIFT features of the eyeball image samples in the training dataset and the test dataset are extracted respectively, and the extracted features are normalized and feature fused;

[0011] A state detection model is constructed by a neural network method, the neural network parameters are determined and the weight threshold is initialized, the input fusion features are learned and classified, the output of each layer and the output error are calculated, and the weight threshold of each layer is corrected by the output error;

[0012] When all training data samples are trained, the accuracy of the state monitoring model is tested using the test data set. When the accuracy meets the preset standard, the trained state detection model is output;

[0013] Obtain the image features and SIFT features of the current user's eye image, import the trained state detection model, obtain the probability distribution of the user's awake state and tired state labels, determine whether the user is in a tired state based on the probability distribution, and output;

[0014] In addition, the key point position distribution of the user in the awake state is set according to the SIFT features of the user's eye image samples. When the user is in a tired state, the position deviation between the current key point position distribution and the key point position distribution in the awake state is obtained, and the user's fatigue level is obtained according to the position deviation.

[0015] In this solution, the image features of the user's eye image samples are extracted, and the eye image samples are classified, specifically:

[0016] The user's eye image is collected through the eye tracking device of the smart glasses, the user's eye image dataset is constructed, invalid image samples are eliminated, and the screened eye image samples are adaptively preprocessed according to the lighting environment;

[0017] The preprocessed eye image samples are imported into the AlexNet convolutional neural network, feature extraction is performed based on the AlexNet convolutional neural network, feature maps are extracted through convolution layers with convolution kernels of different sizes, and the extracted feature maps are activated and input into the pooling layer;

[0018] In the pooling layer, the dimension of the feature map is reduced, and feature extraction is further performed, and the feature map after the pooling operation is imported into the fully connected layer to obtain a one-dimensional feature vector output by the fully connected layer;

[0019] The one-dimensional feature vector is used to cluster the eyeball image samples, and the awake state image subset and the fatigue state image subset are divided according to the distance difference between the samples, and the label information of the eyeball image samples is set.

[0020] In this solution, when it is determined that the current state of the user is fatigued, a warning message is generated and the user is warned through a microphone, specifically:

[0021] After determining that the user is in a state of fatigue, an early warning sound is issued through the microphone according to the preset prompt sound, a fatigue status monitoring task is generated for the user, an early warning frequency threshold is set according to the user's current scenario, and the cumulative number of early warnings within a preset time interval is determined. If the number exceeds the early warning frequency threshold, a rest reminder sound is issued through the microphone;

[0022] If the warning frequency threshold is not exceeded, interaction information is generated based on the current scenario and sent to the user through a microphone based on the warning timestamp, wherein the interaction information is generated based on a preset large language model and user preference information.

[0023] In this solution, the massage device is activated based on the fatigue state to massage the user, specifically:

[0024] Based on the user's fatigue state image subset, different fatigue states are divided, the division result is mapped with the fatigue degree, and the fatigue degree division interval is generated, the massage intensity interval of the massage device is read, and the massage intensity interval is divided according to the division ratio of the fatigue degree division interval;

[0025] After determining that the user is in a state of fatigue, the massage intensity is determined according to the massage intensity range in which the user's current fatigue level falls, and the massage device is configured based on the massage intensity to massage the user to eliminate the user's fatigue.

[0026] A second aspect of the present invention provides a pair of smart glasses, comprising: a middle frame, an eye tracking device, and temples;

[0027] The eye tracking device is arranged at the upper left corner or the upper right corner of the middle frame of the smart glasses, and includes electronics for capturing image data of the user's eyeballs;

[0028] A microphone is provided on the temple, and when the user is in a state of fatigue, the microphone sends a warning sound to the user for warning;

[0029] The temples are provided with a massage device, which can eliminate the user's fatigue when the user is in a fatigued state.

[0030] A third aspect of the present invention provides an early warning system, including an eyeball image acquisition and processing unit, a state detection modeling unit, a fatigue state detection unit, a fatigue early warning unit, and a massage device unit;

[0031] The eyeball image acquisition and processing unit is responsible for acquiring the user's real-time eyeball image data and preprocessing the acquired eyeball image data;

[0032] The state detection modeling unit is responsible for building a state detection model based on a neural network method and performing model training according to a user's eye image data set;

[0033] The fatigue state detection unit is responsible for importing the eyeball image data collected in real time into the state detection model to identify the user's fatigue state and fatigue degree;

[0034] The fatigue warning unit is responsible for generating warning information when the user is in a fatigue state, warning the user through a microphone, and activating the massage device unit based on the fatigue state to adaptively generate massage intensity to massage the user and eliminate the user's fatigue state.

[0035] A fourth aspect of the present invention provides a computer-readable storage medium, which includes an early warning method program for smart glasses. When the early warning method program for smart glasses is executed by a processor, the steps of the early warning method for smart glasses are implemented.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention extracts features of user eyeball images based on the AlexNet convolutional neural network, performs clustering processing using the one-dimensional feature vector output by the network's fully connected layer, optimizes data integration and data analysis, improves the accurate classification of user status data, and provides a data source basis for subsequent user fatigue status detection and identification.

[0038] The state detection model in the present invention is constructed through a neural network method, which provides effective information for user state warning, improves the accuracy and practicality of the user fatigue monitoring system in the smart glasses scenario, improves the safety of user work or driving, and reduces the risk of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0040] Figure 1 A schematic diagram of a scenario in which a user wears smart glasses in an embodiment is shown;

[0041] Figure 2 A schematic diagram of the structure of smart glasses in an embodiment is shown;

[0042] Figure 3 A schematic diagram showing the eye tracking device collecting user eye images in an embodiment is shown;

[0043] Figure 4 The user's eyeball image obtained when the user is awake in the embodiment is shown;

[0044] Figure 5 The user's eyeball image obtained when the user is tired in the embodiment is shown;

[0045] Figure 6 A schematic diagram showing the positions of the microphone and the massage device in the embodiment is shown;

[0046] Figure 7 A flowchart of a smart glasses early warning method according to an embodiment is shown;

[0047] Figure 8 A flowchart of an embodiment for generating a fatigue warning is shown;

[0048] Fig. 9 An embodiment is shown to construct a state detection model to identify the user's fatigue state

[0049] Fig.10 A block diagram of a smart glasses early warning system according to an embodiment is shown;

[0050] Fig.11 A schematic diagram of a processor of smart glasses in an embodiment is shown;

[0051] Fig.12 A schematic diagram of a processor of smart glasses in an embodiment is shown;

[0052] Description of the drawings: 101 - smart glasses, 103 - user, 201 - middle frame, 202 - base, 130 - eye tracking device, 203 - temples, 205 - user's right eye, 501, 502 - microphones, 204, 207 - massage devices. DETAILED DESCRIPTION

[0053] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0055] It should be noted that smart glasses are a type of wearable device with an independent operating system. Users can install software, games and other programs provided by software service providers to achieve a variety of functions. Through voice or motion control, users can complete operations such as adding schedules, map navigation, interacting with friends, taking photos and videos, and making video calls. In addition, smart glasses can also achieve wireless network access through mobile communication networks. As a wearable device, smart glasses can free the user's hands. Compared with smartphones, users are more convenient to use, and they have the characteristics of being easy to use and small in size. Figure 1 As shown, user 103 wears smart glasses 101. Through the smart glasses 101, user 103 can play music, listen to music, and take photos and videos. Preferably, smart glasses 101 can also provide user 103 with more functions and services, such as games, social interaction, navigation, and movie watching.

[0056] like Figure 2 As shown, the first embodiment of the present invention provides a smart glasses 101, including: a middle frame 201, an eye tracking device 130, and temples 203; the middle frame 201 is connected to the temples 203 through the studs 202; when the user device wears the smart glasses to look forward, the eye tracking device 130 can obtain the state of the eyeball, and the eye tracking device 130 includes an electronic processing circuit, a camera and a structured optical element; it can capture the image of the eyeball in real time.

[0057] The eye tracking device 130 is arranged on the frame of the smart glasses 101. Figure 3 As shown, the eye tracking device 130 is located at the upper right corner of the glasses, or at the upper left corner; wherein, the eye tracking device 130 at the upper right corner can obtain the eye image of the user's right eye 205 in real time. The eye tracking device at the upper left corner can obtain the eye image of the user's left eye in real time. When the user is not tired, the obtained user eye image is as follows Figure 4 As shown in Figure 2, the user's eyeball image obtained when the user is tired is as follows: Figure 5 As shown, there is a big difference between the two types of eye images. Smart glasses use eye tracking devices to capture Figure 4 or Figure 5 After the eyeball image shown in FIG. 5 is captured, the state detection model constructed based on the neural network can be used to learn and analyze whether the user is currently awake or tired. When the eyeball image captured by the eyeball tracking device shows that the user is tired, the smart glasses warn the user through the microphone 501 or massage the user through the massage device 207 to eliminate the user's tired state. When it is detected that the user is awake, the microphone 501 does not work and the massage device 207 does not work.

[0058] like Figure 6As shown, a massage device 207 is installed on the right temple of the smart glasses 101 for massaging the right temple of the user. A microphone 501 is also installed on the right temple for outputting a warning sound to the right ear of the user to remind the user of the fatigue state and thus wake up the user; correspondingly, a microphone 502 and a massage device 204 are also provided on the other opposite side of the smart glasses. The massage device is arranged on the temple, and a soft sponge device is installed on the motor. When no massage instruction is received, that is, when the user is detected to be in an awake state, the massage device is not extended; when a massage instruction is detected, that is, when the user is detected to be in a fatigued state, the massage device is extended from the temple and roughly aimed at the user's temple, and a pressing operation is performed through the motor, so as to wake up the user and eliminate the user's fatigue.

[0059] like Figure 7 , Figure 8 As shown, the second embodiment of the present invention provides an early warning method for smart glasses, comprising the following steps:

[0060] S802, obtaining an eye image of the current user, preprocessing the collected eye image of the user, and extracting image features;

[0061] S804, inputting the image features into a state detection model, and obtaining the current state of the user through training and analysis of the state detection model;

[0062] S806, when it is determined that the current state of the user is a fatigue state, an early warning message is generated, the user is warned through a microphone, and a massage device is activated based on the fatigue state to massage the user.

[0063] It should be noted that the eye tracking device of the smart glasses collects the user's eye images in a preset time period, constructs the user's eye image dataset, eliminates invalid image samples caused by the user closing their eyes, and performs adaptive pre-processing on the screened eye image samples according to the lighting environment, adjusts the image's brightness, contrast, saturation and other aspects, and improves the quality of the eye image.

[0064] According to an embodiment of the present invention, Fig. 9 As shown in the figure, a state detection model is constructed to identify the user's fatigue state, specifically:

[0065] S902, obtaining a user eyeball image dataset, preprocessing eyeball image samples in the user eyeball image dataset, extracting image features of the user eyeball image samples, classifying the eyeball image samples, and marking them according to the user's awake state and tired state;

[0066] S904, dividing the eyeball image samples of each state category in the user eyeball image data set into a training data set and a test data set according to the same ratio, extracting image features and SIFT features of the eyeball image samples in the training data set and the test data set respectively, normalizing the extracted features, and performing feature fusion;

[0067] S906, constructing a state detection model through a neural network method, determining neural network parameters and initializing weight thresholds, learning and classifying the input fusion features, calculating the output of each layer and the output error, and correcting the weight thresholds of each layer through the output error;

[0068] S908, when all training data samples are trained, the accuracy of the state monitoring model is tested using the test data set, and when the accuracy meets the preset standard, the trained state detection model is output;

[0069] S910, obtaining image features and SIFT features of the current user's eye image, importing the trained state detection model, obtaining probability distribution of user awake state and fatigue state labels, judging whether the user is in a fatigue state based on the probability distribution, and outputting the result;

[0070] S912, setting the key point position distribution of the user in the awake state according to the SIFT features of the user's eye image sample, when the user is in a tired state, obtaining the position deviation between the current key point position distribution and the key point position distribution in the awake state, and obtaining the user's fatigue level according to the position deviation.

[0071] It should be noted that the pre-processed eye image samples are imported into the AlexNet convolutional neural network, and feature extraction is performed based on the AlexNet convolutional neural network. The AlexNet convolutional neural network has a total of 8 weighted network layers, including 5 convolutional layers and 3 fully connected layers, wherein the 1st, 2nd, and 5th convolutional layers each contain a maximum pooling layer. Feature maps are extracted through convolutional layers with convolution kernels of different sizes, and the extracted feature maps are activated using the ReLU activation function and input into the pooling layer; the dimension of the feature map is reduced in the pooling layer, and feature extraction is further performed, and the feature map after the pooling operation is imported into the fully connected layer to obtain the one-dimensional feature vector output by the fully connected layer.

[0072] The one-dimensional feature vector is used to cluster the eye image samples, the number of clusters is initialized, and the initial cluster center is selected. The distance between the eye image sample and the initial cluster center is calculated, and the eye image sample is divided into the category to which the nearest cluster center belongs to form a clustering result. The cluster center is updated by iterative clustering. When the standard measurement function meets the preset standard or the number of iterations is greater than or equal to the maximum number of iterations, the last clustering result is selected to obtain the awake state image subset and the fatigue state image subset, and the label information of the eye image sample is set.

[0073] A massive amount of user awake state images and fatigue state images are obtained to construct a user eye image dataset, and a BP neural network is trained to construct a state detection model to identify the user's fatigue state. The eye image samples of each state category in the user eye image dataset are divided into a training dataset and a test dataset in the same proportion, and the image features and SIFT features of the eye image samples in the training dataset and the test dataset are extracted respectively. The extracted features are normalized, and the normalized features are connected in series to achieve feature fusion, and a high-dimensional feature vector is generated as the fusion feature; the image features of the eye image samples are composed of a one-dimensional feature vector output by the AlexNet convolutional neural network, and the SIFT features of the eye image samples are calculated and counted by the SIFT algorithm to construct scale space, detect spatial extreme points, locate key points, determine key points, and other steps to generate key point descriptions.

[0074] A state detection model is constructed through a neural network method, and the neural network parameters, learning rate and Dropout ratio are determined and the weight threshold is initialized. Convolution, pooling and full connection operations are performed on the input fusion features to obtain the final Softmax output, and the output and output errors of each layer are calculated. The weight threshold of each layer is corrected by the output error, and the network parameters are updated. When all training data samples are trained, the accuracy of the state monitoring model is tested using a test data set. When the accuracy meets the preset standard, the trained state detection model is output, otherwise, the Dropout ratio of the neural network is adjusted.

[0075] It should be noted that after determining that the user is in a state of fatigue, a warning sound is issued through the microphone according to the preset prompt sound, a fatigue status monitoring task is generated for the user, and the warning frequency threshold is set according to the user's current scenario, and the cumulative number of warnings within the preset time interval is determined. If the warning frequency threshold is exceeded, a rest reminder sound is issued through the microphone; if the warning frequency threshold is not exceeded, interactive information is generated based on the current scenario, and sent to the user through the microphone based on the warning timestamp. The interactive information is generated based on a preset large language model and user preference information. For example, in a driving scenario, the preset large language model is used to ask the user whether he needs to open the window, or a window opening suggestion is given. In addition, based on the user's song preference when using smart glasses, music that meets the user's preferences is recommended for playback.

[0076] Different fatigue states are divided based on a subset of the user's fatigue state images, the division results are mapped with the fatigue degree, fatigue degree division intervals are generated, the massage intensity intervals of the massage device are read, and the massage intensity intervals are divided according to the division ratio of the fatigue degree division intervals; after determining that the user is in a fatigue state, the massage intensity is determined according to the massage intensity interval into which the user's current fatigue degree falls, and the massage device is configured based on the massage intensity to massage the user to eliminate the user's fatigue state.

[0077] like Fig.11 As shown, the smart glasses have built-in memory and processor, the warning module includes a microphone and a massage device, and in the monitoring and warning stage of the user status, the memory includes a warning method program for the smart glasses, and the processor is used to run the program, wherein when the program is running, any of the above-mentioned warning methods for the smart glasses is executed. Fig.12 As shown, in another preferred embodiment of the present application, a server is set up in the cloud based on the cloud platform, and the monitoring and early warning of the user status can be executed by the server, specifically by a processor in the server, and the processor executes any one of the above-mentioned early warning methods of the smart glasses. The smart glasses have built-in memory and communication module, and the user's eye image is collected and temporarily stored in the memory. The eye image is sent to the cloud server by using the communication module, and the server processing result is received by using the communication module, and the judgment result is sent to the early warning module, and early warning and massage wake-up are performed through a microphone and a massage device.

[0078] The third embodiment of the present invention provides an early warning system 10 for smart glasses, such as Fig.10 As shown, it includes an eyeball image acquisition and processing unit 1001, a state detection modeling unit 1002, a fatigue state detection unit 1003, a fatigue warning unit 1004 and a massage device unit 1005;

[0079] The eyeball image acquisition and processing unit is responsible for acquiring the user's real-time eyeball image data and preprocessing the acquired eyeball image data;

[0080] The state detection modeling unit is responsible for building a state detection model based on a neural network method and performing model training according to a user's eye image data set;

[0081] The fatigue state detection unit is responsible for importing the eyeball image data collected in real time into the state detection model to identify the user's fatigue state and fatigue degree;

[0082] The fatigue warning unit is responsible for generating warning information when the user is in a fatigue state, warning the user through a microphone, and activating the massage device unit based on the fatigue state to adaptively generate massage intensity to massage the user and eliminate the user's fatigue state.

[0083] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes an early warning method program for smart glasses. When the early warning method program for smart glasses is executed by a processor, the steps of the early warning method for the smart glasses are implemented.

[0084] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0085] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0087] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0088] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0089] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A warning method for smart glasses, characterized in that: Applied to smart glasses, the method comprises the following steps: Obtain the current user's eyeball image, pre-process the collected user's eyeball image, and extract image features; Inputting the image features into a state detection model, and obtaining the current state of the user through training and analysis by the state detection model; When it is determined that the current state of the user is a fatigue state, an early warning message is generated, the user is warned through a microphone, and the massage device is activated to massage the user based on the fatigue state.

2. The early warning method according to claim 1, characterized in that: The state detection model is constructed using a neural network, specifically: Acquire a user eyeball image dataset, preprocess the eyeball image samples in the user eyeball image dataset, extract image features of the user eyeball image samples, classify the eyeball image samples, and mark them according to the user's awake state and tired state; The eyeball image samples of each state category in the user eyeball image dataset are divided into a training dataset and a test dataset in the same proportion, and the image features and SIFT features of the eyeball image samples in the training dataset and the test dataset are extracted respectively, and the extracted features are normalized and feature fused; A state detection model is constructed by a neural network method, the neural network parameters are determined and the weight threshold is initialized, the input fusion features are learned and classified, the output of each layer and the output error are calculated, and the weight threshold of each layer is corrected by the output error; When all training data samples are trained, the accuracy of the state monitoring model is tested using the test data set. When the accuracy meets the preset standard, the trained state detection model is output; Obtain the image features and SIFT features of the current user's eye image, import the trained state detection model, obtain the probability distribution of the user's awake state and tired state labels, determine whether the user is in a tired state based on the probability distribution, and output; The key point position distribution of the user in the awake state is set according to the SIFT features of the user's eye image samples. When the user is in a tired state, the position deviation between the current key point position distribution and the key point position distribution in the awake state is obtained, and the user's fatigue level is obtained according to the position deviation.

3. The early warning method according to claim 2, characterized in that: Extracting image features of the user's eyeball image samples and classifying the eyeball image samples is specifically as follows: The user's eye image is collected through the eye tracking device of the smart glasses, the user's eye image dataset is constructed, invalid image samples are eliminated, and the screened eye image samples are adaptively preprocessed according to the lighting environment; The preprocessed eye image samples are imported into the AlexNet convolutional neural network, feature extraction is performed based on the AlexNet convolutional neural network, feature maps are extracted through convolution layers with convolution kernels of different sizes, and the extracted feature maps are activated and input into the pooling layer; In the pooling layer, the dimension of the feature map is reduced, and feature extraction is further performed, and the feature map after the pooling operation is imported into the fully connected layer to obtain a one-dimensional feature vector output by the fully connected layer; The one-dimensional feature vector is used to cluster the eyeball image samples, and the awake state image subset and the tired state image subset are divided according to the distance difference between the samples, and the label information of the eyeball image samples is set.

4. The early warning method according to claim 1, characterized in that: When it is determined that the current state of the user is fatigued, a warning message is generated and the user is warned through a microphone, specifically: After determining that the user is in a state of fatigue, an early warning sound is issued through the microphone according to the preset prompt sound, a fatigue status monitoring task is generated for the user, an early warning frequency threshold is set according to the user's current scenario, and the cumulative number of early warnings within a preset time interval is determined. If the number exceeds the early warning frequency threshold, a rest reminder sound is issued through the microphone; If the warning frequency threshold is not exceeded, interaction information is generated based on the current scenario and sent to the user through a microphone based on the warning timestamp, wherein the interaction information is generated based on a preset large language model and user preference information.

5. The early warning method according to claim 1, characterized in that: Based on the fatigue state, the massage device is activated to massage the user, specifically: Based on the user's fatigue state image subset, different fatigue states are divided, the division result is mapped with the fatigue degree, and the fatigue degree division interval is generated, the massage intensity interval of the massage device is read, and the massage intensity interval is divided according to the division ratio of the fatigue degree division interval; After determining that the user is in a state of fatigue, the massage intensity is determined according to the massage intensity range in which the user's current fatigue level falls, and the massage device is configured based on the massage intensity to massage the user to eliminate the user's fatigue.

6. A smart glasses with built-in memory and processor, characterized in that: The memory and the processor store and execute the early warning method program according to any one of claims 1 to 5, wherein the smart glasses include: a middle frame, an eye tracking device, and temples; The eye tracking device is arranged at the upper left corner or the upper right corner of the middle frame of the smart glasses, and includes electronics for capturing image data of the user's eyeballs; The temples are provided with microphones, which can send warning sounds to the user when the user is tired. The temples are provided with a massage device, and when the user is in a tired state, the massage device can be used to eliminate the tired state of the user.

7. An early warning system, characterized in that: Implementing the early warning method as described in any one of claims 1 to 5, comprising an eyeball image acquisition and processing unit, a state detection modeling unit, a fatigue state detection unit, a fatigue early warning unit and a massage device unit; The eyeball image acquisition and processing unit is responsible for acquiring the user's real-time eyeball image data and preprocessing the acquired eyeball image data; The state detection modeling unit is responsible for building a state detection model based on a neural network method and performing model training according to a user's eye image data set; The fatigue state detection unit is responsible for importing the eyeball image data collected in real time into the state detection model to identify the user's fatigue state and fatigue degree; The fatigue warning unit is responsible for generating warning information when the user is in a fatigue state, warning the user through a microphone, and activating the massage device unit based on the fatigue state to adaptively generate massage intensity to massage the user and eliminate the user's fatigue state.

Images