Safe eye protection method and system for realizing free screen based on Internet of Things

Abstract

The invention relates to the technical field of the Internet of Things, and discloses a safe eye protection method and system for realizing a free screen based on the Internet of Things, and the method comprises the steps: detecting a human face region of a target user under a screen display picture, calculating the region area and the head posture in the human face region, and calculating a fixation point in the human face region through the head posture; recognizing the blinking frequency of the face area, and calculating the face-screen display distance between the face area and the free screen; analyzing the visual fatigue of the target user under the screen display picture, and determining the eye state of the target user under the screen display picture; and when the eye state is abnormal, identifying an external environment and screen display characteristics of the screen display picture, extracting environment characteristics of the external environment, mining influence factors influencing the eye state based on the screen display characteristics and the environment characteristics, adjusting picture parameters of the free screen according to the influence factors and the eye state, and generating a safe eye protection report of the free screen. The safe eye protection effect of the free screen can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a safe eye protection method and system for realizing a free screen based on the Internet of Things. Background Art

[0002] FreeScreen's safe eye protection refers to a technology or system that aims to protect the user's eye health by monitoring and adjusting screen parameters, reducing eye fatigue and potential vision problems caused by long-term screen use. FreeScreen's safe eye protection method improves the safety and comfort of screen use through technical means, thereby reducing the negative impact of long-term screen use on the eyes.

[0003] At present, the safe eye protection method of Free Screen is to reduce the blue light intensity to reduce potential damage to the eyes. This method may cause screen color distortion and affect image quality because turning on the low blue light mode. At the same time, even if the blue light intensity is reduced, improper use of Free Screen by users will also affect eye health. Summary of the invention

[0004] The present invention provides a method and system for realizing safe eye protection of a free screen based on the Internet of Things, the main purpose of which is to enhance the safe eye protection effect of the free screen.

[0005] To achieve the above purpose, the present invention provides a safe eye protection method for realizing a free screen based on the Internet of Things, comprising:

[0006] The screen display of the free screen is collected by using the Internet of Things, and the target user under the screen display is monitored in real time by the monitoring camera corresponding to the free screen, the face area of ​​the target user under the screen display is detected, the area and the head posture of the face area are calculated, and the gaze point in the face area is calculated by using the head posture;

[0007] Identify the number of blinks of the face area, and calculate the face-screen distance between the face area and the free screen;

[0008] Determine the visual fatigue of the target user on the screen according to the area, the gaze point, the number of blinks and the face-screen distance, and determine the eye state of the target user on the screen according to the visual fatigue;

[0009] When the eye state is in an abnormal state, the external environment and screen display features of the screen display are identified, and the environmental features of the external environment are extracted. Based on the screen display features and the environmental features, the influencing factors affecting the eye state are mined, and the gaze point data corresponding to the eye state is extracted to determine the pupil size and gaze duration of the eye state, and the picture parameters of the free screen are obtained, the picture parameters include screen brightness, resolution and scanning frequency, the light brightness in the influencing factors is determined, and the optimal screen brightness of the free screen is determined according to the light brightness and the pupil size, the optimal resolution of the free screen is calculated according to the gaze point data and the influencing factors, and the optimal scanning frequency of the free screen is calculated according to the influencing factors and the visual fatigue corresponding to the eye state, and after adjusting the picture parameters according to the optimal screen brightness, the optimal resolution and the optimal scanning frequency, the step of determining the visual fatigue of the target user under the screen display according to the area of ​​the region, the gaze point, the number of blinks and the face-screen display distance is returned to execute;

[0010] When there is no abnormality in the eye state, a safety eye protection report for the free screen is generated.

[0011] Optionally, the calculating the area and head posture in the face region includes:

[0012] Acquire a facial image of the facial area, and detect facial edges of the facial image;

[0013] Constructing a polygon of the face image according to the face edge, and identifying the vertices of the polygon;

[0014] Based on the vertices, marking the facial contour of the facial image;

[0015] Constructing a two-dimensional coordinate system of the face contour, and determining the vertex coordinates of the vertices according to the two-dimensional coordinate system;

[0016] According to the vertex coordinates, the area of ​​the face region is calculated using the following formula:

[0017]

[0018] Where M represents the area, i represents the i-th vertex corresponding to the face contour, n represents the total number of vertices corresponding to the face contour, and p i represents the horizontal coordinate of the i-th vertex corresponding to the face contour, p i+1 represents the horizontal coordinate of the i+1th vertex corresponding to the face contour, q i Indicates the ordinate of the i-th vertex corresponding to the face contour, q i+1Indicates the ordinate of the i+1th vertex corresponding to the face contour;

[0019] Constructing a three-dimensional face model of the face area according to the face image;

[0020] Constructing a three-dimensional coordinate system of the three-dimensional face model, and determining the nose tip coordinates, left eye coordinates, and right eye coordinates of the face region according to the three-dimensional coordinate system;

[0021] According to the nose tip coordinates, the left eye coordinates and the right eye coordinates, the pitch angle, yaw angle and roll angle of the three-dimensional face model are calculated using the following formula:

[0022]

[0023] Among them, α represents the pitch angle, β represents the yaw angle, γ represents the roll angle, arctan represents the inverse tangent function, and x b represents the x-axis coordinate of the nose tip, y b The y-axis coordinate of the nose tip, z b represents the z-axis coordinate of the nose tip, x l Indicates the x-axis coordinate of the left eye, y l Indicates the y-axis coordinate of the left eye, z l Indicates the z-axis coordinate of the left eye, x r Indicates the x-axis coordinate of the right eye, y r Indicates the y-axis coordinate of the right eye, z r Represents the z-axis coordinate of the right eye coordinate;

[0024] The head posture of the face region is determined according to the pitch angle, the yaw angle, and the roll angle.

[0025] Optionally, constructing a three-dimensional face model of the face area according to the face image includes:

[0026] Constructing a basic three-dimensional face model of the face area;

[0027] Detecting feature points of the face image, and mapping model corresponding points of the basic three-dimensional model according to the feature points;

[0028] Determine a rotation matrix and a translation vector of the basic three-dimensional model based on the feature points and the model corresponding points;

[0029] According to the feature points, the model corresponding points, the rotation matrix and the translation vector, the posture parameters and shape parameters of the model of the basic three-dimensional model are calculated using the following formula:

[0030]

[0031] Among them, ZT represents the posture parameter, G represents the shape parameter, arg represents the function parameter, min represents the minimization function, N represents the number of feature points or model corresponding points, and T j represents the jth feature point, D j represents the jth model corresponding point, A represents the rotation matrix, H represents the translation vector, Proj represents the projection transformation, B j represents the change in the corresponding point of the j-th model;

[0032] According to the posture parameters and the shape parameters, the basic three-dimensional model is mapped to a two-dimensional image space to obtain mapped feature points;

[0033] Calculating the error between the feature point and the mapped feature point,

[0034] When the error is greater than a preset error threshold, determining an objective function of the basic three-dimensional model and the face image according to the error;

[0035] Calculating the parameter gradient of the objective function relative to the posture parameter and the shape parameter, adjusting the posture parameter and the shape parameter according to the parameter gradient to obtain adjusted posture parameter and adjusted shape parameter, and returning to the step of mapping the basic three-dimensional model to a two-dimensional image space according to the posture parameter and the shape parameter to obtain mapped feature points;

[0036] When the error is less than a preset error threshold, the basic three-dimensional model is used as the three-dimensional face model of the face area.

[0037] Optionally, calculating the gaze point in the face region by using the head posture includes:

[0038] Constructing an initial direction vector of the head posture;

[0039] Determining a head center axis and an eye center axis of the head posture according to the head posture;

[0040] Calculating the head direction vector of the head central axis through the rotation matrix corresponding to the head posture and the initial direction vector;

[0041] Identifying a pupil center of the head posture, and determining an eye direction vector of the eye center axis based on the pupil center;

[0042] The center axis distance between the center axis of the head and the center axis of the eye is calculated, and the gaze point in the face area is calculated according to the center axis distance, the head direction vector and the eye direction vector.

[0043] Optionally, the calculating the face-screen display distance between the face area and the free screen includes:

[0044] A three-dimensional coordinate system of the face area and the free screen is constructed with the surveillance camera corresponding to the free screen as the origin, and the spatial parameters of the free screen in the three-dimensional coordinate system are determined. According to the three-dimensional coordinate system and the spatial parameters, the three-dimensional coordinates of the free screen center of the free screen are determined, and the internal parameters and external parameters of the surveillance camera are extracted, wherein the spatial parameters include position parameters and direction parameters;

[0045] Analyze the pixel coordinates of the face image corresponding to the face area, and convert the pixel coordinates into three-dimensional face coordinates using the following formula according to the internal parameters and external parameters:

[0046] R(X f , Y f , Z f )=PerspectiveTransform(S(E f , F f ),intrinsic,extrinsic)

[0047] Among them, R(X f , Y f , Z f ) represents the three-dimensional coordinates of the face, X f The X-axis coordinates of the face's three-dimensional coordinates, f The Y-axis coordinate of the face's three-dimensional coordinates, Z f represents the Z-axis coordinate of the three-dimensional coordinates of the face, PerspectiveTransform represents the function of pixel coordinate transformation, S(E f , F f ) represents the pixel coordinates, E f Indicates the X-axis coordinate of the pixel coordinate, F f Indicates the Y-axis coordinate of the pixel coordinate, intrinsic indicates internal parameters, and extrinsic indicates external parameters;

[0048] Based on the three-dimensional coordinates of the face and the three-dimensional coordinates of the center of the free screen, a face-screen display distance between the face area and the free screen is determined.

[0049] Optionally, determining the visual fatigue of the target user on the screen display according to the area of ​​the region, the gaze point, the number of blinks, and the face-screen display distance includes:

[0050] Obtaining the screen resolution of the displayed image, determining the gaze point data of the gaze point, wherein the gaze point data includes: a gaze point distribution state and a gaze point movement mode,

[0051] Calculating the gaze area of ​​the target user under the screen display according to the area of ​​the area, the gaze point data and the screen resolution;

[0052] Based on the number of blinks, determining the screen gaze time of the target user;

[0053] Determining the visual pressure of the target user according to the face-screen distance;

[0054] The visual fatigue degree of the target user on the displayed image is calculated based on the gaze area, the screen gaze time, and the visual pressure.

[0055] Optionally, determining the eye state of the target user on the displayed image according to the visual fatigue degree includes:

[0056] Extracting a facial image corresponding to the target user, and identifying eye features of the facial image;

[0057] Based on the eye features, determining the pupil state and eye wetness of the target user;

[0058] Determining the eye-face movement state of the target user according to the gaze point data corresponding to the visual fatigue degree;

[0059] Determining an eye health index of the target user according to the pupil state, the eye wetness, and the eyelid movement state;

[0060] The eye state of the target user on the screen display is determined according to the visual fatigue degree and the eye health index.

[0061] Optionally, mining the influencing factors affecting the eye state based on the screen display features and the environmental features includes:

[0062] Respectively calculating the correlation between the display feature and the environment feature and the eye state to obtain a display-eye correlation and an environment-eye correlation;

[0063] When the screen display-eye correlation degree and the environment-eye correlation degree are both greater than a preset threshold, extracting correlation features from the screen display features and the environment features;

[0064] According to the associated features, an influencing factor affecting the eye state is determined.

[0065] Optionally, extracting environmental features of the external environment includes:

[0066] Extracting initial environmental data of the external environment using a preset sensor network;

[0067] De-noising the initial environmental data to obtain de-noised environmental data, and identifying abnormal values ​​of the de-noised environmental data;

[0068] Performing logarithmic transformation on the abnormal value to obtain environmental data;

[0069] Calculate the light parameters and the ambient color temperature parameters of the external environment according to the environmental data, wherein the light parameters include light intensity, light intensity change rate and light distribution, and the ambient color temperature parameters include: color temperature change range and color temperature change frequency;

[0070] The environmental characteristics of the external environment are determined according to the light parameters and the environmental color temperature parameters.

[0071] In order to solve the above problems, the present invention also provides a safe eye protection system for realizing a free screen based on the Internet of Things, the system comprising:

[0072] A gaze point analysis module is used to collect the screen display of the free screen by using the Internet of Things, and monitor the target user under the screen display in real time through the monitoring camera corresponding to the free screen, detect the face area of ​​the target user under the screen display, calculate the area and head posture of the face area, and calculate the gaze point in the face area by using the head posture;

[0073] A face-display distance calculation module, used to identify the number of blinks of the face area and calculate the face-display distance between the face area and the free screen;

[0074] an eye state analysis module, for determining the visual fatigue of the target user on the screen display according to the area, the gaze point, the number of blinks and the face-screen distance, and determining the eye state of the target user on the screen display according to the visual fatigue;

[0075] A screen display automatic adjustment module, used for identifying the external environment and screen display features of the screen display picture when the eye state is abnormal, and extracting the environmental features of the external environment, mining the influencing factors affecting the eye state based on the screen display features and the environmental features, and extracting the gaze point data corresponding to the eye state, determining the pupil size and gaze duration of the eye state, obtaining the picture parameters of the free screen, the picture parameters including screen brightness, resolution and scanning frequency, determining the light brightness in the influencing factors, determining the optimal screen brightness of the free screen according to the light brightness and the pupil size, calculating the optimal resolution of the free screen according to the gaze point data and the influencing factors, calculating the optimal scanning frequency of the free screen according to the influencing factors and the visual fatigue corresponding to the eye state, and adjusting the picture parameters according to the optimal screen brightness, the optimal resolution and the optimal scanning frequency, and returning to the step of determining the visual fatigue of the target user under the screen display picture according to the area of ​​the region, the gaze point, the number of blinks and the face-screen display distance;

[0076] The safety eye protection report generation module is used to generate a safety eye protection report for the free screen when there is no abnormality in the eye state.

[0077] The embodiment of the present invention can determine the line of sight of the detected user according to the user's face area and gaze point by detecting the face area of ​​the target user under the displayed image, automatically adjust the screen display parameters, and reduce eye fatigue caused by screen discomfort. Optionally, the embodiment of the present invention can help analyze the concentration and eye fatigue state of the target user by identifying the number of blinks of the face area; the embodiment of the present invention determines the eye state of the target user under the displayed image according to the visual fatigue; optimizes the screen display settings by analyzing the visual fatigue, such as adjusting the brightness, contrast, font size, etc., to reduce visual fatigue and protect eye health; the embodiment of the present invention can determine the eye environment of the target user by identifying the external environment and display features of the displayed image when the eye state is abnormal. , thereby automatically enabling screen filtering technology or eye protection mode, such as reducing flicker, filtering harmful light, etc., to protect the eyes from harm; the embodiment of the present invention can automatically adjust the screen display parameters by mining the influencing factors that affect the eye state based on the screen display characteristics and the environmental characteristics, and provide the user with feedback on visual comfort, guide the user to adjust the screen position or angle, remind the user to reduce the screen usage time, and encourage regular rest; especially when it is detected that the visual fatigue reaches a preset value, remind the user to rest, which is helpful to assist the user in eye protection. Finally, the embodiment of the present invention generates a safe eye protection report for the free screen when there is no abnormal state in the eye state. The report analyzes the user's eye habits and screen usage environment, and provides personalized eye protection suggestions accordingly, thereby reminding the user to maintain good eye habits and assisting the user in eye protection. Therefore, the present invention can enhance the eye protection effect of the free screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A schematic diagram of a flow chart of a method for implementing a free screen safety eye protection method based on the Internet of Things provided by an embodiment of the present invention;

[0079] Figure 2 A functional module diagram of a safe eye protection system for realizing a free screen based on the Internet of Things provided by an embodiment of the present invention;

[0080] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0081] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0082] The embodiment of the present application provides a method for safe eye protection based on the Internet of Things to implement a free screen. The execution subject of the method for safe eye protection based on the Internet of Things to implement a free screen includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for safe eye protection based on the Internet of Things to implement a free screen can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0083] Reference Figure 1 FIG. 1 is a flow chart of a method for implementing a safe eye protection for a free screen based on the Internet of Things according to an embodiment of the present invention. In this embodiment, the method for implementing a safe eye protection for a free screen based on the Internet of Things includes:

[0084] S1. Use the Internet of Things to collect the screen display of the free screen, and use the monitoring camera corresponding to the free screen to monitor the target user under the screen display in real time, detect the face area of ​​the target user under the screen display, calculate the area and head posture within the face area, and use the head posture to calculate the gaze point within the face area.

[0085] The embodiment of the present invention can detect the face area of ​​the target user under the screen display image, determine the line of sight direction of the detected user according to the face area and gaze point of the user, automatically adjust the screen display parameters, and reduce eye fatigue caused by screen discomfort. Wherein, the screen display image refers to the screen display image, which usually refers to the visual content presented on a certain display device (such as a liquid crystal display, LED display, electronic paper display, etc.).

[0086] In this specification, the free screen refers to a display device, and the free screen includes at least one of the following devices: a tablet computer, a laptop computer, an extended screen, a display, and a mobile terminal with a display screen.

[0087] The embodiment of the present invention can fully understand the target user's experience when using the free screen by monitoring the target user under the screen display image in real time through the monitoring camera corresponding to the free screen. The monitoring camera refers to an image acquisition device used to capture and record the activities in front of the screen in real time.

[0088] The embodiment of the present invention can optimize the layout and design of the screen content according to the face area and gaze point of the user by detecting the face area of ​​the target user under the screen display, thereby improving the safety eye protection function. The face area refers to a specific area identified and marked in the monitoring picture.

[0089] Optionally, as an embodiment of the present invention, the face area of ​​the target user under the screen display may be detected by image processing technology and computer vision algorithm. The image processing technology refers to a series of algorithms and methods for analyzing, modifying and optimizing digital images. The computer vision algorithm refers to a series of algorithms for extracting, processing and interpreting information from images or videos.

[0090] The embodiment of the present invention can help understand the user's attention direction and interest points by calculating the area and head posture in the face area, thereby improving the user experience and protecting the user's eyesight. The area refers to the pixel area covered by the face area detected in the image. The head posture refers to the position and direction of a person's head relative to a reference coordinate system (usually a horizontal plane and a vertical line).

[0091] As an embodiment of the present invention, the calculating the area and head posture in the face region includes:

[0092] Acquire a facial image of the facial area, and detect facial edges of the facial image;

[0093] Constructing a polygon of the face image according to the face edge, and identifying the vertices of the polygon;

[0094] Based on the vertices, marking the facial contour of the facial image;

[0095] Constructing a two-dimensional coordinate system of the face contour, and determining the vertex coordinates of the vertices according to the two-dimensional coordinate system;

[0096] According to the vertex coordinates, the area of ​​the face region is calculated using the following formula:

[0097]

[0098] Where M represents the area, i represents the i-th vertex corresponding to the face contour, n represents the total number of vertices corresponding to the face contour, and p i represents the horizontal coordinate of the i-th vertex corresponding to the face contour, p i+1 represents the horizontal coordinate of the i+1th vertex corresponding to the face contour, q i Indicates the ordinate of the i-th vertex corresponding to the face contour, q i+1Indicates the ordinate of the i+1th vertex corresponding to the face contour;

[0099] Constructing a three-dimensional face model of the face area according to the face image;

[0100] Constructing a three-dimensional coordinate system of the three-dimensional face model, and determining the nose tip coordinates, left eye coordinates, and right eye coordinates of the face region according to the three-dimensional coordinate system;

[0101] According to the nose tip coordinates, the left eye coordinates and the right eye coordinates, the pitch angle, yaw angle and roll angle of the three-dimensional face model are calculated using the following formula:

[0102]

[0103] Among them, α represents the pitch angle, β represents the yaw angle, γ represents the roll angle, arctan represents the inverse tangent function, and x b represents the x-axis coordinate of the nose tip, y b The y-axis coordinate of the nose tip, z b represents the z-axis coordinate of the nose tip, x l Indicates the x-axis coordinate of the left eye, y l Indicates the y-axis coordinate of the left eye, z l Indicates the z-axis coordinate of the left eye, x r Indicates the x-axis coordinate of the right eye, y r Indicates the y-axis coordinate of the right eye, z r Represents the z-axis coordinate of the right eye coordinate;

[0104] The head posture of the face region is determined according to the pitch angle, the yaw angle, and the roll angle.

[0105] The face image refers to visual data containing a person's facial features captured by an image acquisition device (such as a camera, surveillance camera or other sensor). The face edge refers to the dividing line between the face contour and the background or other non-face area in the face image. The polygon refers to a closed figure formed by approximating the shape in the image as a series of straight line segments in order to simplify the representation and processing of complex shapes in computer vision and image processing. The vertex refers to one of the basic components of a polygon in computer graphics and image processing, which is used to define the shape and boundary of a polygon. The face contour refers to the contour line of the face edge in the face image. The face three-dimensional model refers to a model that can represent a face in the form of three-dimensional space through professional image processing and three-dimensional modeling software. The pitch angle refers to the rotation angle from the horizontal plane to the head. The yaw angle refers to the rotation angle of the head around the Y axis (perpendicular to the horizontal plane). The roll angle refers to the rotation angle of the head around the X axis (perpendicular to the vertical and horizontal planes).

[0106] Optionally, the facial edge of the facial image may be detected by a Canny edge detection algorithm, wherein the Canny edge detection algorithm refers to an edge detection algorithm of non-maximum suppression (NMS) and double threshold processing.

[0107] Optionally, the polygon for constructing the face image according to the face edge may be constructed by a polygon approximation method, and the polygon approximation includes: piecewise linear approximation, Bezier curve fitting, spline curve fitting, and the like.

[0108] Optionally, constructing a three-dimensional face model of the face area according to the face image includes:

[0109] Constructing a basic three-dimensional face model of the face area;

[0110] Detecting feature points of the face image, and mapping model corresponding points of the basic three-dimensional model according to the feature points;

[0111] Determine a rotation matrix and a translation vector of the basic three-dimensional model based on the feature points and the model corresponding points;

[0112] According to the feature points, the model corresponding points, the rotation matrix and the translation vector, the posture parameters and shape parameters of the basic three-dimensional model are calculated using the following formula:

[0113]

[0114] Among them, ZT represents the posture parameter, G represents the shape parameter, arg represents the function parameter, min represents the minimization function, N represents the number of feature points or model corresponding points, and Tj represents the jth feature point, D j represents the jth model corresponding point, A represents the rotation matrix, H represents the translation vector, Proj represents the projection transformation, B j represents the change in the corresponding point of the j-th model;

[0115] According to the posture parameters and the shape parameters, the basic three-dimensional model is mapped to a two-dimensional image space to obtain mapped feature points;

[0116] Calculating the error between the feature point and the mapped feature point,

[0117] When the error is greater than a preset error threshold, determining an objective function of the basic three-dimensional model and the face image according to the error;

[0118] Calculating the parameter gradient of the objective function relative to the posture parameter and the shape parameter, adjusting the posture parameter and the shape parameter according to the parameter gradient to obtain adjusted posture parameter and adjusted shape parameter, and returning to the step of mapping the basic three-dimensional model to a two-dimensional image space according to the posture parameter and the shape parameter to obtain mapped feature points;

[0119] When the error is less than a preset error threshold, the basic three-dimensional model is used as the three-dimensional face model of the face area.

[0120] Among them, the feature points refer to specific points in the face image used to identify the key parts of the face, such as eyes, nose, ears, etc. The model corresponding points are points on the three-dimensional model corresponding to the feature points detected in the two-dimensional image. The rotation matrix refers to a mathematical representation of the rotation of an object in three-dimensional space. It is a 3x3 matrix used to transform a point in three-dimensional space from an original coordinate system to a target coordinate system. The translation vector refers to a vector that describes the change in the position of an object in three-dimensional space. For the posture estimation of the three-dimensional face model, the translation vector is used to describe the movement of the basic three-dimensional model in three-dimensional space relative to the initial position. The posture parameters refer to a set of parameters that describe the rotation and displacement state of an object or human body in space. The shape parameters refer to parameters used to describe the shape of an object in computer vision applications such as three-dimensional modeling and face recognition. The corresponding points on the three-dimensional basic face model are mapped to points on the two-dimensional image plane through a certain transformation (usually a projection transformation). The error refers to the difference between the projection points (mapped feature points) of the three-dimensional basic model on the two-dimensional image and the feature points actually detected in the two-dimensional image. The objective function refers to a mathematical expression used to measure the difference between the model and actual observation data during the three-dimensional model optimization process, and its purpose is to minimize this difference by adjusting parameters.

[0121] Optionally, the model corresponding points of the basic three-dimensional model mapped according to the feature points can be obtained by an iterative closest point algorithm, wherein the iterative closest point algorithm refers to an algorithm for three-dimensional model alignment, which can automatically find the best matching point between two models.

[0122] Optionally, the rotation matrix and translation vector of the basic three-dimensional model based on the feature points and the corresponding points of the model can be determined by a nonlinear optimization method, wherein the nonlinear optimization refers to an algorithm that takes into account the nonlinear relationship between rotation and translation and can handle more complex geometric transformations, such as the Levenberg-Marquardt algorithm.

[0123] The embodiment of the present invention can analyze the user's attention level on a specific screen area by calculating the gaze point in the face area using the head posture, optimize the screen display of the free screen, and thus improve the user experience. The gaze point refers to the position where the target user's line of sight is focused on a specific screen or display area.

[0124] As an embodiment of the present invention, the calculating the gaze point in the face region by using the head posture includes:

[0125] Constructing an initial direction vector of the head posture;

[0126] Determining a head center axis and an eye center axis of the head posture according to the head posture;

[0127] Calculating the head direction vector of the head central axis through the rotation matrix corresponding to the head posture and the initial direction vector;

[0128] Identifying a pupil center of the head posture, and determining an eye direction vector of the eye center axis based on the pupil center;

[0129] The center-axis distance between the center axis of the head and the center axis of the eyes is calculated, and the gaze point in the face area is calculated according to the center-axis distance, the head direction vector and the eye direction vector.

[0130] Wherein, the initial direction vector refers to the starting direction used to determine the center axis of the head and the center axis of the eye during the head posture estimation process. The center axis of the head refers to a virtual axis assumed to pass through the center of the head in a face image or a three-dimensional model. The center axis of the eye refers to an axis passing through the center point of the pupil and perpendicular to the surface of the eye in a face image. The head direction vector refers to a vector describing the direction of the center axis of the head in a face image or a three-dimensional model. This vector is usually pointing from the center point of the head (such as the tip of the nose) to the direction of the center axis of the head. The eye direction vector refers to a vector describing the direction of the center axis of the eye in a face image or a three-dimensional model. This vector is usually pointing from the center point of the eye (usually the center of the pupil) to the direction of the center axis of the eye. The center axis distance refers to the distance between the center axis of the head and the center axis of the eye in a face image or a three-dimensional model. This distance can be used to describe the relative position between the head and the eye.

[0131] S2. Identify the number of blinks of the face area, and calculate the face-screen display distance between the face area and the free screen.

[0132] The embodiment of the present invention can help analyze the concentration and eye fatigue state of the target user by identifying the number of blinks in the face area.

[0133] Optionally, as an embodiment of the present invention, the number of blinks of the face region can be identified by a time series analysis method, wherein the time series analysis refers to a method of identifying the frequency and pattern of blinks by calculating the intervals between blinks.

[0134] The embodiment of the present invention can calculate the face-screen distance between the face area and the free screen and remind the user to adjust the screen position according to the distance calculation result to maintain the optimal face-screen distance. The face-screen distance refers to the spatial distance between the target user and the screen.

[0135] As an embodiment of the present invention, the calculating the face-screen display distance between the face area and the free screen includes:

[0136] A three-dimensional coordinate system of the face area and the free screen is constructed with the surveillance camera corresponding to the free screen as the origin, and the spatial parameters of the free screen in the three-dimensional coordinate system are determined. According to the three-dimensional coordinate system and the spatial parameters, the three-dimensional coordinates of the free screen center of the free screen are determined, and the internal parameters and external parameters of the surveillance camera are extracted, wherein the spatial parameters include position parameters and direction parameters;

[0137] Analyze the pixel coordinates of the face image corresponding to the face area, and convert the pixel coordinates into three-dimensional face coordinates using the following formula according to the internal parameters and external parameters:

[0138] R(X f , Y f , Z f )=PeerspectiveTransform(S(E f , F f ),intrinsic,extrinsic)

[0139] Among them, R(X f , Y f , Z f ) represents the three-dimensional coordinates of the face, X f The X-axis coordinates of the face's three-dimensional coordinates, f The Y-axis coordinate of the face's three-dimensional coordinates, Z f represents the Z-axis coordinate of the three-dimensional coordinates of the face, PerspectiveTransform represents the function of pixel coordinate transformation, S(E f , F f ) represents the pixel coordinates, E f Indicates the X-axis coordinate of the pixel coordinate, F f Indicates the Y-axis coordinate of the pixel coordinate, intrinsic indicates internal parameters, and extrinsic indicates external parameters;

[0140] Based on the three-dimensional coordinates of the face and the three-dimensional coordinates of the center of the free screen, a face-screen display distance between the face area and the free screen is determined.

[0141] The spatial parameters refer to a set of parameters that describe the position and direction of an object in three-dimensional space. The position parameters refer to the position of the free screen in three-dimensional space. The direction parameters refer to the direction of the free screen in three-dimensional space. The internal parameters refer to the properties of the camera itself, which define the geometric and optical properties inside the camera. The external parameters refer to the position and direction of the camera relative to the three-dimensional space. The pixel coordinates refer to the coordinate position of each pixel in a digital image.

[0142] Optionally, the face-display distance between the face area and the free screen determined based on the three-dimensional coordinates of the face and the three-dimensional coordinates of the center of the free screen can be calculated by a Euclidean distance formula.

[0143] S3. Determine the visual fatigue of the target user on the displayed screen according to the area of ​​the region, the gaze point, the number of blinks and the face-display distance, and determine the eye state of the target user on the displayed screen according to the visual fatigue.

[0144] The embodiment of the present invention determines the visual fatigue of the target user under the screen display according to the area, the gaze point, the number of blinks and the face-screen distance, which can be used for in-depth analysis, such as identifying visual fatigue patterns and evaluating the impact of different screen settings. The visual fatigue refers to the degree of fatigue that an individual's eyes and brain experience when processing visual information, evaluated through a series of indicators and methods.

[0145] As an embodiment of the present invention, determining the visual fatigue of the target user on the screen display according to the area of ​​the region, the gaze point, the number of blinks, and the face-screen distance includes:

[0146] Obtaining the screen resolution of the displayed image, determining the gaze point data of the gaze point, wherein the gaze point data includes: a gaze point distribution state and a gaze point movement mode,

[0147] Calculating the gaze area of ​​the target user under the screen display according to the area of ​​the area, the gaze point data and the screen resolution;

[0148] Based on the number of blinks, determining the screen gaze time of the target user;

[0149] Determining the visual pressure of the target user according to the face-screen distance;

[0150] The visual fatigue degree of the target user on the displayed image is calculated based on the gaze area, the screen gaze time, and the visual pressure.

[0151] Among them, the screen resolution refers to the number of pixels that can be displayed on the free screen. The gaze point data refers to the position information of the specific point that the user is gazing at on the screen or display device. The gaze point distribution state refers to the position information of the specific point that the user is gazing at on the screen or display device. The gaze point movement mode refers to the movement trajectory and behavior pattern of the user's eye gaze point on the screen within a certain period of time. The gaze area refers to the main area where the user's eyes focus when looking at the screen. The visual pressure refers to the pressure on the eyes caused by visual task requirements and screen parameters (such as resolution, contrast, brightness, etc.) when looking at the screen.

[0152] Optionally, the gaze point data for determining the gaze point may be determined by a gaze point detection method based on image processing and pattern recognition.

[0153] Optionally, the calculating the visual fatigue of the target user on the displayed image based on the gaze area, the screen gaze time, and the visual pressure includes:

[0154] Determine a gaze area weight, a screen gaze time weight, and a visual stress weight of the gaze area, the screen gaze time, and the visual stress;

[0155] The visual fatigue degree of the target user under the screen display is calculated using the following formula according to the gaze area weight, the screen gaze time weight and the visual pressure weight:

[0156] VF=WGA×GA+WGD×GD+WVS×VS

[0157] Among them, VF represents visual fatigue, WGA represents gaze area weight, GA represents gaze area, WGD represents screen gaze time weight, GD represents screen gaze time, WVS represents visual stress weight, and VS represents visual stress. In actual application, the gaze area weight, the screen gaze time weight, and the visual stress weight are adjusted based on the eye usage. For example, the gaze area weight, the screen gaze time weight, and the visual stress weight can be set to 0.3, 0.3, and 0.4, respectively.

[0158] The embodiment of the present invention determines the eye state of the target user under the screen display according to the visual fatigue degree, and can optimize the screen display settings by analyzing the visual fatigue degree, such as adjusting the brightness, contrast, font size, etc., to reduce visual fatigue and protect eye health. The eye state refers to the health status, functional performance, and physiological and psychological characteristics related to vision of the eyes.

[0159] As an embodiment of the present invention, determining the eye state of the target user on the screen display according to the visual fatigue degree includes:

[0160] Extracting a facial image corresponding to the target user, and identifying eye features of the facial image;

[0161] Based on the eye features, determining the pupil state and eye wetness of the target user;

[0162] Determining the eye-face movement state of the target user according to the gaze point data corresponding to the visual fatigue degree;

[0163] Determining an eye health index of the target user according to the pupil state, the eye wetness, and the eyelid movement state;

[0164] The eye state of the target user on the screen display is determined according to the visual fatigue degree and the eye health index.

[0165] Among them, the eye features refer to the various identifiable and quantifiable parts of the eye and their properties. The pupil state refers to the various physiological and morphological characteristics of the pupil, which can reflect the individual's physiological state, emotional changes, and response to external stimuli. The eye wetness refers to the moisture state of the eye surface, which reflects the degree of lubrication of the eyeball surface and is an important indicator for measuring eye comfort and health. The eyelid movement state refers to the eyelid movement state over a period of time. The eye health index is a series of parameters used to evaluate the function and structure of the eye.

[0166] Optionally, the identifying of the eye features of the face image may be performed by a local binary pattern, where the local binary pattern refers to an algorithm that describes the local structure of an image by comparing a pixel with pixel values ​​in its neighborhood.

[0167] S4. When the eye state is abnormal, identify the external environment and display features of the screen display, extract the environmental features of the external environment, mine the influencing factors affecting the eye state based on the display features and the environmental features, extract the gaze point data corresponding to the eye state, determine the pupil size and gaze duration of the eye state, obtain the picture parameters of the free screen, the picture parameters include screen brightness, resolution and scanning frequency, determine the light brightness in the influencing factors, determine the optimal screen brightness of the free screen according to the light brightness and the pupil size, calculate the optimal resolution of the free screen according to the gaze point data and the influencing factors, calculate the optimal scanning frequency of the free screen according to the influencing factors and the visual fatigue corresponding to the eye state, adjust the picture parameters according to the optimal screen brightness, the optimal resolution and the optimal scanning frequency, and then return to execute the step of determining the visual fatigue of the target user on the screen display according to the area of ​​the area, the gaze point, the number of blinks and the face-screen distance.

[0168] The embodiment of the present invention can determine the target user's eye environment by identifying the external environment and screen display features of the screen display when the eye state is abnormal, thereby automatically enabling screen filtering technology or eye protection mode, such as reducing flicker, filtering harmful light, etc., to protect the eyes from damage. Among them, the abnormal state refers to the state when the visual fatigue reaches moderate visual fatigue. For example, when the visual fatigue is 3-6, the eyes will feel dry, sore, painful, foreign body sensation, blurred vision, etc. The external environment refers to the various conditions and factors surrounding the screen display system and the user's environment. The screen display features refer to the various attributes and technical specifications of the screen display system in terms of visual presentation, which directly affect the user's visual experience and eye health.

[0169] The embodiment of the present invention can help analyze the visual interference of the external environment to the target user and clarify the stimulation factors to the eyes of the target user by extracting the environmental features of the external environment. The environmental features refer to various conditions in the user's surrounding environment that affect the screen display effect and the user's visual experience.

[0170] As an embodiment of the present invention, the extracting environmental features of the external environment includes:

[0171] Extracting initial environmental data of the external environment using a preset sensor network;

[0172] De-noising the initial environmental data to obtain de-noised environmental data, and identifying abnormal values ​​of the de-noised environmental data;

[0173] Performing logarithmic transformation on the abnormal value to obtain environmental data;

[0174] Calculate the light parameters and the ambient color temperature parameters of the external environment according to the environmental data, wherein the light parameters include light intensity, light intensity change rate and light distribution, and the ambient color temperature parameters include: color temperature change range and color temperature change frequency;

[0175] The environmental characteristics of the external environment are determined according to the light parameters and the environmental color temperature parameters.

[0176] Among them, the preset sensor network refers to the preset sensor network, which refers to a series of sensors pre-installed and configured in a specific application scenario, such as light sensors, temperature sensors, etc. The initial environmental data refers to the original data about the external environment collected by the preset sensor network. The abnormal value refers to the value that deviates from the normal value in the data set. The light parameter refers to the parameter used to describe the characteristics of light in environmental monitoring. The light intensity refers to the description of the brightness of the light in the environment. The light intensity change rate refers to the description of the speed of change of light intensity over time. The light distribution refers to the description of the distribution of light in space. The environmental color temperature parameter refers to the parameter that describes the color temperature characteristics of the environmental lighting. The color temperature change range refers to the description of the change range of the environmental lighting color temperature within a certain period of time. The color temperature change frequency refers to the description of the frequency of the change of the environmental lighting color temperature over time.

[0177] Optionally, the denoising of the initial environmental data can be achieved through filter technology, wherein the filter technology refers to the use of specific algorithms in signal processing and image processing to filter or smooth signal or image data to remove noise, interference or outliers, thereby improving the quality and analyzability of the data.

[0178] The embodiment of the present invention can automatically adjust the screen display parameters by mining the influencing factors that affect the eye state based on the screen display features and the environmental features, and provide visual comfort feedback to the user, guide the user to adjust the screen position or angle, remind the user to reduce the screen usage time, and encourage regular rest. The influencing factors refer to those factors that can affect the eye state, which may include environmental factors, screen display features, user personal habits, etc.

[0179] As an embodiment of the present invention, mining the influencing factors affecting the eye state based on the screen display features and the environmental features includes:

[0180] Respectively calculating the correlation between the display feature and the environment feature and the eye state to obtain a display-eye correlation and an environment-eye correlation;

[0181] When the screen display-eye correlation degree and the environment-eye correlation degree are both greater than a preset threshold, extracting correlation features from the screen display features and the environment features;

[0182] According to the associated features, an influencing factor affecting the eye state is determined.

[0183] The display-eye correlation refers to the correlation between display features (such as brightness, contrast, resolution, etc.) and eye status (such as pupil size, blinking frequency, visual fatigue, etc.). The environment-eye correlation refers to the correlation between environmental features (such as light intensity, temperature, humidity, etc.) and eye status. The correlation features refer to those features that have significant correlation between display features and environmental features and eye status. These features can help determine the key factors affecting eye status.

[0184] Optionally, the respectively calculating the correlation between the display feature and the environment feature and the eye state can be obtained by calculating a correlation coefficient.

[0185] The embodiment of the present invention extracts the gaze point data corresponding to the eye state to analyze the user's gaze behavior, thereby evaluating the user's visual fatigue level.

[0186] The embodiment of the present invention can understand the target user's eye usage and analyze the target user's eye health risks by determining the pupil size and gaze duration of the eye state. The pupil size refers to the diameter of the pupil in the eye under certain conditions. The gaze duration refers to the length of time the user gazes at a specific point or area on the screen.

[0187] Optionally, as an embodiment of the present invention, the pupil size and gaze duration for determining the eye state may be determined by video eye tracking technology.

[0188] The embodiment of the present invention can better analyze the impact of the free screen image on the target user's eyes by obtaining the image parameters of the free screen. The image parameters refer to various technical specifications and settings involved in displaying an image on the free screen, and the image parameters include screen brightness, resolution, and scanning frequency.

[0189] The embodiment of the present invention can provide a data basis for adjusting the resolution later by determining the light brightness in the influencing factor.

[0190] Optionally, the light brightness in determining the influencing factor can be obtained by an optical sensor.

[0191] The embodiment of the present invention determines the optimal screen brightness of the free screen according to the light brightness and the pupil size, thereby ensuring that the screen brightness matches the light brightness of the surrounding environment, reducing the need for the eyes to adjust to different brightness levels, and thus reducing eye fatigue. The optimal screen brightness refers to the brightness level that the screen should be set to under specific lighting conditions in order to reduce eye fatigue, protect eyesight, and provide a comfortable visual experience.

[0192] Optionally, as an embodiment of the present invention, determining the optimal screen brightness of the free screen according to the light illumination and the pupil size includes:

[0193] determining a standard screen brightness of the free screen,

[0194] According to the standard screen brightness, the light brightness and the pupil size, the optimal screen brightness of the free screen is calculated using the following formula:

[0195] Br=Br0+pQ light +qD pupil

[0196] Among them, Br represents the optimal screen brightness, Br0 represents the standard screen brightness, p represents the illumination adjustment coefficient, Q light represents the light intensity, q represents the pupil adjustment coefficient, D pupil Indicates pupil size; in actual applications, the illumination adjustment coefficient and the pupil adjustment coefficient can be adjusted according to eye usage, for example, the illumination adjustment coefficient is set to 0.4, and the pupil adjustment coefficient is set to 0.6.

[0197] Calculating the optimal resolution of the free screen based on the gaze point data and the influencing factors can improve the clarity and detail display of the image, thereby enhancing the user's visual experience and protecting eye health. The optimal resolution refers to the pixel density at which the free screen can clearly display the image in order to provide the best visual experience and comfort under specific usage conditions.

[0198] The embodiment of the present invention can reduce the visual fatigue of the target user and improve the user experience by calculating the optimal scanning frequency of the free screen according to the influencing factor and the visual fatigue corresponding to the eye state. The optimal scanning frequency refers to the number of times the free screen refreshes the screen per second in order to provide the best visual experience and comfort under specific usage conditions.

[0199] The embodiment of the present invention can adapt to the user's visual needs, protect eye health, and improve user experience by adjusting the image parameters according to the optimal screen brightness, the optimal resolution, and the optimal scanning frequency.

[0200] Optionally, the picture parameters may be adjusted according to the optimal screen brightness, the optimal resolution and the optimal scanning frequency through intelligent identification and adjustment by an adaptive adjustment system.

[0201] S5. When there is no abnormality in the eye state, generate a safety eye protection report for the free screen.

[0202] The embodiment of the present invention generates a safe eye protection report for the free screen when there is no abnormal state in the eye state. The report analyzes the user's eye habits and screen usage environment, and provides personalized eye protection suggestions accordingly, thereby reminding the user to maintain good eye habits, educating the user on the knowledge and importance of eye protection, and improving the user's understanding of eye health and self-protection awareness.

[0203] In actual applications, when the eye state is abnormal (i.e., when visual fatigue is detected to reach a preset value), a prompt message is sent to the free screen for display through the Internet of Things, wherein the prompt information includes: the current face-screen distance and the face-screen distance that is most conducive to eye protection, influencing factors, number of blinks, visual fatigue, and environmental characteristics.

[0204] The embodiment of the present invention can determine the line of sight of the detected user according to the user's face area and gaze point by detecting the face area of ​​the target user under the displayed image, automatically adjust the screen display parameters, and reduce eye fatigue caused by screen discomfort. Optionally, the embodiment of the present invention can help analyze the concentration and eye fatigue state of the target user by identifying the number of blinks of the face area; the embodiment of the present invention determines the eye state of the target user under the displayed image according to the visual fatigue; optimizes the screen display settings by analyzing the visual fatigue, such as adjusting the brightness, contrast, font size, etc., to reduce visual fatigue and protect eye health; the embodiment of the present invention can determine the eye environment of the target user by identifying the external environment and display features of the displayed image when the eye state is abnormal. , thereby automatically enabling screen filtering technology or eye protection mode, such as reducing flicker, filtering harmful light, etc., to protect the eyes from harm; the embodiment of the present invention can automatically adjust the screen display parameters by mining the influencing factors that affect the eye state based on the screen display characteristics and the environmental characteristics, and provide the user with feedback on visual comfort, guide the user to adjust the screen position or angle, remind the user to reduce the screen usage time, and encourage regular rest; especially when it is detected that the visual fatigue reaches a preset value, remind the user to rest, which is helpful to assist the user in eye protection. Finally, the embodiment of the present invention generates a safe eye protection report for the free screen when there is no abnormal state in the eye state. The report analyzes the user's eye habits and screen usage environment, and provides personalized eye protection suggestions accordingly, thereby reminding the user to maintain good eye habits and assisting the user in eye protection. Therefore, the present invention can enhance the eye protection effect of the free screen.

[0205] like Figure 2 As shown, it is a functional module diagram of a safe eye protection system for realizing a free screen based on the Internet of Things provided by an embodiment of the present invention.

[0206] The safety eye protection system 200 for realizing a free screen based on the Internet of Things of the present invention can be installed in an electronic device. According to the functions implemented, the safety eye protection system 200 for realizing a free screen based on the Internet of Things may include a gaze point analysis module 201, a face-screen display distance calculation module 202, an eye state analysis module 203, a screen display automatic adjustment module 204 and a safety eye protection report generation module 205. The module described in the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0207] In this embodiment, the functions of each module / unit are as follows:

[0208] The gaze point analysis module 201 is used to collect the screen display of the free screen by using the Internet of Things, and monitor the target user under the screen display in real time through the monitoring camera corresponding to the free screen, detect the face area of ​​the target user under the screen display, calculate the area and head posture of the face area, and calculate the gaze point in the face area by using the head posture;

[0209] The face-display distance calculation module 202 is used to identify the number of blinks of the face area and calculate the face-display distance between the face area and the free screen;

[0210] The eye state analysis module 203 is used to determine the visual fatigue of the target user on the screen according to the area, the gaze point, the number of blinks and the face-screen distance, and determine the eye state of the target user on the screen according to the visual fatigue;

[0211] The screen display automatic adjustment module 204 is used to identify the external environment and screen display features of the screen display picture when the eye state is in an abnormal state, and extract the environmental features of the external environment, based on the screen display features and the environmental features, dig out the influencing factors that affect the eye state, and extract the gaze point data corresponding to the eye state, determine the pupil size and gaze duration of the eye state, obtain the picture parameters of the free screen, the picture parameters include screen brightness, resolution and scanning frequency, determine the light brightness in the influencing factors, determine the optimal screen brightness of the free screen according to the light brightness and the pupil size, calculate the optimal resolution of the free screen according to the gaze point data and the influencing factors, calculate the optimal scanning frequency of the free screen according to the influencing factors and the visual fatigue corresponding to the eye state, and after adjusting the picture parameters according to the optimal screen brightness, the optimal resolution and the optimal scanning frequency, return to the step of determining the visual fatigue of the target user under the screen display picture according to the area, the gaze point, the number of blinks and the face-screen display distance;

[0212] The safety eye protection report generating module 205 is used to generate a safety eye protection report for the free screen when there is no abnormality in the eye state.

[0213] An embodiment of the present invention provides an electronic device for implementing a safe eye protection method for a free screen based on the Internet of Things; by implementing the electronic device for implementing a safe eye protection method for a free screen based on the Internet of Things, by detecting the face area of ​​the target user under the displayed image, the line of sight of the detected user can be determined according to the face area and gaze point of the user, and the screen display parameters can be automatically adjusted to reduce eye fatigue caused by screen discomfort. Optionally, the embodiment of the present invention can help analyze the concentration and eye fatigue state of the target user by identifying the number of blinks in the face area; the embodiment of the present invention determines the eye state of the target user under the displayed image based on the visual fatigue; by analyzing the visual fatigue, the screen display settings are optimized, such as adjusting the brightness, contrast, font size, etc., to reduce visual fatigue and protect eye health; the embodiment of the present invention identifies the abnormal state of the eye when there is an abnormal state. The external environment and screen display features of the screen display can determine the target user's eye environment, thereby automatically enabling screen filtering technology or eye protection mode, such as reducing flicker, filtering harmful light, etc., to protect the eyes from damage; the embodiment of the present invention can automatically adjust the screen display parameters by mining the influencing factors that affect the eye state based on the screen display features and the environmental features, and provide the user with feedback on visual comfort, guide the user to adjust the screen position or angle, remind the user to reduce the screen usage time, and encourage regular rest; especially when it is detected that the visual fatigue reaches a preset value, remind the user to rest, which is helpful to assist the user in eye protection. Finally, the embodiment of the present invention generates a safe eye protection report for the free screen when there is no abnormal state in the eye state. The report analyzes the user's eye habits and screen usage environment, and provides personalized eye protection suggestions accordingly, thereby reminding the user to maintain good eye habits and assisting the user in eye protection. Therefore, the present invention can enhance the eye protection effect of the free screen.

Claims

1. A safe eye protection method for realizing free screen based on Internet of Things, characterized in that: The method comprises: The screen display of the free screen is collected by using the Internet of Things, and the target user under the screen display is monitored in real time by the monitoring camera corresponding to the free screen, the face area of ​​the target user under the screen display is detected, the area and the head posture of the face area are calculated, and the gaze point in the face area is calculated by using the head posture; Identify the number of blinks of the face area, and calculate the face-screen distance between the face area and the free screen; Determine the visual fatigue of the target user on the screen according to the area, the gaze point, the number of blinks and the face-screen distance, and determine the eye state of the target user on the screen according to the visual fatigue; When the eye state is in an abnormal state, the external environment and screen display features of the screen display are identified, and the environmental features of the external environment are extracted. Based on the screen display features and the environmental features, the influencing factors affecting the eye state are mined, and the gaze point data corresponding to the eye state is extracted to determine the pupil size and gaze duration of the eye state, and the picture parameters of the free screen are obtained, the picture parameters include screen brightness, resolution and scanning frequency, the light brightness in the influencing factors is determined, and the optimal screen brightness of the free screen is determined according to the light brightness and the pupil size, the optimal resolution of the free screen is calculated according to the gaze point data and the influencing factors, and the optimal scanning frequency of the free screen is calculated according to the influencing factors and the visual fatigue corresponding to the eye state, and after adjusting the picture parameters according to the optimal screen brightness, the optimal resolution and the optimal scanning frequency, the step of determining the visual fatigue of the target user under the screen display according to the area of ​​the region, the gaze point, the number of blinks and the face-screen display distance is returned to execute; When there is no abnormality in the eye state, a safety eye protection report for the free screen is generated.

2. The method for protecting eyes with a safe screen based on the Internet of Things as claimed in claim 1, characterized in that: The calculating the area and head posture of the face region includes: Acquire a facial image of the facial area, and detect facial edges of the facial image; Constructing a polygon of the face image according to the face edge, and identifying the vertices of the polygon; Based on the vertices, marking the facial contour of the facial image; Constructing a two-dimensional coordinate system of the face contour, and determining the vertex coordinates of the vertices according to the two-dimensional coordinate system; According to the vertex coordinates, the area of ​​the face region is calculated using the following formula: Where M represents the area, i represents the i-th vertex corresponding to the face contour, n represents the total number of vertices corresponding to the face contour, and p i represents the horizontal coordinate of the i-th vertex corresponding to the face contour, p i+1 represents the horizontal coordinate of the i+1th vertex corresponding to the face contour, q i Indicates the ordinate of the i-th vertex corresponding to the face contour, q i+1 Indicates the ordinate of the i+1th vertex corresponding to the face contour; Constructing a three-dimensional face model of the face area according to the face image; Constructing a three-dimensional coordinate system of the three-dimensional face model, and determining the nose tip coordinates, left eye coordinates, and right eye coordinates of the face region according to the three-dimensional coordinate system; According to the nose tip coordinates, the left eye coordinates and the right eye coordinates, the pitch angle, yaw angle and roll angle of the three-dimensional face model are calculated using the following formula: Among them, α represents the pitch angle, β represents the yaw angle, γ represents the roll angle, arctan represents the inverse tangent function, and x b represents the x-axis coordinate of the nose tip, y b The y-axis coordinate of the nose tip, z b represents the z-axis coordinate of the nose tip, x l Indicates the x-axis coordinate of the left eye, y l Indicates the y-axis coordinate of the left eye, z l Indicates the z-axis coordinate of the left eye, x r Indicates the x-axis coordinate of the right eye, y r Indicates the y-axis coordinate of the right eye, z r Represents the z-axis coordinate of the right eye coordinate; The head posture of the face region is determined according to the pitch angle, the yaw angle, and the roll angle.

3. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 2, characterized in that: The step of constructing a three-dimensional face model of the face area according to the face image includes: Constructing a basic three-dimensional face model of the face area; Detecting feature points of the face image, and mapping model corresponding points of the basic three-dimensional model according to the feature points; Determine a rotation matrix and a translation vector of the basic three-dimensional model based on the feature points and the model corresponding points; According to the feature points, the model corresponding points, the rotation matrix and the translation vector, the posture parameters and shape parameters of the model of the basic three-dimensional model are calculated using the following formula: Among them, ZT represents the posture parameter, G represents the shape parameter, arg represents the function parameter, min represents the minimization function, N represents the number of feature points or model corresponding points, and T j represents the jth feature point, D j represents the jth model corresponding point, A represents the rotation matrix, H represents the translation vector, Proj represents the projection transformation, B j represents the change in the corresponding point of the j-th model; According to the posture parameters and the shape parameters, the basic three-dimensional model is mapped to a two-dimensional image space to obtain mapped feature points; Calculating the error between the feature point and the mapped feature point, When the error is greater than a preset error threshold, determining an objective function of the basic three-dimensional model and the face image according to the error; Calculating the parameter gradient of the objective function relative to the posture parameter and the shape parameter, adjusting the posture parameter and the shape parameter according to the parameter gradient to obtain adjusted posture parameter and adjusted shape parameter, and returning to the step of mapping the basic three-dimensional model to a two-dimensional image space according to the posture parameter and the shape parameter to obtain mapped feature points; When the error is less than a preset error threshold, the basic three-dimensional model is used as the three-dimensional face model of the face area.

4. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 1, characterized in that: The calculating the gaze point in the face region by using the head posture includes: Constructing an initial direction vector of the head posture; Determining a head center axis and an eye center axis of the head posture according to the head posture; Calculating the head direction vector of the head central axis through the rotation matrix corresponding to the head posture and the initial direction vector; Identifying a pupil center of the head posture, and determining an eye direction vector of the eye center axis based on the pupil center; The center axis distance between the center axis of the head and the center axis of the eye is calculated, and the gaze point in the face area is calculated according to the center axis distance, the head direction vector and the eye direction vector.

5. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 1, characterized in that: The calculating the face-screen display distance between the face area and the free screen includes: A three-dimensional coordinate system of the face area and the free screen is constructed with the surveillance camera corresponding to the free screen as the origin, and the spatial parameters of the free screen in the three-dimensional coordinate system are determined. According to the three-dimensional coordinate system and the spatial parameters, the three-dimensional coordinates of the free screen center of the free screen are determined, and the internal parameters and external parameters of the surveillance camera are extracted, wherein the spatial parameters include position parameters and direction parameters; Analyze the pixel coordinates of the face image corresponding to the face area, and convert the pixel coordinates into three-dimensional face coordinates using the following formula according to the internal parameters and external parameters: R(X f ,Y f ,Z f )=PerspectiveTransform(S(E f ,F f ),intrinsic,extrinsic) Among them, R(X f , Y f , Z f ) represents the three-dimensional coordinates of the face, X f The X-axis coordinates of the face's three-dimensional coordinates, f The Y-axis coordinate of the face's three-dimensional coordinates, Z f represents the Z-axis coordinate of the three-dimensional coordinates of the face, PerspectiveTransform represents the function of pixel coordinate transformation, S(E f , F f ) represents the pixel coordinates, E f Indicates the X-axis coordinate of the pixel coordinate, F f Indicates the Y-axis coordinate of the pixel coordinate, intrinsic indicates internal parameters, and extrinsic indicates external parameters; Based on the three-dimensional coordinates of the face and the three-dimensional coordinates of the center of the free screen, a face-screen display distance between the face area and the free screen is determined.

6. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 1, characterized in that: The determining the visual fatigue degree of the target user on the screen display according to the area of ​​the region, the gaze point, the number of blinks and the face-screen distance includes: Obtaining the screen resolution of the displayed image, determining the gaze point data of the gaze point, wherein the gaze point data includes: a gaze point distribution state and a gaze point movement mode, Calculating the gaze area of ​​the target user under the screen display according to the area of ​​the area, the gaze point data and the screen resolution; Based on the number of blinks, determining the screen gaze time of the target user; Determining the visual pressure of the target user according to the face-screen distance; The visual fatigue degree of the target user on the displayed image is calculated based on the gaze area, the screen gaze time, and the visual pressure.

7. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 1, characterized in that: The step of determining the eye state of the target user on the screen display according to the visual fatigue degree includes: Extracting a facial image corresponding to the target user, and identifying eye features of the facial image; Based on the eye features, determining the pupil state and eye wetness of the target user; Determining the eye-face movement state of the target user according to the gaze point data corresponding to the visual fatigue degree; Determining an eye health index of the target user according to the pupil state, the eye wetness, and the eyelid movement state; The eye state of the target user on the screen display is determined according to the visual fatigue degree and the eye health index.

8. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 1, characterized in that: The mining of influencing factors affecting the eye state based on the screen display features and the environment features includes: Respectively calculating the correlation between the display feature and the environment feature and the eye state to obtain a display-eye correlation and an environment-eye correlation; When the screen display-eye correlation degree and the environment-eye correlation degree are both greater than a preset threshold, extracting correlation features from the screen display features and the environment features; According to the associated features, an influencing factor affecting the eye state is determined.

9. The method for realizing a safe eye protection of a free screen based on the Internet of Things as claimed in claim 1, characterized in that: The extracting the environmental features of the external environment includes: Extracting initial environmental data of the external environment using a preset sensor network; De-noising the initial environmental data to obtain de-noised environmental data, and identifying abnormal values ​​of the de-noised environmental data; Performing logarithmic transformation on the abnormal value to obtain environmental data; Calculate the light parameters and the ambient color temperature parameters of the external environment according to the environmental data, wherein the light parameters include light intensity, light intensity change rate and light distribution, and the ambient color temperature parameters include: color temperature change range and color temperature change frequency; The environmental characteristics of the external environment are determined according to the light parameters and the environmental color temperature parameters.

10. A safe eye protection system based on the Internet of Things to achieve a free screen, characterized in that: The system is used to execute the safe eye protection method for realizing a free screen based on the Internet of Things as described in any one of claims 1 to 9, and the system comprises: A gaze point analysis module is used to collect the screen display of the free screen by using the Internet of Things, and monitor the target user under the screen display in real time through the monitoring camera corresponding to the free screen, detect the face area of ​​the target user under the screen display, calculate the area and head posture of the face area, and calculate the gaze point in the face area by using the head posture; A face-display distance calculation module, used to identify the number of blinks of the face area and calculate the face-display distance between the face area and the free screen; an eye state analysis module, for determining the visual fatigue of the target user on the screen display according to the area, the gaze point, the number of blinks and the face-screen distance, and determining the eye state of the target user on the screen display according to the visual fatigue; A screen display automatic adjustment module, used for identifying the external environment and screen display features of the screen display picture when the eye state is in an abnormal state, and extracting the environmental features of the external environment, mining the influencing factors affecting the eye state based on the screen display features and the environmental features, and extracting the gaze point data corresponding to the eye state, determining the pupil size and gaze duration of the eye state, and obtaining the picture parameters of the free screen, wherein the picture parameters include screen brightness, resolution, and scanning frequency, calculating the optimal screen brightness of the free screen according to the gaze point data and the influencing factors, determining the light brightness in the influencing factors, determining the optimal resolution of the free screen according to the light brightness and the pupil size, calculating the optimal scanning frequency of the free screen according to the influencing factors and the visual fatigue corresponding to the eye state, and adjusting the picture parameters according to the optimal screen brightness, the optimal resolution, and the optimal scanning frequency, and returning to the step of determining the visual fatigue of the target user under the screen display picture according to the area of ​​the region, the gaze point, the number of blinks, and the face-screen display distance; The safety eye protection report generation module is used to generate a safety eye protection report for the free screen when there is no abnormality in the eye state.

Images