Method and mobile device for preventing myopia

Abstract

The present invention provides a method for preventing myopia and a mobile device, which are used to remind users to use their eyes correctly to prevent myopia. The method includes: when the screen is turned on, tracking the user's facial image through a front camera, and calibrating the coordinates of each feature point on the facial image; determining whether the user has a correct viewing posture based on the coordinates of each feature point on the facial image; if not, determining the duration of the user maintaining an incorrect viewing posture; when the duration is greater than a first threshold, sending a first prompt message to the user, the first prompt message being used to prompt the user to maintain a correct eye posture.

Description

[Technical field]

[0001] The invention relates to the field of eye detection, and in particular to a method for preventing myopia and a mobile device. [Background technology]

[0002] During the growth process of children, children read, write, and use various electronic devices at a high frequency. Myopia, strabismus, hunchback, etc. are common diseases among school students. The main reason is that students have improper sitting postures during the long-term learning process. Students' bad habits are formed over a long period of time and it is difficult to solve them simply through teachers' reminders. Especially myopia, in recent years, the incidence of myopia has gradually increased. At the same time, the onset of myopia also has a trend of getting lower and lower in age. In the process of learning, it is extremely important for students to maintain a correct sitting posture, a suitable reading and viewing angle, a suitable reading light intensity, and moderate eye use to prevent myopia. Various devices or equipment have appeared in the prior art for correcting students' bad eye habits, such as sitting posture correctors and various desk lamps, etc., to prevent myopia.

[0003] It should be noted that now more and more myopia is caused by the use of mobile devices such as mobile phones, iPads, etc. Therefore, how to correct users' bad eye habits when using mobile devices is an urgent problem that needs to be solved. [Summary of the invention]

[0004] In order to solve the problem of myopia caused by bad eye habits of users when using mobile devices, the present invention provides a method for preventing myopia and a mobile device.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for preventing myopia, the method is applied to a mobile device, and comprises the following steps: when the screen is turned on, tracking the user's facial image through the front camera, and calibrating the coordinates of each feature point on the facial image; determining whether the user meets the correct viewing posture according to the coordinates of each feature point on the facial image; if not, determining the duration of the user maintaining the incorrect viewing posture; when the duration is greater than a first threshold, sending a first prompt message to the user, the first prompt message being used to prompt the user to maintain the correct eye posture.

[0006] Preferably, the first prompt message is used to prompt the user that the viewing distance is unhealthy; or, the first prompt message is used to prompt the user that the viewing angle is unhealthy; or, the first prompt message is used to prompt the user that the viewing posture is unhealthy.

[0007] Preferably, when the first prompt message is used to prompt the user that the viewing distance is unhealthy, determining whether the user complies with the correct viewing posture based on the coordinates of each feature point on the facial image includes: determining the comprehensive imaging size of the user's characteristic parts based on the coordinates of each feature point; calculating the viewing distance based on the comprehensive imaging size of the characteristic parts and preset standard data; when the viewing distance is greater than the eye safety distance, determining that the user does not comply with the correct viewing posture.

[0008] Preferably, the characteristic part is a pupil, the integrated imaging size of the characteristic part is a pupil distance, and determining the integrated imaging size of the characteristic part of the user comprises: calculating the pupil distance L between the two eyes of the user by the following formula: The coordinates of the left pupil are (x1, y1), and the coordinates of the right pupil are (x2, y2).

[0009] The calculating of the viewing distance includes: calculating the horizontal viewing angle and the vertical viewing angle according to the deviation of the left pupil and the right pupil from the origin of the picture based on the following formula: α=|(x2+x1)|; β=|(y2+y1)|; wherein, the α is related to the horizontal viewing angle, the β is related to the vertical viewing angle, and the center point of the picture is a pre-calibrated origin of the coordinate system; when the α is less than a second threshold and the β is less than a third threshold, calculating the observation angle according to the horizontal viewing angle and the vertical viewing angle; calculating the viewing distance according to the observation angle and the pupil distance between the two eyes of the user by the following formula: M=L / cosδ; wherein, the δ is used to represent the observation angle, and the L is used to represent the pupil distance between the two eyes of the user.

[0010] Preferably, when the α is greater than the second threshold or the β is greater than the third threshold, a first prompt message is sent to the user to prompt the user that the viewing angle is unhealthy.

[0011] Preferably, when the first prompt message is used to prompt the user that the viewing posture is unhealthy, determining whether the user conforms to the correct viewing posture includes: connecting a line from the center point of the pupil to the philtrum in the facial image and extending it in both directions to segment the facial image; counting the sum of skin color pixels in the segmented facial image to determine the left face area and the right face area respectively; calculating the strabismus degree according to the formula d=(LR) / (L+R), wherein d is used to represent the strabismus degree, L is used to represent the left face area, and R is used to represent the right face area; when the strabismus degree is greater than a fourth threshold, determining that the user does not conform to the correct viewing posture.

[0012] Preferably, when the first prompt message is used to prompt the user that the viewing posture is unhealthy, determining whether the user conforms to the correct viewing posture includes: determining that the user is viewing a mobile device based on the user's facial image; inferring the facial spatial posture from the relative spatial position of the mobile phone and the face, and the horizontal posture data of the nine-axis gyroscope in the mobile device; if the horizontal posture data is within a preset range and the facial spatial posture is within the preset range, determining whether the user is viewing in a side-lying or supine position, and determining that the user does not conform to the correct viewing posture.

[0013] Preferably, the first prompt message includes: being displayed above the screen of the mobile device; or covering the screen of the mobile device; or sending a message to a preset phone number.

[0014] The present invention also provides a mobile device for preventing myopia, comprising: a camera, which is used to track the user's facial image and calibrate the coordinates of each feature point on the facial image when the screen is turned on; a processor, which is used to determine whether the user complies with the correct viewing posture based on the coordinates of each feature point on the facial image; if not, determine the duration for which the user maintains the incorrect viewing posture; and a display module, which is used to send a first prompt message to the user when the duration is greater than a first threshold, and the first prompt message is used to prompt the user to maintain the correct eye posture.

[0015] The embodiment of the present application provides a method for preventing myopia, which specifically includes: when the screen is turned on, tracking the user's facial image through the front camera and calibrating the coordinates of each feature point on the facial image; determining whether the user meets the correct viewing posture according to the coordinates of each feature point on the facial image; if not, determining the duration of the user maintaining the incorrect viewing posture; when the duration is greater than a first threshold, sending a first prompt message to the user, the first prompt message is used to prompt the user to maintain the correct eye posture. When it is determined that the user is using the eyes incorrectly during the use of the mobile device, the user is reminded to maintain the correct eye posture to prevent myopia.

Brief Description of the Drawings

[0016] Figure 1 It is a flow chart of a method for preventing myopia provided in an embodiment of the present application;

[0017] Figure 2 A schematic diagram of a flow chart for determining whether a user has a correct viewing posture is provided in an embodiment of the present application. [Specific implementation method]

[0018] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In order to solve the problem of myopia caused by bad eye habits of users when using mobile devices, the present invention provides a method for preventing myopia and a mobile device, which are used to prompt users to correct bad eye habits when using mobile devices to prevent myopia.

[0020] See also Figure 1 , which is a flow chart of a method for preventing myopia provided by the present invention, specifically comprising the following steps:

[0021] 101. When the screen is on, the user's facial image is tracked through the front camera, and the coordinates of each feature point on the facial image are calibrated;

[0022] When a user is using a mobile device and the screen of the mobile device is displayed brightly, the face image of the user is tracked by the front camera of the mobile device. After the front camera captures the face image, the coordinates of each feature point are calibrated on the face image. It should be noted that the feature points in the face image may be pupils, eye corner feature points, eye center feature points, etc., which are not specifically limited here.

[0023] In addition, there are many ways to calibrate the coordinates of each feature point on the face image, such as preprocessing the face image, extracting corner points in the preprocessed face image, filtering and merging the corner points to obtain the connected area of ​​the corner points; extracting the centroid of the connected area of ​​the corner points, and the specific extraction method can be: calculating the brightness difference between the current pixel and the surrounding pixels according to a predefined 3X3 template, and extracting the pixel points whose brightness difference is greater than or equal to a first threshold as the corner points; the 3X3 template is composed of pixels centered on the current pixel and to the left, right, top, bottom, upper left, upper right, lower left and lower right of the current pixel. Then the extracted centroid is matched with the face template, the matching probability between the centroid and the face template is calculated, and the area formed by the centroids whose matching probability is greater than or equal to a predetermined value is positioned as a candidate face area, wherein the face template may be a rectangular template, including at least three points, for example, each point is represented by (P, w, h), wherein P is the two-dimensional coordinate of the point, w is the maximum horizontal range allowed to appear left and right of the point, and h is the maximum vertical range allowed to appear up and down of the point.

[0024] Alternatively, the coordinates of each feature point on the face image can also be calibrated by AI face recognition, for example, 2.14 points, 5 points, and 6 points are marked. The five most critical points on the face are the left and right corners of the mouth, the center of the two eyes, and the nose. These five key points belong to the key points inside the face, and the posture of the face can be calculated based on them. Of course, there were also schemes for marking 4 points and 6 points in the early days. FRGC-V2 (Face Recognition Grand Challenge Version 2.0) released in 2005 marked 5 key points, including eyes, nose, mouth, and chin. The Caltech 10000 Web Faces dataset released in 2007 marked 4 key points, including eyes, nose, and mouth. The AFW dataset in 2013 marked 6 key points, including eyes, nose, and lips, of which the lips have 3 points. The MTFL / MAFL dataset released in 2014 marked 5 key points, including eyes, nose, and 2 corners of the mouth. Or 68-point annotation is the most common annotation scheme today. It was proposed in the Xm2vtsdb dataset in 1999. The 300W dataset and XM2VTS dataset also adopted the 68 key point scheme, which was adopted by the Dlib algorithm in OpenCV.

[0025] There are also some different versions of the annotation of 68 key points. Here we introduce the most common version in Dlib, which divides the key points of the face into internal key points and contour key points. The internal key points include eyebrows, eyes, nose, and mouth, totaling 51 key points, and the contour key points include 17 key points. The 68 key points of the face annotation used by Dlib can be seen in the figure above. There are 5 key points for a single eyebrow, which are evenly sampled from the left boundary to the right boundary, totaling 5×2=10. The eyes are divided into 6 key points, which are the left and right boundaries, and the upper and lower eyelids are evenly sampled, totaling 6×2=12. The lips are divided into 20 key points, except for the 2 at the corners of the mouth, which are divided into upper and lower lips. The outer boundaries of the upper and lower lips are evenly sampled with 5 points each, and the inner boundaries of the upper and lower lips are evenly sampled with 3 points each, totaling 20. The annotation of the nose adds 4 key points to the bridge of the nose, and 5 key points are evenly sampled at the tip of the nose, totaling 9 key points. The facial contour is evenly sampled with 17 key points. If the forehead is also added, more key points can be obtained, such as 81. Therefore, the technology of calibrating each feature point on the face image is an existing technology, and the details will not be repeated here.

[0026] Therefore, there are many ways to obtain the coordinates of each feature point on the face image, which are not specifically limited.

[0027] 102. Determine whether the user has a correct viewing posture according to the coordinates of each feature point on the face image;

[0028] 103. If not, determining the duration for which the user maintains the incorrect viewing posture;

[0029] After determining the coordinates of each feature point on the face image, determine whether the user meets the correct viewing posture according to the coordinates of each feature point, wherein the correct viewing posture may include the user's correct viewing distance, correct viewing angle, and correct viewing posture. Figure 2 , which is a flow chart of judging whether a user has a correct viewing posture provided by the present invention, including:

[0030] 1021. Determine the comprehensive imaging size of the characteristic part of the user according to the coordinates of each characteristic point

[0031] In practical applications, one or more groups of feature points in the face image may be pupils, or feature points equivalent to pupils, such as corner feature points, eye center feature points, etc. In this application, the pupil is used as an example of a feature part, and the comprehensive imaging size of the feature part is the pupil distance.

[0032] Specifically, the pupil distance L between the two eyes of the user can be calculated by the following formula:

[0033]

[0034] The coordinates of the left pupil are (x1, y1), and the coordinates of the right pupil are (x2, y2).

[0035] 1022. Calculate the viewing distance according to the comprehensive imaging size of the characteristic part and preset standard data;

[0036] After determining the comprehensive imaging size of the user's characteristic part, i.e., the pupil distance between the two eyes, the horizontal viewing angle and the vertical viewing angle are calculated according to the deviation of the left pupil and the right pupil from the origin of the picture. Specifically, the horizontal viewing angle and the vertical viewing angle are calculated by the following formula:

[0037] α=|(x2+x1)|;

[0038] β=|(y2+y1)|;

[0039] The α is used to indicate correlation with the horizontal viewing angle, and the β is used to indicate correlation with the vertical viewing angle.

[0040] Optionally, the horizontal and vertical viewing angles can be calculated as follows:

[0041] α=|(x2+x1)|;

[0042] β=|(y2+y1)|;

[0043] Wherein, the α is related to the horizontal viewing angle, the β is related to the vertical viewing angle, and the center point of the picture is a pre-calibrated origin of the coordinate system.

[0044] When the α is smaller than the second threshold and the β is smaller than the third threshold,

[0045] for:

[0046] The viewing distance is calculated according to the observation angle and the pupil distance between the two eyes of the user.

[0047] The viewing distance is calculated by the following formula: M=L* / cosδ; wherein, δ is used to represent the observation angle, and L is used to represent the pupil distance between the two eyes of the user.

[0048] 1023. When the viewing distance is greater than the eye safety distance, it is determined that the user does not conform to the correct viewing posture.

[0049] After the viewing distance is obtained, it is compared with the eye safety distance. When the viewing distance is greater than the eye safety distance, it is determined that the user does not conform to the correct viewing posture.

[0050] Optionally, in the embodiment of the present application, the method of determining whether the user conforms to the correct viewing posture may also include the following methods:

[0051] 1. When the α is greater than the second threshold or the β is greater than the third threshold, a first prompt message is sent to the user to prompt the user that the viewing angle is unhealthy.

[0052] 2. Draw a line from the center of the pupil to the philtrum in the facial image and extend it in both directions to segment the facial image; count the total skin color pixels in the segmented facial image to determine the left face area and the right face area respectively; calculate the strabismus degree according to the formula d=(LR) / (L+R), where d is used to represent the strabismus degree, L is used to represent the left face area, and R is used to represent the right face area; when the strabismus degree is greater than a fourth threshold, determine that the user does not meet the correct viewing posture.

[0053] In addition, in smartphones, the gyroscope is a sensor that is mainly used to detect the posture of the phone. It is indispensable for users to play somatosensory games, and it is also used for anti-shake when taking photos with some mobile phones; in addition, mobile phone navigation sometimes needs to use it to achieve better positioning. The earliest gyroscopes were mechanical, relatively large in size, and had high-speed rotating gyroscopes inside. Mechanical things have high requirements for processing accuracy and are afraid of vibration. Therefore, the accuracy of navigation systems based on mechanical gyroscopes has always been not very high. In today's smartphones, the gyroscope sensor has evolved into a small chip. It is a sensor and an upgraded version of the accelerometer. The accelerometer can monitor and sense linear motion in a certain axis, while the gyroscope can detect and sense linear motion and motion in 3D space. This makes it possible to identify direction, determine posture, and calculate angular velocity.

[0054] In view of this, the embodiments of the present application can also use a gyroscope to determine whether the user conforms to the correct viewing posture. Specifically, three, based on the facial image of the user, determine that the user is watching a mobile device; infer the facial spatial posture from the relative spatial position of the mobile phone and the face, and the horizontal posture data of the nine-axis gyroscope in the mobile device; if the horizontal posture data is within a preset range and the facial spatial posture is within the preset range, determine whether the user is in a side-lying or supine position, and determine that the user does not conform to the correct viewing posture.

[0055] In summary, the embodiments of the present application provide multiple methods for determining whether a user has a correct viewing posture. Multiple methods can be combined and integrated into the mobile device to more accurately determine whether the user's viewing posture is incorrect and prompt the user to use eye health to prevent myopia.

[0056] 104. When the duration is greater than a first threshold, a first prompt message is sent to the user, where the first prompt message is used to prompt the user to maintain a correct eye posture.

[0057] When the duration is greater than the first threshold, a first prompt message is sent to the user to prompt the user to maintain the correct eye posture. Specifically, the first prompt message may be to display the first prompt message above the screen of the mobile device, or to cover the screen of the mobile device to prompt the user that the distance from the screen is less than the safe eye distance; or, to turn off the screen and voice broadcast the first prompt message; or to send the first prompt message to a preset phone number for notification. For example, when the viewing distance is too close, the first prompt message is a warning signal for an unhealthy viewing distance; or when the viewing angle is greater than the second threshold, the first prompt message is a warning signal for an unhealthy viewing angle; or when the degree of strabismus is greater than the third threshold, it is determined that the user does not meet the correct viewing posture, and the first prompt message is a warning signal for an unhealthy viewing posture.

[0058] In addition, an embodiment of the present application also provides a mobile device for preventing myopia, including: a camera, which is used to track the user's facial image and calibrate the coordinates of each feature point on the facial image when the screen is lit; a processor, which is used to determine whether the user complies with the correct viewing posture based on the coordinates of each feature point on the facial image; if not, determine the duration for which the user maintains an incorrect viewing posture; and a display module, which is used to send a first prompt message to the user when the duration is greater than a first threshold, and the first prompt message is used to prompt the user to maintain a correct eye posture.

[0059] The above description is only one or several embodiments of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for preventing myopia, the method being applied to a mobile device, characterized in that: The steps include: When the screen is on, the user's face image is tracked through the front camera, and the coordinates of each feature point on the face image are calibrated; Determining whether the user has a correct viewing posture according to the coordinates of each feature point on the face image; If not, determining the duration for which the user maintains the incorrect viewing posture; When the duration is greater than a first threshold, a first prompt message is sent to the user, wherein the first prompt message is used to prompt the user to maintain a correct eye posture. The first prompt message is used to prompt the user that the viewing distance is unhealthy; Or, the first prompt message is used to prompt the user that the viewing angle is unhealthy; Or, the first prompt message is used to prompt the user that the viewing posture is unhealthy, When the first prompt message is used to prompt the user that the viewing distance is unhealthy, determining whether the user conforms to the correct viewing posture according to the coordinates of each feature point on the face image includes: Determining the comprehensive imaging size of the characteristic parts of the user according to the coordinates of each characteristic point; Calculating the viewing distance according to the comprehensive imaging size of the characteristic part and preset standard data; When the viewing distance is greater than the eye safety distance, it is determined that the user does not conform to the correct viewing posture. The characteristic part is a pupil, the integrated imaging size of the characteristic part is a pupil distance, and determining the integrated imaging size of the characteristic part of the user includes: The pupil distance L between the two eyes of the user is calculated by the following formula: Among them, the coordinates of the left pupil are (x1, y1), and the coordinates of the right pupil are (x2, y2). The calculating the viewing distance comprises: Based on the following formula, the horizontal viewing angle and the vertical viewing angle are calculated according to the deviation of the left pupil and the right pupil from the origin of the picture: α=|(x2+x1)|; β=|(y2+y1)|; Wherein, the α is related to the horizontal viewing angle, the β is related to the vertical viewing angle, and the center point of the picture is the origin of the pre-calibrated coordinate system; When the α is less than a second threshold value and the β is less than a third threshold value, calculating an observation angle according to the horizontal viewing angle and the vertical viewing angle; According to the observation angle and the pupil distance between the two eyes of the user, the viewing distance is calculated by the following formula: M = L / cosδ; Wherein, δ is used to represent the observation angle, and L is used to represent the pupil distance between the two eyes of the user. When the α is greater than the second threshold or the β is greater than the third threshold, a first prompt message is sent to the user to prompt the user that the viewing angle is unhealthy.

2. The method according to claim 1, characterized in that When the first prompt message is used to prompt the user that the viewing posture is unhealthy, determining whether the user conforms to the correct viewing posture includes: Connecting a line from the pupil center point to the Ren Zhong acupoint in the face image and extending the line in both directions to segment the face image; According to the segmented face image, the total amount of skin color pixels is counted to determine the left face area and the right face area respectively; Calculate the degree of strabismus according to the formula d=(LR) / (L+R), wherein d is used to represent the degree of strabismus, L is used to represent the area of ​​the left face, and R is used to represent the area of ​​the right face; When the squint degree is greater than a fourth threshold, it is determined that the user does not conform to a correct viewing posture.

3. The method according to claim 1, characterized in that When the first prompt message is used to prompt the user that the viewing posture is unhealthy, determining whether the user conforms to the correct viewing posture includes: Determining, based on the facial image of the user, that the user is viewing the mobile device; Inferring the spatial posture of the face from the relative spatial position of the mobile phone and the face and the horizontal posture data of the nine-axis gyroscope in the mobile device; If the horizontal posture data is within the preset range and the face space posture is within the preset range, it is determined whether the user is in a side-lying or supine viewing position, and it is determined that the user does not meet the correct viewing posture.

4. The method according to claim 1, characterized in that: The first prompt message includes: Displayed on the top of the screen of the mobile terminal; or, Covering the screen of a mobile terminal; or, Send a message to a preset phone number.

5. A mobile device for preventing myopia, characterized in that: include: The camera is used to track the user's facial image and calibrate the coordinates of each feature point on the facial image when the screen is lit; A processor, configured to determine whether the user conforms to a correct viewing posture according to the coordinates of each feature point on the face image; if not, determine a duration for which the user maintains an incorrect viewing posture; The display module is configured to send a first prompt message to the user when the duration is greater than a first threshold, wherein the first prompt message is configured to prompt the user to maintain a correct eye posture. Wherein, the first prompt message is used to prompt the user that the viewing distance is unhealthy; Or, the first prompt message is used to prompt the user that the viewing angle is unhealthy; Or, the first prompt message is used to prompt the user that the viewing posture is unhealthy, When the first prompt message is used to prompt the user that the viewing distance is unhealthy, determining whether the user conforms to the correct viewing posture according to the coordinates of each feature point on the face image includes: Determining the comprehensive imaging size of the characteristic parts of the user according to the coordinates of each characteristic point; Calculating the viewing distance according to the comprehensive imaging size of the characteristic part and preset standard data; When the viewing distance is greater than the eye safety distance, it is determined that the user does not conform to the correct viewing posture. The characteristic part is a pupil, the integrated imaging size of the characteristic part is a pupil distance, and determining the integrated imaging size of the characteristic part of the user includes: The pupil distance L between the two eyes of the user is calculated by the following formula: Among them, the coordinates of the left pupil are (x1, y1), and the coordinates of the right pupil are (x2, y2). The calculating the viewing distance comprises: Based on the following formula, the horizontal viewing angle and the vertical viewing angle are calculated according to the deviation of the left pupil and the right pupil from the origin of the picture: α=|(x2+x1)|; β=|(y2+y1)|; Wherein, the α is related to the horizontal viewing angle, the β is related to the vertical viewing angle, and the center point of the picture is the origin of the pre-calibrated coordinate system; When the α is less than a second threshold value and the β is less than a third threshold value, calculating an observation angle according to the horizontal viewing angle and the vertical viewing angle; According to the observation angle and the pupil distance between the two eyes of the user, the viewing distance is calculated by the following formula: M = L / cosδ; Wherein, δ is used to represent the observation angle, and L is used to represent the pupil distance between the two eyes of the user. When the α is greater than the second threshold or the β is greater than the third threshold, a first prompt message is sent to the user to prompt the user that the viewing angle is unhealthy.

Images