Tear secretion promoting method
Through video data acquisition and identification model, personalized vibration control of tear secretion instruments is realized, the accuracy problems caused by manual operation in the existing technology are solved, and the effectiveness of tear secretion instruments is improved.
Patent Information
- Application Number
- CN202410875048.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-11-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing tear protactic instruments require manual operation by users, resulting in different control accuracy and poor use effects, and personalized and precise control cannot be achieved.
By collecting user video data, determining the vibration duration based on the video recognition model, controlling the vibration of the tear prostate, and monitoring and adjustment in real time during the vibration to achieve personalized and precise control.
It realizes precise control of the tear secretion instrument based on the user's real-time image, improves the effect of the tear secretion instrument and ensures personalized treatment effect.
Smart Images

Figure CN118892415A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for promoting tear secretion. Background Art
[0002] With the development of society, the factors leading to dry eyes are increasing. Dry eyes are becoming an increasingly serious ophthalmic disease that seriously affects vision, quality of life, work efficiency and social interaction, causing great trouble to users. Dry eyes have become a more serious public health problem.
[0003] The tear secretion promoter can stimulate the body's own tears through the extranasal nerves, fundamentally treating dry eyes and achieving true "open source". At present, the tear secretion promoter requires manual operation by the user, which makes the control accuracy different for different users, resulting in poor use of the tear secretion promoter. Summary of the invention
[0004] 1. Technical issues to be resolved
[0005] In order to solve the above problems, the present invention provides a method for promoting tear secretion.
[0006] (II) Technical solution
[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:
[0008] A method for promoting tear secretion, comprising:
[0009] S101, collecting a first video of a user, where the first video includes eyes of the user;
[0010] S102, determining a vibration duration according to the first video;
[0011] S103, controlling the tear secretion promoting device to vibrate for a certain vibration duration, and collecting a second video of the user while the tear secretion promoting device is vibrating, wherein the second video includes the user's eyes;
[0012] S104, after the tear secretion promoting device vibrates for the vibration time, stopping the vibration according to the second video.
[0013] Optionally, S102 includes:
[0014] S102-1, obtaining the pre-stored standard number of blinks per minute of the user And the user's pre-stored standard vibration duration
[0015] S102-2, determining the current number of blinks of the user according to the first video
[0016] S102-3, according to Determine the adjustment ratio w b ;
[0017] S102-4, according to w b , and Determine the duration of the vibration.
[0018] Optionally, S102-2 includes:
[0019] S102-2-1, determining the eyelid height difference of each frame in the first video Form the first eyelid height difference set Where i1 is the frame identifier in the first video. For the i1th frame in the first video, if there is no eye in i1, the eyelid height difference is determined. Otherwise, determine the eyelid height difference is the ordinate of the center of the upper eyelid of the eye in i1 determined by the pre-trained eyelid recognition model, is the ordinate of the lower eyelid center of the eye in i1 determined by the pre-trained eyelid recognition model;
[0020] S102-2-2, confirmed The longest sequence of consecutive non-∞ elements in
[0021] S102-2-3, if The total number of elements in is less than 4320, then determine Otherwise, according to The values of each element in
[0022] Optionally, according to The values of each element in include:
[0023] Get the standard element value se0;
[0024] Sure The value of each element in Among them, j is The element identifier in for The element value of the jth element in ;
[0025] Sure The number of consecutive subsequences whose element values are positive
[0026] Sure in, for The total number of elements in .
[0027] Optionally, before S102, the method further includes:
[0028] S201, obtaining multiple third videos, where the third videos include eyes of a user;
[0029] S202, determining the eyelid height difference of each frame in each third video segment Form a second eyelid height difference set for each third video Where x is the third video ID, v x is the frame identifier in the xth third video. x Frame, if v x If there is no eye in the image, determine the eyelid height difference Otherwise, determine the eyelid height difference is v determined by the pre-trained eyelid recognition model x The ordinate of the center of the upper eyelid of the middle eye, is v determined by the pre-trained eyelid recognition model x the ordinate of the center of the lower eyelid of the middle eye;
[0030] S203, determine each The sequence of consecutive non-∞ second elements in ;
[0031] S204, selecting a second element sequence in which the number of elements is not less than 2 to form a third element sequence;
[0032] S205, determining the minimum value, maximum value and minimum value of the element values in each third element sequence;
[0033] S206, determining a comparison value according to the maximum value and the minimum value of each third element sequence;
[0034] S207, among all the minimum values, the minimum value that is not greater than the comparison value is formed into a final minimum value;
[0035] S208, OK Among them, a is the final minimum value mark, max{} is the maximum value function, min a is the ath final minimum;
[0036] S209, determine the value of each element in each third element sequence Where c is the third element sequence identifier, b is the element identifier in the cth third element sequence, is the element value of the cth element in the cth third element sequence;
[0037] S210, OK Among them, avg{} is the averaging function, is the number of consecutive subsequences with positive element values in the cth third element sequence, is the total number of elements in the cth third element sequence.
[0038] Optionally, determining the comparison value according to the maximum value and the minimum value of each third element sequence includes:
[0039] Determine the difference Δ between the maximum and minimum values of each third element sequence c =max c -min c , where max c is the maximum value in the cth third element sequence, min c is the minimum value in the cth third element sequence;
[0040] in, For all differences Δ c The standard deviation of .
[0041] Optionally, according to Determine the adjustment ratio w b ,include:
[0042] like Then b =1;
[0043] like Then b =2;
[0044] like but Among them, min{} is the minimum value function;
[0045] otherwise,
[0046] Optionally, according to w b , and Determine the duration of the vibration, including:
[0047] like but
[0048] otherwise, Among them, min{} is the minimum value function.
[0049] Optionally, S104 includes:
[0050] S104-1, identifying the number of consecutive frames in which tears appear in the second video using a pre-trained tear recognition model;
[0051] S104-2, if the number of consecutive frames is greater than the preset threshold, the vibration is stopped; otherwise, the vibration continues for the standard vibration duration of the user stored in advance half.
[0052] Optionally, for 30 seconds.
[0053] (III) Beneficial effects
[0054] The beneficial effects of the present invention are as follows: the present invention relates to a method for promoting tear secretion, the method comprising: collecting a first video of a user, the first video including the user's eyes; determining a vibration duration according to the first video; controlling a tear secretion promoting instrument to vibrate for the vibration duration, collecting a second video of the user while the tear secretion promoting instrument is vibrating, the second video including the user's eyes; after the tear secretion promoting instrument vibrates for the vibration duration, stopping the vibration according to the second video. The method of the present invention controls the vibration of the tear secretion promoting instrument based on a video including the user's eyes, and can achieve precise control of the tear secretion promoting instrument according to the user's real-time image, thereby improving the effect of the tear secretion promoting instrument. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 A schematic diagram of a process for promoting tear secretion provided in one embodiment of the present application;
[0056] Figure 2 A schematic diagram of a curve graph formed by a third element sequence provided in an embodiment of the present application. DETAILED DESCRIPTION
[0057] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.
[0058] The tear secretion promoter can stimulate the body's own tears through the extranasal nerves, fundamentally treating dry eyes and achieving true "open source". At present, the tear secretion promoter requires manual operation by the user, which makes the control accuracy different for different users, resulting in poor use of the tear secretion promoter.
[0059] Based on this, the present invention proposes a tear secretion promoting method, which includes: collecting a first video of the user, the first video includes the user's eyes; determining the vibration duration according to the first video; controlling the tear secretion promoting instrument to vibrate for the vibration duration, collecting a second video of the user while the tear secretion promoting instrument is vibrating, the second video includes the user's eyes; after the tear secretion promoting instrument vibrates for the vibration duration, stopping the vibration according to the second video. The method involved in the present invention controls the vibration of the tear secretion promoting instrument based on the video including the user's eyes, and can realize accurate control of the tear secretion promoting instrument according to the real-time image of the user, thereby improving the effect of the tear secretion promoting instrument.
[0060] The present embodiment provides a tear secretion promoting method, which is executed when a user needs to use a tear secretion promoting device to control the vibration of the tear secretion promoting device. Before using the method of the present embodiment, the tear secretion promoting device needs to be connected to a device (such as a server) that executes the method of the present embodiment, and the device (such as a server) that executes the method of the present embodiment needs to be connected to an image acquisition device (such as a mobile phone), so that the device (such as a server) that executes the method of the present embodiment can obtain the latest user data in real time through the image acquisition device (such as a mobile phone), and control the vibration of the tear secretion promoting device based on the data.
[0061] In addition, when the method provided by this embodiment is used for the first time, the basic information of the user will be extracted first (that is, the standard number of blinks per minute of the user will be extracted). Standard element value se0), and store se0, and each time the method provided in this embodiment is performed to promote tear secretion, it will be based on se0 provides more precise control over the tear secretion promoting instrument.
[0062] If you change users later, want to store the latest data, or have other reasons, you can repeat the extraction. se0 process.
[0063] Among them, extraction The process of se0 is as follows:
[0064] S201, obtaining multiple third videos.
[0065] Among them, the third video includes the user's eyes.
[0066] Before the user uses the tear secretion promoting device, a communication connection is established between the image acquisition device (such as a mobile phone), the device for executing the method of this embodiment (such as a server), and the tear secretion promoting device. During the process of se0, the user's current video containing eyes (ie, the third video) will be captured through an image capture device (such as a mobile phone) and transmitted back to a device (such as a server) executing the method of this embodiment.
[0067] The third video may be an eye video or a face video including eyes.
[0068] In addition, the third video needs to have a certain length (eg, more than 3 minutes, or more than 5 minutes, etc.).
[0069] In addition, since this process ultimately obtains the user's basic data, in order to ensure accuracy, multiple third videos can be obtained in different time periods to improve the diversity of the third videos and the number of third videos.
[0070] S202, determining the eyelid height difference of each frame in each third video segment Form a second eyelid height difference set for each third video
[0071] Where x is the third video ID, v x is the frame identifier in the xth third video. x Frame, if v x If there is no eye in the image, determine the eyelid height difference Otherwise, determine the eyelid height difference is v determined by the pre-trained eyelid recognition model x The ordinate of the center of the upper eyelid of the middle eye, is v determined by the pre-trained eyelid recognition model x The ordinate of the center of the lower eyelid of the middle eye.
[0072] This step uses the existing face recognition method to identify the eye position in each frame of each third video. When no eyes are recognized, it means that the frame image does not contain eyes, and there is no eyelid height difference at this time, so it is determined to be -∞ (it can also be other values in specific implementation, such as null value, or NULL, etc., as long as the value representing no eyes and the distance difference between the upper and lower eyelids of normal eyes is pre-set). If the eyes are recognized, the center position of the upper and lower eyelids can be identified by the pre-trained eyelid recognition model, and the ordinates of the upper and lower eyelid centers are determined, and the ordinate difference of the upper and lower eyelid centers is determined as the eyelid height difference of the frame.
[0073] It should be noted that the present embodiment does not limit the establishment of the coordinate system of the coordinates, as long as the unit length of the vertical coordinate is the same, because it is only necessary to obtain the vertical coordinate difference between the centers of the upper and lower eyelids, that is, a relative difference. In other words, the establishment of different coordinate systems may lead to different coordinates of the centers of the upper and lower eyelids, but the difference in the vertical coordinates only represents the number of unit lengths of the vertical coordinates in the current coordinate system. Therefore, as long as the unit length of the vertical coordinate is the same, the difference is comparable no matter how the coordinate system is established.
[0074] After step S202, each segment of the third video corresponds to a second eyelid height difference set, and the number of elements in the second eyelid height difference set is the same as the number of frames in the corresponding third video.
[0075] If the value of an element in any second eyelid height difference set is -∞, it means that there is no eye in the frame image. If the value of an element in any second eyelid height difference set is 0, it means that there is an eye in the frame image, but the eye is closed. If the value of an element in any second eyelid height difference set is a positive number, it means that there is an eye in the frame image, but the eye is open, and different values indicate different degrees of eye opening. The larger the value, the more open the eye is.
[0076] However, there are recognition errors, so after obtaining all the second eyelid height difference sets, they will be preprocessed. That is, if there is an element value less than 0 and greater than -∞, it is considered that the corresponding frame includes the eye, but the upper and lower eyelids may be misidentified after the eyes are closed, resulting in a vertical coordinate recognition error. At this time, the eye will be considered closed, and its value will be uniformly reassigned to 0. If there is a value that is significantly greater than the maximum position difference between the upper and lower eyelids, it is considered that the recognition is wrong, and its value is uniformly reassigned to -∞.
[0077] After preprocessing, the second eyelid height difference set only includes -∞ and normal value elements. Elements with a value of -∞ either have no eyes in the corresponding frame or are incorrectly identified, and the frames corresponding to this value are all invalid frames.
[0078] S203, determine each The sequence of consecutive non-∞ second elements in .
[0079] Since the second eyelid height difference set obtained in step S202 only includes elements with -∞ and normal values. Elements with a value of -∞ either have no eyes in the corresponding frame or are incorrectly identified, and the frames corresponding to this value are all invalid frames. Then the -∞ value elements in the second eyelid height difference set segment the second eyelid height difference set, and a series of continuous non-∞ elements between the two -∞ value elements form a segment, which indicates that it is a valid eye data. Each segment is a second element sequence, that is, a second element sequence is a valid data.
[0080] From the above analysis, it can be seen that a third video corresponds to a second eyelid height difference set, corresponding to at least one second element sequence.
[0081] For example, the second eyelid height difference set is {1,-∞,-
[0082] ∞,0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0,-∞,-∞,1,2,3,2,1,-∞,2}, then the second element sequence has four, namely {1}, {0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0},
[0083] 1,2,3,2,1},{2}.
[0084] S204, selecting a second element sequence whose number of elements is not less than 2 to form a third element sequence.
[0085] Since the duration of a human blink is no less than 1 second, and there are 24 frames in one second of video, and each frame corresponds to an element in the second element sequence, if there are less than 2 elements in a second element sequence, it means that it is not a complete blink data and is considered invalid data. The valid data forms the third element sequence.
[0086] For example, the second element sequence has four elements, namely {1},
[0087] 0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0}, {1,2,3,2,1}, {2}. The number of elements in the first and fourth second element sequences are both less than 2, so they are not used as the third element sequences. Finally, there are only 2 third element sequences, namely {0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0}, {1,2,3,2,1}.
[0088] Finally, the third element sequence is valid data, and the final se0.
[0089] S205, determining the minimum value, maximum value and minimum value of the element values in each third element sequence.
[0090] The maximum value and the minimum value are overall concepts, that is, there is no value larger or smaller than it in the third element sequence, while the minimum value is a regional concept, and there is no value smaller than it in the region. In other words, the maximum value and the minimum value are both one value, and the minimum value is multiple values.
[0091] For example, if the third element sequence is {0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0}, the maximum value is 6, the minimum value is 0, and the minimum value is 0 (the four positions where 0 appears are the minimum values of their corresponding regions). If the third element sequence is {1,2,3,2,1}, the maximum value is 3, the minimum value is 1, and the minimum value is 1 (the two positions where 1 appears are the minimum values of their corresponding regions).
[0092] In specific applications, the third element sequence can be formed into a curve graph, and each valley point is a minimum value. If the third element sequence is {0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0}, then its curve graph is as follows Figure 2 shown.
[0093] S206: Determine a comparison value according to the maximum value and the minimum value of each third element sequence.
[0094] The implementation process of this step is:
[0095] S206-1, determining the difference Δ between the maximum value and the minimum value of each third element sequence c =max c -min c .
[0096] Among them, max c is the maximum value in the cth third element sequence, min c is the minimum value in the c-th third element sequence.
[0097] S206-2,
[0098] in, For all differences Δ c The standard deviation of , where avg{} is the averaging function.
[0099] In step S201, multiple third videos are obtained, each of which corresponds to a second eyelid height difference set, each of which corresponds to at least one third element sequence, and each of which corresponds to a difference value Δ c All differences Δ c The standard deviation of All differences Δ c The mean value of
[0100] The comparison value is used as a subsequent comparison standard. If it is larger than the comparison value, it is considered that the eyes are open, and if it is smaller than the comparison value, it may be closed. c The maximum distance between the upper and lower eyelids of each valid data. is the maximum distance between the upper and lower eyelids of the user when the eyes are open on average, which is used as a reference value. If the distance difference between the upper and lower eyelids is greater than the reference value (i.e. ) is one-fourth smaller, then the eyes may be closed, but the data may have errors during collection. It represents the degree of dispersion between the maximum distances between the upper and lower eyelids in the valid data collected each time. The smaller the value, the smaller the maximum distances between the upper and lower eyelids in the valid data collected each time. The larger the value, the larger the maximum distances between the upper and lower eyelids in the valid data collected each time. As an adjustment value for the difference between each acquisition, an adjustment is made to the quarter point, and the final comparison standard, namely the comparison value, which can be used as a comparison standard for identifying whether the eyes are closed is obtained.
[0101] S207, among all the minimum values, the minimum value that is not greater than the comparison value is formed into the final minimum value.
[0102] If the minimum value is greater than the comparison standard (ie, the comparison value), it is considered that the eye movement does not involve closing the eyes, and it is not used as the final minimum value. Therefore, the final minimum value represents the point where the eyes may be closed.
[0103] S208, determine the standard element value
[0104] Among them, a is the final minimum value mark, max{} is the maximum value function, min a is the ath final minimum.
[0105] The standard element value se0 is the maximum value among all the final minimum values, that is, the maximum value of the distance between the upper and lower eyelids when the eyes are closed. This value is used as the standard element value for subsequent comparisons, representing the maximum possible distance between the upper and lower eyelids when the user closes his eyes.
[0106] Because some people cannot close their eyes completely, that is, there will be a gap between the upper and lower eyelids when they close their eyes (such as loose closure caused by eyelid muscles, or loose closure of children, etc.), the standard element value se0 is a standard value of the upper and lower eyelid distance when closing the eyes taking this situation into consideration.
[0107] S209, determine the value of each element in each third element sequence
[0108] Where c is the third element sequence identifier, b is the element identifier in the cth third element sequence, is the element value of the cth element in the cth third element sequence in .
[0109] se0 is the distance between the upper and lower eyelids when the user closes his eyes, The actual eye opening and closing conditions relative to the user's own characteristics (i.e., the distance between the upper and lower eyelids when the user closes his eyes) can be obtained.
[0110] For example, the third element sequence is {0,1,2,3,6,5,3,2,1,0,0,1,2,3,4,3,2,1,0}. If se0=1, the third element sequence after step S209 is {-1,0,1,2,5,4,2,1,0,-1,-1,0,1,2,3,2,1,0,-1}.
[0111] It can be seen that the consecutive positive numbers between two 0 elements in the third element sequence after step S209 are a blink process. For example, the third element sequence after the above processing has two blink processes, namely {1,2,5,4,2,1,} and {1,2,3,2,1}.
[0112] S210, determining the user's standard blink frequency per minute
[0113] Among them, avg{} is the averaging function, is the number of consecutive subsequences with positive element values in the cth third element sequence, is the total number of elements in the cth third element sequence.
[0114] In step S201, multiple third videos are obtained, each of which corresponds to a second eyelid height difference set, each of which corresponds to at least one third element sequence, and the number of continuous subsequences with positive element values in each third element sequence obtained in step S209 is That is, the number of blinks included in the third element sequence, is the number of frames in the valid data, and one second is 24 frames, one minute is 24*60=1440 frames, so through You can get the duration of each valid data, that is, is the duration of the third element sequence (in minutes), is the number of blinks per minute, That is, the average number of blinks per minute obtained by all third element sequences. This value is the user's baseline value. The user's standard number of blinks per minute
[0115] In getting the above After se0, it will store se0, so that tear secretion can be promoted subsequently by the method provided in this embodiment.
[0116] In addition, the user's standard vibration duration will be set and stored As the basic duration of each vibration, that is, if there is no correction, the tear secretion instrument will be controlled to vibrate each time Then stop.
[0117] The value of is determined based on the tear secretion meter. For example, by using the same tear secretion meter by different users, the average time from the start of vibration to tearing is monitored, such as 30 seconds. That is 30 seconds.
[0118] See also Figure 1The tear secretion promoting method provided in this embodiment is implemented as follows:
[0119] S101, collecting a user's first video.
[0120] The first video includes the user's eyes.
[0121] Before the user uses the tear secretion promoter, a communication connection will be established among the image acquisition device (such as a mobile phone), the device that executes the method of this embodiment (such as a server), and the tear secretion promoter. Therefore, in step 101, the user's current video containing the eyes (i.e., the first video) will be captured by the image acquisition device (such as a mobile phone) and transmitted back to the device that executes the method of this embodiment (such as a server).
[0122] The first video may be an eye video or a face video including eyes.
[0123] In addition, the first video needs to have a certain length (such as more than 3 minutes, or more than 5 minutes, etc.), so as to ensure the effect of the method provided in this embodiment.
[0124] S102: Determine the vibration duration according to the first video.
[0125] This step will determine the vibration duration of the tear secretion promoting device based on the first video. Since the first video is the user's current video, the video reflects the user's current eye condition. Based on the first video, the vibration duration that is most suitable for the user can be accurately customized for the user, ensuring the accuracy of the control of the tear secretion promoting device.
[0126] The implementation process of step S102 is as follows:
[0127] S102-1, obtaining the pre-stored standard number of blinks per minute of the user And the user's pre-stored standard vibration duration
[0128] The data obtained here are obtained from step S201 to step S210. and pre-set and stored
[0129] S102-2, determining the current number of blinks of the user according to the first video
[0130] The implementation process of step S102-2 is as follows:
[0131] S102-2-1, determining the eyelid height difference of each frame in the first video Form the first eyelid height difference set
[0132] Where i1 is the frame identifier in the first video. For the i1th frame in the first video, if there is no eye in i1, the eyelid height difference is determined. Otherwise, determine the eyelid height difference is the ordinate of the center of the upper eyelid of the eye in i1 determined by the pre-trained eyelid recognition model, is the ordinate of the center of the lower eyelid of the eye in i1 determined by the pre-trained eyelid recognition model.
[0133] The implementation process of step S102-2-1 is similar to that of step S202, except that the processed videos are different. It will not be described here in detail, and the implementation process of step S202 can be referred to.
[0134] S102-2-2, confirmed The longest sequence of consecutive non-∞ elements in
[0135] The implementation process of step S102-2-2 is similar to that of step S203, except that the set of eyelid height differences to be processed is different, which will not be described in detail here, and the implementation process of step S203 may be referred to.
[0136] S102-2-3, if The total number of elements in is less than 4320, then determine Otherwise, according to The values of each element in
[0137] The first video has 1440 frames per minute, 3 minutes means 3*1440=4320 frames. If the total number of elements in is less than 4320, it means that the valid data duration is less than 3 minutes. This will result in less valid data, which will affect the accuracy of judging the current number of blinks of the user. Therefore, the default data is used, that is, like If the total number of elements in is less than 4320, the actual blinking of the user is determined based on the valid data.
[0138] Specifically, according to The values of each element in The process is:
[0139] S301, obtaining the standard element value se0.
[0140] What is obtained here is se0 obtained from step S201 to step S208.
[0141] S302, confirm The value of each element in
[0142] Among them, j is The element identifier in for The element value of the jth element in .
[0143] The implementation process of step S302 is similar to that of step S209, except that the processing sequence is different, which will not be described here in detail, and the implementation process of step S209 may be referred to.
[0144] S303, confirm The number of consecutive subsequences whose element values are positive
[0145] It can be seen that after step S302 The continuous positive numbers between two 0 elements are a blink process. The number of consecutive subsequences whose element values are positive That is, the number of blinks corresponding to the longest valid data in the first video reflecting the current real situation of the user obtained in step S101.
[0146] S304, confirm
[0147] in, for The total number of elements in .
[0148] is the number of frames in the current valid data, and there are 24 frames in one second and 24*60=1440 frames in one minute, so You can get the duration of each valid data, that is, is the length of the longest valid data in the first video reflecting the current real situation of the user obtained in step S101 (in minutes), The current number of blinks per minute of the user.
[0149] S102-3, according to Determine the adjustment ratio w b .
[0150] The number of blinks is related to the dryness of the eyes. Normally, the eyes blink 8 to 16 times per minute. If you blink 7 to 8 times per minute when looking at a computer or other electronic device or staring at a certain place, it will cause dry eyes. If the number of blinks per minute is less than 7 times or more than 16 times, it means that the eyes are abnormal. Step S102-3 is based on the user's standard blinking number (i.e. ) to determine the user's normal eye condition, namely:
[0151] like This means that the user's eyes are normal under normal circumstances, so wb =1, that is, no adjustment is made.
[0152] like This indicates that the user may have dry eyes. b =2, i.e. doubled.
[0153] like This means that the user's eyes are abnormal. That is, it is adjusted between 2 and 3 times according to the abnormal situation. The more abnormal it is, the greater the adjustment multiple.
[0154] Among them, min{} is the minimum value function.
[0155] Otherwise (i.e. ), it means the user's eyes are abnormal. That is, the adjustment will be between 2 and 3 times according to the abnormal situation, with the maximum adjustment being 3 times. The more abnormal the situation, the greater the adjustment multiple.
[0156] S102-4, according to w b , and Determine the duration of the vibration.
[0157] like This means that the user's eyes are normal at present, so the vibration is performed according to the default duration, that is, Should Pre-set and stored
[0158] Otherwise, it means that the user's eyes are currently dry, which will increase the vibration duration.
[0159] Among them, min{} is the minimum value function.
[0160] w b Represents the user's general adjustment multiple. It represents the adjustment factor caused by the abnormality between the current blinking situation and the user's normal blinking situation. It is the adjustment multiple under the current situation of the user, which not only takes into account the differences among users themselves, but also takes into account the current actual situation of the users, thus ensuring the accurate control of the tear secretion promoting instrument.
[0161] S103, controlling the tear secretion promoting device to vibrate for a certain vibration duration, and collecting a second video of the user while the tear secretion promoting device is vibrating.
[0162] The second video includes the user's eyes.
[0163] After executing step S102, it is monitored whether the user has placed the tear secretion promoter in the correct position (such as the vibration head of the tear secretion promoter is in contact with the skin near the user's nasal bone and lateral nasal cartilage), and the tear secretion promoter is controlled to vibrate, and the vibration duration is the vibration duration obtained in S102-4. Vibration energy can be transmitted to the human body through the vibration, thereby stimulating the user's external nasal nerves and promoting the body's tear secretion through the nasolacrimal reflex, which helps to improve dry eye symptoms and treat dry eye.
[0164] Among them, there are many methods to monitor whether the user has placed the tear secretion promoter in the correct position. For example, after the user places the tear secretion promoter in the correct position, a confirmation signal is sent to the device (such as a server) that executes the method of this embodiment through a mobile phone or a special trigger device on the tear secretion promoter. If the device (such as a server) that executes the method of this embodiment detects the signal, it is considered that the user has placed the tear secretion promoter in the correct position.
[0165] In addition, during the vibration of the tear secretion promoting device, the image acquisition device (such as a mobile phone) will still capture the user's eye video (ie, the second video) in real time, and transmit the second video to the device (such as a server) that executes the method of this embodiment.
[0166] S104, after the tear secretion promoting device vibrates for the vibration time, stopping the vibration according to the second video.
[0167] This step will determine whether to stop the vibration after the vibration duration is reached based on the second video.
[0168] S104-1, using a pre-trained tear recognition model, identifying the number of consecutive frames in which tears appear in the second video.
[0169] In this step, a model can be trained through a large amount of sample data. The model video can detect whether there are tears in the eyes. Through this model, it can be obtained whether the user sheds tears during the vibration of the tear secretion promoter. If tears are shed, and the duration of tears is greater than the preset threshold, the purpose of this vibration has been achieved. If no tears are shed, or the flow duration is not greater than the preset threshold, the purpose of this vibration has not been achieved.
[0170] S104-2, if the number of consecutive frames is greater than the preset threshold, the vibration is stopped. Otherwise, the vibration continues for the pre-stored standard vibration duration of the user. half.
[0171] The number of frames with tears obtained in step S104-1 can represent the duration of the user's tears. If the number of consecutive frames is not greater than the preset threshold, there is no tearing or it is considered that the tearing is not closely related to the vibration of the tear secretion promoting device. At this time, if the vibration duration has been reached, the vibration will be extended. Otherwise, it is considered that the purpose of the tear secretion promoting device has been achieved, and the vibration will stop when the vibration duration is reached.
[0172] In this way, in addition to personalizing the duration of each vibration, the vibration duration can also be flexibly controlled according to the current vibration effect of the tear secretion promoter, further ensuring the control effect.
[0173] The present embodiment provides a tear secretion promoting method, which includes collecting a first video of a user, the first video including the user's eyes; determining a vibration duration based on the first video; controlling a tear secretion promoting device to vibrate for the vibration duration, collecting a second video of the user while the tear secretion promoting device is vibrating, the second video including the user's eyes; after the tear secretion promoting device vibrates for the vibration duration, stopping the vibration based on the second video. The method provided in the present embodiment controls the vibration of the tear secretion promoting device based on a video including the user's eyes, and can achieve precise control of the tear secretion promoting device based on real-time images of the user, thereby improving the effect of the tear secretion promoting device.
[0174] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.
[0175] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.
[0176] Finally, it should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for promoting tear secretion, It is characterized in that The method comprises: S101, collecting a first video of a user, where the first video includes eyes of the user; S102, determining a vibration duration according to the first video; S103, controlling the tear secretion promoting instrument to vibrate for the vibration duration, and collecting a second video of the user while the tear secretion promoting instrument is vibrating, wherein the second video includes the user's eyes; S104, after the tear secretion promoting device vibrates for the vibration time, stopping the vibration according to the second video.
2. The method according to claim 1, It is characterized in that The S102 includes: S102-1, obtaining the pre-stored standard number of blinks per minute of the user and the pre-stored standard vibration duration of the user S102-2, determining the current number of blinks of the user according to the first video S102-3, according to Determine the adjustment ratio w b ; S102-4, according to w b , and Determine the duration of the vibration.
3. The method according to claim 2, It is characterized in that The S102-2 includes: S102-2-1, determining the eyelid height difference of each frame in the first video Form the first eyelid height difference set Among them, i 1 is a frame identifier in the first video. 1 Frame, if i 1 If there is no eye in the image, determine the eyelid height difference Otherwise, determine the eyelid height difference is i determined by the pre-trained eyelid recognition model 1 The ordinate of the center of the upper eyelid of the middle eye, is i determined by the pre-trained eyelid recognition model 1 the ordinate of the center of the lower eyelid of the middle eye; S102-2-2, confirmed The longest sequence of consecutive non-∞ elements in S102-2-3, if The total number of elements in is less than 4320, then determine Otherwise, according to The values of each element in 4. The method according to claim 3, It is characterized in that The basis The values of each element in include: Get the standard element value se 0 ; Sure The value of each element in Among them, j is The element identifier in for The element value of the jth element in ; Sure The number of consecutive subsequences whose element values are positive Sure in, for The total number of elements in .
5. The method according to claim 4, It is characterized in that Before S102, the method further includes: S201, obtaining a plurality of third videos, wherein the third videos include the eyes of the user; S202, determining the eyelid height difference of each frame in each third video segment Form a second eyelid height difference set for each third video Where x is the third video identifier, v x is the frame identifier in the xth third video. x Frame, if v x If there is no eye in the image, determine the eyelid height difference Otherwise, determine the eyelid height difference is v determined by the pre-trained eyelid recognition model x The ordinate of the center of the upper eyelid of the middle eye, is v determined by the pre-trained eyelid recognition model x the ordinate of the center of the lower eyelid of the middle eye; S203, determine each The sequence of consecutive non-∞ second elements in ; S204, selecting a second element sequence in which the number of elements is not less than 2 to form a third element sequence; S205, determining the minimum value, maximum value and minimum value of the element values in each third element sequence; S206, determining a comparison value according to the maximum value and the minimum value of each third element sequence; S207, among all the minimum values, the minimum value that is not greater than the comparison value is formed into a final minimum value; S208, OK Among them, a is the final minimum value mark, max{} is the maximum value function, min a is the ath final minimum; S209, determine the value of each element in each third element sequence Where c is the third element sequence identifier, b is the element identifier in the cth third element sequence, is the element value of the cth element in the cth third element sequence; S210, OK Among them, avg{} is the averaging function, is the number of consecutive subsequences with positive element values in the cth third element sequence, is the total number of elements in the cth third element sequence.
6. The method according to claim 5, It is characterized in that The step of determining the comparison value according to the maximum value and the minimum value of each third element sequence comprises: Determine the difference Δ between the maximum and minimum values of each third element sequence c =max c -min c , where max c is the maximum value in the cth third element sequence, min c is the minimum value in the cth third element sequence; Sure in, For all differences Δ c The standard deviation of .
7. The method according to claim 2, It is characterized in that The basis Determine the adjustment ratio w b ,include: like Then b =1; like Then b =2; like but Among them, min{} is the minimum value function; otherwise, 8. The method according to claim 2, It is characterized in that According to w b , and Determine the duration of the vibration, including: like but otherwise, Among them, min{} is the minimum value function.
9. The method according to claim 1, It is characterized in that The S104 includes: S104-1, identifying the number of consecutive frames in which tears appear in the second video using a pre-trained tear recognition model; S104-2, if the number of consecutive frames is greater than the preset threshold, the vibration is stopped; otherwise, the vibration continues for the pre-stored standard vibration duration of the user. half.
10. The method according to claim 2, It is characterized in that Said for 30 seconds.
Citation Information
Cited By
Dynamic lacrimal river parameter measurement method for dry eye assessment
CN119417886A
