Eye-controlled staring interaction method and system for asymptomatic patient

Abstract

The invention relates to the technical field of visual interaction, and particularly discloses an eye-controlled staring interaction method and system for an asymptomatic patient, and the method comprises the steps: selecting a photographing point position, and synchronously determining photographing parameters; acquiring the position of a patient in real time, and constructing an interaction option library containing an interaction sequence according to the position; displaying an interaction option library containing an interaction sequence based on an interaction device, obtaining eye data of a patient in real time, determining interaction options according to the eye data, and generating an interaction instruction; and acquiring a patient video in real time based on the camera shooting point location, identifying the patient video, and updating the determination process of the interaction options. According to the invention, the eye data is acquired, the eye data is used as input, an effect similar to a mouse is simulated, the system is used for controlling various intelligent devices, the operation process is very convenient, and the system is matched with a large number of intelligent devices, so that the nursing cost can be greatly reduced, and meanwhile, a patient can obtain great satisfaction.

Description

Technical Field

[0001] The present invention relates to the field of visual interaction technology, and in particular to an eye-controlled gaze interaction method and system for patients with amyotrophic lateral sclerosis (ALS). Background Art

[0002] For some people with limited mobility, such as patients with ALS, their daily lives require the care of caregivers. Caregivers can be family members or staff members. Regardless of the type of caregiver, a lot of costs are required, including time costs and remuneration. However, these people with limited mobility are not completely unable to move. With the increasing popularity of smart devices today, these people with limited mobility can live on their own. Caregivers only need to perform simple emergency monitoring. In fact, allowing these people with limited mobility to live on their own is also beneficial to their physical and mental health. Therefore, how to provide a convenient interactive control solution for smart devices to improve the convenience of life for people with limited mobility is the technical problem that the technical solution of the present invention wants to solve. Summary of the Invention

[0003] The purpose of the present invention is to provide an eye-controlled gaze interaction method and system for ALS patients to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions: An eye-controlled gaze interaction method for patients with ALS, the method comprising: Receive an interaction request containing authorization information from a patient, obtain spatial planning information of the interaction area, select imaging points based on the spatial planning information, and simultaneously determine imaging parameters; wherein the selected imaging points and determined imaging parameters are used to ensure that the total imaging area is not less than the interaction area; Acquire the patient's location in real time, and construct an interaction option library containing an interaction sequence based on the location; Based on the interactive device displaying an interactive option library containing an interactive sequence, the patient's eye data is acquired in real time, the interactive options are determined based on the eye data, and an interactive instruction is generated; the eye data includes gaze direction and gaze duration; The patient video is acquired in real time based on the camera position, the patient video is identified, and the determination process of the interactive options is updated; wherein the update target is the threshold condition of the eye data.

[0005] As a further solution of the present invention, the step of obtaining the patient's position in real time and constructing an interaction option library containing an interaction sequence according to the position includes: Obtain the patient's location in real time, generate time items and location items, and insert them into the database; Calculating the distance between the patient's position and the interaction position of the interaction option, and determining the interaction order according to the distance; the interaction order is in ascending order of distance; Updates the interaction options library based on the interaction sequence.

[0006] As a further solution of the present invention, the steps of displaying an interaction option library containing an interaction sequence based on an interactive device, acquiring the patient's eye data in real time, determining the interaction options based on the eye data, and generating the interaction instructions include: Displaying an interaction option library containing an interaction sequence based on a display installed on the interaction device; Obtain the patient's eye data based on the camera installed on the interactive device to obtain the patient's gaze direction at each moment; determining an interaction location on the display based on the gaze direction, and determining a gaze duration at the interaction location; When the gaze duration reaches a preset duration threshold, the interaction options at the interaction position are read and an interaction instruction is generated.

[0007] As a further solution of the present invention, the steps of acquiring patient video in real time based on camera positions, identifying the patient video, and updating the determination process of interactive options include: Acquire patient videos in real time based on camera locations, identify the patient videos, and extract the patient's temporal behavioral features; Each time an interaction instruction is generated, the behavior features within a preset time range before the interaction instruction generation moment are queried to obtain a feature group; Count all feature groups of each interaction instruction and determine the average feature group of the interaction instruction; Based on the current moment, the behavioral characteristics within the preset time range are read in real time, the behavioral characteristics are compared with the average feature group of each interaction instruction, and the matching degree is calculated; The duration threshold of the interaction option corresponding to each interaction instruction is adjusted according to the matching degree.

[0008] As a further solution of the present invention, the step of counting all feature groups of each interactive instruction and determining the average feature group of the interactive instruction includes: For each interaction instruction, count the feature groups that generate the interaction instruction each time; Get the union of all feature groups to get the total behavioral feature group; For each behavioral feature in the total behavioral feature group, query the number of times it appears in the feature group; When the number of times reaches a preset number threshold, it is used as an average feature; All average features are counted to determine the average feature group of the interaction instruction.

[0009] As a further solution of the present invention, the step of reading the behavior characteristics within a preset time range in real time based on the current moment, comparing the behavior characteristics with the average feature group of each interaction instruction, and calculating the matching degree includes: In real time, the current moment is used as the end moment, and behavioral features within the preset time range are obtained forward to construct a real-time behavior group; Read the average feature group of each interaction instruction in turn; Select behavioral features in the real-time behavior group in turn, compare them with all behavioral features in the average feature group, and determine the maximum similarity; Calculate the mean of the maximum similarity of each behavior feature of the real-time behavior group as the matching degree; The process of adjusting the duration threshold of the interaction options corresponding to each interaction instruction according to the matching degree includes: Where, is the adjusted duration threshold, is the preset standard duration threshold, For matching, is the preset correction factor.

[0010] The technical solution of the present invention also provides an eye-controlled gaze interaction system for patients with ALS, the system comprising: A point calibration module is configured to receive an interaction request containing authorization information from a patient, obtain spatial planning information of the interaction area, select imaging points based on the spatial planning information, and simultaneously determine imaging parameters; wherein the selected imaging points and determined imaging parameters are used to ensure that the total imaging area is not smaller than the interaction area; An interaction preprocessing module, configured to obtain the patient's position in real time and construct an interaction option library containing an interaction sequence according to the position; An interaction instruction generation module is configured to obtain the patient's eye data in real time based on an interaction option library containing an interaction sequence displayed on an interaction device, determine interaction options based on the eye data, and generate interaction instructions; the eye data includes gaze direction and gaze duration; The video recognition application module is used to obtain patient videos in real time based on camera points, identify the patient videos, and update the determination process of interactive options; wherein the update target is the threshold condition of eye data.

[0011] As a further solution of the present invention: the interactive preprocessing module includes: A location acquisition unit is used to acquire the patient's location in real time, generate time items and location items, and insert them into the database; An order determination unit, configured to calculate a distance between a patient's position and an interaction position of an interaction option, and determine an interaction order based on the distance; the interaction order is in ascending order of distance; The update execution unit is used to update the interaction option library based on the interaction sequence.

[0012] As a further solution of the present invention: the interactive instruction generation module includes: A display unit, configured to display an interaction option library containing an interaction sequence based on a display installed on the interaction device; An eye recognition unit, configured to obtain the patient's eye data based on a camera installed on the interactive device, and obtain the patient's gaze direction at each moment; a duration recording unit, configured to determine an interaction position on the display according to the gaze direction, and determine a gaze duration at the interaction position; The generation execution unit is used to read the interaction options at the interaction position and generate interaction instructions when the gaze duration reaches a preset duration threshold.

[0013] As a further solution of the present invention: the video recognition application module includes: A behavior feature extraction unit is used to obtain patient videos in real time based on camera locations, identify the patient videos, and extract the patient's behavior features including time; A feature group generating unit is configured to query the behavior features within a preset time range before the generation of the interaction instruction each time an interaction instruction is generated, to obtain a feature group; an average feature determination unit, configured to count all feature groups of each interaction instruction and determine an average feature group of the interaction instruction; A matching degree calculation unit is used to read the behavior characteristics within a preset time range in real time based on the current moment, compare the behavior characteristics with the average feature group of each interaction instruction, and calculate the matching degree; The duration threshold adjustment unit is used to adjust the duration threshold of the interaction option corresponding to each interaction instruction according to the matching degree.

[0014] Compared with the existing technology, the beneficial effects of the present invention are: the present invention obtains eye data, uses the eye data as input, simulates a mouse-like effect, and is used to control various smart devices. The operation process is extremely convenient. In conjunction with a large number of smart devices, it can greatly reduce nursing costs and also allow patients to gain great satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention.

[0016] Figure 1 The flowchart of the eye-controlled gaze interaction method for patients with ALS is shown.

[0017] Figure 2 This is a first sub-flow flowchart of the eye-controlled gaze interaction method for ALS patients.

[0018] Figure 3 This is a second sub-flow flowchart of the eye-controlled gaze interaction method for ALS patients.

[0019] Figure 4 This is the third sub-flow block diagram of the eye-controlled gaze interaction method for ALS patients.

[0020] Figure 5 This is a structural block diagram of the eye-controlled gaze interaction system for ALS patients. DETAILED DESCRIPTION

[0021] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] Figure 1 The flowchart of the eye-controlled gaze interaction method for ALS patients is as follows. In an embodiment of the present invention, an eye-controlled gaze interaction method for ALS patients includes: Step S100: receiving an interaction request containing authorization information from a patient, obtaining spatial planning information of the interaction area, selecting imaging points based on the spatial planning information, and simultaneously determining imaging parameters; wherein the selected imaging points and determined imaging parameters are used to ensure that the total imaging area is not smaller than the interaction area; The technical solution of the present invention needs to obtain the patient's behavioral data and facial data. Therefore, it requires the patient's explicit authorization to proceed. When sending an interaction request, the patient also sends authorization information. In subsequent steps, the present invention defaults to having authority; the interaction process occurs in the interaction area. The conventional interaction area is the home scene. The spatial planning information of the interaction area is obtained, and then some camera points are determined, and the camera parameters of the camera points are determined. In actual applications, cameras are generally installed at the camera points, and the camera parameters are the working parameters of the camera; it should be noted that the selected camera points and the determined camera parameters are used to determine that the total camera area is not less than the interaction area, which means that the set of acquisition areas of all cameras completely covers the entire interaction area.

[0023] It is worth mentioning that the space planning information generally refers to walls.

[0024] Step S200: obtaining the patient's position in real time, and constructing an interaction option library containing an interaction sequence according to the position; With the help of a camera, the patient's location at each moment can be obtained (the location is obtained very frequently). Based on the patient's location, the interaction needs that the patient may want to perform can be determined, and then an interaction option library containing an interaction sequence can be constructed. In actual applications, the interaction option library containing an interaction sequence is generally a UI interface containing multiple options. Specifically, regarding the interaction needs that the patient may want to perform based on the patient's location, for example, if the patient is next to a washing machine, then the activity he may want to perform is washing clothes.

[0025] Step S300: Based on the interactive device displaying an interactive option library containing an interactive sequence, the patient's eye data is acquired in real time, interactive options are determined based on the eye data, and interactive instructions are generated; the eye data includes gaze direction and gaze duration; The interactive device is generally a device that the patient carries with him / her, such as a display installed on a wheelchair with a camera installed on the display. This display with a camera can be compared to a tablet computer with lower performance requirements. The interactive option library containing the interactive sequence is determined according to the patient's position and displayed on the interactive device. The patient's facial data is obtained based on the camera on the interactive device, and the facial data is recognized. The eye data can be located and analyzed to determine which interactive option the patient wants to select. The interactive instructions are generated according to the selected interactive option.

[0026] Among them, the eye data includes gaze direction and gaze duration. Since the patient's eye position is known and the position of the display is also known, the two positions and the gaze direction can be combined to determine the corresponding position of the patient's line of sight on the display, which can provide an effect similar to that of a mouse. The gaze duration is the time the patient gazes at a certain position, which can provide an effect similar to that of clicking.

[0027] Step S400: acquiring a patient video in real time based on the camera position, identifying the patient video, and updating the interactive option determination process; wherein the update target is a threshold condition of the eye data; During the patient's interaction process, the patient's video is obtained in real time based on the camera installed at the camera point (not the camera of the interactive device). By identifying the patient's video, its behavior can be identified, and then the relationship between the behavior and the interactive options can be determined, and then the interaction process can be optimized. The optimization goal is to achieve the conditions for the gaze duration. Generally speaking, if the probability of a certain behavior corresponding to a certain choice is extremely high, then the achievement condition can be very small, making it easier for the gaze duration to achieve the condition, thereby improving the interaction efficiency.

[0028] Figure 2This is a first sub-flow diagram of an eye-controlled gaze interaction method for ALS patients. The steps of obtaining the patient's position in real time and constructing an interaction option library containing an interaction sequence based on the position include: Step S201: Acquire the patient's location in real time, generate time items and location items, and insert them into the database; Step S202: Calculating the distance between the patient's position and the interaction position of the interaction option, and determining the interaction order according to the distance; the interaction order is in ascending order of distance; Step S203: updating the interaction option library based on the interaction sequence.

[0029] In an example of the technical solution of the present invention, the construction process of the interaction option library is explained. The patient's location is obtained in real time, time items and location items are generated, and inserted into the database. At this time, the database records the patient's historical location data; for each interaction option in the interaction scene, it is an interaction with a real device, so there is an interaction position, and the distance between the patient's location and the interaction position of the interaction option is calculated. All interaction options are sorted from near to far according to the distance to obtain an interaction option library; in the sorted interaction option library, the interaction option that is closer to the patient is ranked higher.

[0030] Figure 3 This is a flowchart of the second sub-process of the eye-controlled gaze interaction method for patients with ALS. The steps of displaying an interaction option library containing an interaction sequence based on an interactive device, acquiring the patient's eye data in real time, determining interaction options based on the eye data, and generating interaction instructions include: Step S301: displaying an interaction option library containing an interaction sequence based on a display installed on an interaction device; Step S302: acquiring the patient's eye data based on a camera installed on the interactive device to obtain the patient's gaze direction at each moment; Step S303: determining an interaction position on the display according to the gaze direction, and determining a gaze duration at the interaction position; Step S304: When the gaze duration reaches a preset duration threshold, the interaction options at the interaction position are read and an interaction instruction is generated.

[0031] In one example of the technical solution of the present invention, an interactive option library containing an interactive sequence is displayed based on a display installed on an interactive device, and the patient's eye data is obtained based on a camera installed on the interactive device, the patient's gaze direction at each moment is obtained, the eye position is obtained, the display position is read, the interactive position on the display is determined according to the gaze direction, and the gaze time at each interactive position is accumulated in real time. When the gaze time reaches a preset time threshold, the interactive option at the interactive position is read and an interactive instruction is generated; the time threshold can be set to two seconds, that is, when the patient gazes at a certain position for two seconds, the interactive option corresponding to the position is read as the selected target, and an interactive instruction is generated.

[0032] Figure 4 This is a block diagram of the third sub-process of the eye-controlled gaze interaction method for ALS patients. The steps of acquiring patient video in real time based on camera positions, identifying the patient video, and updating the determination process of interaction options include: Step S401: acquiring a patient video in real time based on camera locations, identifying the patient video, and extracting the patient's behavioral features including time; Step S402: Every time an interaction instruction is generated, the behavior characteristics within a preset time range before the interaction instruction generation moment are searched to obtain a feature group; Step S403: Count all feature groups of each interaction instruction and determine the average feature group of the interaction instruction; Step S404: Reading the behavior characteristics within a preset time range in real time based on the current moment, comparing the behavior characteristics with the average feature group of each interaction instruction, and calculating the matching degree; Step S405: adjusting the duration threshold of the interaction options corresponding to each interaction instruction according to the matching degree.

[0033] In one example of the technical solution of the present invention, a camera installed at a camera point is used to obtain a patient video in real time, and the patient video is identified to obtain the patient's behavioral characteristics containing time. The behavioral characteristics extraction process can adopt existing technology, and the extracted behavioral characteristics are image features representing the patient's actions.

[0034] Furthermore, each time an interaction instruction is generated, it means that the patient has selected an interaction option. The behavioral characteristics within a preset time range before the interaction instruction is generated are queried to obtain a feature group. The time range generally does not exceed 30 seconds, assuming it is 5 seconds. The above process is to query the behavioral characteristics within 5 seconds before the interaction instruction is generated to obtain a feature group; an interaction instruction can be generated at different times, and each time it is generated, a feature group can be obtained. All feature groups are counted to determine the average feature group as the average feature group of the interaction instruction.

[0035] After the average feature group of each interactive instruction is determined, the application link is carried out. The behavioral characteristics within the preset time range are read in real time based on the current moment. This represents the patient's most recent behavioral characteristics. The behavioral characteristics are compared with the average feature group of each interactive instruction, and the matching degree is calculated. The greater the matching degree, the more the behavioral characteristics match the average feature group. At this time, the duration threshold of the corresponding interactive option can be smaller, so that the interactive instruction can be triggered even when the gaze duration is shorter. In actual application, even if the gaze duration of 2 seconds is shortened to 1.9 seconds, efficiency can be improved.

[0036] As a preferred embodiment of the technical solution of the present invention, the step of counting all feature groups of each interaction instruction and determining the average feature group of the interaction instruction includes: For each interaction instruction, count the feature groups that generate the interaction instruction each time; Get the union of all feature groups to get the total behavioral feature group; For each behavioral feature in the total behavioral feature group, query the number of times it appears in the feature group; When the number of times reaches a preset number threshold, it is used as an average feature; All average features are counted to determine the average feature group of the interaction instruction.

[0037] In an example of the technical solution of the present invention, a specific average feature group generation scheme is provided. For each interaction instruction, the feature group generated each time the interaction instruction is counted, the union of all feature groups is obtained, and the total behavior feature group is obtained. For each behavior feature in the total behavior feature group, the number of times it appears in the feature group is queried. In other words, for each behavior feature that appears in the feature group, the number of times it appears in all feature groups of the same interaction instruction is queried. When the number of appearances is large enough, it is marked as an average feature; the average features of all marks are counted to obtain the average feature group of the interaction instruction.

[0038] As a preferred embodiment of the technical solution of the present invention, the step of reading the behavior characteristics within a preset time range in real time based on the current moment, comparing the behavior characteristics with the average feature group of each interaction instruction, and calculating the matching degree includes: In real time, the current moment is used as the end moment, and behavioral features within the preset time range are obtained forward to construct a real-time behavior group; Read the average feature group of each interaction instruction in turn; Select behavioral features in the real-time behavior group in turn, compare them with all behavioral features in the average feature group, and determine the maximum similarity; The mean of the maximum similarity of each behavior feature in the real-time behavior group is calculated as the matching degree.

[0039] In an example of the technical solution of the present invention, the calculation process of the matching degree is explained. The current moment is taken as the last moment in real time, and the behavioral features within the preset time range are obtained forward. A real-time behavior group is constructed, and the average feature group of each interactive instruction is read in turn. The behavioral features are selected in the real-time behavior group in turn (each behavioral feature in the real-time behavior group needs to be analyzed once), and compared with all the behavioral features in the average feature group to determine the maximum similarity. The maximum similarity of each behavioral feature in the real-time behavior group is read, and the average is calculated as the matching degree.

[0040] It is worth mentioning that the feature group, real-time behavior group and average feature group of the present invention are all sets. Sets are disordered, and the elements therein are not related to order.

[0041] The process of adjusting the duration threshold of the interaction options corresponding to each interaction instruction according to the matching degree includes: Where, is the adjusted duration threshold, is the preset standard duration threshold, For matching, is the preset correction factor.

[0042] The above content provides the calculation process of the duration threshold. First, the staff pre-sets a standard duration threshold, such as two seconds. On this basis, the weight is determined according to the matching degree, and the original standard duration threshold is adjusted. The greater the matching degree, the smaller the adjusted duration threshold. Among them, since the matching degree uses similarity, which is generally in the range of zero to one, this makes The value range of the item is very small, so this application introduces a correction coefficient to enlarge or reduce the matching degree according to needs.

[0043] It is worth mentioning that Generally, it takes a positive value. In some cases, it can take zero. When it takes zero, it means that the duration threshold remains unchanged. Generally, it does not take a negative value because the effect of taking a negative value is that the greater the matching degree, the greater the duration threshold. This may have certain practical significance. For example, the more matching the interaction options, the more carefully they should be determined. This generally occurs in the testing phase of behavioral characteristics and is used to determine whether there is a certain relationship between behavioral characteristics and interactive instructions.

[0044] Figure 5 : is a structural block diagram of an eye-controlled gaze interaction system for ALS patients. In an embodiment of the present invention, an eye-controlled gaze interaction system for ALS patients is provided. The system 10 includes: The point calibration module 11 is used to receive an interaction request containing authorization information sent by the patient, obtain spatial planning information of the interaction area, select camera points based on the spatial planning information, and simultaneously determine camera parameters; wherein the selected camera points and determined camera parameters are used to ensure that the total camera area is not smaller than the interaction area; An interaction pre-processing module 12 is used to obtain the patient's position in real time and construct an interaction option library containing an interaction sequence according to the position; An interaction instruction generation module 13 is configured to acquire the patient's eye data in real time based on an interaction option library containing an interaction sequence displayed by the interaction device, determine interaction options based on the eye data, and generate interaction instructions; the eye data includes gaze direction and gaze duration; The video recognition application module 14 is used to obtain patient videos in real time based on camera locations, identify the patient videos, and update the determination process of the interactive options; wherein the update target is the threshold condition of the eye data.

[0045] Furthermore, the interaction pre-processing module 12 includes: A location acquisition unit is used to acquire the patient's location in real time, generate time items and location items, and insert them into the database; An order determination unit, configured to calculate a distance between a patient's position and an interaction position of an interaction option, and determine an interaction order based on the distance; the interaction order is in ascending order of distance; The update execution unit is used to update the interaction option library based on the interaction sequence.

[0046] Specifically, the interaction instruction generating module 13 includes: A display unit, configured to display an interaction option library containing an interaction sequence based on a display installed on the interaction device; An eye recognition unit, configured to obtain the patient's eye data based on a camera installed on the interactive device, and obtain the patient's gaze direction at each moment; a duration recording unit, configured to determine an interaction position on the display according to the gaze direction, and determine a gaze duration at the interaction position; The generation execution unit is used to read the interaction options at the interaction position and generate interaction instructions when the gaze duration reaches a preset duration threshold.

[0047] Furthermore, the video recognition application module 14 includes: A behavior feature extraction unit is used to obtain patient videos in real time based on camera locations, identify the patient videos, and extract the patient's behavior features including time; A feature group generating unit is configured to query the behavior features within a preset time range before the generation of the interaction instruction each time an interaction instruction is generated, to obtain a feature group; an average feature determination unit, configured to count all feature groups of each interaction instruction and determine an average feature group of the interaction instruction; A matching degree calculation unit is used to read the behavior characteristics within a preset time range in real time based on the current moment, compare the behavior characteristics with the average feature group of each interaction instruction, and calculate the matching degree; The duration threshold adjustment unit is used to adjust the duration threshold of the interaction option corresponding to each interaction instruction according to the matching degree.

[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An eye-controlled gaze interaction method for patients with ALS, characterized in that: The method comprises: Receive an interaction request containing authorization information from a patient, obtain spatial planning information of the interaction area, select imaging points based on the spatial planning information, and simultaneously determine imaging parameters; wherein the selected imaging points and determined imaging parameters are used to ensure that the total imaging area is not less than the interaction area; Acquire the patient's location in real time, and construct an interaction option library containing an interaction sequence based on the location; Based on the interactive device displaying an interactive option library containing an interactive sequence, the patient's eye data is acquired in real time, the interactive options are determined based on the eye data, and an interactive instruction is generated; the eye data includes gaze direction and gaze duration; The patient video is acquired in real time based on the camera position, the patient video is identified, and the determination process of the interactive options is updated; wherein the update target is the threshold condition of the eye data.

2. The eye-controlled gaze interaction method for ALS patients according to claim 1, characterized in that: The step of obtaining the patient's position in real time and constructing an interaction option library containing an interaction sequence according to the position includes: Obtain the patient's location in real time, generate time items and location items, and insert them into the database; Calculating the distance between the patient's position and the interaction position of the interaction option, and determining the interaction order according to the distance; the interaction order is in ascending order of distance; Updates the interaction options library based on the interaction sequence.

3. The eye-controlled gaze interaction method for ALS patients according to claim 1, characterized in that: The steps of displaying an interaction option library containing an interaction sequence based on an interaction device, acquiring eye data of a patient in real time, determining interaction options according to the eye data, and generating interaction instructions include: Displaying an interaction option library containing an interaction sequence based on a display installed on the interaction device; Obtain the patient's eye data based on the camera installed on the interactive device to obtain the patient's gaze direction at each moment; determining an interaction location on the display based on the gaze direction, and determining a gaze duration at the interaction location; When the gaze duration reaches a preset duration threshold, the interaction options at the interaction position are read and an interaction instruction is generated.

4. The eye-controlled gaze interaction method for ALS patients according to claim 1, characterized in that: The steps of acquiring patient video in real time based on camera positions, identifying the patient video, and updating the interactive options include: Acquire patient videos in real time based on camera locations, identify the patient videos, and extract the patient's temporal behavioral features; Each time an interaction instruction is generated, the behavior features within a preset time range before the interaction instruction generation moment are queried to obtain a feature group; Count all feature groups of each interaction instruction and determine the average feature group of the interaction instruction; Based on the current moment, the behavioral characteristics within the preset time range are read in real time, the behavioral characteristics are compared with the average feature group of each interaction instruction, and the matching degree is calculated; The duration threshold of the interaction option corresponding to each interaction instruction is adjusted according to the matching degree.

5. The eye-controlled gaze interaction method for ALS patients according to claim 4, characterized in that: The step of counting all feature groups of each interaction instruction to determine the average feature group of the interaction instruction includes: For each interaction instruction, count the feature groups that generate the interaction instruction each time; Get the union of all feature groups to get the total behavioral feature group; For each behavioral feature in the total behavioral feature group, query the number of times it appears in the feature group; When the number of times reaches a preset number threshold, it is used as an average feature; All average features are counted to determine the average feature group of the interaction instruction.

6. The eye-controlled gaze interaction method for ALS patients according to claim 5, characterized in that: The step of reading the behavior characteristics within a preset time range in real time based on the current moment, comparing the behavior characteristics with the average feature group of each interaction instruction, and calculating the matching degree includes: In real time, the current moment is used as the end moment, and behavioral features within the preset time range are obtained forward to construct a real-time behavior group; Read the average feature group of each interaction instruction in turn; Select behavioral features in the real-time behavior group in turn, compare them with all behavioral features in the average feature group, and determine the maximum similarity; Calculate the mean of the maximum similarity of each behavior feature of the real-time behavior group as the matching degree; The process of adjusting the duration threshold of the interaction options corresponding to each interaction instruction according to the matching degree includes: Where, is the adjusted duration threshold, is the preset standard duration threshold, For matching, is the preset correction factor.

7. An eye-controlled gaze interaction system for patients with ALS, characterized in that: The system comprises: A point calibration module is configured to receive an interaction request containing authorization information from a patient, obtain spatial planning information of the interaction area, select imaging points based on the spatial planning information, and simultaneously determine imaging parameters; wherein the selected imaging points and determined imaging parameters are used to ensure that the total imaging area is not smaller than the interaction area; An interaction preprocessing module, configured to obtain the patient's position in real time and construct an interaction option library containing an interaction sequence according to the position; An interaction instruction generation module is configured to obtain the patient's eye data in real time based on an interaction option library containing an interaction sequence displayed on an interaction device, determine interaction options based on the eye data, and generate interaction instructions; the eye data includes gaze direction and gaze duration; The video recognition application module is used to obtain patient videos in real time based on camera points, identify the patient videos, and update the determination process of interactive options; wherein the update target is the threshold condition of eye data.

8. The eye-controlled gaze interaction system for ALS patients according to claim 7, characterized in that: The interactive preprocessing module includes: A location acquisition unit is used to acquire the patient's location in real time, generate time items and location items, and insert them into the database; An order determination unit, configured to calculate a distance between a patient's position and an interaction position of an interaction option, and determine an interaction order based on the distance; the interaction order is in ascending order of distance; The update execution unit is used to update the interaction option library based on the interaction sequence.

9. The eye-controlled gaze interaction system for ALS patients according to claim 7, characterized in that: The interactive instruction generation module includes: A display unit, configured to display an interaction option library containing an interaction sequence based on a display installed on the interaction device; An eye recognition unit, configured to obtain the patient's eye data based on a camera installed on the interactive device, and obtain the patient's gaze direction at each moment; a duration recording unit, configured to determine an interaction position on the display according to the gaze direction, and determine a gaze duration at the interaction position; The generation execution unit is used to read the interaction options at the interaction position and generate interaction instructions when the gaze duration reaches a preset duration threshold.

10. The eye-controlled gaze interaction system for ALS patients according to claim 7, characterized in that: The video recognition application module includes: A behavior feature extraction unit is used to obtain patient videos in real time based on camera locations, identify the patient videos, and extract the patient's behavior features including time; A feature group generating unit is configured to query the behavior features within a preset time range before the generation of the interaction instruction each time an interaction instruction is generated, to obtain a feature group; an average feature determination unit, configured to count all feature groups of each interaction instruction and determine an average feature group of the interaction instruction; A matching degree calculation unit is used to read the behavior characteristics within a preset time range in real time based on the current moment, compare the behavior characteristics with the average feature group of each interaction instruction, and calculate the matching degree; The duration threshold adjustment unit is used to adjust the duration threshold of the interaction option corresponding to each interaction instruction according to the matching degree.

Images