A telemetry type eye movement behavior classification method and device for people with visual field defects

Abstract

The application provides a kind of telemetering eye movement behavior classification method and device for visual field defect population, comprising: visual field area mapping, measuring visual field range, marking each visual field area, simulating and generating visible fixation point of calibration plane, establishing the mapping of visual field area from virtual visual field plane to real calibration plane;Gaze data correction, collect gaze data, get gaze drop point through visual field area mapping model, and carry out global area clustering of gaze drop point data, and correct the overall visual field area, invisible visual field area and extended visual field area;Eye movement behavior classification, by judging the gaze drop point area, the continuous gaze drop point trajectory is divided into different visual field area data sections, and the eye movement behavior in different visual field areas is classified.The application solves the dynamic accurate mapping problem of visual field range in telemetry scene, realizes efficient eye movement behavior classification in different visual field areas, and has the characteristics of strong expansibility, high robustness and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, in particular, especially relates to a telemetry eye movement behavior classification method and device for visual field loss population. BACKGROUND

[0002] Visual field loss mainly has two kinds: one is the damage of concentric visual field caused by retinitis, choroid loss or glaucoma, also known as tubular visual field; the other is the loss of hemianopia, most of which is hemianopia on the same side. By fixing the prism module near the upper and / or lower part of the residual central visual field, an artificial visual aid around the human eye is formed, which can help to expand the visual field range of the visual field loss population. Compared with the normal visual field population, the visible visual field range of the visual field loss population is reduced, the visible fixation points are reduced, and the visual behavior is changed, and the saccade path and fixation frequency are obviously different.

[0003] Most of the traditional eye movement behavior classification methods are based on head-mounted visual line tracking devices, which can analyze the eye movement behavior of normal population, but when analyzing the visual field loss population, due to the lack of accurate visual field range mapping, it is usually impossible to establish the relationship between eye movement behavior and real visual field range. In addition, when the visual field loss population wears a head-mounted optical visual aid, there are many eye movement behaviors to expand the visual field range, which affects the normal fixation habit, and if an additional head-mounted visual line tracking device is used, it may increase the visual load of the visual field loss population. SUMMARY

[0004] Therefore, the present application provides a telemetry eye movement behavior classification method and device for visual field loss population, which considers using telemetry devices for visual field range mapping, and correcting and optimizing the mapped visual field range of the visual field loss population through dynamic visual field area adjustment. And according to the motion characteristics of eye movement behavior in different visual field ranges, the eye movement behavior classification is carried out respectively. The present application solves the problem of dynamic and accurate mapping of visual field range in telemetry scene, realizes efficient eye movement behavior classification in different visual field areas, and has the characteristics of strong expansibility and high robustness.

[0005] The technical means adopted by the present application are as follows:

[0006] On the one hand, the present application provides a telemetry eye movement behavior classification method for visual field loss population, comprising:

[0007] S1, visual field area mapping: obtaining the real visual field range of the visual field loss population through visual field measurement, and quantifying it as a virtual human eye visual field range parameter; simulating and generating visible fixation points on the calibration plane, and iteratively adjusting the visual field area mapping model; using the calibration plane reference fixation points in the visible visual field range to realize the mapping of the visual field area from the virtual visual field plane to the real calibration plane;

[0008] S2, gaze data correction: based on a remote gaze tracking device, collecting the gaze data of the visual field defect population; using a visual field region mapping model, mapping the gaze data to a calibration plane, and calculating the gaze landing points; performing global region clustering on all gaze landing point data; performing position correction on the overall visual field region, and sequentially correcting the invisible region and the extended region, and calculating the corrected gaze landing point data;

[0009] S3, eye movement behavior classification: dividing the continuous gaze landing point trajectory into visible visual field region data segment, extended visual field region data segment and invisible visual field region data segment by judging the gaze landing point region, and classifying the eye movement behavior in different visual field regions.

[0010] Further, the real visual field range of the visual field defect population is obtained by visual field measurement, and is quantified as a virtual human eye visual field range parameter, including:

[0011] The visual field measurement result is converted into a gray quantization value in the visual field coordinate system, and the center of the visual field coordinate system is Euler zero degree angle;

[0012] The visual field measurement result is smoothed by using Gaussian filtering, and the boundary of the measurement result is reduced by using morphological erosion operation;

[0013] The visible visual field region and the invisible visual field region are labeled, and the reduced visible visual field region and invisible visual field region both have high confidence;

[0014] If the visual field defect population wears an optical visual aid, it further includes: labeling the extended visual field region.

[0015] Further, the visible fixation points are simulated and generated on the calibration plane, and the visual field region mapping model is iteratively adjusted, including:

[0016] The distance from the visual field defect population to the calibration plane is measured, the center position of the human eye on the calibration plane is labeled, the projection mapping of the human eye visual field range on the calibration plane is calculated by using the trigonometric function relationship, and the region mark of the visual field coordinate system is converted into the region mark of the calibration plane coordinate; the center of the calibration plane coordinate system is the center position of the human eye on the calibration plane;

[0017] The center point of the visible visual field region is calculated, and the distance from the point to the upper, lower, left and right boundaries is calculated, and the vertical and horizontal star rays of the point to the boundary are made, and the star ray center point is taken as the reference calibration point of the visible region;

[0018] The visual field defect population adjusts the head and eyeball position until all the reference calibration points enter the visible visual field region; if there are multiple visible visual field regions in the center, the largest visible visual field region is selected; if the visible visual field regions are scattered, the center point of the visible visual field region is taken as the reference calibration point.

[0019] Further, the mapping from the virtual visual field plane to the real calibration plane is established by using the calibration plane in the visible visual field range as the reference gaze point, including:

[0020] Establishing a mapping relationship between the human eye features and the visible calibration points;

[0021] Establishing a mapping relationship between the virtual visual field plane and the real calibration plane;

[0022] The required transformation parameters are learned by using joint minimization of re-projection error constraints, which are solved by a differential evolution method, and the line of sight landing points are mapped to the calibration plane.

[0023] Further, the line of sight data of the visual field defect population is collected, including: collecting human eye images of the visual field defect population by using a human eye camera, and positioning the pupil center position by pupil detection.

[0024] Further, all line of sight landing point data are globally regionally clustered, including:

[0025] A density-based clustering method is used to cluster the line of sight landing points in the visible visual field area, filter low-density areas and find dense areas;

[0026] According to the existing visual field mapping results, the clustering results are corresponded to different visual field areas;

[0027] For global line of sight landing points, the dense sample area should be located in the visible visual field area or the extended visual field area; the edge points connecting different clustering results are used to convert the clustering results into a clustering graph, and the areas with an area less than a threshold value are deleted.

[0028] Further, the visual field area is corrected, including:

[0029] The visible visual field area is taken as the reference, the clustering graph in the visible visual field area is intersected with the mapped visual field area, if the coincidence rate of the visual field area and the clustering graph is greater than 90%, it is judged that the visual field area does not need to be corrected, otherwise, the real value of the visible visual field area center on the calibration plane is manually labeled, the scale and translation parameters of the calibration plane are solved;

[0030] The invisible area correction includes: the clustering graph in the invisible area is generally ignored and not processed, if the clustering graph is located at the boundary between the visible area and the invisible area, the invisible area is corrected, the area is morphologically eroded, and the area is merged into the visible area;

[0031] The extended area correction includes: intersecting the cluster graph in the extended area with the mapped extended field of view area, if the coincidence rate of the extended field of view area and the area cluster graph is greater than 90%, it is judged that the extended field of view area does not need to be corrected, otherwise, the range of the extended field of view area needs to be re-labeled;

[0032] The line of sight landing point data correction includes: according to the scale and translation parameters, the line of sight landing point data in the visible field of view area and the extended field of view area is translated to the new line of sight landing point position.

[0033] Further, the eye movement behavior of different field of view areas is classified, including:

[0034] In the visible field of view area data segment, the multi-level hidden Markov model is used for coarse classification and fine classification of eye movement behavior data, first, the maximum expectation algorithm is used to solve the data threshold parameter in the first layer hidden Markov model, and the data is divided into fixation behavior and saccade behavior with large momentum difference, and then the maximum expectation algorithm is used to solve the data threshold parameter in the second layer hidden Markov model, and the fixation behavior data is further divided into fixation behavior and smooth trailing behavior;

[0035] In the extended field of view area data segment, the single-level hidden Markov model is used for classification of eye movement behavior data, and the threshold parameter of the extended field of view area data is solved by using the hidden Markov model, and the data is divided into fixation behavior and saccade behavior;

[0036] For the case that the range of the extended field of view is large, the visible field of view area data segment is referred to, which is divided into fixation behavior, saccade behavior and smooth trailing behavior; for the data segment connecting the visible area and the extended area, i.e. the landing point moving from the visible field of view area to the extended field of view area, it is separately regarded as a kind of eye movement behavior; in the invisible field of view area data segment, only the continuous motion data segment is labeled, and the sparse data is ignored;

[0037] The eye movement behavior classification results of the visible field of view area, the extended field of view area and the invisible field of view area are combined.

[0038] Further, the line of sight landing point area discrimination includes:

[0039] The motion momentum and acceleration of the current line of sight landing point are calculated; in the case of uniform sampling, the time factor is ignored, the motion momentum value is the Euclidean distance between the current line of sight landing point and the line of sight landing point at the previous time, and the acceleration value is the difference between the current motion momentum and the motion momentum at the previous time;

[0040] According to the line of sight landing point area automatic standard, the region label of the line of sight landing point is determined, i.e. the visible field of view area, the invisible field of view area or the extended field of view area;

[0041] According to the continuous region label result, the gaze landing point data is divided into data segments of different visual field regions, and the classification of the gaze landing point data is completed.

[0042] Further, the output eye movement behaviors include: fixation behaviors, saccade behaviors, smooth pursuit behaviors, region jump behaviors in the visible visual field region, fixation behaviors, saccade behaviors in the extended visual field region, and continuous eye movement behaviors in the invisible visual field region.

[0043] In another aspect, the application also provides a telemetry-based eye movement behavior classification device for people with visual field defects, which performs gaze tracking based on a telemetry-based gaze tracking device, and the classification device comprises:

[0044] A visual field region mapping module is configured to obtain the real visual field range of the people with visual field defects through visual field measurement, and quantize the real visual field range into virtual human eye visual field range parameters; simulate and generate visible fixation points on a calibration plane, and iteratively adjust a visual field region mapping model; and map the visual field region from a virtual visual field plane to a real calibration plane by using the calibration plane reference fixation points in the visible visual field range.

[0045] A gaze data correction module is configured to collect gaze data of the people with visual field defects; map the gaze data to the calibration plane by using the visual field region mapping model, and calculate gaze landing points; perform global region clustering on all gaze landing point data; perform position correction on the overall visual field region, and sequentially correct the invisible region and the extended region, and calculate corrected gaze landing point data.

[0046] An eye movement behavior classification module is configured to divide continuous gaze landing point trajectories into visible visual field region data segments, extended visual field region data segments and invisible visual field region data segments by judging the gaze landing point regions, and classify eye movement behaviors in different visual field regions.

[0047] Compared with the prior art, the application has the following advantages:

[0048] The existing line-of-sight tracking and eye movement behavior method is generally only for normal visual field population without considering the particularity of visual field defect population. In the scene of visual training, work and study, etc., the precise visual field range mapping of the visual field defect population needs to be established when using the remote line-of-sight tracking device. The various main steps of the present application bring beneficial effects: 1. The line-of-sight calibration is carried out by using the visible visual field range to eliminate the mapping error between the virtual visual field plane and the real calibration plane; 2. The visual field range is corrected by using the line-of-sight landing point clustering, and the visual field areas of the visible visual field, the invisible visual field and the extended visual field are iteratively adjusted, so that the precise visual field range can be obtained; 3. The motion characteristics of the eye movement behavior in different visual field ranges are considered, and the eye movement behavior classification results are obtained by respectively processing. The present application can be used for eye movement behavior estimation of the visual field defect population with or without wearing optical visual aid, and can also be used for eye movement behavior estimation of the normal visual field population after wearing optical extension device. The present application solves the dynamic and accurate mapping problem of the visual field range in the remote scene, realizes the efficient eye movement behavior classification in different visual field areas, and has the characteristics of strong expansibility and high robustness. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0050] Figure 1 The flow chart of a remote eye movement behavior classification method for visual field defect population in the embodiment of the present application is shown.

[0051] Figure 2 The visual field area mapping schematic diagram in the embodiment of the present application is shown.

[0052] Figure 3 The eye movement behavior schematic diagram in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0054] The application provides a telemetry eye movement behavior classification method for visual field defect population, which is used for non-contact line-of-sight tracking and eye movement behavior classification in the case of wearing or not wearing optical visual aids by visual field defect population. The method is mainly applied to visual training of visual field defect population, understanding of visual behavior of visual field defect population in the case of wearing or not wearing optical visual aids, and evaluation of auxiliary technology and visual training effect. The application can also be applied to visual field extension visual training of normal visual field population.

[0055] As shown in Figure 1 The application provides a telemetry eye movement behavior classification method for visual field defect population, which mainly includes the steps of visual field region mapping, line-of-sight data correction and eye movement behavior classification. The specific technical solutions are as follows:

[0056] S1, visual field region mapping: the real visual field range of the visual field defect population is obtained through visual field measurement, and is quantified as a virtual human eye visual field range parameter, the visible visual field region, the invisible visual field region and the extended visual field region are labeled, the visible fixation point is simulated and generated on the calibration plane, the visual field region mapping model is iteratively adjusted, the calibration plane reference fixation point in the visible visual field range is used to establish the mapping from the virtual visual field plane to the real calibration plane.

[0057] The specific steps are as follows:

[0058] S11, visual field measurement: the existing visual field meter is usually used, such as plane visual field meter, arc visual field meter and automatic visual field meter. First, the visual field measurement result is converted into gray quantization value in visual field coordinate system, and different color scales are used to represent visual acuity information. The center of visual field coordinate system is Euler zero degree angle. Then, the visual field measurement result is smoothed by using Gaussian filtering, and the boundary of the measurement result is reduced by using morphological erosion operation. Finally, the visible visual field region and the invisible visual field region are labeled, wherein the reduced visible visual field region and invisible visual field region have high confidence. If the visual field defect population wears optical visual aids, the extended visual field region needs to be labeled in this step.

[0059] S12, line-of-sight calibration: in the calibration plane, a virtual visual field area mark is projected, reference calibration points are generated in the visible visual field area, and line-of-sight calibration is completed using these visible calibration points. First, measure the distance from the visual field defect population to the calibration plane, mark the center position of the human eye on the calibration plane, calculate the projection mapping of the human eye visual field range on the calibration plane using trigonometric functions, and convert the area mark of the visual field coordinate system to the area mark of the calibration plane coordinate. The center of the calibration plane coordinate system is the center position of the human eye on the calibration plane. Then, calculate the center point of the visible visual field area, and calculate the distance from the point to the upper, lower, left and right boundaries. The vertical and horizontal star rays of the point to the boundary are made, and the center point of the star ray is taken as the reference calibration point of the visible area. The visual field defect population adjusts the head and eyeball position until all the reference calibration points enter the visible visual field area. For the visual field defect population, there may be multiple visible visual field area centers, so the largest visible visual field area is selected. If the visible visual field area is relatively scattered, the center point of the visible visual field area is taken as the reference calibration point. The calibration step needs to be repeated two to three times, which can be based on the current star ray, rotated clockwise by 30 degrees, and then the star ray of the point to the boundary is made, and the center point of the star ray is taken as the reference calibration point of the visible area. Each time 5 reference calibration points can be collected, of which 1 is the center point of the visible visual field area, and 4 are the center points of the star rays.

[0060] S13, visual field area mapping: the mapping relationship between the human eye features and the visible calibration points is established. First, the human eye successively gazes at the reference calibration points of the visible visual field, and the human eye features corresponding to the reference calibration points are collected. Here, the coordinate position of the pupil in the human eye image is taken as the human eye feature. The center position of the pupil or iris can be obtained by using the pupil detection method, and the two-dimensional vector of the center position is the human eye feature. Then, the mapping relationship between the virtual visual field plane X vf and the real calibration plane X screen is established:

[0061]

[0062] wherein,

[0063]

[0064]

[0065] T = [t x t y t z ] T .

[0066] Finally, the transformation parameters required are learned by using joint minimization of re-projection error constraints, solved by differential evolution method, and the line-of-sight landing point is mapped to the calibration plane:

[0067]

[0068] where N is the number of reference calibration points, {x gt ,y gt} is the real calibration point position, {x pd ,y pd} is the predicted calibration position.

[0069] S2, gaze data correction: collect the gaze data of the visual field loss population, map the existing gaze data to the calibration plane using the visual field area mapping model, and calculate the gaze landing point. Cluster all gaze landing point data globally, and according to statistical observation, the dense area of the gaze landing point of the visual field loss population should be located in the visible visual field area. If they wear optical visual aids (such as prism visual aids), the extended visual field area will also have a dense area of gaze landing points, and the invisible visual field area should have very few gaze landing points. At this time, the position of the overall visual field area is corrected, and the invisible area and the extended area are corrected in turn to calculate the corrected gaze landing point data.

[0070] The specific steps are as follows:

[0071] S21, gaze data collection: collect human eye images using a human eye camera, and locate the pupil center position through pupil detection. Map the pupil center feature to the gaze landing point on the calibration plane using the visual field area mapping model.

[0072] S22, global area clustering of gaze landing point data: use a density-based clustering method to cluster the gaze landing points in the visible visual field area, filter the low-density area, and find the dense area. According to the existing visual field mapping result, the clustering result is corresponded to different visual field areas. For the global gaze landing point, the dense sample area should be located in the visible visual field area or the extended visual field area. Connect the edge points of different clustering results, convert the clustering results into a clustering graph, and delete the areas with an area less than a threshold value.

[0073] S23, visual field area correction: take the visible visual field area as the benchmark, and perform intersection processing on the clustering graph in the visible visual field area and the mapped visual field area. If the overlap rate of the visual field area and the clustering graph is greater than 90%, it is judged that the visual field area does not need to be corrected, otherwise, the real value of the center of the visible visual field area on the calibration plane needs to be manually labeled, and the scale and translation parameters of the calibration plane are solved:

[0074]

[0075] where s x and s y are scale parameters, w x and w y are translation parameters, Xpd is the uncorrected line of sight landing point data, is the calibration plane.

[0076] S24, invisible region correction: the cluster graph of the invisible region is generally ignored and not processed, if the cluster graph is located at the boundary of the visible region and the invisible region, the invisible region is corrected, the morphological erosion processing is performed on the region, and the region is merged into the visible region.

[0077] S25, extended region correction: since the extended region is separately labeled, the cluster graph of the extended region should be mostly located in the extended region, and a small part of noise data is located in the visible field of view region. The cluster graph in the extended region is intersected with the mapped extended field of view region, if the coincidence rate of the extended field of view region and the cluster graph in the region is greater than 90%, it is judged that the extended field of view region does not need to be corrected, otherwise, the range of the extended field of view region needs to be re-labeled.

[0078] S26, line of sight landing point data correction: according to the scale and translation parameters, the line of sight landing point data in the visible field of view region and the extended field of view region is translated to a new line of sight landing point position.

[0079] S3, eye movement behavior classification: by judging the line of sight landing point region, the continuous line of sight landing point trajectory is divided into different field of view region data segments, and the eye movement behavior in different field of view regions is classified.

[0080] The eye movement behavior of the visual field defect population is different in the visible field and the extended field, and the movement law is also different. Therefore, the eye movement behavior of the visual field defect population in different visual field regions is classified to understand the degree of damage of visual field defects to visual behavior. The eye movement behavior classification divides the continuous gaze point trajectory into visible field region data segment, extended field region data segment and invisible field region data segment by judging the gaze point region. In the visible field region data segment, the multi-level hidden Markov model is used for coarse classification and fine classification of eye movement behavior data. First, the maximum expectation algorithm is used to solve the data threshold parameter in the first layer hidden Markov model, and the data is divided into fixation behavior and saccade behavior with large momentum difference. Then, the maximum expectation algorithm is used to solve the data threshold parameter in the second layer hidden Markov model, and the fixation behavior data is further divided into fixation behavior and smooth pursuit behavior. In the extended field region data segment, the single-level hidden Markov model is used for classification of eye movement behavior data. The threshold parameter of the extended field region data is solved by using the hidden Markov model, and the data is divided into fixation behavior and saccade behavior. Here, for the case of a larger extended field range, the visible field region data segment can be processed, which is divided into fixation behavior, saccade behavior and smooth pursuit behavior. For the data segment connecting the visible region and the extended region, i.e. the gaze point moving from the visible field region to the extended field region, it is separately regarded as a kind of eye movement behavior. In the invisible field region data segment, only the continuous movement data segment is marked, and the relatively sparse data is ignored.

[0081] Finally, the eye movement behavior classification results of the visible field region, the extended field region and the invisible field region are merged. A total of seven kinds of eye movement behaviors are output, including fixation behavior, saccade behavior, smooth pursuit behavior and region jump behavior in the visible field region, fixation behavior and saccade behavior in the extended field region, and continuous eye movement behavior in the invisible field region.

[0082] The specific steps are as follows:

[0083] S31, gaze point region discrimination: according to the space-time relationship of the gaze point data, the gaze point data is divided into continuous data segments of different visual field regions. First, the movement momentum and acceleration of the current gaze point are calculated. In the case of uniform sampling, the time factor is ignored, the movement momentum value is the Euclidean distance between the current gaze point and the gaze point at the previous time, and the acceleration value is the difference between the current movement momentum and the previous movement momentum. Then, according to the gaze point region automatic standard, the region label of the gaze point is obtained, i.e. visible field region, invisible field region or extended field region. Finally, according to the continuous region label result, the gaze point data is divided into data segments of different visual field regions, and the classification of the gaze point data is completed.

[0084] S32, visible field of view area data segment processing: the data in the visible field of view area includes fixation behavior, saccade behavior, smooth pursuit behavior and area jump behavior, the data in the area is roughly classified and finely classified according to the eye movement behavior data. The hidden Markov model is a time series statistical model based on the hidden state Markov process. The ternary eye movement classification can be expressed as a first-order three-state hidden Markov model problem, the hidden state is fixation behavior, saccade behavior and smooth pursuit behavior, and the probability distribution of different eye movement behavior types is usually represented by a continuous Gaussian distribution. Fixation behavior and saccade behavior have very different position dispersion and speed characteristics, and the stable state of smooth pursuit usually contains part of saccade, which is not easy to distinguish. First, the eye movement characteristics are preprocessed, including the visual line landing point position, momentum, acceleration, the hidden state sequence that maximizes the joint probability is established, and the maximum expectation algorithm is used to solve the first layer hidden Markov model to perform rough classification of eye movement behavior, and the data is divided into saccade behavior and fixation behavior, wherein the fixation behavior still retains part of the smooth pursuit. Then, the eye movement behavior is finely classified, the maximum expectation algorithm is used to solve the second layer hidden Markov model, and the fixation behavior data is further divided into smooth pursuit and fixation behavior.

[0085] S33, extended field of view area data segment processing: the data in the extended field of view area includes fixation behavior and saccade behavior, and the data in the area is single classified according to the eye movement behavior data. The binary eye movement classification can be expressed as a first-order two-state hidden Markov model problem, the hidden state is fixation behavior and saccade behavior, and the probability distribution of different eye movement behavior types is usually represented by a continuous Gaussian distribution. The hidden Markov model is solved by using the maximum expectation algorithm, and the eye movement behavior data in the area is divided into fixation behavior and saccade behavior.

[0086] S34, invisible field of view area data segment processing: there is part of continuous eye movement behavior in the invisible field of view area, the behavior data is retained and no classification processing is performed.

[0087] In the above embodiment, the visual line calibration is performed by using the visible field of view range, the mapping error of the virtual field of view plane and the real calibration plane is eliminated, the field of view range is corrected by using the visual line landing point clustering, and the field of view areas of the visible field of view, the invisible field of view and the extended field of view are iteratively adjusted, so that the accurate field of view range can be obtained. The motion characteristics of the eye movement behavior in different field of view ranges are considered, and the eye movement behavior classification results are obtained by respectively performing targeted processing. The method solves the dynamic and accurate mapping problem of the field of view range in the telemetry scene, realizes the efficient eye movement behavior classification in different field of view areas, and has the characteristics of strong expansibility and high robustness.

[0088] In another embodiment, the application also provides a telemetry type eye movement behavior classification device for people with visual field defects, which performs visual line tracking based on a telemetry type visual line tracking device, and the classification device comprises:

[0089] A visual field region mapping module is configured to obtain the real visual field range of the visual field defect population through visual field measurement, and quantize the real visual field range as a virtual human eye visual field range parameter; simulate and generate a visible fixation point on a calibration plane, and iteratively adjust the visual field region mapping model; and use the calibration plane reference fixation point in the visible visual field range to establish mapping of the visual field region from the virtual visual field plane to the real calibration plane.

[0090] A gaze data correction module is configured to collect gaze data of the visual field defect population; use the visual field region mapping model to map the gaze data to the calibration plane, and calculate the gaze landing point; perform global region clustering on all gaze landing point data; perform position correction on the overall visual field region, and sequentially correct the invisible region and the extended region, and calculate the corrected gaze landing point data.

[0091] An eye movement behavior classification module is configured to divide the continuous gaze landing point trajectory into a visible visual field region data segment, an extended visual field region data segment and an invisible visual field region data segment by judging the gaze landing point region, and classify the eye movement behavior in different visual field regions.

[0092] The above-mentioned embodiment of the remote eye movement behavior classification device for the visual field defect population solves the dynamic and accurate mapping problem of the visual field range in the remote scene, realizes efficient eye movement behavior classification in different visual field regions, and has the characteristics of strong expansibility and high robustness.

[0093] For the remote eye movement behavior classification device of the embodiment of the present application, since it corresponds to the remote eye movement behavior classification method in the above embodiment, the description is relatively simple, and the relevant similar parts can be referred to the description of the remote eye movement behavior classification method part in the above embodiment. Therefore, it will not be described in detail here.

[0094] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A telemetry-based eye movement behavior classification method for people with visual field loss, characterized in that, The method comprises the following steps: S1, visual field area mapping: obtaining the real visual field range of the visual field defect population through visual field measurement, and quantifying it as a virtual human eye visual field range parameter; Simulating and generating visible fixation points on the calibration plane, and iteratively adjusting the visual field area mapping model; Using the calibration plane reference fixation points within the visible visual field range to establish the mapping of the visual field area from the virtual visual field plane to the real calibration plane; S2, gaze data correction: performing gaze tracking based on a remote gaze tracking device, and collecting gaze data of the visual field defect population; Using the visual field area mapping model, mapping the gaze data to the calibration plane to calculate the gaze landing point; Performing global regional clustering on all gaze landing point data; Performing position correction on the overall visual field area, and sequentially correcting the invisible area and the extended area to calculate the corrected gaze landing point data; S3, eye movement behavior classification: dividing the continuous gaze landing point trajectory into visible visual field area data segment, extended visual field area data segment and invisible visual field area data segment by judging the gaze landing point area, and classifying the eye movement behavior in different visual field areas; Wherein, the classification of eye movement behavior in different visual field areas comprises: In the visible visual field area data segment, multi-level hidden Markov model is used for coarse classification and fine classification of eye movement behavior data. First, the maximum likelihood algorithm is used to solve the data threshold parameter in the first layer hidden Markov model, and the data is divided into fixation behavior and saccade behavior with large momentum difference. Then, the maximum likelihood algorithm is used to solve the data threshold parameter in the second layer hidden Markov model, and the fixation behavior data is further divided into fixation behavior and smooth trailing behavior; In the extended visual field area data segment, single-level hidden Markov model is used for classification of eye movement behavior data. The threshold parameter of the extended visual field area data is solved by using hidden Markov model, and the data is divided into fixation behavior and saccade behavior; For the case of large extended visual field range, refer to the processing of the visible visual field area data segment, divide it into fixation behavior, saccade behavior and smooth trailing behavior; For the data segment connecting the visible area and the extended area, i.e. the landing point moving from the visible visual field area to the extended visual field area, it is separately regarded as a kind of eye movement behavior; In the invisible visual field area data segment, only the continuous motion data segment is marked, and the sparse data is ignored; Merge the eye movement behavior classification results of the visible visual field area, the extended visual field area and the invisible visual field area. Wherein, the gaze landing point area judgment comprises: Calculate the momentum and acceleration of the current gaze landing point. In the case of uniform sampling, ignore the time factor, the momentum value is the Euclidean distance between the current gaze landing point and the gaze landing point at the previous time, and the acceleration value is the difference between the current momentum and the previous momentum; According to the automatic standard of the gaze landing point area, the area label of the gaze landing point is determined, i.e. visible visual field area, invisible visual field area or extended visual field area; According to the continuous regional label result, the gaze landing point data is divided into different visual field area data segments, and the classification of the gaze landing point data is completed.

2. The method of claim 1, wherein the method is for a visual field defect population. The real visual field range of the visual field defect population is obtained by visual field measurement, and is quantified as a virtual human eye visual field range parameter, including: The visual field measurement result is converted into a gray quantization value in the visual field coordinate system, and the center of the visual field coordinate system is Euler zero degree angle; Gaussian filtering is used to smooth the visual field measurement result, and morphological erosion operation is used to reduce the boundary of the measurement result; The visible visual field area and the invisible visual field area are labeled, and the reduced visible visual field area and invisible visual field area have high confidence; If the visual field defect population wears an optical visual aid, it further includes: labeling the extended visual field area.

3. The method of claim 2, wherein the method is for a population with visual field defects. The visible fixation point is simulated and generated on the calibration plane, and the visual field area mapping model is iteratively adjusted, including: The distance from the visual field defect population to the calibration plane is measured, the center position of the human eye on the calibration plane is labeled, the projection mapping of the human eye visual field range on the calibration plane is calculated by using the trigonometric function relationship, and the area mark of the visual field coordinate system is converted into the area mark of the calibration plane coordinate; the center of the calibration plane coordinate system is the center position of the human eye on the calibration plane; The center point of the visible visual field area is calculated, and the distance from the point to the upper, lower, left and right boundaries is calculated, and the vertical and horizontal star rays of the point to the boundary are made, and the star ray center point is taken as the reference calibration point of the visible area; The visual field defect population adjusts the head and eyeball position until all the reference calibration points enter the visible visual field area; if there are multiple visible visual field areas in the center, the largest visible visual field area is selected; if the visible visual field areas are scattered, the center point of the visible visual field area is taken as the reference calibration point.

4. The method of claim 3, wherein the method is for a visual field defect population. The calibration plane reference fixation point in the visible visual field range is used to establish the mapping from the virtual visual field plane to the real calibration plane, including: The mapping relationship between the human eye features and the visible calibration point is established; The mapping relationship between the virtual visual field plane and the real calibration plane is established; The required transformation parameters are learned by using joint minimization re-projection error constraint, and are solved by using differential evolution method, and the line of sight landing point is mapped to the calibration plane.

5. The telemetry-based eye movement behavior classification method for visual field defect population according to claim 1, wherein, The line of sight data of the visual field defect population is collected, including: the human eye image of the visual field defect population is collected by using the human eye camera, and the pupil center position is located by pupil detection.

6. The method of claim 5, wherein the method is for a population with visual field defects. All the line of sight landing point data are globally regionally clustered, including: A density-based clustering method is used to cluster the line of sight landing points in the visible visual field area, filter low-density areas and find dense areas; According to the existing visual field mapping result, the clustering result is corresponded to different visual field areas; For the global line of sight landing point, the dense sample area should be located in the visible visual field area or the extended visual field area; the edge points of different clustering results are connected, the clustering result is converted into a clustering graph, and the areas with an area less than a threshold value are deleted.

7. The method of claim 6, wherein the method is for a population with visual field defects. The visual field area is corrected, including: The visible visual field area is taken as the reference, the clustering graph in the visible visual field area is intersected with the mapped visual field area, if the coincidence rate of the visual field area and the clustering graph is greater than 90%, it is judged that the visual field area does not need to be corrected, otherwise, the real value of the visible visual field area center on the calibration plane is manually labeled, the scale and translation parameters of the calibration plane are solved; The invisible region correction includes: generally ignoring the cluster graphics in the invisible region, if the cluster graphics are located at the boundary between the visible region and the invisible region, the invisible region is corrected by morphological erosion, and the invisible region is merged into the visible region; The extended region correction includes: intersecting the cluster graphics in the extended region with the mapped extended field of view region, if the coincidence rate of the extended field of view region and the cluster graphics in the region is greater than 90%, it is judged that the extended field of view region does not need to be corrected, otherwise, the range of the extended field of view region needs to be re-labeled; The line of sight landing point data correction includes: according to the scale and translation parameters, the line of sight landing point data in the visible field of view region and the extended field of view region are translated to new line of sight landing point positions.

8. A telemetry-based eye movement behavior classification device for people with visual field loss, characterized by, The line of sight tracking is performed based on a telemetry line of sight tracking device, and the classification device includes: A field of view region mapping module is configured to obtain the real field of view range of the field of view loss population by field of view measurement, and quantize the real field of view range into virtual human eye field of view range parameters; simulate and generate visible fixation points on a calibration plane, iteratively adjust the field of view region mapping model, and use the calibration plane reference fixation points in the visible field of view range to establish the mapping of the field of view region from the virtual field of view plane to the real calibration plane; A line of sight data correction module is configured to collect line of sight data of the field of view loss population, map the line of sight data to the calibration plane by using the field of view region mapping model, calculate the line of sight landing points, perform global region clustering on all line of sight landing point data, correct the position of the overall field of view region, and sequentially correct the invisible region and the extended region to calculate the corrected line of sight landing point data; An eye movement behavior classification module is configured to divide the continuous line of sight landing point trajectory into a visible field of view region data segment, an extended field of view region data segment and an invisible field of view region data segment by judging the line of sight landing point region, and classify the eye movement behaviors in different field of view regions; The classification of the eye movement behaviors in different field of view regions includes: In the visible field of view region data segment, a multi-level hidden Markov model is used for coarse classification and fine classification of the eye movement behavior data, first, a maximum likelihood algorithm is used to solve the data threshold parameter in the first layer hidden Markov model, and the data is divided into fixation behavior and saccade behavior with large momentum difference, and then a maximum likelihood algorithm is used to solve the data threshold parameter in the second layer hidden Markov model, and the fixation behavior data is further divided into fixation behavior and smooth trailing behavior; In the extended field of view region data segment, a single-level hidden Markov model is used for classification of the eye movement behavior data, and a hidden Markov model is used to solve the threshold parameter of the extended field of view region data, and the data is divided into fixation behavior and saccade behavior; For the case that the extended field of view range is large, the visible field of view region data segment is referred to, and the data is divided into fixation behavior, saccade behavior and smooth trailing behavior; for the data segment connecting the visible region and the extended region, i.e. the landing points moving from the visible field of view region to the extended field of view region, it is separately classified as a type of eye movement behavior; in the invisible field of view region data segment, only the continuous motion data segment is labeled, and the sparse data is ignored. The eye movement behavior classification results of the visible visual field region, the extended visual field region and the invisible visual field region are combined. The line-of-sight landing point region discrimination comprises: The motion momentum and acceleration of the current line-of-sight landing point are calculated; in the case of uniform sampling, the time factor is ignored, the motion momentum value is the Euclidean distance between the current line-of-sight landing point and the line-of-sight landing point at the previous moment, and the acceleration value is the difference between the motion momentum at the current moment and the motion momentum at the previous moment; According to the line-of-sight landing point region automatic standard, the region label of the line-of-sight landing point is obtained, i.e. the visible visual field region, the invisible visual field region or the extended visual field region; According to the continuous region label results, the line-of-sight landing point data is divided into data segments of different visual field regions, and the classification of the line-of-sight landing point data is completed.

Images