An industrial equipment interaction interface optimization method based on eye tracking

Abstract

The application discloses an industrial equipment interaction interface optimization method based on eye movement tracking and relates to the technical field of industrial man-machine interaction. The method comprises the following steps: collecting the gaze coordinates and the pupil diameter of an operator; detecting a micro-saccade event and obtaining a motion direction angle and an amplitude; determining an expected gaze direction of a safety-critical alarm region and calculating an included angle; calculating a deviation contribution value, updating a direction cumulative deviation amount according to an asymmetric attenuation rule; and when the cumulative amount continuously exceeds a threshold value and there is no micro-saccade in the direction, applying dynamic visual disturbance to an alarm icon. The application can improve the alarm ignoring problem caused by fatigue or attention fixation in an industrial monitoring scene by predicting the ignoring tendency before gaze shift by using the micro-saccade direction characteristics.

Description

Technical Field

[0001] This invention relates to the field of industrial human-computer interaction technology, and more specifically, to a method for optimizing the interactive interface of industrial equipment based on eye tracking. Background Technology

[0002] In the process industry, operators need to spend long periods of time monitoring the interface of distributed control systems or data acquisition and monitoring systems to monitor production operation status and respond to various alarm information. The monitoring interface typically has designated safety-critical alarm areas to display the highest priority alarm icons, such as emergency shutdown, over-temperature / over-pressure, and toxic gas leaks. When a hazardous condition triggers an alarm, the corresponding alarm icon changes color or flashes.

[0003] In real-world production environments, operators are prone to fatigue and a gradual decline in attention span when under prolonged low-load monitoring. Even if alarm icons are activated within their field of vision, operators may still fail to notice them in time—a phenomenon known in cognitive psychology as change blindness. Ignoring critical safety alarms can lead to operational delays and create safety hazards.

[0004] To address the aforementioned issues, existing methods have attempted to optimize monitoring interfaces using eye-tracking technology. One type of method assesses the interface layout by analyzing the operator's gaze heatmap, representing offline adjustment rather than real-time intervention. Another type of method highlights or flashes the alarm area after real-time monitoring indicates that the operator's gaze has not fallen on the alarm area for a certain period. A common characteristic of these methods is that they rely on gaze results for post-event triggering, meaning they can only respond after the operator has not gazed at the alarm area for an extended period, and cannot predict or proactively intervene during the formation of a neglect tendency. Therefore, an eye-tracking-based method for optimizing the user interface of industrial equipment is proposed to address the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an industrial equipment interactive interface optimization method based on eye tracking. This method addresses the problem that existing methods rely on gaze results for post-event triggering, and cannot predict and intervene in the tendency to ignore safety-critical alarms before operator ignoring behavior occurs.

[0006] To achieve the above objectives, the present invention provides the following technical solution: An eye-tracking-based method for optimizing the user interface of industrial equipment includes the following steps: S1: Real-time acquisition of the operator's gaze coordinates and pupil diameter on the industrial monitoring interface, wherein at least one safety-critical alarm area is defined on the industrial monitoring interface for displaying the highest priority alarm icon.

[0007] The collected gaze coordinates and pupil diameters provide raw data for subsequent microsaccade detection and operator alertness assessment.

[0008] S2: Detect micro-saccade events online from real-time collected gaze coordinates, and obtain the motion direction angle and motion amplitude of each micro-saccade.

[0009] The microsaccades are rapid, involuntary saccades of less than 0.5° produced by the operator's eyeballs during fixation. The direction of the microsaccades reflects the operator's implicit attentional shifting tendency; that is, before the operator actually moves their fixation point, their microsaccades will already be biased towards the direction in which their attention will be redirected. Based on this characteristic, by continuously monitoring the direction of the microsaccades, the operator's attentional tendency can be obtained before they actually shift their fixation, providing a basis for early prediction of neglect tendencies.

[0010] Furthermore, the method for online detection of microsaccade events in step S2 is as follows: using 6 times the median eye movement velocity within the sliding window as a dynamic threshold, eye movement segments with instantaneous eye movement velocity exceeding the dynamic threshold, duration exceeding 6ms, and movement amplitude exceeding 0.05° are identified as microsaccade events.

[0011] The dynamic threshold adapts to the statistical characteristics of the data within the sliding window, which can accommodate the differences in eye movement patterns among different operators and the changes in eye movement under different fatigue states of the same operator. It does not require a preset fixed threshold, thus improving the robustness and applicability of micro-saccade detection.

[0012] S3: For the safety-critical alarm area, take the average gaze coordinate within a preset time period before the micro-eye saccade event as the current gaze coordinate, take the direction from the current gaze coordinate to the center of the safety-critical alarm area as the desired gaze direction, and calculate the angle between the motion direction angle and the desired gaze direction.

[0013] This angle directly reflects the degree of deviation of the micro-eye saccade direction from the safety-critical alarm area. When the operator's micro-eye saccades continuously deviate in a certain direction and move away from the safety-critical alarm area, the angle increases, indicating that their implicit attention may be deviating from that area. This angle is the basis for subsequent quantification of attentional deviation.

[0014] S4: Calculate the deviation contribution value of each microsaccade, which is 0 when the movement direction angle of the microsaccade is consistent with the desired gaze direction, and increases as the angle increases.

[0015] The deviation contribution value is used to quantify the contribution of a single micro-saccade event to attentional deviation, converting the directional deviation of the micro-saccade into an accumulative numerical indicator. When the direction of the micro-saccade is exactly pointing towards the safety-critical alarm area, no deviation contribution is generated; when the direction of the micro-saccade is away from this area, a corresponding deviation contribution is generated, and the larger the deviation angle, the higher the contribution value.

[0016] Furthermore, the deviation contribution value mentioned in step S4 The calculation formula is:

[0017] in, The angle between the motion direction angle and the desired gaze direction. The amplitude of the movement, for The square root of.

[0018] This formula makes the deviation contribution value vary with the included angle. It exhibits non-linear growth, in When the size is small, the growth is gradual and it is tolerant of normal microsaccade directional tremors, avoiding excessive deviation contribution due to occasional eye movement fluctuations; exist Larger amplitude microsaccharides increase rapidly and contribute higher deviation values ​​to obvious directional deviations, ensuring that systematic attentional deviations can be effectively captured. Simultaneously, multiplying by the square root of the movement amplitude assigns higher weight to larger microsaccharides, as larger microsaccharides often indicate a stronger intention to shift attention.

[0019] S5: Update the cumulative deviation in direction according to the deviation contribution value. The update rule is as follows: when the included angle exceeds the deviation threshold, the current cumulative deviation in direction is attenuated at the first attenuation rate and then added to the current deviation contribution value; when the included angle does not exceed the deviation threshold, the current cumulative deviation in direction is attenuated separately at the second attenuation rate, where the second attenuation rate is greater than the first attenuation rate.

[0020] The cumulative directional deviation represents the degree to which operator attention systematically moves away from the safety-critical alarm area. In the update rule, moving towards the micro-saccade triggers a faster decay rate, causing the cumulative amount to be rapidly reduced, reflecting the trend of operator implicit attention re-approaching the safety-critical alarm area; moving away from the micro-saccade triggers a slower decay rate, causing the cumulative amount to continuously increase, reflecting the trend of operator implicit attention continuously moving away from the safety-critical alarm area. Therefore, the changing trend of the cumulative directional deviation can reflect the dynamic deviation of operator attention relative to the safety-critical alarm area in real time.

[0021] Further, the first attenuation rate in step S5 is denoted as The second decay rate is denoted as , and All are exponentially decaying factors and satisfy the following conditions: .

[0022] because much smaller The decay rate of directional cumulative deviation for directional micro-saccades is significantly greater than that for divergent micro-saccades, thus simultaneously achieving two objectives in the dynamic change of directional cumulative deviation: firstly, cumulative tracking of continuous divergent behavior, and secondly, rapid response to sudden directional behavior. This asymmetric design ensures that the directional cumulative deviation decreases rapidly when the operator's attention truly begins to return to the safety-critical alarm area, reducing unnecessary disturbance triggering.

[0023] Furthermore, step S5 also includes: performing linear fitting on the pupil diameter within a preset sliding time window to obtain the slope of the pupil diameter change over time; when the slope is negative, increasing... The value, The upper limit is 0.99.

[0024] Sustained pupil constriction is a physiological indicator of decreased operator alertness. When a trend of pupil constriction is detected, it suggests that the operator may have entered a state of fatigue or inattention. In this case, increasing pupil diameter is recommended. This slows down the decay rate of cumulative directional deviation, allowing deviation information to be retained longer under fatigue conditions, making it easier for the cumulative directional deviation to reach the accumulation threshold. This modulation mechanism correlates the operator's physiological state with the sensitivity of the prediction of neglect tendency, automatically increasing the prediction sensitivity when the operator's alertness decreases, thus enhancing the ability to prevent potential neglect risks.

[0025] S6: When the cumulative deviation of the direction exceeds the cumulative threshold, and the state is maintained for a preset time, and no micro-slap event with the included angle less than the orientation threshold occurs within the preset time, it is determined that the operator has a tendency to ignore the safety critical alarm area, and dynamic visual perturbation is applied to the alarm icon in the safety critical alarm area.

[0026] A sustained cumulative deviation exceeding a cumulative threshold indicates that the operator's attention has systematically drifted away from the critical safety alarm area, while the absence of directional micro-saccades within a preset time excludes the possibility of attention returning. When both conditions are met simultaneously, it suggests that the operator is highly likely to ignore critical alarm information within that area. Under these conditions, applying dynamic visual perturbation to the alarm icon can proactively capture the operator's peripheral visual attention, guiding their gaze to the critical safety alarm area and intervening in attention before the ignoring behavior occurs.

[0027] Furthermore, the dynamic visual perturbation in step S6 is as follows: during interface rendering, a periodic rotation and scaling offset controlled by polar coordinate angle and time is applied to the texture coordinates of the alarm icon. The perturbation amplitude is positively correlated with the cumulative deviation in the direction, so that the alarm icon presents a logarithmic spiral motion shape.

[0028] Logarithmic spiral motion comprises both rotational and radial scaling components. Compared to simple color changes or flashing, this motion pattern selectively activates motion-sensitive areas in the visual cortex, resulting in a strong bottom-up attention capture effect on peripheral vision. The perturbation amplitude adaptively varies with the cumulative deviation in direction, making the perturbation more intense when the tendency to ignore is stronger, effectively attracting attention while avoiding excessive interference with slight deviations.

[0029] Furthermore, the frequency range of the periodic rotation and scaling is 6Hz to 8Hz.

[0030] This frequency range corresponds to the preferred response band of motor-sensitive neurons in the visual cortex. Motion stimuli within this band can elicit a strong neural response with a low perceptual load, thereby achieving efficient capture of operator attention while minimizing visual fatigue.

[0031] Furthermore, the method also includes S7: continuously monitoring the operator's gaze coordinates after applying the dynamic visual disturbance; when a saccade is detected and the coordinates of the saccade's landing point are within the safety-critical alarm area, canceling the dynamic visual disturbance and resetting the cumulative directional deviation to 0, wherein the amplitude and speed of the saccade are both greater than the micro-saccade event.

[0032] A saccade is a behavioral indicator of an operator actively shifting their gaze, signifying that the operator's attention has been successfully guided to a safety-critical alarm area. Removing the disturbance at this point prevents the ongoing dynamic effects from interfering with the operator's subsequent information processing and operations. Simultaneously, it resets the accumulated directional deviation to zero, returning the system to its initial state and preparing for the next round of neglect-prone prediction.

[0033] Furthermore, the cancellation mentioned in S7 refers to linearly decaying the disturbance amplitude to 0 within 50ms.

[0034] Linear decay allows alarm icons to smoothly return from a motion state to a static display, avoiding operator discomfort or secondary distraction caused by the sudden disappearance of motion effects, and ensuring that operators can focus on reading the information and subsequent handling after noticing the alarm information.

[0035] Furthermore, the industrial monitoring interface is a distributed control system monitoring interface or a data acquisition and monitoring control system monitoring interface.

[0036] This type of interface is the primary interactive interface for operators in process industries to perform long-term monitoring. Operators need to continuously stare at the screen to monitor production status, which can easily lead to blind spotting due to fatigue or fixation of attention. This method, in monitoring scenarios with such interfaces, can effectively reduce the risk of unintentionally ignoring safety-critical alarms by real-time analysis of micro-saccade directions and timely intervention of dynamic visual disturbances.

[0037] The technical effects and advantages of this invention are as follows: First, this invention enables early prediction of neglect tendency by analyzing the directional characteristics of microsaccades. Microsaccades are involuntary, rapid eye movements that occur during fixation, and their direction is coupled with the operator's implicit attention shift. This invention calculates the deviation contribution value of each microsaccade and updates the cumulative directional deviation using an asymmetric decay rule: microsaccades moving towards the safety-critical alarm area have their cumulative value reduced rapidly at a larger decay rate, while microsaccades moving away from the alarm area have their cumulative value continuously increased at a smaller decay rate. This rule allows the cumulative directional deviation to reflect the dynamic deviation of attention relative to the alarm area in real time, providing a basis for predicting neglect tendency and shifting the intervention time from reactive to proactive.

[0038] Secondly, this invention employs dynamic visual perturbation using a logarithmic spiral motion pattern for attention guidance. When the cumulative directional deviation continuously exceeds a threshold and no directional micro-saccades occur, a periodic rotation and scaling offset, controlled by both polar coordinate angle and time, is applied to the texture coordinates of the alarm icon. The perturbation amplitude is positively correlated with the cumulative deviation. The logarithmic spiral motion effectively activates the motion-sensitive area of ​​the visual cortex, creating an attention capture effect on the peripheral visual field and guiding the operator to shift their gaze to the alarm area. After detecting that the operator has scanned towards the alarm area, the perturbation is smoothly withdrawn within 50ms, and the cumulative directional deviation is simultaneously reset, avoiding interference with subsequent operations.

[0039] Third, this invention utilizes the pupil diameter change trend to dynamically modulate the attenuation rate. Pupil constriction is a physiological manifestation of decreased operator alertness. When a negative slope of pupil diameter change is detected, the value of the first attenuation rate is increased to slow the attenuation of accumulated directional deviation, allowing deviation information to be retained longer under fatigue conditions and automatically improving the sensitivity of neglect tendency prediction. This modulation mechanism correlates the operator's physiological state with prediction parameters, helping to maintain effective monitoring of neglect risk at different alert levels. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the overall implementation of the method of the present invention.

[0041] Figure 2 This is a detailed flowchart of the neglect tendency prediction and disturbance triggering of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1 As attached Figures 1 to 2 The following is a method for optimizing the user interface of industrial equipment based on eye tracking, and its specific implementation details are as follows: This system is implemented based on the industrial monitoring interface currently viewed by the operator. The industrial monitoring interface can be a distributed control system monitoring interface or a data acquisition and monitoring control system monitoring interface. At least one safety-critical alarm zone is pre-defined on the interface by the system administrator using configuration tools. Each safety-critical alarm zone displays the highest priority alarm icon, such as an emergency shutdown alarm icon, a reactor over-temperature alarm icon, a pressure vessel over-pressure alarm icon, or a toxic gas leak alarm icon. Each safety-critical alarm zone is defined by its boundary vertex coordinates, and its geometric center coordinates serve as the center of the safety-critical alarm zone. The boundary vertex coordinates and center coordinates of the safety-critical alarm zone are configured during the system initialization phase.

[0044] The entire method is executed in the order of steps S1 to S7.

[0045] S1: Real-time acquisition of the operator's gaze coordinates and pupil diameter on the industrial monitoring interface.

[0046] The gaze coordinates and pupil diameter are acquired in real time using an infrared corneal reflective eye-tracking device positioned in front of the operator. The gaze coordinates are the two-dimensional visual coordinates of the operator's gaze points on the industrial monitoring interface, measured in degrees, representing the horizontal and vertical deflection angles at the intersection of the line of sight and the screen. The pupil diameter is the real-time measurement of the operator's pupils.

[0047] The selection of the acquisition frequency needs to balance the accuracy requirements of micro-eye saccade detection with the computational load of data processing. In one example, an acquisition frequency of 1000Hz is chosen, suitable for scenarios with high accuracy requirements for micro-eye saccade detection. In another example, an acquisition frequency of 500Hz is chosen, which still meets the detection requirements for micro-eye saccade events while reducing the real-time pressure on data processing, making it suitable for scenarios with limited computing resources in the processing module.

[0048] The collected gaze coordinates and pupil diameters are transmitted to the processing module in the subsequent steps in the form of a time series with timestamps. Each frame of data contains a gaze coordinate, a pupil diameter value, and a timestamp corresponding to the sampling time.

[0049] S2: Detect micro-saccade events online from real-time collected gaze coordinates, and obtain the motion direction angle and motion amplitude of each micro-saccade.

[0050] Microsaccades are tiny, rapid, involuntary saccades that occur in the operator's eyes during fixation, typically less than 0.5° in amplitude. The direction of microsaccades can reflect the operator's tendency to shift implicit attention.

[0051] When detecting micro-saccade events online, a sliding time window is first set. This sliding time window slides forward with each new frame of gaze coordinate data, ensuring that the window always contains gaze coordinate data within the most recent window length. The selection of the sliding time window length affects the smoothness of the dynamic threshold and the response speed to changes in eye movement.

[0052] In one example, the sliding time window is 500 ms long, containing more sampling points, resulting in a smoother median velocity estimate, suitable for monitoring scenarios where eye movement patterns are relatively stable. In another example, the sliding time window is 200 ms long, with data within the window more closely reflecting the current eye movement state, making it more sensitive to changes in eye movement velocity.

[0053] In the initial stage of the method, when the duration of collected gaze coordinate data is less than a window length, a fixed velocity threshold of 30° / s is used for micro-saccade detection. 30° / s is the typical upper limit of eye movement velocity under normal gaze conditions, which can effectively distinguish micro-saccades from gaze drift at this stage. After the accumulated data reaches a window length, the method switches to a dynamic threshold method based on the middle value of the sliding window.

[0054] For the gaze coordinate sequence within the sliding time window, each pair of adjacent gaze coordinates is processed sequentially. The Euclidean distance between the gaze coordinates of adjacent frames is used as the displacement, and the time interval between the two frames is used as the time difference. Dividing the displacement by the time difference yields the instantaneous eye movement velocity value corresponding to that adjacent frame. The same calculation is performed on all adjacent frame pairs within the window to obtain an instantaneous eye movement velocity sequence. The median of this sequence is taken, and multiplied by 6 to obtain a dynamic threshold. The dynamic threshold adaptively changes with the statistical characteristics of the data within the window, eliminating the need to set individual threshold parameters for different operators or different fatigue states.

[0055] Subsequently, sampling points where the instantaneous eye movement velocity exceeds the dynamic threshold are marked as candidate points, and temporally consecutive candidate points are grouped into a single candidate segment. When the duration of a candidate segment exceeds 6 ms and the displacement amplitude from the start to the end of the segment exceeds 0.05°, the candidate segment is identified as a micro-saccade event. The start time of this micro-saccade event is the timestamp of the first frame sampling point of the candidate segment, and the end time is the timestamp of the last frame sampling point. 6 ms is the minimum duration of a micro-saccade, and 0.05° is the minimum amplitude of a micro-saccade; these two conditions together effectively distinguish micro-saccades from measurement noise in eye-tracking devices.

[0056] In another example, the duration threshold is set to 8ms and the displacement amplitude threshold is set to 0.1°, which is suitable for eye-tracking devices with high noise levels.

[0057] For the identified micro-saccade events, extract their motion amplitude. And the direction and angle of motion. Amplitude of motion. The distance from the starting point to the ending point of the micro-saccade event is expressed in Euclidean degrees. The motion direction angle is the angle between the direction vector of the micro-saccade event from the starting point to the ending point and the horizontal reference axis of the industrial monitoring interface, with counterclockwise as the positive direction.

[0058] S3: For safety-critical alarm areas, the direction from the current gaze coordinates to the center of the safety-critical alarm area will be taken as the desired gaze direction.

[0059] To obtain a stable gaze position at the moment of a microsaccade event, all gaze coordinate sampling points acquired by S1 within 100 ms prior to the start of the microsaccade event are collected. The 100 ms time window length strikes a balance between smoothing gaze coordinate fluctuations and maintaining real-time position. The arithmetic mean of the x-coordinates and y-coordinates of these sampling points is taken, and the resulting average coordinate point is used as the current gaze coordinate.

[0060] In another example, the median of the x and y coordinates of the gaze coordinate sampling points within the time window is used to form the current gaze coordinates, in order to reduce the impact of individual outlier sampling points on the current position estimation.

[0061] For each safety-critical alarm zone, a direction vector is calculated from the current gaze coordinates to the center of the safety-critical alarm zone. The angle between this direction vector and the horizontal reference axis of the industrial monitoring interface is the desired gaze direction, indicating the gaze shift direction that the operator should take when focusing their attention on the safety-critical alarm zone.

[0062] The angle between the microsaccade direction angle and the desired fixation direction is denoted as . . The calculation method is as follows: take the absolute value of the difference between the two. If the absolute value exceeds 180°, then subtract the absolute value from 360° as the result. ,thus The value range is from 0° to 180°. When When it is 0°, it means that the direction of the microsaccade movement is completely consistent with the desired fixation direction; when When the angle is 180°, it means that the direction of the microsaccade movement is completely opposite to the desired gaze direction.

[0063] When there are multiple safety-critical alarm areas on the industrial monitoring interface, the desired gaze direction is determined for each safety-critical alarm area, and the corresponding included angle is calculated. .

[0064] S4: Calculate the deviation contribution value for each microsaccade. .

[0065] Deviation from contribution value The formula used to quantify the deviation between the direction of the microsaccade movement and the desired gaze direction is as follows: ,in The included angle is determined in S3. The amplitude of motion extracted from S2. for The square root of.

[0066] This formula makes Follow The increase is non-linear: when When smaller The growth is gradual and it is tolerant of normal microsaccade directional tremors; when When larger Rapid growth contributes a high deviation value to obvious directional divergences.

[0067] Multiply The larger the amplitude of microsaccades, the more it deviates from the contribution value. It occupies a larger weight.

[0068] S5: Based on deviation contribution value Update direction cumulative deviation .

[0069] Cumulative deviation of direction This reflects the cumulative degree to which the operator's implicit attention has systematically drifted away from the target safety-critical alarm area. This parameter is continuously updated throughout the entire method's operation, unaffected by ignore bias decisions or applied disturbances. (Cumulative directional deviation) The initial value is set to 0.

[0070] The system operates according to a fixed time step. Update accordingly. In one example, this time step is 10ms. A 10ms time step ensures that... The update frequency is sufficient to reflect changes in micro-saccade events without introducing excessive computational overhead. Within each time step, all detected micro-saccade events are counted. If at least one micro-saccade event is detected within that time step, an event-driven update is performed sequentially for each micro-saccade event, with the update rules described below. If no micro-saccade event is detected within this time step, time-driven decay is performed, which accumulates the deviation in the current direction. Multiply by the first decay rate ,Right now .

[0071] Before updating, first set the bias threshold. This is used to distinguish whether the microsaccade direction deviates from or moves towards the desired gaze direction. In one example, the deviation threshold... A 30° angle is suitable for general scenarios where the safety-critical alarm area is of moderate size. In another example, the angle is biased towards the threshold. A 20° angle is suitable for scenarios where the safety-critical alarm area is small and more sensitive to attention deviation. In another example, the bias threshold... A 40° angle is suitable for scenarios where the safety-critical alarm area is large and a certain directional tolerance is permissible.

[0072] Simultaneously set the first decay rate Second decay rate In one example, Take 0.95, Taking 0.60, the cumulative directional deviation under this value combination The accumulation is relatively stable, and the orientation reset is relatively rapid, making it suitable for scenarios where high accuracy in predicting the neglect tendency is required. In another example, Take 0.90, Taking 0.50, the cumulative directional deviation under this value combination The overall range of change is larger, and the response to fluctuations in attention is more sensitive. and satisfy . The closer it is to 1, the greater the cumulative deviation in direction. The more it is retained with each decay; The smaller the value, the greater the cumulative deviation in direction. The decay is faster when moving toward microsaccades.

[0073] The rules for event-driven updates are as follows: For each pending micro-saccade event, check its included angle. .

[0074] When the included angle Exceeding the bias threshold At that time, the cumulative deviation of the current direction will be calculated. Multiply by the first decay rate In addition, the current deviation contribution value of micro-saccades The updated cumulative deviation of direction is obtained. ,Right now .

[0075] When the included angle Not exceeding the bias threshold At that time, the cumulative deviation of the current direction will be calculated. Multiply by the second decay rate The updated cumulative deviation of direction is obtained. ,Right now .

[0076] because much smaller Micro-saccades directed toward the desired gaze direction will cause a cumulative deviation in gaze. Decay at a faster rate. If there are multiple micro-saccade events within this time step, update the timer sequentially using each event according to the rules described above. The previous event after the update The value serves as input for the next event update.

[0077] In a preferred example, this step also includes using pupil diameter data to determine the first decay rate. Dynamic modulation is performed. Specifically, a sliding time window of 1 second is set, and all pupil diameter sampling points acquired by S1 within this time window are linearly fitted to obtain the slope of the pupil diameter change over time. .when A negative value indicates that the operator's pupils are constricting and alertness is decreasing. At this time, [the following action should be taken]. Increase to ,in The preset modulation coefficients, The upper limit is 0.99. When When it is a non-negative value, Keep the original settings unchanged. In one example... The initial value is 0.95. The value of is such that when the operator is in a typical state of fatigue and the pupil constriction slope is... When a significantly negative value is reached, The increase is approximately 0.04, that is... It was modulated to approximately 0.99.

[0078] S6: Ignore biased judgments and the application of dynamic visual perturbations.

[0079] First, set the cumulative threshold. Preset duration and toward the threshold Three decision parameters.

[0080] Cumulative threshold Cumulative deviation in direction The threshold for judgment. In one example, the cumulative threshold. This is determined during the system calibration phase. The calibration process involves: during the initial system deployment, operators performing simulated monitoring tasks under normal conditions, recording the cumulative directional deviation throughout the process. The cumulative threshold is determined by taking a value below which 95% of the data points on the curve are lower than a given threshold value. This method can determine personalized cumulative thresholds based on the individual eye movement characteristics of different operators.

[0081] Preset duration Cumulative deviation in direction Exceeding the cumulative threshold consecutively The minimum duration to be maintained. In one example, the preset duration. A duration of 200ms effectively filters out momentary threshold exceedances caused by occasional eye movement fluctuations, ensuring that deviations only occur due to accumulated directional deviations. The perturbation is triggered only when the value remains consistently high. In another example, a preset duration is used. A time limit of 150ms is suitable for scenarios where a fast response time is required to address the tendency to ignore certain factors.

[0082] Towards the threshold This is used to identify whether the direction of microsaccade movement is close enough to the desired gaze direction. In one example, the direction towards the threshold... Using a 10° angle, the orientation judgment is relatively strict, only considering it an orientation when the micro-saccade direction is very close to the desired gaze direction. In another example, the orientation threshold... Using 15° provides a relatively lenient approach to determining orientation.

[0083] When the cumulative deviation of direction Exceeding the cumulative threshold At that time, a timer is started. During the timing process, the angle of each subsequently detected micro-saccade event is determined. Is it less than the threshold? .

[0084] If the timer reaches the preset duration And no angle was detected within the preset time period. Less than the threshold If a micro-eye jump event occurs, it is determined that the operator has a tendency to ignore the safety-critical alarm area, and dynamic visual perturbation is applied to the alarm icon in the safety-critical alarm area.

[0085] If the preset time is reached... Angle detected previously Less than the threshold In the event of a micro-eye jump, the timer immediately resets to zero, waiting for the accumulated directional deviation. Exceeding the cumulative threshold again Then restart the timer.

[0086] Dynamic visual perturbation is achieved by applying real-time transformations to the texture coordinates of the alarm icon during the interface rendering phase. In the graphics processor shader program, for each pixel covered by the alarm icon, let its original texture coordinates before perturbation be... The texture coordinates of the center of the alarm icon are The radius of this pixel relative to the center of the icon is... The polar angle is .in, The distance from that pixel to the center of the icon is determined by... Calculated; The direction angle of the pixel relative to the center of the icon is determined by the relative coordinates of the pixel and the center of the icon.

[0087] when When, that is, when the pixel is the center point of the icon, let The value is set to 0 to avoid the logarithmic function being undefined. The texture coordinates of the pixel after perturbation. Determine by the following formula: ; ; in, , The frequency of periodic rotation and scaling; The spiral tightness coefficient is a preset constant value. Polar radius The natural logarithm, when hour Set to 0; The current time; The amplitude of the disturbance.

[0088] In this transformation, the phase component of the offset is determined by time. Polar diameter The natural logarithm and polar angle Together, the magnitude of the offset is determined to be the disturbance amplitude. Because the phase contains In this type of device, pixels at different radii have different phase shifts at the same time, causing each pixel on the alarm icon to have different displacements at different times. The overall device exhibits a logarithmic spiral motion around the center of the icon, that is, it performs rotational motion and radial scaling motion simultaneously.

[0089] Frequency of periodic rotation and scaling Selected within the range of 6Hz to 8Hz. This frequency range corresponds to the frequency band in the visual cortex where motion-sensitive neurons respond most strongly, providing optimal automatic attention capture for the operator's peripheral visual field. In one example, 7Hz is chosen, which is the center value of this band. Spiral tightness coefficient. Take a preset constant value, in one example Take 1.0.

[0090] Disturbance amplitude Cumulative deviation from the current direction Proportional, that is , This is the proportionality coefficient. The size of the alarm icon is determined based on its dimensions and the desired maximum disturbance amplitude. The alarm icon size is measured in degrees (view coordinates). In one example, the width of the alarm icon is set to... Degree, expected cumulative deviation in direction Reaching the cumulative threshold When the value is twice the size of the alarm icon, the disturbance amplitude reaches 10% of the alarm icon width. This value ensures that the disturbance amplitude varies within a reasonable range.

[0091] In the first rendering frame where a tendency to ignore is determined, the accumulated deviation is calculated in the current direction. Calculate the initial disturbance amplitude The alarm icon then begins to move according to the aforementioned transformation. In each subsequent rendered frame, the deviation accumulates with the direction. Real-time changes, disturbance amplitude Synchronized updates.

[0092] When there are multiple safety-critical alarm zones on the industrial monitoring interface, each safety-critical alarm zone independently maintains its own cumulative directional deviation. Each safety-critical alarm area is then individually ignored. When a specific safety-critical alarm area meets the triggering condition, dynamic visual perturbation is applied only to the alarm icon within that area; the directional cumulative deviation of the remaining safety-critical alarm areas continues to be maintained independently. When dynamic visual perturbation is applied to multiple safety-critical alarm areas simultaneously, the perturbation of each area is rendered independently without interference, and the perturbation amplitude of each area is determined solely by its own directional cumulative deviation. Decide.

[0093] S7: After applying a dynamic visual disturbance, continuously monitor the operator's gaze coordinates. When a scan is detected and the coordinates of the scan's landing point are within a safety-critical alarm area, cancel the dynamic visual disturbance and accumulate the directional deviation. Reset to 0.

[0094] Salivation is an eye movement behavior that occurs when an operator actively shifts their gaze; its amplitude and speed are significantly greater than microsaccades. The criteria for determining saccades are: the instantaneous eye movement speed exceeds a preset speed threshold, and the amplitude of the eye movement segment exceeds a preset amplitude threshold. In one example, the preset speed threshold is 30° / s, and the preset amplitude threshold is 0.5°; this set of thresholds can effectively distinguish between saccades and microsaccades. In another example, the preset speed threshold is 40° / s, and the preset amplitude threshold is 1°, which is suitable for scenarios where the operator's overall eye movement amplitude is relatively large.

[0095] After detecting a scan event that meets the criteria, it is determined whether the coordinates of the scan's landing point are located within any safety-critical alarm area where dynamic visual disturbances have been applied. If it is located within an area, the disturbance undoing operation for that area is executed at the beginning of the next rendering frame.

[0096] The disturbance is removed by gradually reducing the disturbance amplitude to zero over a preset smooth decay period. In one example, a linear decay method is used, with a decay period of 50ms. (Based on the current disturbance amplitude...) Starting with this, in each subsequent rendered frame, the amount of perturbation to be reduced in that frame is calculated based on the frame interval. Specifically, the reduction amount in that frame is equal to... Multiply by the frame interval time and divide by 50ms to subtract the reduction in frame rate from the current disturbance amplitude until the disturbance amplitude is 0. Linear attenuation ensures that the movement amplitude of the alarm icon decreases uniformly, resulting in a smooth and natural recovery process.

[0097] In another example, exponential decay is used. In each rendered frame, the current disturbance amplitude is multiplied by a preset decay coefficient, which is less than 1. This causes the disturbance amplitude to decrease rapidly at first and then slowly, with the decay process essentially completed within approximately 80ms. Exponential decay can quickly reduce the disturbance amplitude in the initial stage, allowing the alarm icon to stabilize rapidly, making it suitable for scenarios with high recovery speed requirements.

[0098] While removing the disturbance, the cumulative deviation in direction will be recorded. Reset directly to 0. When multiple safety-critical alarm areas exist on the industrial monitoring interface, ensure that redirection detection and disturbance removal are performed independently for each safety-critical alarm area with applied dynamic visual disturbance. When the scan point falls within a safety-critical alarm area, only remove the disturbance for that safety-critical alarm area and accumulate its directional deviation. Resetting to 0 will not affect other safety-critical alarm areas.

[0099] Application scenario examples To facilitate understanding of the overall execution process of the method of this invention, the following description is based on a typical industrial monitoring application scenario.

[0100] In a chemical production monitoring room, an operator is faced with a distributed control system monitoring interface. At the top of the interface is a reactor over-temperature alarm icon, located within a preset safety-critical alarm zone. The operator is fatigued after prolonged monitoring, their gaze fixed on the production process display area in the center of the interface. At this moment, the reactor over-temperature alarm is triggered, and the alarm icon changes color at the top of the interface. The operator then experiences a series of micro-saccades, these movements generally veering downwards and to the left of the interface, systematically moving away from the direction of the alarm icon.

[0101] The S1 collects the operator's gaze coordinates and pupil diameter in real time at a frequency of 1000Hz, with gaze coordinates in degrees.

[0102] S2 detects these micro-saccade events from the gaze coordinates, with a sliding time window length of 500ms, a duration threshold of 6ms, and a displacement amplitude threshold of 0.05°, and obtains the motion direction angle and motion amplitude of each micro-saccade. .

[0103] S3 takes the average gaze coordinates within 100ms before the start of each micro-saccade as the current gaze coordinates, calculates the expected gaze direction from the current gaze coordinates to the center of the alarm area, and obtains the included angle corresponding to each micro-saccade. .

[0104] S4 calculates the deviation contribution value for each microsaccade. Due to the angle between the direction of microsaccades and the desired fixation direction. Larger, deviating significantly from the contribution value The level remains high.

[0105] S5 accumulates the directional deviation in 10ms time steps. Update Take 0.95, Take 0.60, Take 30°. Within each time step, if a micro-saccade event occurs and the angle is... Exceed Then press each micro-eye twitch in sequence. renew Cumulative deviation in direction Gradually rising; If there is no microtwitching event, then according to Attenuation. Due to operator fatigue, the slope of the change in pupil diameter... It is negative and significant. The modulation increased to 0.99. The decline has slowed further, and the overall trend remains upward.

[0106] When the cumulative deviation of direction Exceeding the cumulative threshold Then, the S6 starts its timer. This is the statistical upper limit value specified for this operator. It takes 200ms. The angle was 10°. Within a preset duration of 200ms, the system continuously monitored subsequent micro-saccade events and detected no angle. For micro-saccades of less than 10°, the system determines that the operator tends to ignore the alarm area. It then applies dynamic visual perturbations to the alarm icon via the graphics processor shader program, with a frequency of... 7Hz, spiral tightness coefficient The disturbance amplitude is 1.0. Cumulative deviation from the current direction Proportional, proportionality coefficient Based on icon width and Determine as in the example above.

[0107] After the operator's peripheral vision perceives the logarithmic spiral motion disturbance of the alarm icon, their attention is automatically captured, resulting in a saccade towards the alarm icon. S7 detects this saccade event; the instantaneous eye movement speed of the saccade exceeds 30° / s and the movement amplitude exceeds 0.5°, and the saccade's landing point is within the boundary of the alarm area. The system determines that the operator's attention has been redirected.

[0108] The system then linearly decayed the disturbance amplitude to 0 within 50ms, the alarm icon returned to its normal static display, and the accumulated directional deviation was recorded. Reset to 0. The operator successfully noticed the alarm message and began taking appropriate action.

[0109] The above description is merely a specific embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing the user interface of industrial equipment based on eye tracking, characterized in that, Includes the following steps: S1: Real-time acquisition of the operator's gaze coordinates and pupil diameter on the industrial monitoring interface, wherein at least one safety-critical alarm area is defined on the industrial monitoring interface for displaying the highest priority alarm icon. S2: Detect micro-saccade events online from the real-time collected gaze coordinates, and obtain the motion direction angle and motion amplitude of each micro-saccade. The micro-saccade event is a rapid saccade with an amplitude of less than 0.5° generated by the operator during gaze. S3: For the safety-critical alarm area, take the average gaze coordinates within a preset time period before the micro-eye saccade event as the current gaze coordinates, take the direction from the current gaze coordinates to the center of the safety-critical alarm area as the desired gaze direction, and calculate the angle between the motion direction angle and the desired gaze direction. S4: Calculate the deviation contribution value for each microsaccade, wherein the deviation contribution value is 0 when the movement direction angle of the microsaccade is consistent with the desired gaze direction, and increases as the angle increases; S5: Update the cumulative deviation in direction according to the deviation contribution value. The update rule is: when the included angle exceeds the deviation threshold, the cumulative deviation in the current direction is attenuated at the first attenuation rate and then added to the current deviation contribution value. When the included angle does not exceed the deviation threshold, the cumulative deviation in the current direction is attenuated separately according to the second attenuation rate, where the second attenuation rate is greater than the first attenuation rate; S6: When the cumulative deviation of the direction exceeds the cumulative threshold, and the state is maintained for a preset time, and no micro-slap event with the included angle less than the orientation threshold occurs within the preset time, it is determined that the operator has a tendency to ignore the safety critical alarm area, and dynamic visual perturbation is applied to the alarm icon in the safety critical alarm area.

2. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 1, characterized in that, The deviation contribution value mentioned in step S4 The calculation formula is: in, The angle between the motion direction angle and the desired gaze direction. The amplitude of the movement, for The square root of.

3. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 1, characterized in that, In step S5, the first attenuation rate is denoted as The second decay rate is denoted as , and All are exponentially decaying factors and satisfy the following conditions: .

4. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 3, characterized in that, Step S5 further includes: performing linear fitting on the pupil diameter within a preset sliding time window to obtain the slope of the pupil diameter change over time; when the slope is negative, increasing... The value, The upper limit is 0.

99.

5. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 1, characterized in that, The dynamic visual perturbation mentioned in step S6 is as follows: during interface rendering, a periodic rotation and scaling offset controlled by polar coordinate angle and time is applied to the texture coordinates of the alarm icon. The perturbation amplitude is positively correlated with the cumulative deviation in the direction, so that the alarm icon presents a logarithmic spiral motion shape.

6. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 5, characterized in that, The frequency range of the periodic rotation and scaling is 6Hz to 8Hz.

7. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 1, characterized in that, Also includes S7: After the dynamic visual disturbance is applied, the operator's gaze coordinates are continuously monitored. When a saccade is detected and the coordinates of the saccade's landing point are within the safety-critical alarm area, the dynamic visual disturbance is canceled and the cumulative directional deviation is reset to 0. The amplitude and speed of the saccade are both greater than the micro-saccade event.

8. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 7, characterized in that, The cancellation mentioned in S7 refers to linearly decaying the disturbance amplitude to 0 within 50ms.

9. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 1, characterized in that, The method for detecting microsaccade events online in step S2 is as follows: using 6 times the median eye movement velocity within the sliding window as a dynamic threshold, eye movement segments with instantaneous eye movement velocity exceeding the dynamic threshold, duration exceeding 6ms, and movement amplitude exceeding 0.05° are identified as microsaccade events.

10. The method for optimizing the interactive interface of industrial equipment based on eye tracking according to claim 1, characterized in that, The industrial monitoring interface is either a distributed control system monitoring interface or a data acquisition and monitoring control system monitoring interface.

Images