Attention detection method and device, equipment, storage medium and program product

Abstract

The invention discloses an attention detection method, device and equipment, a storage medium and a program product. The method comprises the steps that when it is detected that a scene condition meets an attention detection condition, an eyeball tracking function is started; displaying a stimulus based on the scene condition, where the stimulus is used to guide a user's gaze; eye information of a user is obtained, and the eye information is collected based on the eyeball tracking device; the attention state of the user is determined based on the eye information, the problem of resource waste caused by continuous attention detection is solved, whether the attention detection condition is met or not is judged by detecting the scene condition, if yes, attention detection is started, the attention of the user needs to be continuously detected, and the user experience is improved. The attention detection only needs to be started when the attention detection condition is met, data processing resources are saved, and effective detection of the attention state is achieved; the user is guided through display stimulation, and the accuracy of the attention detection result is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of eye tracking, and in particular to an attention detection method, device, equipment, storage medium and program product. BACKGROUND

[0002] In many fields, the monitoring of attention is widely used. Especially in the field of vehicle driving, it is necessary to detect the driving state of the user in time, and to prompt in time when the user is in fatigue driving or inattentive, so as to avoid danger and the like. Due to the development of eye tracking technology, many detections of fatigue and attention of users gradually adopt schemes of analyzing the fixation point position and eye features.

[0003] At present, the analysis of the state of the user based on eye tracking is usually uninterrupted analysis of the eye image of the user, but real-time tracking of the fixation point and analysis of the state of the user require real-time calculation of a large number of eye images, and the calculation amount and consumption are large. Moreover, the process of the user from attention concentration to inattention is relatively long, and real-time tracking calculation also causes a lot of waste of calculation. SUMMARY

[0004] The present application provides an attention detection method, device, equipment, storage medium and program product to solve the problem of resource waste caused by continuous attention detection.

[0005] According to an aspect of the present application, an attention detection method is provided, comprising:

[0006] When it is detected that the scene condition meets the attention detection condition, the eye tracking function is started;

[0007] The stimulus is displayed based on the scene condition, wherein the stimulus is used to guide the user to fixate;

[0008] The eye information of the user is acquired, wherein the eye information is collected based on the eye tracking device;

[0009] The attention state of the user is determined based on the eye information.

[0010] According to another aspect of the present application, an attention detection device is provided, comprising:

[0011] The stimulus display module is configured to start the eye tracking function when it is detected that the scene condition meets the attention detection condition;

[0012] The stimulus display module is configured to display the stimulus based on the scene condition, wherein the stimulus is used to guide the user to fixate;

[0013] The eye information acquisition module is configured to acquire the eye information of the user, wherein the eye information is collected based on the eye tracking device;

[0014] An attention analysis module is used to determine the user's attention state based on the eye information.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor, and a memory communicatively connected to said at least one processor;

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the attention detection method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the attention detection method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the attention detection method according to any embodiment of the present invention.

[0020] The technical solution of this invention enables eye-tracking when the scene conditions meet the attention detection criteria; displays stimuli based on the scene conditions, wherein the stimuli are used to guide the user's gaze; acquires the user's eye information, wherein the eye information is collected based on the eye-tracking device; and determines the user's attention state based on the eye information. This solves the problem of resource waste caused by continuous attention detection. By detecting the scene conditions to determine whether the attention detection criteria are met, if so, the eye-tracking device is activated, and stimuli are displayed according to the scene conditions to guide the user's gaze. The eye-tracking device collects the user's eye information after the stimuli appear and analyzes the user's attention state. The attention detection method provided by this application does not require continuous detection of the user's attention; it only needs to be activated when the attention detection criteria are met, saving data processing resources and achieving effective detection of attention state. By guiding the user through displayed stimuli, the accuracy of attention detection results can be improved. Furthermore, since the stimuli are generated based on various different situations within the driving scenario, the display of stimuli does not need to be preset and can be directly integrated with various display functions in the cockpit without adding additional stimuli display burden. This integrates attention detection into the driving scenario, which is beneficial to ensuring the timeliness of attention detection and thus ensuring driving safety.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an attention detection method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of an attention detection method provided according to Embodiment 2 of the present invention;

[0025] Figure 3 This is a schematic diagram of a stimulus display provided according to Embodiment 2 of the present invention;

[0026] Figure 4 This is a schematic diagram of another stimulus provided according to Embodiment 2 of the present invention;

[0027] Figure 5 This is a schematic diagram of another stimulus provided according to Embodiment 2 of the present invention;

[0028] Figure 6 This is a schematic diagram of another stimulus provided according to Embodiment 2 of the present invention;

[0029] Figure 7a This is a schematic diagram of another stimulus provided according to Embodiment 2 of the present invention;

[0030] Figure 7b This is a schematic diagram of another stimulus provided according to Embodiment 2 of the present invention;

[0031] Figure 8 This is a schematic diagram of another stimulus provided according to Embodiment 2 of the present invention;

[0032] Figure 9 This is a schematic diagram of an attention detection device according to Embodiment 3 of the present invention;

[0033] Figure 10 This is a schematic diagram of the structure of an electronic device that implements the attention detection method of this invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1 This is a flowchart of an attention detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations involving attention detection. The method can be executed by an attention detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0038] S101. When the scene conditions meet the attention detection criteria, enable the eye-tracking function.

[0039] The attention detection method provided in this application can be applied to scenarios such as vehicle driving, airport tower monitoring, monitoring pilot status during flight, monitoring rooms in factories that require real-time monitoring of large equipment data, or passenger flow monitoring rooms in high-speed rail stations and train stations.

[0040] In this embodiment, the scene situation can be understood as the changes that occur both outside and inside the device when the user is driving a vehicle, an airplane, or using monitoring equipment. Scene situations can be divided into two categories: changing and unchanged. Changing scenes include the addition, reduction, or positional change of targets, while unchanged scenes include no target and no change with a target. By analyzing the scene situation, it can be determined whether the environment requiring the user's attention has changed; the occurrence of scene situations is uncertain. Attention detection conditions can be understood as the conditions used to determine whether attention detection should be performed, such as a sudden change in vehicle speed, the sudden appearance of other vehicles in front of the vehicle, or flashing of indicator lights on the vehicle's dashboard. The eye-tracking device can be used to calculate the user's gaze point position and can also acquire eye feature data based on collected eye images for attention situation analysis and calculation, thereby achieving the purpose of attention detection. In other words, when the scene situation meets the attention detection conditions, the eye-tracking function is activated to acquire user eye information for attention detection analysis.

[0041] The system detects scene conditions using image acquisition devices and sensors, and analyzes whether the scene conditions meet the attention detection criteria. If they do, the eye-tracking function is activated to detect the attention state; otherwise, attention state detection is not required.

[0042] S102. Display stimuli based on the scene, wherein the stimuli are used to guide the user's gaze.

[0043] In this embodiment, the stimulus can be a graphic, animation, etc., such as a circle, a rectangle, an animated character, etc., used to guide the user's attention. The location requiring the user's concentration may differ in different scenarios. The intelligent functions equipped in the smart cockpit analyze the scenario, such as the HUD display function, intelligent alarms, and intelligent prompts, and different intelligent prompts will appear. Generally, the appearance of an intelligent prompt indicates that the user needs to pay attention to the prompt content. Therefore, the content to be displayed in the intelligent prompt can be defaulted to the stimulus to be displayed; that is, the stimulus is determined based on the intelligent prompts during driving. Finally, the location where the stimulus is displayed is the location that the user needs to pay attention to in the current scenario, and the stimulus is displayed accordingly.

[0044] S103. Obtain the user's eye information, wherein the eye information is collected based on an eye-tracking device.

[0045] In this embodiment, eye information can be understood as information describing the condition of the user's eyes. After activation, the eye-tracking device can periodically collect the user's eye information at certain time intervals. The eye-tracking device can be fixed in front of the user in a position that does not obstruct the user's line of sight, or it can be worn by the user on their head, for example, a wearable eye tracker.

[0046] S104. Determine the user's attention state based on eye information.

[0047] In this embodiment, attention state is used to indicate whether the user's attention is focused, and can be focused / unfocused, distracted / focused, etc. Eye information is analyzed to determine changes and states in the user's eyes. Based on these changes, the user's attention state is determined. When analyzing the user's attention state, one method can be used to analyze eye information to determine the attention state; or, multiple methods can be used to analyze eye information, obtaining data on different types of eye states and changes, thereby determining the analysis results for each method. Finally, the analysis results of multiple attention states are combined to determine the final attention state. For example, if more than half of the user's attention states are found to be unfocused, the user is determined to be unfocused; or, if at least one attention state is found to be unfocused, the user is determined to be unfocused.

[0048] This invention provides an attention detection method. When the scene conditions meet the attention detection criteria, an eye-tracking function is activated. Stimuli are displayed based on the scene conditions, whereby the stimuli guide the user's gaze. The user's eye information is acquired, whereby the eye information is collected using the eye-tracking device. The user's attention state is determined based on the eye information. This method solves the problem of resource waste caused by continuous attention detection. By detecting the scene conditions, it determines whether the attention detection criteria are met. If they are, the eye-tracking device is activated, and stimuli are displayed according to the scene conditions to guide the user's gaze. The eye-tracking device collects the user's eye information and analyzes the user's attention state. The attention detection method provided in this application does not require continuous detection of the user's attention; it only needs to activate attention detection when the attention detection criteria are met, saving data processing resources and achieving effective detection of attention state. Guiding the user by displaying stimuli improves the accuracy of attention detection results. Moreover, the stimuli are determined based on intelligent driving prompts, and the display of stimuli does not need to be preset according to rules or conditions, increasing the randomness of stimulus display. Because the displayed stimuli are integrated with intelligent prompts during driving, the timing of the displayed stimuli also indicates when the user needs to increase their attention, making the timing of attention detection more appropriate and helping to ensure the user's driving safety.

[0049] Example 2

[0050] Figure 2 This is a flowchart of an attention detection method provided in Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiments. Figure 2As shown, the method includes:

[0051] S201. When the scene conditions meet the attention detection criteria, enable the eye-tracking function.

[0052] Optionally, the scene conditions that satisfy the attention detection criteria may include, but are not limited to, at least one of the following:

[0053] The degree of change of the target objects around the equipment meets the preset conditions;

[0054] The equipment parameters meet the preset thresholds;

[0055] The device is displaying an alarm message.

[0056] In this embodiment, the target object can be a vehicle, pedestrian, etc.; the device can be a vehicle, aircraft, etc., and the types of devices and target objects may differ in different attention detection scenarios. Preset conditions can be changes in the position or quantity of the target object; preset thresholds can be understood as values ​​used to determine whether the device parameters are normal, and different types of device parameters may correspond to different preset thresholds. Alarm prompts can be provided through light prompts, sound prompts, text prompts, icon prompts, etc.

[0057] Preset preset conditions, detect target objects around the device, determine the degree of change of target objects based on the position and number of target objects at different times, and judge whether the degree of change of target objects meets the preset conditions. If it does, it can be determined that the attention detection conditions are met.

[0058] Pre-set thresholds for different types of parameters. Different application scenarios and devices have different accuracy requirements and impacts on attention detection. The preset thresholds can also be set according to the application scenario and device type. Device parameters, such as speed and acceleration, are collected in real time. The collected device parameters are compared with the corresponding preset thresholds. If the parameters are not within the range required by the preset thresholds, the attention detection conditions are determined to be met.

[0059] Check if the detection equipment issues any alarms, such as low fuel level or tire pressure warnings. If the equipment issues an alarm, the attention detection conditions are met.

[0060] If a scenario meets one or more of the above conditions, it can be considered to meet the attention detection criteria.

[0061] Optionally, the degree of change of the target objects around the device meets at least one of the following preset conditions:

[0062] The number and relative position of the target objects remain unchanged within the preset time period;

[0063] Changes in the number of targets for the target object;

[0064] The relative position of the target object has changed.

[0065] In this embodiment, the target quantity can be understood as the number of target objects; the preset duration can be set in advance, for example, it can be determined by analyzing historical data, or it can be customized by the user, etc.

[0066] The system detects and identifies objects around the equipment, determining their location and quantity. It then determines whether the quantity of these objects changes within a preset time period, and whether their relative positions to the equipment change. If one or more of the following occurs within the preset time period: the quantity and relative position of the objects remain unchanged, or the quantity and relative position change: the degree of change in the objects around the equipment is considered to meet the preset conditions.

[0067] Optionally, the device parameters must meet at least one of the preset thresholds, including:

[0068] The speed change of the equipment exceeds the preset change threshold;

[0069] The device's steering angle exceeds the threshold.

[0070] The device has shifted at an angle exceeding the threshold.

[0071] The frequency of changes in the device's speed information exceeds a preset frequency threshold;

[0072] The oil level in the equipment is below the oil level threshold.

[0073] In this embodiment, the change threshold can be understood as a threshold for speed changes, which can be preset based on the impact of speed changes on safety; the speed information change frequency can be understood as the frequency of speed information changes, such as the frequency of acceleration changes, the frequency of speed changes, etc.; the frequency threshold can be understood as a threshold for the change frequency. When the speed information change frequency includes multiple types of change frequencies, the corresponding frequency thresholds can be different. For example, the frequency threshold corresponding to the frequency of acceleration changes is different from the frequency threshold corresponding to the frequency of speed changes. The fuel level threshold can be preset. Some devices will also alarm when the fuel level is below a certain threshold, such as the fuel level alarm of a vehicle. The fuel level threshold in this application also uses the fuel level alarm threshold of the device itself, etc.

[0074] The system uses sensors to monitor real-time information such as speed, steering angle, offset angle, acceleration, and fuel level. Based on the speed values ​​at different times, it determines the speed change value and compares it with a preset threshold. If the speed change value exceeds the threshold, it indicates a sudden acceleration or braking of the device, and the device parameters are considered to meet the preset threshold. A steering angle threshold is determined. This threshold can be preset to a fixed value or updated in real-time based on road conditions. For example, on straight roads, the steering angle threshold may not exceed 40°, and on left-turn roads, it may not exceed 50°, etc. To distinguish between left and right turns, the steering angle thresholds can be indicated by positive and negative signs. The steering angle is compared with the corresponding threshold. If the steering angle exceeds the threshold, the current steering is deemed unreasonable, and the device parameters are considered to meet the preset threshold. Finally, the offset angle is compared with the corresponding threshold. If the offset angle exceeds the threshold, the vehicle is deemed to have deviated, and the device parameters are considered to meet the preset threshold. Analyze velocity and acceleration information at different times to determine the frequency of velocity changes. This frequency can be the frequency of velocity changes or the frequency of acceleration changes. Compare the velocity change frequency with a corresponding frequency threshold. If the velocity change frequency exceeds the threshold, the equipment speed is unstable, and the equipment parameters can be considered to meet the preset threshold. Compare the oil level with a oil level threshold. If the oil level is less than the threshold, the oil level is insufficient, and the equipment parameters can be considered to meet the preset threshold.

[0075] S202. Determine the display method and display position of the stimuli based on the scene.

[0076] In this embodiment, the display method can be to display stimuli through animation, images, graphics, etc., and may also include the size of the stimulus, etc. Generally, the display method of the stimulus is consistent with the display method of the intelligent prompt; the display position can be represented in the form of coordinates, and the display position can correspond to the center point of the stimulus or any point of the stimulus, and the display position should also be consistent with the display position of the intelligent prompt. Analyze the scene situation to determine the current position that needs attention. For example, if the scene situation involves a change in the number of target objects, the position that needs attention can be the changed target, such as the position of the added target object, or the position of the remaining target object when the target object is reduced, etc. After determining the position that needs attention, this position can be used as the display position, or the display position can be set near this position. The display method can be preset. For example, different display methods can be set for different scenarios. The display methods corresponding to each scenario are stored in a data table in advance. When selecting the display method, factors such as time can also be considered. For example, a rectangular box can be displayed as a stimulus during the day, while a brightly colored animation can be displayed at night to attract attention more easily. Alternatively, the display method can be divided into time periods, and different display methods can be selected for different time periods. In this way, a more obvious display method can be selected to display stimuli when people are prone to drowsiness. Therefore, the display method can also be selected based on the current time.

[0077] The display method and location of stimuli are determined based on the scenario, including determining the intelligent prompt content based on the scenario, and determining the display method and content of stimuli based on the intelligent prompt content. Intelligent prompt content refers to the intelligent functions within the smart cockpit analyzing and judging the scenario in a driving scenario, and determining the intelligent prompt content based on the scenario requiring prompts. Intelligent prompt content may include, but is not limited to, displaying a flashing frame on a target object, flashing a fuel level indicator light, etc. Therefore, the display location and display method of the intelligent prompt content are the same as the display method and display location of the stimuli.

[0078] Of course, in some cases, the driver's driving status can also be captured by the camera inside the vehicle. The camera inside the vehicle can be a regular scene camera or an infrared camera set in an eye-tracking device. It can capture the user's driving status in a timely manner. If the image recognition technology identifies a problem with the user's driving status, such as the user exhibiting fatigue postures such as frequent body swaying or frequent head nodding, a smart prompt can be triggered at any time to display a stimulus and activate the eye-tracking function to detect the user's eye information when the stimulus is displayed, in order to determine the user's distraction.

[0079] S203. Display the corresponding stimulus at the display position according to the display method.

[0080] The corresponding stimulus is generated based on the display method, and then displayed at the appropriate display position. For example, the display method is a rectangle, and the display position is (x, y). In this case, the rectangle is displayed, and the center point of the rectangle is (x, y). Of course, the display method and display position can also be preset, and the stimulus can be displayed directly at the preset display position and preset display method based on the analysis and judgment of the scene.

[0081] S204. Obtain the user's eye information, wherein the eye information is collected based on an eye-tracking device.

[0082] Optionally, the eye information includes at least one of the following: fixation point position, horizontal difference between binocular fixations, pupil information, saccade information, blink duration, and orbital information.

[0083] S205. Determine the user's attention state based on eye information.

[0084] Analyzing eye information includes pupil information such as left and right pupil deviation, pupil size, and pupil size variability; and orbital information such as whether the user's eyes are closed and the distance between the upper and lower eyelids. Analyzing eye information over a period of time can determine saccade information, blink duration, etc. Saccade information can be expressed as saccade frequency, and blink duration as blink frequency. Based on eye information, it's possible to determine whether the user is focused and their attentional state. For example, the location of the fixation point can be used to determine if the user is not fixating on a stimulus, thus identifying a distracted state.

[0085] Optionally, eye information includes the horizontal difference in gaze level between the two eyes. Based on this eye information, the user's attentional state is determined, including:

[0086] Determine whether the horizontal position difference between the two fixation points is within a first preset range; wherein, the horizontal position difference between the two fixation points is the absolute value of the difference between the coordinates of the left eye fixation point and the coordinates of the right eye fixation point on the horizontal axis;

[0087] If so, determine that the user's attention state is distracted.

[0088] A first preset range is set in advance. It is determined whether the horizontal position difference between the fixation points of the two eyes is within the first preset range. If so, the user's attention state is determined to be a distracted state; otherwise, the user's attention can be considered to be focused.

[0089] Optionally, eye information includes saccade information, which includes at least one of saccade frequency, saccade duration, variability in saccade duration, and variability in maximum saccade rate. Determining the user's attention state based on eye information includes:

[0090] A user's attention state is determined to be distracted when at least one of the following conditions is met:

[0091] The number of eye saccades is within the second preset range;

[0092] The duration of eye saccades is within the third preset range;

[0093] The variability in saccade duration is within the fourth preset range;

[0094] The maximum variability of eye saccade rate is within the fifth preset range.

[0095] A second, third, fourth, and fifth preset range are pre-set. These ranges can be established by analyzing sample data. The user's attentional state is determined to be distracted if at least one of the following conditions is met: the number of saccades is within the second preset range; the duration of saccades is within the third preset range; the variability in the duration of saccades is within the fourth preset range; and the variability in the maximum rate of saccades is within the fifth preset range. Otherwise, the user's attention can be considered focused.

[0096] Alternatively, you can set the user's attention state to be a distracted state when all four conditions above are met, or when at least two conditions are met.

[0097] Optionally, eye information includes blink duration, and the user's attention state is determined based on this eye information, including:

[0098] Determine if the blink duration is within the sixth preset range;

[0099] If so, determine that the user's attention state is distracted.

[0100] A sixth preset range is set in advance. It is determined whether the blinking time is within the sixth preset range. If so, the user's attention state is determined to be a distracted state; otherwise, the user's attention can be considered to be focused.

[0101] Optionally, when the eye information includes pupil information, and the pupil information includes pupil size and / or pupil size variability, determining the user's attention state based on the eye information includes:

[0102] A user's attention state is determined to be distracted when at least one of the following conditions is met:

[0103] Pupil size is within the seventh preset range;

[0104] Pupil size variability is within the eighth preset range.

[0105] A seventh and eighth preset range are pre-set, which can be determined by analyzing sample data. The user's attention state is determined to be distracted if at least one of the following conditions is met: pupil size is within the seventh preset range, or pupil size variability is within the eighth preset range; otherwise, the user's attention is considered focused.

[0106] Alternatively, you can set the user's attention state to be considered distracted when both of the above conditions are met.

[0107] Compared to a focused state, in a distracted state, there are differences in indicators such as the horizontal difference of binocular fixation points, the number of saccades, the duration of saccades, the variability of saccade duration, the variability of maximum saccade rate, the average number of blinks, the longest blink duration, pupil diameter, and pupil diameter variability. The corresponding indicators for each type of data (i.e., the corresponding preset range) can be calculated in advance within a certain time window, for example, a time window set to 10 seconds and a sample size of 1000.

[0108] 1. Horizontal Gazing Difference (Gaze_difference_X): The absolute difference between the coordinates of the left and right eye fixations on the X-axis. Gaze_difference_X = |Gaze_point_left_X – Gaze_point_Right_X|. Compared to a focused state, the horizontal gazing difference is larger when the mind wanders. After collecting and calculating a large amount of data, the values ​​are as follows: P50 is 35.89px, P75 is 57.91px, P95 is 112.41px, and the average is 77.31px; while in a distracted state, P50 is 52.66px, P75 is 83.46px, P95 is 200.50px, and the average is 77.31px. Based on the data obtained above, a first preset range was set.

[0109] 2. Saccade Counts: Calculates the number of saccades that occur within a time window. Fewer saccades occur during a distracted state compared to a focused state. A second preset range is set based on the data obtained above.

[0110] 3. Saccade Duration: Calculate the average saccade duration for all saccade behaviors within the time window. Saccade duration is longer in a distracted state compared to a focused state. Based on the data obtained above, set a third preset range.

[0111] 4. Saccade Duration Standard Variation: Saccade duration variation refers to the duration of all saccade behaviors within a time window. The standard deviation of saccade duration is calculated. Compared to the state of focused attention, the variability of saccade duration is greater in the state of inattention. Based on the data obtained above, a fourth preset range is set.

[0112] 5. Saccade Velocity Peak Standard Variation: Saccade velocity peak standard variation refers to the maximum rate of all saccade behaviors within a time window, calculating the standard deviation of all maximum saccade rates. Compared to a focused state, the variability of saccade velocity peak standard variation is greater in a distracted state. Based on the data obtained above, a fifth preset range is set.

[0113] 6. Blink Duration: Calculates the average duration of all blinks within a time window. Blink duration is longer when the mind is wandering compared to a focused state. When focused, the average blink duration is 127.40 ms, P50 is 116.50 ms, and P95 is 224.75 ms; when wandering, the average blink duration is 156.67 ms, P50 is 172.00 ms, and P95 is 335 ms. Based on the above data, a sixth preset range is set.

[0114] 7. Pupil diameter: The pupil diameter is larger when the mind is wandering compared to when attention is focused. Calculate the pupil diameter when the mind is wandering, and set the seventh preset range based on the obtained data.

[0115] 8. Pupil diameter variability: Pupil diameter variability is the standard deviation of all valid pupil diameter data samples within the time window. The variability of pupil diameter is greater in a distracted state compared to a focused state. An eighth preset range is set based on the data obtained from statistical analysis.

[0116] In this embodiment of the application, the preset ranges (e.g., the first preset range, ..., the eighth preset range) can be determined based on the normal distribution formed by the collected data. Eye information falls within a preset range, which can be p50, p70, p95, etc. Alternatively, a range can be directly preset.

[0117] Optionally, eye information includes the fixation point location. Based on this eye information, the user's attention state is determined, including steps A1-A2:

[0118] A1. Obtain the display range of the stimulus.

[0119] After the stimulus is displayed, the display range of the stimulus can be determined accordingly. The display range of the stimulus can be determined directly based on the display method and display position of the stimulus, or it can be determined based on the coordinate position covered by the stimulus after the stimulus is displayed.

[0120] A2. Determine whether the gaze point is within the display range of the stimulus within a preset time. If so, determine that the user's attention state is a distracted state.

[0121] In this embodiment, the preset time can be set according to factors such as the device's movement speed and safety requirements. After the stimulus appears, it is determined whether the gaze point is within the display range of the stimulus within the preset time. If so, it can be determined that the user's gaze point has not moved towards the stimulus location after the stimulus appears. Generally, it can be assumed that the user's attention is not focused and cannot detect the change in time, thus determining that the user's attention state is a distracted state.

[0122] Optionally, eye information includes the fixation point location. Based on this eye information, the user's attention state is determined, including steps B1-B7:

[0123] B1. Based on the appearance of the stimulus, obtain the initial position information of the fixation point and the initial fixation time.

[0124] In this embodiment, the initial fixation point position information can be coordinate information, used to describe the initial fixation point position information; the initial fixation time is used to describe the time when the fixation stimulus begins.

[0125] When a stimulus occurs, the time of stimulus occurrence is taken as the initial fixation time, and the user's fixation point location at this time is taken as the initial fixation point location information. The fixation point location can be obtained based on an eye-tracking device.

[0126] B2. Obtain the display range of the stimulus.

[0127] B3. Based on the fact that the gaze point falls within the display range of the stimulus, obtain the gaze point termination position information and gaze termination time.

[0128] In this embodiment, the fixation point termination position information can be coordinate information, used to describe the position when the fixation point falls within the display range of the stimulus; the fixation termination time is used to describe the time it takes to fixate on the stimulus.

[0129] Based on the position of the fixation point, determine whether the fixation point falls within the display range of the stimulus. When the fixation point falls within the display range of the stimulus, obtain the time when the fixation point falls within the display range of the stimulus and the position of the fixation point at this time. Determine this position as the fixation point termination position information and use this time as the fixation termination time.

[0130] B4. Determine the fixation point movement time based on the fixation initiation time and fixation termination time.

[0131] In this embodiment, fixation point movement time can be understood as the time it takes for the fixation point to move from its initial position to the location of the stimulus after the stimulus appears. The fixation point movement time is obtained by subtracting the fixation initiation time and fixation termination time.

[0132] B5. Determine the distance the gaze point moves based on the initial and final positions of the gaze point.

[0133] In this embodiment, the fixation point movement distance can be understood as the distance the fixation point moves from its initial position to the location of the stimulus after the stimulus appears. The fixation point movement distance is calculated using the initial fixation point position information and the final fixation point position information, combined with the distance calculation formula.

[0134] B6. Determine the movement speed based on the distance the gaze point moves and the time the gaze point moves.

[0135] The movement speed is obtained by dividing the distance the gaze point moves by the time the gaze point moves from one position to another.

[0136] B7. When the moving speed is less than the preset speed threshold, the attention state of the object to be detected is determined to be a distracted state.

[0137] In this embodiment, the preset speed threshold can be set in advance. The preset speed threshold can be determined through experiments, simulations, or based on experience. By comparing the moving speed with the preset speed threshold, when the moving speed is less than the preset speed threshold, it is determined that the user's reaction speed is slow and the user's attention state is distracted. The more focused the user's attention is, the faster the reaction should be.

[0138] Optionally, eye information includes the fixation point location. Based on this eye information, the user's attention state is determined, including steps C1-C5:

[0139] C1. Obtain the initial fixation time based on the appearance of the stimulus.

[0140] C2. Once the fixation point falls within the display range of the stimulus, determine the fixation termination time.

[0141] C3. Obtain the positions of all gaze points within the gaze initiation and gaze termination times.

[0142] While collecting the user's gaze point position, the collection time is recorded. Based on the gaze initiation time and gaze termination time, the collection time of the gaze point position is filtered, and all time points within the gaze initiation time and gaze termination time are retained. The gaze point position collected at each time point is then determined.

[0143] C4. Determine the degree of dispersion when the fixation point moves based on the position of each fixation point.

[0144] Analyze the positions of each fixation point to determine whether there is dispersion or drift during the movement of the fixation point, and then determine the degree of dispersion when the fixation point moves. For example, by fitting the positions of each fixation point, determine whether the path of the fixation point movement is a straight line. Determine the degree of dispersion based on the equation or line segment obtained from the fitting. Generally, the more focused the attention, the closer the fixation point should be to a straight line when it moves, or the smaller the degree of drift.

[0145] C5. If the degree of distraction is high, the user's attention state is determined to be a distracted state.

[0146] Optionally, when the eye information includes pupil deviation information, the user's attention state is determined based on the eye information, including steps D1-D2:

[0147] D1. Determine the offset of the user's two pupils based on the pupil offset information.

[0148] Based on the identification of pupil position in the collected user eye images, the system determines whether the user's left and right pupils have shifted according to pupil offset information. For example, the left pupil is shifted to the left, while the right pupil is not shifted.

[0149] D2. If the deviation of the user's two pupils is inconsistent, the user's attention state is determined to be a distracted state.

[0150] Compare whether the deviation of the two pupils is consistent. When attention is focused, the pupils of both eyes should be coordinated, for example, both deviating to the left or right. When attention is distracted, the deviation of the two pupils may be inconsistent, for example, one pupil deviates to the left while the other does not. If the deviation of the two pupils is inconsistent, the user's attention state is determined to be a distracted state.

[0151] This invention provides an attention detection method that solves the problem of resource waste caused by continuous attention detection. It analyzes the scene from multiple aspects, including the number and relative position of target objects, device parameters, and alarm prompts, to determine whether attention detection conditions are met. If met, an eye-tracking device is activated to promptly detect the user's attention state, avoiding omissions. Appropriate display positions and display methods are selected based on the scene conditions to display stimuli, guiding the user's gaze and improving the detection of their attention state. The method can analyze the user's attention state from multiple angles, including fixation point position, horizontal difference between binocular fixations, pupil information, saccade information, blink duration, and orbital information, achieving accurate attention state analysis. The attention detection method provided in this application does not require continuous monitoring of the user's attention; it only needs to activate attention detection when the conditions are met, saving data processing resources and achieving effective attention state detection. Guiding the user through displayed stimuli improves the accuracy of attention detection results.

[0152] For example, the embodiments of this application illustrate the display of stimuli in different scenarios. Taking a vehicle driving on a highway as an example, the scenario can be divided into scenarios that are basically unchanged and scenarios that change.

[0153] 1. The scene remains basically unchanged.

[0154] 1) No goal, no change

[0155] If, within a preset time period, the image acquisition device captures a scene in front of the subject where there is no target (moving vehicle, pedestrian, etc.) and the scene shows no significant change, then a stimulus is automatically generated. For example, Figure 3 A schematic diagram of stimulus display is provided, using stimulus 31 as an example, such as... Figure 3 As shown, stimuli can appear on road signs; stimuli can also appear on a prominent logo on the side. Stimuli can be colored marker boxes that attract user attention.

[0156] 2) Keep your goals in place.

[0157] If, within a predetermined time period, the image acquisition device captures only one (or more) target objects (moving vehicles) in the frame, and their relative positions remain largely unchanged, then a stimulus is actively generated. The stimulus appears on the captured target object (if there is only one target object, it is displayed on that object; if there are multiple target objects, the stimulus can be displayed on the target object closest to the user). For example, Figure 4 Another stimulus display diagram is provided, such as Figure 4 As shown, stimulus 31 is displayed on a target object in front.

[0158] 2. Scene Changes

[0159] 1) Increase in target objects:

[0160] When a new target object (nearby vehicles) enters the scene, a stimulus is displayed on the new target object. For example, Figure 5 An alternative stimulus display diagram is provided, where the number of target objects is increased from 1 to 2, and the stimulus is displayed on the newly appearing target objects.

[0161] 2) Reduction of target objects

[0162] When the number of target objects in the image decreases, the stimulus is displayed on the remaining target objects. If there are multiple remaining target objects, one can be selected to display the stimulus, for example, the most recent target object. For example, Figure 6 An alternative stimulus display diagram is provided, in which the number of target objects is reduced from 2 to 1, and the stimulus is displayed on the remaining target objects.

[0163] 3) Changes in the relative position of the target object and the equipment

[0164] When the position of the target object in the image changes significantly from the device (i.e., the vehicle itself), such as when the user changes lanes or overtakes, or when the target object's speed decreases, the positions of the target and the user will change. The user can select the target object currently closest to them to display stimuli, or multiple target objects can be selected simultaneously. For example, Figure 7a Another illustration of stimulus display is provided, where the relative positions of the target objects change, and the stimulus can be displayed on the nearest target object. For example, Figure 7b An alternative stimulus display diagram is provided, in which the relative positions of the target objects change, and the stimulus can be displayed on multiple target objects.

[0165] in, Figures 3-7b This can be a stimulus display within a HUD (Head-Up Display) scenario, or the stimulus can be displayed inside the vehicle, for example, on the vehicle's dashboard or screens. For example, Figure 8 An alternative stimulus display diagram is provided, showing stimulus 31 on the vehicle's dashboard. Figure 8 The triangle with an exclamation mark is stimulus 31.

[0166] When performing attention state detection, it may be necessary to determine eye data under different states to assist in the detection of attention state. This application embodiment also provides methods for obtaining various eye data when the user's attention is focused, distracted, and in a natural state.

[0167] 1. Experimental setup

[0168] The experimental setup is used to test the eye characteristics of users when faced with sudden stimuli, whether they are in a distracted, focused, or natural state.

[0169] The display screen shows images to provide visual stimulation. The standard stimulus image is a 7x5 arrangement of green dots, and the biased stimulus image is also a 7x5 arrangement of dots, containing 34 green dots and 1 red dot, with the red dot appearing at 35 locations. Alternatively, the standard stimulus image is a green dot, and the biased stimulus image is a red dot. The diameter of the dots is approximately 1°, and the dot array appears within a horizontal range of 60° (-30° to +30°) and a vertical range of 55° (-30° to +25°) of visual field.

[0170] A camera that takes pictures containing eyes;

[0171] A loudspeaker provides auditory stimulation and is used to play voice prompts;

[0172] Buttons are used for user confirmation of operations, such as button [1] and button [0];

[0173] The computing chip receives images and analyzes information such as eye saccades, blinks, eye sockets, fixation point position, and pupil deflection in different states of the person's eyes, generating corresponding eye feature data.

[0174] 2. Work Process

[0175] Before each experimental task begins, experimental instructions will be displayed on the screen. After the test subject fully understands the instructions, the practice program will begin. Each subject will complete 20 practice trials before the formal experiment begins. The probability of the biased stimulus appearing during the practice program is 20%.

[0176] The first stage is a state of focused attention, which requires the test users to concentrate when completing the task.

[0177] The loudspeaker indicates that the test is currently in a focused state.

[0178] The monitor randomly displays standard stimulus images and deviated stimulus images. The probability of a standard stimulus appearing is 95%, and the probability of a deviated stimulus appearing is 5%. Each stimulus lasts for 1500 ms, and there is a 50 ms interval between two stimuli (during which time the screen displays white), for a total of 100 times. The 100 times can be divided into 4 blocks, with each block forming a class. One class has a higher probability of deviated stimuli appearing, and the other class has a lower probability of deviated stimuli appearing. The total probability of deviated stimuli appearing in both classes is 5%.

[0179] The test user presses the confirmation button when a deviating stimulus image is seen based on the content of the stimulus image [1];

[0180] The camera captures images of the user's eyes and sends them to the computing chip;

[0181] After 10 trials, a probe appears randomly at a frequency of 10%. The loudspeaker prompts, "Have you been distracted before this? If yes, press [1]; otherwise, press [0]."

[0182] When the user being tested realizes that they are being distracted, press [1].

[0183] The second stage is the distracted state, which requires the test users to actively think about things unrelated to the current task while completing the task.

[0184] The loudspeaker indicates that the test is currently in a distraction state.

[0185] The monitor randomly displays standard stimulus images and deviated stimulus images. The probability of a standard stimulus appearing is 95%, and the probability of a deviated stimulus appearing is 5%. Each stimulus lasts for 1500 ms, and there is a 50 ms interval between two stimuli (during which time the screen displays white), for a total of 100 times. The 100 times can be divided into 4 blocks, with each block forming a class. One class has a higher probability of deviated stimuli appearing, and the other class has a lower probability of deviated stimuli appearing. The total probability of deviated stimuli appearing in both classes is 5%.

[0186] The test user presses the confirmation button when a deviating stimulus image is seen based on the content of the stimulus image [1];

[0187] The image acquisition device captures images of the user's eyes and sends them to the computing chip;

[0188] After 10 trials, a probe appears randomly at a frequency of 10%. The loudspeaker prompts, "Have you been distracted before this? If yes, press [1]; otherwise, press [0]."

[0189] When the subject realized that he or she was being distracted, he or she pressed [1].

[0190] The third stage is the natural state.

[0191] The loudspeaker indicates that the test is currently in a natural state.

[0192] The monitor randomly displays standard stimulus images and deviated stimulus images. The probability of a standard stimulus appearing is 95%, and the probability of a deviated stimulus appearing is 5%. Each stimulus lasts for 1500 ms, and there is a 50 ms interval between two stimuli (during which time the screen displays white), for a total of 800 times. These 800 times can be divided into 8 blocks, with each block consisting of 4 blocks as a class. One class has a higher probability of deviated stimuli appearing, and the other class has a lower probability of deviated stimuli appearing. The total probability of deviated stimuli appearing in both classes is 5%, and the probability is balanced in the ABBA format.

[0193] The test user presses the confirmation button when a deviating stimulus image is seen based on the content of the stimulus image [1];

[0194] The image acquisition device captures images of the user's eyes and sends them to the computing chip;

[0195] After 10 trials, a probe appears randomly at a frequency of 10%. The loudspeaker prompts, "Have you been distracted before this? If yes, press [1]; otherwise, press [0]."

[0196] When the user being tested realizes that they are being distracted, press [1].

[0197] If the user being tested needs a break, they can press [R] to pause the test.

[0198] The computing chip analyzes the user's eye characteristics in three states: distracted, focused, or natural. These include the speed (V) of the user's gaze point movement, the degree of dispersion during gaze point movement, the position of the two pupils, saccades, blinks, and the condition of the eye sockets.

[0199] Example 3

[0200] Figure 9 This is a schematic diagram of an attention detection device provided in Embodiment 3 of the present invention. Figure 9 As shown, the device includes: a stimulus display module 41, a stimulus display module 42, an eye information acquisition module 43, and an attention analysis module 44.

[0201] Among them, the stimulus display module 41 is used to enable the eye tracking function when the scene situation is detected to meet the attention detection conditions;

[0202] Stimulus display module 42 is used to display stimuli based on scene conditions, wherein the stimuli are used to guide the user's gaze;

[0203] The eye information acquisition module 43 is used to acquire the user's eye information, wherein the eye information is acquired based on the eye tracking device;

[0204] Attention analysis module 44 is used to determine the user's attention state based on the eye information.

[0205] This invention provides an attention detection device that solves the problem of resource waste caused by continuous attention detection. It determines whether attention detection conditions are met by detecting the scene conditions. If so, it activates an eye-tracking device, displays stimuli according to the scene conditions, guides the user's gaze through the stimuli, collects the user's eye information through the eye-tracking device, and analyzes the user's attention state. The attention detection method provided in this application does not require continuous detection of the user's attention; it only needs to activate attention detection when the conditions are met, saving data processing resources and achieving effective detection of attention state. Guiding the user through displayed stimuli improves the accuracy of attention detection results.

[0206] Optionally, the scene conditions that satisfy the attention detection criteria include at least one of the following:

[0207] The degree of change of the target objects around the equipment meets the preset conditions;

[0208] The equipment parameters meet the preset thresholds;

[0209] The device is displaying an alarm message.

[0210] Optionally, the degree of change of the target object around the device satisfies at least one of the following preset conditions:

[0211] The number and relative position of the target objects remain unchanged within the preset time period;

[0212] Changes in the number of targets for the target object;

[0213] The relative position of the target object has changed.

[0214] Optionally, the device parameters must meet at least one of the preset thresholds, including:

[0215] The speed change of the equipment exceeds the preset change threshold;

[0216] The device's steering angle exceeds the threshold.

[0217] The device has shifted at an angle exceeding the threshold.

[0218] The frequency of changes in the device's speed information exceeds a preset frequency threshold;

[0219] The oil level in the equipment is below the oil level threshold.

[0220] Optionally, the stimulus display module 42 includes:

[0221] The display mode and position determination unit is used to determine the display mode and display position of the stimulus according to the scene conditions;

[0222] The display unit is used to display the corresponding stimulus at the display position according to the display method.

[0223] Optionally, the eye information includes at least one of the following: fixation point position, horizontal difference between the fixation points of the two eyes, pupil information, saccade information, blink duration, and orbital information.

[0224] Optionally, the eye information includes the horizontal position difference between the two gaze points, and the attention analysis module 44 is specifically used to: determine whether the horizontal position difference between the two gaze points is within a first preset range; wherein, the horizontal position difference between the two gaze points is the absolute value of the difference between the coordinates of the left gaze point and the coordinates of the right gaze point on the horizontal axis; if so, determine that the user's attention state is a distracted state.

[0225] Optionally, the eye information includes saccade information, which includes at least one of saccade count, saccade duration, variability in saccade duration, and variability in maximum saccade rate. The attention analysis module 44 is specifically used for:

[0226] The user's attention state is determined to be a distracted state when at least one of the following conditions is met:

[0227] The number of eye sacral counts is within a second preset range;

[0228] The duration of the eye saccade is within a third preset range;

[0229] The variability in the duration of eye saccades is within a fourth preset range;

[0230] The maximum variability of the eye saccade rate is within the fifth preset range.

[0231] Optionally, the eye information includes blink duration, and the attention analysis module 44 is specifically used to: determine whether the blink duration is within a sixth preset range; if so, determine that the user's attention state is a distracted state.

[0232] Optionally, the eye information includes the horizontal position difference of the fixation points of both eyes, and the attention analysis module 44 is specifically used to: obtain the display range of the stimulus; determine whether the position of the fixation point is within the display range of the stimulus within a preset time; if so, determine that the user's attention state is a distracted state.

[0233] Optionally, the eye information includes the fixation point position, and the attention analysis module 44 is specifically used for: obtaining the initial fixation point position information and the initial fixation time based on the appearance of the stimulus; obtaining the display range of the stimulus; obtaining the fixation point termination position information and the fixation termination time based on the fixation point falling into the display range of the stimulus; determining the fixation point movement time based on the fixation initial time and the fixation termination time; determining the fixation point movement distance based on the fixation initial position information and the fixation point termination position information; determining the movement speed based on the fixation point movement distance and the fixation point movement time; and determining the user's attention state as a distracted state when the movement speed is less than a preset speed threshold.

[0234] Optionally, the eye information includes the fixation point position, and the attention analysis module 44 is specifically used for: obtaining the fixation initiation time based on the appearance of the stimulus; determining the fixation termination time when the fixation point falls into the display range of the stimulus; obtaining all fixation point positions within the fixation initiation time and the fixation termination time; judging the degree of dispersion when the fixation point moves based on each fixation point position; if the degree of dispersion is high, determining that the user's attention state is a distracted state.

[0235] Optionally, the eye information includes pupil information, which includes pupil size and / or pupil size variability. The attention analysis module 44 is specifically used to determine that the user's attention state is a distracted state when at least one of the following conditions is met: the pupil size is within a seventh preset range; the pupil size variability is within an eighth preset range.

[0236] The attention detection device provided in the embodiments of the present invention can execute the attention detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0237] Example 4

[0238] Figure 10 A schematic diagram of an electronic device 50 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0239] like Figure 10As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 and a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded into the RAM 53 from storage unit 58. The RAM 53 can also store various programs and data required for the operation of the electronic device 50. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0240] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, etc.; output unit 57, such as various types of monitors, speakers, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0241] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as attention detection methods.

[0242] In some embodiments, the attention detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the attention detection method described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to perform the attention detection method by any other suitable means (e.g., by means of firmware).

[0243] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0244] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0245] This invention provides a computer program product, which includes a computer program that, when executed by a processor, implements the attention detection method described in any embodiment of this invention.

[0246] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0247] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0248] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0249] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0250] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0251] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An attention detection method characterized by, The method comprises: starting an eye tracking function when it is detected that a scene condition meets an attention detection condition; displaying a stimulus based on the scene condition, wherein the stimulus is used to guide the user to gaze; obtaining eye information of the user, wherein the eye information is collected based on the eye tracking device; determining an attention state of the user based on the eye information.

2. The method of claim 1, wherein, The scene condition meeting the attention detection condition comprises at least one of the following: a degree of change of a target object around the device meeting a preset condition; a device parameter of the device meeting a preset threshold value; the device giving an alarm prompt.

3. The method of claim 2, wherein, The degree of change of the target object around the device meeting the preset condition comprises at least one of the following: a target number and a relative position of the target object being unchanged within a preset time length; a target number of the target object changing; a relative position of the target object changing.

4. The method of claim 2, wherein, The device parameter of the device meeting the preset threshold value comprises at least one of the following: a speed change value of the device exceeding a preset change threshold value; a turning angle of the device exceeding a threshold value; an offset angle of the device exceeding a threshold value; a speed information change frequency of the device exceeding a preset frequency threshold value; an oil amount of the device being less than an oil amount threshold value.

5. The method of claim 1, wherein, The displaying of the stimulus based on the scene condition comprises: determining a display mode and a display position of the stimulus according to the scene condition; and displaying a corresponding stimulus at the display position according to the display mode.

6. The method of claim 1, wherein, The eye information comprises at least one of the following: a gaze point position, a horizontal difference between gaze points of two eyes, pupil information, eye blink information, a blink length, and eye socket information.

7. The method of claim 6, wherein, The eye information comprises a horizontal difference between gaze points of two eyes, and the determining of the attention state of the user based on the eye information comprises: determining whether the horizontal difference between gaze points of two eyes is within a first preset range, wherein the horizontal difference between gaze points of two eyes is a difference absolute value of a left eye gaze point coordinate and a right eye gaze point coordinate on a horizontal axis; and if yes, determining that the attention state of the user is a distraction state.

8. The method of claim 6, wherein, The eye information comprises eye blink information, and the eye blink information comprises at least one of the following: an eye blink number, an eye blink length, a variability of the eye blink length, and a maximum rate variability of the eye blink, and the determining of the attention state of the user based on the eye information comprises: if at least one of the following conditions is met, determining that the attention state of the user is a distraction state: the eye blink number is within a second preset range; the eye blink length is within a third preset range; the variability of the eye blink length is within a fourth preset range; and the maximum rate variability of the eye blink is within a fifth preset range.

9. The method of claim 6, wherein, The eye information comprises a blink length, and the determining of the attention state of the user based on the eye information comprises: determining whether the blink length is within a sixth preset range; and if yes, determining that the attention state of the user is a distraction state.

10. The method of claim 6, wherein, When the eye information comprises a gaze point position, the determining of the attention state of the user based on the eye information comprises: obtaining a display range of the stimulus; determining whether the gaze point position is within the display range of the stimulus within a preset time; and if yes, determining that the attention state of the user is a distraction state.

11. The method of claim 6, wherein, The eye information comprises a gaze point position, and the determining of the attention state of the user based on the eye information comprises: acquire gaze point initial position information and gaze initial time based on the occurrence of the stimulus; acquire display range of the stimulus; acquire gaze point terminal position information and gaze terminal time based on the gaze point falling into the display range of the stimulus; determine gaze point movement time according to the gaze initial time and the gaze terminal time; determine gaze point movement distance according to the gaze point initial position information and the gaze point terminal position information; determine movement speed according to the gaze point movement distance and the gaze point movement time; when the movement speed is less than a preset speed threshold, determine that the attention state of the user is a distraction state.

12. The method of claim 6, wherein, The eye information includes gaze point positions, and the determination of the attention state of the user based on the eye information includes: acquire gaze initial time based on the occurrence of the stimulus; determine gaze terminal time based on the gaze point falling into the display range of the stimulus; acquire all gaze point positions within the gaze initial time and the gaze terminal time; determine the degree of dispersion of the gaze point movement according to each gaze point position; if the degree of dispersion is high, determine that the attention state of the user is a distraction state.

13. The method of claim 6, wherein, The eye information includes pupil information, and the pupil information includes pupil size and / or pupil size variability, and the determination of the attention state of the user based on the eye information includes: when at least one of the following conditions is met, determine that the attention state of the user is a distraction state: the pupil size is within a seventh preset range; the pupil size variability is within an eighth preset range.

14. An attention detection device, characterized by The method includes: a stimulus display module, configured to start an eye tracking function when it is detected that a scene condition meets an attention detection condition; a stimulus display module, configured to display a stimulus based on a scene condition, wherein the stimulus is used to guide the user to gaze; an eye information acquisition module, configured to acquire eye information of a user, wherein the eye information is acquired based on the eye tracking device; an attention analysis module, configured to determine the attention state of the user based on the eye information.

15. An electronic device, comprising: The electronic device includes: at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the attention detection method in any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to execute the attention detection method in any one of claims 1-13 when executed.

17. A computer program product, characterised in that, The computer program product includes a computer program that, when executed by a processor, implements the attention detection method according to any one of claims 1-13. The computer program product includes a computer program that, when executed by a processor, implements the attention detection method according to any one of claims 1-13.

Images