Line-of-sight focus prediction method and device, electronic equipment and readable storage medium

Abstract

The invention provides a sight line focus prediction method and device, electronic equipment and a readable storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining eye movement data which comprises eye movement images of a plurality of continuous time steps; feature extraction is conducted on the eye movement image of each time step, n feature maps corresponding to each time step under different scale conditions are obtained, and n is an integer larger than 1; calculating a weight corresponding to each feature map in the n feature maps corresponding to each time step, wherein the weight is used for representing a difference condition between the feature map corresponding to each time step and the feature map corresponding to the previous time step; based on the n feature maps corresponding to each time step and the weight corresponding to each feature map, calculating to obtain a target feature of the eye movement image in each time step; and predicting the sight focus position based on the target feature of the eye movement image in each time step. The method can improve the prediction accuracy of the sight focus.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a line of sight focus prediction method, device, electronic device and readable storage medium. Background Art

[0002] Eye tracking can use the device to capture the user's line of sight data in real time, including browsing screens and exposure focus, thereby improving the effect of user and front-end interaction. In related technologies, the user's line of sight focus is determined by obtaining eye movement data and environmental data of the user's gaze area, and processing the eye movement data and environmental data through a convolutional neural network model. However, in related technologies, there is still a certain deviation between the position predicted by processing eye movement data through a convolutional neural network and the actual position, and there is a problem of low prediction accuracy of line of sight focus.

[0003] It can be seen that the related technology has the problem of low prediction accuracy of sight focus. Summary of the invention

[0004] The embodiments of the present application provide a line of sight focus prediction method, device, electronic device and readable storage medium to solve the problem of low line of sight focus prediction accuracy in the prior art.

[0005] To solve the above problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for predicting a sight focus, comprising:

[0007] Acquire eye movement data, wherein the eye movement data includes eye movement images of a plurality of consecutive time steps;

[0008] Performing feature extraction on the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1;

[0009] Calculate the weight corresponding to each feature map in the n feature maps corresponding to each time step, where the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step;

[0010] Based on the n feature maps corresponding to each time step and the weight corresponding to each feature map, a target feature of the eye movement image at each time step is calculated;

[0011] The sight focus position is predicted based on the target features of the eye movement image at each time step.

[0012] In a second aspect, an embodiment of the present application further provides a sight focus prediction device, comprising:

[0013] A first acquisition module, used to acquire eye movement data, wherein the eye movement data includes eye movement images of a plurality of consecutive time steps;

[0014] An extraction module, used to extract features from the eye movement image at each time step, to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1;

[0015] A first calculation module, used to calculate the weight corresponding to each feature map of the n feature maps corresponding to each time step, wherein the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step;

[0016] A second calculation module, configured to calculate a target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and a weight corresponding to each feature map;

[0017] A prediction module is used to predict the sight focus position based on the target features of the eye movement image at each time step.

[0018] In a third aspect, an embodiment of the present application further provides an electronic device, including a transceiver and a processor.

[0019] The transceiver is used to obtain eye movement data, wherein the eye movement data includes eye movement images of multiple consecutive time steps;

[0020] The processor is used to extract features from the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1;

[0021] The processor is further used to calculate a weight corresponding to each feature map in the n feature maps corresponding to each time step, wherein the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step;

[0022] The processor is further configured to calculate a target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and a weight corresponding to each feature map;

[0023] The processor is further used to predict the sight focus position based on the target features of the eye movement image at each time step.

[0024] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the line of sight focus prediction method as described in the first aspect above are implemented.

[0025] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the line of sight focus prediction method as described in the first aspect above are implemented.

[0026] In a sixth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the line of sight focus prediction method as described in the first aspect above.

[0027] In an embodiment of the present application, eye movement data is obtained, and the eye movement data includes eye movement images of multiple consecutive time steps; feature extraction is performed on the eye movement images of each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1; the weight corresponding to each feature map in the n feature maps corresponding to each time step is calculated, and the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step; based on the n feature maps corresponding to each time step and the weight corresponding to each feature map, the target feature of the eye movement image at each time step is calculated; and the sight focus position is predicted based on the target feature of the eye movement image at each time step. In this way, by calculating the n feature maps corresponding to each time step under different scale conditions, and calculating the weight corresponding to each feature map, the target feature is calculated by the weight corresponding to each feature map, so that the target feature can more accurately capture the temporal changes of eye movement behavior, thereby improving the temporal sensitivity and prediction accuracy of the algorithm, and then improving the prediction accuracy of the sight focus. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 is a flow chart of a sight focus prediction method provided by an embodiment of the present application;

[0030] Figure 2 It is a schematic diagram of calculating the weights corresponding to the feature graph provided in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of multi-scale feature extraction provided in an embodiment of the present application;

[0032] Figure 4 is a flow chart of extracting target features provided by an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of calculating self-attention information provided by an embodiment of the present application;

[0034] Figure 6 This is a flow chart of obtaining target features through feature fusion provided in an embodiment of the present application;

[0035] Figure 7 This is a flow chart of optimizing the sight focus position through the user portrait vector provided by an embodiment of the present application;

[0036] Figure 8 It is a schematic diagram of constructing a user portrait vector provided in an embodiment of the present application;

[0037] Fig. 9 is a flow chart of a sight focus prediction system provided by an embodiment of the present application;

[0038] Fig.10 is a structural diagram of a sight focus prediction device provided in an embodiment of the present application;

[0039] Fig.11 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] See also Figure 1 , Figure 1 is a flow chart of a method for predicting sight focus provided by an embodiment of the present application, such as Figure 1 As shown, the following steps are included:

[0042] Step 101: Acquire eye movement data, where the eye movement data includes eye movement images of multiple consecutive time steps.

[0043] The above eye movement data is obtained by collecting the user's eye movements through a data acquisition module. The data acquisition module establishes a connection with the user through a high-precision eye tracking device. The data acquisition module usually includes a camera, an infrared sensor, and a dedicated eye tracking software. The data acquisition module can accurately capture the user's eye movements. Specifically, when a user browses a web page, the user's eyes will focus on the area of ​​interest, forming a series of eye movement data points or eye movement images, that is, forming eye movement data.

[0044] The above-mentioned time step is the time point difference between two adjacent eye movement images collected by the data acquisition module during the process of collecting eye movement images. Eye movement images of multiple consecutive time steps are collected through the data acquisition model, so that the multiple eye movement images can retain the characteristics on the time scale. Subsequent data processing can improve the accuracy of the predicted visual focus.

[0045] After the data acquisition module collects the eye movement data, it transmits the eye movement data to the system in real time to realize the analysis and prediction of the eye movement data. The transmission process needs to ensure the stability and accuracy of the data, so the module will adopt an efficient data transmission protocol and error checking mechanism. At the same time, in order to ensure the synchronization of the data, the data acquisition module will also record the timestamp corresponding to each eye movement data point, so that the system can restore the eye movement data through the timestamp.

[0046] In some implementations, the data acquisition module may store the eye movement data in a standardized format, so that a subsequent system can obtain the eye movement data from the data acquisition module for further analysis and processing.

[0047] In some embodiments, the eye movement data includes the user's gaze point and scanning path. Furthermore, the eye movement data may also include the user's pupil size, blinking frequency and other physiological index information. The user's physiological index information, as well as the user's gaze point and scanning path together constitute the eye movement data of the user's complete eye movement behavior when browsing web pages.

[0048] Step 102: extract features from the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1.

[0049] It should be noted that eye movement data has obvious multi-scale characteristics, which is specifically manifested in that different behavioral patterns may be exhibited at different time scales, such as rapid eye movements (Saccades) and fixations. The features extracted by traditional convolutional neural networks (CNN) are difficult to reflect the changes in eye movement data in the time series. Therefore, in the embodiments of the present application, the features that need to be extracted must reflect the changes in the eye movement data and the time dependence of the eye movement data.

[0050] In some implementations, the characteristics of the eye movement data may be extracted based on a Multi-Variable Time-Series Convolutional Neural Network (MTS-CNN) algorithm.

[0051] Specifically, by analyzing the characteristics of eye tracking data, such as the morphological features of the pupil, eyelid, and white of the eye, a series of optimized convolution kernels in MTS-CNN are designed. The convolution kernels can more effectively capture the key features of eye images, such as the precise position of the pupil center and the degree of eye opening, thereby improving the accuracy of line of sight focus recognition.

[0052] First, we design convolution kernels {Ks1, Ks2, ..., Ksn} of different scales, where s1, s2, ..., sn represent different scales. For each image at time step t, Use convolution kernels of different scales to extract features:

[0053]

[0054] Among them, F t,s is a feature map of scale s, * represents the convolution operation, b s is the bias term, and ReLU() is the linear rectification function. In this way, the feature map can be extracted under different scale conditions through convolution kernels of different sizes.

[0055] In some optional embodiments, before extracting feature maps from eye movement data using convolution kernels of different scales, the eye movement data may be preprocessed, wherein the preprocessing includes processing the eye movement image {I 1 ,I 2 ,...I T}Perform preprocessing operations such as resizing and normalization to obtain the image to be processed by MTS-CNN Where T is the length of the time series of the entire eye movement data.

[0056] Step 103: Calculate the weight corresponding to each feature map in the n feature maps corresponding to each time step, where the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step.

[0057] Step 104: Calculate the target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and the weight corresponding to each feature map.

[0058] It should be noted that, in view of the temporal nature of eye movement data, the embodiment of the present application introduces a dynamic weight adjustment mechanism, which adaptively adjusts the fusion weights of features of different scales based on the feature graph of the previous time step and the feature graph of the current time step. Figure 2 As shown, by calculating the weight corresponding to each feature map, the target feature of each time step can be obtained based on the weight corresponding to each feature map.

[0059] Among them, the weight represents the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step. The target feature is obtained by calculating the n feature maps corresponding to each time step and the weight corresponding to each feature map. The target feature can more accurately capture the temporal changes of eye movement behavior, such as rapid eye movements and gaze point changes, thereby improving the algorithm's temporal sensitivity and prediction accuracy.

[0060] For example, Figure 3 As shown, the present application forms a multi-scale feature extraction layer by designing a multi-scale convolution layer (i.e., convolution kernels of different scales), and realizes the extraction of feature maps of the input eye movement image at different scales through the multi-scale feature extraction layer; designs a weighted fusion layer, and calculates the weight corresponding to each feature map through the weighted fusion layer; designs a fully connected layer, and realizes the fusion of feature maps and weights corresponding to feature maps through the fully connected layer to calculate the target features.

[0061] In the embodiment of the present application, the flowchart for extracting target features is as follows: Figure 4 As shown, the eye movement data to be extracted by MTS-CNN is obtained through data preprocessing, and the feature maps under different scale conditions are extracted through multi-scale feature extraction. Then, the target features are obtained through time-sensitive feature fusion. Specifically, the weights corresponding to the feature maps are calculated, and then the n feature maps corresponding to each time step and the weights corresponding to each feature map are calculated to obtain the target features, so that the target features can more accurately capture the temporal changes of eye movement behavior, so that the target features can improve the temporal sensitivity and prediction accuracy of the algorithm.

[0062] Step 105: Predict the sight focus position based on the target features of the eye movement image at each time step.

[0063] Specifically, the gaze focus position is predicted based on the target features of the eye movement image at each time step, and the calculated target features at each time step are input into a prediction model, and the gaze focus position is predicted by the prediction model.

[0064] In an embodiment of the present application, eye movement data is obtained, and the eye movement data includes eye movement images of multiple consecutive time steps; feature extraction is performed on the eye movement images of each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1; the weight corresponding to each feature map in the n feature maps corresponding to each time step is calculated, and the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step; based on the n feature maps corresponding to each time step and the weight corresponding to each feature map, the target feature of the eye movement image at each time step is calculated; and the sight focus position is predicted based on the target feature of the eye movement image at each time step. In this way, by calculating the n feature maps corresponding to each time step under different scale conditions, and calculating the weight corresponding to each feature map, the target feature is calculated by the weight corresponding to each feature map, so that the target feature can more accurately capture the temporal changes of eye movement behavior, thereby improving the temporal sensitivity and prediction accuracy of the algorithm, and thus achieving the improvement of the prediction accuracy of the sight focus.

[0065] In some optional implementations, an interactive art exhibition based on the eye focus prediction method of the present application can be designed, and the audience can browse and interact with the artwork through eye movement control to obtain a more personalized and immersive experience. In addition, the application of eye tracking in creative advertising can also be explored, and the display method and content of the advertisement can be customized by capturing the user's eye movement trajectory to improve the advertising effect.

[0066] Furthermore, in addition to network front-end interaction, the line of sight focus prediction method can also be applied to online education. By analyzing students' eye movement data to evaluate students' learning status and points of interest, teachers can be provided with more accurate teaching feedback and personalized guidance. In addition, in autonomous driving and assisted driving systems, the driver's line of sight direction and attention status can be detected in real time, and warnings can be issued in time or corresponding measures can be taken to improve driving safety. For example, if the driver's intersection point appears to be irregularly free, it is considered that the driver is not paying attention, and a warning broadcast can be issued to remind the driver.

[0067] In one embodiment, calculating the weight corresponding to each feature map in the n feature maps corresponding to each time step includes:

[0068] Acquire the target feature of the eye movement image at the previous time step;

[0069] Based on the n feature maps and the target feature of the previous time step, calculating a difference metric corresponding to each feature map;

[0070] According to the difference metric corresponding to each feature map, a weight corresponding to each feature map is determined.

[0071] It should be noted that in order to achieve dynamic adjustment of weights and capture the temporal changes in eye movement data, the target features of the previous time step are introduced in the embodiment of the present application, and the weights corresponding to the feature graphs of each time step are adjusted by comparing the feature graphs with the target features of the previous time step.

[0072] Specifically, the target feature of the previous time step is F′ t-1 , the feature map of the current time step is F t,s , the difference measure between each feature map and the target feature of the previous time step is D(F′ t-1 ,F t,s ), D( ) is used to calculate the difference metric. The weight corresponding to each feature map is then determined by the difference metric.

[0073] In some implementations, the weight corresponding to each feature map can be calculated by a fully connected layer in a neural network. Specifically, the weight corresponding to each feature map calculated by a preset neural network is expressed by the formula: t,s =N(D(F′ t-1 ,F t,s )).

[0074] Among them, N() is a fully connected layer with an activation function in the preset neural network, which is used to convert the difference measure into weights.

[0075] In some implementations, in addition to the above formula, a correspondence between weights and difference values ​​may be preset, and the difference values ​​may be converted into weights corresponding to the feature graphs through the correspondence.

[0076] For example, the corresponding relationship can be a linear function, in which case the weight k can be expressed by the linear function k=a*D(F' t-1 ,F t,s )+b, where a and b are constants.

[0077] In the embodiment of the present application, the target feature of the eye movement image at the previous time step is obtained; based on the n feature graphs and the target feature of the previous time step, the difference metric corresponding to each feature graph is calculated; the difference metric corresponding to each feature graph is calculated by a preset neural network to obtain the weight corresponding to each feature graph. In this way, by introducing the target feature of the previous time step, the weight corresponding to each feature graph is calculated.

[0078] In some implementations, the weight corresponding to the feature graph may be calculated by calculating the difference metric between the feature graph at the previous time step and the current time step, and then calculating the difference metric between the feature graph at the previous time step and the current time step.

[0079] The difference measure between the feature map in the previous time step and the current time step can be expressed as D(F t-1 ,F t,s ), F t-1 is the feature map of the previous time step. The weight corresponding to each feature map is calculated by the preset neural network and expressed by the formula: W t,s =N(D(F t-1 ,F t,s )).

[0080] In some embodiments, based on the n feature maps corresponding to each time step and the weight corresponding to each feature map, the target feature of the eye movement image at each time step is calculated. Specifically, each time step and the weight corresponding to the time step are weighted summed to obtain the target feature of each time step.

[0081] Specifically, the target feature can be calculated by the following formula: t ′=Σ s W t,s ·F t,s .

[0082] In one embodiment, the predicting of the sight focus position based on the target feature of the eye movement image at each time step includes:

[0083] Obtaining the timing adaptive factor corresponding to the previous time step and the position code of each time step;

[0084] Based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step and the position encoding of each time step, the self-attention information corresponding to each time step is calculated;

[0085] Fusing the self-attention information corresponding to each time step with the target feature of the eye movement image at each time step to obtain the target feature of the eye movement image after being updated at each time step;

[0086] The sight focus position is obtained based on the target feature prediction of the eye movement image after being updated at each time step.

[0087] It should be noted that eye movement data is not just a static image feature, but also a dynamic time series. After extracting each feature map and the weight corresponding to the feature map through MTS-CNN, in order to more effectively process the extracted each feature map and the weight corresponding to the feature map, an adaptive timing coding mechanism is introduced in this application to capture the dynamic changes in the eye movement sequence. Adaptive timing coding converts the input sequence (such as eye movement data) into a format that is easier for the model to process, and adds a unique "label" or "marker" to the data at each time step so that the model can distinguish information at different time points, allowing the model to more effectively process the timing information in the eye movement data.

[0088] In an embodiment of the present application, a timing adaptive factor is set as an adaptive timing coding to obtain a target feature with higher recognition accuracy. In some embodiments, the processing of each feature map and the weight corresponding to the feature map can be achieved through a conversion model (such as a SAT-Transformer model). The SAT-Transformer sequence modeling module effectively processes these features in a time series context to capture the long-term dependence and dynamic changes of eye movements. The target features of the MTS-CNN feature extraction module are used as input by SAT-Transformer, and these features are deeply encoded and modeled in combination with the timing characteristics of the eye movement data, so that the SAT-Transformer sequence modeling module can simultaneously capture the instantaneous features and long-term dependencies of eye movements, providing a powerful feature representation for subsequent line of sight focus prediction. Through SAT-Transformer, it is possible to maintain temporal consistency when calculating attention, and has greater flexibility and adaptability when processing eye movement sequences of different lengths and rates.

[0089] The above-mentioned time series adaptive factor is a time series adaptive factor outputted at the same time as the target feature is outputted in the previous time step, and is used to adjust the influence of position coding; the above-mentioned position coding is used to characterize the position information of each time step relative to the entire time series. The target feature is marked by the time series adaptive factor and position coding to obtain the input feature of each time step.

[0090] Specifically, the input features can be calculated by the following formula: E t =F t ′+A t ·PE(t). In the formula, F t ′ is the target feature corresponding to time step t, A t is the timing adaptation factor, and PE(t) is the position code.

[0091] Furthermore, the self-attention mechanism allows the model to focus on information at different positions in the sequence when processing the sequence. The attention mask is a tool used to control the "field of vision" of the model when paying attention to information. In order to ensure that the model only pays attention to the information at the current time step and before, and follows the correct timing order when processing the sequence, the embodiment of the present application introduces a timing-aware attention mask into the self-attention mechanism to maintain timing consistency. In this way, the model can better capture the timing patterns in the eye movement data.

[0092] Specifically, Figure 5 As shown, the self-attention information corresponding to each time step is calculated based on the target features of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step and the position encoding of each time step. The self-attention information enables the model to accurately focus on the updated target features fused with the self-attention information, and the updated target features are used to predict the line of sight focus position.

[0093] Furthermore, after calculating the self-attention information, it is necessary to fuse the self-attention information with the target feature to obtain an updated target feature. In some embodiments, the updated target feature can be calculated by weighted averaging, concatenating, element-by-element multiplication, etc. of the self-attention information and the target feature.

[0094] In some embodiments, the fused updated target features can be fed into a fully connected layer or other classifier to produce a final output decision, including the predicted gaze focus position. This fusion strategy can improve the model's recognition accuracy of eye movement behavior, especially when dealing with rapid eye movements and gaze transitions.

[0095] In an embodiment of the present application, the timing adaptive factor corresponding to the previous time step and the position coding of each time step are obtained; based on the target features of the eye movement image at each time step, the timing adaptive factor corresponding to the previous time step and the position coding of each time step, the self-attention information corresponding to each time step is calculated; the self-attention information corresponding to each time step and the target features of the eye movement image at each time step are fused to obtain the target features of the eye movement image after being updated at each time step; the sight focus position is predicted based on the target features of the eye movement image after being updated at each time step. In this way, the self-attention information is calculated through the timing adaptive factor and the position coding, and the updated target features are fused with the self-attention information and the target features, so that the model can distinguish the timing information in the updated target features, and then can more effectively process the timing information in the eye movement data, and improve the accuracy of the sight focus position finally predicted.

[0096] Among them, Figure 6As shown in the figure, the target features are marked by introducing adaptive realization factors and self-attention mechanism, and finally the updated target features are obtained through feature fusion.

[0097] In some implementations, the timing adaptation factor may be obtained as follows:

[0098] The preset model receives the output features of the previous time step, and the output features may include various attributes of the eye movement data, such as the gaze point position, gaze duration, eye movement speed, etc.;

[0099] The output features are input into one or more neural network layers for processing. These neural network layers may include fully connected layers, convolutional layers, recurrent layers (such as Long Short-Term Memory (LSTM) or Gate Recurrent Unit (GRU)), etc., depending on the design of the model. In the neural network, the output features will undergo a series of linear transformations, nonlinear activations (such as ReLU, Sigmoid, etc.) and possible normalization operations to extract higher-level feature representations.

[0100] In the final stage of neural network processing, the preset model dynamically calculates the timing adaptation factor A based on the extracted features. t This process may involve a specific output layer that transforms the previous features into A t A scalar value (i.e. a single number).

[0101] Furthermore, the timing adaptation factor A t The value of may be constrained by some activation function (such as Sigmoid or Tanh) to ensure that it is within a reasonable range (such as between 0 and 1).

[0102] In one embodiment, the calculating the self-attention information corresponding to each time step based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step, and the position encoding of each time step includes:

[0103] Calculating the input feature of the eye movement image at each time step, the input feature is the sum of the target feature and the adjustment parameter at each time step, and the adjustment parameter is the product of the timing adaptive factor corresponding to the previous time step and the position encoding at each time step;

[0104] Calculate the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step;

[0105] Based on the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step, and the timing adaptation factor corresponding to the previous time step, the self-attention information corresponding to each time step is calculated.

[0106] The above input features can be calculated by the following formula: t =F t ′+A t ·PE(t). In the formula, F t ′ is the target feature corresponding to time step t, A t is the timing adaptation factor, and PE(t) is the position code.

[0107] The above query vector, key vector and value vector can be calculated by a preset weight matrix. The query vector is obtained by mapping the input features to the query vector space through the preset query vector weight matrix; the key vector is obtained by mapping the input features to the key vector space through the preset key vector weight matrix; and the value vector is obtained by mapping the input features to the value vector space through the preset value vector weight matrix.

[0108] After the query vector, key vector, and value vector are calculated, the self-attention information is calculated by the following formula:

[0109]

[0110] Where Q, K, and V are query, key, and value vectors respectively. k is the dimension of the key vector, used to prevent gradient disappearance.

[0111] In the embodiment of the present application, the input features of the eye movement image at each time step are calculated, the input features are the sum of the target features and the adjustment parameters at each time step, and the adjustment parameters are the product of the timing adaptive factor corresponding to the previous time step and the position encoding of each time step; the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step are calculated; based on the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step, and the timing adaptive factor corresponding to the previous time step, the self-attention information corresponding to each time step is calculated. In this way, the query vector, key vector and value vector are calculated through the input features, and the self-attention information is calculated through the query vector, key vector and value vector, so that the updated target features can be obtained by fusing the self-attention information with the target features.

[0112] In one embodiment, after predicting the sight focus position based on the target feature of the eye movement image at each time step, the method further includes:

[0113] Obtaining a user portrait vector of a target user, wherein the eye movement data is the eye movement data of the target user;

[0114] Fusing the user portrait vector and the sight focus position to obtain a fusion vector;

[0115] An updated sight focus position is predicted based on the fusion vector.

[0116] It should be noted that the eye movement data of different users have strong individual characteristics. In order to provide users with a more personalized, intuitive and natural interactive experience, achieve more accurate personalized content recommendations, and push information that is more in line with their interests and needs, user portraits are introduced in this application. The user's line of sight focus position is optimized through user portraits, thereby improving the interaction effect.

[0117] In some embodiments, the intersection position is achieved by combining the user portrait vector optimization prediction through a personalized dynamic prediction of gaze focus (PVFP-Dynamic) method. Among them, the PVFP-Dynamic gaze focus prediction module uses dynamically adjusted attention weights to predict the gaze focus based on the SAT-Transformer sequence modeling module and combined with the user portrait information. By considering the user's personalized characteristics and eye movement habits, the PVFP-Dynamic gaze focus prediction module can more accurately predict the user's gaze focus.

[0118] Among them, the process of optimizing the sight focus position by combining the user portrait vector is as follows: Figure 7 As shown, a user portrait vector is first constructed, and then the user portrait vector and the sight focus position are fused to obtain a fusion vector; finally, dynamic prediction is implemented based on the fusion vector to obtain an updated sight focus position.

[0119] In the embodiment of the present application, the eye movement data is the eye movement data of the target user by obtaining the user portrait vector of the target user; the user portrait vector and the sight focus position are fused to obtain a fusion vector; and the updated sight focus position is predicted based on the fusion vector. In this way, the user's sight focus position is optimized through the user portrait, thereby improving the effect of human-computer interaction.

[0120] In one embodiment, the user portrait vector is obtained in the following manner:

[0121] Obtain at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings;

[0122] Extracting at least one of an eye movement feature, a behavior feature, and a preference feature based on at least one of the eye movement tracking data, browsing history, click behavior, and preference settings of the target user;

[0123] At least one of the eye movement features, behavior features and preference features is fused to obtain the user portrait vector.

[0124] At least one of the eye tracking data, browsing history, click behavior, and preference settings of the target user may be obtained through log files, user configuration, and direct user feedback in the terminal system used by the user. In some embodiments, when obtaining the eye tracking data, browsing history, click behavior, and preference settings of the target user, the data may be pre-processed to delete invalid data or obviously erroneous data to ensure the accuracy of the obtained data.

[0125] like Figure 8 As shown, the eye movement features are used to characterize the characteristics of the target user's eye movements, such as average fixation duration, rapid eye movement frequency, fixation point distribution, etc. The behavioral features are used to characterize the characteristics of the target user's actions other than eye movements, such as click-through rate, browsing depth, active time period, etc. The preference features are used to characterize the target user's preference orientation, such as the topic or content type that the user explicitly expresses interest in.

[0126] In some embodiments, when at least one of the eye movement feature, the behavior feature, and the preference feature is fused to obtain a user portrait vector, the eye movement feature, the behavior feature, and the preference feature need to be normalized, and then a preset weight value is assigned to the eye movement feature, the behavior feature, and the preference feature, and the eye movement feature, the behavior feature, and the preference feature are weighted to obtain a fused feature. The weight value can be adjusted according to actual conditions to ensure that the contribution of different features is reasonably balanced.

[0127] In some implementations, at least one of the eye movement feature, the behavior feature, and the preference feature is fused to obtain the user portrait vector, which can be specifically expressed by the following formula:

[0128]

[0129] Among them, G t represents the predicted sight focus position at time step t, H t represents the output of SAT-Transformer, U is the user portrait vector, FC represents the fully connected layer, Indicates the feature fusion operation. In this way, by fusing the output of the SAT-Transformer modeling module with the user portrait vector, a fusion vector that incorporates the user's personalized information is obtained, and then the fully connected layer or other regression model is used to predict the updated sight focus position.

[0130] In addition, other information about the target user, such as voice information, can be obtained. By combining voice information and target features, cross-modal eye tracking can be achieved to provide a more comprehensive user behavior analysis.

[0131] In the embodiment of the present application, at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings is obtained; at least one of the eye movement features, behavior features, and preference features is extracted based on at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings; at least one of the eye movement features, behavior features, and preference features is fused to obtain a user portrait vector. In this way, the user portrait vector of the target user is constructed through at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings, thereby optimizing the predicted sight focus position.

[0132] In some embodiments, after the user portrait vector of the target user is obtained by fusion, the user portrait vector can be optimized by model training. Specifically, the user portrait vector is trained by a machine learning algorithm (such as clustering, classification, regression, etc.) to obtain a training result and / or a loss value. In the case where the training result and / or the loss value do not meet the preset conditions, the preset weight values ​​assigned to the eye movement features, behavioral features, and preference features can be adjusted to obtain an updated user portrait vector. Repeat the adjustment and training until the training results and loss values ​​meet the preset conditions.

[0133] Among them, the preset conditions may include that the loss value is less than a set loss threshold, and / or the accuracy included in the training result is greater than or equal to a set accuracy threshold.

[0134] In some implementations, before extracting features from the eye movement data, the eye movement data may be preprocessed to improve the accuracy of the eye movement data. The eye movement data may be preprocessed by a data preprocessing module.

[0135] It should be noted that eye movement data usually comes from different subjects and experimental conditions, and may exist in different formats and structures. It is necessary to use a data preprocessing module to process the data into data with a consistent format and structure, making it easier to understand and analyze, and also facilitating comparison and integration between different data sets.

[0136] Furthermore, there are often missing values, outliers, and duplicate values ​​in eye movement data. These special values ​​may be caused by equipment failure, subject head movement, etc., which may have a negative impact on subsequent analysis, such as biased or inaccurate analysis results. These problems can be handled by filling missing values ​​(such as using interpolation methods) or deleting rows containing missing values ​​through the data preprocessing module, thereby reducing the impact of missing values ​​on the analysis.

[0137] Specifically, the data preprocessing module removes or corrects invalid, abnormal or duplicate data through data cleaning, and then smoothes the original data to reduce the impact of noise and mutation on the data. For missing values, you can delete the rows or columns containing missing values, use central metrics such as mean, median or mode to fill missing data, and model missing values ​​by establishing classification or regression models. Finally, the processed data is input into the MTS-CNN feature extraction module for feature extraction.

[0138] Among them, in order to solve the noise interference problem that may exist in the current system, a more efficient filtering algorithm can be used to improve the accuracy and reliability of eye movement data. At the same time, the model training process can be optimized to reduce overfitting and improve the generalization ability of the model.

[0139] Furthermore, eye movement data contains rich information, but not all information is useful for subsequent analysis. The data preprocessing module can extract the most explanatory features for the research question and perform necessary dimensionality reduction to simplify the complexity of subsequent analysis and improve computational efficiency.

[0140] like Fig. 9 As shown, the embodiment of the present application also provides a flow chart of a gaze focus prediction system, which collects eye movement data through a data acquisition module, preprocesses eye movement data through a data preprocessing module, extracts target features through MTS-CNN and SAT-Transformer, and finally predicts gaze focus through PVFP-Dynamic. This system can effectively enhance the application effect of eye tracking in network interaction through a dynamic adaptive manner combined with multi-scale image processing technology, solves key issues in eye tracking such as accurate feature extraction, time series data processing, personalized interactive experience, and flexible adaptation to different scenarios, thereby improving the accuracy of the predicted gaze focus.

[0141] See also Fig.10 , Fig.10 is a structural diagram of a sight focus prediction device provided in an embodiment of the present application, such as Fig.10 As shown, the sight focus prediction 1000 includes:

[0142] A first acquisition module 1001 is used to acquire eye movement data, where the eye movement data includes eye movement images of a plurality of consecutive time steps;

[0143] An extraction module 1002 is used to extract features from the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1;

[0144] A first calculation module 1003 is used to calculate the weight corresponding to each feature map in the n feature maps corresponding to each time step, where the weight is used to represent the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step;

[0145] A second calculation module 1004 is used to calculate the target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and the weight corresponding to each feature map;

[0146] The prediction module 1005 is used to predict the sight focus position based on the target features of the eye movement image at each time step.

[0147] In one embodiment, the first calculation module 1003 includes:

[0148] A first acquisition unit, used for acquiring a target feature of the eye movement image at the previous time step;

[0149] A first calculation unit, configured to calculate a difference metric corresponding to each feature map based on the n feature maps and the target feature of the previous time step;

[0150] The second calculation unit is used to determine the weight corresponding to each feature map according to the difference metric corresponding to each feature map.

[0151] In one embodiment, the prediction module 1005 includes:

[0152] A second acquisition unit, used to acquire the timing adaptive factor corresponding to the previous time step and the position code of each time step;

[0153] A third calculation unit is used to calculate the self-attention information corresponding to each time step based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step and the position encoding of each time step;

[0154] A fusion unit, used to fuse the self-attention information corresponding to each time step with the target feature of the eye movement image at each time step to obtain the target feature of the eye movement image after being updated at each time step;

[0155] A prediction unit is used to predict the sight focus position based on the target features of the eye movement image after being updated at each time step.

[0156] In one embodiment, the third computing unit includes:

[0157] A first calculation subunit is used to calculate the input feature of the eye movement image at each time step, where the input feature is the sum of the target feature and an adjustment parameter at each time step, and the adjustment parameter is the product of the timing adaptive factor corresponding to the previous time step and the position code at each time step;

[0158] A second calculation subunit is used to calculate the query vector, the key vector and the value vector corresponding to the input features of the eye movement image at each time step;

[0159] The third computing subunit is used to calculate the self-attention information corresponding to each time step based on the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step, and the timing adaptive factor corresponding to the previous time step.

[0160] In one embodiment, after the prediction module 1005, the sight focus prediction 1000 further includes:

[0161] A second acquisition module is used to acquire a user portrait vector of a target user, wherein the eye movement data is the eye movement data of the target user;

[0162] A fusion module, used to fuse the user portrait vector and the sight focus position to obtain a fusion vector;

[0163] An updating module is used to predict an updated sight focus position based on the fusion vector.

[0164] In one embodiment, the user portrait vector is obtained in the following manner:

[0165] Obtain at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings;

[0166] Extracting at least one of an eye movement feature, a behavior feature, and a preference feature based on at least one of the eye movement tracking data, browsing history, click behavior, and preference settings of the target user;

[0167] At least one of the eye movement features, behavior features and preference features is fused to obtain the user portrait vector.

[0168] The data processing device provided in the embodiment of the present application is capable of implementing each process of each embodiment of the above-mentioned line of sight focus prediction method. The technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be described here.

[0169] For details, see Fig.11An embodiment of the present application further provides an electronic device, including a bus 1101, a transceiver 1102, an antenna 1103, a bus interface 1104, a processor 1105 and a memory 1106.

[0170] Wherein, the transceiver 1102 is used to obtain eye movement data, and the eye movement data includes eye movement images of multiple consecutive time steps;

[0171] The processor 1105 is used to extract features from the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1;

[0172] The processor 1105 is further configured to calculate a weight corresponding to each feature map in the n feature maps corresponding to each time step, wherein the weight is used to characterize a difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step;

[0173] The processor 1105 is further configured to calculate a target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and a weight corresponding to each feature map;

[0174] The processor 1105 is further configured to predict a sight focus position based on target features of the eye movement image at each time step.

[0175] In one embodiment, calculating the weight corresponding to each feature map in the n feature maps corresponding to each time step includes:

[0176] Acquire the target feature of the eye movement image at the previous time step;

[0177] Based on the n feature maps and the target feature of the previous time step, calculating a difference metric corresponding to each feature map;

[0178] According to the difference metric corresponding to each feature map, a weight corresponding to each feature map is determined.

[0179] In one embodiment, the calculating the target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and the weight corresponding to each feature map includes:

[0180] Obtaining the timing adaptive factor corresponding to the previous time step and the position code of each time step;

[0181] Based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step and the position encoding of each time step, the self-attention information corresponding to each time step is calculated;

[0182] Fusing the self-attention information corresponding to each time step with the target feature of the eye movement image at each time step to obtain the target feature of the eye movement image after being updated at each time step;

[0183] The sight focus position is obtained based on the target feature prediction of the eye movement image after being updated at each time step.

[0184] In one embodiment, the calculating the self-attention information corresponding to each time step based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step, and the position encoding of each time step includes:

[0185] Calculating the input feature of the eye movement image at each time step, the input feature is the sum of the target feature and the adjustment parameter at each time step, and the adjustment parameter is the product of the timing adaptive factor corresponding to the previous time step and the position encoding at each time step;

[0186] Calculate the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step;

[0187] Based on the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step, and the timing adaptation factor corresponding to the previous time step, the self-attention information corresponding to each time step is calculated.

[0188] In one embodiment, the transceiver 1102 is further used to obtain a user portrait vector of a target user, and the eye movement data is the eye movement data of the target user;

[0189] The processor 1105 is further configured to fuse the user portrait vector and the sight focus position to obtain a fused vector;

[0190] The processor 1105 is further configured to predict an updated sight focus position based on the fusion vector.

[0191] In one embodiment, the user portrait vector is obtained in the following manner:

[0192] Obtain at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings;

[0193] Extracting at least one of an eye movement feature, a behavior feature, and a preference feature based on at least one of the eye movement tracking data, browsing history, click behavior, and preference settings of the target user;

[0194] At least one of the eye movement features, behavior features and preference features is fused to obtain the user portrait vector.

[0195] exist Fig.11 In the embodiment, the bus architecture (represented by bus 1101) may include any number of interconnected buses and bridges, and bus 1101 links various circuits including one or more processors represented by processor 1105 and memory represented by memory 1106. Bus 1101 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 1104 provides an interface between bus 1101 and transceiver 1102. Transceiver 1102 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 1105 is transmitted on a wireless medium via antenna 1103, and further, antenna 1103 also receives data and transmits the data to processor 1105.

[0196] The processor 1105 is responsible for managing the bus 1101 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 1106 can be used to store data used by the processor 1105 when performing operations.

[0197] Optionally, the processor 1105 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0198] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the embodiment of the above-mentioned line of sight focus prediction method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0199] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the embodiment of the above-mentioned sight focus prediction method is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0200] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the embodiment of the above-mentioned line of sight focus prediction method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0201] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0202] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0203] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for predicting sight focus, characterized in that: include: Acquire eye movement data, wherein the eye movement data includes eye movement images of a plurality of consecutive time steps; Performing feature extraction on the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1; Calculate the weight corresponding to each feature map in the n feature maps corresponding to each time step, where the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step; Based on the n feature maps corresponding to each time step and the weight corresponding to each feature map, a target feature of the eye movement image at each time step is calculated; The sight focus position is predicted based on the target features of the eye movement image at each time step.

2. The method according to claim 1, characterized in that The calculating the weight corresponding to each feature map in the n feature maps corresponding to each time step includes: Acquire the target feature of the eye movement image at the previous time step; Based on the n feature maps and the target feature of the previous time step, calculating a difference metric corresponding to each feature map; According to the difference metric corresponding to each feature map, a weight corresponding to each feature map is determined.

3. The method according to claim 1, characterized in that The predicting of the sight focus position based on the target feature of the eye movement image at each time step includes: Obtaining the timing adaptive factor corresponding to the previous time step and the position code of each time step; Based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step and the position encoding of each time step, the self-attention information corresponding to each time step is calculated; Fusing the self-attention information corresponding to each time step with the target feature of the eye movement image at each time step to obtain the target feature of the eye movement image after being updated at each time step; The sight focus position is obtained based on the target feature prediction of the eye movement image after being updated at each time step.

4. The method according to claim 3, characterized in that The calculating the self-attention information corresponding to each time step based on the target feature of the eye movement image at each time step, the timing adaptation factor corresponding to the previous time step and the position encoding of each time step includes: Calculating the input feature of the eye movement image at each time step, the input feature is the sum of the target feature and the adjustment parameter at each time step, and the adjustment parameter is the product of the timing adaptive factor corresponding to the previous time step and the position encoding at each time step; Calculate the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step; Based on the query vector, key vector and value vector corresponding to the input features of the eye movement image at each time step, and the timing adaptation factor corresponding to the previous time step, the self-attention information corresponding to each time step is calculated.

5. The method according to any one of claims 1 to 4, characterized in that After obtaining the sight focus position based on the target feature prediction of the eye movement image at each time step, the method further includes: Obtaining a user portrait vector of a target user, wherein the eye movement data is the eye movement data of the target user; Fusing the user portrait vector and the sight focus position to obtain a fusion vector; An updated sight focus position is predicted based on the fusion vector.

6. The method according to claim 5, characterized in that The user portrait vector is obtained in the following way: Obtain at least one of the target user's eye tracking data, browsing history, click behavior, and preference settings; Extracting at least one of an eye movement feature, a behavior feature, and a preference feature based on at least one of the eye movement tracking data, browsing history, click behavior, and preference settings of the target user; At least one of the eye movement features, behavior features and preference features is fused to obtain the user portrait vector.

7. A sight focus prediction device, characterized in that: include: A first acquisition module, used to acquire eye movement data, wherein the eye movement data includes eye movement images of a plurality of consecutive time steps; An extraction module, used to extract features from the eye movement image at each time step, to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1; A first calculation module, used to calculate the weight corresponding to each feature map of the n feature maps corresponding to each time step, wherein the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step; A second calculation module, configured to calculate a target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and a weight corresponding to each feature map; A prediction module is used to predict the sight focus position based on the target features of the eye movement image at each time step.

8. An electronic device, characterized in that: Including transceiver and processor, The transceiver is used to obtain eye movement data, wherein the eye movement data includes eye movement images of multiple consecutive time steps; The processor is used to extract features from the eye movement image at each time step to obtain n feature maps corresponding to each time step under different scale conditions, where n is an integer greater than 1; The processor is further used to calculate a weight corresponding to each feature map in the n feature maps corresponding to each time step, wherein the weight is used to characterize the difference between the feature map corresponding to each time step and the feature map corresponding to the previous time step; The processor is further configured to calculate a target feature of the eye movement image at each time step based on the n feature maps corresponding to each time step and a weight corresponding to each feature map; The processor is further used to predict the sight focus position based on the target features of the eye movement image at each time step.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps in the line of sight focus prediction method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sight focus prediction method according to any one of claims 1 to 6 are implemented.

11. A computer program product, characterized in that It comprises computer instructions, which, when executed by a processor, implement the steps of the sight focus prediction method as described in any one of claims 1 to 6.

Images