A method for selecting a stimulus image for an eye movement experiment based on a discrimination measure

Abstract

The application relates to an eye movement experiment stimulus image selection method based on discriminability measurement and belongs to the field of image processing. Eye movement data of a subject observing a stimulus image is acquired, and the stimulus image and the eye movement data are divided into a training set and a test set; an agent model is constructed, the input eye movement data and the stimulus image are used to obtain a corresponding predicted category; the agent model is iteratively trained based on the training set; based on the agent model trained in each iteration, CSE score and CSD score are evaluated, discriminability score is acquired, a Pearson coefficient of the two scores is calculated, and it is judged whether the Pearson coefficient is greater than a consistency threshold value; if the Pearson coefficient is less than the consistency threshold value, the agent model is continuously iteratively trained; if the Pearson coefficient is greater than the consistency threshold value, the discriminability score of the present iteration is taken as a final discriminability score, and an eye movement stimulus image is selected according to the discriminability score.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to a method for evaluating the ability to distinguish stimulus images in eye-tracking experiments. Specifically, it establishes a classification model based on dynamic eye-tracking data as a surrogate model to assess group differences and individual consistency among subjects, mines stimulus image discrimination information to measure image discrimination ability, and realizes an image discrimination ability evaluation method based on a classification model; thereby enabling the selection of stimulus images. Background Technology

[0002] Eye tracking is a key method for revealing human perception and social behavior. Traditional eye tracking research focuses on exploring common mechanisms among different observers, such as attention models, salient object detection, and saccade path prediction. In recent years, researchers have begun to utilize different attributes of eye movements for group identification, aiming to solve practical applications in the medical, security, and commercial fields.

[0003] However, current eye-tracking recognition research often involves subjective or random stimulus selection, lacking quantitative standards to assess image discrimination capabilities and contributions to eye-tracking recognition research. For example, a significant application of eye-tracking recognition is the identification of autism spectrum disorder (ASD). Data-driven eye-tracking models can reduce subjectivity and reliance on physicians in traditional assessment methods, enabling scalable models and facilitating early and objective identification. However, current ASD research often uses subjectively or randomly selected stimuli (i.e., the images viewed by subjects), lacking standardization. Recent studies have shown that stimulus heterogeneity makes comparing and validating results difficult, hindering research progress in this area. Furthermore, selecting multiple stimuli for multiple durations of eye-tracking experiments is challenging for individuals with ASD. Therefore, developing models to assess and select stimuli in eye-tracking recognition experiments is an important but unresearched problem. Summary of the Invention

[0004] The technical problem to be solved by this invention is:

[0005] Current experimental setups for eye-tracking technology research exhibit high heterogeneity. For example, results from many autism eye-tracking studies are difficult to cross-validate, hindering in-depth research into eye movement differences between autistic individuals and typical developmental groups. A significant aspect of this heterogeneity is the heterogeneity of eye-tracking stimulus images; there is currently no standardized approach to evaluating and selecting these images. Therefore, exploring a unified standard for selecting eye-tracking stimulus images is crucial. The proposed method for evaluating the discriminative power of eye-tracking stimulus images provides a standard for selecting images with discriminative power. Furthermore, selecting more discriminative images can eliminate noise interference from poorly discriminative stimuli, thereby improving the accuracy of the corresponding recognition algorithm and shortening the duration of eye-tracking experiments. This alleviates the time-consuming nature of eye-tracking experiments; for example, in autism eye-tracking recognition, this invention can reduce the number of images required for diagnosis, thus reducing diagnostic time and playing a vital role in developing child-friendly diagnostic processes.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for selecting eye-tracking stimulus images based on discrimination strength, characterized in that it includes:

[0008] Obtain eye movement data of the subject observing the stimulus image, and divide the stimulus image and eye movement data into training set and test set;

[0009] A proxy model is constructed, which is an eye-tracking-based classification model that takes eye-tracking data and stimulus images as input to obtain the corresponding predicted category; the proxy model is then iteratively trained based on the training set.

[0010] Based on the surrogate model trained in each iteration, the CSE score and CSD score are evaluated to obtain the discrimination score. The Pearson coefficient of the two scores is calculated, and it is determined whether the Pearson coefficient is greater than the consistency threshold. If it is less than the consistency threshold, the surrogate model is trained iteratively. If it is greater than the consistency threshold, the discrimination score of this iteration is taken as the final discrimination score, and the eye-tracking stimulus image is selected based on the discrimination score.

[0011] A further technical solution of the present invention: the proxy model uses cross-entropy loss as the loss function during training and updates the model parameters through the SGD algorithm.

[0012] A further technical solution of the present invention: the evaluation of CSE score and CSD score specifically includes:

[0013] Using the ASD and TD groups of the subjects corresponding to the eye-tracking data as label information, the probability distribution of each image corresponding to the ASD group and the TD group was calculated by estimating the probability of each saccade predicted by the model. and The class entropy measure, i.e., the CSE score of different groups, is obtained by combining the class probability distribution with label information. and The weighted scores of the two categories are used to stimulate the final CSE score;

[0014] For the CSD score, the KL divergence is calculated between the probability distribution of each saccade path and the class probability distribution of its corresponding group. The results of all saccade paths corresponding to all different groups are weighted separately to obtain the final CSD score.

[0015] A further technical solution of the present invention: the acquisition of the discrimination score specifically includes:

[0016] The weighted CSE and CSD scores are then inversely calculated to obtain the final discriminative power score of the image, where a higher score indicates a higher discriminative power of the stimulus.

[0017] A further technical solution of the present invention: the calculation of the Pearson coefficient for the two scores is specifically as follows:

[0018] The discrimination scores of the stimulus images obtained from one process are sorted and compared with the discrimination scores obtained from the previous process to calculate the Pearson coefficient, which measures the consistency of the discrimination scores between the two processes.

[0019] A computer system is characterized by comprising: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0020] A computer-readable storage medium is characterized by storing computer-executable instructions, which, when executed, are used to implement the above-described method.

[0021] A computer program product is characterized by including computer-executable instructions, which, when executed, are used to implement the above-described method.

[0022] The beneficial effects of this invention are as follows:

[0023] The present invention provides a method for selecting eye-tracking stimulus images based on discrimination strength, which has the following advantages:

[0024] 1) This invention proposes a stimulus selection criterion and an eye-tracking research paradigm, which helps to solve the problems of heterogeneity of stimulus images and long experiment duration in eye-tracking experiments.

[0025] 2) This invention proposes a novel framework for measuring stimulus discrimination ability. The framework estimates CSE and CSD scores based on a surrogate model, and quantifies the group differences and individual consistency among different subjects in a stimulus image through class entropy and KL divergence measures, respectively.

[0026] 3) This invention proposes an iterative learning mechanism that uses the estimated discrimination score as feedback and employs progressive purification loss to guide the model to focus on stimuli with high discrimination ability, thereby iteratively optimizing the discrimination score.

[0027] 4) This invention proposes a discrimination-based aggregation method, which aggregates the results of all stimulus images according to the discrimination score of the stimulus images. This can be effectively used as the confidence level for the aggregation of results of different stimulus images, solving the problem that the previous average aggregation method is difficult to weight the results of different stimulus images.

[0028] 5) The stimulus image discrimination ability obtained by this invention corroborates the results of previous studies and yields some new quantitative results. Therefore, it can serve as a standard for selecting stimulus images in subsequent eye-tracking experiments. Furthermore, the quantitative conclusions obtained by this algorithm can serve as a reference for subsequent stimulus image selection. Attached Figure Description

[0029] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0030] Figure 1 This is a flowchart illustrating the overall implementation of the present invention;

[0031] Figure 2 This is a general framework diagram of the model of the present invention;

[0032] Figure 3 This is a schematic diagram of the CSE score calculation process of this invention;

[0033] Figure 4 This is a comparison diagram of the predicted probability distributions of the two stimuli in this invention;

[0034] Figure 5 This is a schematic diagram of the CSD score calculation process of this invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0036] This invention provides a method for selecting eye-tracking stimulus images based on discriminative power, where determining the discriminative power of quantified stimuli is a key issue. To address this issue, this invention proposes a novel and effective method that utilizes a surrogate model (i.e., a recognition model based on saccade paths) to extract discriminative power information from stimuli. The recognition model can encode this discriminative power information and integrate low-level texture features and high-level semantic features to better measure the discriminative power of stimuli. Furthermore, this invention notes that simpler stimuli tend to lead to greater inter-class differences; therefore, this invention considers methods for measuring difficulty levels, which have been discussed in active learning (AL) and hard negative example mining (HNM) methods. However, existing methods in these fields are difficult to apply to the model of this invention. First, the paradigms of these methods are incompatible with the settings of this invention. Second, the discriminative power of stimuli is a group characteristic between the ASD group and the typical developmental group, but these methods have not taken this relationship into account in previous tasks. Therefore, it is difficult to consistently assess the discriminative power of stimuli for subjects using existing methods.

[0037] To solve the above problems, the present invention proposes a discrimination score for evaluating group differences and individual consistency among subjects. Specifically, the present invention considers the predicted probability distribution of each stimulus after Softmax to explore discrimination information. The stimulus discrimination ability is a group attribute related to the differences between groups. For this purpose, the present invention introduces a cross-subject entropy (CSE) score to measure the group differences. The present invention first uses label information and Monte Carlo (MC) sampling to marginalize the subject probability distribution, which can obtain the between-group representation and decouple the input stimulus and saccade path. Then, the present invention proposes an entropy-like between-group representation method to measure the differences in the within-group population trends. However, the present invention notes that the CSE score considers the differences between groups while ignoring the separability within the group, which may lead to severe generalization degradation. Based on this, the present invention further proposes a cross-subject divergence (CSD) score to select more consistent stimuli. The CSD score uses the Kullback-Leibler (KL) divergence to calculate the distance between the cross-subject probability and the between-group representation to measure individual consistency. This score can achieve consistent and robust cross-subject discrimination estimation to improve the generalization ability. Finally, the present invention combines the CSE score and the CSD score to obtain the discrimination score, which is an effective general indicator for evaluating the stimulus discrimination ability.

[0038] In addition, the present invention introduces an iterative learning mechanism for further discrimination purification. Specifically, the estimated discrimination score is affected by the predicted probability distribution, which is affected by noise or uncertainty when training and testing. For this purpose, the present invention incorporates the feedback from the estimated discrimination score into the model training process. The present invention proposes a progressive purification loss to挖掘 the stimulus attention weighted for the training samples. This method allows the model to focus on discriminative stimuli, thereby refining the predicted probability distribution and the estimated discrimination score. In addition, the present invention notes that previous methods only adopt a basic aggregation strategy to obtain the subject results by averaging the prediction results of the saccade path. Through the stimulus discrimination ability, the present invention provides a performance-weighted method that aggregates the results of the saccade path prediction under the weight of the stimulus discrimination ability.

[0039] The above technical solutions specifically include the following parts:

[0040] (1) Training the base model

[0041] It should be noted that there is an unclear expression "挖掘" in the original text of item (3), which is tentatively translated as "挖掘" here. If there is a more accurate expression, it can be adjusted accordingly.Construct an eye movement-based classification model as a surrogate model, and then use the training set data to train the surrogate model. The surrogate model is a classification model based on eye movement data, which takes the eye movement data and the stimulus image as inputs to obtain the corresponding predicted category. The model uses the cross-entropy loss as the loss function and updates the model parameters through the SGD (Stochastic Gradient Descent) algorithm.

[0042] (2) Evaluate the CSE score and the CSD score

[0043] Based on the trained surrogate model, first use MC Dropout to obtain more stable probability estimates. Take the group (ASD and TD) of the subject corresponding to the eye movement data as the label information, and calculate the class probability distributions corresponding to the ASD group and the TD group for each image through the probability estimates of each saccade path predicted by the model. and Obtain the class entropy measure by combining the class probability distribution with the label information, that is, the CSE score for different groups. and Weight the two types of scores to obtain the final CSE score of the stimulus. For the CSD score, calculate the KL divergence between the probability distribution of each group of saccade paths and the class probability distribution of its corresponding group, and weight all the results corresponding to all different groups of saccade paths of the stimulus to obtain the final CSD score.

[0044] (3) Obtain the discrimination score

[0045] Weight the obtained CSE score and CSD score, and perform the reverse operation to obtain the final discrimination score of the image. The higher the score, the higher the discrimination of the stimulus.

[0046] (4) Calculate the Pearson coefficient of the two scores

[0047] Regard the above process as a process, sort the discrimination scores of the stimulus images obtained in one process, and calculate the Pearson coefficient with the discrimination score ranking obtained in the previous process to measure the consistency of the discrimination scores of the two processes.

[0048] (5) Iteratively learn and train the model based on the discrimination score

[0049] If the Pearson coefficient is less than a certain threshold, iteratively train the surrogate model to obtain a new discrimination score. Calculate the attention weight of each stimulus using the discrimination score of the stimulus image. More discriminative stimuli have higher weights. Introduce stimulus-based attention through the purification loss to promote the training process to focus on discriminative stimuli, and finally train to obtain new model parameters.

[0050] (6) Obtain the final discrimination score

[0051] When the Pearson coefficient is greater than a certain threshold, that is, there is no significant change in the discrimination score obtained in this process compared with the discrimination score in the previous process, stop the iterative update of the model, and use the discrimination score of this process as the final discrimination score.

[0052] To enable those skilled in the art to better understand the present invention, the present invention will be described in detail below with reference to specific embodiments.

[0053] Example 1:

[0054] Step 1: Train and test the base model

[0055] The base model is as shown in Figure 2 (a). Through the base model, the present invention can obtain the discrimination ability of stimuli from one process (a complete training and testing process) without iterative learning and updating of model parameters. Specifically, the present invention separates the training set (stimulus images and eye movement data) and the test set . Given an input sample x from the training set i,j (eye movement data), the prediction result of the surrogate model with parameters θ can be described as follows:

[0056]

[0057] where represents the predicted saccade path probability distribution (i.e., the result corresponding to the saccade path) in the first process. The cross-entropy (CE) loss is used to evaluate the difference between the prediction and the true value y i in terms of the saccade path, so as to update all parameters of the surrogate model, which can be expressed as:

[0058]

[0059] where and are the number of training objects and the number of stimuli respectively. Generally speaking, the surrogate model init is trained on the initial parameters θ and the original training set to obtain the trained model parameters. After the training process, the trained surrogate model 0 with parameters θ is evaluated to obtain the discrimination score for each stimulus I j . The above process can be mathematically summarized as follows:

[0060]

[0061] where θ 0 represents the parameters of the model after being trained in the first process . represents the stimulus discrimination ability I in the first process j . is a function for obtaining the discrimination score

[0062] Step 2: Obtain the discrimination score using the trained surrogate model

[0063] Considering that the stimulus discrimination ability is a group property involving differences between groups, the present invention uses a novel method to evaluate the discrimination ability based on the saccade path probability distribution for modeling special properties. The proposed discrimination score comes from two aspects, namely the CSE score and the CSD score, and these two aspects are finally combined to evaluate the stimulus discrimination ability

[0064] (1) Cross-subject entropy

[0065] In summary, the proposed CSE scoring aims to mine the differences in group trends within the group through the surrogate model. Inspired by the active learning and hard negative example mining methods, the present invention uses the prediction probability distribution containing the confidence information of the input samples to explore the discriminative information

[0066] Figure 2 (c) shows the calculation process of the discrimination score. The present invention first uses MC dropout to obtain a more stable probability estimate of the classification result. Specifically, given the saccade path and its corresponding image where I is the subject index and j is the image index, the present invention formulates the probability that the input belongs to class c through the saccade path-based recognition model as follows

[0067]

[0068] where represents the prediction probability in one process, and T MC represents the number of runs of MC dropout. Different from the active learning paradigm which involves selecting unlabeled samples for annotation, the training set of the present invention contains complete annotations for stimulus selection. For this reason, the present invention can combine class-related information to guide more accurate confidence estimation. Specifically, the present invention first divides the training subjects set according to the class labels as follows

[0069]

[0070] where and represent the index sets of the ASD group and the TD group, respectively. Then, in order to separate the stimuli from the coupled inputs and capture the population trends of the stimuli, the present invention is based on the sets and and adopts the joint probability to calculate the marginal probability as Figure 3 shown. Specifically, the joint probability can be expressed as follows:

[0071]

[0072] MC sampling is used to estimate the probability P i,j (s i,j |,I j ), and calculate the marginal probability Furthermore, label information is introduced to distinguish between the ASD and TD groups in order to obtain the class probability distribution, which represents the population trend within the group and decouples the input stimuli and saccade paths. Taking the ASD group as an example, the present invention can mathematically summarize the above process as follows:

[0073]

[0074]

[0075] where represents the class probability distribution of ASD for the image I j . Similarly, the class probability distribution of TD for the image I j is labeled as In addition, the class probability distribution represents the population trend exhibited by the probability distribution. In machine learning, entropy is usually used to measure the uncertainty of a probability distribution. For example, active learning methods use entropy to select samples that are difficult to label. The goal of the present invention is to measure the population differences in the responses of the ASD and TD groups to various stimuli. To this end, the present invention uses a class entropy measurement based on additional label-related information to quantify the difficulty of the stimuli and thus determine the discriminative ability of the stimuli. This class entropy measure CSE is defined mathematically by the following formula:

[0076]

[0077] where represents the indicator function, and represent the CSE scores of the ASD group and the TD group, respectively. To measure the difference between the two groups, the two CSE scores and are added with the weight coefficients ψ and (1 - ψ), which can be expressed as follows:

[0078]

[0079] where CSE j represents the final CSE score of image I j . The weight coefficients and are calculated based on the number of subjects in each group, which reflects the confidence level of each group, thus providing a more stable estimate.

[0080] (2) Cross-subject divergence

[0081] The CSE score helps to evaluate the difficulty level of stimuli to distinguish ASD patients from TD patients. However, the CSE score only represents the group trend within a group because it is calculated by averaging the marginalized distribution that aggregates the distributions of each subject. In other words, the CSE score does not take into account individual differences, resulting in non-robust and inconsistent measurement results and showing limited generalization potential. For example, Figure 4 shows a comparison between the predicted probability distributions of two stimuli. The present invention observes that Stimulus 1 and Stimulus 2 have the same CSE score. However, the predicted probability distributions of some subjects are quite different from the central probability distribution on Stimulus 1, which reflects the inconsistency among subjects in the image. The predicted probabilities of some subjects on Stimulus 1 exhibit high entropy, which is difficult to classify correctly. Compared with Stimulus 1, the predicted probability distribution on Stimulus 2 has a compact and consistent distribution. Based on the above observations, the present invention believes that the selected discriminative stimulus provides a more consistent and compact probability distribution for each subject, which can produce a more robust discriminative estimate and has greater stability when generalized to different subjects.

[0082] The discriminative score not only needs to evaluate the average group trend within a group but also needs to measure the differences between different subjects. In this way, the score can provide a measure of the consistency and robustness of cross-subject data to improve the generalization ability. Therefore, the present invention proposes the CSD score to measure the consistency among subjects in a group, which indicates how different the predictions involving the current subject are from the corresponding predictions generated by other subjects, as Figure 5 shown. The present invention also uses category information to distinguish ASD subjects from the TD group to more accurately measure the consistency within the group. The present invention uses the KL divergence function to measure the class difference between the individual distribution and the group probability distribution. For image , the CSD score can be defined as follows:

[0083]

[0084] where represents the KL divergence function. A low CSD score means that the prediction is more consistent with the prediction based on the same set of data.

[0085] Combined with the proposed CSE and CSD scores, the present invention can robustly and consistently estimate the stimulus discrimination ability. Note that low CSE and CSD scores mean high discrimination ability for the corresponding stimuli. Therefore, the present invention performs a reverse operation to obtain a final measure of the discrimination score, such that a higher score represents a more discriminative stimulus, which can be described as follows:

[0086] C j = CSE j + λ·CSD j , (18)

[0087] D j = 1 - C j , (19)

[0088] where λ represents the trade-off coefficient between the CSE score and the CSD score.

[0089] Step 3: Iterative learning for discrimination purification

[0090] The discrimination score can be obtained from the base model. However, the predicted probability distribution used to estimate the discrimination score may be affected by noise, resulting in an inaccurate discrimination score. To solve this problem, the present invention further proposes an iterative learning mechanism for purifying the discrimination score, which provides feedback to the model training process step by step in the subsequent process to further refine the discrimination score. The iterative process gradually enhances the accuracy of the surrogate model. Due to more accurate confidence estimation, the trained model is more precise and can better estimate the discrimination score.

[0091] Specifically, given the discrimination scores D 0 of a stimulus, the present invention first integrates them into the next process to feed the discrimination scores back into the surrogate model and retrain the model parameters. Generally speaking, the parameter update process can be summarized as follows:

[0092]

[0093] where θ 0 and θ t respectively represent the parameters of the surrogate model in process 0 and t. D t-1 represents the discrimination score in the (t - 1)-th process. Note that the model parameters in the t-th process are the training parameters θ t-1Initialized. To integrate the discrimination scores into the training process, the present invention proposes a stimulus-based attention mechanism that focuses on stimuli with high discrimination scores. Specifically, the discrimination scores contain the difficulty information of the samples; thus, paying more attention to discriminative stimuli is beneficial for aggregating all the saccade path results of a subject to produce better subject results. The stimulus-based attention mechanism weights each stimulus, where the weight is the feedback of the discrimination score obtained from the previous process. For each stimulus I j , its attention weight ω j can be formulated as follows:

[0094]

[0095] where represents the weight of stimulus I j in the t-th process. represents the discrimination score of stimulus I j in the (t-1)-th process. The weight coefficients are assigned to each stimulus, where higher weights are assigned to more discriminative stimuli. The present invention proposes a progressive purification loss to introduce stimulus-based attention, aiming to promote the training process to focus on discriminative stimuli. The formula for this progressive purification loss is as follows:

[0096]

[0097] where represents the prediction result in the t-th process. The proxy model is trained with the progressive purification loss and generates the model parameters θ t . Then, the present invention calculates the discrimination score under the new parameters θ t :

[0098]

[0099] where represents the discrimination score of stimulus I j in the t-th process. The estimated discrimination scores are iteratively purified by using the progressive purification loss, so that the discrimination scores become more accurate in this process.

[0100] A stopping strategy is designed for iterative learning: when the estimated discrimination scores do not change significantly between different processes. Specifically, the present invention calculates the consistency between the discrimination scores obtained in the current process and those obtained in the previous process. When the consistency is high, the estimation of the discrimination scores converges and the iterative process stops. The calculation process can be expressed as follows:

[0101]

[0102] where σ t is the consistency of the discrimination scores of two processes. is a consistency metric used to measure the consistency between two discrimination scores and to judge the convergence of the discrimination scores in the relevant processes. In a specific implementation of the present invention, the present invention uses the Pearson correlation between a pair of sorted discrimination results as the consistency metric σ t :

[0103]

[0104] where s t-1 (·) and s t (·) represent two sorting results, represents the Pearson correlation. When σ exceeds a certain threshold, the iterative process stops. Note that the feedback-based iterative learning is an online learning process that allows updating of model parameters based on test results. However, in many medical scenarios, due to privacy and commercial issues, the model parameters cannot be made public. Therefore, only the discrimination scores can be tested based on the predicted probability distribution, and the iterative learning process cannot be performed to update the model. In this case, the present invention evaluates the discrimination scores based on the base model, and this approach is also referred to as the "offline" approach.

[0105] Step 4: Discrimination score evaluation

[0106] (1) Evaluation of training

[0107] Given the number of selected stimuli, the present invention first selects stimuli from the stimulus set in the order of discrimination scores to generate a selected stimulus subset According to the subset select the saccade path set to form a training set Based on this selected subset train the recognition model which can be expressed as follows:

[0108]

[0109] where θ e represents the training model parameters based on the selected stimuli. Note that for a fair comparison, the initial parameters θ init are independent of the discrimination scores. The CE loss is also used to train the recognition model, which can be formulated as follows:

[0110]

[0111] where represents the corresponding prediction result,

[0112] (2) Discrimination-based Aggregation

[0113] Since the evaluation is carried out at the subject level, the results at the subject level are crucial for comparing model performance. Each subject observes multiple stimuli, and the saccade paths of all stimuli are recorded as eye-tracking data. Recognition models usually predict results based on individual saccade paths, that is, results based on saccade paths. To obtain the results at the subject level, the present invention needs to comprehensively aggregate the results of all saccade paths.

[0114] A natural way to generate subject results is to equally aggregate all saccade path results, which can be formulated as follows:

[0115]

[0116] where P i (c) represents the probability that subject i belongs to class c. The present invention considers the process of aggregating the results of individual saccade paths from the perspective of model ensemble (ME) to obtain subject results. From this perspective, previous methods used a simple mixing method that averaged the results on the saccade paths to obtain the subject results. However, since the performances of different stimuli vary, equally aggregating the results is not a suitable method. Therefore, the present invention proposes a new aggregation method that uses the discrimination ability of stimuli as an estimate of the corresponding performance of stimuli. Specifically, the calculated discrimination scores can be regarded as the confidence levels of different stimuli. The present invention uses the discrimination scores as the confidence levels of the corresponding stimuli and summarizes their stimulus results as follows:

[0117]

[0118] where represents the probability that subject i belongs to class c predicted by discriminative aggregation. The final predicted class y i is the corresponding class with the maximum probability.

[0119] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for selecting eye-tracking stimulus images based on discrimination strength, characterized in that, include: Obtain eye movement data of the subject observing the stimulus image, and divide the stimulus image and eye movement data into training set and test set; A proxy model is constructed, which is an eye-tracking-based classification model that takes eye-tracking data and stimulus images as input to obtain the corresponding predicted category; the proxy model is then iteratively trained based on the training set. Based on the agent model trained in each iteration, evaluate the CSE score and CSD score to obtain the discrimination score, calculate the Pearson coefficient of the two scores, and determine whether the Pearson coefficient is greater than the consistency threshold. If it is less than the consistency threshold, continue to iterate and train the agent model. If the score is greater than the consistency threshold, the discrimination score of this iteration is taken as the final discrimination score, and the eye-tracking stimulation image is selected based on the discrimination score. The assessment of CSE and CSD scores is as follows: Using the ASD and TD groups of the subjects corresponding to the eye-tracking data as label information, the probability distribution of each image corresponding to the ASD group and the TD group was calculated by estimating the probability of each saccade predicted by the model. and The class entropy measure, i.e., the CSE score of different groups, is obtained by combining the class probability distribution with label information. and The weighted average of the two categories of scores is used to stimulate the final CSE score. For the CSD score, the KL divergence is calculated between the probability distribution of each group of saccade paths and the class probability distribution of its corresponding group. All results of all different groups of saccade paths are weighted and summed to obtain the final CSD score. The calculation of the Pearson coefficient for the two scores is as follows: The discrimination scores of the stimulus images obtained from one process are sorted and compared with the discrimination scores obtained from the previous process to calculate the Pearson coefficient, which measures the consistency of the discrimination scores between the two processes.

2. The method for selecting eye-tracking stimulus images based on discrimination strength according to claim 1, characterized in that, The surrogate model uses cross-entropy loss as the loss function during training and updates the model parameters using the SGD algorithm.

3. The method for selecting eye-tracking stimulus images based on discrimination strength according to claim 1, characterized in that, The specific steps for obtaining the discrimination score are as follows: The weighted CSE and CSD scores are then inversely calculated to obtain the final discriminative power score of the image, where a higher score indicates a higher discriminative power of the stimulus.

4. A computer system, characterized in that... include: One or more processors, a computer-readable storage medium for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of claim 1.

5. A computer-readable storage medium, characterized in that... The device stores computer-executable instructions, which, when executed, are used to implement the method of claim 1.

6. A computer program product, characterized in that... It includes computer-executable instructions, which, when executed, are used to implement the method of claim 1.

Images