Joint attention potential evaluation method and system based on step-by-step prompting and process characteristics

Abstract

The application discloses a kind of based on step-by-step prompt and process feature's joint attention potential evaluation method and system.The method includes the following steps: constructing joint attention task and step-by-step prompt control flow, and in the flow, multi-modal data acquisition is carried out synchronously;The data collected are processed to extract eye movement process features;Based on the attention process features and behavior results, a hierarchical sequential potential evaluation model is constructed;The model is used to infer joint attention potential, and the evaluation results and explanations are output.The application introduces a hierarchical sequential potential evaluation model that conforms to the step-by-step prompt logic, and modulates the behavior response probability using eye movement process features, thereby achieving stable inference of children's potential joint attention ability.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary technical field of artificial intelligence, psychometrics and special education, and in particular relates to a method and system for assessing joint attentional potential based on step-by-step prompting and process characteristics. Background Technology

[0002] Joint attention refers to an individual's ability to share attention with others on a particular object or event through nonverbal communication means such as eye contact and gestures. Its delayed development is one of the core early characteristics of children with autism spectrum disorder. Accurate assessment of this ability is not only the basis for diagnosis but also a prerequisite for developing personalized intervention plans and implementing rehabilitation training.

[0003] Existing assessment techniques for joint attention are mainly divided into clinical observation methods and computer-based task assessment methods. Clinical observation methods rely on the assessor's subjective judgment, are time-consuming, and struggle to quantify subtle differences in ability. With the development of human-computer interaction technology, eye-tracking-based computer-aided assessment systems have gradually become a research hotspot. In these assessment tasks, tiered cues are a commonly used paradigm in clinical and experimental design; that is, when the child does not respond, gaze, verbal, or gestural cues are provided sequentially from weakest to strongest until the child performs the target behavior or the task is completed.

[0004] However, existing technologies still have significant limitations when using the above paradigm for computerized assessment. First, while existing methods employ a tiered prompting process, they often overlook the inherent sequential dependencies between prompt levels at the data analysis level. Existing technologies often use simple statistical indicators, such as calculating the highest prompt level required for a child's success or calculating the overall success rate. This approach fails to integrate behavioral responses under different prompting conditions into a unified probability model, making it difficult to decouple the subject's potential abilities from the external support provided by the prompts. Consequently, it cannot accurately calculate the child's intrinsic ability threshold to overcome prompt dependence.

[0005] Secondly, existing eye-tracking data analysis often focuses on outcome classification or single-feature statistics, lacking refined modeling of the task execution process. In actual assessments, children's failure to complete joint attention tasks may stem from insufficient cognitive abilities or from ineffective allocation of attentional resources. For example, a child may possess the ability to understand the task but fail to capture key visual cues within the appropriate time window, leading to task failure. Current technologies fail to incorporate eye-tracking signals as a procedural modulation variable that can explain behavioral outcomes into the assessment model. This results in the model's inability to distinguish between failures due to ability deficiencies and failures due to attentional modulation failures, thus leading to biased estimates of children's potential joint attention abilities.

[0006] Therefore, there is an urgent need for an assessment method that can adapt to a sequential structure of progressive cues and can incorporate eye movement characteristics to correct behavioral probabilities, in order to achieve quantitative inference of the potential joint attention ability of children with autism. Summary of the Invention

[0007] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method and system for assessing joint attention potential based on step-by-step prompting and process characteristics, which can achieve stable inference of children's potential joint attention ability.

[0008] To achieve the above objectives, according to one aspect of the present invention, a method for assessing joint attentional potential based on step-by-step prompting and process characteristics is provided, comprising the steps of: Load tasks for children and dynamically adjust task prompt levels during task execution. When dynamically adjusting prompt levels, follow a step-by-step prompting method from weak to strong. If the child does not make a target response to the current task at the current prompt level, the prompt level is increased step by step until the child makes a target response or still does not make a target response even after reaching the strongest prompt level. The process of a child performing a task is considered as a round. Behavioral outcome data and eye movement data are collected for each round. The behavioral outcome data includes the task completion status, the final triggered cue level, and time information for each round. Calculate the feature vector of eye movement process that reflects children's attention allocation strategy based on eye movement data; A hierarchical sequential potential assessment model is constructed based on behavioral outcome data and eye movement process feature vectors. Based on the hierarchical sequential potential assessment model, children's joint attention potential is inferred.

[0009] Preferably, the progressively stronger prompting method includes no prompting level, gaze prompting level, gaze and speech combined prompting level, and gaze and gesture combined prompting level.

[0010] Preferably, the collection of behavioral outcome data and eye-tracking data for each round includes the following steps: Simultaneously record the child's behavioral results and eye movement data for each round, and align the eye movement data with the trigger time of the cue level to form eye movement data corresponding to each cue stage.

[0011] Preferably, the step of calculating the eye movement process feature vector reflecting the child's attention allocation strategy based on eye movement data includes the following steps: Define the correct target interest region, distractor interest region, and social cue source interest region on the user interface; Using the triggering time of each cue level as a reference, at least two consecutive analysis time windows are constructed to characterize the attention response characteristics at different stages. The time window closer to the cue triggering time is defined as the early window, which reflects the child's immediate orientation response to social cues, while the time window further away from the cue triggering time is defined as the late window, which reflects the child's continuous processing of target information and decision-making process. Based on the above window division, the ratio of the number of fixation points falling into the correct target interest region in each window to the total number of effective fixation points in that window is calculated, and these multiple ratios are used as a multi-dimensional eye movement process feature vector. Based on the above window division, the ratio of the number of fixations falling into the social cue source interest region within each window to the total number of effective fixations within that window is calculated, and these multiple ratios are used as multi-dimensional eye movement process feature vectors.

[0012] Preferably, the step of calculating the eye movement process feature vector reflecting the child's attention allocation strategy based on eye movement data includes the following steps: A target-locking feature after prompting is constructed. The target-locking feature after prompting is defined as a binary variable. If, within any valid time window after the prompt is triggered, the child's gaze enters the correct target area of ​​interest and remains there continuously for a period of time exceeding a preset threshold, it is determined that a valid target lock has occurred, and the target-locking feature after prompting is recorded as a valid value. Otherwise, it is recorded as an invalid value, and the target-locking feature after prompting is used as the eye movement process feature vector.

[0013] Preferably, the step of constructing a hierarchical sequential potential assessment model based on behavioral outcome data and eye-tracking process feature vectors includes the following steps: It has Children awaiting evaluation, recorded as Each child The joint attention task for each round is denoted as... Each round contains A progressively increasing number of prompt levels, denoted as... Define children The joint attentional potential is a latent variable. , No. The difficulty threshold parameter for level-based hints is: Furthermore, the difficulty thresholds for different prompt levels satisfy the monotonicity constraint; the higher the prompt level, the lower the corresponding difficulty threshold for children. In the Round number The feature vector of eye movement process under level prompts is express; Hierarchical sequential potential assessment model specification: In a given , , Under these conditions, children In the Round number The conditional probability of successfully completing a task under level-one prompts is equal to the Sigmoid function mapping of the linear combination value, where the linear combination value is determined by... , With feature weight vector The inner product, and A joint decision.

[0014] Preferably, the hierarchical sequential potential assessment model includes: ; in, Let represent a binary random variable, when the child In the Round number The value is 1 if the task is successfully completed under the given prompt, and 0 otherwise. Indicates matrix transpose. Indicates that in a given , , Under these conditions, children In the Round number The conditional probability of successfully completing the task under level-one prompts; ; Children In the The round ends at the 1st step. The joint probability of level hints, The loop variable for calculating the product represents the sequence from the no-hint level up to the level preceding the current level. Indicates that in a given , , Under these conditions, children In the Round number The conditional probability of successfully completing a task under a given prompt.

[0015] Preferably, the hierarchical sequential potential assessment model includes: ; Indicates when children At the highest alert level The probability of not making a target response after the event ends.

[0016] Preferably, the inference of children's joint attention potential based on the hierarchical sequential potential assessment model includes the following steps: The actual observation results of each round are expanded into a binary response sequence indicating the alert level. Among them, the failure level will Recorded as 0, the level will be [recorded] upon first success. Recorded as 1, subsequent levels that are not triggered will not be expanded, and the highest level will be displayed. If no target response is made after the event concludes, all observable cue levels will be [affected / indicated]. Recorded as 0; For threshold parameters Spacing parameterization and monotonic constraints are applied to ensure that the threshold is satisfied. ; set up , express , , Under these conditions, children In the Round number The system prompts whether the task prediction result is complete; if successful... =1, if it fails =0, σ(.) denotes the Sigmoid function, and we construct the marginal log-likelihood objective function for the entire sample. The objective function is written as: ; in , The regularization coefficient is used. For the set of unknown parameters in the model Perform joint estimation and calculate the converged result. The value is output to the user as a joint attention potential value.

[0017] According to another aspect of the present invention, a joint attention potential assessment system based on step-by-step prompting and process characteristics is provided, comprising: The task interaction and control module is used to load tasks for children and dynamically adjust the task prompt level during the child's task execution. The prompt level is adjusted in a step-by-step manner from weak to strong. If the child does not make a target response to the current task at the current prompt level, the prompt level is increased step by step until the child makes a target response or still does not make a target response even after reaching the strongest prompt level. The multimodal data synchronous acquisition module is used to treat the process of a child performing a task as a round, and collect behavioral result data and eye movement data for each round. The behavioral result data includes the task completion status, the final triggered prompt level, and the reaction time for each round. The attention process feature extraction module is used to calculate eye movement process feature vectors that reflect children's attention allocation strategies based on eye movement data; The hierarchical sequential potential assessment model calculation module is used to construct a hierarchical sequential potential assessment model based on behavioral outcome data and eye movement process feature vectors, and to infer children's joint attention potential based on the hierarchical sequential potential assessment model.

[0018] Overall, the beneficial effects of the above-mentioned technical solutions conceived in this invention compared with the prior art are reflected in the following aspects.

[0019] First, this invention effectively addresses the problem of existing assessment techniques neglecting the inherent sequential dependencies of tiered cue data. Existing techniques often isolate and statistically analyze performance at each cue level, while this invention constructs a hierarchical sequential potential assessment model that describes the conditional probability relationships between cue levels. This modeling approach aligns with the clinical logic that "higher-level cues are only needed after lower-level cues fail," thus making the inference of children's abilities more rigorous and avoiding assessment biases caused by improper data structure processing.

[0020] Secondly, this invention significantly improves the purity and accuracy of potential assessment by introducing eye-tracking process features as a probability modulation variable. In traditional behavioral assessments, children may fail tasks due to brief distractions or inappropriate visual search strategies, which is often misjudged as a lack of cognitive ability. This invention uses eye-tracking data to correct the probability of behavioral outcomes, decoupling the attention allocation process from intrinsic cognitive abilities. This means the model can identify children who "possess the ability but fail due to inattention," thereby more accurately uncovering their potential abilities within their zone of proximal development.

[0021] Furthermore, this invention fully utilizes the information value of incomplete task samples by modeling right-censored events. In the assessment of children with low-functioning autism, complete failure is a common phenomenon. Existing technologies often struggle to handle such data, while this invention treats it as right-censored data and incorporates it into the likelihood function calculation, ensuring the model's assessment sensitivity in the low-ability range and expanding the applicability of the assessment method to a wider population.

[0022] Finally, the assessment results output by this invention are highly interpretable and clinically instructive. The system not only provides an ability score but also quantifies the contribution weight of different eye-movement features to task success rate. This provides therapists with a direct basis for developing personalized intervention plans, such as determining whether to prioritize basic attention training or focus on cognitive training for social intention understanding based on feature weights, thus achieving an effective connection between assessment and intervention. Attached Figure Description

[0023] Figure 1 This is a diagram illustrating the architecture and application scenarios of a children's joint attention potential assessment system provided in an embodiment of the present invention. Figure 2 A flowchart of a method for assessing children's joint attention potential provided in an embodiment of the present invention; Figure 3 A step-by-step prompting interaction logic diagram provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of eye movement feature extraction and time window provided in an embodiment of the present invention; Figure 5 This is a region of interest distribution map provided for an embodiment of the present invention; Figure 6 This is a schematic diagram of a hierarchical sequential potential evaluation model based on attention modulation, provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0025] In the description of the embodiments of this application, the term "multiple" means two or more.

[0026] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0027] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] This invention provides a method and system for assessing joint attentional potential based on progressive cues and process features. The core of this invention lies in introducing a sequential probability structure that conforms to progressive cues logic and modulating behavioral response probabilities using eye-tracking process features, thereby achieving stable inference of children's potential joint attentional abilities even in the presence of numerous failures and auxiliary dependencies.

[0030] like Figure 1 As shown, the joint attention potential assessment system of this invention includes a hardware interactive terminal and a software processing module.

[0031] The hardware interaction terminal mainly includes a high-performance computing unit, a visual stimulus presentation unit, and a non-invasive eye-tracking unit. The visual stimulus presentation unit typically uses a high-resolution display to present children with interactive task scenarios involving virtual robots and target building blocks. The Tobii series non-invasive eye-tracking unit is located below the display and is used to capture the coordinates of the child's gaze points and pupil diameter data in real time at a sampling rate of at least 60Hz.

[0032] The software processing module runs on a high-performance computing unit and integrates a task interaction and control module, a multimodal data synchronous acquisition module, an attention process feature extraction module, a hierarchical sequential potential assessment model calculation module, and a potential inference and report generation module. These modules communicate at high speed via a data bus, achieving closed-loop control from stimulus presentation and data acquisition to real-time analysis.

[0033] like Figure 2 As shown, an embodiment of the present invention provides a method for assessing joint attention potential based on step-by-step prompting and process characteristics, comprising the following steps: S101, Pay attention to the task and the step-by-step prompt control steps. Load a task for the child. The task can be a preset joint attention game script. During the child's execution of the task, the task prompt level is dynamically adjusted according to the child's real-time behavior. When dynamically adjusting the prompt level, a step-by-step prompting method from weak to strong is followed. If the child does not make a target response to the current task at the current prompt level, the prompt level is increased step by step until the child makes a target response or still does not make a target response even after reaching the strongest prompt level.

[0034] S102, Multimodal Data Synchronous Acquisition Step. The process of a child performing a task is considered a round, and behavioral outcome data and eye-tracking data are collected for each round. Behavioral outcome data includes round completion status, final trigger cue level, round end time, and round duration; the round time information is used for eye-tracking sequence slice alignment, time window feature extraction, and data quality control. Data can be segmented and labeled based on the cue trigger event and round time boundaries.

[0035] S103, Attention Process Feature Extraction Step. This step calculates eye-tracking process feature vectors reflecting children's attention allocation strategies based on eye-tracking data. The raw eye-tracking data can be cleaned and spatiotemporally sliced ​​based on the timestamps of cue triggers, and then process indicators reflecting children's attention allocation strategies can be calculated.

[0036] S104, Calculation steps of the sequential potential assessment model. A hierarchical sequential potential assessment model is constructed based on behavioral outcome data and eye movement process feature vectors. Based on the hierarchical sequential potential assessment model, the joint attention potential of children is inferred.

[0037] It may also include step S105, the assessment result output step. Based on the converged model parameters, the system outputs a quantitative score of the child's potential ability and a weighted analysis report of attention characteristics in each dimension, providing rehabilitation therapists with intuitive intervention suggestions.

[0038] like Figure 3 As shown, the sequential interaction logic in step S101 of this embodiment is designed as a unidirectional progressive state machine. If the child does not make a target response under the current conditions, the prompt level increases progressively according to the preset prompt levels. The prompt levels constitute a sequential structure from weak to strong, specifically including no prompt level, gaze prompt level, gaze and language combination prompt level, and gaze and gesture combination prompt level. The task follows a strict sequential mechanism, that is, the system will only trigger the next level prompt when the child has not made a target response to complete the task under all prompt conditions below the current level. Different prompt levels correspond to different ability threshold parameters, which are used to define the intrinsic ability threshold required for the child to get rid of external support in the model.

[0039] In one embodiment, at the start of each task round, the system first enters a no-cue phase (k=0), where the virtual robot simply gazes at a target block, and the system monitors whether the child responds with joint attention within a preset time. If the child does not respond, the state machine automatically transitions to the first-level cue phase (k=1), where the virtual robot plays a voice command while maintaining gaze. If the child still does not respond, the system further triggers the second-level cue phase (k=2), where the virtual robot adds a hand gesture pointing to the target block. If the child still cannot complete the task after this phase, the system triggers the third-level cue phase (k=3), which combines gaze, gesture, and verbal cues. Specifically, if the child fails to respond effectively at all preset cue levels, the system classifies the round as a task failure and marks it as right-censored data in the mathematical model. This design not only conforms to the principle of assisted withdrawal in clinical intervention, but more importantly, it establishes a strict conditional dependency between different cue levels, meaning that the triggering of a higher-level cue necessarily presupposes the failure of a lower-level cue.

[0040] Step S102, multimodal data acquisition, specifically includes the following: During the execution of each task round, the system synchronously records the child's behavioral outcome data and eye movement data. The behavioral outcome data includes the task completion status of each round, the final triggered cue level, and the reaction time. The eye movement data is acquired at a high-frequency sampling rate using an eye-tracking device and includes timestamps, fixation point coordinates, and eye saccade path information. The system aligns the eye movement data with the trigger time of the cue level to form an eye movement trajectory segment corresponding to each cue stage.

[0041] like Figure 4 and Figure 5 As shown in the example, this embodiment further illustrates how to extract key procedural features from raw eye-tracking data.

[0042] like Figure 5 As shown, the present invention first defines three static areas of interest (AOI) in the game screen coordinate system, which respectively cover the face and hand areas of the virtual robot (defined as social cue source AOI), the target block area (defined as correct target AOI), and the non-target block area (defined as interference item AOI).

[0043] In order to capture children's dynamic processing of cues, such as Figure 4 As shown, this invention constructs a dual-stage dynamic time window, using the trigger moment of each cue level as the zero-point baseline, to characterize the attention response features at different stages. The time window closer to the cue trigger moment is defined as the early window, and the time window further away from the cue trigger moment is defined as the late window. Specifically, 0 to 1.5 seconds after the cue trigger is defined as the early window, within which eye movement behavior mainly reflects the child's automatic orientation ability to social cues; 1.5 to 3.0 seconds after the cue trigger is defined as the late window, within which eye movement behavior mainly reflects the child's information processing and decision-making execution abilities.

[0044] Based on the aforementioned window division, this invention calculates and generates a multi-dimensional attention process feature vector. Specifically, this includes the target gaze ratio within each window, i.e., the ratio of the number of gaze points falling into the correct target AOI to the total number of effective gaze points within that window. In this embodiment, the correct target AOI is defined as the target block region. Both early and late windows calculate the gaze ratio based on this target AOI. Additionally, social cue source gaze can be counted as an independent eye-tracking feature. Its calculation method is: within a given analysis window, the ratio of the number of effective gaze points falling into the social cue source AOI (such as the face / hand of a virtual robot) to the total number of effective gaze points within that window. If necessary, the corresponding ratios can be further calculated separately for early and late windows to characterize the temporal changes in social cue utilization.

[0045] In practice, the start and end duration of the time window can be flexibly set according to task design requirements, prompt format, or subject characteristics. For example, it can be set as a fixed duration window, a proportional duration window, or an event-triggered adaptive window. This invention does not limit this.

[0046] In another embodiment, the time window can be further subdivided into multiple sub-windows to extract more refined attention change features, including but not limited to: the target fixation ratio in each sub-window, the slope of the fixation ratio change between sub-windows, the segmented index of the initial entry latency into the target AOI, the segmented index of continuous fixation duration, and the fixation shift frequency. Specifically, the segmented index of the initial entry latency into the target AOI refers to the time difference from the prompt triggering time to the initial entry into the target AOI, and is discretely encoded according to a preset interval, that is, several intervals of the preset time difference, encoded according to which interval the observed time difference belongs to; the segmented index of continuous fixation duration refers to the continuous fixation duration within the target AOI (which can be the longest continuous duration or the cumulative continuous duration), and is segmented and encoded according to a preset interval; the fixation shift frequency refers to the number of times the fixation switches between different AOIs per unit time, used to characterize the stability and exploratory nature of visual search. Through the above sub-window features, the dynamic process of attention orientation, maintenance, and shift after prompting can be characterized in a more granular way.

[0047] Furthermore, this invention specifically constructs a post-cue target locking feature, defined as a binary variable, to determine whether a child has formed stable joint attention after being prompted. The specific calculation logic is as follows: if, within any valid time window after the prompt is triggered, the child's gaze enters the correct target AOI and remains continuously there for more than a preset threshold (preferably 0.8 seconds in this embodiment), then it is determined that effective target locking has occurred, and the feature value is recorded as 1; otherwise, it is recorded as 0. This feature can effectively distinguish between two distinct cognitive states: "saccades" and "stable joint attention."

[0048] All the above-mentioned eye-tracking attention process features can be combined to form a multi-dimensional eye-tracking process feature vector, which can be used to characterize the attention regulation process of children under specific cueing conditions.

[0049] like Figure 6 As shown, this embodiment elaborates on the mathematical principles and computational path of the core algorithm model. The core of this embodiment lies in constructing a probabilistic model that can decouple "intrinsic capabilities" from "process-oriented attention".

[0050] This invention first defines the set of basic variables in the evaluation task. Assume the system has... Children awaiting evaluation, recorded as Each child undergoes The joint attention task for each round is denoted as... Each round contains A progressively increasing number of prompt levels, denoted as... (In this embodiment) ).definition For children In the The level indicated when the mission is completed at the end of the round. Indicates right deletion (i.e., in all cases) (All levels failed). In a specific data embodiment, the corresponding data can be retrieved according to the task configuration. value.

[0051] To quantify the real-time impact of attention on task completion rate, this invention constructs a hierarchical sequential potential evaluation model based on attention feature modulation. This model assumes that at specific cue levels… The conditional probability of a child making an effective response is determined by their potential ability, their current eye-tracking attention state, and the difficulty of the task at that cue level.

[0052] Specifically, defining children The joint attentional potential is a latent variable. , No. The difficulty threshold parameter for level-based hints is: Furthermore, the difficulty thresholds for different prompt levels satisfy the monotonicity constraint; the higher the prompt level, the lower the corresponding difficulty threshold for children. In the Round number The feature vector of eye movement process under level prompts is express; Hierarchical sequential potential assessment model specification: In a given , , Under these conditions, children In the Round number The conditional probability of successfully completing a task under level-one prompts is equal to the Sigmoid function mapping of the linear combination value, where the linear combination value is determined by... , With feature weight vector The inner product, and They are jointly determined. The specific mathematical expression encompasses the positive contribution of latent variables, the moderating effect of eigenvectors, and the negative intercept effect of the threshold parameter.

[0053] In one embodiment, the hierarchical sequential potential assessment model includes the mathematical formula (1): ; in, Let represent a binary random variable, when the child In the Round number The value is 1 if the task is successfully completed under the prompt, and 0 otherwise. It is a latent variable that cannot be directly observed, and it is also the core evaluation index that this invention ultimately seeks to solve through model inversion. The larger its value, the stronger the capability. The vector corresponds to the gaze ratio, lock-on delay, and other features extracted in Example 4; it reflects the quality of the child's current attention allocation. This represents the feature weight vector, used to quantify the contribution of different dimensions of attention features to the task success rate, and consists of the model parameters to be estimated. Since more cue information makes the task easier, the parameters... It must satisfy the monotonically decreasing constraint, that is , Indicates matrix transpose. Indicates that in a given , , Under these conditions, children In the Round number The conditional probability of successfully completing a task under a given prompt.

[0054] Furthermore, the step of constructing the hierarchical sequential potential assessment model also includes establishing sequential structure constraints and right censoring modeling.

[0055] Furthermore, based on the step-by-step prompting process described in this invention, the task results have a strict sequential dependency. That is, the child only receives the result on the first prompting stage. to Only after failing to complete the task under all the prompts will you have a chance to enter the next level. Level prompts. Therefore, children in the first... The round ends at the 1st step. The joint probability of a level prompt should be equal to the product of the failure probabilities of all previous levels and the success probability of the current level. This logic is expressed by the following formula (2): ; In the above formula (2), This indicates the level of achievement of the task by the children as actually observed. The loop variable for the product represents the sequence from the no-hint level up to the level preceding the current level. When When the product term is 1, the value of the product term is 1.

[0056] Furthermore, addressing the common "unresponsiveness" phenomenon among children with autism, this invention introduces a right censoring modeling mechanism. For cases where a task is not completed at all preset prompt levels, this invention models it as a right censoring event. The probability of a right censoring event is defined as the joint probability that the child fails at all preset prompt levels. This modeling approach allows the model to fully utilize information from incomplete task samples, improving the accuracy of assessments for children in the low-ability range.

[0057] When the child is at the highest preset prompt level by the system If no effective response is provided after the failure, the system not only records the failure but also uses this information to correct the probability distribution of the low-capability range. Let the prompt level for a successful response be... (common +1 level), then the right deletion event corresponds to all All failures under the prompt level are defined by the probability of occurrence as shown in formula (3): ; Solving for model parameters includes the following steps: (1) Data reconstruction: Expand the observation results of each round into a binary response sequence of the cue level. Among them, failure level Recorded as 0, first success level Record it as 1, and subsequent levels that have not been triggered will not be expanded.

[0058] (2) Censoring: For right censored rounds, press the failure event on all observable cue levels ( (Recorded as 0) is included in the likelihood to preserve information in the low-potential range.

[0059] (3) Parameter constraints: for threshold parameters Spacing parameterization and monotonic constraints are applied to ensure that the threshold is satisfied. This is to align with the task setting that "the higher the prompt, the lower the success threshold."

[0060] (4) Objective function: Construct the negative log-likelihood function for all samples, and add a regularization term to improve the estimation stability under small sample conditions.

[0061] To achieve parameter estimation, this invention constructs and minimizes a regularized negative log-likelihood objective function. Let...

[0062] express , , Under these conditions, children In the Round number The system prompts whether the task prediction result is complete; if successful... =1, if it fails =0, σ(z) represents the Sigmoid function, which is the probability value that maps the linear output to the interval (0,1).

[0063] Then the expanded level binary response data The objective function can be written as follows: ; The first term comes from the likelihood of the observed data, and the latter two are regularization terms. , is the regularization coefficient. The relationship between this objective function and the aforementioned formula is: Formula (1) gives the single-level success probability. Formulas (2) and (3) give the sequential success / right censoring probabilities at the round level; the round results are expanded according to the sequential rules as follows: Subsequently, the objective function described above is statistically equivalent to the likelihood expression at the round level, and is convenient for numerical optimization.

[0064] (5) Numerical optimization and convergence determination: Gradient-type numerical methods are used for parameter estimation, and the capability parameters are output. Threshold parameters With feature weights The validity is verified by considering the convergence state, the stability of the objective function, and the interpretability of the parameters.

[0065] (6) Output results: The system outputs individual ability scores and feature contribution explanations, which are used for subsequent group comparisons, failure mechanism classification and individualized intervention suggestions.

[0066] The system not only provides a capability score, but also outputs the weights that quantify the contribution of different eye-tracking features to the task success rate. w At the application level, features are first standardized, then stable contribution features are determined by combining weight direction, absolute value, and confidence interval, and aggregated interpretation is performed according to the cognitive dimension. It should be noted that "first fixation latency" here refers to the time difference between the first entry into the target AOI after cue triggering; "first entry into the target AOI latency segmentation index" is a discrete feature obtained by segmenting and encoding based on thresholds on this continuous variable. The former is used for continuous effect estimation, and the latter is used for hierarchical interpretation and rule discrimination. If attention orientation-related features (such as first fixation latency and early target interest area fixation ratio) are the main adverse factors, basic attention orientation training is prioritized; if social cue utilization-related features are the main adverse factors, social intention understanding training is prioritized; if both types of factors are significant, a phased joint intervention plan is adopted. Through this mechanism, a closed-loop connection of "assessment results - mechanism interpretation - intervention strategy" is achieved.

[0067] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for assessing joint attentional potential based on hierarchical prompting and process characteristics, characterized in that, Including the following steps: Load tasks for children and dynamically adjust task prompt levels during task execution. When dynamically adjusting prompt levels, follow a step-by-step prompting method from weak to strong. If the child does not make a target response to the current task at the current prompt level, the prompt level is increased step by step until the child makes a target response or still does not make a target response even after reaching the strongest prompt level. The process of a child performing a task is considered as a round. Behavioral outcome data and eye movement data are collected for each round. The behavioral outcome data includes the task completion status, the final triggered cue level, and time information for each round. Calculate the feature vector of eye movement process that reflects children's attention allocation strategy based on eye movement data; A hierarchical sequential potential assessment model is constructed based on behavioral outcome data and eye movement process feature vectors. Based on the hierarchical sequential potential assessment model, children's joint attention potential is inferred.

2. The method for assessing joint attentional potential based on progressive prompting and process characteristics as described in claim 1, characterized in that, The progressively stronger prompting method includes no prompting level, gaze prompting level, gaze and speech combined prompting level, and gaze and gesture combined prompting level.

3. The method for assessing joint attentional potential based on progressive prompting and process characteristics as described in claim 1, characterized in that, The process of collecting behavioral outcome data and eye-tracking data for each round includes the following steps: Simultaneously record the child's behavioral results and eye movement data for each round, and align the eye movement data with the trigger time of the cue level to form eye movement data corresponding to each cue stage.

4. The method for assessing joint attentional potential based on step-by-step prompting and process characteristics as described in claim 1, characterized in that, The process of calculating an eye-tracking feature vector reflecting a child's attention allocation strategy based on eye-tracking data includes the following steps: Define the correct target interest region, distractor interest region, and social cue source interest region on the user interface; Using the triggering time of each cue level as a reference, at least two consecutive analysis time windows are constructed to characterize the attention response characteristics at different stages. The time window closer to the cue triggering time is defined as the early window, which reflects the child's immediate orientation response to social cues, while the time window further away from the cue triggering time is defined as the late window, which reflects the child's continuous processing of target information and decision-making process. Based on the above window division, the ratio of the number of fixation points falling into the correct target interest region in each window to the total number of effective fixation points in that window is calculated, and these multiple ratios are used as a multi-dimensional eye movement process feature vector. Based on the above window division, the ratio of the number of fixations falling into the social cue source interest region within each window to the total number of effective fixations within that window is calculated, and these multiple ratios are used as multi-dimensional eye movement process feature vectors.

5. The method for assessing joint attentional potential based on step-by-step prompting and process characteristics as described in claim 1, characterized in that, The process of calculating an eye-tracking feature vector reflecting a child's attention allocation strategy based on eye-tracking data includes the following steps: A target-locking feature after prompting is constructed. The target-locking feature after prompting is defined as a binary variable. If, within any valid time window after the prompt is triggered, the child's gaze enters the correct target area of ​​interest and remains there continuously for a period of time exceeding a preset threshold, it is determined that a valid target lock has occurred, and the target-locking feature after prompting is recorded as a valid value. Otherwise, it is recorded as an invalid value, and the target-locking feature after prompting is used as the eye movement process feature vector.

6. The method for assessing joint attentional potential based on progressive prompting and process characteristics as described in claim 1, characterized in that, The construction of the hierarchical sequential potential assessment model based on behavioral outcome data and eye-tracking process feature vectors includes the following steps: It has Children awaiting evaluation, recorded as Each child The joint attention task for each round is denoted as... Each round contains A progressively increasing number of prompt levels, denoted as... Define children The joint attentional potential is a latent variable. , No. The difficulty threshold parameter for level-based hints is: Furthermore, the difficulty thresholds for different prompt levels satisfy the monotonicity constraint; the higher the prompt level, the lower the corresponding difficulty threshold for children. In the Round number The feature vector of eye movement process under level prompts is express; Hierarchical sequential potential assessment model specification: In a given , , Under these conditions, children In the Round number The conditional probability of successfully completing a task under level-one prompts is equal to the Sigmoid function mapping of the linear combination value, where the linear combination value is determined by... , With feature weight vector The inner product, and A joint decision.

7. The method for assessing joint attentional potential based on step-by-step prompting and process characteristics as described in claim 6, characterized in that, The hierarchical sequential potential assessment model includes: ； in, Let represent a binary random variable, when the child In the Round number The value is 1 if the task is successfully completed under the given prompt, and 0 otherwise. Indicates matrix transpose. Indicates that in a given , , Under these conditions, children In the Round number The conditional probability of successfully completing the task under level-one prompts; ； Children In the The round ends at the 1st step. The joint probability of level hints, The loop variable for calculating the product represents the sequence from the no-hint level up to the level preceding the current level. Indicates that in a given , , Under these conditions, children In the Round number The conditional probability of successfully completing a task under a given prompt.

8. The method for assessing joint attentional potential based on step-by-step prompting and process characteristics as described in claim 7, characterized in that, The hierarchical sequential potential assessment model includes: ； Indicates when children At the highest alert level The probability of not making a target response after the event ends.

9. The method for assessing joint attentional potential based on step-by-step prompting and process characteristics as described in claim 8, characterized in that, The method for inferring children's joint attentional potential based on the hierarchical sequential potential assessment model includes the following steps: The actual observation results of each round are expanded into a binary response sequence indicating the alert level. Among them, the failure level will Recorded as 0, the level will be [recorded] upon first success. Recorded as 1, subsequent levels that are not triggered will not be expanded, and the highest level will be displayed. If no target response is made after the event concludes, all observable cue levels will be [affected / indicated]. Recorded as 0; For threshold parameters Spacing parameterization and monotonic constraints are applied to ensure that the threshold is satisfied. ; set up , express , , Under these conditions, children In the Round number The system prompts whether the task prediction result is complete; if successful... =1, if it fails =0, σ(.) denotes the Sigmoid function, and we construct the marginal log-likelihood objective function for the entire sample. The objective function is written as: ； in , The regularization coefficient is used. For the set of unknown parameters in the model Perform joint estimation and calculate the converged result. The value is output to the user as a joint attention potential value.

10. A joint attention potential assessment system based on hierarchical prompting and process characteristics, characterized in that, include: The task interaction and control module is used to load tasks for children and dynamically adjust the task prompt level during the child's task execution. The prompt level is adjusted in a step-by-step manner from weak to strong. If the child does not make a target response to the current task at the current prompt level, the prompt level is increased step by step until the child makes a target response or still does not make a target response even after reaching the strongest prompt level. The multimodal data synchronous acquisition module is used to treat the process of a child performing a task as a round, and collect behavioral result data and eye movement data for each round. The behavioral result data includes the task completion status, the final triggered prompt level, and the reaction time for each round. The attention process feature extraction module is used to calculate eye movement process feature vectors that reflect children's attention allocation strategies based on eye movement data; The hierarchical sequential potential assessment model calculation module is used to construct a hierarchical sequential potential assessment model based on behavioral outcome data and eye movement process feature vectors, and to infer children's joint attention potential based on the hierarchical sequential potential assessment model.

Images