An interactive interface adaptive adjustment method, system, device and medium suitable for ASD+ADHD patients

Abstract

A method, system, device and medium for adaptive adjustment of an interactive interface for ASD+ADHD patients, relating to the field of human-computer interaction. The method comprises: obtaining target object attribute data, rendering an interactive interface that masks negative feedback information and contains target elements; obtaining multi-dimensional interaction sequence data during task execution; real-time monitoring of the data, rendering a soothing guidance interface when determining that the target object is in an interaction defense state; when not in the state, time-aligning the data and extracting first and second dimensional behavior feature data; determining the hit state of the target element based on the first dimensional behavior feature data, and then calculating a composite interaction influence index; based on the above feature data and influence index, adjusting the element presentation parameters of the interactive interface. The present application improves the accuracy of the interface adjustment strategy through dynamic analysis of multi-dimensional behavior features, and provides an interactive system suitable for psychological assessment and intervention of ASD+ADHD patients.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, specifically to a method, system, device, and medium for adaptive adjustment of the interactive interface suitable for ASD+ADHD patients. Background Technology

[0002] With the rapid development of artificial intelligence and medical information technology, interactive systems have become an important channel for psychological assessment and intervention. The design and presentation of various interactive interfaces are constantly evolving, allowing users to complete diverse assessment and intervention tasks. As a crucial tool for improving user compliance and task effectiveness, the adaptive adjustment and dynamic presentation of the interactive interface have a significant impact on the system's assessment and intervention outcomes.

[0003] Currently, adaptive adjustment methods for user interfaces mainly rely on statistical analysis of user task execution results. By analyzing single-dimensional task performance data such as user operation accuracy or reaction time, adjustment strategies for the difficulty or presentation speed of interface elements are determined to meet the user's ability adaptation needs during task execution.

[0004] However, relying solely on single-dimensional task execution data makes it difficult to accurately identify the interactive defensive states (such as aimless and frequent clicking, prolonged gaze deviation, etc.) of users with autism and attention deficit hyperactivity disorder (ASD+ADHD) during interaction. This results in the determined interface adjustment strategies being detached from the user's true psychological and behavioral state, thereby reducing the accuracy of the interface adjustment strategies. Summary of the Invention

[0005] To address the aforementioned shortcomings, a method, system, device, and medium for adaptive adjustment of the interactive interface suitable for ASD+ADHD patients are provided.

[0006] In a first aspect, the present invention provides an adaptive adjustment method for the interactive interface suitable for patients with ASD+ADHD, characterized in that the method includes: Obtain the attribute data of the target object (ASD+ADHD patient), and render an interactive interface that masks negative feedback information and contains target elements based on the target object attribute data. The negative feedback information includes interface prompt elements that represent operation errors or task failures, and the target elements include graphic elements that allow the target object to perform touch or tracking operations. During the execution of tasks based on the interactive interface, multi-dimensional interactive sequence data is acquired, including image sequence data and operation trajectory data. The system monitors the multi-dimensional interaction sequence data in real time. If the operation frequency of the operation trajectory data is greater than a preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than a preset duration threshold, the system determines that the target object is in an interactive defense state and renders a soothing guidance interface. When the target object is not in the interactive defense state, the multidimensional interaction sequence data is time-aligned, and a first-dimensional behavioral feature data and a second-dimensional behavioral feature data are extracted based on the aligned multidimensional interaction sequence data. The first-dimensional behavioral feature data includes the target object's accuracy performance data in the emotion recognition task, and the second-dimensional behavioral feature data includes the target object's reaction speed or attention maintenance performance data during the task. Based on the first dimension behavioral feature data, the hit status of the target element is determined. In the case of no hit, the fluctuation range of the second dimension behavioral feature data relative to the global historical data of the target object within the preset first time window is calculated to obtain the composite interaction influence index. In the case of a hit, the composite interaction influence index is set to the preset default value. Based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index, the element presentation parameters of the interactive interface are adjusted.

[0007] By employing the aforementioned technical solution, the target object attribute data is acquired, and an interactive interface containing target elements is rendered while shielding negative feedback information. This avoids the interference of negative prompts on the target object's emotions and maintains the target object's emotional stability. During task execution based on the interactive interface, multi-dimensional interactive sequence data, including image sequence data and operation trajectory data, is acquired and monitored in real time. If the operation frequency of the operation trajectory data exceeds a preset frequency threshold or the gaze deviation duration of the image sequence data exceeds a preset duration threshold, the target object is determined to be in an interactive defense state, and a soothing guidance interface is rendered. This promptly identifies abnormal behavior generated by the target object and alleviates its negative emotions through the soothing guidance interface, avoiding data distortion caused by continuing the task when the target object is in a poor state. When the target object is not in an interactive defense state, the multi-dimensional interactive sequence data is time-aligned, and first-dimensional behavioral feature data and second-dimensional behavioral feature data are extracted, eliminating time differences between different data sources. Based on the first-dimensional behavioral feature data, the hit status of the target element is determined. In the case of a miss, the fluctuation amplitude of the second-dimensional behavioral feature data within a preset first time window relative to the target object's global historical data is calculated to obtain a composite interactive impact index, quantifying the degree of abnormal behavioral changes accompanying the target object's operational errors. Ultimately, the element presentation parameters of the interactive interface were adjusted based on the first-dimensional behavioral characteristic data, the second-dimensional behavioral characteristic data, and the composite interaction influence index. This allowed the interface adjustment to comprehensively consider the behavioral performance of the target object and the interaction influence between dimensions, thereby improving the accuracy of the interface adjustment strategy and providing an interactive system suitable for psychological assessment and intervention of ASD+ADHD patients.

[0008] Secondly, the present invention provides an adaptive adjustment system for the interactive interface suitable for patients with ASD+ADHD, the system comprising: The interface rendering module is used to acquire target object attribute data and render an interactive interface that masks negative feedback information and contains target elements based on the target object attribute data. The negative feedback information includes interface prompt elements that represent operation errors or task failures. The target elements include graphic elements that allow the target object to perform touch or tracking operations. The data acquisition module is used to acquire multi-dimensional interaction sequence data during the execution of tasks based on the interactive interface. The multi-dimensional interaction sequence data includes image sequence data and operation trajectory data. The status monitoring module is used to monitor the multi-dimensional interaction sequence data in real time. If the operation frequency of the operation trajectory data is greater than a preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than a preset duration threshold, the module determines that the target object is in an interactive defense state and renders a soothing guidance interface. The feature extraction module is used to perform time alignment on the multidimensional interaction sequence data when the target object is not in the interaction defense state, and extract first-dimensional behavioral feature data and second-dimensional behavioral feature data based on the aligned multidimensional interaction sequence data. The first-dimensional behavioral feature data includes the accuracy performance data of the target object in the emotion recognition task, and the second-dimensional behavioral feature data includes the reaction speed or attention maintenance performance data of the target object during the task. The index calculation module is used to determine the hit status of the target element based on the first dimension behavioral feature data. In the case of no hit, it calculates the fluctuation range of the second dimension behavioral feature data relative to the global historical data of the target object within a preset first time window to obtain the composite interaction influence index. In the case of a hit, the composite interaction influence index is set to a preset default value. The parameter adjustment module is used to adjust the element presentation parameters of the interactive interface based on the first dimension behavioral feature data, the second dimension behavioral feature data, and the composite interaction influence index.

[0009] Thirdly, the present invention provides an electronic device, including a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the aforementioned adaptive adjustment method for the interactive interface suitable for ASD+ADHD patients when executing the computer program.

[0010] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when read and run by a processor, implements the above-described adaptive adjustment method for the interactive interface suitable for ASD+ADHD patients. Attached Figure Description

[0011] Figure 1 A flowchart illustrating an adaptive adjustment method for the interactive interface of patients with ASD+ADHD, provided in an embodiment of the present invention; Figure 2 A schematic diagram illustrating a multidimensional interactive sequence data feature extraction and fusion process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a composite interaction influence index calculation and interface adjustment logic provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the architecture of an adaptive adjustment system for the interactive interface of patients with ASD+ADHD, provided in an embodiment of the present invention. Detailed Implementation

[0012] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention. Example

[0013] like Figure 1 As shown, an adaptive adjustment method for the user interface suitable for patients with ASD+ADHD is proposed, the method comprising: Obtain target object attribute data, and render an interactive interface that masks negative feedback information and includes target elements based on the target object attribute data. Negative feedback information includes interface prompt elements that represent operation errors or task failures, and target elements include graphic elements that allow the target object to be touched or tracked. During the execution of tasks based on the interactive interface, multi-dimensional interactive sequence data is acquired, which includes image sequence data and operation trajectory data. Real-time monitoring of multi-dimensional interaction sequence data; if the operation frequency of the operation trajectory data is greater than the preset frequency threshold or the duration of the gaze deviation represented by the image sequence data is greater than the preset duration threshold, the target object is determined to be in an interactive defense state and a soothing guidance interface is rendered. When the target object is not in an interactive defense state, the multidimensional interaction sequence data is time-aligned, and the first-dimensional behavioral feature data and the second-dimensional behavioral feature data are extracted based on the aligned multidimensional interaction sequence data. The first-dimensional behavioral feature data includes the target object's accuracy performance data in the emotion recognition task, and the second-dimensional behavioral feature data includes the target object's reaction speed or attention maintenance performance data during the task. The hit status of the target element is determined based on the first dimension behavioral feature data. In the case of a miss, the fluctuation range of the second dimension behavioral feature data within the preset first time window relative to the global historical data of the target object is calculated to obtain the composite interaction influence index. In the case of a hit, the composite interaction influence index is set to the preset default value. Based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index, the element presentation parameters of the interactive interface are adjusted.

[0014] Specifically, the electronic device first acquires the target object's attribute data, which may include basic information such as the target object's age and medical history. Based on the acquired target object attribute data, it renders an interactive interface that masks negative feedback information and includes the target elements. In practical applications, the system hides or does not load interface prompt elements (i.e., negative feedback information, such as error pop-ups or red warning icons) that indicate operational errors or task failures during rendering. It presents the target elements that the target object needs to click or track (such as emoticons for touch selection or dynamic graphics for eye tracking) in a clean visual environment, thereby providing the target object with a safe and stress-free task execution environment and maintaining its emotional stability.

[0015] During the process of the target object performing tasks based on the interactive interface, the system simultaneously acquires multi-dimensional interaction sequence data. This multi-dimensional interaction sequence data specifically includes image sequence data acquired through the image acquisition module and operation trajectory data acquired through the touch interaction module. The system then monitors this multi-dimensional interaction sequence data in real time to determine the target object's current true interaction state. If the operation frequency of the operation trajectory data exceeds a preset frequency threshold, or the duration of gaze deviation represented by the image sequence data exceeds a preset duration threshold, it indicates that the target object is exhibiting aimless and frequent touching or severely wandering attention. At this point, the system determines that the target object is in an interactive defensive state, immediately pauses the current task, and renders a soothing guidance interface. By presenting relaxing guidance content, the system aims to alleviate the target object's resistance and anxiety, avoiding the acquisition of distorted interaction data.

[0016] If the target object is not in an interactive defensive state, the interaction task can proceed normally. At this point, the multi-dimensional interaction sequence data is time-aligned to eliminate time differences between different data sources. Based on the aligned multi-dimensional interaction sequence data, first-dimensional behavioral feature data and second-dimensional behavioral feature data are extracted. The first-dimensional behavioral feature data includes the target object's accuracy performance in the emotion recognition task, and the second-dimensional behavioral feature data includes the target object's reaction speed or attention maintenance performance during the task. Next, the hit status of the target element is determined based on the first-dimensional behavioral feature data. In the case of a miss, the fluctuation range of the second-dimensional behavioral feature data within a preset first time window relative to the target object's global historical data is calculated, thus obtaining a composite interaction impact index. This index accurately quantifies the cascading impact of a target object's error in one dimension on the behavior of other dimensions. In the case of a hit, the composite interaction impact index is set to a preset default value.

[0017] Finally, based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index, the element presentation parameters of the interactive interface are comprehensively adjusted. For example, based on the above calculation results, the presentation duration, movement speed, or complexity of interference items of target elements in subsequent interactive tasks can be dynamically changed. Through the above complete process, electronic devices can accurately capture the real interactive state of the target object in multiple dimensions and the mutual influence between dimensions, and accordingly provide a highly matched interface adaptive adjustment strategy, providing an interactive system suitable for psychological assessment and intervention of ASD+ADHD patients. Example

[0018] As a preferred embodiment of the present invention, based on Example 1, the multidimensional interaction sequence data is time-aligned, and the first-dimensional behavioral feature data and the second-dimensional behavioral feature data are extracted based on the aligned multidimensional interaction sequence data, including: Based on a preset sliding window, image sequence data and operation trajectory data are timestamped to generate a multimodal time series matrix; The multimodal time series matrix is ​​input into a pre-trained cross-modal attention network for feature fusion; The cross-modal attention network outputs first-dimensional behavioral feature data and second-dimensional behavioral feature data. The first-dimensional behavioral feature data represents the hit probability of the target element, and the second-dimensional behavioral feature data represents the sustained attention decay slope and impulsive touch rate.

[0019] Specifically, such as Figure 2As shown, image sequence data and operation trajectory data are synchronously traversed on a unified time axis using a preset sliding window. In this embodiment, the window length is set to 500ms and the step size is set to 250ms. For image frames falling within the same window, facial motion unit activation intensity vector, eye-tracking gaze coordinate sequence, and head posture angle are extracted. For operation events falling within the same window, touch point coordinates, time interval from target element presentation to touch response (reaction time), and touch frequency are extracted. The above features are concatenated into the multimodal feature vector of the window. With the task start time as zero, the feature vectors of all windows are arranged in order of window step size to form a multimodal time series matrix with the number of rows equal to the total number of windows and the number of columns equal to the total dimension of the concatenated features. This unifies the two originally asynchronous data streams into a structured matrix that can be analyzed segment by segment. After the matrix is ​​constructed, it is input into a pre-trained cross-modal attention network. The pre-training process of the cross-modal attention network specifically includes: obtaining a multimodal time-series sample matrix of historical target objects in multiple sets of baseline interaction tasks, and extracting the corresponding true hit probability, sustained attention decay slope, and impulsive touch rate as training labels by combining expert evaluation or standardized test results; constructing an initial network model that includes a multi-head cross-modal attention mechanism; inputting the sample matrix into the initial network model, calculating the loss value between the model output features and the training labels using the mean squared error (MSE) or cross-entropy loss function, and iteratively updating the network parameters using a backpropagation algorithm (such as the Adam optimizer) until the loss value converges, thus obtaining the pre-trained cross-modal attention network.

[0020] In practical applications, this network uses image feature subsequences as query ends and operation trajectory feature subsequences as key ends. It calculates the temporal correlation weights between the two modalities through a cross-modal attention mechanism, enabling the model to establish a cross-frequency and cross-semantic granular temporal correspondence between visual attention direction and actual touch behavior, and capture the temporal coupling pattern between the target object's gaze state and touch action. The network outputs first-dimensional behavioral feature data and second-dimensional behavioral feature data on the branch of the fused representation.

[0021] The first dimension of behavioral feature data represents the hit probability of the target element in a softmax-normalized probability form. Specifically, it is the joint probability that within the effective time window after the target element is presented, the target object's eye-tracking gaze hotspot falls on the target element area and the touch point coordinates simultaneously hit the area. The higher the hit probability, the more accurate the target object's perception and selection of the target element. The second dimension of behavioral feature data includes two sub-indicators. The first is the continuous attention decay slope, which is represented by the slope coefficient obtained by fitting the gaze deviation frequency to a linear regression throughout the task. The larger the absolute value of the slope, the more significant the decrease in attention maintenance ability over time. The second is the impulsive touch rate, which is defined as the proportion of touch events with a reaction time lower than a preset lower threshold to all effective touch events. In this embodiment, the threshold is set to 200ms. Touches below this value are judged as impulsive touches that the target object has responded to before fully perceiving the target element.

[0022] Taking the emotion decoding maze task targeting an 8-year-old boy with ASD and ADHD attention deficit as an example, during the task execution, the system acquired facial images at 30 frames per second and recorded touch operations with millisecond precision. After alignment with a sliding window, a cross-modal attention network fused and analyzed the image features and operation features throughout the process: In the early stage of the task, the target boy's eye-tracking fixation hotspot could stably fall on the option area within about 800ms after the target emotion icon was presented, and the touch time was basically synchronized with the fixation point, with the hit probability maintained at a moderate level; In the middle and later stages of the task, the fixation coordinates frequently paused briefly outside the option area before returning, the touch reaction time shortened to less than 180ms, the impulsive touch rate increased to 32% of all touches, and the sustained attention decay slope increased from -0.03 / minute in the early stage to -0.11 / minute, indicating that the attention maintenance ability decreased significantly as the task progressed; The feature data of the above two dimensions were then passed to the composite interaction influence index calculation step to provide quantitative input for subsequent adaptive adjustment. Example

[0023] As a preferred embodiment of the present invention, based on Example 1, the fluctuation range of the second-dimensional behavioral feature data within a preset first time window relative to the global historical data of the target object is calculated to obtain a composite interaction influence index, including: Determine the target time point where the first dimension of behavioral feature data represents the target element that was not hit; Extract the local attention decay slope from the target time point to the end of the preset first time window, and obtain the global average attention decay slope of the target object in the task; Calculate the ratio of the difference between the local attention decay slope and the global average attention decay slope; The composite interaction influence index is calculated based on the difference ratio and the preset weighting coefficient.

[0024] Specifically, such as Figure 3 As shown, the system continuously monitors the first-dimensional behavioral feature data. Once it detects that the hit probability represented by the first-dimensional behavioral feature data is lower than a preset judgment threshold, it is determined that the target element has not been hit, and the timestamp of the interaction is recorded as the target time point. Starting from the target time point, an interaction sequence segment of length equal to the first time window is extracted. In this embodiment, the first time window is set to 30 seconds. The second-dimensional behavioral feature data within this time window is extracted and fitted to obtain the local attention decay slope. This local slope reflects the trend of attention change in a short period of time after the miss event occurs. At the same time, the system retrieves the target object since the task started. The global average attention decay slope is calculated from the second-dimensional behavioral feature data across all valid time windows up to the current moment. This global slope represents the baseline attention maintenance level of the target object under normal interaction. Then, the global average attention decay slope is subtracted from the local attention decay slope, and the difference is divided by the absolute value of the global average attention decay slope to obtain the difference ratio. This ratio directly reflects the deviation of the attention deterioration caused by the missed event from the baseline level. Finally, this difference ratio is multiplied by a preset weighting coefficient (set to 1.5 in this embodiment) to calculate the composite interaction impact index. Based on this composite interaction impact index and multi-dimensional features, a current interaction state vector is constructed. After being input into the reinforcement learning model, it outputs interface adjustment action commands to dynamically update interface parameters. The larger the index, the more significant the negative impact of the first-dimensional missed event on the second-dimensional behavioral features.

[0025] Taking a complex interactive task involving emotion decoding and continuous tracking as an example, in the first 5 minutes of the task, the target's global average attention decay slope remained at a stable baseline level of -0.05 / minute. At 5 minutes and 12 seconds, the target failed to correctly select the matching emoji icon in the emotion decoding stage. The system marked this moment as the target time point and immediately started a first time window with a duration of 30 seconds. Data analysis showed that within this 30-second window, the target's tracking error distance increased significantly and the frequency of deviations rose sharply. The fitted local attention decay slope reached -0.15 / minute. The system calculated that the difference between the local slope and the global slope was 200%, indicating that the attention decay rate after the miss event reached three times the baseline level. Combined with a weighting coefficient of 1.5, the final calculated composite interaction impact index was 3.0. This high composite interaction impact index immediately triggered the system to lower the element presentation parameters of subsequent interfaces, such as reducing the movement speed of target elements and reducing the number of distracting items, to help the target smoothly pass through the cognitive fluctuation period. Example

[0026] As a preferred embodiment of the present invention, based on Example 3, the multidimensional interaction sequence data further includes physiological sensor data. Before calculating the composite interaction influence index based on the difference ratio and preset weighting coefficients, the method further includes: Extract the rate of change in physiological arousal within a preset first time window from physiological sensor data; The rate of change in physiological arousal is mapped to a weight value, and the weight value is used as a preset weighting coefficient.

[0027] Specifically, while acquiring image sequence data and operation trajectory data, the system simultaneously receives physiological sensing data from wearable sensing devices. This physiological sensing data mainly includes heart rate sequences and skin conductance activity sequences. After determining the target time point representing the first dimension of behavioral feature data as a missed target element, the system simultaneously extracts the heart rate variability index and skin conductance activity conductivity from the target time point to the end of the first time window, and calculates the percentage change relative to the global baseline level established at the beginning of the task. This comprehensive percentage is used as the physiological arousal change rate, which positively represents the degree of tension or anxiety caused by the missed event in the target object. Subsequently, the system uses a preset linear mapping function or step mapping table to map the calculated physiological arousal change rate to specific weight values. For example, when the physiological arousal change rate is in the normal fluctuation range, it is mapped to a base weight of 1.0, and when the change rate increases significantly, it is mapped to an amplified weight greater than 1.0. Finally, the system uses this dynamically generated weight value as a preset weighting coefficient and multiplies it by the difference ratio calculated in Example 3 to complete the calculation of the composite interaction influence index.

[0028] Taking the joint training task of emotion and impulse regulation performed by the target object as an example, when the target object fails to identify a specific emotional element, the system records the target time point and initiates a first time window of 30 seconds. During this period, the system monitors that the target object's heart rate rapidly increases from the baseline of 85 beats per minute to 110 beats per minute, while the skin conductance activity index shows a significant upward trend. The physiological arousal change rate is calculated to be 45%. The system inputs this 45% physiological arousal change rate into a preset mapping function, obtains the corresponding weight value of 1.8, and uses this 1.8 directly as the weighting coefficient. Assuming that the difference between the local attention decay slope calculated based on the operation trajectory and the global average attention decay slope is 150%, the final composite interaction influence index is 2.7. This index, which incorporates high physiological arousal characteristics, will prompt the system to immediately determine that the target object is in a state of high emotional load, thereby triggering a rapid reduction in the parameters of the interface elements, such as automatically inserting elements of the breathing relaxation guidance interface, effectively preventing the target object from entering a deep interactive defense state due to continuous frustration. Example

[0029] As a preferred embodiment of the present invention, based on Example 1, when the operation frequency of the operation trajectory data is greater than a preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than a preset duration threshold, it is determined that the target object is in an interactive defense state and a soothing guidance interface is rendered, including: The reaction time of a preset number of consecutive operations within a preset second time window is calculated in real time, and the operation frequency of the operation trajectory data is determined based on the reaction time, as well as the coordinates of the gaze point in the image sequence data are extracted. If the frequency of operation exceeds the preset frequency threshold, or if the duration of the gaze point deviating from the core area of ​​the task in the interactive interface exceeds the preset duration threshold and there is no touch action, the target object is determined to be in an interactive defense state. Intercept the task progress of the interactive interface and reduce the display density of elements in the interactive interface; A soothing, low-cognitive-load-based guided interface is overlaid on the interactive interface until the coordinates of the gaze point are detected and refocused on the core area of ​​the task.

[0030] Specifically, the system continuously monitors the status of tasks in the background, calculating in real time the reaction time of a preset number of consecutive operations within a pre-defined second time window. In this embodiment, the second time window is set to 10 seconds, and the preset number of consecutive operations is set to 5. By statistically analyzing the time interval of consecutive touch events within this window, the system determines the operation frequency of the operation trajectory data based on the reaction time. Simultaneously, an eye-tracking algorithm is used to extract the coordinates of the gaze point in the image sequence data in real time. Subsequently, the system performs dual-condition judgment. If the operation frequency is greater than a preset frequency threshold, such as more than 3 rapid disordered clicks per second, or if the gaze point coordinates deviate from the core area of ​​the task in the interactive interface for a duration greater than a preset duration threshold without any touch action, in this embodiment, the duration threshold is set to 8 seconds. These correspond to the target's impulsive venting and visual avoidance behaviors, respectively, and the system determines that the target is in an interactive defensive state. Once this state is confirmed, the system immediately intercepts the task progress on the interactive interface, pauses the current countdown and task logic, and reduces the display density of elements in the interactive interface. Background animations, non-critical decorations, and complex distractions are faded out to remove unnecessary visual stimuli. Next, the system overlays a low-cognitive-load soothing guidance interface on the interactive interface, such as presenting a soft-toned graphic that slowly expands and contracts to guide deep breathing. The system continuously monitors eye movement data until it detects that the gaze point coordinates have refocused on the core area of ​​the task and remain stable, indicating that the target has regained attention. Only then does the system remove the soothing guidance interface and smoothly resume the original task progress.

[0031] Taking a highly challenging visual search maze task as an example, as the difficulty of the level increases, the target becomes anxious due to the inability to find the correct path. The system detects that within a 10-second second time window, the target makes seven consecutive blind touches on the blank area of ​​the screen with extremely short reaction times. The calculated operation frequency is far greater than the preset frequency threshold, and the system immediately determines that the target has entered an interactive defense state. The system then intercepts the progress of the maze task, fades out the dense maze walls and interfering elements in the interface to reduce the element display density, and overlays a soothing guide interface with a water ripple effect in the center of the screen. After about 15 seconds of visual guidance, the system detects that the target's gaze has refocused on the core task area in the center of the screen, indicating that the target's emotions and attention have calmed down. The system then automatically exits the soothing guide interface, restores the normal display density of the maze elements, and continues the task. Example

[0032] As a preferred embodiment of the present invention, based on Example 1, the element presentation parameters of the interactive interface are adjusted based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index, including: The first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index are used to construct the current interaction state vector; Input the current interaction state vector into the preset reinforcement learning model and output the interface adjustment action command; Based on interface adjustment action commands, dynamically update the dwell time, movement speed, or number of distracting items of the target element in the interactive interface.

[0033] Specifically, during task execution, the system extracts in real time the first-dimensional behavioral feature data (e.g., target hit rate or reaction time), the second-dimensional behavioral feature data (e.g., attention duration or trajectory deviation), and the composite interaction influence index calculated in Example 3 within the current time window. After normalizing these three types of multi-source heterogeneous data, they are concatenated to construct a high-dimensional current interaction state vector. This vector comprehensively represents the target object's current overall cognitive and operational state. Subsequently, the system inputs this current interaction state vector into a pre-trained reinforcement learning model (e.g., Deep Q-Network (DQN) or Proximal Policy Optimization (PPO) model).

[0034] The pre-training process of the reinforcement learning model specifically includes: constructing a Markov Decision Process (MDP), where the state space is defined as a vector space containing the first dimension behavioral feature data, the second dimension behavioral feature data, and a composite interaction influence index; the action space is defined as a set of adjustment instructions for interface element presentation parameters (such as target element dwell time, movement speed, and number of distractors); constructing an offline experience replay buffer using historical user interaction logs; during the pre-training phase, the model samples state transition sequence data (i.e., current state, action taken, environmental feedback reward value, and next state) from the experience replay buffer, and updates the network parameters using the temporal difference (TD) algorithm or policy gradient algorithm to maximize long-term cumulative reward. After the above pre-training, the model can evaluate the expected benefits of various adjustment strategies in the preset action space based on the real-time input state vector, and output the interface adjustment action instruction with the maximum expected reward; finally, the system's rendering engine receives the action instruction and immediately executes the corresponding parameter modification on the front-end interface, dynamically updating the target element's dwell time, movement speed, or number of distractors in the interactive interface to match the target object's current cognitive load level in real time.

[0035] Taking the task of multi-target dynamic tracking and recognition of a target object as an example, at a certain stage of the task, the system extracts first-dimensional behavioral feature data indicating that the target hit rate has dropped to 60%, and second-dimensional behavioral feature data indicating that the trajectory tracking error has increased significantly, with the composite interaction impact index reaching as high as 2.5. The system normalizes these data to construct the current interaction state vector and inputs it into a preset reinforcement learning model. After analyzing this high-load state vector, the model outputs an interface adjustment action instruction aimed at reducing cognitive stress: "Increase the target element dwell time by 0.5 seconds, reduce the movement speed by 20%, and reduce the number of interference items by 2." After receiving the instruction, the system front end instantly completes a smooth transition update of the interface rendering parameters. After the adjustment, facing a slower task pace and a clearer visual environment, the target object's hit rate and tracking accuracy gradually recover within the following minute. Based on this, the system feeds back a positive reward signal to the reinforcement learning model, further optimizing the model's decision-making strategy. Example

[0036] As a preferred embodiment of the present invention, based on Example 6, the preset reinforcement learning model updates the policy in the following manner: Obtain the task completion score and interaction effort score of the target object after the task is executed. The interaction effort score is determined based on the effective gaze duration and reasonable number of touches within a preset time. Based on the task completion score and interaction effort score, a purely positive reward function value is calculated. The reward function value does not contain any negative penalty terms and is always greater than or equal to zero. The network parameters of the reinforcement learning model are updated based on the purely positive reward function value using the proximal policy optimization algorithm.

[0037] Specifically, at the end of each interactive task cycle or at a preset evaluation node, the system first obtains the target object's task completion score, which objectively reflects the achievement of the task objective (such as hit rate, progress, etc.). Simultaneously, the system calculates the interaction effort score, which focuses on evaluating the target object's subjective engagement. This score is calculated by weighting the effective gaze duration (i.e., the duration of time the gaze falls on the task-related area) and the reasonable number of touches (the number of effective operations after excluding high-frequency invalid random points) within a preset time period. Subsequently, the system fuses the task completion score and the interaction effort score (e.g., linear weighted summation) and performs ReLU function or bottom-line truncation to calculate a purely positive reward function value, ensuring that this value does not contain any negative penalty terms and is always greater than or equal to zero. Finally, the system uses this purely positive reward function value as an environmental feedback signal, calculates the advantage function using the Proximal Policy Optimization (PPO) algorithm, and updates the parameters of the Actor and Critic networks in the reinforcement learning model under the constraint of limiting the difference between the old and new policies (KL divergence), thereby optimizing the next interface adjustment action command.

[0038] Taking a child's attention training task as an example, after a 3-minute training cycle, the system calculated the child's task completion score to be only 55 out of 100, indicating a mediocre performance. However, the system also extracted that the effective gaze duration within that cycle reached 140 seconds, and the number of reasonable touches was 45, indicating that the child maintained a high level of focus and effort. Based on this, the interaction effort score was calculated to be 85. The system combined the task completion score and interaction effort with weights of 0.4 and 0.6, respectively, to calculate a comprehensive score of 73, which was directly used as the pure positive reward function value (i.e., a reward value of 73, with no deductions). After receiving this positive reward value, the PPO algorithm recognized that although the current strategy did not bring extremely high task scores, it successfully maintained the target's high effort. Therefore, it calculated a positive advantage estimate and smoothly updated the network parameters. Through this purely positive incentive mechanism, the model tends to maintain a moderate level of challenge in subsequent rounds, rather than drastically reducing the number of distractors or slowing down movement speed due to low task scores, thus effectively ensuring the continued effectiveness of focus training. Example

[0039] Reference Figure 4 The present invention provides an adaptive adjustment system for the interactive interface suitable for patients with ASD+ADHD, the system comprising: The interface rendering module is used to obtain the target object's attribute data and render an interactive interface that masks negative feedback information and contains target elements based on the target object's attribute data. Negative feedback information includes interface prompt elements that indicate operation errors or task failures, and target elements include graphic elements that allow the target object to perform touch or tracking operations. The data acquisition module is used to acquire multi-dimensional interaction sequence data during the execution of tasks based on the interactive interface. The multi-dimensional interaction sequence data includes image sequence data and operation trajectory data. The status monitoring module is used to monitor multi-dimensional interaction sequence data in real time. When the operation frequency of the operation trajectory data is greater than the preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than the preset duration threshold, it determines that the target object is in an interactive defense state and renders a soothing guidance interface. The feature extraction module is used to perform time alignment on the multidimensional interaction sequence data when the target object is not in the interaction defense state, and extract first-dimensional behavioral feature data and second-dimensional behavioral feature data based on the aligned multidimensional interaction sequence data. The first-dimensional behavioral feature data includes the target object's accuracy performance data in the emotion recognition task, and the second-dimensional behavioral feature data includes the target object's reaction speed or attention maintenance performance data during the task. The index calculation module is used to determine the hit status of the target element based on the first dimension behavioral feature data. In the case of no hit, it calculates the fluctuation range of the second dimension behavioral feature data within the preset first time window relative to the global historical data of the target object to obtain the composite interaction influence index. In the case of a hit, the composite interaction influence index is set to the preset default value. The parameter adjustment module is used to adjust the element presentation parameters of the interactive interface based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index. Example

[0040] An electronic device, including a memory and a processor; Memory, used to store computer programs; The processor is used to implement the above-described adaptive adjustment method for the interactive interface for patients with ASD+ADHD when executing a computer program. Example

[0041] A computer-readable storage medium storing a computer program, which, when read and run by a processor, implements the aforementioned adaptive adjustment method for the interactive interface suitable for ASD+ADHD patients.

[0042] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for adaptive adjustment of the user interface suitable for patients with ASD+ADHD, characterized in that, The method includes: Obtain target object attribute data, and render an interactive interface that masks negative feedback information and contains target elements based on the target object attribute data. The negative feedback information includes interface prompt elements that represent operation errors or task failures, and the target elements include graphic elements that allow the target object to perform touch or tracking operations. During the execution of tasks based on the interactive interface, multi-dimensional interactive sequence data is acquired, including image sequence data and operation trajectory data. The system monitors the multi-dimensional interaction sequence data in real time. If the operation frequency of the operation trajectory data is greater than a preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than a preset duration threshold, the system determines that the target object is in an interactive defense state and renders a soothing guidance interface. When the target object is not in the interactive defense state, the multidimensional interaction sequence data is time-aligned, and a first-dimensional behavioral feature data and a second-dimensional behavioral feature data are extracted based on the aligned multidimensional interaction sequence data. The first-dimensional behavioral feature data includes the target object's accuracy performance data in the emotion recognition task, and the second-dimensional behavioral feature data includes the target object's reaction speed or attention maintenance performance data during the task. Based on the first dimension behavioral feature data, the hit status of the target element is determined. In the case of no hit, the fluctuation range of the second dimension behavioral feature data relative to the global historical data of the target object within the preset first time window is calculated to obtain the composite interaction influence index. In the case of a hit, the composite interaction influence index is set to the preset default value. Based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index, the element presentation parameters of the interactive interface are adjusted.

2. The adaptive adjustment method for the interactive interface for ASD+ADHD patients according to claim 1, characterized in that, The step of performing time alignment on the multidimensional interaction sequence data and extracting first-dimensional behavioral feature data and second-dimensional behavioral feature data based on the aligned multidimensional interaction sequence data includes: The image sequence data and the operation trajectory data are time-stamped and aligned based on a preset sliding window to generate a multimodal time series matrix; The multimodal time series matrix is ​​input into a pre-trained cross-modal attention network for feature fusion; The cross-modal attention network outputs first-dimensional behavioral feature data and second-dimensional behavioral feature data. The first-dimensional behavioral feature data represents the hit probability of the target element, and the second-dimensional behavioral feature data represents the sustained attention decay slope and impulsive touch rate.

3. The adaptive adjustment method for the interactive interface for ASD+ADHD patients according to claim 1, characterized in that, The fluctuation range of the second-dimensional behavioral feature data within a preset first time window relative to the global historical data of the target object is used to obtain a composite interaction influence index, including: Determine the target time point in time when the first dimension of behavioral feature data represents the target element that was not hit; Extract the local attention decay slope from the target time point to the end of the preset first time window, and obtain the global average attention decay slope of the target object in the task; Calculate the ratio of the difference between the local attention decay slope and the global average attention decay slope; The composite interaction influence index is calculated based on the difference ratio and the preset weighting coefficient.

4. The adaptive adjustment method for the interactive interface for ASD+ADHD patients according to claim 3, characterized in that, The multidimensional interaction sequence data also includes physiological sensor data. Before calculating the composite interaction influence index based on the difference ratio and a preset weighting coefficient, the method further includes: Extract the rate of change in physiological arousal within a preset first time window from the physiological sensor data; The rate of change in physiological arousal is mapped to a weight value, and the weight value is used as the preset weighting coefficient.

5. The adaptive adjustment method for the interactive interface for ASD+ADHD patients according to claim 1, characterized in that, When the operation frequency of the operation trajectory data is greater than a preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than a preset duration threshold, determining that the target object is in an interactive defense state and rendering a soothing guidance interface includes: The reaction time of a preset number of consecutive operations within a preset second time window is calculated in real time, and the operation frequency of the operation trajectory data is determined based on the reaction time, and the coordinates of the gaze point in the image sequence data are extracted. If the operation frequency is greater than a preset frequency threshold, or if the duration of the gaze point coordinate deviating from the core task area in the interactive interface is greater than a preset duration threshold and there is no touch action, the target object is determined to be in the interactive defense state. Intercept the task progress of the interactive interface and reduce the display density of elements in the interactive interface; A soothing, low-cognitive-load guidance interface is overlaid on the interactive interface until the coordinates of the gaze point are detected to refocus on the core area of ​​the task.

6. The adaptive adjustment method for the interactive interface for ASD+ADHD patients according to claim 1, characterized in that, The step of adjusting the element presentation parameters of the interactive interface based on the first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index includes: The first-dimensional behavioral feature data, the second-dimensional behavioral feature data, and the composite interaction influence index are used to construct the current interaction state vector; The current interaction state vector is input into a preset reinforcement learning model, and the interface adjustment action command is output. Based on the interface adjustment action instructions, the dwell time, movement speed, or number of distracting items of the target element in the interactive interface are dynamically updated.

7. The adaptive adjustment method for the interactive interface for ASD+ADHD patients according to claim 6, characterized in that, The preset reinforcement learning model updates the policy in the following ways: The task completion score and interaction effort score of the target object after performing the task are obtained. The interaction effort score is determined based on the effective gaze duration and reasonable number of touches within a preset time. Based on the task completion score and the interaction effort score, a purely positive reward function value is calculated, which does not contain any negative penalty terms and is always greater than or equal to zero. Based on the purely positive reward function value, the network parameters of the reinforcement learning model are updated using the proximal policy optimization algorithm.

8. An adaptive adjustment system for the interactive interface suitable for patients with ASD+ADHD, characterized in that, The system includes: The interface rendering module is used to acquire target object attribute data and render an interactive interface that masks negative feedback information and contains target elements based on the target object attribute data. The negative feedback information includes interface prompt elements that represent operation errors or task failures. The target elements include graphic elements that allow the target object to perform touch or tracking operations. The data acquisition module is used to acquire multi-dimensional interaction sequence data during the execution of tasks based on the interactive interface. The multi-dimensional interaction sequence data includes image sequence data and operation trajectory data. The status monitoring module is used to monitor the multi-dimensional interaction sequence data in real time. If the operation frequency of the operation trajectory data is greater than a preset frequency threshold or the gaze deviation duration represented by the image sequence data is greater than a preset duration threshold, the module determines that the target object is in an interactive defense state and renders a soothing guidance interface. The feature extraction module is used to perform time alignment on the multidimensional interaction sequence data when the target object is not in the interaction defense state, and extract first-dimensional behavioral feature data and second-dimensional behavioral feature data based on the aligned multidimensional interaction sequence data. The first-dimensional behavioral feature data includes the accuracy performance data of the target object in the emotion recognition task, and the second-dimensional behavioral feature data includes the reaction speed or attention maintenance performance data of the target object during the task. The index calculation module is used to determine the hit status of the target element based on the first dimension behavioral feature data. In the case of no hit, it calculates the fluctuation range of the second dimension behavioral feature data relative to the global historical data of the target object within a preset first time window to obtain the composite interaction influence index. In the case of a hit, the composite interaction influence index is set to a preset default value. The parameter adjustment module is used to adjust the element presentation parameters of the interactive interface based on the first dimension behavioral feature data, the second dimension behavioral feature data, and the composite interaction influence index.

9. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the adaptive adjustment method for the interactive interface for patients with ASD+ADHD as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when read and run by a processor, implements the adaptive adjustment method for the interactive interface for ASD+ADHD patients as described in any one of claims 1-7.

Images