An interface dynamic layout optimization method and system based on user preferences

Abstract

This invention relates to the field of graphical user interface interaction technology, and more particularly to a method and system for dynamic layout optimization of interfaces based on user preferences. The invention provides a method for dynamic layout optimization of interfaces based on user preferences, comprising the following steps: S1: collecting behavioral sequence data and assistive technology requirement type data of the user during interaction with the graphical user interface; S2: inferring the user's implicit state data based on the behavioral sequence data and assistive technology requirement type data; S3: generating layout data to be deployed based on the implicit state data and the user's assistive technology requirement type data. This invention, by introducing interaction noise modeling for disabled users into dynamic layout optimization for the first time, utilizes a multi-type probability model to separate true preferences from task-irrelevant noise from highly biased behavioral sequences, significantly improving the layout accuracy for screen readers, users with motor disabilities, and users with cognitive impairments.

Description

Technical Field

[0001] This invention relates to the field of graphical user interface interaction technology, and in particular to a method and system for dynamic layout optimization of interfaces based on user preferences. Background Technology

[0002] With the popularization of mobile internet and smart terminals, the layout of graphical user interfaces has gradually evolved from static design to dynamic layout that adjusts in real time according to user behavior. Existing dynamic layout optimization methods are usually based on explicit interaction signals of user clicks, pauses, and swipes. They use recommendation algorithms, reinforcement learning, or responsive frameworks to dynamically adjust the arrangement order, size, and visibility of interface elements to improve click-through rates and conversion rates for ordinary users. However, existing technologies have the following significant drawbacks: Firstly, when disabled users use screen readers, magnifying glasses, voice control, or switch scanning devices, they generate a lot of behavioral noise that is unrelated to their actual preferences. Traditional preference modeling methods treat this noise as negative feedback, causing the layout optimization results to deviate significantly from the actual needs of disabled users. Secondly, the existing system is completely unaware of whether the user has enabled assistive technology, and it does not incorporate the key success criteria in WCAG into the constraints of layout generation. This easily leads to technical defects such as scaling failure, reading jumps, and keyboard traps, causing disabled users to immediately churn after dynamic layout changes. Thirdly, disabled users are highly sensitive to frequent changes in interface layout, while the existing system uses a uniform dynamic frequency and change amplitude, failing to adjust the step size and priority of layout changes according to the user's assistive technology type, current cognitive load, or state of frustration perception, creating a fundamental contradiction between personalization and stability. Fourthly, in the cold start phase without historical behavioral data, the existing system uses popular layouts or random trial strategies, which easily exposes the above defects, causing disabled users to abandon the system on their first use. Therefore, there is an urgent need to develop a method and system for dynamic interface layout optimization based on user preferences, which can sense interaction noise of disabled users, enforce accessibility hard constraints, and adaptively adjust the intensity of layout changes according to the user's real-time cognitive state. Summary of the Invention

[0003] To overcome the shortcomings of existing dynamic layout optimization methods, such as ignoring interaction noise of disabled users, lacking hard constraints on compatibility with assistive technologies, failing to adapt to the stability requirements of disabled users, and having a high churn rate of disabled users during the cold start phase, this invention provides a user preference-based dynamic interface layout optimization method and system.

[0004] The technical implementation of this invention is: a method for dynamic layout optimization of an interface based on user preferences, comprising the following steps: S1: Collect behavioral sequence data and assistive technology requirement type data of users during the interaction process between users and the graphical user interface; S2: Infer the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data; S3: Generate deployment layout data based on the implicit state data and the user's assistive technology requirement type data; S4: Construct a directed graph based on the layout data to be deployed and the current layout metadata, and obtain the optimal layout change sequence from the current layout to the target layout based on the directed graph; S5: Execute dynamic layout update of the interface according to the optimal layout change sequence.

[0005] Preferably, the collection of behavioral sequence data and user assistance technology requirement type data during the interaction between the user and the graphical user interface includes: the behavioral sequence data includes focus dwell time, touch jitter amplitude, repeated click interval, API call frequency, and focus rewind number, and the behavioral sequence data is merged into a single total sequence according to timestamps; the assistance technology requirement type data is data collected during the cold start guidance process, including at least one of the following: screen reader type, zoom function enabled status, voice control enabled status, switch scanning device connection status, and interface dynamic change sensitivity.

[0006] Preferably, the step of inferring the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data includes: taking the most recent... Each behavioral sequence data point is binned and discretized to obtain composite observation symbols. An observation sequence is constructed based on the composite observation symbols. ,in It is a multi-dimensional feature vector; the auxiliary type set is defined according to the auxiliary technology requirement type data. ,in Total number of auxiliary types; for each type Configure state transition probability matrix Observation probability matrix and initial state distribution ,in Indicates from hidden state Transition to hidden state The probability of; The number of hidden states; For the first The hidden state at any given moment; Indicates the hidden state Composite observation symbols observed below The probability of; The total number of categories for all composite observation symbols; This indicates that the user is in an invisible state at the start of the interaction. The probability of the state transition; implicit state data is obtained by forward recursion based on the state transition probability matrix, the observation probability matrix and the initial state distribution.

[0007] Preferably, the step of obtaining implicit state data through forward recursion based on the state transition probability matrix, observation probability matrix, and initial state distribution includes: defining forward variables. Indicated to Observed up to time And the current hidden state is Joint probability: ,initialization Perform recursion ,in ;time In hidden state The posterior probability is ,in Given the total joint probability of all hidden states in the entire observation sequence, the hidden state with the highest posterior probability is selected as the current hidden state. At the same time, obtain the state confidence of this state. The implicit state data includes the current implicit state and the state confidence level.

[0008] Preferably, generating the layout data to be deployed based on the implicit state data and the user's assistive technology requirement type data includes: defining an interface component library. , where each component Assign a D-dimensional semantic vector The initial state distribution vector is directly defined based on the current hidden state. If the state confidence level is lower than the state confidence threshold, then the initial state distribution vector is optimized to obtain a new state distribution vector. If the state confidence level is higher than the state confidence threshold, then the initial state distribution vector is the state distribution vector; the demand distribution vector is defined according to the auxiliary technology demand type data. , ,in The type of assistive technology required by the current user; It is a mapping matrix; It is a one-hot encoding function; The normalized exponential function is used; the total demand distribution vector is obtained based on the state distribution vector and the demand distribution vector. ; Obtain the layout data to be deployed based on the total demand distribution vector.

[0009] Preferably, obtaining the deployment layout data based on the total demand distribution vector includes: based on the total demand distribution vector... Constructing the hyperparameters of the Dirichlet prior distribution , ,in It is the concentration coefficient, and The hyperparameters are then modified based on WCAG hard constraints, which include scaling failure constraints, reading jump constraints, keyboard trap constraints, and touch target size constraints. The layout topic distribution is inferred based on these hyperparameters as input to the Dirichlet assignment model. The layout data to be deployed is obtained based on the inferred layout theme distribution. The layout data to be deployed includes the definite visibility, arrangement order weight, minimum touch size and focus priority of each component.

[0010] Preferably, the step of constructing a directed graph based on the layout data to be deployed and the current layout metadata includes: comparing the layout data to be deployed with the current layout metadata, and constructing a directed graph using each component in the interface and its visibility, arrangement order weight, minimum touch size, and focus priority as state dimensions. , where nodes Represents the state dimension of each component, edges This represents atomic transformation operations from one state dimension to another; the dynamic weight vector is determined using user-aided technology requirement type data and user implicit state data. For each edge Assigning a disability cost vector , ,in Changes in focus order; The distance to move to the target; Number of times the viewport unit is modified; This is a sign indicating a change in the escape mechanism; The reading order deviation is defined as the cumulative cost vector, which represents the sum of all types of disability costs accumulated in the path history. A penalty function is defined based on the cumulative cost vector. .

[0011] Preferably, obtaining the optimal layout change sequence from the current layout to the target layout based on the directed graph includes: according to the dynamic weight vector. Disability cost vector and penalty function Define edge Actual weights: ,in Represents the immediate cost of disability; Represents cumulative interference penalty. The penalty factor coefficient; based on the node To the target node Define the lower bound of disability cost The lower bound of the disability cost is used as a heuristic function to obtain the optimal layout change sequence through a look-ahead search algorithm. And maintain the cumulative cost vector in the state node.

[0012] Preferably, the step of performing dynamic layout updates of the interface based on the optimal layout change sequence includes: defining a sliding window of length M, and recording the total cost of the most recent M successfully executed optimal layout change sequences. And calculate its normalization cost. The number of atomic transformation operations in the defined sequence is... Define the average step size cost The stability coefficient is calculated based on the normalization cost and the average step size cost. , ,in The initial stability coefficient; The average step size cost threshold; ; A preset stability sensitivity coefficient is used; the actual number of atomic operations performed is obtained based on the stability coefficient. When performing dynamic layout updates to the interface, only the first elements of the optimal layout change sequence are executed. One operation is performed, and the remaining operations are temporarily stored and executed during the next interaction interval or after the user displays confirmation.

[0013] Preferably, the user preference-based dynamic interface layout optimization system further includes: The data acquisition module collects behavioral sequence data and assistive technology requirement type data of users during their interaction with the graphical user interface. The implicit state inference module infers the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data; The layout generation module generates layout data to be deployed based on the implicit state data and the user's auxiliary technology requirement type data; The layout change planning module constructs a directed graph based on the layout data to be deployed and the current layout metadata, and obtains the optimal layout change sequence from the current layout to the target layout based on the directed graph. The dynamic layout execution module performs dynamic layout updates of the interface based on the optimal layout change sequence.

[0014] This invention has the following advantages: Firstly, it introduces interactive noise modeling for users with disabilities into dynamic layout optimization. Utilizing multi-type probability models, it separates true preferences from task-irrelevant noise in highly biased behavioral sequences, significantly improving layout accuracy for screen readers, users with motor impairments, and users with cognitive impairments. Secondly, it achieves deep integration of WCAG accessibility hard constraints and layout generation. During the semantic mapping stage, it enforces rules prohibiting the definition of interactive targets in vw, forcing the edit distance between keyboard focus order and DOM tree order to be less than or equal to 1, and binding an Esc escape mechanism to all custom controls, ensuring the accessibility of dynamic layouts from the source. Furthermore, it pioneers… A disability-cost-oriented dynamic path planning and adaptive execution mechanism was developed. The disability cost weights were adaptively adjusted according to different assistive technology types and real-time hidden states, and a cumulative interference penalty function was introduced. This achieved the optimization goal transformation from minimum operation steps to minimum disability interference, significantly reducing the churn risk of users with cognitive impairments and autism. Finally, a complete cold-start disability-friendly strategy was constructed. By actively collecting assistive technology demand types, loading a pre-validated accessibility and safety baseline layout, and combining it with a low-risk exploration mechanism, users with disabilities can obtain a stable and operable interface on their first use. This effectively fills the gap in the support for users with disabilities in the cold start phase of existing dynamic layout systems. Attached Figure Description

[0015] Figure 1 A flowchart of a user-preference-based dynamic interface layout optimization method; Figure 2 This is a schematic diagram of a system structure for dynamic interface layout optimization based on user preferences. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1: A method for dynamic interface layout optimization based on user preferences, such as... Figure 1 As shown, it includes the following steps: S1: Collect behavioral sequence data and assistive technology requirement type data of users during the interaction process between users and the graphical user interface; The behavioral sequence data includes focus dwell time, touch jitter amplitude, repeated click interval, API call frequency, and focus rewind count. The behavioral sequence data is merged into a single total sequence according to timestamps. The auxiliary technology requirement type data is data collected during the cold start boot process and includes at least one of the following: screen reader type, zoom function enabled status, voice control enabled status, switch scanning device connection status, and interface dynamic change sensitivity.

[0018] It should also be noted that: the focus dwell time is the time the user spends on each focusable element; the touch jitter amplitude is the standard deviation of the distance between touch trajectory points; the repeated click interval is the time interval between two consecutive clicks on the same target; the API call frequency is the number of API calls per unit time; the focus rewind count is the number of times the focus moves in the opposite direction immediately after rewinding per unit time; the cold start guide process is the process the user enters when opening the app for the first time, collecting data in the form of a wizard-style dialog box: detecting the status of the screen reader and voice control through the browser API, detecting dynamic change sensitivity through CSS, and detecting the system settings API. The system reads the zoom level and the connection status of the on / off scanning device; the screen reader type is used to determine the focus order preference and the consistency requirements between the reading order and the DOM order; the zoom function enable status is used to determine the minimum touch target size threshold and the vw unit disable rule; the voice control enable status is used to determine the integrity requirements of the semantic tags of interface elements; the on / off scanning device connection status is used to determine the stability requirements of the focus traversal order; the interface dynamic change sensitivity is used to determine the upper limit constraint of the layout change range; the minimum touch target size threshold is dynamically set according to the user's zoom function enable status and the WCAG2.1 success standard.

[0019] S2: Infer the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data; Take the nearest Each behavioral sequence data point is binned and discretized to obtain composite observation symbols. An observation sequence is constructed based on the composite observation symbols. ,in It is a multi-dimensional feature vector; the auxiliary type set is defined according to the auxiliary technology requirement type data. ,in Total number of auxiliary types; for each type Configure state transition probability matrix Observation probability matrix and initial state distribution ,in Indicates from hidden state Transition to hidden state The probability of; The number of hidden states; For the first The hidden state at any given moment; Indicates the hidden state Composite observation symbols observed below The probability of; The total number of categories for all composite observation symbols; This indicates that the user is in an invisible state at the start of the interaction. The probability of the state transition; implicit state data is obtained by forward recursion based on the state transition probability matrix, the observation probability matrix and the initial state distribution.

[0020] Define forward variables Indicated to Observed up to time And the current hidden state is Joint probability: ,initialization Perform recursion ,in ;time In hidden state The posterior probability is ,in Given the total joint probability of all hidden states in the entire observation sequence, the hidden state with the highest posterior probability is selected as the current hidden state. At the same time, obtain the state confidence of this state. The implicit state data includes the current implicit state and the state confidence level.

[0021] It should also be noted that the nearest one is taken. Each behavioral sequence data point is used to construct an observation sequence in chronological order after obtaining composite observation symbols. The step size of the sliding window is set to For each type of assistive technology Behavioral sequence data of at least 500 users with similar disabilities were collected in advance during a standardized testing task. Accessibility experts then labeled the latent states of users in each time window, and initial parameters were calculated using maximum likelihood estimation. , , For each user, the Baum-Welch algorithm is used for personalized fine-tuning. The specific rules for binning discretization are based on the percentiles of each feature of the training samples to determine the binning boundaries of the behavioral sequence data: the focus dwell time is taken as the 30th and 70th percentiles of all samples as short / medium / long boundaries, the touch shake amplitude is taken as 5px and 20px as fixed boundaries, the repeated click interval is taken as a threshold of 500mx to distinguish between repeated clicks and normal clicks, the API call frequency is taken as 5 times / min and 20 times / min as boundaries, and the number of focus backs is taken as 0.9 and 1.1 as boundaries to distinguish between no backs, single backs, and multiple backs. For scenarios where training samples cannot be obtained, the above fixed values ​​are used as boundaries.

[0022] The assistance type is generated based on the user's assistance technology requirement type data. The assistance type is determined according to the assistance technology requirement type data selected by the user during the cold start guidance process. The assistance type includes no assistance, screen reader assistance, zoom assistance, language assistance, scanning assistance, and movement obstacle assistance. The latent state includes low cognitive load state, high cognitive load state, frustration perception state, and fatigue accumulation state.

[0023] Current hidden state The state confidence level of this state. To avoid layout instability caused by frequent changes in hidden states in a short period of time, a smoothing mechanism is introduced: a state history queue of length U is maintained. If the states predicted by consecutive windows are consistent, the state is output. If they are inconsistent, the state with the highest frequency in the history queue is taken as the output until consecutive consistent windows are re-established. The initial default value of the length U is 3.

[0024] S3: Generate deployment layout data based on the implicit state data and the user's assistive technology requirement type data; Define UI component library , where each component Assign a D-dimensional semantic vector The initial state distribution vector is directly defined based on the current hidden state. If the state confidence level is lower than the state confidence threshold, then the initial state distribution vector is optimized to obtain a new state distribution vector. If the state confidence level is higher than the state confidence threshold, then the initial state distribution vector is the state distribution vector; the demand distribution vector is defined according to the auxiliary technology demand type data. , ,in The type of assistive technology required by the current user; It is a mapping matrix; It is a one-hot encoding function; The normalized exponential function is used; the total demand distribution vector is obtained based on the state distribution vector and the demand distribution vector. ; Obtain the layout data to be deployed based on the total demand distribution vector.

[0025] Based on the total demand distribution vector Constructing the hyperparameters of the Dirichlet prior distribution , ,in It is the concentration coefficient, and The hyperparameters are then modified based on WCAG hard constraints, which include scaling failure constraints, reading jump constraints, keyboard trap constraints, and touch target size constraints. The layout topic distribution is inferred based on these hyperparameters as input to the Dirichlet assignment model. The layout data to be deployed is obtained based on the inferred layout theme distribution. The layout data to be deployed includes the definite visibility, arrangement order weight, minimum touch size and focus priority of each component.

[0026] It should also be noted that, Let D be a semantic vector; assign a fixed distribution vector to each hidden state, and define a neutral distribution. Taking a state of high cognitive load as an example, when D=4, an initial state distribution vector is assigned to it based on a fixed distribution vector. neutral distribution If the corresponding state confidence is lower than the state confidence threshold, then the state distribution vector... according to Calculations are performed to obtain the final state distribution vector; if the corresponding state confidence is higher than the state confidence threshold, then the state distribution vector... The state confidence threshold is dynamically calculated based on the statistical distribution of posterior probabilities in user history interactions, taking the median of the state confidence scores of the most recent N successful inferences. When no historical data is available, a default threshold of 0.6 is used. The mapping matrix is ​​obtained by offline collection of historical interaction data from users of various assistive technology types, using one-hot encoded assistive types as input and the distribution of layout topics actually selected or preferred by the user as a supervision signal, and trained using the cross-entropy loss function. The total demand distribution vector... ,in The preset fusion weights are dynamically set based on the type of auxiliary technology requirements: for users who rely on screen readers or on / off scanning devices, the weights are increased. To emphasize the need for assistive technologies, for latent states with high cognitive load or low confidence, increase... To enhance the influence of state distribution, the default value is... .

[0027] The concentration coefficient is customized based on the user's current implicit state confidence and historical interaction stability: the concentration coefficient is increased when the state confidence is high or the recent layout acceptance rate is good, and decreased otherwise. The default initial value is 10. The scaling failure constraint is that if the assistance type includes scaling assistance or motion obstacle assistance, then no component is allowed to use vw units to define its size, all themes containing vw size attributes are forced to have their probability set to zero, and the physical size of all interactive targets at 400% scaling must be greater than or equal to 44×44pt. The reading jump constraint is that if the assistance type is screen reader assistance, then the edit distance between the keyboard focus order and the natural order of the DOM tree must be <= 1 jump, and the screen reader reading order must be completely consistent with the visual presentation order. Consistent with the following rules, themes that alter the visual order using CSS properties are prohibited; the keyboard trap constraint requires that if the accessibility type includes screen reader accessibility, scanning accessibility, or motion impairment accessibility, any modal window, dropdown menu, or custom control must be bound to an Esc key escape mechanism, and the focus must be able to completely move out of the control using Tab; modal themes without an escape mechanism are prohibited; the touch target size constraint requires that if the accessibility type includes zoom accessibility or motion impairment accessibility, the minimum size of all clickable areas must not be less than 56×56pt; the hyperparameter correction based on WCAG hard constraints involves setting the hyperparameters corresponding to layout themes that violate WCAG hard constraints to zero; and the optimal layout theme distribution is inferred using a latent Dirichlet allocation model. From Dirichlet a priori Medium sampling layout topic distribution The initial value is set to all 1s for each component in the interface. ,from Sampling a topic According to the theme semantic vectors of components This determines the layout properties of the component; the Dirichlet distribution is a multivariate continuous distribution defined on a D-1 dimensional simplex, and its probability density function is... ,in It is a multivariate beta function; its key property is the expectation of the Dirichlet distribution. for: ,therefore Thematic distribution of layout under Dirichlet a priori conditions The expectation.

[0028] S4: Construct a directed graph based on the layout data to be deployed and the current layout metadata, and obtain the optimal layout change sequence from the current layout to the target layout based on the directed graph; The layout data to be deployed is compared with the current layout metadata. A directed graph is constructed using each component in the interface and its visibility, arrangement order weight, minimum touch size, and focus priority as state dimensions. , where nodes Represents the state dimension of each component, edges This represents atomic transformation operations from one state dimension to another; the dynamic weight vector is determined using user-aided technology requirement type data and user implicit state data. For each edge Assigning a disability cost vector , ,in Changes in focus order; The distance to move to the target; Number of times the viewport unit is modified; This is a sign indicating a change in the escape mechanism; The reading order deviation is defined as the cumulative cost vector, which represents the sum of all types of disability costs accumulated in the path history. A penalty function is defined based on the cumulative cost vector. .

[0029] According to the dynamic weight vector Disability cost vector and penalty function Define edge Actual weights: ,in Represents the immediate cost of disability; Represents cumulative interference penalty. The penalty factor coefficient; based on the node To the target node Define the lower bound of disability cost The lower bound of the disability cost is used as a heuristic function to obtain the optimal layout change sequence through a look-ahead search algorithm. And maintain the cumulative cost vector in the state node.

[0030] It should also be noted that the dynamic weight vector is determined by the user's assistive technology requirement type data and the user's implicit state data. It is a five-dimensional non-negative real vector, whose components correspond to the weights of focus order change, target movement distance, number of viewport unit modifications, escape mechanism change flag, and reading order deviation, respectively; through a comparative experiment on disabled users: users rated the interference degree of each type of disability cost in five typical layout change scenarios from 1 to 5 points, and after normalization, the median was used to establish a deterministic mapping table; the dynamic weight vector is obtained according to the deterministic mapping table.

[0031] The change in focus order is obtained by calculating the edit distance of the focus index sequence of all focusable elements in the interface before and after the atomic change operation, and normalizing the edit distance to the [0, 1] interval. The target movement distance is obtained by calculating the change in position of the same component in the screen coordinate system before and after the operation, and normalizing it by dividing its Euclidean distance by the length of the screen diagonal. If the component's visibility or size changes, the displacement of the component's center point is used. The number of times the viewport unit is modified is obtained by counting the number of instances of adding, deleting, or modifying the viewport unit in the CSS property during the atomic change operation. The escape mechanism change flag is obtained by checking the escape behavior of any custom control being bound or unbound by the Esc key to exit, close the pop-up, or cancel the operation before and after the operation. If the escape mechanism state of any control changes, the flag is set to 1; otherwise, it is 0. The reading order deviation is obtained by calculating the Kendall tau distance of the screen reader's sequence before and after the operation and dividing it by the maximum inversion logarithm. The normalized deviation is the reading order deviation. Penalty function ,in For the first The cost of disability; For the cumulative threshold, different cumulative upper limits are preset for each type of disability cost based on the user's assistive technology type and hidden state. The default value is the default threshold set recommended by WCAG. To maximize the penalty, a fixed semantic upper limit is preset for the cost of each type of disability, including a focus order change of 2, a target movement distance of 2.5, a viewport unit modification count of 3, an escape mechanism change of 5, and a reading order deviation of 2. All values ​​are determined based on empirical risk analysis and do not rely on real-time data. To control sensitivity, a default initial value of 0.8 is uniformly set.

[0032] Cumulative interference penalty middle The penalty factor coefficient uses the default value. The lower limit of the cost of disability Defined as ,in This represents the minimum value in each dimension of the dynamic weight vector; The number of attribute dimensions that differ between the current layout and the target layout; In order to eliminate the first A set of atomic operations with distinct characteristics; initialization of the optimal layout change sequence. , start node Join, its , ,in Represents the distance from the starting node to the node. The actual cumulative cost, i.e., from the starting point to the node. The sum of the residual costs of all edges on the path; evaluated by the evaluation function. Pop the smallest node ,like If the target state is reached, stop and reconstruct the path; otherwise, generate all atomic operations. For each successor node Calculate its immediate cost Cumulative penalty items actual weight of the edge ,renew , , then calculate , and insert If the optimal layout change sequence If the node is empty or the maximum search steps are exceeded, progressive target relaxation is triggered: the node... For attributes that are difficult to achieve in the middle, relax the search, search again, and record the relaxed attributes and the extent of relaxation. Add a warning log to the final generated optimal layout change sequence. The maximum number of search steps is calculated by multiplying the square root of the number of interactive components in the interface by 10 and then rounding up to the nearest integer, while not exceeding the preset upper limit of 200.

[0033] S5: Execute dynamic layout update of the interface according to the optimal layout change sequence.

[0034] Define a sliding window of length M to record the total cost of the most recent M successfully executed optimal layout change sequences. And calculate its normalization cost. The number of atomic transformation operations in the defined sequence is... Define the average step size cost The stability coefficient is calculated based on the normalization cost and the average step size cost. , ,in The initial stability coefficient; The average step size cost threshold; ; A preset stability sensitivity coefficient is used; the actual number of atomic operations performed is obtained based on the stability coefficient. When performing dynamic layout updates to the interface, only the first elements of the optimal layout change sequence are executed. One operation is performed, and the remaining operations are temporarily stored and executed during the next interaction interval or after the user displays confirmation.

[0035] It should also be noted that the total cost of the optimal layout change sequence ,in For the first Subatomic transformation operations; normalization cost ,in This represents the average of the total historical costs within the window. It is a very small positive number; the initial stability coefficient ,in State confidence; The state confidence threshold is defined as follows: the average step size cost threshold is set to 0.2 times the number of interactive components on the current interface and is limited to between 5 and 30; the preset stability sensitivity coefficient is defined as follows: A preset fixed coefficient value is set based on the user's current hidden state type, with an initial default value of 0.5; the preset stable sensitivity coefficient A fixed coefficient value is preset based on the sensitivity of the interface to dynamic changes collected by the user during the cold start boot process, with an initial default value of 0.3; the actual number of atomic operations executed. When performing dynamic layout updates of the interface, if the stability coefficient... If the stability coefficient is less than the preset stability threshold, only one atomic operation will be performed in this instance, and the user will be prompted non-intrusively: "The interface is being optimized for you, do you want to continue?" If the value exceeds the preset stability threshold, then execute all actions within the current interaction interval. One operation; the preset stability threshold adopts the default value of 0.2.

[0036] Example 2: Based on Example 1, a user-preference-based dynamic interface layout optimization system, such as... Figure 2 As shown, it also includes: The data acquisition module collects behavioral sequence data and assistive technology requirement type data of users during their interaction with the graphical user interface. The implicit state inference module infers the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data; The layout generation module generates layout data to be deployed based on the implicit state data and the user's auxiliary technology requirement type data; The layout change planning module constructs a directed graph based on the layout data to be deployed and the current layout metadata, and obtains the optimal layout change sequence from the current layout to the target layout based on the directed graph. The dynamic layout execution module performs dynamic layout updates of the interface based on the optimal layout change sequence.

[0037] The above embodiments are provided for those skilled in the art to implement or use the present invention. Those skilled in the art can make various modifications or changes to the above embodiments without departing from the inventive concept of the present invention. Therefore, the protection scope of the present invention is not limited to the above embodiments, but should be the maximum scope that conforms to the innovative features mentioned in the claims.

Claims

1. A method for dynamic layout optimization of user interfaces based on user preferences, characterized in that, Includes the following steps: S1: Collect behavioral sequence data and assistive technology requirement type data of users during the interaction process between users and the graphical user interface; S2: Infer the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data; S3: Generate deployment layout data based on the implicit state data and the user's assistive technology requirement type data; S4: Construct a directed graph based on the layout data to be deployed and the current layout metadata, and obtain the optimal layout change sequence from the current layout to the target layout based on the directed graph; S5: Execute dynamic layout update of the interface according to the optimal layout change sequence.

2. The user preference-based dynamic interface layout optimization method according to claim 1, characterized in that, The collection of behavioral sequence data and user assistance technology requirement type data during the interaction between the user and the graphical user interface includes: the behavioral sequence data includes focus dwell time, touch jitter amplitude, repeated click interval, API call frequency, and focus rewind number, and the behavioral sequence data is merged into a single total sequence according to timestamps; the assistance technology requirement type data is data collected during the cold start boot process, including at least one of the following: screen reader type, zoom function enabled status, voice control enabled status, switch scanning device connection status, and interface dynamic change sensitivity.

3. The user preference-based dynamic interface layout optimization method according to claim 1, characterized in that, The process of inferring the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data includes: taking the most recent... Each behavioral sequence data point is binned and discretized to obtain composite observation symbols. An observation sequence is constructed based on the composite observation symbols. ,in It is a multi-dimensional feature vector; the auxiliary type set is defined according to the auxiliary technology requirement type data. ,in Total number of auxiliary types; for each type Configure state transition probability matrix Observation probability matrix and initial state distribution ,in Indicates from hidden state Transition to hidden state The probability of; The number of hidden states; For the first The hidden state at any given moment; Indicates the hidden state Composite observation symbols observed below The probability of; The total number of categories for all composite observation symbols; This indicates that the user is in an invisible state at the start of the interaction. The probability of the state transition; implicit state data is obtained by forward recursion based on the state transition probability matrix, the observation probability matrix and the initial state distribution.

4. The user preference-based dynamic interface layout optimization method according to claim 3, characterized in that, The step of obtaining implicit state data through forward recursion based on the state transition probability matrix, observation probability matrix, and initial state distribution includes: defining forward variables. Indicated to Observed up to time And the current hidden state is Joint probability: ,initialization Perform recursion ,in ;time In hidden state The posterior probability is ,in Given the total joint probability of all hidden states in the entire observation sequence, the hidden state with the highest posterior probability is selected as the current hidden state. At the same time, obtain the state confidence of this state. The implicit state data includes the current implicit state and the state confidence level.

5. A user preference-based dynamic interface layout optimization method according to claim 4, characterized in that, The step of generating the layout data to be deployed based on the implicit state data and the user's accessibility technology requirement type data includes: defining the interface component library. , where each component Assign a D-dimensional semantic vector The initial state distribution vector is directly defined based on the current hidden state. If the state confidence level is lower than the state confidence threshold, then the initial state distribution vector is optimized to obtain a new state distribution vector. If the state confidence level is higher than the state confidence threshold, then the initial state distribution vector is the state distribution vector; the demand distribution vector is defined according to the auxiliary technology demand type data. , ,in The type of assistive technology required by the current user; It is a mapping matrix; It is a one-hot encoding function; The normalized exponential function is used; the total demand distribution vector is obtained based on the state distribution vector and the demand distribution vector. ; Obtain the layout data to be deployed based on the total demand distribution vector.

6. The user preference-based dynamic interface layout optimization method according to claim 1, characterized in that, The step of obtaining the deployment layout data based on the total demand distribution vector includes: based on the total demand distribution vector Constructing the hyperparameters of the Dirichlet prior distribution , ,in It is the concentration coefficient, and The hyperparameters are then modified based on WCAG hard constraints, which include scaling failure constraints, reading jump constraints, keyboard trap constraints, and touch target size constraints. The layout topic distribution is inferred based on these hyperparameters as input to the Dirichlet assignment model. The layout data to be deployed is obtained based on the inferred layout theme distribution. The layout data to be deployed includes the definite visibility, arrangement order weight, minimum touch size and focus priority of each component.

7. A user preference-based dynamic interface layout optimization method according to claim 1, characterized in that, The step of constructing a directed graph based on the layout data to be deployed and the current layout metadata includes: comparing the layout data to be deployed with the current layout metadata, and constructing a directed graph using each component in the interface and its visibility, arrangement order weight, minimum touch size, and focus priority as state dimensions. , where nodes Represents the state dimension of each component, edges This represents atomic transformation operations from one state dimension to another; the dynamic weight vector is determined using user-aided technology requirement type data and user implicit state data. For each edge Assigning a disability cost vector , ,in Changes in focus order; The distance to move to the target; Number of times the viewport unit is modified; This is a sign indicating a change in the escape mechanism; The reading order deviation is defined as the cumulative cost vector, which represents the sum of all types of disability costs accumulated in the path history. A penalty function is defined based on the cumulative cost vector. .

8. A user preference-based dynamic interface layout optimization method according to claim 7, characterized in that, The step of obtaining the optimal layout change sequence from the current layout to the target layout based on the directed graph includes: according to the dynamic weight vector. Disability cost vector and penalty function Define edge Actual weights: ,in Represents the immediate cost of disability; Represents cumulative interference penalty. The penalty factor coefficient; based on the node To the target node Define the lower bound of disability cost The lower bound of the disability cost is used as a heuristic function to obtain the optimal layout change sequence through a look-ahead search algorithm. And maintain the cumulative cost vector in the state node.

9. A user preference-based dynamic interface layout optimization method according to claim 1, characterized in that, The step of performing dynamic layout updates of the interface based on the optimal layout change sequence includes: defining a sliding window of length M, and recording the total cost of the most recent M successfully executed optimal layout change sequences. And calculate its normalization cost. The number of atomic transformation operations in the defined sequence is... Define the average step size cost The stability coefficient is calculated based on the normalization cost and the average step size cost. , ,in The initial stability coefficient; The average step size cost threshold; ; A preset stability sensitivity coefficient is used; the actual number of atomic operations performed is obtained based on the stability coefficient. When performing dynamic layout updates to the interface, only the first elements of the optimal layout change sequence are executed. One operation is performed, and the remaining operations are temporarily stored and executed during the next interaction interval or after the user displays confirmation.

10. A user-preference-based dynamic interface layout optimization system, used to implement the user-preference-based dynamic interface layout optimization method described in any one of 1-9, characterized in that it further... include: The data acquisition module collects behavioral sequence data and assistive technology requirement type data of users during their interaction with the graphical user interface. The implicit state inference module infers the user's implicit state data based on the behavioral sequence data and the auxiliary technology requirement type data; The layout generation module generates layout data to be deployed based on the implicit state data and the user's auxiliary technology requirement type data; The layout change planning module constructs a directed graph based on the layout data to be deployed and the current layout metadata, and obtains the optimal layout change sequence from the current layout to the target layout based on the directed graph. The dynamic layout execution module performs dynamic layout updates of the interface based on the optimal layout change sequence.

Images