A dynamic web interface modeling method and system based on reinforcement learning

Abstract

The present invention proposes a dynamic web interface modeling method and system based on reinforcement learning, the method comprising: defining a data structure for each element on the web interface and binding an event, processing and recording the event data when a triggering event occurs; defining a dynamic data set structure, based on user input and interface element status; collecting real-time interface element status information, binding an event processing function for the real-time interface element, standardizing the data, and storing it locally or on the server; defining a reinforcement learning model to optimize interface linkage rules; and designing a model-independent meta-learning model to adaptively configure the interface. The present invention automatically optimizes and adjusts the linkage rules of web interface elements through reinforcement learning, self-supervised learning, and meta-learning algorithms, significantly improving the system's intelligence level and user experience.

Description

Technical Field

[0001] The present invention belongs to the field of dynamic web interface modeling, and in particular relates to a dynamic web interface modeling method and system based on reinforcement learning. Background Art

[0002] In modern web development, with the increasing demand for user interaction and the complexity of business logic, dynamically adjusting the linkage rules of interface elements has become a very common development application scenario. When a user enters or changes a value in an interface element, the value or state of other interface elements in the current interface will be dynamically changed according to certain rules to implement complex business logic. This dynamic linkage function has become an indispensable part in many actual business scenarios, such as automatic form filling, data filtering, and real-time data display.

[0003] There are two main traditional solutions: hard coding and scripting. The hard coding method is to enumerate all user scenario rules and directly implement the scenario cascade rules through code. The main problem with this method is that it is difficult to exhaust all user scenarios, and when user needs change or new scenarios are added, the code needs to be adjusted frequently, which has high maintenance costs and poor flexibility. On the other hand, the scripting method allows users to directly write JavaScript scripts to control the data linkage of web interface elements. This method is more adaptable than the hard coding method, but it has higher technical requirements for users. Users need to be familiar with script writing, and when the scenario is complex, script writing and maintenance are more difficult. In addition, both methods lack intelligence and automation, and cannot dynamically optimize and adjust the linkage rules of interface elements according to the actual operations and needs of users.

[0004] These problems are particularly prominent in low-code platforms. The goal of low-code platforms is to simplify the development process so that users without programming experience can build complex applications. However, the dynamic interface linkage function in low-code platforms currently still relies on certain hard-coding or scripting methods, and fails to fully utilize modern artificial intelligence technology to improve user experience and system intelligence. Therefore, a new method is needed that can combine modern artificial intelligence algorithms to automatically optimize and adjust the linkage rules of interface elements, lower the user's technical threshold, and improve the flexibility and intelligence of the system. Summary of the invention

[0005] The purpose of the present invention is to propose a dynamic web interface modeling method and system based on reinforcement learning, which automatically optimizes and adjusts the linkage rules of web interface elements through reinforcement learning, self-supervised learning and meta-learning algorithms, thereby significantly improving the intelligence level of the system and user experience.

[0006] In order to achieve the above object, the present invention provides a dynamic web interface modeling method based on reinforcement learning, the method comprising:

[0007] S1. Define a data structure for each element on the web interface and bind an event, wherein each element is bound to a single or multiple events. When a trigger event occurs, the event data is processed and recorded, and the event data is transmitted in JSON format. At the same time, a data transmission standard interface is defined to transmit the processed time data;

[0008] S2. Define the dynamic data set structure, dynamically generate SQL query scripts and parameter configurations based on user input and interface element status, optimize the dynamic data set using a reinforcement learning-based optimization method, transfer the data set using the SON format, and define a standard interface for data transfer of the dynamic data set; wherein the dynamic data set includes SQL scripts, parameters, and return data;

[0009] S3. Collect the status information of real-time interface elements, and then use a unified data structure to represent the status information. At the same time, bind event processing functions to real-time interface elements, introduce data processing functions, standardize the data, and store it locally or on the server. Then use JSON format to transfer data, define a standard interface to call real-time data, design a distributed data collection system architecture to collect and manage data, and then define a data analysis function to analyze and process the data collected in real time;

[0010] S4. Define the reinforcement learning model optimization interface linkage rules, introduce the action-state mapping matrix based on the reinforcement learning model, select the optimal action through matrix operation, and feed back the data set query results to the reinforcement learning model for strategy update through the dynamic data set feedback mechanism; wherein, the reinforcement learning includes state representation, action space, reward function and strategy network;

[0011] The state is represented by S={ElementState, UserAction, DynamicDataset}, where ElementState represents the state information of the interface element, UserAction represents the user's operation data, and DynamicDataset represents the state of the dynamic data set;

[0012] The action space is A={change_value, change_style, trigger_event, query_dataset}, where change_value represents the action of changing the element value, change_style represents the action of changing the element style, trigger_event represents the action of triggering the element event, and query_dataset represents the action of querying the dynamic dataset;

[0013] The reward function is R t =f(user_satisfaction,system_performance,task_completion), where, R t is the reward value at time step t, user_satisfaction represents user satisfaction, system_performance represents system performance, and task_completion represents task completion;

[0014] S5. Design a model-independent meta-learning model to adaptively configure the interface, calculate the optimal configuration action based on the model-independent meta-learning model, and design a meta-learning distributed system architecture to collect and manage data. Introduce an adaptive feedback mechanism to feed back the configuration results to the model-independent meta-learning model for strategy update and optimize the web interface configuration.

[0015] Preferably, the binding of event data in S1 also introduces event weights, and each event data is assigned different weights according to its importance and frequency, as shown below:

[0016]

[0017] Among them, w ij represents the weight of the jth event triggered by the ith element, f ij represents the frequency of the jth event triggered by the ith element, and m is the total number of events that can be triggered by the ith element.

[0018] Preferably, in the S1, the event data processing further introduces an event influence matrix to quantify the influence of the event, and the event influence matrix is ​​defined as:

[0019] I={I ij}

[0020] Among them, I ij Indicates the influence of the jth event triggered by the i-th element;

[0021] I ij =w ij ·R ij

[0022] Among them, R ij Indicates the reward value of the jth event triggered by the i-th element, which depends on the contribution of the event to the interface linkage;

[0023] The update formula of event influence is:

[0024]

[0025] in, represents the influence of the jth event triggered by the i-th element in the t+1th iteration, It represents the influence of the jth event triggered by the i-th element in the t-th iteration. α is the learning rate, which ranges from [0,1].

[0026] Preferably, metadata definition is introduced in S2, including SQL script template and parameter configuration, wherein the structure of the metadata is expressed as follows:

[0027] Metadata={template_id,sql_template,param_config}

[0028] Among them, template_id is the unique identifier of the template; sql_template is the template of the SQL query script; param_config is the parameter configuration, including parameter name, type and default value.

[0029] Preferably, in S2, the objective function for optimizing the dynamic data set is:

[0030]

[0031] Among them, J(θ) represents the optimization objective function, π θ represents the reinforcement learning policy, τ represents the trajectory sampled from the policy, T represents the time step, γ represents the discount factor, R t represents the reward at time step t;

[0032] Wherein, the optimization process is:

[0033] Initialize reinforcement learning policy π θ ;

[0034] Calculate the value of the current strategy through the strategy evaluation function;

[0035] According to the evaluation results, optimize the policy parameters θ to maximize the objective function J(θ);

[0036] Verify the optimized data set.

[0037] Preferably, the distributed data acquisition system architecture includes a front-end data acquisition module, a data processing module and a data storage module;

[0038] Among them, the front-end data collection module is used to implement event binding and status collection on the front end, and capture user operations and interface element status through JavaScript code;

[0039] The data processing module is used to implement data processing functions locally or on the server side, and to verify and standardize the collected data;

[0040] The data storage module is used to store the processed data in a local database or a cloud database to ensure data persistence and security.

[0041] Preferably, in S4, a proximal strategy optimization algorithm is used to optimize reinforcement learning, comprising the following steps:

[0042] The advantage function is calculated and expressed as follows:

[0043]

[0044] Among them, A t represents the advantage function at time step t, V(S t ) indicates state S t The value function of

[0045] The update policy network is expressed as follows:

[0046]

[0047] Among them, α represents the learning rate, π θ Indicates the current strategy, represents the old strategy, ∈ represents the clipping parameter. Preferably, the action-state mapping matrix is ​​expressed as follows:

[0048] M={M ij}

[0049] Among them, M ij Represents the mapping relationship between state i and action j, which is expressed as follows:

[0050] M ij =P(A j |S i )

[0051] By calculating the mapping matrix M, we select the action that maximizes the expected reward, which is expressed as follows:

[0052]

[0053] Preferably, the training steps of the model-independent meta-learning model include:

[0054] Sample a set of tasks from the task distribution p(T):

[0055] {T i}~p(T)

[0056] Among them, T i is the i-th task, p(T) is the task distribution;

[0057] Adaptive learning rate and weighted loss function are introduced, and the adaptive inner loop update formula is defined as follows:

[0058]

[0059] Among them, α i Represents task T i The adaptive learning rate, Represents task T i The gradient of the loss function, λ∥θ∥ 2 represents the L2 regularization term, λ represents the regularization coefficient; G ii Represents the cumulative gradient square matrix of the diagonal elements, which is used for adaptive learning rate adjustment; δ represents a small value to prevent the denominator from being zero;

[0060] The meta-learning rate and weighted total loss function are introduced to calculate the importance of different tasks, and the adaptive outer loop update formula is defined:

[0061]

[0062] Where β is the meta-learning rate, w i It is task T i The weights of , used to weight the total loss, It is task T i The loss gradient is λ∥θ∥ 2 is the L2 regularization term, λ is the regularization coefficient, H ii is the square matrix of accumulated gradients with diagonal elements, used for adaptive learning rate adjustment, and δ is a small value to prevent the denominator from being zero.

[0063] In another aspect of the present invention, a dynamic web interface modeling system based on reinforcement learning is provided, the system comprising:

[0064] The web interface initialization unit is used to define the data structure and bind events for each element on the web interface, wherein each element is bound to one or more events, and when a trigger event occurs, the event data is processed and recorded, and the event data is transmitted in JSON format, and a data transmission standard interface is defined to transmit the processed time data;

[0065] A dynamic data set construction unit is used to define the dynamic data set structure, dynamically generate SQL query scripts and parameter configurations according to user input and interface element status, optimize the dynamic data set using a reinforcement learning-based optimization method, transfer the data set using the SON format, and define a standard interface for data transfer of the dynamic data set; wherein the dynamic data set includes SQL scripts, parameters, and return data;

[0066] The analysis unit of the interface element is used to collect the status information of the real-time interface elements, and then use a unified data structure to represent the status information. At the same time, the event processing function is bound to the real-time interface elements, and the data processing function is introduced to standardize the data and store it locally or on the server. Then the JSON format is used to transmit the data, and a standard interface is defined to call the real-time data. A distributed data collection system architecture is designed to collect and manage the data, and then a data analysis function is defined to analyze and process the data collected in real time;

[0067] An optimization interface unit is used to define the linkage rules of the reinforcement learning model optimization interface, and introduce an action-state mapping matrix based on the reinforcement learning model, select the optimal action through matrix operations, and feed back the data set query results to the reinforcement learning model for strategy update through a dynamic data set feedback mechanism; wherein the reinforcement learning includes state representation, action space, reward function and strategy network;

[0068] The state is represented by S={ElementState, UserAction, DynamicDataset}, where ElementState represents the state information of the interface element, UserAction represents the user's operation data, and DynamicDataset represents the state of the dynamic data set;

[0069] The action space is A={change_value, change_style, trigger_event, query_dataset}, where change_value represents the action of changing the element value, change_style represents the action of changing the element style, trigger_event represents the action of triggering the element event, and query_dataset represents the action of querying the dynamic dataset;

[0070] The reward function is R t =f(user_satisfaction,system_performance,task_completion), where, R tis the reward value at time step t, user_satisfaction represents user satisfaction, system_performance represents system performance, and task_completion represents task completion;

[0071] The adaptive configuration unit is used to design a model-independent meta-learning model to adaptively configure the interface. The optimal configuration action is calculated based on the model-independent meta-learning model. At the same time, a meta-learning distributed system architecture is designed to collect and manage data. An adaptive feedback mechanism is introduced to feed back the configuration results to the model-independent meta-learning model for strategy update and optimization of the web interface configuration.

[0072] The beneficial technical effects of the present invention are at least as follows:

[0073] (1) The present invention uses a reinforcement learning algorithm, and can automatically optimize the linkage rules of a dynamic web interface based on the user's operation data. The reinforcement learning agent adjusts the attributes and linkage rules of the interface elements according to the state of the interface elements and the user's operation, and evaluates the effects of these adjustments through a reward function. This can avoid the rigidity and high maintenance cost of the traditional hard-coding method, and also reduce the technical threshold for users to write scripts.

[0074] (2) The present invention uses a self-supervised learning method to automatically generate and optimize dynamic data sets. The self-supervised learning model can process and understand the data input by the user, and automatically generate SQL scripts and parameter configurations based on the user's business needs and data characteristics. This not only improves the efficiency and accuracy of data linkage, but also reduces the dependence on the user's technical capabilities, allowing users of low-code platforms to easily implement complex data linkage functions.

[0075] (3) The present invention uses a meta-learning algorithm to achieve adaptive configuration and optimization of dynamic interfaces. The meta-learning model can quickly adapt to the needs and operating habits of different users, and adjust and optimize the configuration of interface elements based on the user's operating data and feedback. This innovation not only improves the intelligence level of the system, allowing the interface to quickly respond to the needs of different users, but also improves the user experience, truly realizing the ease of use and flexibility of the low-code platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0077] Figure 1 This is a flow chart of a dynamic web interface modeling method based on reinforcement learning of the present invention.

[0078] Figure 2 This is a framework diagram of a dynamic web interface modeling system based on reinforcement learning in the present invention. DETAILED DESCRIPTION

[0079] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0080] like Figure 1 As shown, an embodiment of the present invention provides a dynamic web interface modeling method based on reinforcement learning, and the method includes the following steps S1-S5:

[0081] S1. Define the data structure and bind events for each element on the web interface, where each element is bound to one or more events. When a triggering event occurs, the event data is processed and recorded, and the event data is transmitted in JSON format. At the same time, a data transmission standard interface is defined to transmit the processed time data.

[0082] Specifically, the following steps are included:

[0083] First, you need to define each element on the interface. These elements can be input boxes, drop-down menus, buttons, etc. Use a unified data structure to represent these interface elements and ensure that they conform to standardized definitions. Define the data structure of interface elements:

[0084] Element={id,type,value,state} (1)

[0085] Among them: id is the unique identifier of the element, type is the type of the element (such as input box, drop-down menu, button, etc.), value is the current value of the element, and state is the state of the element (such as display, hide, enable, disable, etc.).

[0086] It is understandable that the definition of interface elements can be further refined and more attributes can be added to adapt to complex scenarios in actual applications, such as: position: the position coordinates of the element in the interface; style: the style attributes of the element; event_handlers: the event handling function bound to the element; the detailed definition of interface elements is as follows:

[0087] Element={id,type,value,state,position,style,event_handlers} (2)

[0088] Furthermore, in order to capture user operations and trigger corresponding linkage rules, events need to be bound to these interface elements. Each element can be bound to multiple events, depending on the type of element. Define the data structure of the event:

[0089] Event={element_id,event_type,timestamp,value} (3)

[0090] Where: element_id is the unique identifier of the element that triggers the event, event_type is the type of event (such as change, click, focus, etc.), timestamp is the timestamp of the event, and value is the value of the element when the event is triggered.

[0091] In order to achieve more intelligent event binding, the concept of event weight is introduced. Each event can be assigned different weights according to its importance and frequency. Define event weight:

[0092]

[0093] Where: w ij represents the weight of the jth event triggered by the ith element, f ij represents the frequency of the jth event triggered by the ith element, and m is the total number of events that can be triggered by the ith element.

[0094] Furthermore, when an event is triggered, the event data needs to be passed to the reinforcement learning model in the subsequent steps for processing. Event data can be uniformly processed and recorded through an event processing function. Event processing function:

[0095] handleEvent(event)(5)

[0096] Where event is an object containing event details.

[0097] It can be understood that the processing process includes the following steps:

[0098] Capture events: When users operate on the interface, corresponding events are triggered.

[0099] Log event data: Logs detailed information about the event, including element ID, event type, timestamp, and current value.

[0100] Pass event data: Pass the recorded event data to the subsequent reinforcement learning model.

[0101] Furthermore, the event influence matrix is ​​introduced to quantify the influence of events and guide the subsequent optimization of linkage rules. Define the event influence matrix:

[0102] I={I ij} (6)

[0103] Where: I ij Indicates the influence of the jth event triggered by the i-th element;

[0104] I ij =w ij ·R ij

[0105] Among them, R ij Indicates the reward value of the jth event triggered by the i-th element, which depends on the contribution of the event to the interface linkage;

[0106] The update formula of event influence is:

[0107]

[0108] in: It represents the influence of the jth event triggered by the i-th element in the t-th iteration. α is the learning rate, which ranges from [0,1].

[0109] Furthermore, in order to ensure the effective transmission of data, it is necessary to define the interface and data format of data transmission. In the present invention, JSON format is used to transmit event data, and a standard interface is defined for subsequent steps to call. Define event data format:

[0110] event_data={element_id,event_type,timestamp,value,weight,influence}(8)

[0111] Among them: element_id is the unique identifier of the element that triggers the event, event_type is the type of event (such as change, click, focus, etc.), timestamp is the timestamp of the event, value is the value of the element when the event is triggered, weight is the weight of the event, and influence is the influence of the event.

[0112] S2. Define the dynamic data set structure, dynamically generate SQL query scripts and parameter configurations based on user input and interface element status, optimize the dynamic data set using a reinforcement learning-based optimization method, use the SON format to transfer the data set, and define a standard interface for data transfer for the dynamic data set; wherein the dynamic data set includes SQL scripts, parameters, and return data.

[0113] Specifically, define the structure of the dynamic dataset. Each dynamic dataset consists of SQL scripts, parameters, and return data. In order to adapt to different scenarios, we use a unified data structure to represent dynamic datasets. Define the data structure of the dynamic dataset:

[0114] DynamicDataset={id,sql_script,parameters,return_data} (9)

[0115] Where: id is the unique identifier of the data set, sql_script is the SQL query script, parameters is the parameter list of the SQL query, and return_data is the data field returned by the SQL query

[0116] Furthermore, in order to improve flexibility, metadata definition is introduced, including SQL script templates and parameter configuration. Define the metadata structure:

[0117] Metadata={template_id,sql_template,param_config} (10)

[0118] Where: template_id is the unique identifier of the template, sql_template is the template of the SQL query script, and param_config is the parameter configuration, including the parameter name, type, and default value.

[0119] Dynamically generate SQL query scripts and parameter configurations based on user input and interface element status. We use self-supervised learning models to automatically generate and optimize these data sets. Define the input and output of the self-supervised learning model:

[0120] Input={element_state,user_input} (11)

[0121] Among them: element_state is the current state of the interface element, user_input is the user's input data;

[0122] Output={generated_sql,generated_params} (12)

[0123] Among them: generated_sql is the generated SQL query script, generated_params is the generated parameter configuration.

[0124] Furthermore, the generation process includes the following steps:

[0125] Feature extraction: extract features from input data and construct feature vectors.

[0126] Model reasoning: Generate SQL scripts and parameter configurations through self-supervised learning models.

[0127] Result verification: Verify the generated SQL scripts and parameter configurations to ensure their correctness and effectiveness.

[0128] Furthermore, in order to ensure that the generated data set can efficiently support interface linkage, the dynamic data set needs to be optimized. We use an optimization method based on reinforcement learning to improve the quality of the data set through continuous iteration and learning. Define the optimization objective function:

[0129]

[0130] Where: J(θ) is the optimization objective function, π θ is the reinforcement learning policy, τ is the trajectory sampled from the policy, T is the time step, γ is the discount factor, and R t is the reward at time step t.

[0131] The optimization process includes the following steps:

[0132] Initialization strategy: Initialize the reinforcement learning strategy π θ .

[0133] Strategy evaluation: Calculate the value of the current strategy through the strategy evaluation function.

[0134] Strategy optimization: Based on the evaluation results, optimize the strategy parameters θ to maximize the objective function J(θ).

[0135] Result verification: Verify the optimized data set to ensure that it can efficiently support interface linkage.

[0136] Furthermore, in order to ensure that the data set can be effectively transferred and used, we define the interface and data format for data transfer. In this invention, the JSON format is used to transfer the data set, and a standard interface is defined for subsequent steps to call. Define the data transfer format:

[0137] data_packet={dataset_id,sql_script,params,return_fields} (14)

[0138] Where: dataset_id is the unique identifier of the dataset, sql_script is the generated SQL query script, params is the generated parameter configuration, return_fields is the data field returned by the query, and defines the data transfer interface:

[0139] sendDataPacket(data_packet)(15)

[0140] Through the definition, generation and optimization of dynamic data sets, reliable data support is provided for the intelligent linkage of the entire system. The self-supervised learning model realizes the automatic generation and optimization of data sets, while the optimization method based on reinforcement learning ensures the efficiency and correctness of data sets. Standardized data transmission format and interface definition ensure that data sets can be seamlessly transferred and used between different steps.

[0141] S3. Collect the status information of real-time interface elements, and then use a unified data structure to represent the status information. At the same time, bind event processing functions to real-time interface elements, introduce data processing functions, standardize the data, and store it locally or on the server. Then use JSON format to transfer data, define a standard interface to call real-time data, design a distributed data collection system architecture to collect and manage data, and then define a data analysis function to analyze and process the data collected in real time. Specifically, the following steps are included:

[0142] Real-time collection of status information of interface elements. This status information includes the value, position, status, etc. of the element. We use a unified data structure to represent this status information. Define the data structure of the interface element status:

[0143] ElementState={id,value,position,state,timestamp} (16)

[0144] Among them: id is the unique identifier of the element, value is the current value of the element, position is the position coordinate of the element in the interface, state is the state of the element (such as display, hide, enable, disable, etc.), and timestamp is the timestamp of state collection.

[0145] In order to capture the user's operation data, it is necessary to bind event processing functions to the interface elements. These functions will be triggered when the user operates and record the operation data. Define the data structure of user operation data:

[0146] UserAction={element_id,action_type,value,timestamp} (17)

[0147] Among them: element_id is the unique identifier of the element that triggers the action, action_type is the type of action (such as click, input, selection, etc.), value is the value of the element when the action is triggered, and timestamp is the timestamp of the action.

[0148] Furthermore, the collected interface element status and user operation data need to be processed and stored for use in subsequent reinforcement learning models. Introduce data processing functions to standardize the data and store it locally or on the server. Data processing function:

[0149] processData(data)(18)

[0150] The processing process includes the following steps:

[0151] Data verification: Check whether the collected data is complete and valid.

[0152] Data standardization: Convert data into a unified format for easy subsequent processing.

[0153] Data storage: Store the processed data locally or on the server.

[0154] In order to ensure that data can be effectively transferred and used, the interface and data format of data transfer are defined. In this invention, JSON format is used to transfer data, and a standard interface is defined for subsequent steps to call. Define data transfer format:

[0155] data_packet={element_state,user_action} (19)

[0156] Among them: element_state is the state information of the interface element; user_action is the user's operation data.

[0157] Define the data transfer interface:

[0158] sendDataPacket(data_packet)(20)

[0159] In order to ensure the real-time and accuracy of data collection, a distributed data collection system architecture is designed. The system includes front-end data collection module, data processing module and data storage module.

[0160] Front-end data collection module: implement event binding and status collection on the front-end, and capture user operations and interface element status through JavaScript code.

[0161] Data processing module: implements data processing functions locally or on the server to verify and standardize the collected data.

[0162] Data storage module: Stores the processed data in a local database or cloud database to ensure data persistence and security.

[0163] Furthermore, in order to better understand user behavior and optimize interface linkage rules, we need to process and analyze the collected data. By analyzing user operation data and interface element status, we can discover common operation modes and linkage requirements.

[0164] Define data analysis functions:

[0165] analyzeData(data)(21)

[0166] The processing process includes the following steps:

[0167] Data cleaning: Remove invalid data and noise to ensure data quality.

[0168] Feature extraction: Extract meaningful features from the data, such as operation frequency, response time, etc.

[0169] Pattern recognition: Identify users’ common operation patterns and linkage needs through clustering and classification algorithms.

[0170] S4. Define the reinforcement learning model optimization interface linkage rules, introduce the action-state mapping matrix based on the reinforcement learning model, select the optimal action through matrix operation, and feed back the data set query results to the reinforcement learning model for strategy update through the dynamic data set feedback mechanism; wherein, the reinforcement learning includes state representation, action space, reward function and strategy network;

[0171] The state is represented by S={ElementState, UserAction, DynamicDataset}, where ElementState represents the state information of the interface element, UserAction represents the user's operation data, and DynamicDataset represents the state of the dynamic data set;

[0172] The action space is A={change_value, change_style, trigger_event, query_dataset}, where change_value represents the action of changing the element value, change_style represents the action of changing the element style, trigger_event represents the action of triggering the element event, and query_dataset represents the action of querying the dynamic dataset;

[0173] The reward function is R t =f(user_satisfaction,system_performance,task_completion), where, R tis the reward value at time step t, user_satisfaction represents user satisfaction, system_performance represents system performance, and task_completion represents task completion.

[0174] The training process includes the following steps:

[0175] Initialize policy network: Initialize the parameters θ of the policy network.

[0176] State observation: Get the current state S t .

[0177] Select action: According to the policy network π θ Select Action A t .

[0178] Execute action: Execute action A in the environment t , get the new state S t+1 and reward R t .

[0179] Update the policy network: Update the policy network parameters θ by gradient ascent to maximize the objective function J(θ). The objective function is defined as:

[0180]

[0181] Where: J(θ) is the optimization objective function, π θ is the policy network, τ is the trajectory sampled from the policy, T is the time step, and γ is the discount factor.

[0182] Furthermore, in order to improve the training efficiency and effect of the model, we use the proximal policy optimization (PPO) algorithm. PPO prevents the strategy from falling into the local optimum by limiting the step size of the strategy update. The optimization steps of the PPO algorithm are:

[0183] Compute the advantage function:

[0184]

[0185] Among them: A t is the advantage function at time step t, V(S t ) is state S t The value function of

[0186] Update policy network:

[0187]

[0188] Where: α is the learning rate, π θ is the current strategy, is the old policy and ∈ is the clipping parameter.

[0189] It is understandable that through the training of the reinforcement learning model, the system can intelligently adjust the linkage rules of interface elements to improve user experience and system performance. The specific optimization steps are as follows:

[0190] Initialization rules: Set the initial linkage rules based on the initial status.

[0191] Status evaluation: Evaluate the current status and linkage effects through the model.

[0192] Action selection: Select the optimal action based on the policy network.

[0193] Rule update: Execute the selected action and update the linkage rules.

[0194] Continuous optimization: Continuously optimize linkage rules through continuous iteration and learning.

[0195] Furthermore, in order to achieve efficient model training, a distributed training system architecture is designed. The system includes data acquisition module, model training module and rule optimization module:

[0196] Data collection module: collects user operation data and interface element status in real time and passes them to the training module.

[0197] Model training module: Model training and optimization based on the collected data.

[0198] Rule optimization module: dynamically adjust and optimize interface linkage rules based on training results.

[0199] Furthermore, in order to effectively map the action to the state space, we introduce the action-state mapping matrix M and select the optimal action through matrix operations. Define the action-state mapping matrix:

[0200] M={M ij} (28)

[0201] Where: M ij Represents the mapping relationship between state i and action j;

[0202] M ij =P(A j |S i )

[0203] By calculating the mapping matrix M, we can choose the action that maximizes the expected reward:

[0204]

[0205] It is understandable that in order to achieve interactive optimization between data sets and linkage rules, a dynamic data set feedback mechanism is introduced to feed back data set query results to the reinforcement learning model for strategy update. Definition of feedback mechanism:

[0206] Feedback(S t ,A t )={query_result,response_time,accuracy} (30)

[0207] Among them: query_result is the result of the dataset query, response_time is the response time of the query, and accuracy is the accuracy of the query result. Through the feedback mechanism, the reinforcement learning model can adjust the strategy in real time to optimize the query and linkage rules of the dataset.

[0208] S5. Design a model-independent meta-learning model to adaptively configure the interface, calculate the optimal configuration action based on the model-independent meta-learning model, and design a meta-learning distributed system architecture to collect and manage data. Introduce an adaptive feedback mechanism to feed back the configuration results to the model-independent meta-learning model for strategy update and optimize the web interface configuration.

[0209] Specifically, the model-agnostic meta-learning (MAML) algorithm is used to achieve adaptive configuration of the interface. MAML enables the model to learn and optimize efficiently with a small amount of data by quickly adapting to new tasks. Define the state representation of the meta-learning model:

[0210] S={ElementState,UserAction,DynamicDataset,RuleSet} (31)

[0211] Among them: ElementState is the state information of the interface element, which is converted into a vector representation through embedding technology:

[0212] ElementStateEmbedding=Embedding(ElementState) (32)

[0213] Among them, Embedding is an embedding function that converts ElementState into a low-dimensional vector; UserAction is the user's operation data, DynamicDataset is the state of the dynamic data set, and RuleSet is the current linkage rule set.

[0214] Furthermore, the objective function of the meta-learning model is defined as:

[0215]

[0216] Where: θ is the initial parameter of the model, T i is the i-th task, is the loss function of the i-th task, α is the learning rate, and f θ It's a model.

[0217] Furthermore, the meta-learning model training includes the following steps:

[0218] Sample a set of tasks from the task distribution p(T):

[0219] {T i}~p(T)(34)

[0220] Among them, T i is the i-th task, and p(T) is the task distribution.

[0221] In order to further optimize the inner loop update formula and make it more suitable for complex scenarios, an adaptive learning rate and a weighted loss function are introduced. This can better handle the importance of different tasks and data distribution differences, and define the adaptive inner loop update formula:

[0222]

[0223] Where: α i It is task T i The adaptive learning rate, It is task T i The gradient of the loss function, λ∥θ∥ 2 is the L2 regularization term, λ is the regularization coefficient, G ii is the square matrix of accumulated gradients with diagonal elements, used for adaptive learning rate adjustment, and δ is a small value to prevent the denominator from being zero.

[0224] In the outer loop update, the meta-learning rate and weighted total loss function are introduced to better reflect the importance of different tasks. This can optimize the adaptability of the model in complex scenarios. Define the adaptive outer loop update formula:

[0225]

[0226] Where: β is the meta-learning rate, w i It is task T i The weights of , used to weight the total loss, It is task T i The loss gradient is λ∥θ∥ 2 is the L2 regularization term, λ is the regularization coefficient, H ii is the square matrix of accumulated gradients with diagonal elements, used for adaptive learning rate adjustment, and δ is a small value to prevent the denominator from being zero.

[0227] These improved formulas introduce adaptive learning rates and weighted losses, which can better adapt to the complexity and diversity of patent scenarios and improve the training efficiency and effectiveness of meta-learning models.

[0228] Preferably, in order to improve the effect of interface adaptive configuration, we introduce a rule-based adaptive strategy, which dynamically adjusts the interface configuration by analyzing user operations and interface element states in real time.

[0229] Define the adaptive configuration strategy:

[0230] Config=f(S,A,R) (37)

[0231] Among them: S is the current state, A is the set of optional actions, R is the set of linkage rules, and f is the adaptive configuration strategy function.

[0232] Furthermore, through the meta-learning model, the system can quickly select and execute the optimal configuration action to improve user experience and interface adaptability. Define the action selection function:

[0233]

[0234] Among them: A * is the best action. is the expected reward of action A in state S.

[0235] Furthermore, in order to achieve efficient training and adaptive configuration of the meta-learning model, we designed a distributed system architecture. The system includes a data collection module, a model training module, and an adaptive configuration module:

[0236] Data collection module: real-time collection of user operation data, interface element status and linkage rules.

[0237] Model training module: trains and optimizes the meta-learning model based on the collected data.

[0238] Adaptive configuration module: dynamically adjust and optimize interface configuration based on training results.

[0239] Furthermore, in order to achieve interactive optimization of adaptive configuration and linkage rules, we introduce an adaptive feedback mechanism to feed back the configuration results to the meta-learning model for strategy update. Define the adaptive feedback mechanism:

[0240] Feedback(S t ,A t )={config_result,user_feedback,system_metrics} (39)

[0241] Among them: config_result is the configuration result, user_feedback is the user's feedback, and system_metrics is the system's performance indicators.

[0242] Through the feedback mechanism, the meta-learning model can adjust the strategy in real time to optimize the adaptive configuration and linkage rules of the interface.

[0243] like Figure 2 As shown, another embodiment of the present invention is provided in another aspect, a dynamic web interface modeling system based on reinforcement learning, the system comprising:

[0244] The web interface initialization unit 501 is used to define a data structure for each element on the web interface and bind an event, wherein each element is bound to a single or multiple events, and when a triggering event occurs, the event data is processed and recorded, and the event data is transmitted in JSON format, and a data transmission standard interface is defined to transmit the processed time data;

[0245] The dynamic data set construction unit 502 is used to define the dynamic data set structure, dynamically generate SQL query scripts and parameter configurations according to user input and interface element status, optimize the dynamic data set using a reinforcement learning-based optimization method, transfer the data set using the SON format, and define a standard interface for data transfer of the dynamic data set; wherein the dynamic data set includes SQL scripts, parameters, and return data;

[0246] The interface element analysis unit 503 is used to collect the status information of the real-time interface elements, and then use a unified data structure to represent the status information, bind event processing functions to the real-time interface elements, introduce data processing functions, standardize the data, and store it locally or on the server, and then use the JSON format to transfer data, define a standard interface to call the real-time data, design a distributed data collection system architecture to collect and manage the data, and then define a data analysis function to analyze and process the data collected in real time;

[0247] The optimization interface unit 504 is used to define the linkage rules of the reinforcement learning model optimization interface, introduce the action-state mapping matrix based on the reinforcement learning model, select the optimal action through matrix operation, and feed back the data set query results to the reinforcement learning model for strategy update through the dynamic data set feedback mechanism; wherein the reinforcement learning includes state representation, action space, reward function and strategy network;

[0248] The state is represented by S={ElementState, UserAction, DynamicDataset}, where ElementState represents the state information of the interface element, UserAction represents the user's operation data, and DynamicDataset represents the state of the dynamic data set;

[0249] The action space is A={change_value, change_style, trigger_event, query_dataset}, where change_value represents the action of changing the element value, change_style represents the action of changing the element style, trigger_event represents the action of triggering the element event, and query_dataset represents the action of querying the dynamic dataset;

[0250] The reward function is R t =f(user_satisfaction,system_performance,task_completion), where, R t is the reward value at time step t, user_satisfaction represents user satisfaction, system_performance represents system performance, and task_completion represents task completion;

[0251] The adaptive configuration unit 505 is used to design a model-independent meta-learning model to adaptively configure the interface, calculate the optimal configuration action based on the model-independent meta-learning model, and design a meta-learning distributed system architecture to collect and manage data. An adaptive feedback mechanism is introduced to feed back the configuration results to the model-independent meta-learning model for strategy update and optimization of the web interface configuration.

[0252] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0253] In addition, for technical details that are not described in detail in this embodiment, reference can be made to the parameter operation method provided in any embodiment of the present invention, and will not be repeated here.

[0254] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0255] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0256] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0257] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A dynamic web interface modeling method based on reinforcement learning, characterized in that: The method comprises: S1. Define a data structure for each element on the web interface and bind an event, wherein each element is bound to a single or multiple events. When a trigger event occurs, the event data is processed and recorded, and the event data is transmitted in JSON format. At the same time, a data transmission standard interface is defined to transmit the processed time data; S2. Define the dynamic data set structure, dynamically generate SQL query scripts and parameter configurations based on user input and interface element status, optimize the dynamic data set using a reinforcement learning-based optimization method, transfer the data set in JSON format, and define a standard interface for data transfer of the dynamic data set; wherein the dynamic data set includes SQL scripts, parameters, and return data; S3. Collect the status information of real-time interface elements, and then use a unified data structure to represent the status information. At the same time, bind event processing functions to real-time interface elements, introduce data processing functions, standardize the data, and store it locally or on the server. Then use JSON format to transfer data, define a standard interface to call real-time data, design a distributed data collection system architecture to collect and manage data, and then define a data analysis function to analyze and process the data collected in real time; S4. Define the reinforcement learning model optimization interface linkage rules, introduce the action-state mapping matrix based on the reinforcement learning model, select the optimal action through matrix operation, and feed back the data set query results to the reinforcement learning model for strategy update through the dynamic data set feedback mechanism; wherein, the reinforcement learning includes state representation, action space, reward function and strategy network; The state is represented by ,in, Indicates the status information of the interface element. Indicates the user's operation data. Represents the state of a dynamic data set; The action space is ,in, Represents the action of changing the value of an element. Represents the action of changing the style of an element. Represents the action that triggers the element event. Represents the action of querying a dynamic data set; The reward function is ,in, is the time step The reward value, Indicates user satisfaction, Indicates system performance, Indicates task completion; S5. Design a model-independent meta-learning model to adaptively configure the interface, calculate the optimal configuration action based on the model-independent meta-learning model, and design a meta-learning distributed system architecture to collect and manage data. Introduce an adaptive feedback mechanism to feed back the configuration results to the model-independent meta-learning model for strategy update and optimize the web interface configuration. Dynamically generate SQL query scripts and parameter configurations based on user input and interface element status, and use self-supervised learning models to automatically generate and optimize data sets; define the input and output of the self-supervised learning model: ； in, is the current state of the interface element, is the user's input data; ； in, is the generated SQL query script, is the generated parameter configuration; The generation process includes the following steps: Feature extraction: extract features from input data and construct feature vectors; Model reasoning: Generate SQL scripts and parameter configurations through self-supervised learning models; Result verification: Verify the generated SQL scripts and parameter configurations to ensure their correctness and effectiveness.

2. According to claim 1, a dynamic web interface modeling method based on reinforcement learning is characterized in that: The binding of event data in S1 also introduces event weights, and each event data is given a different weight according to its importance and frequency, as shown below: ； in, Indicates The element triggers the The weight of the event, Indicates The element triggers the The frequency of events, It is The total number of events that can be triggered by an element.

3. A dynamic web interface modeling method based on reinforcement learning according to claim 2, characterized in that: In the above S1, the event data processing also introduces an event influence matrix to quantify the influence of the event, and the event influence matrix is ​​defined as: ； in, Indicates The element triggers the the impact of the event; ； in, Indicates The element triggers the The reward value of an event depends on the contribution of the event to the interface linkage; The update formula of event influence is: ； in, Indicates The element triggers the The influence of this event in the t+1th iteration, Indicates The element triggers the The event in The impact in the iteration, is the learning rate, ranging from .

4. The method for modeling a dynamic web interface based on reinforcement learning according to claim 1, characterized in that: The metadata definition is introduced into the S2, including the SQL script template and parameter configuration, wherein the structure of the metadata is represented as follows: ； in, is a unique identifier for the template; It is a template for SQL query script; It is the parameter configuration, including parameter name, type and default value.

5. A dynamic web interface modeling method based on reinforcement learning according to claim 4, characterized in that: In S2, the objective function optimized for the dynamic data set is: ； in, represents the optimization objective function, represents the reinforcement learning strategy, represents the trajectory sampled from the policy, represents the time step, represents the discount factor, Indicates the time step Rewards; Wherein, the optimization process is: Initializing the reinforcement learning policy ; Calculate the value of the current strategy through the strategy evaluation function; Optimize strategy parameters based on evaluation results , so that the objective function maximize; Verify the optimized data set.

6. The method for dynamic web interface modeling based on reinforcement learning according to claim 1, characterized in that: The distributed data acquisition system architecture includes a front-end data acquisition module, a data processing module and a data storage module; Among them, the front-end data collection module is used to implement event binding and status collection on the front end, and capture user operations and interface element status through JavaScript code; The data processing module is used to implement data processing functions locally or on the server side, and to verify and standardize the collected data; The data storage module is used to store the processed data in a local database or a cloud database to ensure data persistence and security.

7. The method for modeling a dynamic web interface based on reinforcement learning according to claim 1, characterized in that: In S4, a proximal strategy optimization algorithm is used to optimize reinforcement learning, including the following steps: The advantage function is calculated and expressed as follows: ； in, Indicates the time step The advantage function of Indicates status The value function of The update policy network is expressed as follows: ； in, represents the learning rate, Indicates the current strategy, represents the old policy, Represents clipping parameters.

8. The method for modeling a dynamic web interface based on reinforcement learning according to claim 7, characterized in that: The action-state mapping matrix is ​​expressed as follows: ； in, Indicates status With action The mapping relationship between them is expressed as follows: ； By calculating the mapping matrix , select the action that maximizes the expected reward, expressed as follows: 。 9. The method for modeling a dynamic web interface based on reinforcement learning according to claim 1, characterized in that: The training steps of the model-independent meta-learning model include: From the task distribution Sample a set of tasks: ； in, It is tasks, is the task distribution; Adaptive learning rate and weighted loss function are introduced, and the adaptive inner loop update formula is defined as follows: ； in, Indicates the task The adaptive learning rate, Indicates the task The gradient of the loss function is represents the L2 regularization term, represents the regularization coefficient; Represents the cumulative gradient square matrix of diagonal elements, used for adaptive learning rate adjustment; Represents small values ​​that prevent the denominator from being zero; The meta-learning rate and weighted total loss function are introduced to calculate the importance of different tasks, and the adaptive outer loop update formula is defined: ； in, is the meta-learning rate, It's a task The weights of , used to weight the total loss, It's a task The loss gradient is is the L2 regularization term, is the regularization coefficient, is the cumulative gradient square matrix with diagonal elements, used for adaptive learning rate adjustment, is a small value that prevents the denominator from reaching zero.

10. A dynamic web interface modeling system based on reinforcement learning, characterized in that: The system comprises: The web interface initialization unit is used to define the data structure and bind events for each element on the web interface, wherein each element is bound to one or more events, and when a trigger event occurs, the event data is processed and recorded, and the event data is transmitted in JSON format, and a data transmission standard interface is defined to transmit the processed time data; A dynamic data set construction unit is used to define the dynamic data set structure, dynamically generate SQL query scripts and parameter configurations according to user input and interface element status, optimize the dynamic data set using a reinforcement learning-based optimization method, transfer the data set in JSON format, and define a standard interface for data transfer of the dynamic data set; wherein the dynamic data set includes SQL scripts, parameters, and return data; The analysis unit of the interface element is used to collect the status information of the real-time interface elements, and then use a unified data structure to represent the status information. At the same time, the event processing function is bound to the real-time interface elements, and the data processing function is introduced to standardize the data and store it locally or on the server. Then the JSON format is used to transmit the data, and a standard interface is defined to call the real-time data. A distributed data collection system architecture is designed to collect and manage the data, and then a data analysis function is defined to analyze and process the data collected in real time; An optimization interface unit is used to define the linkage rules of the reinforcement learning model optimization interface, and introduce an action-state mapping matrix based on the reinforcement learning model, select the optimal action through matrix operations, and feed back the data set query results to the reinforcement learning model for strategy update through a dynamic data set feedback mechanism; wherein the reinforcement learning includes state representation, action space, reward function and strategy network; The state is represented by ,in, Indicates the status information of the interface element. Indicates the user's operation data. Represents the state of a dynamic data set; The action space is ,in, Represents the action of changing the value of an element. Represents the action of changing the style of an element. Represents the action that triggers the element event. Represents the action of querying a dynamic data set; The reward function is ,in, is the time step The reward value, Indicates user satisfaction, Indicates system performance, Indicates task completion; The adaptive configuration unit is used to design a model-independent meta-learning model to adaptively configure the interface. The optimal configuration action is calculated based on the model-independent meta-learning model. At the same time, a meta-learning distributed system architecture is designed to collect and manage data. An adaptive feedback mechanism is introduced to feed back the configuration results to the model-independent meta-learning model for strategy update and optimization of the web interface configuration. Dynamically generate SQL query scripts and parameter configurations based on user input and interface element status, and use self-supervised learning models to automatically generate and optimize data sets; define the input and output of the self-supervised learning model: ； in, is the current state of the interface element, is the user's input data; ； in, is the generated SQL query script, is the generated parameter configuration; The generation process includes the following steps: Feature extraction: extract features from input data and construct feature vectors; Model reasoning: Generate SQL scripts and parameter configurations through self-supervised learning models; Result verification: Verify the generated SQL scripts and parameter configurations to ensure their correctness and effectiveness.

Images