Experimental path sorting method based on user behavior perception feedback

Abstract

The invention discloses an experimental path sorting method based on user behavior perception feedback, which comprises the following steps: acquiring a full path data set which comprises a plurality of experimental paths; constructing a perceptual response model, wherein the perceptual response model adopts a reinforcement learning framework; the perception response model performs iterative updating according to the state information and the award value obtained by the behaviors under different interactions; and performing iterative optimization on the perception response model to obtain current state information, extracting a current experimental path set from the full path data set according to the current state information, and sorting the current experimental path set through the perception response model in the iterative optimization process to obtain an experimental path sorting result. According to the technical scheme, through perception and response to user behaviors, a reinforcement learning algorithm framework is applied, and path sets meeting requirements are sequenced and output in a mode of maximizing actual demands of users.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent decision-making technology, and in particular relates to an experimental path sorting method based on user behavior perception feedback. Background Art

[0002] According to the invention "Experimental Planning Method and System Based on Optimal Path" (authorization number: CN117252314B), for a data system, its starting data is a set of elements and a set of associations between elements, and the path is data dynamically generated based on the associations between elements. However, the experimental initial environment (starting element) and the experimental target (end point element) can theoretically be a combination of any two factors. There are many types of paths between the starting point and the end point of each experiment.

[0003] Although in actual operations, we tend to aggregate and generate effective paths with high activity and ignore most of the paths that exist in theory, there are still a huge number of paths between two elements. When users search for paths, how to return multiple paths in the form of a partially ordered set that meets user requirements becomes an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes an experimental path sorting method based on user behavior perception feedback to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides an experimental path sorting method based on user behavior perception feedback, comprising:

[0006] Acquire a full path data set, wherein the full path data set includes several experimental paths;

[0007] Constructing a perception-response model, wherein the perception-response model adopts a reinforcement learning framework;

[0008] Iteratively optimizing the perception response model, wherein a single process in the iterative optimization process includes: obtaining state information, extracting a corresponding experimental path set from the full path data set according to the state information, sorting the experimental path set by the perception response model, obtaining user behavior according to the sorting result, obtaining a reward value according to the user behavior, optimizing the policy parameters of the perception response model according to the reward value, and completing a single optimization;

[0009] The current state information is obtained, and the current experimental path set is extracted from the full path data set according to the current state information. The current experimental path set is sorted by the perception response model in the iterative optimization process to obtain the experimental path sorting result; wherein the perception response model is optimized based on the current state information according to a single process in the iterative optimization process.

[0010] Optionally, the full path data set is directed graph structure data, wherein a single experimental path in the full path data set includes a number of directional elements and experiments.

[0011] Optionally, the perception response model includes a state space, an action space, a reward function and a state transfer function, wherein the state space includes different states, wherein the state includes input conditions, user data and search constraint variables; the action space includes several different sorting actions, and the reward function represents the relationship between executing a sorting action in a state and the reward value; the state transfer function represents the probability of executing a sorting action in a certain state and transferring to the next state.

[0012] Optionally, the user behavior includes selecting a path, browsing, exiting browsing, turning pages, modifying and creating a path.

[0013] Optionally, the process of obtaining a reward value based on user behavior includes:

[0014] The user behavior is calculated through the reward function in the perception response model to obtain the reward value;

[0015] The reward values ​​include the reward value for selecting a path, the reward value for clicking to browse, and the reward value for modifying / building a path;

[0016] The reward value acquisition process of the selected path includes:

[0017] r choose =k choose *p choose *y, where p choose It is the inverse of the ratio of the row number position of the selected path to the total number of rows, indicating that the ranking effect will meet the user's expectations more if the user selects a path that is closer to the front.

[0018] The reward value acquisition process of the click-to-browse reward value includes:

[0019] r read =k read *p read *y, where p read is the normalized coefficient value of browsing time, indicating the user's attention to the browsing results in this query;

[0020] The reward value acquisition process of the modification / construction path includes:

[0021] r update =k update *p update *y, where p update For p read *m update , where p read Represents the importance of the user to this search, m update It is the inverse of the measurement of the modification range and proportion. The product of the two represents the comprehensive measurement of the user's attention and the degree of conformity of the returned results to their needs; k is the overall reward adjustment coefficient, k choose , k read and k update It represents the overall reward adjustment coefficient corresponding to the reward value for selecting a path, the reward value for clicking and browsing, and the reward value for modifying / building a path, and y is the unit reward score.

[0022] Optionally, the process of ranking experimental paths through the sensory response model includes:

[0023] The optimal action is selected according to the optimal strategy through the perception response model in the iterative optimization process, and the current experimental path is sorted by the optimal action; the optimal strategy is the strategy for executing the optimal action under the current state information, and the optimal strategy is:

[0024]

[0025] Among them, π * is the optimal strategy, π represents the strategy, k represents the prediction interval time step, γ k ∈[0,1) is the discount rate, r t+k represents the reward at time step t+k, s t Represents the state at time step t, s represents arbitrary state information, S represents the state space, and t represents the time step.

[0026] Optionally, the state transfer function is optimized by a gradient algorithm or a policy approximation algorithm.

[0027] On the other hand, the present invention provides an experimental path sorting system based on user behavior perception feedback, characterized in that it is used to execute the above method.

[0028] Compared with the prior art, the present invention has the following advantages and technical effects:

[0029] The present invention proposes an experimental path sorting method based on user behavior perception and response. Through this method, the algorithm framework of reinforcement learning can be applied to perceive and respond to user behavior, and a path set that meets the requirements can be sorted and output in a way that maximizes the actual needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 A schematic diagram of a single experimental path data structure according to an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of the data structure of the associated experimental path according to an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of an experimental path between input and output of an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of a directed structure of variable relationships in an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of simplified input factors and output factors of an embodiment of the present invention;

[0036] Figure 6 A schematic diagram of the relationship between factors and experiments in the full path data of an embodiment of the present invention;

[0037] Figure 7 A schematic diagram of a single experimental path result extracted from the full path data of an embodiment of the present invention;

[0038] Figure 8 The figure is a schematic diagram of a method flow of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] like Figure 8 As shown, the present invention provides an experimental path sorting method based on user behavior perception response, comprising:

[0042] Step S1: Dataset preparation

[0043] All experimental path datasets form the full path dataset Set1.

[0044] The experimental path dataset includes different experimental path data, such as Figure 1 As shown in the figure, each experimental path data includes an input variable set I = (i1, i2…i n ) is transformed into an output variable set O = (o1, o2…o n ) process (where o * ∈O is the main output variable, that is, the experimental result, and the others are secondary outputs), the experiment is recorded as p (here I and O are only variable sets, do not contain the measurement values ​​of the variables, and only represent the form of the experiment rather than an instance). For example, if a chlorophyll extraction experiment is to be conducted, the factors include organic solvents, experimental materials, grinding time, grinding materials, filter paper, etc. as experimental inputs, that is, input variable sets, and also include the technical parameters and specifications required for the experimental process as experiments, and chlorophyll solution as experimental output, that is, output variable sets.

[0045] Experimental input is the factor g related to the experimental process n Set G = (g1, g2…g n ).

[0046] The objective environmental factors or the factors that can be obtained through simple measurement are recorded as environmental variables e j , the environment variable set is E:

[0047] F=(f1,f2…f j )

[0048] The factors that require a certain amount of input for more complex measurements or other experiments to obtain the measured values ​​are recorded as experimental calculation variables c t , experimental calculation variable set C:

[0049] C=(c1,c2…c t )

[0050] All experiments p k Let it be set P:

[0051] P=(p1,p2…p k )

[0052] p i The output set of (i=1,2,3…,k) is ρ(O) is the power set of the elements in O. (Assume that the result sets of different experimental outputs are not completely the same or contain a unique identifier of an experimental instance).

[0053] set up Then G=F∪O',p i (i=1,2,3…,k) is the mapping from experimental input to output:

[0054] pi :G→O′

[0055] The corresponding input set is: (Assuming that the parameter sets of different experimental inputs are not exactly the same or contain a unique identifier of an experimental instance), you can Figure 1 The input and output of a single data is combined into the input and output of all data in the set, such as Figure 2 The path relationship between two related experimental data is shown in Figure 1. The output of the experiment p is passed through environmental factors and the experimental output of the previous step, and is used as the experimental input of the next experiment p1.

[0056] like Figure 3 As shown, the final process between the input and output of an overall experimental data is a directed graph, in which the input can obtain the output through different paths, including paths of different experiments, p11, p21 and p3, p12 and p3, p31, p22 and p3, p31, p23 and p3, to obtain the relevant required output.

[0057] by Figure 3 Based on the structure of , we can construct a directed graph structure containing arcs of elements of set P, edges of elements of sets I, O, and E - "variable relationship graph", such as Figure 4 As shown, the top path is used as an illustrative example, where environmental factors e1 and e2 are combined with input factor I1 to obtain output O1 through experiment p1, with output O1 as part of the input, and environmental factor e3 is combined with input I3 to obtain output O4 through experiment p4, and output O4 is combined with e10 to prepare for the next input.

[0058] The experimental path from an experimental input to an experimental output can be simplified as follows: Figure 5 The structure of experimental input 1 is as follows: the experimental result O is obtained through experiments (including sub-experiment 1, sub-experiment 2...sub-experiment n). The meaning of the experimental path is the path between the experimental input element and the experimental output element.

[0059] right Figure 5 The structure of is fully combined and displayed to form Figure 6 The directed graph structure shown in Figure 6 As shown in the figure, a directed graph consisting of input element-path-output element is separated. This directed graph is an experimental path. Figure 7 for Figure 6 Part of the structure Figure 7 The partial structures shown are Figure 6 and Figure 7The structure of is explained, factor a is the original input, and factor b and factor k are obtained through experiments 1 and 3 respectively. Factor k is obtained through experiment 4 to obtain factor I. Factor I can obtain factor b according to experiment 5. Factor b and factor I are obtained through experiment 6 to obtain the final output, i.e., factor j. Figure 7 For a single experimental path extracted from the full path data set structure, the result input of the path is in the text or graph structure of input factor-experiment-output factor→input-experiment-output factor→input factor-experiment-output factor.

[0060] The input set, output set and experiments representing the paths in all experimental path data sets are formed into a directed graph structure, and the directed graph structure is saved to form a full path data set Set1.

[0061] Step S2: Model preparation

[0062] Construct a perception response model. The perception response model is agent: M = (S, A, R, T), where M represents the perception response model, S represents the state space, A represents the state space, R represents the reward function, and T represents the state transfer function. The contents of the above perception response model are described in detail:

[0063] 1) State space: S = (E, H), which contains all the state information that the agent can perceive, where E = (e1, e2, …, e n ) is the environment scene variable where the user is located, H=(h1,h2,…,h n ) are search constraint variables.

[0064] When searching for related paths, you need to enter the starting element ele b and the end element ele e , collect environmental information and search constraint information at the same time, let E = (e1, e2, ..., e n ) is the environment scene variable where the user is located, including the first n c The input and response of the search, user portrait attributes, user historical operation information, etc., H = (h1,h2,…,h m ) is the search constraint variable, including the conditions and requirements entered during the search, such as timeliness requirements, various cost requirements, etc.

[0065] 2) Action space: A = (a1, a2, ..., a n ), where a i It represents the i-th action, i=1, 2...n, and includes all the actions that the agent can take in each state, which is a series of possible path sorting strategies.

[0066] 3) Reward function: R(s,a,s') represents the reward value obtained by the agent from the environment when executing action a in state s and transferring to state s';

[0067] 4) State Transfer Function: T: S×A×S'→[0,1], T(s,a,s') represents the probability of executing action a in state s and transferring to state s'. The state transfer function consists of a series of policy parameters θ(θ1,θ2,…,θ m ), where T(s,a,s') can be optimized by optimizing the state transfer parameters, and there is a direct relationship between T(s,a,s').

[0068] Step S3: User search context awareness

[0069] When a user searches:

[0070] 1. User search information collection: H

[0071] It includes the conditions and requirements entered during the search. The conditions entered during the search include the element information entered by the user, such as the starting point and the end point, etc. The requirements include the resource constraints set by the user, such as equipment and accessories, etc., the path length, duration, cost and other constraints entered by the user, as well as other requirements.

[0072] 2. Scene environment information collection: E

[0073] The system will collect environmental information of the user's scenario, including the input and response of the previous n searches, user portrait attributes, user historical operation information, etc.

[0074] Step S4: Search results returned

[0075] The full path dataset Set1 is matched through the user search condition information collected in S3, and the corresponding result set is returned.

[0076] Among them, according to the user search condition information, the full path data set is searched to find relevant factors or corresponding data. After the corresponding data search is completed, the connected related paths are searched in the directed graph structure in the full path data set based on the starting point and end point of the input element information, and the paths are extracted. In the process of searching the path, the resources involved in the path are compared with the above requirements as the restriction conditions. When the resources contain equipment or accessories that are not in the resource restriction conditions, the paths are removed. At the same time, the paths involved in the path search process are calculated in real time to calculate the length, duration and cost of the experimental path. If the calculation process does not meet the above requirements, the path is removed and the path search is no longer continued along this path to the end point. Or after searching the experimental path, the paths that do not meet the requirements are removed according to the above requirements, and all the paths that meet the requirements are retained and returned as the corresponding result set.

[0077] Step S5: Sorting the result set

[0078] The E and H sets obtained from S3 are used as the overall environmental information for perception, and are input into the agent model M for decision-making. The specific action completion ranking is obtained, and the experimental path ranking result set is obtained, which is then returned to the user for subsequent operations.

[0079] After obtaining the result set or experimental path, the perception response model selects the optimal action for the current state according to the obtained state, that is, the overall environmental information (E, H). The optimal action is the optimal execution function a * ∈A=(a1,a2,…,a n ), which represents the sorting strategies that can be adopted, including: sorting by efficiency, sorting by cost, sorting by the minimum cost of a certain resource, sorting by the minimum time consumption, etc., divided into the following two steps: the perception response model obtains the corresponding optimal strategy according to the current state, and obtains the action a with the highest reward in the optimal strategy. * , with a * The determined sorting strategy sorts the result set to obtain the experimental path sorting result set, and its actions are not adjusted during the optimization process. At the same time, it should be noted that the actions can be adjusted according to actual needs, and the optimization needs to be performed again after the adjustment.

[0080] The optimal action corresponding to the current state is a *Continuous optimization is performed based on gradual strategy adjustments. The strategy is to perform a certain action in a certain state. The optimal strategy is to perform the action with the highest reward in a certain state. After performing an action, the action will trigger subsequent predicted behaviors, and then predict its reward value. By comparing the size of the total reward value, the optimal action can be judged. The action with the highest reward under the optimal strategy is not fixed, and will be adjusted and optimized according to the subsequent reward function. The predicted behavior is predicted based on the state transition function.

[0081] Step S6: User Behavior

[0082] Users make decisions and take actions on the sorted result set obtained in S5, such as selecting a path, browsing, exiting browsing, turning pages, modifying and creating a path, etc.

[0083] After obtaining the sorted result set, the agent intelligent body will provide the sorted result set to the user, and the user, that is, the external environment, will make decisions and behaviors on the sorted result set, such as selecting a path, browsing, exiting browsing, turning pages, modifying and creating paths, etc., thereby providing the agent intelligent body with feedback on user behavior, that is, the external environment, and the agent intelligent body will learn and optimize based on the feedback.

[0084] Step S7: Reward Function

[0085] After receiving the user's behavior, the perception-response model agent calculates its reward according to the reward function.

[0086] The agent is rewarded based on the user's behavior. The specific form of the reward is to assign reward points to the user's positive behavior and 0 to the rest.

[0087] Positive user behaviors include:

[0088] Path selection behavior, browsing behavior, page turning behavior, path editing behavior, new path creation behavior, etc.

[0089] Negative user behaviors include exiting, closing, etc.

[0090] Including reward functions for different user behaviors. In the reward function, k∈R is the overall reward adjustment coefficient. Different reward types have different adjustment coefficients according to their important procedures, representing the external weight of the reward. The k value is preset, and p is the reward ratio. The ratio shows that the weight of this behavior in this type of reward belongs to the internal weight of the reward. y is the unit reward score:

[0091] The reward value of the selected path: r choose =k choose *p choose *y, where p chooseIt is the inverse of the ratio of the row number position of the selected path to the total number of rows, indicating that the ranking effect will meet the user's expectations more if the user selects a path that is closer to the front.

[0092] Click to view the reward value: read =k read *p read *y, where p read It is the normalized coefficient value of browsing time, indicating the user's attention to the browsing results in this query.

[0093] Modify / build path reward value: r update =k update *p update *y, where p update For p read *m update , where p read Represents the importance of the user to the search, m update It is the inverse of the measurement of the modification range and proportion (m=1 when constructing a path). The product of the two represents a comprehensive measure of the user's attention and the degree to which the returned results are consistent with their needs. At the same time, the modification and construction of the path will also update the full path set in the form of a new path between the start and end elements.

[0094] Other rewarding behaviors include:

[0095] Continue to search / turn pages, modify query conditions, modify constraints, stay on the interface, etc. This type of setting has a unified k value and p value, which is lower than the p value of the aforementioned key reward behavior. In order to prevent this type of reward from causing the model to tend to guide users to search and turn pages continuously, a smaller weight is set. The weight value is much smaller than the above-mentioned clear selection path, click to browse, and modify / build path reward value, such as less than 1 / 3 of it.

[0096] Step S8: Optimization function

[0097] The agent with reinforcement learning algorithm framework perceives the current environment state s t , select action a from the action space A r Execution; After the environment receives the action selected by the Agent, it gives the Agent the corresponding reward signal feedback r t+1 , and transfer to the new environment state s t+1 , waiting for the Agent to make a new decision.

[0098] In the process of interaction with the environment, the agent's goal is to find an optimal strategy. The real-time search and sorting strategy based on reinforcement learning controls the optimal strategy π * , through the optimal strategy π *Continuously select actions so that it can obtain the highest long-term cumulative reward in any state s and any time step t, that is:

[0099]

[0100] Among them, π * is the optimal strategy, π represents the strategy, k represents the prediction interval time step, γ k ∈[0,1) is the discount rate, r t+k represents the reward at time step t+k, s t Represents the state at time step t, s represents arbitrary state information, S represents the state space, and t represents the time step.

[0101] The goal of reinforcement learning is to find the optimal state-action-value function:

[0102]

[0103] Among them, a t Represents the action at time step t, a represents any action, and under this optimal state-action-value function, the action for the state is the optimal action.

[0104] The above-mentioned perception response model of the present invention adopts a reinforcement learning framework, wherein the above-mentioned long-term cumulative reward is calculated based on a series of subsequent user behaviors predicted by the selected action. In the above-mentioned long-term cumulative reward calculation process, The coefficient of n represents the number of subsequent steps, so as to prevent the model from guiding users to search continuously for rewards, and make the model biased towards allowing users to obtain the desired results quickly and efficiently.

[0105] Step S9: Update parameters

[0106] The parameters θ(θ1,θ2,…,θ m ) is optimized to optimize the model performance so that the agent's response feedback can obtain a higher reward value, thereby completing the model iteration.

[0107] The state transfer function is T:S×A×S→[0,1], where T(s,a,s') represents the execution of action a in state s. Its specific form is: T=T(s,a,s':θ),θ=(θ1,θ2,…,θ m ).

[0108] The optimization process of the model is to optimize (θ1, θ2, …, θ m), θ determines the probability of executing action a in state s to become state s', and s' determines the final reward value. Therefore, in order to maximize the reward of the agent, it is necessary to learn and adjust the parameters of the state transfer function T(s,a,s':θ) during the training process.

[0109] It should be noted that s' mainly represents the user's historical operation information in the subsequent state information. The user's historical operation information represents the user's existing historical operation information. The above-mentioned state transfer function is adjusted or optimized according to the user's historical operation information. By selecting the execution action and the parameters in the state transfer function, an action can be performed according to the change of the state. The user's possible different operation behaviors can be predicted and the reward value of the operation behavior can be calculated. According to the long-term predicted reward value, the optimal action currently executed can be obtained.

[0110] In the present invention, an agent is used to decide which sorting method to use to meet the needs of the user, and then the optimal sorting result determined by the model is returned.

[0111] The appropriate ranking results determine the user's next behavior, and the user's next behavior represents the user's satisfaction with the search results. The essence of agent design is to provide a result that satisfies the user and tune its own model based on user feedback.

[0112] Specific implementation content:

[0113] The user's scenario environment is:

[0114] Step 1: First, enter the search conditions such as starting point "iron oxide" and end point "LK-99" in the search box and click Search.

[0115] Step 2: After matching the database, a path result set is obtained.

[0116] Step 3: After the agent perceives the environment information, it processes the feedback of the result set, obtains a sorted set and returns it to the user.

[0117] Step 4: The user makes decisions and takes actions based on the returned results.

[0118] Step 5: The agent collects user behavior information and updates the model.

[0119] Step 6: The updated model senses and responds to user behavior again.

[0120] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An experimental path sorting method based on user behavior perception feedback, characterized in that: include: Acquire a full path data set, wherein the full path data set includes several experimental paths; Constructing a perception-response model, wherein the perception-response model adopts a reinforcement learning framework; Iteratively optimizing the perception response model, wherein a single process in the iterative optimization process includes: obtaining state information, extracting a corresponding experimental path set from the full path data set according to the state information, sorting the experimental path set by the perception response model, obtaining user behavior according to the sorting result, obtaining a reward value according to the user behavior, optimizing the policy parameters of the perception response model according to the reward value, and completing a single optimization; The current state information is obtained, and the current experimental path set is extracted from the full path data set according to the current state information. The current experimental path set is sorted by the perception response model in the iterative optimization process to obtain the experimental path sorting result; wherein the perception response model is optimized based on the current state information according to a single process in the iterative optimization process.

2. The method according to claim 1, characterized in that The full path data set is directed graph structure data, wherein a single experimental path in the full path data set includes a number of directional elements and experiments.

3. The method according to claim 1, characterized in that The perception response model includes a state space, an action space, a reward function and a state transfer function, wherein the state space includes different states, wherein the state includes input conditions, user data and search constraint variables; the action space includes several different sorting actions, and the reward function represents the relationship between executing a sorting action in a state and the reward value; the state transfer function represents the probability of executing a sorting action in a certain state and transferring to the next state.

4. The method according to claim 1, characterized in that: The user behaviors include selecting a path, browsing, exiting browsing, turning pages, modifying and creating a path.

5. The method according to claim 4, characterized in that The process of obtaining reward value based on user behavior includes: The user behavior is calculated through the reward function in the perception response model to obtain the reward value; The reward values ​​include the reward value for selecting a path, the reward value for clicking to browse, and the reward value for modifying / building a path; The reward value acquisition process of the selected path includes: r choose =K choose *p choose *y, where p choose It is the inverse of the ratio of the row number position of the selected path to the total number of rows, indicating that the ranking effect will meet the user's expectations more if the user selects a path that is closer to the front. The reward value acquisition process of the click-to-browse reward value includes: r read =K read *p read *y, where p read is the normalized coefficient value of browsing time, indicating the user's attention to the browsing results in this query; The reward value acquisition process of the modification / construction path includes: r update =k update *p update *y, where p update For p read *m update , where p read Represents the importance of the user to this search, m update It is the inverse of the measurement of the modification range and proportion. The product of the two represents the comprehensive measurement of the user's attention and the degree of conformity of the returned results to their needs; k is the overall reward adjustment coefficient, k choose , k read and k update It represents the overall reward adjustment coefficient corresponding to the reward value for selecting a path, the reward value for clicking and browsing, and the reward value for modifying / building a path, and y is the unit reward score.

6. The method according to claim 1, characterized in that The process of ranking experimental paths through the sensory response model involves: The optimal action is selected according to the optimal strategy through the perception response model in the iterative optimization process, and the current experimental path is sorted by the optimal action; the optimal strategy is the strategy for executing the optimal action under the current state information, and the optimal strategy is: Among them, π * is the optimal strategy, π represents the strategy, k represents the prediction interval time step, γ k ∈[0,1) is the discount rate, r t+k represents the reward at time step t+k, s t Represents the state at time step t, s represents arbitrary state information, S represents the state space, and t represents the time step.

7. The method according to claim 3, characterized in that The state transfer function is optimized by a gradient algorithm or a strategy approximation algorithm.

8. An experimental path ranking system based on user behavior perception feedback, characterized in that: Used to execute the method described in any one of claims 1 to 8.

Images