Intelligent routing jump recommendation method based on reinforcement learning

Abstract

The invention discloses an intelligent routing jump recommendation method based on reinforcement learning, and relates to the technical field of computer networks. Comprising the steps that 1, multi-dimensional data are collected, the multi-dimensional data comprise user role information, page types and user feedback data, 2, a state space S is defined, the collected data are subjected to structured coding, the state of each system is represented by a three-dimensional vector, all dimensions correspond to user role codes, page type codes and user feedback state codes respectively, and the user role codes, the page type codes and the user feedback state codes are stored in the state space S; 3, creating a Q table, initializing a Q value table according to the size of the state space S and the size of the action space A, the dimension of the Q table being S * A, and initializing each element Q (s, a) into a fixed value Q0, 4, carrying out action coding, carrying out unique coding on a jump target of each page, and establishing a mapping relation table of action codes and actual page jump logic, 5, optimizing the jump action based on a recommendation strategy of Q-Learning; and 6, generating and outputting a recommendation result, and receiving feedback.

Description

Technical Field

[0001] The present invention discloses an intelligent routing jump recommendation method based on reinforcement learning, and relates to the technical field of computer networks. Background Art

[0002] In existing business systems, after users submit or complete a page operation, they generally face the problem of unclear next steps. Current systems often use fixed-rule jump methods, such as pre-set business process jump logic. This approach has many drawbacks: First, because the rules are fixed, they cannot adapt to the personalized operating habits and needs of different users. For example, users with different roles have different levels of familiarity with business processes, and the required jump guidance should also be different; second, in the face of business changes, such as adding new business functions or adjusting business processes, the modification and maintenance costs of fixed rules are high, and it is difficult to respond to changes quickly; third, the rigid jump method cannot be dynamically adjusted based on real-time user feedback, resulting in a poor user experience during operation and low jump efficiency, which in turn affects user satisfaction and efficiency with the system. Summary of the Invention

[0003] The present invention aims to solve the problems of unclear jump paths in existing business systems, rigid fixed rule jump methods, inability to adapt to changes in different users or businesses, and poor user experience. Through an intelligent routing jump recommendation method based on reinforcement learning, dynamic self-learning and personalized page recommendations are achieved, thereby improving user jump efficiency and experience and reducing the cost of fixed rule configuration and maintenance.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides an intelligent route jump recommendation method based on reinforcement learning, comprising:

[0006] Step 1: Collect multi-dimensional data, including user role information, page type, and user feedback data.

[0007] Step 2: Define the state space S, perform structured coding on the collected data, and represent each system state with a three-dimensional vector. Each dimension corresponds to the user role code, page type code, and user feedback state code.

[0008] Step 3: Create a Q table and initialize the Q value table according to the size of the state space S and the action space A. The dimension of the Q table is |S|×|A|, and each element Q(s,a) is initialized to a fixed value Q0.

[0009] Step 4: Perform action coding, uniquely encode the jump target of each page, and establish a mapping relationship table between the action code and the actual page jump logic.

[0010] Step 5: Optimize jump actions based on Q-Learning recommendation strategy:

[0011] At each time step t, when the system is in state s t When generating a random number r in the interval [0,1], if r<∈, the initial value of ∈ is 0.2, from the action space A(s t ) randomly selects an action, otherwise selects the action with the largest Q value in the current state; the user performs action a t Then enter the new state s t+1 , calculate the instant reward r according to the preset rules based on user feedback t , if the valid operation r is completed t =+5, if you click the cancel button r t = -6, if no operation is performed within 3 seconds t =-3; according to Q-Learning update formula:

[0012]

[0013] Update the Q table with a learning rate of α = 0.1 and a discount factor of γ = 0.9;

[0014] Step 6: Generate and output recommendation results and receive feedback: When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table * , calculate the difference ΔQ between the maximum Q value and the second largest Q value; take action a * The corresponding target page business name, jump URL and ΔQ are encapsulated and returned to the front end through the back end; the front end sets a confidence threshold θ and compares the received ΔQ with θ. If ΔQ ≥ θ, a recommended jump pop-up window is displayed at the appropriate position on the page, and the page jump operation is triggered when the user clicks the recommended link; when ΔQ < θ, the front end does not display the jump pop-up window and executes the preset fallback rule.

[0015] Furthermore, step 1 of the intelligent route jump recommendation method based on reinforcement learning includes:

[0016] Step 11: Collect user role information and obtain user identity tags, which include administrator role tags, ordinary employee role tags, and customer role tags;

[0017] Step 12: Collect the page type. When the backend receives the frontend request, the page type is determined based on the requested URL path or interface identifier and the pre-established mapping relationship.

[0018] Step 13: Collect user feedback data and, through the front-end event monitoring mechanism, record the user's response to the recommended jump within 3 seconds of the recommendation prompt message being displayed. The response actions include clicking the jump link, clicking the cancel button, and not performing any action.

[0019] Furthermore, in step 2 of the intelligent route jump recommendation method based on reinforcement learning, each system state is represented by a three-dimensional vector, which is expressed as: s = [role, page_type, feedback_status],

[0020] Each dimension corresponds to the user role code, page type code and user feedback status code respectively.

[0021] Furthermore, step 6 of the intelligent route jump recommendation method based on reinforcement learning specifically includes:

[0022] S61: Optimal action selection. When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table. * =argmax a∈A(s) Q(s,a), and calculate the difference between the maximum Q value and the second largest Q value ΔQ=Q(s,a max )-Q(s,a second ), and return ΔQ along with the recommendation results to the front end;

[0023] S62: Recommended information assembly, action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated into JSON format data and returned to the front-end through the back-end API interface;

[0024] S63: Front-end display and jump. After receiving the recommended data, the front-end sets the confidence threshold θ and compares the received ΔQ with θ:

[0025] If ΔQ ≥ θ, a recommended jump pop-up window will be displayed at an appropriate location on the page.

[0026] The present invention also provides an intelligent route jump recommendation device based on reinforcement learning, comprising: an acquisition module, a data encoding module, a Q table management module, an action encoding module, a strategy management module and a generation output module.

[0027] The acquisition module collects multi-dimensional data, including user role information, page type, and user feedback data.

[0028] The data encoding module defines the state space S, performs structured encoding on the collected data, and represents each system state by a three-dimensional vector. Each dimension corresponds to the user role code, page type code, and user feedback state code.

[0029] The Q table management module creates a Q table and initializes the Q value table according to the size of the state space S and the action space A. The dimension of the Q table is |S|×|A|, and each element Q(s,a) is initialized to a fixed value Q0.

[0030] The action coding module performs action coding, uniquely codes the jump targets of each page, and establishes a mapping relationship table between the action coding and the actual page jump logic.

[0031] The policy management module optimizes jump actions based on Q-Learning's recommended strategy:

[0032] At each time step t, when the system is in state s t When generating a random number r in the interval [0,1], if r<∈, the initial value of ∈ is 0.2, from the action space A(s t ) randomly selects an action, otherwise selects the action with the largest Q value in the current state; the user performs action a t Then enter the new state s t+1 , calculate the instant reward r according to the preset rules based on user feedback t , if the valid operation r is completed t =+5, if you click the cancel button r t = -6, if no operation is performed within 3 seconds t =-3; according to Q-Learning update formula:

[0033]

[0034] Update the Q table with a learning rate of α = 0.1 and a discount factor of γ = 0.9;

[0035] The output module generates and outputs recommendation results and receives feedback: When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table * , calculate the difference ΔQ between the maximum Q value and the second largest Q value; take action a * The corresponding target page business name, jump URL and ΔQ are encapsulated and returned to the front end through the back end; the front end sets a confidence threshold θ and compares the received ΔQ with θ. If ΔQ ≥ θ, a recommended jump pop-up window is displayed at the appropriate position on the page, and the page jump operation is triggered when the user clicks the recommended link; when ΔQ < θ, the front end does not display the jump pop-up window and executes the preset fallback rule.

[0036] Furthermore, the acquisition module of the intelligent route jump recommendation device based on reinforcement learning performs:

[0037] Step 11: Collect user role information and obtain user identity tags, which include administrator role tags, ordinary employee role tags, and customer role tags;

[0038] Step 12: Collect the page type. When the backend receives the frontend request, the page type is determined based on the requested URL path or interface identifier and the pre-established mapping relationship.

[0039] Step 13: Collect user feedback data and, through the front-end event monitoring mechanism, record the user's response to the recommended jump within 3 seconds of the recommendation prompt message being displayed. The response actions include clicking the jump link, clicking the cancel button, and not performing any action.

[0040] Furthermore, the data encoding module of the intelligent route jump recommendation device based on reinforcement learning uses a three-dimensional vector to represent each system state, which is expressed as: s = [role, page_type, feedback_status],

[0041] Each dimension corresponds to the user role code, page type code and user feedback status code respectively.

[0042] Furthermore, the generation output module of the intelligent route jump recommendation device based on reinforcement learning performs:

[0043] S61: Optimal action selection. When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table. * =argmax a∈A(s) Q(s,a), and calculate the difference between the maximum Q value and the second largest Q value ΔQ=Q(s,a max )-Q(s,a second ), and return ΔQ along with the recommendation results to the front end;

[0044] S62: Recommended information assembly, action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated into JSON format data and returned to the front-end through the back-end API interface;

[0045] S63: Front-end display and jump. After receiving the recommended data, the front-end sets the confidence threshold θ and compares the received ΔQ with θ:

[0046] If ΔQ ≥ θ, a recommended jump pop-up window will be displayed at an appropriate location on the page.

[0047] The benefits of the present invention are:

[0048] 1. Unlike the traditional fixed rule jump method, this invention does not require frequent rule modifications to adapt to business changes and user needs. The system can automatically optimize the recommendation strategy through reinforcement learning, greatly reducing the manpower and time costs of rule configuration and maintenance;

[0049] 2. The present invention can adapt to the personalized operating habits of different users and the dynamic changes of their business. Whether it is a new or old user, whether it is an existing business process or a new business function, the system can continuously adjust the recommendation strategy through learning to maintain good recommendation results, thereby improving the versatility and scalability of the system.

[0050] 3. The introduction of the rule-based guarantee mechanism of the present invention avoids user operation interruption or recommendation interference caused by recommendation anomalies, thereby enhancing the reliability and usability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0053] Example 1

[0054] The present invention provides an intelligent route jump recommendation method based on reinforcement learning, comprising:

[0055] Step 1: Collect multi-dimensional data, including user role information, page type, and user feedback data.

[0056] For example, the collection module is deployed at key operation nodes such as the expense reimbursement form filling page, review page, and voucher upload page in the expense reimbursement system to achieve real-time collection of user behavior data, including:

[0057] S11: User Role Information Collection: The system's backend permission management module interacts with the enterprise identity authentication center via the OAuth 2.0 protocol to obtain the logged-in user's role information. If the user is identified as an ordinary employee, the "Ordinary Employee" role tag is assigned and coded as 0.

[0058] S12: Page Type Collection: When a user enters the reimbursement form page, the backend receives the request sent by the frontend using the Axios library and parses the URL as " / reimbursement / apply." Based on the pre-set mapping relationships {" / login":"Login Page"," / reimbursement / apply":"Reimbursement Form Page"," / reimbursement / review":"Review Page"}, the backend uses regular expression matching rules to determine that the page is the reimbursement form page and assigns it a code of 1.

[0059] S13: User Feedback Data Collection: After the system recommends a "View Reimbursement Standards" link to the user, the front-end uses JavaScript's setTimeout function to start a 3-second timer and, through the addEventListener event listening mechanism, records the user's feedback status. If the user clicks the link within 3 seconds, the value is 1; if the user clicks the Cancel button, the value is -1; if no action is taken after the timer expires, the value is 0.

[0060] Step 2: Define the state space S and perform structured encoding on the collected data. Each system state is represented by a three-dimensional vector, with each dimension corresponding to the user role code, page type code, and user feedback status code. Each state s∈S is represented by a three-dimensional vector s=[role,page_type,feedback_status]. To facilitate system storage and processing, binary encoding is used to compress the information in each dimension. For example, if a user clicks the recommended jump on the expense report page, the corresponding state vector s=[0,1,1] is converted to binary and stored as "000101". The system establishes a mapping table between state codes and original data to facilitate data recovery and analysis.

[0061] Step 3: Create a Q-table. Initialize the Q-value table based on the size of the state space S and the action space A. The Q-table dimension is |S| × |A|, and each element Q(s,a) is initialized to a fixed value Q0. At system startup, a Q-table with dimensions of 15 × 6 is created based on the state space S, which contains 15 different states covering different combinations of roles, pages, and feedback states, and the action space A, which contains six possible jump actions on the reimbursement form page: "View Reimbursement Standards," "Add Expense Details," "Upload Attachments," "Submit Reimbursement Form," "Return to Homepage," and "View Reimbursement History." The initial value of each element Q(s,a) in the Q-table is set to 0.3.

[0062] Step 4: Perform action coding, uniquely encoding the jump target for each page, and establish a mapping table between action codes and actual page jump logic. For action coding, encode the jump target for the reimbursement form page: "View Reimbursement Standards" is coded as 0, "Add Expense Details" is coded as 1, "Upload Attachments" is coded as 2, "Submit Reimbursement Form" is coded as 3, "Return to Home Page" is coded as 4, and "View Historical Reimbursement Records" is coded as 5. During system initialization, all action codes are checked for uniqueness and conflicts. A mapping table between action codes and actual page jump logic is also established to facilitate rapid transitions between recommendation result generation and page jumps.

[0063] Step 5: Optimize jump actions based on Q-Learning recommendation strategy:

[0064] At each time step t, when the system is in state s t When generating a random number r in the interval [0,1], if r<∈, the initial value of ∈ is 0.2, from the action space A(s t ) randomly selects an action, otherwise selects the action with the largest Q value in the current state; the user performs action a t Then enter the new state s t+1 , calculate the instant reward r according to the preset rules based on user feedback t , if the valid operation r is completed t =+5, if you click the cancel button r t = -6, if no operation is performed within 3 seconds t =-3; according to Q-Learning update formula:

[0065]

[0066] Update the Q table with a learning rate of α = 0.1 and a discount factor of γ = 0.9.

[0067] The specific process can be:

[0068] S51: Action selection strategy: At a certain time step t, the system is in state s t , such as s t =[0,1,0], indicating that ordinary employees did not take the recommended action on the reimbursement form filling page, and set the initial value of ∈ to 0.2. Generate a random number r in the interval [0,1]. If r<∈, then select the action from the action space A(s t ) randomly selects an action; otherwise, the action with the largest Q value in the current state is selected. Where ∈ decays from 0.2 to 0.05 according to the preset decay function ∈ = 0.2-0.0001×n, where n is the number of iterations.

[0069] S52: State transfer and reward calculation: User performs action a tAfter that, the system enters the new state s t+1 . The user enters the s by performing the "add expense details" action on the reimbursement form filling page. t+1 =[0,1,1] state as an example, calculate the reward r according to the rule t :If the valid operation r is completed t =+5.

[0070] S53: Q value update, adjust the Q table according to the Q-Learning update formula:

[0071]

[0072] Given a learning rate of α = 0.1 and a discount factor of γ = 0.9, t In the state, execute the action code 1 of "add expense details" as an example. The original Q(s t ,1)=0.3,in s t+1 In state, According to the above formula, substitute the data to calculate: Q(s t ,1)←0.3+0.1[5+0.9*0.3-0.3]=0.3+0.1(5+0.27-0.3)=

[0073] 0.3 + 0.1 * 4.97 = 0.797, completing the Q-value update. The system uses a distributed transaction processing mechanism to ensure the accuracy and consistency of Q-value updates. Distributed locks control access to the Q table during multi-user operations and log each update, including information such as the Q-value before the update, the state-action pair, and the reward value.

[0074] Step 6: Generate and output recommendation results and receive feedback: When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table * , calculate the difference ΔQ between the maximum Q value and the second largest Q value; take action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated and returned to the front-end through the back-end. The front-end sets a confidence threshold θ and compares the received ΔQ with θ. If ΔQ ≥ θ, a recommended jump pop-up window is displayed at an appropriate location on the page, and the page jump operation is triggered when the user clicks the recommended link. When ΔQ < θ, the front-end does not display the jump pop-up window and executes the preset fallback rule. The specific process is as follows:

[0075] Optimal action selection: When the user enters the reimbursement form filling page, state s = [0,1,0], the system traverses the Q value of each action in the action space A(s) and determines the action a corresponding to the maximum Q value *, and record the next largest Q value, and calculate the difference ΔQ between the two. For example, assuming Q(s,0)=0.3, Q(s,1)=0.797, Q(s,2)=0.3, Q(s,3)=0.3, Q(s,4)=0.3, Q(s,5)=0.3, then a * For "add expense details" (code 1), Q(s,a max )=0.797,Q(s,a second )=0.3,ΔQ=0.797-0.3=0.497. The system uses parallel computing to improve the efficiency of Q value calculation and sets a jitter threshold. When ΔQ is less than the jitter threshold and greater than the confidence threshold, the action with the largest Q value or the second largest Q value is randomly selected as the recommendation.

[0076] S6.2, Recommended information assembly: a * The business name, redirect URL, and ΔQ=0.497 of the corresponding page are encapsulated as JSON data: {"business_name":"Add expense details","url":" / reimbursement / add_detail","delta_Q":0.497}.

[0077] S6.3. The frontend sets a confidence threshold of θ = 0.3. After receiving the backend recommendation data, it compares ΔQ with θ. When ΔQ = 0.497 > 0.3, a recommendation jump pop-up window is displayed at an appropriate location on the page. When the user clicks the recommended link, the page is redirected using the window.location.href method, and the current page status information is recorded.

[0078] Rule fallback mechanism: When ΔQ < 0.3, the front-end does not display the jump pop-up window and the fallback rule is executed. Because ΔQ of this operation = 0.497 > 0.3, the pop-up window is still displayed and the fallback is not triggered.

[0079] In summary, this embodiment's method constructs a state space by collecting user roles, page types, and feedback behaviors, and dynamically optimizes page redirection strategies based on the Q-Learning algorithm. Through Q-table updates and optimal action selection, personalized recommendations are achieved. Combined with a confidence judgment mechanism and rule-based logic, this ensures accurate and reliable recommendations, effectively improving user redirection efficiency and system intelligence.

[0080] Example 2

[0081] The present invention also provides an intelligent route jump recommendation device based on reinforcement learning, comprising: an acquisition module, a data encoding module, a Q table management module, an action encoding module, a strategy management module and a generation output module.

[0082] The acquisition module collects multi-dimensional data, including user role information, page type, and user feedback data.

[0083] The data encoding module defines the state space S, performs structured encoding on the collected data, and represents each system state by a three-dimensional vector. Each dimension corresponds to the user role code, page type code, and user feedback state code.

[0084] The Q table management module creates a Q table and initializes the Q value table according to the size of the state space S and the action space A. The dimension of the Q table is |S|×|A|, and each element Q(s,a) is initialized to a fixed value Q0.

[0085] The action coding module performs action coding, uniquely codes the jump targets of each page, and establishes a mapping relationship table between the action coding and the actual page jump logic.

[0086] The policy management module optimizes jump actions based on Q-Learning's recommended strategy:

[0087] At each time step t, when the system is in state s t When generating a random number r in the interval [0,1], if r<∈, the initial value of ∈ is 0.2, from the action space A(s t ) randomly selects an action, otherwise selects the action with the largest Q value in the current state; the user performs action a t Then enter the new state s t+1 , calculate the instant reward r according to the preset rules based on user feedback t , if the valid operation r is completed t =+5, if you click the cancel button r t = -6, if no operation is performed within 3 seconds t =-3; according to Q-Learning update formula:

[0088]

[0089] Update the Q table with a learning rate of α = 0.1 and a discount factor of γ = 0.9;

[0090] The output module generates and outputs recommendation results and receives feedback: When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table * , calculate the difference ΔQ between the maximum Q value and the second largest Q value; take action a *The corresponding target page business name, jump URL and ΔQ are encapsulated and returned to the front end through the back end; the front end sets a confidence threshold θ and compares the received ΔQ with θ. If ΔQ ≥ θ, a recommended jump pop-up window is displayed at the appropriate position on the page, and the page jump operation is triggered when the user clicks the recommended link; when ΔQ < θ, the front end does not display the jump pop-up window and executes the preset fallback rule.

[0091] Since the information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0092] Likewise, the benefits of the device of the present invention are:

[0093] 1. There is no need to frequently modify rules to adapt to business changes and user needs, as is the case with traditional fixed-rule jump methods. The system can automatically optimize recommendation strategies through reinforcement learning, significantly reducing the manpower and time costs of rule configuration and maintenance.

[0094] 2. It can adapt to the personalized operating habits of different users and dynamic changes in business. Whether it is a new or old user, whether it is an existing business process or a new business function, the system can continuously adjust the recommendation strategy through learning to maintain good recommendation results, thereby improving the system's versatility and scalability;

[0095] 3. A safety net mechanism is introduced to avoid user operation interruption or recommendation interference caused by abnormal recommendations, thereby enhancing the reliability and usability of the system.

[0096] It should be noted that not all steps and modules in the above-mentioned processes and device structures are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0097] The above embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. An intelligent route jump recommendation method based on reinforcement learning, characterized by include: Step 1: Collect multi-dimensional data, including user role information, page type, and user feedback data. Step 2: Define the state space S, perform structured coding on the collected data, and represent each system state with a three-dimensional vector. Each dimension corresponds to the user role code, page type code, and user feedback state code. Step 3: Create a Q table and initialize the Q value table according to the size of the state space S and the action space A. The dimension of the Q table is |S|×|A|, and each element Q(s,a) is initialized to a fixed value Q0. Step 4: Perform action coding, uniquely encode the jump target of each page, and establish a mapping relationship table between the action code and the actual page jump logic. Step 5: Optimize jump actions based on Q-Learning recommendation strategy: At each time step t, when the system is in state s t When generating a random number r in the interval [0,1], if r<∈, the initial value of ∈ is 0.2, from the action space A(s t ) randomly selects an action, otherwise selects the action with the largest Q value in the current state; the user performs action a t Then enter the new state s t+1 , calculate the instant reward r according to the preset rules based on user feedback t , if the valid operation r is completed t =+5, if you click the cancel button r t = -6, if no operation is performed within 3 seconds t =-3; according to Q-Learning update formula: Update the Q table with a learning rate of α = 0.1 and a discount factor of γ = 0.9; Step 6: Generate and output recommendation results and receive feedback: When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table * , calculate the difference ΔQ between the maximum Q value and the second largest Q value; take action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated and returned to the front-end through the back-end. The front-end sets a confidence threshold θ and compares the received ΔQ with θ. If ΔQ ≥ θ, a recommended jump pop-up window is displayed at an appropriate location on the page. When the user clicks the recommended link, the page jump operation is triggered. When ΔQ<θ, the front-end does not display the jump pop-up window and executes the preset fallback rule.

2. The intelligent route jump recommendation method based on reinforcement learning according to claim 1 is characterized in that Step 1 includes: Step 11: Collect user role information and obtain user identity tags, which include administrator role tags, ordinary employee role tags, and customer role tags; Step 12: Collect the page type. When the backend receives the frontend request, the page type is determined based on the requested URL path or interface identifier and the pre-established mapping relationship. Step 13: Collect user feedback data and, through the front-end event monitoring mechanism, record the user's response to the recommended jump within 3 seconds of the recommendation prompt message being displayed. The response actions include clicking the jump link, clicking the cancel button, and not performing any action.

3. The intelligent route jump recommendation method based on reinforcement learning according to claim 1 is characterized in that In step 2, each system state is represented by a three-dimensional vector, expressed as: s = [role, page_type, feedback_status], Each dimension corresponds to the user role code, page type code and user feedback status code respectively.

4. The intelligent route jump recommendation method based on reinforcement learning according to claim 1 is characterized in that Step 6 specifically includes: S61: Optimal action selection. When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table. * =argmax a∈A(s) Q(s,a), and calculate the difference between the maximum Q value and the second largest Q value ΔQ=Q(s,a max )-Q(s,a second ), and return ΔQ along with the recommendation results to the front end; S62: Recommended information assembly, action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated into JSON format data and returned to the front-end through the back-end API interface; S63: Front-end display and jump. After receiving the recommended data, the front-end sets the confidence threshold θ and compares the received ΔQ with θ: If ΔQ ≥ θ, a recommended jump pop-up window will be displayed at an appropriate location on the page.

5. An intelligent route jump recommendation device based on reinforcement learning, characterized by include: Acquisition module, data encoding module, Q table management module, action encoding module, strategy management module and generation output module, The acquisition module collects multi-dimensional data, including user role information, page type, and user feedback data. The data encoding module defines the state space S, performs structured encoding on the collected data, and represents each system state by a three-dimensional vector. Each dimension corresponds to the user role code, page type code, and user feedback state code. The Q table management module creates a Q table and initializes the Q value table according to the size of the state space S and the action space A. The dimension of the Q table is |S|×|A|, and each element Q(s,a) is initialized to a fixed value Q0. The action coding module performs action coding, uniquely codes the jump targets of each page, and establishes a mapping relationship table between the action coding and the actual page jump logic. The policy management module optimizes jump actions based on Q-Learning's recommended strategy: At each time step t, when the system is in state s t When generating a random number r in the interval [0,1], if r<∈, the initial value of ∈ is 0.2, from the action space A(s t ) randomly selects an action, otherwise selects the action with the largest Q value in the current state; the user performs action a t Then enter the new state s t+1 , calculate the instant reward r according to the preset rules based on user feedback t , if the valid operation r is completed t =+5, if you click the cancel button r t = -6, if no operation is performed within 3 seconds t =-3; according to Q-Learning update formula: Update the Q table with a learning rate of α = 0.1 and a discount factor of γ = 0.9; The output module generates and outputs recommendation results and receives feedback: When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table * , calculate the difference ΔQ between the maximum Q value and the second largest Q value; take action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated and returned to the front-end through the back-end. The front-end sets a confidence threshold θ and compares the received ΔQ with θ. If ΔQ ≥ θ, a recommended jump pop-up window is displayed at an appropriate location on the page. When the user clicks the recommended link, the page jump operation is triggered. When ΔQ<θ, the front-end does not display the jump pop-up window and executes the preset fallback rule.

6. The intelligent route jump recommendation device based on reinforcement learning according to claim 5 is characterized in that The acquisition module executes: Step 11: Collect user role information and obtain user identity tags, which include administrator role tags, ordinary employee role tags, and customer role tags; Step 12: Collect the page type. When the backend receives the frontend request, the page type is determined based on the requested URL path or interface identifier and the pre-established mapping relationship. Step 13: Collect user feedback data and, through the front-end event monitoring mechanism, record the user's response to the recommended jump within 3 seconds of the recommendation prompt message being displayed. The response actions include clicking the jump link, clicking the cancel button, and not performing any action.

7. The intelligent route jump recommendation device based on reinforcement learning according to claim 5 is characterized by The data encoding module uses a three-dimensional vector to represent each system status, expressed as: s = [role, page_type, feedback_status], Each dimension corresponds to the user role code, page type code and user feedback status code respectively.

8. The intelligent route jump recommendation device based on reinforcement learning according to claim 5 is characterized by To generate the output module execute: S61: Optimal action selection. When the user enters a certain page state, the system selects the action a with the largest Q value from the action space A(s) corresponding to the current state s according to the Q table. * =argmax a∈A(s) Q(s,a), and calculate the difference between the maximum Q value and the second largest Q value ΔQ=Q(s,a max )-Q(s,a second ), and return ΔQ along with the recommendation results to the front end; S62: Recommended information assembly, action a * The corresponding target page business name, jump URL, and ΔQ are encapsulated into JSON format data and returned to the front-end through the back-end API interface; S63: Front-end display and jump. After receiving the recommended data, the front-end sets the confidence threshold θ and compares the received ΔQ with θ: If ΔQ ≥ θ, a recommended jump pop-up window will be displayed at an appropriate location on the page.

Images