Menu navigation method, apparatus, device, and medium

Abstract

The present application relates to the technical field of natural language processing and human-computer interaction, and provides a menu navigation method, device, equipment and medium, comprising obtaining a user query and a current system state; extracting a local subgraph in a menu knowledge graph based on this; inputting the user query, the current system state and the local subgraph into a menu navigation model to obtain a navigation decision action with a topological association relationship of an action space limited to the local subgraph; verifying and executing the navigation decision action to update the state and output a menu navigation result; wherein the menu navigation model is obtained by reinforcement learning training of a rule reward function based on system business rules and graph topology constraints. The present application strictly constrains the reasoning action space of a large model through a local subgraph, and guides the model to make a compliant decision in combination with a rule reward function, effectively reducing the reasoning illusion of the model, and thereby improving the accuracy and controllability of menu navigation in a complex system.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and human-computer interaction, and in particular to a menu navigation method, apparatus, device, and medium. Background Technology

[0002] In the mobile application ecosystem, menu navigation is the core hub connecting users with complex functions. Faced with limited screen space and a massive demand for services, traditional hierarchical navigation often results in key functions being hidden deep within the system and lengthy operation paths. As users shift from passive browsing to actively achieving their goals, static menus can no longer meet efficiency expectations. Current trends are driving navigation systems towards intelligent evolution, integrating scene awareness, personalized recommendations, and intent recognition, aiming to simplify multi-layered clicks into a single, direct access, and transforming navigation from a mere display of functions to intelligent service guidance.

[0003] With the rapid development of large language models, their application in intent understanding and dialogue systems is becoming increasingly widespread. However, when faced with navigation requirements involving multiple steps, large language models are prone to planning invalid operation paths, such as frequently generating menu nodes that do not exist in the actual system, or outputting broken paths that cannot be accessed coherently. On the other hand, when faced with navigation tasks with complex business premises, large language models often output decision actions that do not conform to business specifications, causing the user's navigation process to be easily interrupted in actual execution. Summary of the Invention

[0004] This invention provides a menu navigation method, apparatus, device, and medium to solve the technical problem in the prior art that invalid operation paths are easily planned and decision actions that do not conform to business specifications are frequently output when menu navigation is based on a large language model.

[0005] This invention provides a menu navigation method, comprising the following steps: Obtain the current system status of the user query and menu navigation system; the current system status includes the current node; Based on the user query and the current node, a local subgraph is retrieved and extracted from the menu knowledge graph of the menu navigation system. The user query, the current system status, and the local subgraph are input into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. Verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

[0006] According to a menu navigation method provided by the present invention, the step of retrieving and extracting local subgraphs from the menu knowledge graph of the menu navigation system based on the user query and the current node includes: Entity recognition and intent recognition are performed on the user query to extract the entity type and query intent; Based on the entity type and the query intent, an initial set of relevant nodes is retrieved from the menu knowledge graph of the menu navigation system. Input the initial set of relevant nodes, the user query, and the current node into the second language model to obtain the candidate root node output by the second language model; Starting from the candidate root node, a bounded breadth-first search is performed in the menu knowledge graph to extract a local subgraph.

[0007] According to a menu navigation method provided by the present invention, the step of inputting the user query, the current system state, and the local subgraph into a menu navigation model to obtain a navigation decision action includes: Based on the local subgraph, the user query, and the current system state, contextual prompts are constructed. The contextual prompts are input into the menu navigation model for constraint reasoning to obtain the navigation decision result output by the menu navigation model; the navigation decision result includes the navigation path; the node jumps in the navigation path are restricted by the topological relationships of the local subgraph; The navigation path in the navigation decision result is analyzed to obtain the navigation decision action.

[0008] According to a menu navigation method provided by the present invention, the step of verifying and executing the navigation decision action to update the current system state includes: The navigation decision action is verified; the verification includes at least one of permission verification, function verification, and path verification. If the verification is successful, the system call or node jump corresponding to the navigation decision action will be executed; Based on the execution result and execution action of the navigation decision action, update the historical dialogues and node states in the current system state.

[0009] According to a menu navigation method provided by the present invention, the step of outputting menu navigation results based on the updated current system state includes: Determine whether the updated current system state meets the task termination conditions; If the task termination condition is not met, then based on the updated current system state, return to the step of retrieving and extracting a local subgraph in the menu knowledge graph based on the user query and the current node, until the task termination condition is met. If the task termination condition is met, then based on the updated current system state, the menu navigation result is output; the menu navigation result includes at least the final target node, the system response message, and the reasoning path corresponding to the node sequence visited in each iteration.

[0010] According to a menu navigation method provided by the present invention, the menu navigation model is trained in the following manner: Obtain a set of thought chain samples; the set of thought chain samples includes several thought chain samples, each of the thought chain samples includes a query question description, as well as a step-by-step navigation reasoning process label and a final navigation decision answer label corresponding to the query question description; Based on the aforementioned thought chain sample set, the first large language model was subjected to supervised fine-tuning to obtain the initial menu navigation model; Obtain a sample set of navigation prompt words from a menu navigation sample system; each navigation prompt word sample in the sample set includes a sample user query, a sample partial subgraph, and a sample current system status. Each navigation prompt word sample is input into the initial menu navigation model for group sampling to obtain multiple sets of candidate navigation action output sequences corresponding to each navigation prompt word sample; Based on the system business rules and graph topology constraints of the menu navigation sample system, a rule reward function is constructed, and the total reward for each group of candidate navigation action output sequences is determined based on the rule reward function. The relative advantage is determined based on the total reward of multiple candidate navigation action output sequences within the same group, and the model parameters of the initial menu navigation model are updated based on the relative advantage to obtain a trained menu navigation model.

[0011] According to a menu navigation method provided by the present invention, determining the total reward for each group of candidate navigation action output sequences includes: The effect reward for each group of candidate navigation action output sequences is determined based on the target intersection of the path of each group of candidate navigation action output sequences and the standard navigation action output path. Based on the node jump relationships in each group of candidate navigation action output sequences and the graph topology constraints, determine the compliance reward for each group of candidate navigation action output sequences. Based on the total number of action jumps and redundant jump paths of each group of candidate navigation action output sequences, determine the efficiency reward for each group of candidate navigation action output sequences. Based on the degree of effective integration of external entity knowledge nodes and the logical rigor of each group of candidate navigation action output sequences, the integrity reward of each group of candidate navigation action output sequences is determined. Based on the performance reward, compliance reward, efficiency reward, and integrity reward, the total reward for each group of candidate navigation action output sequences is determined.

[0012] According to a menu navigation method provided by the present invention, the menu knowledge graph is constructed in the following manner: Obtain menu nodes within the menu navigation system, as well as business entity knowledge nodes extracted from the business data of the menu navigation system; Construct topological relationships between nodes; the topological relationships include at least structural relationships, semantic relationships, and inference constraint relationships. A menu knowledge graph is constructed based on the menu nodes, the business entity knowledge nodes, and the topological relationships.

[0013] According to a menu navigation method provided by the present invention, the construction of topological associations between nodes includes: The structural relationship is constructed based on the hierarchical relationship between nodes, the dependency relationship between functions and data, and the temporal relationship of business processes; The semantic relationship is constructed based on the synonym mapping relationship and hierarchical classification relationship between nodes; The inference constraint relationship is constructed based on the mutual exclusion constraints between the functional jump paths and parameter states in node interactions.

[0014] The present invention also provides a menu navigation device, comprising: The acquisition module is used to acquire the current system status of the user query and menu navigation system; the current system status includes the current node; The graph retrieval module is used to retrieve and extract local subgraphs from the menu knowledge graph of the menu navigation system based on the user query and the current node. The large model reasoning module is used to input the user query, the current system state, and the local subgraph into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. The output module is used to verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the menu navigation method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the menu navigation method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the menu navigation method as described above.

[0018] The menu navigation method, apparatus, device, and medium provided by this invention use local subgraphs in the menu knowledge graph as the reasoning knowledge context of the menu navigation model, and constrain the action space of the menu navigation model with the topological relationships of the local subgraphs. This ensures that each navigation decision strictly follows the topological structure of the menu knowledge graph, effectively suppressing the reasoning illusion of the menu navigation model. At the same time, the menu navigation model is trained through reinforcement learning using a rule reward function constructed based on system business rules and graph topological constraints. This enables the menu navigation model to generate navigation decision actions that conform to business logic under complex business constraints, improving the accuracy of menu navigation decisions while maintaining the flexibility of natural language interaction. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the menu navigation method provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of a menu knowledge graph provided in an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the menu navigation device provided in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] It should be noted that in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] The menu navigation method provided in the embodiments of the present invention can be executed by any electronic device with data processing capabilities, such as a server, processor, or terminal device. Any device with data processing capabilities can serve as the execution subject in the embodiments of the present invention. In one application scenario, the above method can be executed by a server deployed with a menu navigation system: the server receives user queries sent by the terminal device, calls a pre-built menu knowledge graph and a trained menu navigation model to understand, plan, and navigate the user query, and returns the final menu navigation result to the terminal device, which then presents it to the user. The embodiments of the present invention do not limit the specific execution subject.

[0028] This invention provides a menu navigation method. Figure 1 This is a flowchart illustrating the menu navigation method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps 110, 120, 130 and 140.

[0029] Step 110: Obtain the current system status of the user query and menu navigation system; the current system status includes the current node.

[0030] User queries refer to the descriptions of query intents expressed by users in natural language, such as: I want to check last month's phone bill, how to unsubscribe from value-added services, etc., which can cover various forms such as colloquial expressions, vague descriptions, and complex queries containing multiple sub-intents.

[0031] In practical applications, when a user needs to perform a business operation or query information, they will input the relevant user query into the menu navigation system. There are no restrictions on how the user query is obtained; it can be obtained through text input, speech recognition to text, etc.

[0032] The current system state refers to the overall environmental state of the menu navigation system when responding to a current user query. Specifically, the current system state includes not only the current node and historical access path representing the user's query location, but also the historical dialogue context between the system and the user, the system's business query cache, and the user's identity permissions and business status.

[0033] Here, the current node refers to the graph node in the menu knowledge graph that corresponds to the user's current interaction position in the menu navigation system. For example, when a user initiates a query on the personal center page, the current node is the menu node in the menu knowledge graph that is identified as the personal center.

[0034] Step 120: Based on the user query and the current node, retrieve and extract local subgraphs from the menu knowledge graph of the menu navigation system.

[0035] In this embodiment, the menu knowledge graph is a pre-constructed structured knowledge base that organizes all functional nodes and their relationships in the menu navigation system in the form of a directed graph. Each node in the menu knowledge graph corresponds to a functional entity in the menu navigation system, and each node carries structured attribute information such as a unique identifier, functional description, access prerequisites, interface, and business constraint rules; the edges in the menu knowledge graph represent the topological relationships between nodes.

[0036] Topological relationships refer to the various directed or undirected connections between nodes in a menu knowledge graph. They are used to represent multi-dimensional association information such as hierarchical subordination, semantic relationships, business dependencies, or functional jump constraints between nodes. These topological relationships collectively constitute the topological structure of the menu knowledge graph, determining the range of neighboring nodes reachable from any given node, as well as the reachability boundaries between nodes in terms of business logic.

[0037] A local subgraph is a structured subgraph extracted from the menu knowledge graph, centered on the current node and extending along topological relationships. It includes the current node and several neighboring nodes directly or indirectly connected to it through topological relationships, as well as the topological relationships and node attribute information between these nodes. Neighboring nodes refer to nodes in the menu knowledge graph that have a topological relationship with the current node and are within a specified hop count range.

[0038] In this embodiment, by constructing a local subgraph, structured local knowledge related to the current navigation location and user query can be accurately provided to the menu navigation model, avoiding the need to input the entire menu knowledge graph. This effectively reduces computational overhead while providing accurate knowledge support for the reasoning of the menu navigation model.

[0039] In one example, the intent information in the user query and the graph location information of the current node can be used to locate the initial node range related to the current query in the menu knowledge graph. Based on the initial node range and the current node, several hop neighbor nodes are retrieved outward according to the topological relationship. Finally, a structured local subgraph including related nodes, topological relationship and node attributes is extracted as the knowledge context input for the subsequent reasoning stage.

[0040] Step 130: Input the user query, the current system status, and the local subgraph into the menu navigation model to obtain a navigation decision action; the action space of the navigation decision action is limited by the topological association relationship of the local subgraph.

[0041] The menu navigation model is obtained by training the first large language model using reinforcement learning based on a rule-based reward function. The first large language model refers to a pre-trained large-scale language model with natural language understanding and generation capabilities. Its specific architecture is not restricted and can be any pre-trained language model based on the Transformer architecture.

[0042] In this embodiment, the menu navigation model is based on the first large language model and is trained by reinforcement learning of the first large language model through a rule reward function specifically designed for menu navigation tasks, so that the menu navigation model has the ability to perform stable reasoning and navigation decisions under the structural constraints of the menu knowledge graph.

[0043] The rule-based reward function is an evaluation function used to quantify and calculate reward signals for the navigation decision actions generated by the menu navigation model during the reinforcement learning training of the first language model.

[0044] Here, the rule-based reward function is constructed based on system business rules and graph topology constraints. System business rules refer to the rule constraints defined by business logic in the menu navigation system, used to evaluate the compliance of navigation decisions generated by the menu navigation model during reinforcement learning training. For example, they check whether the navigation path meets user permission requirements, whether the business operation meets precondition constraints, and whether the service response contains necessary business information. Graph topology constraints refer to the reachability constraints between nodes in the menu knowledge graph defined by topological relationships. These constraints require that each node jump involved in the navigation decision actions generated by the menu navigation model must follow the topological relationships existing in the menu knowledge graph to ensure the compliance of the navigation path in the menu knowledge graph structure.

[0045] It should be understood that by injecting system business rules and graph topology constraints into the reinforcement learning training process in the form of rule reward functions, the menu navigation model can obtain structured and automatically computed reward signals during training, thereby guiding the menu navigation model to gradually learn to generate navigation decision actions that conform to business logic and follow the structural constraints of the menu knowledge graph.

[0046] In the specific reasoning process, the user query, the current system state, and the local subgraph are structurally combined into the input context of the menu navigation model. The menu navigation model performs semantic understanding and constraint reasoning based on this input context and outputs navigation decision actions.

[0047] It should be noted that navigation decision actions refer to the specific execution instructions determined by the menu navigation model under the constraints of the current system state and the local subgraph. These can include jumping to a neighboring node in the local subgraph, executing the business function corresponding to the current node, clarifying the user's query intent, or returning specific business information to the user. The action space of navigation decision actions is limited by the topological relationships of the local subgraph. That is, when outputting navigation decision actions, the menu navigation model can only select from the set of operations allowed by the range of nodes contained in the local subgraph and the corresponding topological relationships, and cannot generate actions outside the topological relationships of the local subgraph.

[0048] Step 140: Verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state.

[0049] Upon receiving the navigation decision action output by the menu navigation model, the system first verifies the action to confirm its executable capability under the current system state. After successful verification, it executes the corresponding system call or node jump operation, obtains the execution result, and updates the current system state based on the result and the executed navigation decision action. For example, it updates the current node to the target node after the jump. The interaction information is recorded in the history of the dialogue to ensure the current system state accurately reflects the latest navigation progress.

[0050] After the current system state is updated, the menu navigation result is output based on the updated current system state. Here, the menu navigation result refers to the navigation target information finally determined by the menu navigation system after one or more navigation decision actions, including but not limited to the final target menu node identifier, the business operation entry point provided to the user, or the execution feedback of the directly executed business function, etc.

[0051] It should be noted that when the navigation task corresponding to the user query is relatively complex and a single navigation decision is insufficient to fully meet the user's query needs, the local subgraph can be retrieved again based on the updated current system state and input into the menu navigation model for the next round of reasoning. Through multiple rounds of iteration, the menu navigation result that meets the user's query needs can be output based on the updated current system state.

[0052] The menu navigation method of this invention uses local subgraphs in the menu knowledge graph as the reasoning knowledge context of the menu navigation model, and constrains the action space of the menu navigation model with the topological relationships of the local subgraphs. This ensures that each navigation decision strictly follows the topological structure of the menu knowledge graph, effectively suppressing the reasoning illusion of the menu navigation model. At the same time, the menu navigation model is trained through reinforcement learning using a rule reward function constructed based on system business rules and graph topological constraints. This enables the menu navigation model to generate navigation decision actions that conform to business logic under complex business constraints, improving the accuracy of menu navigation decisions while maintaining the flexibility of natural language interaction.

[0053] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0054] In some embodiments, the step of retrieving and extracting local subgraphs from the menu knowledge graph of the menu navigation system based on the user query and the current node includes: Entity recognition and intent recognition are performed on the user query to extract the entity type and query intent; Based on the entity type and the query intent, an initial set of relevant nodes is retrieved from the menu knowledge graph of the menu navigation system. Input the initial set of relevant nodes, the user query, and the current node into the second language model to obtain the candidate root node output by the second language model; Starting from the candidate root node, a bounded breadth-first search is performed in the menu knowledge graph to extract a local subgraph.

[0055] Entity recognition is the process of identifying named entities with specific business semantics contained in user queries. The recognition result is described by entity type, which indicates the business category to which the entity belongs. Entity types include, but are not limited to, product categories, business function categories, and account status categories.

[0056] Intent recognition is the process of analyzing and classifying the operational intent contained in a user's query. The recognition result describes the type of operation the user expects the menu navigation system to perform, based on the query intent. Query intents include, but are not limited to, status queries, function processing, rights and benefits queries, and billing queries. Furthermore, a single user query may contain multiple query intents simultaneously, forming a compound query.

[0057] The initial relevant node set refers to the set of nodes in the menu knowledge graph that are related to the entity type and query intent of the current user's query, including two types of nodes: menu nodes and business entity knowledge nodes.

[0058] Specifically, based on the entity type extracted by entity recognition, business entity knowledge nodes matching the entity type are retrieved in the menu knowledge graph; simultaneously, based on the query intent extracted by intent recognition, menu nodes corresponding to the query intent are retrieved in the menu knowledge graph; the retrieved menu nodes and business entity knowledge nodes are merged to form an initial set of related nodes.

[0059] In this embodiment, the second large language model refers to a large language model used to perform semantic analysis on the initial set of relevant nodes, generate intent summaries, and output candidate root nodes. The second large language model and the menu navigation model can be the same large language model, or they can be different large language models that are deployed separately; this embodiment does not impose any restrictions on this.

[0060] Specifically, the attribute information of each node in the initial relevant node set, the user query text, and the current node identifier are input into the second language model. The second language model performs intent summarization on the user query and, combined with the location information of the current node, sorts and analyzes the relevance of each node in the initial relevant node set to the user query, thereby determining the most suitable candidate root node as the starting point for local subgraph extraction.

[0061] Here, the candidate root node can be a single node or multiple nodes sorted by priority; when there are multiple candidate root nodes, the candidate root node with the highest priority is selected as the starting point for the subsequent bounded breadth-first search.

[0062] Bounded breadth-first search refers to starting from the candidate root node, traversing the topological relationships in the knowledge graph within the set maximum hop count, and extracting all nodes reachable within the hop count and the topological relationships between these nodes, thereby forming a structured local subgraph containing the candidate root node and its several hop neighbors.

[0063] The maximum number of hops can be adjusted based on the menu hierarchy depth and computing resource constraints of the actual business scenario, and there are no restrictions on it.

[0064] The menu navigation method of this invention combines semantic understanding of user queries with structured retrieval of menu knowledge graphs, and uses the semantic reasoning capabilities of a second language model to intelligently filter candidate nodes, ensuring that the extracted local subgraphs are highly relevant to the current user query.

[0065] In some embodiments, inputting the user query, the current system state, and the local subgraph into the menu navigation model to obtain a navigation decision action includes: Based on the local subgraph, the user query, and the current system state, contextual prompts are constructed. The contextual prompts are input into the menu navigation model for constraint reasoning to obtain the navigation decision result output by the menu navigation model; the navigation decision result includes the navigation path; the node jumps in the navigation path are restricted by the topological relationships of the local subgraph; The navigation path in the navigation decision result is analyzed to obtain the navigation decision action.

[0066] It should be understood that contextual prompts are text inputs that organize and arrange the structured information of the local subgraph, the user query text, and the current system state according to the format requirements specified by the menu navigation model. This information is then passed to the menu navigation model in a way that the menu navigation model can process, providing a complete contextual environment for the constraint reasoning of the menu navigation model.

[0067] In one example, the contextual prompts can first describe the role positioning and decision-making constraints of the menu navigation model, explicitly requiring the menu navigation model to strictly follow the topological relationships of the local subgraph when making decisions; secondly, the local subgraph is presented as the knowledge input of the menu navigation model in a structured text format; then, the current node and the list of executable actions determined based on the local subgraph are indicated, clearly informing the menu navigation model of the currently available action space; finally, the user query text and the historical dialogue records in the current system state are input together as the complete interactive context required for this round of reasoning.

[0068] Constrained reasoning refers to the reasoning process by which a menu navigation model analyzes the intent of a user query, evaluates and compares each action in the list of executable actions, and ultimately determines the optimal navigation decision under the constraints of the topological relationships of the local subgraph defined by contextual prompts.

[0069] The navigation decision result is the complete decision content output by the menu navigation model based on constraint reasoning, which typically includes the following three fields: Navigation path: Describe the node jump path planned for this navigation in the format of current node → action to be performed → target node. Each node jump in the navigation path is limited by the existing topological relationships in the local subgraph and must not jump to nodes outside the local subgraph. System response script: Natural language response content for users, used to explain to users the operation that will be performed; Reasoning chain: This records the reasoning process of the menu navigation model when making this navigation decision, including the analysis of user intent, the evaluation and comparison of each executable action, and the final decision rationale, providing a basis for the interpretability of the navigation decision results.

[0070] In this embodiment, after obtaining the navigation decision result output by the menu navigation model, the navigation decision result output by the menu navigation model is parsed, and the specific operation instructions planned for this navigation, i.e., the navigation decision action, are extracted from the navigation path field.

[0071] Among them, the navigation decision action can be a node jump action that jumps to a neighbor node in a local subgraph, a system call action that calls the business function interface corresponding to the current node, an information return action that returns specific business information to the user, or a clarification action that initiates an intent clarification request to the user.

[0072] The menu navigation method of this invention, by structuring and encoding local subgraphs, user queries, and the current system state into contextual prompts and inputting them into the menu navigation model for constraint reasoning, ensures that each node jump in the navigation path generated by the menu navigation model is strictly limited by the topological relationships of the local subgraph. At the same time, the reasoning chain field included in the navigation decision results provides a complete and traceable record of the basis for each navigation decision, effectively improving the interpretability of the menu navigation system.

[0073] In some embodiments, verifying and executing the navigation decision action to update the current system state includes: The navigation decision action is verified; the verification includes at least one of permission verification, function verification, and path verification. If the verification is successful, the system call or node jump corresponding to the navigation decision action will be executed; Based on the execution result and execution action of the navigation decision action, update the historical dialogues and node states in the current system state.

[0074] Before executing a navigation decision, the decision is validated to ensure it can be performed under the current system state. Specifically, the validation includes at least one or more of the following validation types: Permission verification: This refers to the process of verifying whether the current user has the necessary access or operation permissions for the target nodes or business functions involved in the navigation decision-making action. This may include verifying the user's login status, real-name authentication status, and account status. Specific verification items are determined based on the access prerequisites configured for the target node. For example, when the navigation decision-making action involves a rights-related menu node, it is necessary to verify whether the user has completed real-name authentication and whether their account status is normal. Functional verification: This refers to the process of verifying whether the target functional nodes involved in the navigation decision action are currently in an available state. The target functional nodes may be temporarily unavailable due to system maintenance, service degradation, or other reasons. Functional verification aims to ensure that the relevant functional services are in a normal and available state before executing the navigation decision action, thus avoiding navigating the user to an unavailable functional entry point.

[0075] Path verification: This refers to the process of verifying whether the node jump path planned by the navigation decision action actually exists in the menu knowledge graph, and whether the user has access rights to each node on the path. By comparing the topological relationship of the menu knowledge graph, it is confirmed that each node jump in the navigation path corresponds to the actual topological relationship in the menu knowledge graph, thus preventing invalid jumps outside the topological structure of the menu knowledge graph.

[0076] After the above verification is passed, the specific operation corresponding to the navigation decision action is executed. Specifically, if the navigation decision action is a node jump action, the current node is updated to the target node pointed to by the navigation path, and the user is provided with access to the business functions of the target node through the API interface or function entry corresponding to the target node; if the navigation decision action is a system call action, the backend API interface specified by the navigation decision action is called to obtain business data, which serves as the data basis for generating the system response script.

[0077] If the verification fails, the execution of the current navigation decision action is interrupted, and a prompt message is generated for the user based on the business constraint rules configured in the corresponding node of the menu knowledge graph. For example, if the permission verification finds that the user's account is in arrears and the current navigation decision action involves the handling of rights and interests, the action is interrupted and a prompt message is sent to the user. At the same time, the navigation of the corresponding node of the payment process is actively triggered to guide the user to complete the operation.

[0078] After completing the navigation decision action, the complete record of this interaction, including but not limited to the current system state before this interaction, the executed navigation decision action, the execution result, and the system response, is added to the history dialogue to continuously maintain the context information of multi-round interactions. At the same time, the node state in the current system state is updated to the latest current node reached after the execution of this navigation decision action to ensure that the reasoning in subsequent rounds can be based on the latest navigation location information.

[0079] The menu navigation method of this invention performs permission verification, function verification, and path verification before executing navigation decision actions. It comprehensively verifies navigation decision actions from the perspectives of user permissions, function availability, and path compliance, effectively preventing invalid navigation caused by insufficient permissions, unavailable functions, or invalid paths. At the same time, by updating historical dialogues and node states in a timely manner after execution, it provides complete and accurate contextual information support for continuous navigation decisions in multi-turn dialogue scenarios.

[0080] In some embodiments, the step of outputting menu navigation results based on the updated current system state includes: Determine whether the updated current system state meets the task termination conditions; If the task termination condition is not met, then based on the updated current system state, return to the step of retrieving and extracting a local subgraph in the menu knowledge graph based on the user query and the current node, until the task termination condition is met. If the task termination condition is met, then based on the updated current system state, the menu navigation result is output; the menu navigation result includes at least the final target node, the system response message, and the reasoning path corresponding to the node sequence visited in each iteration.

[0081] Here, the task termination condition refers to the judgment made based on the updated current system state, that is, whether the currently executed navigation decision action sequence has fully satisfied the navigation requirements corresponding to the user's query, and there is no need to execute any additional navigation decision actions.

[0082] In one example, if the business function corresponding to the updated current node can directly respond to all the user's query intent, or if the menu navigation model explicitly outputs a task end marker in the navigation path when generating navigation decision results, then the task termination condition is determined to be met.

[0083] When the updated system state is determined not to meet the task termination condition, it indicates that the current navigation task is not yet complete, and the user's navigation needs have not been fully met. At this point, using the updated system state as input, the process returns to re-examining and extracting local subgraphs from the menu knowledge graph based on the updated current node. Reasoning and decision-making are then performed on the new local subgraph and user query, forming an iterative closed loop of local subgraph retrieval, constraint reasoning, verification execution, state update, and termination judgment, until the task termination condition is met. Through multiple iterations, the menu navigation system can handle complex composite queries that require multiple node jumps and system calls to be fully satisfied.

[0084] It should be noted that the menu navigation result in this embodiment is the final result output by the menu navigation system to the user and the upper-level system after completing the current navigation task, and it includes at least the following three components: The final target node: This refers to the menu knowledge graph node identified by the current node in the current system state at the end of this navigation task, representing the final business function location reached by this navigation task. The target node can be a leaf node that directly carries the business function required by the user's query, or a function node that directly returns business data after executing a system call action; System response script: refers to the natural language response content addressed to the user, which describes the final execution result of this navigation task, including the execution result of the business function corresponding to the target node, the business information provided to the user, and the description of optional follow-up actions.

[0085] Inference Path: A complete record of the sequence of nodes visited in each iteration throughout the navigation task, presented as a sequence of node identifiers. This serves as an interpretable log output for the menu navigation system, providing comprehensive audit evidence for subsequent menu navigation result tracing, problem investigation, and continuous optimization. Simultaneously, the menu navigation system collects successful and failed interaction trajectories for further optimization of the menu knowledge graph structure or fine-tuning of the menu navigation model, enabling continuous learning and system optimization.

[0086] The menu navigation method of this invention, by judging the task termination condition and supporting multi-round iterative closed loop, enables the menu navigation system to handle complex user queries that require multiple steps and multiple node jumps to be fully satisfied, thus improving the coverage of complex intent query scenarios. At the same time, the reasoning path is output as an interpretable log, so that the complete process of each navigation decision can be traced and audited, meeting the business scenario's requirements for the interpretability of the navigation process.

[0087] In some embodiments, the menu navigation model is trained in the following manner: Obtain a set of thought chain samples; the set of thought chain samples includes several thought chain samples, each of the thought chain samples includes a query question description, as well as a step-by-step navigation reasoning process label and a final navigation decision answer label corresponding to the query question description; Based on the aforementioned thought chain sample set, the first large language model was subjected to supervised fine-tuning to obtain the initial menu navigation model; Obtain a sample set of navigation prompt words from a menu navigation sample system; each navigation prompt word sample in the sample set includes a sample user query, a sample partial subgraph, and a sample current system status. Each navigation prompt word sample is input into the initial menu navigation model for group sampling to obtain multiple sets of candidate navigation action output sequences corresponding to each navigation prompt word sample; Based on the system business rules and graph topology constraints of the menu navigation sample system, a rule reward function is constructed, and the total reward for each group of candidate navigation action output sequences is determined based on the rule reward function. The relative advantage is determined based on the total reward of multiple candidate navigation action output sequences within the same group, and the model parameters of the initial menu navigation model are updated based on the relative advantage to obtain a trained menu navigation model.

[0088] It should be noted that the Mind Chain sample set is a labeled training dataset used for supervised fine-tuning of the first major language model. Each Mind Chain sample consists of three parts: Query Query Description: Refers to a complete description of a specific menu navigation task scenario, including the user query content in this scenario, the current system status, and the structured representation of relevant local subgraphs extracted from the menu knowledge graph; Step-by-step navigation reasoning process tag: refers to the step-by-step breakdown description of the navigation reasoning process corresponding to the query question description, recording the complete reasoning chain from user intent analysis to path constraint evaluation, from candidate action comparison to decision selection; Final navigation decision answer label: refers to the label content of the optimal navigation decision result corresponding to the description of the query question, which is the target answer that the menu navigation model should learn to output.

[0089] In one example, the construction of the thought chain sample set adopts a combination of manual annotation and synthetic generation. For typical business scenarios, business experts manually annotate the step-by-step navigation reasoning process and the final navigation decision answer labels; for long-tail complex scenarios, large-scale language models with preliminary reasoning capabilities can be used to assist in synthesizing reasoning trajectories, which are then manually reviewed and corrected, thereby constructing a thought chain sample set covering multiple business domains and multiple query complexities.

[0090] It should be understood that supervised fine-tuning refers to a training process that uses a set of thought chain samples labeled with step-by-step navigation reasoning process labels and final navigation decision answer labels to perform supervised updates on the model parameters of the first language model with the goal of minimizing the error between the output of the first language model and the labeled answer.

[0091] Specifically, the first step involved fine-tuning the primary language model using a simple menu navigation scenario with a single turn and short inference chain, enabling it to quickly establish a basic understanding of the format and specifications for menu navigation tasks. Subsequently, the inference chain length and query complexity of the training samples were gradually increased, introducing training samples with multi-turn dialogues, multi-intent composite queries, and complex navigation scenarios with business constraints. This allowed the primary language model to gradually master the ability to perform long-chain reasoning under the constraints of the menu knowledge graph structure and output standardized navigation decisions. After supervised fine-tuning, the resulting initial menu navigation model possessed basic navigation reasoning capabilities, could understand the input and output formats of menu navigation tasks, and generate relatively reasonable navigation decisions in common navigation scenarios.

[0092] It should be understood that the menu navigation sample system is a simulated menu navigation system used to provide a navigation task environment during the reinforcement learning training phase. It includes a menu knowledge graph for training and corresponding business rule configurations. Each navigation prompt word sample in the navigation prompt word sample set corresponds to a specific menu navigation scenario, which consists of three parts: the sample user query, the sample local subgraph, and the sample current system state. These three parts constitute the complete input context required for the menu navigation model to make inference decisions in this scenario, and its format is consistent with the context prompt word format in the actual application inference phase.

[0093] Group sampling refers to the process of independently sampling from the current initial menu navigation model using certain parameters to generate N independent candidate navigation action output sequences for the same navigation prompt word sample. Here, N is a preset group size, for example, N=4.

[0094] Each group of candidate navigation action output sequences contains a complete navigation decision result. However, due to the randomness of sampling, the N groups of candidate navigation action output sequences corresponding to the same navigation prompt word sample may differ in terms of reasoning path, decision basis, or system response language. Through a group sampling mechanism, without relying on manual feedback annotation, the relative merits of each candidate navigation action output sequence are evaluated by horizontal comparison within the group.

[0095] In this embodiment, during the reinforcement learning training phase, the quality of each group of candidate navigation action output sequences is used to calculate a numerical total reward through a rule-based reward function, thereby automating the training process of the initial menu navigation model. Here, the total reward... Determined by the weighted sum of rewards from each dimension: ;in, The preset weighting coefficients for each reward dimension, Let k be the reward value for each dimension, and k be the number of reward dimensions.

[0096] After calculating the total reward for each group of candidate navigation action output sequences, the average of the total rewards for the N groups of candidate navigation action output sequences within the same group is calculated as the baseline value. , Then, output sequence y for each group of candidate navigation actions within the same group. j Calculate its relative advantage , ;like >0 indicates the candidate navigation action output sequence Its performance in this group is better than average; therefore, the probability of the initial menu navigation model generating this type of output should be increased through gradient updates. If the value is less than 0, the probability of the initial menu navigation model generating this type of output should be reduced.

[0097] Based on the aforementioned relative advantages, the following loss function L is used. GRPO Update the model parameters of the initial menu navigation model: ; in, For new strategies to be optimized, This is the old strategy used when sampling groups. The importance weights are used to measure the degree of deviation between the old and new strategies; N is the preset group size. Output the sequence for the j-th candidate navigation action; The probability of generating the j-th candidate navigation action output sequence; The relative advantage of the output sequence for the j-th candidate navigation action determines the direction and intensity of the gradient update; is the KL divergence penalty coefficient.

[0098] By optimizing and iteratively updating the model parameters of the initial menu navigation model using the strategies described above, the initial menu navigation model gradually learns to generate navigation decision results that follow graph topology constraints, conform to system business rules, have efficient paths, and complete reasoning, ultimately resulting in a well-trained menu navigation model.

[0099] The menu navigation method of this invention first establishes basic navigation reasoning capabilities through supervised fine-tuning based on a thought chain sample set, and then introduces a rule reward function constructed based on system business rules and graph topology constraints for group strategy optimization training, forming a phased hybrid training strategy. This enables the menu navigation model to further learn the ability to generate high-quality navigation decision results under complex business constraints and graph topology constraints, based on mastering basic reasoning, effectively improving the reasoning generalization ability of the menu navigation model.

[0100] In some embodiments, determining the total reward for each group of candidate navigation action output sequences includes: The effect reward for each group of candidate navigation action output sequences is determined based on the target intersection of the path of each group of candidate navigation action output sequences and the standard navigation action output path. Based on the node jump relationships in each group of candidate navigation action output sequences and the graph topology constraints, determine the compliance reward for each group of candidate navigation action output sequences. Based on the total number of action jumps and redundant jump paths of each group of candidate navigation action output sequences, determine the efficiency reward for each group of candidate navigation action output sequences. Based on the degree of effective integration of external entity knowledge nodes and the logical rigor of each group of candidate navigation action output sequences, the integrity reward of each group of candidate navigation action output sequences is determined. Based on the performance reward, compliance reward, efficiency reward, and integrity reward, the total reward for each group of candidate navigation action output sequences is determined.

[0101] Here, the standard navigation action output path refers to the correct navigation path that is pre-marked or determined by the rules of the menu navigation sample system for the navigation task corresponding to the current navigation prompt word sample.

[0102] The performance reward is measured by calculating the target intersection between the navigation path in the candidate navigation action output sequence and the standard navigation action output path. The larger the target intersection, the higher the degree of overlap between the navigation path planned by the candidate navigation action output sequence and the standard navigation action output path, and the higher the performance reward value; conversely, the smaller the target intersection, the lower the performance reward value. The performance reward is used to guide the initial menu navigation model to gradually learn to output navigation decision results that highly match the standard navigation path during the training process.

[0103] The compliance reward is a binary reward metric used to determine whether each node jump in the navigation path of the candidate navigation action output sequence strictly corresponds to the actual topological relationships existing in the menu knowledge graph: if all node jumps in the navigation path of the candidate navigation action output sequence conform to the graph topological constraints of the menu navigation sample system, the compliance reward is 1; if any node jump in the candidate navigation action output sequence violates the graph topological constraints, the compliance reward is 0. The compliance reward is used to prevent the initial menu navigation model from generating illusory paths that do not exist in the menu knowledge graph during training.

[0104] Efficiency rewards are used to measure the simplicity and efficiency of the navigation paths in candidate navigation action output sequences: Candidate navigation action output sequences with fewer total steps and no redundant jump paths receive higher efficiency rewards; candidate navigation action output sequences with repeated jump paths that visit already visited nodes, or redundant jump paths where the total number of steps far exceeds the necessary steps to complete the current navigation task, are penalized. Efficiency rewards guide the initial menu navigation model during training to learn to complete the menu navigation task with the fewest steps, avoiding the generation of redundant navigation paths containing invalid detours.

[0105] The integrity reward is used to comprehensively evaluate the candidate navigation action output sequence from two dimensions. First, it assesses the effectiveness of the candidate navigation action output sequence in integrating business entity knowledge nodes from the menu knowledge graph within the inference chain. Specifically, it evaluates whether the candidate navigation action output sequence fully utilizes the business entity knowledge nodes related to the user's query in the menu knowledge graph to ensure sufficient knowledge basis for navigation inference. Second, it assesses the logical rigor of the inference chain of the candidate navigation action output sequence and the standardization of the system response language fields. Specifically, it evaluates whether the inference process conforms to logical inference standards and whether the system response language contains all the necessary business information required by the user. The integrity reward is output after comprehensively scoring these two dimensions using a reward model. This guides the initial menu navigation model to fully utilize the business entity knowledge nodes in the menu knowledge graph during training, generating high-quality navigation decision results with sufficient knowledge basis, rigorous inference logic, and standardized system response language.

[0106] In this embodiment, the rewards of the above four dimensions are weighted and summed to determine the total reward for each group of candidate navigation action output sequences. The preset weight coefficients of each reward dimension can be adjusted and configured according to the importance of each dimension in the actual business scenario.

[0107] The menu navigation method of this invention constructs a multi-dimensional rule reward function from four dimensions: effectiveness, compliance, efficiency, and completeness. This transforms the core evaluation criteria for navigation path accuracy, graph structure compliance, path simplicity and efficiency, and the completeness of business knowledge utilization in menu navigation tasks into automatically calculated quantitative reward signals, effectively guiding the menu navigation model to learn and generate high-quality navigation decision results under complex constraints.

[0108] In some embodiments, the menu knowledge graph is constructed as follows: Obtain menu nodes within the menu navigation system, as well as business entity knowledge nodes extracted from the business data of the menu navigation system; Construct topological relationships between nodes; the topological relationships include at least structural relationships, semantic relationships, and inference constraint relationships. A menu knowledge graph is constructed based on the menu nodes, the business entity knowledge nodes, and the topological relationships.

[0109] Menu nodes are structured representations of functional entry points at each level in a menu navigation system, corresponding to all menu functions within the system. By comprehensively analyzing the menu navigation system, all levels of menu nodes are extracted to cover all intent distribution scenarios supported by the system. Each menu node carries structured attribute information such as function identifier, hierarchy, detailed function description, operation flow, access prerequisites, API interface information, and business constraint rules. During the organization of menu nodes, data cleaning and standardized representation processing are required. For example, colloquial expressions like "check traffic" in function names are standardized to "traffic query," eliminating semantic ambiguity caused by inconsistent naming.

[0110] Business entity knowledge nodes are structured representations of various business entities within the business domain covered by the menu navigation system. In this embodiment, business entity knowledge nodes are extracted from the business data of the menu navigation system. The sources of this business data include, but are not limited to, business documents, frequently asked questions databases, and historical dialogue records. Key business concepts, product entity information, and thesaurus are extracted from these data to form an extensible semantic network. Each business entity knowledge node carries multi-dimensional attribute information, including a unique identifier, standard name, thesaurus list, concept attribution, entity attribute characteristics, business tags, associated menu entry identifiers, and business constraint rules.

[0111] Taking the operator's APP menu navigation scenario as an example, the M Video VIP monthly card can be used as a business entity knowledge node. Its synonym list includes various commonly used colloquial expressions such as M Video monthly card, M Video membership monthly card, and M Video VIP monthly fee. Its associated menu entry identifiers include menu_031, menu_001, etc. By setting the synonym list and associated menu entry identifiers, the menu navigation model can accurately identify the same business entity that the user is pointing to using different expressions during inference, and locate the corresponding menu node accordingly.

[0112] Topological relationships are the core connecting elements of the menu knowledge graph. By modeling the various semantic and logical relationships between menu nodes and business entity knowledge nodes, between menu nodes themselves, and between business entity knowledge nodes, a graph relationship network is formed that supports structured reasoning in the menu navigation model. Among these, structural relationships, semantic relationships, and reasoning constraint relationships together constitute the core hierarchical system of topological relationships.

[0113] In this embodiment, after business experts review and confirm the accuracy of the menu nodes, business entity knowledge nodes, and topological relationships, all menu nodes and business entity knowledge nodes are imported into the graph database. A globally unique ID is assigned to each node, and various topological relationships are established in the graph database. Simultaneously, the business entity knowledge nodes are mounted to their corresponding menu nodes, constructing a complete menu knowledge graph. Furthermore, based on domain-specific reasoning rules, the menu knowledge graph is supplemented with knowledge, enabling transitive expansion of node attributes and establishing a constraint consistency check mechanism. This enhances the menu knowledge graph's ability to fully cover business domain logic and handle complex menu navigation tasks.

[0114] For ease of understanding, please refer to Figure 2 The following example illustrates a partial scene from the menu knowledge graph. For instance... Figure 2 As shown, the menu knowledge graph constructed in this embodiment breaks away from the structural fragmentation of traditional menu navigation that relies solely on menu hierarchy, and integrates menu nodes (such as...) Figure 2 The right-angled rectangles represent the homepage, service hall, rights and interests area, rights and interests processing, and rights and interests cancellation, respectively, and the business entity knowledge nodes (such as...). Figure 2 The basic package, brand A package, and audio / video membership shown in the rounded rectangle have been deeply integrated.

[0115] This menu knowledge graph models three core topological relationships that support constrained reasoning in large models: Structural relationships: See Figure 2As indicated by the solid arrows; for example, "Home" points to "Service Hall", "Service Hall" points to "Rights and Benefits Zone", and "Rights and Benefits Zone" further branches to "Rights and Benefits Processing" and "Rights and Benefits Cancellation"; Semantic relation: see Figure 2 As shown by the dashed arrows; for example, there is a hierarchical semantic relationship between the business entity knowledge nodes "Basic Package" and "Brand A Package"; there is a semantic relationship of association between "Brand A Package" and "Service Hall"; while the business entity "Audio and Video Membership" is directly attached to the menu node "Rights and Benefits Processing" across levels through the semantic relationship of association with the processing entry. Inference constraints: see Figure 2 As shown by the dashed arrow at the midpoint; for example. Figure 2 The "Benefits Processing" and "Benefits Cancellation" menu nodes are mutually exclusive, meaning that users cannot directly cancel their subscriptions for the same business entity if the subscription is not active. Furthermore, there is a state-based inference constraint between the "Brand A Package" and "Benefits Processing" menu nodes. If business rules require that only "Brand A Package" users can subscribe to a specific "Audio / Video Membership," the large model will prioritize verifying the user's state when planning the navigation path to the "Benefits Processing" node. If the state is inconsistent, the large model's inference path will be blocked by this constraint, thus avoiding the generation of illusory paths that are illegal or inoperable.

[0116] The menu navigation method of this invention incorporates menu nodes of the menu navigation system and business entity knowledge nodes extracted from business data into a menu knowledge graph, and establishes a topological association system covering three dimensions: structural relationship, semantic relationship, and reasoning constraint relationship. This transforms discrete menu functions into a structured knowledge base with rich semantic associations, providing comprehensive and accurate knowledge support for graph-constrained reasoning of the menu navigation model.

[0117] In some embodiments, the construction of topological relationships between nodes includes: The structural relationship is constructed based on the hierarchical relationship between nodes, the dependency relationship between functions and data, and the temporal relationship of business processes; The semantic relationship is constructed based on the synonym mapping relationship and hierarchical classification relationship between nodes; The inference constraint relationship is constructed based on the mutual exclusion constraints between the functional jump paths and parameter states in node interactions.

[0118] It should be noted that structural relationships define the skeleton structure of the menu knowledge graph. By modeling the system hierarchy and business process relationships between menu nodes in the menu navigation system, the basic topological skeleton of the menu knowledge graph is formed. Specifically, structural relationships consist of the following three types of sub-relationships: Hierarchical relationship: describes the parent-child relationship between menu nodes. By mapping the menu tree hierarchy of the menu navigation system, a hierarchical topology extending from the root node to the leaf node is established in the menu knowledge graph, so that each menu node can trace its complete menu path through the hierarchical relationship. Functional and data dependencies: This describes the dependencies between menu nodes and business entity knowledge nodes, or between menu nodes themselves, at the functional implementation or data access level. For example, the "Benefit Activation" menu node depends on the existence of the "Purchased Benefits" business entity knowledge node, and there is a data dependency between the "Package Status Inquiry" menu node and the "Package Information" business entity knowledge node.

[0119] Business process sequence relationship: describes the sequential constraints between menu nodes in a specific business process due to the order of business steps. For example, there is a sequence dependency between the menu node corresponding to the identity verification step and the menu node corresponding to the business processing step. The former must be completed before the latter in the business process.

[0120] Semantic relations form the semantic understanding network of the menu knowledge graph. By modeling the semantic relationships between menu nodes and business entity knowledge nodes, the menu knowledge graph can support natural language semantic understanding of the menu navigation model. Specifically, semantic relations are composed of the following two types of sub-relationships: Synonym mapping describes the equivalence relationships between different expressions of the same function or business entity, including the mapping between function aliases, dialect expressions, colloquial terms, etc., and standard function names. For example, there is a synonym mapping relationship between various user colloquial expressions such as "check phone bill" and "see how much money is left" and the "bill inquiry" menu node; there is a synonym mapping relationship between "M Video Monthly Card" and "M Video Membership Monthly Card" and the "M Video VIP Monthly Card" business entity knowledge node. Through synonym mapping, the menu navigation model can accurately map diverse user natural language expressions to corresponding function nodes or business entity knowledge nodes in the menu knowledge graph; Hyper- and hypo-categorized relationships: These describe the inclusion and being-included relationships between functional concepts or business entities in terms of semantic scope. For example, the concept node "video membership benefits" includes multiple lower-level business entity knowledge nodes such as M video VIP monthly card and N video VIP monthly card. Through hyper- and hypo-categorized relationships, the menu knowledge graph can support the menu navigation model in summarizing and refining concept categories during reasoning, thereby reasonably narrowing or expanding the functional scope when the user's query semantics are ambiguous.

[0121] Inference constraints characterize the dynamic interaction patterns in the menu knowledge graph. By modeling the mutual exclusion constraints between common function jump paths and business parameters encountered by users during actual menu navigation, they provide a constraint basis for the rationality of navigation paths in the inference decisions of the menu navigation model. Specifically, inference constraints consist of the following two types of sub-relationships: Functional navigation paths describe the typical node jump sequences that users traverse when completing a specific business objective in a menu navigation system, as well as multiple alternative paths to achieve the same objective. For example, both the "My Package" path and the "Package Inquiry" path can achieve the goal of querying package status, and they are alternative paths to each other. By explicitly modeling these validated feasible navigation paths in the menu knowledge graph, a reference basis for path selection is provided for the menu navigation model. Mutual exclusion constraints between parameter states: These describe constraints that prevent certain menu function nodes or business operations from being executed simultaneously under specific parameter states or user states. For example, there is a mutual exclusion constraint between an account being suspended due to overdue payments and a rights processing operation. By modeling mutual exclusion constraints between parameter states in the menu knowledge graph, a structured constraint basis is provided for the menu navigation model to perform business logic compliance checks during inference, ensuring that the navigation decision results output by the menu navigation model conform to business constraint rules.

[0122] The menu navigation method of this invention establishes structural relationships, semantic relationships, and reasoning constraints from three dimensions: system structure, semantic understanding, and dynamic interaction. This forms a multi-dimensional topological relationship system that includes hierarchical relationships, dependency relationships, temporal relationships, synonym mapping relationships, hierarchical classification relationships, functional jump paths, and mutual exclusion constraints. This enables the constructed menu knowledge graph to fully support the menu navigation model in performing accurate structured reasoning under complex business logic constraints, thereby improving the accuracy of the menu navigation system in parsing complex intentions and the compliance of navigation decisions.

[0123] The menu navigation device provided in the embodiments of the present invention is described below. The menu navigation device described below can be referred to in correspondence with the menu navigation method described above.

[0124] The menu navigation device of this invention, such as Figure 3 As shown, it includes the following modules: The acquisition module 310 is used to acquire the current system status of the user query and menu navigation system; the current system status includes the current node; Graph retrieval module 320 is used to retrieve and extract local subgraphs in the menu knowledge graph of the menu navigation system based on the user query and the current node. The large model reasoning module 330 is used to input the user query, the current system state, and the local subgraph into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. Output module 340 is used to verify and execute the navigation decision action to update the current system state, and output menu navigation results based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

[0125] The menu navigation device of this invention uses local subgraphs in the menu knowledge graph as the reasoning knowledge context of the menu navigation model, and constrains the action space of the menu navigation model with the topological relationships of the local subgraphs. This ensures that each navigation decision strictly follows the topological structure of the menu knowledge graph, effectively suppressing the reasoning illusion of the menu navigation model. At the same time, the menu navigation model is trained through reinforcement learning using a rule reward function constructed based on system business rules and graph topological constraints. This enables the menu navigation model to generate navigation decision actions that conform to business logic under complex business constraints, improving the accuracy of menu navigation decisions while maintaining the flexibility of natural language interaction.

[0126] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a menu navigation method, which includes: Obtain the current system status of the user query and menu navigation system; the current system status includes the current node; Based on the user query and the current node, a local subgraph is retrieved and extracted from the menu knowledge graph of the menu navigation system. The user query, the current system status, and the local subgraph are input into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. Verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

[0127] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., media capable of storing program code.

[0128] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of executing the menu navigation method provided by each of the above methods, the method comprising: Obtain the current system status of the user query and menu navigation system; the current system status includes the current node; Based on the user query and the current node, a local subgraph is retrieved and extracted from the menu knowledge graph of the menu navigation system. The user query, the current system status, and the local subgraph are input into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. Verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

[0129] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the menu navigation method provided by each of the above methods, the method comprising: Obtain the current system status of the user query and menu navigation system; the current system status includes the current node; Based on the user query and the current node, a local subgraph is retrieved and extracted from the menu knowledge graph of the menu navigation system. The user query, the current system status, and the local subgraph are input into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. Verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

[0130] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in each of the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A menu navigation method, characterized in that, include: Obtain the current system status of the user query and menu navigation system; the current system status includes the current node; Based on the user query and the current node, a local subgraph is retrieved and extracted from the menu knowledge graph of the menu navigation system. The user query, the current system status, and the local subgraph are input into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. Verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

2. The menu navigation method according to claim 1, characterized in that, The step of retrieving and extracting local subgraphs from the menu knowledge graph of the menu navigation system based on the user query and the current node includes: Entity recognition and intent recognition are performed on the user query to extract the entity type and query intent; Based on the entity type and the query intent, an initial set of relevant nodes is retrieved from the menu knowledge graph of the menu navigation system. Input the initial set of relevant nodes, the user query, and the current node into the second language model to obtain the candidate root node output by the second language model; Starting from the candidate root node, a bounded breadth-first search is performed in the menu knowledge graph to extract a local subgraph.

3. The menu navigation method according to claim 1, characterized in that, The step of inputting the user query, the current system state, and the local subgraph into the menu navigation model to obtain navigation decision actions includes: Based on the local subgraph, the user query, and the current system state, contextual prompts are constructed. The contextual prompts are input into the menu navigation model for constraint reasoning to obtain the navigation decision result output by the menu navigation model; the navigation decision result includes the navigation path; the node jumps in the navigation path are restricted by the topological relationships of the local subgraph; The navigation path in the navigation decision result is analyzed to obtain the navigation decision action.

4. The menu navigation method according to claim 1, characterized in that, The process of verifying and executing the navigation decision action to update the current system state includes: The navigation decision action is verified; the verification includes at least one of permission verification, function verification, and path verification. If the verification is successful, the system call or node jump corresponding to the navigation decision action will be executed; Based on the execution result and execution action of the navigation decision action, update the historical dialogues and node states in the current system state.

5. The menu navigation method according to claim 4, characterized in that, The output menu navigation results based on the updated current system state include: Determine whether the updated current system state meets the task termination conditions; If the task termination condition is not met, then based on the updated current system state, return to the step of retrieving and extracting a local subgraph in the menu knowledge graph based on the user query and the current node, until the task termination condition is met. If the task termination condition is met, then based on the updated current system state, the menu navigation result is output; the menu navigation result includes at least the final target node, the system response message, and the reasoning path corresponding to the node sequence visited in each iteration.

6. The menu navigation method according to claim 1, characterized in that, The menu navigation model was trained in the following way: Obtain a set of thought chain samples; the set of thought chain samples includes several thought chain samples, each of the thought chain samples includes a query question description, as well as a step-by-step navigation reasoning process label and a final navigation decision answer label corresponding to the query question description; Based on the aforementioned thought chain sample set, the first large language model was subjected to supervised fine-tuning to obtain the initial menu navigation model; Obtain a sample set of navigation prompts from a menu navigation sample system; Each navigation prompt word sample in the navigation prompt word sample set includes a sample user query, a sample partial subgraph, and a sample current system status; Each navigation prompt word sample is input into the initial menu navigation model for group sampling to obtain multiple sets of candidate navigation action output sequences corresponding to each navigation prompt word sample; Based on the system business rules and graph topology constraints of the menu navigation sample system, a rule reward function is constructed, and the total reward for each group of candidate navigation action output sequences is determined based on the rule reward function. The relative advantage is determined based on the total reward of multiple candidate navigation action output sequences within the same group, and the model parameters of the initial menu navigation model are updated based on the relative advantage to obtain a trained menu navigation model.

7. The menu navigation method according to claim 6, characterized in that, Determining the total reward for each group of candidate navigation action output sequences includes: The effect reward for each group of candidate navigation action output sequences is determined based on the target intersection of the path of each group of candidate navigation action output sequences and the standard navigation action output path. Based on the node jump relationships in each group of candidate navigation action output sequences and the graph topology constraints, determine the compliance reward for each group of candidate navigation action output sequences. Based on the total number of action jumps and redundant jump paths of each group of candidate navigation action output sequences, determine the efficiency reward for each group of candidate navigation action output sequences. Based on the degree of effective integration of external entity knowledge nodes and the logical rigor of each group of candidate navigation action output sequences, the integrity reward of each group of candidate navigation action output sequences is determined. Based on the performance reward, compliance reward, efficiency reward, and integrity reward, the total reward for each group of candidate navigation action output sequences is determined.

8. The menu navigation method according to claim 1, characterized in that, The menu knowledge graph is constructed in the following way: Obtain menu nodes within the menu navigation system, as well as business entity knowledge nodes extracted from the business data of the menu navigation system; Establish topological relationships between nodes; The topological relationships include at least structural relationships, semantic relationships, and inference constraint relationships; A menu knowledge graph is constructed based on the menu nodes, the business entity knowledge nodes, and the topological relationships.

9. The menu navigation method according to claim 8, characterized in that, The construction of topological relationships between nodes includes: The structural relationship is constructed based on the hierarchical relationship between nodes, the dependency relationship between functions and data, and the temporal relationship of business processes; The semantic relationship is constructed based on the synonym mapping relationship and hierarchical classification relationship between nodes; The inference constraint relationship is constructed based on the mutual exclusion constraints between the functional jump paths and parameter states in node interactions.

10. A menu navigation device, characterized in that, include: The acquisition module is used to acquire the current system status of the user query and menu navigation system; the current system status includes the current node; The graph retrieval module is used to retrieve and extract local subgraphs from the menu knowledge graph of the menu navigation system based on the user query and the current node. The large model reasoning module is used to input the user query, the current system state, and the local subgraph into the menu navigation model to obtain navigation decision actions; the action space of the navigation decision actions is limited by the topological relationships of the local subgraph. The output module is used to verify and execute the navigation decision action to update the current system state, and output the menu navigation result based on the updated current system state; The menu navigation model is obtained by reinforcement learning training the first language model based on a rule-based reward function, which is constructed based on system business rules and graph topology constraints.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the menu navigation method as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the menu navigation method as described in any one of claims 1 to 9.

Images