Search recommendation method based on inference trajectory generation of weighted directed inference tree set

Abstract

The application provides a search recommendation method based on a weighted directed reasoning tree set generated by reasoning track generation, and relates to the technical field of search recommendation. The method comprises the following steps: collecting historical text data under a specific business scenario according to user demand and extracting semantic information, using the extracted semantic information to construct a weighted directed reasoning tree set with semantic information and containing complete reasoning tracks; using a vector space mapping and reasoning path fusion method to convert the weighted directed reasoning trees in the weighted directed reasoning tree set into vector representations; using a large model as a strategy model, using a large model-based reasoning track generation method to generate complete reasoning paths, constructing a preference data set, and using a direct preference optimization algorithm to optimize the strategy model; obtaining a search term provided by a user and inputting the search term into the optimized strategy model to obtain a final recommendation result of the search term, so as to provide more personalized and high-quality search results and recommendation contents for the user.

Description

Technical Field

[0001] This invention relates to the field of search recommendation technology, and in particular to a search recommendation method based on a weighted directed inference tree set generated from inference trajectories. Background Technology

[0002] Search recommendation technology combines the advantages of search engines and recommendation systems, aiming to provide users with personalized and high-quality search results and recommended content. This technology intelligently analyzes users' historical behavior and interests, combining this with the characteristics of the data itself to deeply explore users' personalized needs, thereby accurately matching and recommending content that meets those needs. Efficient and accurate search recommendation technology not only significantly enhances the user's search experience and alleviates information overload, but also quickly discovers content that users are interested in. Therefore, it has broad application prospects in many fields such as information retrieval, e-commerce, medical assistance, and educational systems.

[0003] Traditional search recommendation methods primarily rely on keyword matching and rule-based retrieval. They use keyword matching to retrieve relevant documents and then reason and rank them according to predetermined rules. However, this approach has certain limitations. First, existing recommendation systems do not fully consider that the information stored in recommendation applications such as product recommendations, medical assistance, and educational systems may contain logically related or sequential directed reasoning chains. Historical data or records contained in this stored information can actually provide traceable reasoning basis for subsequent recommendation searches. Second, single or multiple discrete search terms are relatively isolated and lack logical relationships, resulting in a lack of semantic coherence in the generated reasoning paths and failing to fully utilize rich contextual information. To overcome these limitations, researchers have proposed knowledge graphs and attention mechanisms. On the one hand, knowledge graphs capture the relationships between semantic entities by constructing directed graph or tree-structured knowledge bases, thus providing support for the reasoning process. However, the relationship networks in knowledge graphs are complex, requiring significant manpower for detailed modeling of domain knowledge, leading to high costs in the construction and maintenance of the knowledge base. On the other hand, attention mechanisms focus on the contextual semantics of sequential data and can capture long-distance dependencies, but they still have limitations when dealing with complex decision-making and reasoning challenges.

[0004] Overall, the methods described above have shortcomings in effectively utilizing domain knowledge and contextual semantics, as well as in efficient planning and reasoning capabilities. This not only limits the ability of existing search and recommendation algorithms to uncover inherent patterns and correlations in data, but also affects the efficiency of guiding search and recommendations by generating reasonable reasoning paths. Therefore, developing a new method that can effectively integrate semantic information with planning and reasoning capabilities is crucial for improving the accuracy and stability of search and recommendation technologies. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention proposes a search recommendation method based on a weighted directed inference tree set generated by inference trajectory, aiming to provide users with more personalized and high-quality search results and recommended content.

[0006] This invention proposes a search recommendation method based on a weighted directed inference tree set generated from inference trajectories, which includes the following steps:

[0007] Step 1: Collect historical text data in specific business scenarios according to user needs, extract semantic information from the historical text data, and use the extracted semantic information to construct a set of weighted directed reasoning trees with semantic information and containing a complete reasoning trajectory.

[0008] Step 2: The weighted directed reasoning trees in the set of weighted directed reasoning trees with semantic information and containing complete reasoning trajectories are converted into vector representations using the method of vector space mapping and reasoning path fusion.

[0009] Step 3: Use the large model as the policy model π θ A large-model-based inference trajectory generation method is adopted, and a complete inference path is generated using the vector representation of the policy model and the weighted directed inference tree; where θ is the parameter of the policy model.

[0010] Step 4: Construct a preference dataset based on the complete inference path

[0011] Step 5: Based on the preference dataset The direct preference optimization algorithm is used to optimize the policy model and obtain the optimized policy model π. * ;

[0012] Step 6: Obtain the user-provided search terms and input them into the optimized strategy model π * In the middle, the optimized strategy model π is used. * The final recommendation results for the search term are obtained;

[0013] Step 1 further includes:

[0014] Step 1.1: Collect several historical text data points based on user needs and specific business scenarios;

[0015] Step 1.2: Clean the collected historical text data, and use each cleaned historical text data as a data sample to construct a data sample set;

[0016] Step 1.3: Use the Information Extraction Model (UIE) to extract information from all data samples in the data sample set, and use a pre-trained model to convert the extracted information into text vector representations with semantic information;

[0017] Step 1.4: Construct a set of weighted directed reasoning trees with semantic information and containing complete reasoning trajectories using text vector representations with semantic information;

[0018] Step 1.3 further includes:

[0019] Step 1.3.1: For the i-th data sample in the data sample set, use the Information Extraction Model (UIE) to extract the entity, the relationship between entities, and key information for judging the importance of the entity and the relationship between entities from the data sample; where i = {1, ..., n}, and n represents the number of data samples in the data sample set;

[0020] Step 1.3.2: Treat each extracted entity as a node and construct a node set N using the extracted entities; construct a directed edge for the relationship between each extracted entity and construct a directed edge set E using the relationship between the extracted entities; use the extracted key information to assign a weight with a value range of [0,1] to each node and each directed edge and construct a weight set W.

[0021] The nodes are divided into: input nodes, intermediate nodes, and output nodes;

[0022] Step 1.3.3: Using a pre-trained model, transform the node set N, directed edge set E, and weight set W from textual representation into text vector representation containing semantic information, obtaining the transformed node set N. i The transformed set of directed edges E i and the transformed weight set W i ;

[0023] The transformed node set N i Noted as: N i ={n1,…,n j ,…,n m}, where n1 represents the text vector representation of the first node; n j The text vector representation of the j-th node; n m The text vector representation of the m-th node;

[0024] The transformed set of directed edges is denoted as: E i ={e1,…,e h ,…,e k}, where e1 represents the text vector representation of the first directed edge; e h The text vector representation of the h-th directed edge; e k The text vector representation of the k-th directed edge;

[0025] The transformed weight set Wi Recorded as: in Let represent the set of directed edge weights after the transformation, and in The text vector representation of the weight of the first directed edge; The text vector representation of the weight of the h-th directed edge; The text vector representation of the weight of the k-th directed edge; This represents the set of node weights after transformation, and in The text vector representation of the weight of the first node; The text vector representation of the weight of the j-th node; The text vector representation of the weight of the m-th node;

[0026] Step 1.4 further includes:

[0027] Step 1.4.1: Utilize the transformed node set N i The transformed set of directed edges E i and the transformed weight set W i Construct the weighted directed inference tree T for the i-th data sample i , denoted as T i =(N i E i W i );

[0028] Step 1.4.2: Construct a weighted directed inference tree for each data sample in the data sample set, and use the weighted directed inference trees of all data samples to construct a set of weighted directed inference trees with semantic information and containing a complete inference trajectory, denoted as T = {T1,…,T...} i ,…,T n}, where T1 and T n These represent the first weighted directed reasoning tree and the nth weighted directed reasoning tree in the set of weighted directed reasoning trees, respectively.

[0029] Step 2 further includes:

[0030] Step 2.1: For the i-th weighted directed reasoning tree T in the set of weighted directed reasoning trees that have semantic information and contain a complete reasoning trajectory i , for T i The set of nodes N in i Define a transition matrix M n , for T i The set of directed edges E in i Define a transition matrix M e The transition matrix M is mentioned above.n and transition matrix M e They are independent of each other;

[0031] Step 2.2: Define the projection function, and use this projection function and the transition matrix M n and transition matrix M e , set N nodes i and the set of directed edges E i Projecting them onto the same vector space yields the projected node vector representation and the directed edge vector representation;

[0032] The projection function is expressed as:

[0033]

[0034] Where n x Let N be the set of nodes i The nodes that participate in the projection; To utilize the transition matrix M n For n x The node vector representation obtained after projection; e y Let E be the set of directed edges i The directed edges that participate in the projection; To utilize the transition matrix M e For e y The directed edge vector representation obtained after projection; where and In the same vector space;

[0035] Step 2.3: Based on the projected node vector representation and directed edge vector representation, calculate the node vector representation that integrates the semantic information of the nodes and the reasoning path information for all nodes in the weighted directed reasoning tree;

[0036] The method for calculating the node vector representation of the fused node semantic information and inference path information is as follows: For an input node in a weighted directed inference tree, the node vector representation after projection of the input node is used as the node vector representation of the fused node semantic information and inference path information of the input node; for an intermediate node or output node in a weighted directed inference tree, the method for calculating the node vector representation of the fused node semantic information and inference path information of the intermediate node or output node is as follows:

[0037]

[0038] in A node vector representation that fuses semantic information and inference path information for intermediate or output nodes; for node n x The node vector representation obtained after projection express Weights of semantic information; express The parent node is represented by the node vector after projection; e px Indicates from arrive The directed edges; Represents a directed edge e px The directed edge vector representation after projection; To represent a directed edge Weights of semantic information;

[0039] Step 2.4: Convert the weighted directed inference tree T i The node vector representations of all nodes in the fusion of node semantic information and reasoning path information are weighted and summed according to their weights to obtain the vector representation of the weighted directed reasoning tree;

[0040] Step 3 further includes:

[0041] Step 3.1: Obtain several sets of search terms x and the corresponding recommendation results y for each set of search terms;

[0042] Step 3.2: Clean the acquired search terms, use a pre-trained model to convert the cleaned search terms and the corresponding recommendation results for each group of search terms into vector representations, and use the converted vector representations to construct a training set. Where x (α) This represents the vector representation of the αth group of search terms after conversion; y (α) This represents the vector representation of the recommended results corresponding to the αth group of search terms;

[0043] Step 3.3: Transfer the training set x in (α) As the starting point of the reasoning path, y (α) As the endpoint of the inference path, the training set is then... Input policy model π θ This generates a complete reasoning path;

[0044] Step 3.4: For the training set Each of the search terms x and the corresponding recommended results y in the search terms x generates a complete reasoning path;

[0045] Step 3.3 further includes:

[0046] Step 3.3.1: Generate an intermediate node on the inference path using the vector representation of the starting point of the inference path and the vector representation of the weight of the starting point of the inference path, and use it as the current intermediate node;

[0047] Step 3.3.2: Determine whether the current intermediate node is the end point of the reasoning path; if yes, generate the complete reasoning path; if no, proceed to step 3.3.3.

[0048] Step 3.3.3: Input the vector representations of all nodes, all directed edges, and all weights in the inference path from the starting point to the current intermediate node into the policy model. Use the policy model to infer the leaf nodes step by step based on the known parts in the inference path and generate an estimated inference path from the current intermediate node to the leaf node.

[0049] Step 3.3.4: The estimated inference path is converted into a vector representation by using vector space mapping and inference path fusion to obtain the vector representation P of the estimated inference path;

[0050] Step 3.3.5: Calculate the semantic structure similarity between the vector representation P of the predicted reasoning path and the vector representations of all weighted directed reasoning trees in the set of weighted directed reasoning trees, and use the obtained semantic similarity as the expected reward of the current intermediate node.

[0051] Step 3.3.6: Determine whether the number of estimated inference paths is greater than the preset maximum number of estimated inference paths. If not, return to step 3.3.3; if yes, select the next node in the estimated inference path with the highest expected reward of the current intermediate node as the child node of the current intermediate node, and then use the child node as the new current intermediate node and return to step 3.3.2.

[0052] Step 3.3.7: Construct a set of possible inference paths using all estimated inference paths other than the complete inference path.

[0053] The preference dataset mentioned in step 4 The construction method is as follows: for the training set x, as the starting point of the reasoning path (α) The generated complete reasoning path will be used as a reference to x. (α) The corresponding correct reasoning path Simultaneously from the predicted inference path set Choose at least one prediction inference path to form a relationship with x (α) Corresponding incorrect reasoning path Using x (α) , and Construct a set of preference data, denoted as For each set of search terms in the training set, a set of preference data is constructed after transformation into vector representations. A preference dataset is then built using all the obtained preference data.

[0054] The objective function of the direct preference optimization algorithm described in step 5 is:

[0055]

[0056] in The objective function of the direct preference optimization algorithm; π ref For reference strategy model; To represent the preference dataset Medium preference data (x,τ) w ,τ l The mathematical expectation of all possible values, τ w For the correct reasoning path, τ l This represents an incorrect inference path; σ is a hyperparameter used to represent the confidence interval; β is a hyperparameter in the objective function; π θ (τ w |x) represents the strategy model π given the search term x. θ Take the correct reasoning path τ w Possibilities; π ref (τ w |x) represents the reference strategy model π given the search term x. ref Take the correct reasoning path τ w Possibilities; π θ (τ l |x) represents the strategy model π given the search term x. θ Taking an incorrect reasoning path τ l Possibilities; π ref (τ l |x) represents the reference strategy model π given the search term x. ref Taking an incorrect reasoning path τ l The possibility;

[0057] Step 6 further includes:

[0058] Step 6.1: Obtain the search term q provided by the user and perform data cleaning. Then, use a pre-trained model to generate a vector representation of the cleaned search term. Finally, concatenate the obtained vector representation with the prompt in the optimized strategy model to obtain a normalized search term vector representation q′.

[0059] Step 6.2: Input the normalized search term vector representation q′ into the optimized strategy model π * In the process, starting from q′, generate u candidate inference paths that reach the output nodes of each weighted directed inference tree in the set of weighted directed inference trees, and construct a candidate inference path set O = {o1, o2, ..., o}. u}; where o1, o2, ..., ou These are the 1st, 2nd, and uth candidate reasoning paths in the candidate reasoning path set, respectively.

[0060] Step 6.3: Calculate the conditional probability scores of all candidate inference paths in the candidate inference path set O, and sort all candidate inference paths in descending order according to their conditional probability scores to obtain an ordered candidate path set. in These are the 1st, 2nd, and uth ordered candidate paths in the ordered candidate path set, respectively.

[0061] Step 6.4: From the ordered candidate path set O * The first S ordered candidate paths are selected, and the endpoints of the first S ordered candidate paths are used as the S recommended results for the search term q, and a set of recommended results is constructed. Where S is the preset maximum number of recommendations; These are the recommendation results for the 1st, 2nd, and Sth ordered candidate paths, respectively.

[0062] Step 6.5: Send the recommendation result set R to the user. The user selects one recommendation result from the recommendation result set R as the final recommendation result. At the same time, add the search term q and the final recommendation result to the training set. Used for continuous optimization of the strategy model.

[0063] The beneficial effects of adopting the above technical solution are as follows:

[0064] This invention innovatively constructs a weighted directed reasoning tree set rich in semantic information, clearly depicting the logical structure and semantic connections of the reasoning process, laying the foundation for semantic matching and planning reasoning. By utilizing a language model, the nodes, edges, and weights in the weighted directed reasoning tree are encoded as vector representations, preserving semantic features and enabling detailed and in-depth modeling of application domains.

[0065] This invention introduces a Large Language Model (LLM) as a policy model to automatically generate inference trajectories, fully leveraging the advantages of LLM in semantic understanding and planning reasoning. Based on a weighted directed reasoning tree set, LLM fully utilizes logically related or sequential directed reasoning paths in historical data or stored information, exploring the optimal inference trajectory through an efficient planning reasoning framework, significantly improving the accuracy of semantic understanding and recommendation.

[0066] The method of this invention uses the optimal inference trajectory automatically generated by LLM to construct a preference dataset. Based on this preference dataset, the Direct Preference Optimization (DPO) algorithm is used to further optimize and train the LLM policy model. DPO optimizes the policy model directly on the user's preference dataset by defining a preference loss function, without the need for reward modeling or reinforcement learning framework, thereby comprehensively improving the execution and optimization efficiency of the method. Attached Figure Description

[0067] Figure 1 This is a flowchart of a search and recommendation method based on a weighted directed reasoning tree set generated from reasoning trajectories in this embodiment;

[0068] Figure 2 This is a schematic diagram of a search and recommendation method based on a weighted directed reasoning tree set generated from reasoning trajectories, as described in this embodiment.

[0069] Figure 3 This is a schematic diagram illustrating the generation of a complete reasoning path in this implementation. Detailed Implementation

[0070] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0071] This invention aims to provide a search recommendation method based on a weighted directed inference tree set generated from inference trajectories. The core idea is to automatically generate inference trajectories using a Large Language Model (LLM). Considering that LLM training requires a large amount of data, but existing data often only contains search terms and final recommendation results, lacking information about the intermediate inference process, this invention first collects a small-scale complete dataset containing existing inference paths, i.e., 0.5% to 1% of the LLM training data. Based on the collected data, a weighted directed inference tree set containing rich semantic information is constructed. Then, leveraging the planning and inference capabilities of LLM, the LLM generates a complete inference trajectory leading to the inference scheme under self-supervision, based on the search terms and final recommendation results. That is, it generates an inference path from the search terms to the output node in the weighted directed inference tree, thereby determining the accurate search recommendation process. A preference dataset is constructed based on the generated complete inference trajectory. Then, the Direct Preference Optimization (DPO) algorithm is used to perform feedback optimization on the LLM (Limited Ledger Path) to fully integrate semantic information and planning reasoning capabilities. This further improves the accuracy of trajectory generation and final recommendation result prediction, ensuring that the generation of final recommendation results no longer relies solely on keyword matching or rule-based retrieval. Instead, the LLM autonomously discovers directed inference paths with logical correlations or connections between search terms and final recommendation results, thereby improving the accuracy and stability of the search recommendation technology. This process comprehensively enhances the recommendation quality and robustness of a search recommendation method based on a weighted directed inference tree set generated from inference trajectories, providing users with more personalized and high-quality search results and recommended content.

[0072] In the process of generating inference trajectories in a large model under self-supervised guidance, in order to ensure the correctness of the final recommendation result obtained through autonomous exploration using only known search terms, the strategy adopted is to ensure that each intermediate inference step is correct as much as possible, thereby increasing the accuracy of the recommendation result. In order to evaluate the accuracy of a certain intermediate inference step, the search terms and the intermediate steps that have been inferred to be as correct as possible are used as premises, and the final recommendation result is predicted based on the premises. The correctness of this intermediate inference step is judged by comparing the consistency between the predicted final recommendation result and the actual result. This avoids the problem that relying solely on a small amount of induced data containing complete paths cannot evaluate the correctness of intermediate results, and also avoids the high cost and potential biases brought about by manual annotation of the inference process.

[0073] This embodiment provides a search recommendation method based on a weighted directed reasoning tree set generated from reasoning trajectories, such as... Figure 1 As shown, the method includes the following steps:

[0074] Step 1: Collect historical text data in specific business scenarios according to user needs, extract semantic information from the historical text data, and use the extracted semantic information to construct a set of weighted directed reasoning trees with semantic information and containing a complete reasoning trajectory.

[0075] Step 1.1: Collect several historical text data points based on user needs and specific business scenarios;

[0076] In this embodiment, user needs may originate from specific problems or goals in actual business operations. By communicating with specific users, the specific problems or goals in actual business operations are identified, and corresponding text information is collected. This provides a foundation for the subsequent construction of weighted directed inference trees and supports the application of these inference trees in real-world scenarios. Since the method proposed in this embodiment has a wide range of applications, the specific business scenario should be determined based on user needs. Historical text data or feedback should be collected for each business scenario. These business scenarios include, but are not limited to, real-world application needs such as product recommendations, medical assistance, and teaching systems. For example, in a product recommendation scenario, users' historical browsing history, purchase records, and preference information can be collected; in a medical assistance scenario, patients' medical information, diagnostic results, and treatment plans can be collected; and in a teaching system, students' learning records, grades, and student feedback can be collected. The collected text information is used to construct a set of weighted directed inference trees with semantic information and serves as a small amount of inductive data for the LLM self-supervised generation of inference trajectories.

[0077] Step 1.2: Clean the collected historical text data, and use each cleaned historical text data as a data sample to construct a data sample set.

[0078] The data cleaning process includes: format standardization, data deduplication, supplementation of missing data, outlier handling, data accuracy verification, and removal of irrelevant data.

[0079] In this implementation, taking a medical diagnosis scenario as an example, real historical text data related to this field is collected, including unstructured data such as medical records, patient information, and follow-up visits. To ensure the quality and consistency of the raw data, the collected real historical text data needs to be cleaned, including: format standardization, deduplication, missing data handling, outlier handling, data accuracy verification, and removal of irrelevant data. First, the collected data is converted into a unified format, including: unified date format, consistent naming conventions, unified data units, consistent data encoding, and data format standardization, for subsequent processing and analysis. For example, the date format is unified to YYYY-MM-DD, named entities use consistent naming conventions, data units are unified to kilograms, millimeters of mercury, etc., a unified encoding standard such as ICD-10 encoding is adopted, and data format consistency is ensured, such as using UTF-8 encoding for text and floating-point numbers for numerical values. For example, for medical diagnostic record data, the format of visit dates can be standardized, such as converting "May 13, 2024" to "2024-05-13"; patient names can be standardized to a consistent spelling, such as unifying "John Smith" and "JohnS." into "John Smith"; disease names can be uniformly coded using ICD-10 encoding, with "hypertension" coded as "I10"; and diagnostic results should be described using consistent diagnostic criteria to avoid using ambiguous or vague diagnostic terms. Duplicate data entries should be identified and removed to avoid analytical bias caused by data duplication, and missing data should be handled using methods such as imputation and interpolation to ensure data integrity and accuracy. Simultaneously, outliers should be handled, identifying and processing values ​​outside the reasonable range or data that do not conform to reality to avoid the impact of outliers on the analysis results. Next, the accuracy of the data should be verified, ensuring data credibility through comparison with other data sources or review by professionals. Finally, irrelevant data should be removed, deleting data that is unrelated to the analysis objective to improve the quality and efficiency of the dataset. These data cleaning steps ensure data quality and consistency, providing a reliable data foundation for subsequent entity relationship extraction.

[0080] Step 1.3: Use the Universal Information Extraction (UIE) model to extract information from all data samples in the data sample set, and use a pre-trained model to convert the extracted information into text vector representations with semantic information.

[0081] Step 1.3.1: For the i-th data sample in the data sample set, use the Information Extraction Model (UIE) to extract the entity, the relationship between entities, and key information for judging the importance of the entity and the relationship between entities from the data sample; where i = {1, ..., n}, and n represents the number of data samples in the data sample set.

[0082] In this embodiment, key information refers to information that is of great significance or plays a decisive role in the reasoning process, which can help the reasoning tree more accurately reflect the actual situation and reasoning rules.

[0083] Step 1.3.2: Treat each extracted entity as a node and construct a node set N using the extracted entities; construct a directed edge for the relationship between each extracted entity and construct a directed edge set E using the relationship between the extracted entities; use the extracted key information to set a weight for each node and each directed edge with a value range of [0,1] and construct a weight set W.

[0084] The nodes are divided into: input nodes, intermediate nodes, and output nodes.

[0085] In this implementation, nodes represent reasoning steps or reasoning conditions. A reasoning step refers to each operation or judgment performed during the reasoning or decision-making process; a reasoning condition refers to the premises or assumptions that need to be met during the reasoning process. Nodes are divided into: input nodes, intermediate nodes, and output nodes. The input node refers to the premises in the reasoning process. The first entity extracted from the data sample is used as the input node; that is, during training, the input node is the first node corresponding to each data sample after data cleaning. During testing or application, the input node is the specific requirement input by the user, i.e., the search term, and this input node is used as the root node of the weighted directed reasoning tree. Intermediate nodes represent intermediate reasoning steps generated according to the reasoning rules and are used to connect the input node and the output node. The output node represents the final decision result of the reasoning.

[0086] In this implementation, directed edges are used to represent dependencies between inference steps or inference conditions. Weights are used to represent the degree of relevance between nodes or the importance of that relationship in the inference process.

[0087] Step 1.3.3: Using a pre-trained model, transform the node set N, directed edge set E, and weight set W from textual representation into text vector representation containing semantic information, obtaining the transformed node set N. i The transformed set of directed edges E i and the transformed weight set W i .

[0088] In this implementation, existing pre-trained models such as BERT and GPT are used to transform the textual representations of nodes, directed edges, and weights—from natural language descriptions easily understood by humans—into text vector representations rich in semantic information, facilitating computation and analysis within the inference tree. This ensures that nodes, directed edges, and weights not only possess tree-like structural relationships but also contain rich semantic information, enabling the subsequently constructed weighted directed inference tree set to better express the logical structure and textual semantics between nodes.

[0089] The transformed node set N i Noted as: N i ={n1,…,n j ,…,n m}, where n1 represents the text vector representation of the first node; n j The text vector representation of the j-th node; n m This represents the text vector representation of the m-th node.

[0090] In this implementation, nodes in a weighted directed inference tree play a role in driving the inference process. For example, considering a medical diagnostic system scenario, the inference steps might involve observing patient symptoms, the results of medical tests, and hypotheses about the condition, while the inference conditions might be restrictions, constraints, or requirements regarding the characteristics of the problem or task. For instance, in a product recommendation system, conditions might include information such as the user's purchase history, preferences, and location. To convert the inference steps and conditional information into semantic information within the nodes, this can be achieved using natural language descriptions, labels, or other symbols. Each inference tree clearly illustrates the structure and logical relationships between all nodes within the tree.

[0091] The transformed set of directed edges is denoted as: E i ={e1,…,e h ,…,e k}, where e1 represents the text vector representation of the first directed edge; e h The text vector representation of the h-th directed edge; e k Let represent the text vector representation of the k-th directed edge.

[0092] In this embodiment, a pre-trained model is used to convert the dependencies between inference steps or conditions from a textual representation of natural language descriptions into a text vector representation of directed edges. The dependencies between inference steps or conditions are similar to the relational form in entity relation triples. For example, the relationship between the entities "Patient No. 1" and "Ward 201" should be a "stay" relationship, and this relationship has a clear directionality from "Patient No. 1" to "Ward 201", and the direction is immutable.

[0093] The transformed weight set W i Recorded as: in Let represent the set of directed edge weights after the transformation, and in The text vector representation of the weight of the first directed edge; The text vector representation of the weight of the h-th directed edge; The text vector representation of the weight of the k-th directed edge; This represents the set of node weights after transformation, and in The text vector representation of the weight of the first node; The text vector representation of the weight of the j-th node; The text vector representation of the weight of the m-th node.

[0094] Step 1.4: Construct a set of weighted directed reasoning trees with semantic information and containing complete reasoning trajectories using text vector representations with semantic information.

[0095] Step 1.4.1: Utilize the transformed node set N i The transformed set of directed edges E i and the transformed weight set W i Construct the weighted directed inference tree T for the i-th data sample i , denoted as T i =(N i E i W i ).

[0096] Step 1.4.2: Construct a weighted directed inference tree for each data sample in the data sample set, and use the weighted directed inference trees of all data samples to construct a set of weighted directed inference trees with semantic information and containing a complete inference trajectory, denoted as T = {T1,…,T...} i ,…,T n}, where T1 and T n Let represent the first weighted directed inference tree and the nth weighted directed inference tree in the set of weighted directed inference trees, respectively.

[0097] In this embodiment, the process corresponding to step 1 above is as follows: taking the historical text data collected according to the corresponding business as a sample, cleaning each piece of historical text data in the sample and inputting it into a general information extraction model, capturing the semantic dependencies within the data sample through an attention mechanism, and identifying information such as entities and relationships related to reasoning from it, and then using a pre-trained model to convert nodes, directed edges and weights into text vector representations with semantic information, thereby constructing a weighted directed reasoning tree. The above operation is repeated until the weighted directed reasoning trees of all historical text data in the entire sample are obtained, and finally a set of weighted directed reasoning trees with semantic information and containing a complete reasoning trajectory is obtained.

[0098] Step 2: The weighted directed inference trees in the set of weighted directed inference trees with semantic information and containing complete inference trajectories are converted into vector representations by using vector space mapping and inference path fusion.

[0099] Step 2.1: For the i-th weighted directed reasoning tree T in the set of weighted directed reasoning trees that have semantic information and contain a complete reasoning trajectory i , for T i The set of nodes N in i Define a transition matrix M n , for T i The set of directed edges E in i Define a transition matrix M e The transition matrix M is mentioned above. n and transition matrix M e They are independent of each other;

[0100] In this embodiment, since the node set N of the weighted directed reasoning tree i and the set of directed edges E i Since nodes and edges reside in their respective vector spaces, they have significant semantic differences. Therefore, two independent transition matrices are needed to map nodes and edges into the same vector space. This allows for the full capture of the reasoning and logical relationships between nodes and edges while preserving their textual semantic information. Furthermore, all nodes within the same weighted directed reasoning tree share a single transition matrix M. n All directed edges also share a single transition matrix M. e This not only reduces the number of parameters, improves the model's generalization ability, and lowers the risk of overfitting, but also simplifies the model structure, making it easier to understand and interpret. Additionally, the transition matrix M... n and M eThe initial values ​​are defined by random initialization, and their dimensions are all d×d, where d is the dimension of the vector representation of the node and the vector representation of the edge; the above transition matrix is ​​one of the parameters that the model needs to learn. During the training process, it is continuously optimized so that the model can better capture the semantic relationships in the knowledge graph or tree structure.

[0101] Step 2.2: Define the projection function, and use this projection function and the transition matrix M n and transition matrix M e , set N nodes i and the set of directed edges E i Projecting them onto the same vector space yields the projected node vector representation and the directed edge vector representation;

[0102] The projection function is expressed as:

[0103]

[0104] Where n x Let N be the set of nodes i The nodes that participate in the projection; To utilize the transition matrix M n For n x The node vector representation obtained after projection; e y Let E be the set of directed edges i The directed edges that participate in the projection; To utilize the transition matrix M e For e y The directed edge vector representation obtained after projection; where and It is a projection in the same vector space.

[0105] Step 2.3: Based on the projected node vector representation and the directed edge vector representation, calculate the node vector representation that integrates the semantic information of the nodes and the reasoning path information for all nodes in the weighted directed reasoning tree.

[0106] The method for calculating the node vector representation of the fused node semantic information and inference path information is as follows: For an input node in a weighted directed inference tree, the node vector representation after projection of the input node is used as the node vector representation of the fused node semantic information and inference path information of the input node; for an intermediate node or output node in a weighted directed inference tree, the method for calculating the node vector representation of the fused node semantic information and inference path information of the intermediate node or output node is as follows:

[0107]

[0108] in A node vector representation that fuses semantic information and inference path information for intermediate or output nodes; for node n x The node vector representation obtained after projection express Weights of semantic information; express The parent node is represented by the node vector after projection; e px Indicates from arrive The directed edges; Represents a directed edge e px The directed edge vector representation after projection; To represent a directed edge Weights of semantic information.

[0109] In this implementation, although nodes and directed edges are projected onto the same vector space, they remain in a discrete, isolated state. To represent the reasoning relationship between nodes and directed edges, each node is represented as a new node vector representation containing the reasoning path information in its corresponding reasoning tree, i.e., a node vector representation that integrates node semantic information and reasoning path information. For each node in the weighted directed reasoning tree, a new node vector representation containing information about all nodes and directed edges on the path leading to that node is calculated on the reasoning path of that node. Specifically, since input nodes have no parent nodes and edges pointing to them, the new node vector representation of the input node is the same as the vector representation obtained after mapping; for non-input nodes, their new node vector representation is a vector representation obtained by reasoning from the node vector representation of the node's parent node after projection and the directed edges pointing to that node, and the resulting vector representation is weighted and summed with the node's own vector representation containing semantic information to obtain a vector representation that integrates the reasoning path. In fact, the vector representation that integrates the reasoning path will contain information about all nodes and edges in the reasoning path before that node. Specifically, for nodes... and its parent node Introducing weights to balance nodes that only carry semantic information Parent node With directed edges The contribution is to incorporate the semantic information of nodes and directed edges into the calculation in the form of weights.

[0110] Step 2.4: Convert the weighted directed inference tree T i The node vector representations of all nodes in the fusion node semantic information and reasoning path information are weighted and summed according to their weights to obtain the vector representation of the weighted directed reasoning tree.

[0111] The vector representation of the weighted directed reasoning tree is expressed as follows:

[0112]

[0113] in Let be the vector representation of the i-th weighted directed inference tree.

[0114] In this implementation, the contributions of semantic information and reasoning path information are balanced by weights in order to capture reasoning relationships and generate a more comprehensive representation.

[0115] Step 3: Use the Large Language Model (LLM) as the policy model π θ The method adopts a large model-based inference trajectory generation method and uses the vector representation of the policy model and weighted directed inference tree to generate a complete inference path; where θ is the parameter of the policy model.

[0116] In this implementation, based on the vector representation of a weighted directed inference tree, an LLM-based inference trajectory generation method is designed. This allows the LLM to generate a complete inference path—a complete inference trajectory from the search term to the final inference result—under self-supervision. This determines the accurate search recommendation process and facilitates the subsequent construction of a preference dataset. Since steps 1-2 process historical text data collected in specific business scenarios, specifically, during the search recommendation process, the user input consists of one or more search terms, and the output only contains the final recommendation result, lacking the intermediate inference path. Furthermore, manual annotation is costly, and using the LLM itself for annotation is ineffective and cannot meet the data volume requirements for LLM training. Therefore, it is necessary to augment the data containing only search terms and recommendation results and construct a preference dataset for optimizing the LLM.

[0117] Step 3.1: Obtain several sets of search terms x and the recommended results y corresponding to each set of search terms.

[0118] In this embodiment, user-inputted search terms and the recommendation results generated by the existing search recommendation system for each set of search terms are collected. A set of search terms input by the user may be a single word or a paragraph containing multiple words.

[0119] Step 3.2: Clean the acquired search terms, use a pre-trained model to convert the cleaned search terms and the corresponding recommendation results for each group of search terms into vector representations, and use the converted vector representations to construct a training set. Where x (α) This represents the vector representation of the αth group of search terms after conversion; y (α) This represents the vector representation of the recommended results corresponding to the αth group of search terms.

[0120] In this implementation, data cleaning is performed on one or more collected search terms, including format standardization, deduplication, missing data handling, outlier handling, data accuracy verification, and removal of irrelevant data, to ensure the accuracy and conciseness of the input text. Additionally, the search terms and final recommendation results need to be converted into vector representations using a pre-trained model; this conversion process is the same as in step 1.3. A training set is then constructed using the converted vector representations. x (α) This represents the vector representation of the α-th group of search terms. Specifically, if this group contains v search terms, the LLM prompt is concatenated with multiple search terms, denoted as prompt + search term 1 | search term 2 | search term 3 ... search term v. The concatenated search terms are then converted into vector representations. The prompt guides the LLM to output the answer; generally, different LLMs use different prompts, or they can be manually specified, as long as it helps the model output better results. It is usually consistent with the prompt used during LLM training. (α) This represents the vector representation of the recommended results corresponding to the α-th search term, which is the input x into the LLM. (α) The final recommendation result, i.e., the label, is then used as the endpoint of the reasoning path.

[0121] Step 3.3: Transfer the training set x in (α) As the starting point of the reasoning path, y (α) As the endpoint of the inference path, the training set is then... Input policy model π θ This generates a complete reasoning path.

[0122] In this embodiment, such as Figure 2 As shown, in order to generate complete and effective inference trajectories to establish accurate inference associations, LLM is used as the policy model π. θ For example, models like LLaMA and ChatGLM are used to explore reasoning paths that start from one or more search terms in the input and reach the final recommendation result in a weighted directed inference tree. Specifically, it allows LLM to start from x... (α)The process begins by establishing a starting point for a complete reasoning path. Then, it explores and gradually generates intermediate nodes along this path, aiming to find a path from the starting point to the endpoint. During the process of reasoning to the endpoint, several intermediate nodes are encountered. However, it's impossible to determine whether a generated intermediate node represents a correct reasoning process, and errors in intermediate nodes greatly increase the likelihood of an incorrect final output node. This is a planning problem. To achieve this goal, we need to explore around each current intermediate node, allowing the LLM (Limited Learning Model) to perform predictive reasoning using information from previous intermediate nodes, generating a reasoning path from the current intermediate node to the predicted endpoint. This process is repeated multiple times, and the solution with the highest expected reward is selected as the next intermediate node, thus choosing a suitable reasoning path to reach the final recommended result.

[0123] Step 3.3.1: Generate an intermediate node on the inference path using the vector representation of the starting point of the inference path and the vector representation of the weight of the starting point of the inference path, and use it as the current intermediate node.

[0124] Step 3.3.2: Determine whether the current intermediate node is the end point of the reasoning path; if yes, generate a complete reasoning path; if no, proceed to step 3.3.3.

[0125] Step 3.3.3: Input the vector representations of all nodes, all directed edges, and all weights in the inference path from the starting point to the current intermediate node into the policy model. Use the policy model to infer the leaf nodes step by step based on the known parts in the inference path and generate an estimated inference path from the current intermediate node to the leaf node.

[0126] In search recommendation tasks, a complete and effective inference path is crucial for establishing accurate inference associations during the search recommendation process. Since the start and end points of the inference path are given, the goal is to construct a complete inference path by generating intermediate nodes that best match the actual recommendation results. Specifically, for a given intermediate node to be generated in the current inference path, assuming its predecessor is the current intermediate node, since the current intermediate node is already determined in the inference path, the vector representation of the current node, all its predecessor nodes, directed edges, and weights is used as input to the LLM (Liquid Probe). An advance prediction from the current node to the leaf node is performed, finally generating the predicted inference path from the current node to the ultimately predicted leaf node. This is a process of generating nodes on the predicted inference path using stepwise reasoning. Here, the leaf node is the search recommendation result inferred by the LLM based on the currently known input conditions, but the leaf node is not necessarily the known endpoint y of the inference path. (α) Consistent.

[0127] Step 3.3.4: The estimated inference path is converted into a vector representation by using the method of vector space mapping and inference path fusion, and the vector representation P of the estimated inference path is obtained.

[0128] In this embodiment, the same vector representation transformation process as in step 2 is performed on the obtained estimated inference path to obtain the vector representation P of the estimated inference path. The nodes, directed edges and weights in the estimated inference path are transformed into a branch in the vector representation of the weighted directed inference tree, because an inference path only involves one branch, and the vector representation of this branch is used to capture the semantic structure information of the path.

[0129] Step 3.3.5: Calculate the semantic structure similarity between the vector representation P of the predicted inference path and the vector representations of all weighted directed inference trees in the set of weighted directed inference trees, and use the obtained semantic similarity as the expected reward of the current intermediate node.

[0130] The method for calculating the semantic structure similarity is as follows:

[0131]

[0132] in The vector representation P of the predicted inference path and the vector representation of the i-th weighted directed inference tree. The semantic structural similarity, where The value range of is [-1, 1], and The closer the value of P is to 1, the better it represents P and The more similar their semantic structures, The closer the value of P is to -1, the better it indicates that P and The less similar their semantic structures are; The vector representation P of the predicted inference path and the vector representation of the i-th weighted directed inference tree. The dot product; ||P|| represents the vector representation of the predicted inference path, which is the magnitude of P; The vector representation of the i-th weighted directed inference tree The model.

[0133] In this embodiment, such as Figure 3 As shown, semantic structural similarity is calculated between the vector representation of the estimated inference path and the vector representations of all weighted directed inference trees in the set of weighted directed inference trees. This yields the semantic structural similarity between the vector representation of the estimated inference path and each weighted directed inference tree. The semantic similarity is then used as the expected reward for intermediate nodes to evaluate the semantic coherence and correctness from the current node to the leaf node, i.e., the value of the intermediate node in the entire inference path. The semantic similarity can be calculated using cosine similarity to measure the degree of similarity between the vector representation of the estimated inference path and the vector representations of all weighted directed inference trees in the set.

[0134] Step 3.3.6: Determine whether the number of estimated inference paths is greater than the preset maximum number of estimated inference paths. If not, return to step 3.3.3; if yes, select the next node in the estimated inference path with the highest expected reward of the current intermediate node as the child node of the current intermediate node, and then use the child node as the new current intermediate node and return to step 3.3.2.

[0135] In this embodiment, the maximum number of estimated inference paths is set to 20. The selection of intermediate nodes can be regarded as a heuristic search process. The goal is to select the optimal intermediate node, which is achieved by executing multiple sets of simulated inference. Each simulated inference will select the next possible intermediate node of the current intermediate node and calculate the expected reward of the current intermediate node. The expected reward of each current intermediate node is used as the evaluation index for selection, and the next node in the estimated inference path with the highest expected reward is selected as the next intermediate node of the current intermediate node.

[0136] Step 3.3.7: Construct a set of possible inference paths using all estimated inference paths other than the complete inference path.

[0137] Step 3.4: For the training set Each of the search terms x and the corresponding recommended results y generates a complete reasoning path.

[0138] Step 4: Construct a preference dataset based on the complete inference path

[0139] The preference dataset The construction method is as follows: for the training set x, as the starting point of the reasoning path (α) The generated complete reasoning path will be used as a reference to x. (α) The corresponding correct reasoning path Simultaneously from the predicted inference path set Choose at least one prediction inference path to form a relationship with x (α) Corresponding incorrect reasoning path Using x (α) , and Construct a set of preference data, denoted as For each set of search terms in the training set, a set of preference data is constructed after transformation into vector representations. A preference dataset is then built using all the obtained preference data.

[0140] In this embodiment, a preference dataset is constructed based on the complete inference path generated in step 3. A set of preference data is represented as in and All are in the predicted inference path set In the middle; and in These represent the estimated inference paths for path 0, path 1, path 2, and path z, respectively.

[0141] Step 5: Based on the preference dataset The direct preference optimization algorithm is used to optimize the policy model and obtain the optimized policy model π. * ;

[0142] The objective function of the direct preference optimization algorithm is:

[0143]

[0144] in The objective function of the direct preference optimization algorithm; π ref For reference strategy model; To represent the preference dataset Medium preference data (x,τ) w ,τ l The mathematical expectation of all possible values, τ w For the correct reasoning path, τ l This represents an incorrect inference path; σ is a hyperparameter used to represent the confidence interval; β is a hyperparameter in the objective function; π θ (τ w |x) represents the strategy model π given the search term x. θ Take the correct reasoning path τ w Possibilities; π ref (τ w |x) represents the reference strategy model π given the search term x. ref Take the correct reasoning path τ w Possibilities; π θ (τ l |x) represents the strategy model π given the search term x. θ Taking an incorrect reasoning path τ l Possibilities; π ref (τ l |x) represents the reference strategy model π given the search term x. ref Taking an incorrect reasoning path τ l The possibility.

[0145] In this embodiment, in order to obtain an optimized strategy model π that can provide search recommendation results with better logic and more in line with human preferences...* Essentially, direct preference optimization (DPO) solves a classification problem based on preference data. Therefore, it doesn't rely on explicit reward modeling or reinforcement learning, making it a simpler and more direct training method. Traditional methods use preference models to train reward models, and then use those reward models to train policy models. Unlike traditional methods, DPO directly defines the preference loss based on the policy, i.e., its objective function. It uses a simple binary cross-entropy objective to optimize the policy model. When the objective function is minimized, the resulting policy model is better and can output more satisfactory recommendation results.

[0146] In reinforcement learning, policy models are typically learned by maximizing expected rewards, while the reference policy model may be a simple, fixed model. By comparing the target policy model and the reference policy model, the quality of the target policy model can be evaluated, guiding its training direction. During policy model optimization, the model is improved by updating parameters θ. The policy model to be optimized is used as the reference policy model, analogous to a physical reference point—a benchmark during model training. This means the reference policy model is not updated during optimization and is considered a fixed benchmark. The direct preference optimization algorithm's objective function... Typically used in reinforcement learning, this optimization objective function compares the policy model π. θ and reference strategy model π ref In the preference dataset The performance of the policy model is evaluated by its π; the hyperparameter β is used to balance the policy model π. θ and reference strategy model π ref The importance of β in the optimization objective function affects the difference between the target policy model and the reference policy model. Typically, the choice of β needs to be adjusted experimentally. As shown in formula (6), this optimization objective function increases the importance of the correct inference path τ in the preference data. w This reduces the likelihood of unfavorable data, i.e., incorrect inference paths τ. l The possibility of this; and using this optimization objective function to train the initial policy model π. θ .

[0147] Step 6: Obtain the user-provided search terms and input them into the optimized strategy model π * In the middle, the optimized strategy model π is used. * The final recommendation results for this search term are obtained.

[0148] In this embodiment, the optimized strategy model is applied to a real search recommendation system. The strategy model generates a complete inference path based on one or more search terms input by the user to achieve a more complete inference process and obtain search recommendation results that conform to human preferences and are more efficient. The aim is to use the optimized strategy model to build a complete search recommendation system model, generate a complete inference path for the input search terms, and achieve better search recommendations.

[0149] Step 6.1: Obtain the search term q provided by the user and perform data cleaning. Then, use a pre-trained model to generate a vector representation of the cleaned search term. Finally, concatenate the obtained vector representation with the prompt in the optimized strategy model to obtain a normalized search term vector representation q′.

[0150] In this embodiment, the method for data cleaning and vector representation operations of the pre-trained model for the search term q is the same as in step 1.3. The LLM prompt and the vector representation of the search term are concatenated to obtain the normalized vector representation q′ of the search term. q′ is mainly used to remind the model to generate inference paths step by step.

[0151] Step 6.2: Input the normalized search term vector representation q′ into the optimized strategy model π * In the process, starting from q′, generate u candidate inference paths that reach the output nodes of each weighted directed inference tree in the set of weighted directed inference trees, and construct a candidate inference path set O = {o1, o2, ..., o}. u}; where o1, o2, ..., o u These are the 1st, 2nd, and uth candidate reasoning paths in the candidate reasoning path set, respectively.

[0152] Step 6.3: Calculate the conditional probability scores of all candidate inference paths in the candidate inference path set O, and sort all candidate inference paths in descending order according to their conditional probability scores to obtain an ordered candidate path set. in These are the 1st, 2nd, and uth ordered candidate paths in the ordered candidate path set, respectively.

[0153] In this embodiment, all candidate inference paths in the candidate inference path set O are calculated in the optimized policy model π. * The conditional probability score under the normalized retrieval word vector representation q′ is used to reflect the quality of the candidate inference path.

[0154] Step 6.4: From the ordered candidate path set O * The first S ordered candidate paths are selected, and the endpoints of the first S ordered candidate paths are used as the S recommended results for the search term q, and a set of recommended results is constructed. Where S is the preset maximum number of recommendations; These are the recommendation results for the 1st, 2nd, and Sth ordered candidate paths, respectively.

[0155] In this embodiment, the S recommended results constitute the final search recommendation scheme for the search term q, which is used to directly recommend it to the user.

[0156] Step 6.5: Send the recommendation result set R to the user. The user selects one recommendation result from the recommendation result set R as the final recommendation result. At the same time, add the search term q and the final recommendation result to the training set. Used for continuous optimization of the strategy model.

[0157] In this implementation, the user selects a recommendation from the recommendation result set R according to their personal needs as the final recommendation result. The user-provided search terms and the user-selected final recommendation result are then used as new training set data to train the policy model, thereby updating the preference dataset. Continuously optimize the strategy model to form a closed-loop optimization process. This optimization process is consistent with steps 3-5, in order to continuously improve the performance of the strategy model and the accuracy of recommendations during use.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A search and recommendation method based on a weighted directed inference tree set generated from inference trajectories, characterized in that, The method includes the following steps: Step 1: Collect historical text data in specific business scenarios according to user needs, extract semantic information from the historical text data, and use the extracted semantic information to construct a set of weighted directed reasoning trees with semantic information and containing a complete reasoning trajectory. Step 2: The weighted directed reasoning trees in the set of weighted directed reasoning trees with semantic information and containing complete reasoning trajectories are converted into vector representations using the method of vector space mapping and reasoning path fusion. Step 2.1: For the set of weighted directed reasoning trees that have semantic information and contain a complete reasoning trajectory, the first... A weighted directed inference tree ,for The set of nodes in Define a transition matrix ,for The set of directed edges in Define a transition matrix The transition matrix mentioned above and transition matrix They are independent of each other; Step 2.2: Define the projection function and use this projection function and the transition matrix. and transition matrix , set of nodes and the set of directed edges Projecting them onto the same vector space yields the projected node vector representation and the directed edge vector representation; The projection function is expressed as: ； ； in For a set of nodes The nodes that participate in the projection; To utilize the transition matrix right The node vector representation obtained after projection; Set of directed edges The directed edges that participate in the projection; To utilize the transition matrix right The directed edge vector representation obtained after projection; where and In the same vector space; Step 2.3: Based on the projected node vector representation and directed edge vector representation, calculate the node vector representation that integrates the semantic information of the nodes and the reasoning path information for all nodes in the weighted directed reasoning tree; The method for calculating the node vector representation of the fused node semantic information and reasoning path information is as follows: For an input node in a weighted directed reasoning tree, the node vector representation after projection of the input node is used as the node vector representation of the fused node semantic information and reasoning path information of the input node. For intermediate or output nodes in a weighted directed reasoning tree, the method for calculating the node vector representation that fuses the semantic information and reasoning path information of the intermediate or output node is as follows: ； in A node vector representation that fuses semantic information and inference path information for intermediate or output nodes; for a node The node vector representation obtained after projection , express Weights of semantic information; express The parent node is represented by the node vector after projection; Indicates from arrive The directed edges; Represents a directed edge The projected directed edge vector representation; To represent a directed edge Weights of semantic information; Step 2.4: Convert the weighted directed reasoning tree The node vector representations of all nodes in the fusion of node semantic information and reasoning path information are weighted and summed according to their weights to obtain the vector representation of the weighted directed reasoning tree; Step 3: Use the large model as the policy model A large-model-based inference trajectory generation method is adopted, and a complete inference path is generated using a policy model and a vector representation of a weighted directed inference tree; whereby... These are the parameters of the strategy model; Step 3.1: Obtain several sets of search terms and the recommended results for each set of search terms. ; Step 3.2: Clean the acquired search terms, use a pre-trained model to convert the cleaned search terms and the corresponding recommendation results for each group of search terms into vector representations, and use the converted vector representations to construct a training set. ,in Indicates the obtained first Vector representation of group search terms after conversion; Indicates the obtained first The vector representation of the recommendation results corresponding to the group search terms; Step 3.3: Transfer the training set In As the starting point of the reasoning path, As the endpoint of the inference path, the training set is then... Input policy model This generates a complete reasoning path; Step 3.4: Prepare the training set All search terms and the search term Corresponding recommendation results Each generates a complete reasoning path; Step 4: Construct a preference dataset based on the complete inference path ; Step 5: Based on the preference dataset The direct preference optimization algorithm is used to optimize the policy model and obtain the optimized policy model. ; Step 6: Obtain the user-provided search terms and input them into the optimized strategy model. In the middle, the optimized strategy model is used. The final recommendation results for this search term are obtained.

2. The search and recommendation method based on a weighted directed inference tree set generated from inference trajectories according to claim 1, characterized in that, Step 1 further includes: Step 1.1: Collect several historical text data entries based on user needs and specific business scenarios; Step 1.2: Clean the collected historical text data, and use each cleaned historical text data as a data sample to construct a data sample set; Step 1.3: Use the Information Extraction Model (UIE) to extract information from all data samples in the data sample set, and use a pre-trained model to convert the extracted information into text vector representations with semantic information; Step 1.4: Construct a set of weighted directed reasoning trees with semantic information and containing complete reasoning trajectories using text vector representations with semantic information.

3. The search and recommendation method based on a weighted directed inference tree set generated from inference trajectories according to claim 2, characterized in that, Step 1.3 further includes: Step 1.3.1: For the first data sample set... From a data sample, the Information Extraction Model (UIE) is used to extract entities, relationships between entities, and key information for determining the importance of entities and the importance of relationships between entities; among which... , This indicates the number of data samples in the data sample set; Step 1.3.2: Treat each extracted entity as a node, and construct a node set using the extracted entities. Draw a directed edge between each extracted entity and the relationships between them, and construct a set of directed edges using the relationships between the extracted entities. Using the extracted key information, a value range is set for each node and each directed edge. Weights and construct a weight set ; The nodes are divided into: input nodes, intermediate nodes, and output nodes; Step 1.3.3: Use a pre-trained model to set up the node set. Set of directed edges and weight set The text representation is converted into a text vector representation containing semantic information, resulting in a converted set of nodes. The transformed set of directed edges and the transformed weight set ; The transformed node set Recorded as: ,in The text vector representation of the first node; Indicates the first The text vector representation of each node; Indicates the first The text vector representation of each node; The transformed set of directed edges is denoted as: ,in The text vector representation of the first directed edge; Indicates the first A text vector representation of a directed edge; Indicates the first A text vector representation of a directed edge; The transformed weight set Recorded as: ;in Let represent the set of directed edge weights after the transformation, and ,in The text vector representation of the weight of the first directed edge; Indicates the first A text vector representation of the weights of directed edges; Indicates the first A text vector representation of the weights of directed edges; This represents the set of node weights after transformation, and ,in The text vector representation of the weight of the first node; Indicates the first Text vector representation of the weights of each node; Indicates the first The text vector representation of the node weights.

4. The search and recommendation method based on a weighted directed inference tree set generated from inference trajectories according to claim 3, characterized in that, Step 1.4 further includes: Step 1.4.1: Utilize the transformed node set The transformed set of directed edges and the transformed weight set Construct the first Weighted directed inference tree of data samples , recorded as ; Step 1.4.2: Construct a weighted directed inference tree for each data sample in the data sample set, and use the weighted directed inference trees of all data samples to construct a set of weighted directed inference trees with semantic information and containing a complete inference trajectory, denoted as . ,in and Represent the first and second weighted directed reasoning trees in the set of weighted directed reasoning trees, respectively. A weighted directed reasoning tree.

5. The search and recommendation method based on a weighted directed inference tree set generated from inference trajectories according to claim 4, characterized in that, Step 3.3 further includes: Step 3.3.1: Generate an intermediate node on the inference path using the vector representation of the starting point of the inference path and the vector representation of the weight of the starting point of the inference path, and use it as the current intermediate node; Step 3.3.2: Determine whether the current intermediate node is the end point of the reasoning path; if yes, generate the complete reasoning path; if no, proceed to step 3.3.

3. Step 3.3.3: Input the vector representations of all nodes, all directed edges, and all weights in the inference path from the starting point to the current intermediate node into the policy model. Use the policy model to infer the leaf nodes step by step based on the known parts in the inference path and generate an estimated inference path from the current intermediate node to the leaf node. Step 3.3.4: The estimated inference path is converted into a vector representation using a vector space mapping and inference path fusion method, thus obtaining the vector representation of the estimated inference path. ; Step 3.3.5: Represent the predicted inference path as a vector. Calculate the semantic structure similarity with the vector representations of all weighted directed reasoning trees in the set of weighted directed reasoning trees, and use the obtained semantic similarity as the expected reward of the current intermediate node; Step 3.3.6: Determine whether the number of estimated inference paths is greater than the preset maximum number of estimated inference paths. If not, return to step 3.3.3; if yes, select the next node in the estimated inference path with the highest expected reward of the current intermediate node as the child node of the current intermediate node, and then use the child node as the new current intermediate node and return to step 3.3.

2. Step 3.3.7: Construct a set of possible inference paths using all estimated inference paths other than the complete inference path. .

6. The search and recommendation method based on a weighted directed inference tree set generated from inference trajectories according to claim 5, characterized in that, The preference dataset mentioned in step 4 The construction method is as follows: for the training set The middle as the starting point of the reasoning path The generated complete reasoning path will be used as a reference for... The corresponding correct reasoning path At the same time, from the set of predicted inference paths Select at least one prediction inference path to form a relation with Corresponding incorrect reasoning path ,use , and Construct a set of preference data, denoted as For each set of search terms in the training set, a set of preference data is constructed after transformation into vector representations. Then, a preference dataset is built using all the obtained preference data. .

7. The search and recommendation method based on a weighted directed inference tree set generated from inference trajectories according to claim 6, characterized in that, The objective function of the direct preference optimization algorithm described in step 5 is: ； in The objective function of the direct preference optimization algorithm; For reference strategy model; To represent the preference dataset Medium preference data The mathematical expectation of all possible values. For the correct reasoning path, This is an incorrect reasoning path; It is a hyperparameter used to represent the confidence interval; To optimize the hyperparameters in the objective function; Indicates that in the given search terms In this case, the strategy model Take the correct reasoning path The possibility; Indicates that in the given search terms In this case, refer to the strategy model Take the correct reasoning path The possibility; Indicates that in the given search terms In this case, the strategy model Taking an incorrect reasoning path The possibility; Indicates that in the given search terms In this case, refer to the strategy model Taking an incorrect reasoning path The possibility.

8. The search and recommendation method based on a weighted directed inference tree set generated according to claim 7, characterized in that, Step 6 further includes: Step 6.1: Obtain the search terms provided by the user The data is cleaned, and a pre-trained model is used to generate vector representations of the cleaned search terms. These vector representations are then concatenated with the prompts from the optimized strategy model to obtain standardized search term vector representations. ; Step 6.2: Represent the normalized search term vectors Input optimized policy model From Start, Generate Find candidate inference paths that reach the output nodes of each weighted directed inference tree in the set of weighted directed inference trees, and construct a set of candidate inference paths. ;in These are the first, second, and third paths in the candidate reasoning path set, respectively. Candidate reasoning paths; Step 6.3: Calculate the set of candidate inference paths The conditional probability scores of all candidate inference paths are calculated, and all candidate inference paths are sorted in descending order according to their conditional probability scores to obtain an ordered set of candidate paths. ;in These are the 1st, 2nd, and 3rd paths in the ordered candidate path set, respectively. Ordered candidate paths; Step 6.4: From the ordered candidate path set Before being selected Ordered candidate paths, will be the first The endpoint of the ordered candidate path is used as the search term. of 1 recommendation result, and construct a recommendation result set. ,in This is the preset maximum number of recommendations; Articles 1, 2, and 3 are respectively. Recommendation results corresponding to the ordered candidate paths; Step 6.5: Set the recommendation results Send to the user, who then selects the recommended results from the set. Select one recommendation result as the final recommendation result, and simultaneously select the search terms. Add the final recommendation results to the training set. It is used to continuously optimize the strategy model.

Images