Knowledge graph and large language model-based interpretable patent recommendation method and device

Abstract

The invention discloses an interpretable patent recommendation method and device based on a knowledge graph and a large language model, and the method comprises the following steps: obtaining patent text information and patent transfer information of a target enterprise in a preset period, and constructing a corresponding patent knowledge graph; a CPC co-occurrence matrix is constructed, and the patent research direction of the target enterprise is obtained; optimizing the patent knowledge graph based on a diffusion model, and performing knowledge reasoning on the optimized patent knowledge graph based on a self-supervised learning model to obtain a patent recommendation path of the target enterprise; and fusing the patent recommendation path and the patent research direction by using a large language model to generate a recommendation text corresponding to the target enterprise. The patent recommendation system effectively solves the problems that an existing patent recommendation system is insufficient in transparency and poor in technology trend prediction capacity, through knowledge reasoning and semantic generation, the accuracy of recommendation results is improved, and the credibility of enterprise technology strategy decisions is remarkably enhanced.

Description

Technical Field

[0001] The present invention relates to the field of patent recommendation technology, and more specifically to an explainable patent recommendation method and device based on knowledge graphs and language models. Background Art

[0002] In today's rapidly evolving global economy, patents, as the core vehicle for technological innovation, have become a crucial strategic resource for companies to maintain their market competitiveness. Companies acquire core technologies through patent transfers, driving product innovation and market expansion. However, with the exponential growth of global patent data, companies face significant challenges in accurately identifying high-value patents that align with their strategic objectives within these vast patent databases.

[0003] Although patent recommendation systems have great potential in assisting decision making, existing research and practice still have the following shortcomings:

[0004] (1) Existing patent recommendation systems have limitations in understanding and predicting technology trends. Specifically, these systems often fail to fully consider the specific needs and strategic goals of enterprises, resulting in recommendation results that are out of touch with the actual needs of enterprises. Many systems rely on simple matching algorithms and lack the ability to deeply understand and analyze patent data, especially in identifying and predicting technology trends. This makes the recommendation results lack foresight and depth.

[0005] (2) The explainability of patent recommendation systems is a key factor in enhancing corporate trust and acceptance. Existing systems are deficient in this regard because they often fail to provide clear reasons for recommendations, making it difficult for companies to understand and trust the recommendations.

[0006] (3) The sparsity problem in patent recommendation tasks limits the recommendation system's ability to learn from corporate behavior. Since a patent is usually assigned to only one company, and a very small number of patents are assigned to two or more companies, the corporate-patent interaction data is extremely sparse, making it difficult for the recommendation system to capture corporate preferences and behavior patterns. In addition, noise and redundant information in the data further exacerbate this problem, potentially causing the model to learn incorrect associations and preferences.

[0007] Therefore, providing a more accurate and comprehensive patent recommendation method and improving the transparency and explainability of the recommendation results are issues that need to be urgently addressed by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides an explainable patent recommendation method and device based on knowledge graph and large language model, optimizes the patent knowledge graph through the diffusion model, recommends patents through a self-supervised learning model, and uses the large language model to generate explainable patent text.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] In a first aspect, the present invention provides an explainable patent recommendation method based on a knowledge graph and a large language model, comprising the following steps:

[0011] Obtain patent text information and patent transfer information of the target enterprise within a preset period and construct a corresponding patent knowledge graph;

[0012] Construct a CPC co-occurrence matrix, identify the core CPC code of the target enterprise through the PageRank algorithm, and derive the patent research direction of the target enterprise;

[0013] Optimizing the patent knowledge graph based on a diffusion model, performing knowledge reasoning on the optimized patent knowledge graph based on a self-supervised learning model, and combining it with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as a patent recommendation path for the target enterprise;

[0014] A large language model is used to integrate the patent recommendation path and patent research direction to generate a recommendation text corresponding to the target enterprise.

[0015] Furthermore, the patent knowledge graph includes entities and relationships between entities;

[0016] The entities include patent ID, inventor ID, applicant address, CPC code and keywords, which serve as nodes of the patent knowledge graph;

[0017] The inter-entity relationships, as edges of the patent knowledge graph, connect the related entities.

[0018] Furthermore, the matrix element m of the CPC co-occurrence matrix ij The co-occurrence frequency of different CPC codes in each patent is expressed as follows:

[0019] m ij =∑ p∈P δ(p,c i )·δ(p,c j );

[0020] Where δ(p,c) represents the membership indicator function between patent p and CPC code c, c i and c j They represent different CPC codes respectively, and P represents all patents of the target enterprise within the preset period.

[0021] Furthermore, the generation process of the diffusion model includes:

[0022] Constructing a patent knowledge graph dataset, encoding the entities and relationships of the patent knowledge graph dataset, and obtaining a binary adjacency matrix as the initial input of the diffusion model;

[0023] gradually adding Gaussian noise to the binary adjacency matrix through a Markov chain and performing forward diffusion to obtain a noisy adjacency matrix;

[0024] Using the noisy adjacency matrix and the binary adjacency matrix as training sets, training a denoising neural network and performing denoising learning; obtaining the mean and covariance of the Gaussian distribution to guide the denoising process of the diffusion model;

[0025] Combine the diffusion loss and the knowledge graph convolution loss to jointly optimize the diffusion model;

[0026] After denoising is completed, the adjacency matrix is ​​reconstructed to obtain the optimized patent knowledge graph G.

[0027] Furthermore, the diffusion loss is used to measure the ability of the diffusion model to recover to its original state after forward diffusion, and to optimize the diffusion model; it is expressed as follows:

[0028]

[0029] Among them, L t represents the loss of the diffusion model at step t; X t represents the noisy adjacency matrix at step t, Represents a given state X t Next, predict the expectation of the original state X0; represents the cumulative noise attenuation coefficient; Indicates that it is based on X t and t predicted X0; represents the uniformly sampled step t in the diffusion steps from 1 to T, T represents the total number of steps; L elbo represents diffusion loss;

[0030] The knowledge graph convolution loss combines the relationship between the enterprise-patent interaction information matrix and the patent knowledge graph, integrates the enterprise interaction information into the denoising process of the patent knowledge graph, and optimizes the diffusion model. The formula is expressed as:

[0031]

[0032] in, represents the enterprise-patent interaction information matrix, Represents the relationship probability matrix predicted by the patent knowledge graph, E u represents the enterprise embedding matrix, E i represents the patent embedding matrix; L ckgc represents the knowledge graph convolution loss.

[0033] Furthermore, the generation process of the self-supervised learning model includes:

[0034] Use the reason weighting function to extract basic semantics from the patent knowledge graph G and calculate the probability of each knowledge triple as the basic principle of collaborative interaction;

[0035] Selecting a preset number of triplets with the highest rationality scores for masking; predicting masked triplets based on the remaining triplets, and calculating the mask reconstruction loss to optimize the self-supervised learning model;

[0036] Adopting the reason perception graph enhancement technology, according to the preset threshold, the corresponding number of triplets with the lowest probability value are deleted to reduce noise and irrelevant data;

[0037] Introducing a contrastive learning task to align signals from knowledge and firm-patent interaction views and enhance the self-supervised learning model's ability to understand and represent data;

[0038] The Bayesian personalized ranking loss function is integrated to optimize the patent ranking based on enterprise preferences in the patent knowledge graph and obtain the corresponding patent recommendation path.

[0039] Furthermore, the usage reason weighting function extracts basic semantics from the optimized patent knowledge graph and calculates the probability of each knowledge triple serving as the basic principle of collaborative interaction; specifically, it includes:

[0040] The probability of calculating the knowledge triple as the basic principle of collaborative interaction is expressed as follows:

[0041]

[0042] The CPC co-occurrence matrix is ​​integrated through the weight function to obtain the weight coefficient w ij , expressed as follows:

[0043]

[0044] The final three-way combination rational score is calculated using the formula:

[0045]

[0046] Where h represents the head entity in the patent knowledge graph, t and t' represent different tail entities in the patent knowledge graph, r and r' represent different relationship entities in the patent knowledge graph; c represents the CPC code, d represents the damping factor; f(h, r, t) and f(h, r', t') represent the different matching degrees between the head entity, relationship and tail entity; ω(h, r, t) represents the final triple combination rationality score; e h 、et 、e r Represent the embedding representation of the head entity, tail entity and relation entity respectively; W Q 、W K represents the training weight; N h Represents the set of neighbor triplets of the head entity h, where the triples exist in the form of (h, r', t'); |N h |Indicates the number of neighbors of the head entity.

[0047] Furthermore, the comparative learning task includes knowledge rational comparative learning and comparative learning between patent knowledge graphs;

[0048] The rational comparative learning of knowledge specifically includes:

[0049] Capturing high-level information in the patent knowledge graph through a graph neural network, generating embedding vectors for patents and target companies, and mapping these embedding vectors into a unified feature space;

[0050] Through comparative learning, the vector distance between the same patents is shortened, while the distance between different patents is expanded, achieving the indirect fusion of patent knowledge graph and interactive view, and obtaining the first loss function of comparative learning;

[0051] The comparative learning between the patent knowledge graphs specifically includes:

[0052] The second loss function of contrastive learning is obtained by comparatively learning the embeddings of enterprises and patents in the original and enhanced patent knowledge graphs.

[0053] In the second aspect, the present invention provides an explainable patent recommendation device based on knowledge graph and language model, including the following modules:

[0054] Data acquisition and graph construction module: used to obtain patent text information and patent transfer information of the target enterprise within a preset period and construct the corresponding patent knowledge graph;

[0055] Patent research direction identification module: used to construct a CPC co-occurrence matrix, identify the core CPC code of the target enterprise through the PageRank algorithm, and derive the patent research direction of the target enterprise;

[0056] Patent recommendation module: used to optimize the patent knowledge graph based on the diffusion model, perform knowledge reasoning on the optimized patent knowledge graph based on the self-supervised learning model, and combine it with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as the patent recommendation path for the target enterprise;

[0057] Recommendation text generation module: used to use a large language model to integrate the patent recommendation path and patent research direction to generate a recommendation text corresponding to the target enterprise.

[0058] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses an explainable patent recommendation method and device based on knowledge graph and large language model, which has the following beneficial effects:

[0059] 1. This invention uses the CPC co-occurrence matrix and PageRank algorithm to accurately locate the core research areas and technological breakthroughs of enterprises, ensuring that the recommended patents are highly consistent with the actual needs of the enterprise, thereby effectively assisting the enterprise's technology research and development decisions.

[0060] 2. The present invention generates a denoising neural network through a diffusion model and combines it with a self-supervised learning model to optimize the knowledge graph, highlight key connections, and reduce noise interference. This not only improves the accuracy of patent recommendations, but also enhances the model's ability to understand data, making the patent recommendation results more credible.

[0061] 3. In the process of generating high-quality recommendation text, the present invention utilizes the natural language generation capability of the large language model to convert the output of the recommendation model into a text with clear logic and detailed content, providing enterprises with detailed recommendation reasons and technical decision-making basis, significantly improving the readability and professionalism of the patent recommendation results, and enhancing the trust and satisfaction of enterprises in the patent recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0063] Figure 1 Flowchart of an explainable patent recommendation method based on knowledge graph and large language model provided in an embodiment of the present invention.

[0064] Figure 2 Structural diagram of an explainable patent recommendation device based on knowledge graph and language model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0066] Example 1

[0067] The embodiment of the present invention discloses an interpretable patent recommendation method based on knowledge graph and large language model, referring to Figure 1 As shown, the following steps are included:

[0068] Obtain patent text information and patent transfer information of the target enterprise within a preset period and construct a corresponding patent knowledge graph;

[0069] Construct a CPC co-occurrence matrix, identify the core CPC codes of the target enterprise through the PageRank algorithm, and derive the patent research direction of the target enterprise;

[0070] De-noising and optimizing the patent knowledge graph based on the trained denoising neural network model;

[0071] Optimizing the patent knowledge graph based on a diffusion model, performing knowledge reasoning on the optimized patent knowledge graph based on a self-supervised learning model, and combining it with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as a patent recommendation path for the target enterprise;

[0072] A large language model is used to integrate the patent recommendation path and patent research direction to generate a recommendation text corresponding to the target enterprise.

[0073] This embodiment uses the method of the present invention to predict technology trends, analyze technology gaps, and assess risks when making decisions on enterprise technology research and development. First, step one is performed to obtain patent data and enterprise data, and to construct a patent knowledge graph to integrate relevant information. Next, step two is performed to construct a CPC co-occurrence matrix and to identify enterprise research directions and technological innovations through the PageRank algorithm. Next, step three is performed to optimize the patent knowledge graph based on a diffusion model and to recommend patents based on a self-supervised learning model. Finally, step four is performed to generate interpretable recommendation text corresponding to the enterprise using a large language model. The present invention can provide enterprises with more accurate, comprehensive, and interpretable patent information services, thereby facilitating technological innovation and industrial development.

[0074] The specific steps of the method of the present invention are described in detail below:

[0075] Step 1 specifically includes:

[0076] First, receive patent data and enterprise data as input;

[0077] During the implementation of this invention, the input received is mainly patent data and transfer data. Since it is difficult to obtain detailed enterprise data, this invention mainly relies on patent data and the transfer information therein to infer the technology needs and research directions of enterprises.

[0078] Specifically, the patent data in this embodiment includes patent text information, primarily the patent title and abstract, and CPC codes. CPC codes are used to classify patents, reflecting the technical field to which they belong and are a crucial component of present invention analysis. Other metadata includes information such as the patent's application date, grant date, and inventor information, all of which contribute to a comprehensive understanding of the patent's context and value.

[0079] Transfer information, which records the transfer of patent ownership, such as the patent's assignee, is key data in this embodiment. By analyzing this transfer information, we can understand the circulation of patents between different companies and, in turn, infer their technological layout and needs. Transfer data is a crucial basis for analyzing corporate technological needs and research directions. By analyzing patent transfer information, we can understand which companies are patent assignees, as well as their level of activity and focus in the technology field. While detailed company data is not available, transfer data can indirectly reflect a company's technological interests and strategic direction, providing valuable information for patent recommendations.

[0080] This example cleans and normalizes patent and transfer data through data preprocessing to ensure data quality and consistency. This example denoises patent text to remove irrelevant characters and repetitive content; standardizes CPC codes to unify the format of classification codes; and categorizes and organizes enterprise data to facilitate subsequent analysis and processing. These preprocessing steps improve data availability and accuracy, providing a more reliable data foundation for subsequent patent recommendations.

[0081] Secondly, build a patent knowledge graph and integrate relevant information;

[0082] In this embodiment of the present invention, entities such as patent IDs, inventor IDs, CPC codes, applicant addresses, and keywords are extracted from patent data. Relationships between entities are defined, including the association between inventors and patents, patent classifications under CPC codes, and semantic connections between patents and keywords. A large language model is then used to identify and extract key entity information from patent titles and abstracts, ensuring the accuracy and relevance of the extraction. Through these steps, a graph structure is constructed that comprehensively reflects the patent knowledge system. Constructing a patent knowledge graph includes the following steps: entity extraction, relationship definition, and knowledge graph integration.

[0083] Entity extraction is the process of extracting entities such as patent ID, inventor ID, CPC code, applicant address, and keywords from patent data. Entities are nodes in the patent knowledge graph, representing various objects and concepts related to patents.

[0084] In this embodiment, the patent ID includes the patent publication number and application number, indicating the identity of the patent; through the inventor ID, all patents of a certain inventor can be quickly retrieved to understand their research direction and technical achievements.

[0085] Specifically, the entity extraction process can use a large language model to identify and extract key entity information from the title and abstract of the patent.

[0086] This example constructs prompt words to guide the model in analyzing the text content and generating output containing key entity information. Due to its powerful natural language understanding and generation capabilities, the large language model can more accurately identify and extract key technical terms and related entities in patents than conventional keyword extraction models. These entities serve as the basic building blocks of the knowledge graph, providing the foundation for subsequent relationship construction and knowledge representation.

[0087] Relationship definitions define the relationships between entities, including the association between inventors and patents, the classification of patents under CPC codes, and the semantic connection between patents and keywords. This relationship is the edge between entities in the patent knowledge graph, describing the interaction and association between entities.

[0088] Specifically, the relationship definition process can be determined by analyzing the structured information and text content in the patent data.

[0089] This example establishes relationships between inventors and patents using a patent's inventor list; and between patents and corresponding technology classifications based on their CPC codes. These relationships connect entities in the patent knowledge graph, forming a complex network structure that reflects the various connections and dependencies within the patent knowledge system.

[0090] Specifically, the relationship definition can also be combined with the citation relationship of the patent to expand the relationship types in the knowledge graph.

[0091] This example analyzes patent citation relationships to establish technical impact relationships between cited and citing patents. These expanded relationships can more comprehensively reflect the role and status of patents in technological development and market layout, providing richer semantic information for patent recommendations.

[0092] Knowledge graph integration integrates the extracted entities and defined relationships into a unified knowledge graph, forming a structured patent knowledge network. The knowledge graph integrates multi-dimensional information about patents, providing a rich knowledge foundation for subsequent patent recommendations.

[0093] The process of constructing the graph structure of the patent knowledge network in this embodiment is as follows:

[0094] Use NetworkX to create a directed multigraph (MultiDiGraph), with companies, patents, and knowledge entities as nodes and interactions and knowledge relationships as edges. Specifically, transfer relationships between companies and patents are added as edges, as are triples in the knowledge graph. This graph structure comprehensively reflects company behavior and knowledge relationships.

[0095] Secondly, knowledge graph integration can be achieved through graph database technology, storing entities and relationships as nodes and edges in a graph structure. Specifically, using a graph database management system such as Neo4j, extracted patent IDs and inventor IDs can be stored as nodes, and relationships between entities as edges. Attributes and weights can be added to each node and edge. This facilitates graph querying, traversal, and analysis, supporting complex graph computations and knowledge reasoning.

[0096] Step 2 specifically includes:

[0097] First, construct the CPC co-occurrence matrix.

[0098] This example collects all transferred patents from an enterprise within a certain period and extracts the CPC code for each patent. It then counts the co-occurrence frequencies between different CPC codes, i.e., the number of times they appear simultaneously in the same patent or the same enterprise's patent portfolio. These frequencies are organized into a matrix, with rows and columns representing different CPC classification codes, and matrix elements representing corresponding co-occurrence frequencies. The specific formula is:

[0099]

[0100] Among them, δ(p,c) is the membership indicator function between patent p and CPC category c, P represents all patents of the target enterprise in the preset period, c i and c j Represents different CPC codes respectively. When the patent contains c i with c j When classifying, the matrix element m ij The value is increased by 1 to form a symmetric co-occurrence matrix.

[0101] Secondly, the PageRank algorithm is used to identify corporate research directions and technological innovations.

[0102] This embodiment analyzes the enterprise's transferred patent data to construct a CPC code co-occurrence matrix. This matrix reveals the interconnectedness between different CPC codes, thereby helping to identify the enterprise's main research direction; it uses the PageRank algorithm to quantify the importance of each CPC code within the company's research focus and identify CPC codes that occupy a core position in the company's patent portfolio.

[0103] The PageRank algorithm applies the following process: The CPC co-occurrence matrix is ​​treated as a graph structure, where nodes represent CPC codes and edge weights represent the frequency of co-occurrence or the strength of association between them. Based on the PageRank algorithm formula, a PageRank score is calculated for each node, reflecting its importance and influence within the entire graph. The algorithm's convergence and accuracy are ensured by setting an appropriate damping coefficient and number of iterations.

[0104] The formula is:

[0105]

[0106] Among them, PR(c i ) represents node c i PageRank score; d is the damping factor, which is set to 0.85 in this embodiment; Z represents the total number of nodes in the CPC co-occurrence matrix; M(c i ) indicates pointing to node c i The node set of L(c j ) represents the slave node c j The number of outgoing links, PR(c j ) represents node c j To further illustrate, the calculation of PageRank is an iterative process. Initially, the PageRank score of each node is usually set to the same value, that is, As iterations proceed, the score of each node is updated based on the scores of its incoming nodes and the number of outgoing nodes.

[0107] In this embodiment, the process of extracting enterprise research directions and technological innovations takes CPC classification numbers and related patents as input, combines the natural language understanding and generation capabilities of the large language model, and performs semantic analysis and knowledge extraction.

[0108] Specifically, by constructing prompt words, the large language model is guided to analyze the technical content, innovations, and application scenarios of these patents, generating detailed descriptions of the company's research directions and technological innovations. Based on information such as technical terminology in patents, the large language model can extract the company's core research directions and key technological breakthroughs in specific technical fields.

[0109] Step three specifically includes:

[0110] (1) Optimizing patent knowledge graph based on diffusion model;

[0111] In this embodiment, the process of generating the diffusion model specifically includes:

[0112] Knowledge graph encoding: Encode the entities and relations of the knowledge graph into a binary adjacency matrix as the initial input of the diffusion model;

[0113] Forward Diffusion Model: Gaussian noise is gradually added to the adjacency matrix through a Markov chain to simulate the gradual destruction of the knowledge graph structure;

[0114] Inverse denoising learning: training a neural network to predict noise and reconstructing the original adjacency matrix from the pure noise state through step-by-step denoising;

[0115] Joint optimization training: Combining diffusion loss and knowledge graph convolution loss to jointly optimize recommendation tasks and graph structure learning;

[0116] Adjacency matrix reconstruction: Restore the optimized adjacency matrix based on the predicted noise, and retain key edges through Topk and the original knowledge graph screening.

[0117] The knowledge graph encoding process is as follows:

[0118] Convert the entities and relations in the knowledge graph into a binary adjacency matrix X0∈{0,1} |E|×|E| ;

[0119] If there is a relationship between entities a and b, then X0[a,b] = 1, otherwise it is 0.

[0120] The forward diffusion model process is as follows: Gaussian noise is gradually added to the adjacency matrix through the Markov chain. After T steps of diffusion, the original adjacency matrix X0 is destroyed into an approximate pure noise X T . The specific formula is:

[0121]

[0122] Where t represents the time step, X t represents the noisy adjacency matrix at step t, represents Gaussian distribution, β t represents the noise scheduling parameter, which is used to control the intensity of noise addition, I represents the unit matrix, q(X t |X t-1 ) represents the state transition probability distribution from time step t-1 to time step t in the forward process of the diffusion model.

[0123] To avoid the time-consuming problem of step-by-step iterative calculation, the forward diffusion process of this embodiment uses a closed-form solution to directly calculate the noisy adjacency matrix X at any time step t. t .

[0124] The specific formula is:

[0125]

[0126] in, represents the cumulative noise attenuation coefficient, t' represents an index variable used to traverse all intermediate steps from 1 to t, β t' represents the variance ratio of the noise added at step t', and ∈ represents standard Gaussian noise.

[0127] The reverse denoising learning process is as follows: the parameterized θ of the denoising neural network model is trained to generate the mean and covariance of the Gaussian distribution to guide the denoising process.

[0128] The specific formula is:

[0129]

[0130] Among them, p θ (X t-1 |X t ) means that given the current state X t Next, restore to the previous state X t-1 The conditional probability distribution of μ θ (X t ,t) represents the mean of Gaussian distribution, ∑ θ (X t ,t) represents the covariance of the Gaussian distribution.

[0131] The joint optimization training process is as follows: define the first loss L of the diffusion model elbo As the diffusion loss, this loss is used to measure the ability of the diffusion model to recover the original state in the first step.

[0132] The specific formula is:

[0133]

[0134]

[0135] Among them, L t represents the loss of the diffusion model at step t; X t represents the noisy adjacency matrix at step t, Represents a given state X t Next, the expectation of state X0; represents the cumulative noise attenuation coefficient; Indicates that it is based on X t and t predicted X0, represents the uniform sampling step t in the diffusion step from 1 to T; L elbo represents diffusion loss.

[0136] In order to make the denoising neural network model trained by the diffusion model more relevant to the recommendation, the diffusion model defines the second loss L of the diffusion model ckgc As a knowledge graph convolution loss, by combining enterprise-patent interaction information and the relationship probability predicted by the knowledge graph, enterprise interaction information is integrated into the optimization process of the knowledge graph, enhancing the model's understanding of enterprise preferences.

[0137] The specific formula is:

[0138]

[0139] in, represents the enterprise-patent interaction information matrix, Represents the relationship probability matrix predicted by the knowledge graph, E u represents the enterprise embedding matrix, E v represents the patent embedding matrix.

[0140] The diffusion model jointly optimizes the training of two losses, which is expressed as follows:

[0141] L diff =(1-λ0)L elbo +λ0L ckgc

[0142] Among them, λ0 is the parameter that controls the loss.

[0143] The adjacency matrix reconstruction process is as follows: Based on the trained denoising neural network, the original adjacency matrix X0 is denoised and optimized to obtain X0';

[0144] The denoising and optimized X0' is double-screened to obtain the optimized adjacency matrix X: the top k edges with the highest prediction weight are retained for each entity node; the edges that do not exist in the original adjacency matrix X0 are removed; and finally it is converted into the knowledge graph G.

[0145] In this embodiment, the patent knowledge graph of the target enterprise obtained in step 1 is optimized by the trained diffusion model to obtain an optimized patent knowledge graph.

[0146] (2) Based on the trained self-supervised learning model, the optimized patent knowledge graph is subjected to knowledge reasoning and combined with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as the patent recommendation path for the target enterprise;

[0147] This embodiment performs denoising optimization on the original adjacency matrix based on the trained diffusion model; then performs self-supervised learning.

[0148] The training process of the self-supervised learning model is as follows: first, a reason-weighted function is used to extract basic semantics from the knowledge graph G, and the probability of each knowledge triple being the basic principle of collaborative interaction is calculated; and the triples with the highest rationality score are selected for masking; the reason-aware graph enhancement technology is used to delete unimportant edges to improve the efficiency of contrastive learning and reduce noise and irrelevant data; a contrastive learning task is introduced to align signals from knowledge and enterprise-patent interaction views and enhance the model's understanding and representation capabilities of the data; and a Bayesian personalized ranking (BPR) loss function is integrated to optimize the patent ranking based on enterprise preferences.

[0149] The application process of the reason weighting function is as follows: the embedded representation of each knowledge triple in the knowledge graph G is input into the function, and the probability value of each triple is calculated in combination with the learnable graph attention mechanism.

[0150] The specific formula is:

[0151]

[0152] Among them, f(h,r,t) represents the matching degree between the head entity h, relation r and tail entity t, e h 、e t 、e r Represent the embedding representations of the head entity, tail entity and relation entity respectively, W Q 、W K are the training weights, and d represents the dimension of the embedding vector.

[0153] By integrating CPC co-occurrence matrix weights, the relationship between specific technical fields and classification codes is strengthened.

[0154] The specific formula is:

[0155]

[0156]

[0157] Among them, w ij represents the weight coefficient of the co-occurrence matrix, h represents the head entity in the patent knowledge graph, t and t' represent different tail entities in the patent knowledge graph, r and r' represent different relationship entities in the patent knowledge graph; c represents the CPC code. When the head entity h and the tail entity t are both CPC codes, the weight coefficient w is calculated using logarithmic weighting. ijf(h,r,t) and f(h,r',t') represent the different matching degrees between the head entity, the relationship and the tail entity; ω(h,r,t) represents the rational score of the final triple combination, N h Represents the set of neighbor triplets of the head entity h, where the triples exist in the form of (h, r', t'); |N h |Indicates the number of neighbors of the head entity.

[0158] The selection and masking process is as follows:

[0159] According to the probability values ​​calculated by the reason weighting function, all knowledge triplets are sorted and the top triplet with the highest score is selected; then a mask operation is performed on these selected triplets, allowing the model to learn how to predict and reconstruct these masked triplets based on the remaining information.

[0160] The specific formula is:

[0161] M k ={(h,r,t)|ω(h,r,t)∈topk(Γ;k m )}

[0162]

[0163] Among them, M k represents the set of triplets selected for masking, Γ represents the distribution set of all ω(h, r, t), L m represents the mask loss, k m represents the number of triplets to be masked, and σ represents the Sigmod activation function.

[0164] The reason-aware graph enhancement process is as follows: according to the rationality score of the knowledge triple, the importance score of each edge is calculated; then, according to a certain ratio or threshold, the edges with lower scores are deleted and the edges with higher scores are retained to form an enhanced knowledge graph.

[0165] The specific formula is:

[0166] S kg ={(h,r,t)|ω(h,r,t)∈topk(Γ;ρ m )}

[0167] Among them, S kg It means that the triples with the highest scores are finally included, which are considered to be the most credible. Γ represents the distribution set of all ω(h, r, t), ρ m represents the selection ratio in control training.

[0168] The self-supervised learning model of this embodiment optimizes the patent ranking based on enterprise preferences through mask reconstruction learning, knowledge rational comparative learning and comparative learning optimization model between patent knowledge graphs.

[0169] Among them, knowledge rational comparative learning includes:

[0170] First, graph neural network models like LightGCN are used to capture high-level information from the knowledge graph and generate embedding vectors for patents and companies. These embedding vectors are then mapped into a unified feature space. Through comparative learning tasks, the vector distances between identical patents are shortened while the distances between different patents are increased, achieving an indirect fusion of the knowledge graph and the interaction graph. In this process, the model learns the implicit relationships between the knowledge graph and company-patent interactions, enabling a better understanding of companies' technological preferences and recommending more relevant patents.

[0171] Among them, the embedding vectors of patents and enterprises are generated, and the specific formula is expressed as:

[0172]

[0173] in, and Represent the embedding vectors of patent v and enterprise u at layer l, N v and N u Represent the neighbor sets of patent v and enterprise u respectively.

[0174] Mapped to a unified feature space, the specific formula is expressed as:

[0175]

[0176] Among them, * indicates variables of different views, representing knowledge graph and interaction graph; represents the representation of patent v in the unified feature space, represents the weight matrix of the first and second layers, which is used to linearly transform the embedding vector. Represents the bias term of the first and second layers, which is used to adjust the mapped result. σ represents the Sigmoid activation function, which is used to map the result after linear transformation to the (0,1) interval.

[0177] Contrastive learning task, the specific formula is expressed as:

[0178]

[0179] Among them, L cl1 represents the loss function of contrastive learning, v represents a patent, belongs to the set V of all patents, v' and v" are randomly sampled negative candidate sets, represents the embedding vector of patent v in the interaction graph, v represents the embedding vector of patent v in the knowledge graph, j represents an index variable used to traverse patent v and negative candidate sets v', v", s(·) represents the cosine similarity of the normalized vector, and τ represents the temperature hyperparameter that controls the hardness of the comparison target.

[0180] At the same time, in order to enhance the model's ability to understand and represent data and learn a more robust and rich knowledge graph representation, this embodiment also introduces comparative learning between graphs. The specific formula is expressed as follows:

[0181]

[0182] Among them, u and u' represent an enterprise in the enterprise set U, v and v' represent different patents in the patent set V, and x' u and x” u represents the embedding of enterprise u in the original knowledge graph and the embedding of the optimized knowledge graph, x' v and x” v represents the embedding of patent i in the original knowledge graph and the embedding of the optimized knowledge graph, s(·) represents the cosine similarity of the normalized vector, and τ represents the temperature hyperparameter that controls the hardness of the comparison target. and They represent the enterprise comparison loss of different views and the patent comparison loss of different views respectively. cl2 represents the contrast loss of different views.

[0183] The final contrastive learning loss function is:

[0184] L CL =L cl1 +L cl2

[0185] Integrated Bayesian Personalized Ranking (BPR) loss function,

[0186] During the training process, this embodiment also integrates a Bayesian personalized ranking (BPR) loss function to calculate the difference in predicted scores between patents with which a company has actually interacted and randomly sampled patents with which it has not. By maximizing the difference between the predicted scores of patents with actual interactions and those with no interactions, the model's parameters are optimized, enabling the model to more accurately predict company preferences and behaviors. Furthermore, a joint learning approach is employed, combining the loss functions of the mask reconstruction task and the contrastive learning task to comprehensively optimize the model's performance.

[0187] Among them, the specific formula of BPR loss function is expressed as:

[0188]

[0189] Where D represents the training data set, which contains triples (u, v, v'). It is a real interaction sample. are randomly sampled negative samples.

[0190] The joint learning method of this embodiment is specifically expressed as follows:

[0191] L=L rec +λ1L m +λ2L CL

[0192] Among them, λ1 and λ2 represent the weights of the mask reconstruction task and the contrastive learning task; L rec represents the BPR loss described in this embodiment; L m represents the mask loss in this embodiment; L CL represents the contrast loss described in this embodiment.

[0193] In this embodiment, the optimized patent knowledge graph is input into the trained self-supervised learning model for learning to obtain the patent recommendation path of the target enterprise.

[0194] Step 4 specifically includes:

[0195] The recommendation model trained in step three is used to calculate the correlation scores between core elements such as enterprises, patents, and keywords to obtain relevant paths for patent recommendations; these paths are input into the large language model together with multi-dimensional information such as the company's specific technical direction, patent details, and keywords; the large language model's excellent language understanding and generation capabilities are used to effectively integrate the input multi-dimensional information to generate content-rich and highly relevant recommendation text.

[0196] The process of integrating multi-dimensional information is as follows: information such as recommendation paths and corporate research directions is structured to form a unified input data set; by setting prompt words and templates, the large language model is guided to conduct comprehensive analysis and understanding of this information.

[0197] The large language model encodes and represents input information to capture key semantics and logical relationships. It also identifies key information in recommended patents, such as technical terms, innovations, and application scenarios. Furthermore, it assesses the relevance and value of recommended patents based on the company's research direction and technical needs. Through these analyses and extractions, the model provides rich semantic content and knowledge support for generating high-quality recommendation text.

[0198] The large language model generates high-quality recommendation text, including detailed explanations of the reasons for the recommendation. Based on the multi-dimensional input information, the large language model uses its internal Transformer architecture and large-scale pre-trained parameters to perform semantic understanding and content generation. The large language model first encodes and represents the input information, capturing key semantics and logical relationships. Based on the encoding results, it then automatically generates the recommendation text. The generated text features clear logic, fluent language, and detailed content, providing valuable decision support for enterprises.

[0199] Example 2

[0200] The embodiment of the present invention provides an explainable patent recommendation device based on knowledge graph and language model, referring to Figure 2 As shown, it includes the following modules:

[0201] Data acquisition and graph construction module: used to obtain patent text information and patent transfer information of the target enterprise within a preset period and construct the corresponding patent knowledge graph;

[0202] Patent research direction identification module: used to construct CPC co-occurrence matrix, identify the core CPC code of the target enterprise through PageRank algorithm, and derive the patent research direction of the target enterprise;

[0203] Patent recommendation module: used to optimize the patent knowledge graph based on the diffusion model, perform knowledge reasoning on the optimized patent knowledge graph based on the self-supervised learning model, and combine it with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as the patent recommendation path for the target enterprise;

[0204] Recommendation text generation module: used to use the large language model to integrate patent recommendation paths and patent research directions to generate recommendation text corresponding to the target enterprise.

[0205] This embodiment aims to provide enterprises with accurate and personalized patent recommendation services by leveraging the advantages of knowledge graphs and large language models, while also offering in-depth rationale analysis to support their decision-making process. This system not only considers the technical content of the patents themselves, but also incorporates multi-dimensional information such as the relationships between different patents and the company's research directions, thereby providing enterprises with more personalized and accurate patent recommendation services. Furthermore, by utilizing these advanced technologies, the system can effectively improve the quality of recommendations and provide enterprises with detailed rationale explanations for recommendations, helping them better understand and evaluate the recommendation results.

[0206] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0207] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0208] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An explainable patent recommendation method based on knowledge graph and large language model, characterized by: The following steps are involved: Obtain patent text information and patent transfer information of the target enterprise within a preset period and construct a corresponding patent knowledge graph; Construct a CPC co-occurrence matrix, identify the core CPC code of the target enterprise through the PageRank algorithm, and derive the patent research direction of the target enterprise; Optimizing the patent knowledge graph based on a diffusion model, performing knowledge reasoning on the optimized patent knowledge graph based on a self-supervised learning model, and combining it with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as a patent recommendation path for the target enterprise; A large language model is used to integrate the patent recommendation path and patent research direction to generate a recommendation text corresponding to the target enterprise.

2. The explainable patent recommendation method based on knowledge graph and large language model according to claim 1 is characterized in that: The patent knowledge graph includes entities and relationships between entities; The entities include patent ID, inventor ID, applicant address, CPC code and keywords, which serve as nodes of the patent knowledge graph; The inter-entity relationships, as edges of the patent knowledge graph, connect the related entities.

3. The explainable patent recommendation method based on knowledge graph and large language model according to claim 1 is characterized in that: The matrix element m of the CPC co-occurrence matrix ij The co-occurrence frequency of different CPC codes in each patent is expressed as follows: m ij =∑ p∈P δ(p,c i )·δ(p,c j ); Where δ(p,c) represents the membership indicator function between patent p and CPC code c, c i and c j They represent different CPC codes respectively, and P represents all patents of the target enterprise within the preset period.

4. The explainable patent recommendation method based on knowledge graph and large language model according to claim 1 is characterized in that: The generation process of the diffusion model includes: Constructing a patent knowledge graph dataset, encoding the entities and relationships of the patent knowledge graph dataset, and obtaining a binary adjacency matrix as the initial input of the diffusion model; gradually adding Gaussian noise to the binary adjacency matrix through a Markov chain and performing forward diffusion to obtain a noisy adjacency matrix; Using the noisy adjacency matrix and the binary adjacency matrix as training sets, training a denoising neural network and performing denoising learning; obtaining the mean and covariance of the Gaussian distribution to guide the denoising process of the diffusion model; Combine the diffusion loss and the knowledge graph convolution loss to jointly optimize the diffusion model; After denoising is completed, the adjacency matrix is ​​reconstructed to obtain the optimized patent knowledge graph G.

5. The explainable patent recommendation method based on knowledge graph and large language model according to claim 4 is characterized in that: The diffusion loss is used to measure the ability of the diffusion model to recover its original state after forward diffusion and to optimize the diffusion model; it is expressed as follows: Among them, L t represents the loss of the diffusion model at step t; X t represents the noisy adjacency matrix at step t, Represents a given state X t Next, predict the expectation of the original state X0; represents the cumulative noise attenuation coefficient; Indicates that it is based on X t and t predicted X0; represents the uniformly sampled step t in the diffusion steps from 1 to T, T represents the total number of steps; L elbo represents diffusion loss; The knowledge graph convolution loss combines the relationship between the enterprise-patent interaction information matrix and the patent knowledge graph, integrates the enterprise interaction information into the denoising process of the patent knowledge graph, and optimizes the diffusion model. The formula is expressed as: in, represents the enterprise-patent interaction information matrix, Represents the relationship probability matrix predicted by the patent knowledge graph, E u represents the enterprise embedding matrix, E i represents the patent embedding matrix; L ckgc represents the knowledge graph convolution loss.

6. The explainable patent recommendation method based on knowledge graph and large language model according to claim 5 is characterized in that: The generation process of the self-supervised learning model includes: Use the reason weighting function to extract basic semantics from the patent knowledge graph G and calculate the probability of each knowledge triple as the basic principle of collaborative interaction; Selecting a preset number of triplets with the highest rationality scores for masking; predicting masked triplets based on the remaining triplets, and calculating the mask reconstruction loss to optimize the self-supervised learning model; Adopting the reason perception graph enhancement technology, according to the preset threshold, the corresponding number of triplets with the lowest probability value are deleted to reduce noise and irrelevant data; Introducing a contrastive learning task to align signals from knowledge and firm-patent interaction views and enhance the self-supervised learning model's ability to understand and represent data; The Bayesian personalized ranking loss function is integrated to optimize the patent ranking based on enterprise preferences in the patent knowledge graph and obtain the corresponding patent recommendation path.

7. The explainable patent recommendation method based on knowledge graph and large language model according to claim 6 is characterized in that: The usage reason weighting function extracts basic semantics from the optimized patent knowledge graph and calculates the probability of each knowledge triple serving as the basic principle of collaborative interaction; specifically, it includes: The probability of calculating the knowledge triple as the basic principle of collaborative interaction is expressed as follows: The CPC co-occurrence matrix is ​​integrated through the weight function to obtain the weight coefficient w ij , expressed as follows: The final three-way combination rational score is calculated using the formula: Where h represents the head entity in the patent knowledge graph, t and t' represent different tail entities in the patent knowledge graph, r and r' represent different relationship entities in the patent knowledge graph; c represents the CPC code, d represents the damping factor; f(h, r, t) and f(h, r', t') represent the different matching degrees between the head entity, relationship and tail entity; ω(h, r, t) represents the final triple combination rationality score; e h 、e t 、e r Represent the embedding representation of the head entity, tail entity and relation entity respectively; W Q 、W K represents the training weight; N h Represents the set of neighbor triplets of the head entity h, where the triples exist in the form of (h, r', t'); |N h |Indicates the number of neighbors of the head entity.

8. The explainable patent recommendation method based on knowledge graph and large language model according to claim 7 is characterized in that: The contrastive learning task includes knowledge rational contrastive learning and contrastive learning between patent knowledge graphs; The rational comparative learning of knowledge specifically includes: Capturing high-level information in the patent knowledge graph through a graph neural network, generating embedding vectors for patents and target companies, and mapping these embedding vectors into a unified feature space; Through comparative learning, the vector distance between the same patents is shortened, while the distance between different patents is expanded, achieving the indirect fusion of patent knowledge graph and interactive view, and obtaining the first loss function of comparative learning; The comparative learning between the patent knowledge graphs specifically includes: The second loss function of contrastive learning is obtained by comparatively learning the embeddings of enterprises and patents in the original and enhanced patent knowledge graphs.

9. An explainable patent recommendation device based on knowledge graph and large language model, characterized by: Includes the following modules: Data acquisition and graph construction module: used to obtain patent text information and patent transfer information of the target enterprise within a preset period and construct the corresponding patent knowledge graph; Patent research direction identification module: used to construct a CPC co-occurrence matrix, identify the core CPC code of the target enterprise through the PageRank algorithm, and derive the patent research direction of the target enterprise; Patent recommendation module: used to optimize the patent knowledge graph based on the diffusion model, perform knowledge reasoning on the optimized patent knowledge graph based on the self-supervised learning model, and combine it with the CPC co-occurrence matrix to obtain an enhanced patent knowledge graph as the patent recommendation path for the target enterprise; Recommendation text generation module: used to use a large language model to integrate the patent recommendation path and patent research direction to generate a recommendation text corresponding to the target enterprise.

Images