LLM-driven semantic association extension and structure completion patent recommendation method and system

Abstract

The invention discloses an LLM-driven semantic association extension and structure completion patent recommendation method and system, and the method comprises the steps: S100, carrying out the concept induction and completion of a discrete patent attribute text based on LLM, so as to generate the background context information of a patent or an enterprise; s200, based on context semantics generated by a large language model, background context correlation between patents and between enterprises is calculated, Top-k most related patents or enterprises are selected according to correlation sorting, a same-field background edge relation is constructed, patent-patent and enterprise-enterprise correlation edges are complemented, and a heterogeneous network structure is obtained; and S300, performing feature propagation and aggregation on the heterogeneous network structure based on a graph convolution model, aggregating high-order heterogeneous neighbor features, optimizing a recommendation relationship in the heterogeneous network structure, and predicting the probability of occurrence of patent recommendation behaviors. According to the method provided by the invention, the accuracy rate is improved by 8.28%, the precision rate is improved by 1.73%, the recall rate is improved by 12.31%, and the F1 score is improved by 6.23%.

Description

Technical Field

[0001] The present invention belongs to the field of patent recommendation technology, and more specifically, relates to a patent recommendation method and system for LLM-driven semantic association expansion and structure completion. Background Art

[0002] Patent technology transfer holds significant strategic importance in promoting scientific and technological progress, facilitating the commercialization of innovative achievements, and elevating industrial technological capabilities. Currently, patent technology trade, particularly through transfer and licensing, has become a crucial means for enterprises, regions, countries, and other market entities to acquire new resources and enhance their innovation capabilities. As of November 2024, enterprises accounted for 73.5% of my country's valid invention patents, an increase of 2.5 percentage points year-on-year. Over 90% of technological innovation information is stored in international and domestic patent repositories. This patent knowledge not only assists expert decision-making but also reduces R&D time by approximately 60% and costs by approximately 40%. Despite this, many patented technological achievements remain untranslated and applied, leaving a significant number of innovative products with unmet patent technology needs. Therefore, addressing the sluggishness of patent product trading, patent recommendation is a key component in addressing patent technology transfer. It helps accurately and rapidly model enterprise-patent relationships, improving the efficiency of transfers between enterprises and patented technologies.

[0003] Faced with a massive and continuously growing number of patents, an efficient recommendation algorithm can not only help the target audience quickly identify the ones they are interested in. However, existing patent recommendation research has problems in the process of utilizing patent attribute knowledge content. These problems include low utilization of patent knowledge, difficulty in achieving dynamic transformation, and difficulties in the decomposition, fusion and reorganization of product patent knowledge. Research based on patent recommendation mainly focuses on patent knowledge recommendation and binary patent recommendation tasks for enterprises. In patent matching indexes, technologies such as topic models, text clustering, and text similarity are often used to achieve personalized recommendations for corporate patents. For example, in terms of patent recommendation, researchers mainly focus on extracting key application scenarios and technical terms from patents as features to distinguish different patent knowledge. In addition, early patent recommendation research mainly focused on patent quantification and feature extraction. Patents are quantified through machine learning models (such as support vector machine models and probabilistic graph models) to be classified or ranked by value for recommendation. However, the above studies lack the application of the large amount of text attributes contained in patent-enterprise relationships, which results in a low utilization rate of patents recommended to enterprises. The present invention focuses on the use of advanced large language model technology to generate enterprise-patent field background context, and to achieve structural completion and enhancement of sparse enterprise-patent heterogeneous networks. It also promotes patent conversion by discovering high-order enterprise-patent conversion opportunities and conversion conditions through lightweight graph convolution, such as the conversion model of heterogeneous networks.

[0004] In recent years, recommendation models based on deep neural network technology have largely met users' needs for daily product recommendations and social media recommendations. However, due to the extensive textual attributes contained in enterprise-patents, a core challenge facing recommendation systems is how to model and enable the models to understand, process, and respond to textual content. Currently, research on patent recommendation based on knowledge graphs (KGs) is increasing. These methods map abstract knowledge into graph-structured content using structured semantic network knowledge bases, leveraging the various information characteristics of patents to facilitate human-computer interaction and expand knowledge, thereby enabling patent recommendation. Although KG-based recommendation systems have received extensive attention in the literature, their modeling complexity is extremely high in the enterprise-patent scenario with diverse attribute content, and their application to patent recommendation requires further research. Furthermore, existing methods face performance and semantic relevance limitations when learning to represent attribute text information. Patents, as intellectually intensive products with high technical content and complex content, involve a large amount of attribute text in the recommendation process, thus differing from the simple buyer-seller relationship applicable to ordinary commodities. Traditional patent recommendation methods still have the following problems: (1) Low patent-enterprise matching: In existing studies, a large number of patent recommendation processes often achieve patent recommendations by matching enterprise-patent content knowledge, which makes the complex structural relationship between enterprises and patents not taken into account, and a large number of text attributes (source domain information and target domain information) are not utilized, resulting in mismatched patents recommended to enterprises or individuals [7]. (2) Lack of efficient use of information such as text attributes: Patent recommendation is essentially a social network of complex heterogeneous information including patent applicants, patent inventors, patent titles, and patent regions. Existing recommendation systems often focus on the information matching degree or semantic relevance between interactive items in the recommendation, while ignoring the domain background context between recommended items, resulting in the rich heterogeneous information data not being fully utilized. Therefore, the present invention deeply mines the feature information and association relationships contained in the patent attribute text data, and systematically generates and infers the technical background context in patent recommendations to improve the accuracy of patent recommendations. (3) Ignoring the complex heterogeneous characteristics in the patent recommendation process: Previous studies only considered the direct recommendation relationship between patents and demanders, ignoring the large number of heterogeneous characteristics in the recommendation link. Patents are complex, heterogeneous relationships involving fields, people, and collaborations, rather than just commodities[7]. Patent recommendation systems based on knowledge graphs can perform text mining on patent information and extract potential connections between patents in terms of technology, collaboration, and other aspects. However, when faced with complex and heterogeneous patent data, the modeling cost of recommendation systems designed based on graph structures is extremely complex. Therefore, how to efficiently utilize the complex and heterogeneous relationships between enterprises and patents is extremely critical. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a patent recommendation method and system driven by LLM-driven semantic association expansion and structure completion, combining the domain background context generation capability and structure completion technology of the large language model to alleviate the data sparsity problem, and model high-order heterogeneous relationship structures to improve the efficiency and accuracy of patent recommendation, and solve the problems of insufficient modeling of complex text attributes and sparse interaction in enterprise-patent heterogeneous networks in existing research methods.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a patent recommendation method for LLM-driven semantic association expansion and structure completion is provided, comprising:

[0007] S100: Based on LLM, the concept of discrete patent attribute text is summarized and completed to generate background context information of the patent or enterprise, and enhance the model's understanding of text semantics;

[0008] S200: Based on the contextual semantic representation generated by the large language model, the contextual relevance between patents and companies is calculated. The top-k most relevant patents or companies are selected based on the relevance ranking, and the contextual edge relationships in the same field are constructed. The patent-patent and company-company relevance edges are completed to obtain a heterogeneous network structure.

[0009] S300: Based on the graph convolution model, feature propagation and aggregation are performed on the heterogeneous network structure, high-order heterogeneous neighbor characteristics are aggregated, the recommendation relationship in the heterogeneous network structure is optimized, and the background context information and the structure characteristics of the heterogeneous network are further integrated to predict the probability of occurrence of patent recommendation behavior.

[0010] Furthermore, in step S100, the generation of background context information of the patent or enterprise includes:

[0011] S201: Use the BERT pre-trained model to capture the contextual relationship of the text to extract the contextual features of the patent representation. Since BERT is a variant of Transformer, d k Represents the dimension of the BERT model output, which constructs fixed-dimensional token vectors and positional encodings for patent / enterprise context segmentation to determine the order of sentence context;

[0012] S202: Use the multi-head self-attention mechanism to fuse the sentence context information, and its output is:

[0013]

[0014] Where Q, K and V represent query, key and value matrices respectively;

[0015] S203: Output the embedded representation of the patent / enterprise background context through a feedforward neural network, layer normalization, and residual connections:

[0016] FFN(x)=max(0,xW1+b1)W2+b2,(2)

[0017]

[0018] Where FFN(x) represents the linear transformation of the long text features of the topic, Represents the background context information of the patent or enterprise after node i is residually connected and layer normalized.

[0019] Furthermore, in step S200, the heterogeneous network structure includes:

[0020] S201: The heterogeneous graph based on enterprise-patent transfer includes the transfer relationships between enterprises and patents, forming a complex heterogeneous relationship network, where nodes represent enterprises or patents and edges represent the transfer relationships between them;

[0021] S202: Cosine similarity is used to measure the contextual semantic similarity of nodes of the same type to evaluate the closeness between patents. The calculation formula of cosine similarity is:

[0022]

[0023]

[0024] in, and The embedding vectors of the context of the two patent fields are the sum of the dot products of the 716-dimensional vectors, A i represents the set of relevance scores between patent i and other patent field context embeddings.

[0025] Furthermore, in step S200, the heterogeneous network structure includes:

[0026] S203: By calculating the cosine similarity of the contextual semantics, the most relevant Top-k nodes are selected and the edge relationship between the two nodes is completed to achieve the cosine similarity Top-k selection of the domain context:

[0027]

[0028] Among them, Asorted represents the domain background relevance weight score.

[0029] 6. The patent recommendation method for LLM-driven semantic association expansion and structure completion according to any one of claims 1 to 4, wherein in step S300, feature propagation and aggregation include:

[0030] S301: For patent node p i , the embedding vector update at layer l depends on the information aggregation of neighboring patent nodes and associated enterprise nodes; let patent p i The patent neighbor node set is N p , the enterprise neighbor node set is N C Based on the patent-patent edge relationship, from the perspective of citation or similar technology, if patent p j ∈N p , its effect on P i The information contribution weight is not only related to the size of the neighboring nodes, but also to the context relevance score generated by the large language model. i With p j The large language model generates context semantic vectors, such as cosine similarity, and the edge weight normalization factor is:

[0031]

[0032] in It is the patent node p j The embedding vector at layer l, N C Represents the neighbor set of a patent.

[0033] Furthermore, in step S300, feature propagation and aggregation include:

[0034] S302: Based on the patent-enterprise edge relationship, the enterprise node acts as a promoter of patent knowledge commercialization. If enterprise C k ∈N C (Pi), Enterprise Node C k P i The information contribution at layer l is:

[0035]

[0036] in is the enterprise node c j The embedding vector at layer l, N P Represents the neighbor set of an enterprise.

[0037] Furthermore, in step S300, feature propagation and aggregation include:

[0038] S303: For patent node p i and enterprise node c jThe weighted summation of the embedding vectors of each layer is performed. In this paper, the cross-layer mean fusion method is used, and the final patent node p i The fusion feature vector is:

[0039]

[0040] where ω k is 1 / k, is the feature of patent i after cross-layer fusion, are the characteristics of enterprise j after cross-layer fusion.

[0041] Furthermore, in step S300, feature propagation and aggregation include:

[0042] S304: The patent subject text technical context under LLM is used as the query vector and the patent subject text technical context as the input of Key and Value to capture the dependency in the context environment for the patent subject technical representation:

[0043]

[0044]

[0045] in, is the context semantic feature vector; is the basic feature vector.

[0046] Furthermore, in step S300, feature propagation and aggregation include:

[0047] S305: Calculate the inner product between the patent node and the transfer object node, and then use the softmax function to convert it into the probability of interaction between the two, that is,

[0048]

[0049] Binary Cross Entropy Loss is used for edge 0-1 classification problems. The final optimization loss is the sum of the binary cross entropy loss function and the multi-relationship enhanced contrast loss.

[0050]

[0051] Where y represents the positive sample set of the interaction between the patent and the assigned enterprise, which comes from the historical interaction records between the patent and the assigned enterprise, y - is a set of negative samples of interaction between patents and assigned enterprises, which is a set of negative samples randomly drawn from the set of assigned enterprises where patents and assigned enterprises have never interacted. ui is the true label, is the predicted score and ξ is the regularization coefficient.

[0052] According to a second aspect of the present invention, a patent recommendation system for LLM-driven semantic association expansion and structure completion is provided, which is used to implement the patent recommendation method for LLM-driven semantic association expansion and structure completion as described in claims 1 to 9, comprising:

[0053] The context information enhancement module is used to summarize and complete the concepts of discrete patent attribute text based on LLM to generate background context information of the patent or enterprise, thereby enhancing the model's understanding of text semantics;

[0054] A heterogeneous network structure construction module is used to calculate the contextual relevance between patents and companies based on the contextual semantics generated by the large language model, select the top-k most relevant patents or companies based on the relevance ranking, construct background edge relationships in the same field, and complete the patent-patent and company-company relevance edges to obtain a heterogeneous network structure.

[0055] The patent recommendation prediction module is used to perform feature propagation and aggregation on the heterogeneous network structure based on the graph convolution model, aggregate high-order heterogeneous neighbor characteristics, optimize the recommendation relationship in the heterogeneous network structure, and further integrate the background context information with the heterogeneous network structure characteristics to predict the probability of occurrence of patent recommendation behavior.

[0056] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0057] 1. The method of the present invention summarizes and completes the discrete patent attribute text based on a large language model, generates patent context or enterprise domain background information, and enhances the model's understanding of text semantics. Secondly, the technical background correlations between patents and between enterprises are calculated respectively, and the correlation edges in the patent-patent network and the enterprise-enterprise network are completed, thereby improving the structural sparsity problem in heterogeneous data networks. Finally, a lightweight graph convolution model is adopted, relying on the patent-enterprise network structure to perform feature propagation and aggregation, optimize the recommendation relationship in the network, and integrate the domain background context features and structural embedding to predict patent recommendation behavior. Finally, through comparative analysis of baseline model experiments, it is found that the accuracy (ACC) of the method proposed in the present invention is improved by 8.28%, the precision (Precision) is improved by 1.73%, the recall rate (Recall) is improved by 12.31%, and the F1 score (F1-score) is improved by 6.23%.

[0058] 2. The method of the present invention uses a large language model to summarize scattered patent attribute texts and generate background context information of the patent or enterprise to enhance the model's ability to understand the text semantics.

[0059] 3. The method of the present invention, based on the contextual semantics generated by a large language model, analyzes the contextual correlation between patents and between enterprises, and completes the correlation edges between patents and enterprises to improve the diversity of neighborhood information and solve the structural sparsity problem in heterogeneous data networks.

[0060] 4. The method of the present invention adopts a lightweight graph convolutional model, relying on the patent-enterprise network structure, to perform high-order propagation and aggregation of features to optimize the recommendation relationship in complex heterogeneous networks.

[0061] 5. Our method integrates domain context and structural features, and calculates the recommendation probability between companies and patents based on historical behavioral interactions, enabling more accurate patent recommendations. Through extensive real-world data and experimental validation, our method significantly outperforms multiple baseline methods, confirming its effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 Schematic diagram of the patent recommendation process based on a large language model in an embodiment of the present invention;

[0063] Figure 2 This is a flowchart of a patent recommendation method based on LLM-driven semantic association expansion and structure completion according to an embodiment of the present invention;

[0064] Figure 3 Schematic diagram of the patent recommendation method framework based on LLM-driven semantic association expansion and structure completion according to an embodiment of the present invention;

[0065] Figure 4 A flowchart for generating active domain context based on a large language model in an embodiment of the present invention;

[0066] Figure 5 Verify the impact of the functional modules on the patent recommendation results in the embodiment of the present invention;

[0067] Figure 6 This is the first visualization result of the patent recommendation in the embodiment of the present invention;

[0068] Figure 7 This is the second visualization result of the patent recommendation in the embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0070] Example 1

[0071] This paper proposes a patent recommendation method that leverages LLM-driven semantic association expansion and structural completion. This method leverages existing enterprise-patent transfer relationships to explore potential patent transfer relationships. The input data for this study includes historical enterprise-patent transfer data and patent-related attribute text data. The output is patent recommendations that align with the enterprise's development value. In patent descriptions, attribute text constitutes a collection of descriptive or conceptual text within a specific technical field. LLM technology enables these dispersed attribute text features to be inferred and summarized into personalized patent representations with descriptive context, which is crucial for the enterprise-patent recommendation task. In this method, the application of LLM technology enables the formation of context-rich patent representations from multiple attribute texts, thus playing a key role in the recommendation system. This personalized representation not only enhances the model's understanding of patent text semantics but also improves the accuracy of enterprise-patent matching. Through this approach, the present invention effectively extracts valuable information from large amounts of patent attribute text and recommends patents that best meet the development needs of enterprises, thereby optimizing patent recommendation efficiency and innovation processes.

[0072] Specifically, if Figure 1 and Figure 2 As shown, the present invention first uses the reasoning ability of the large language model to perform conceptual induction and completion on discrete patent attribute texts to generate background context information of patents or enterprises, thereby enhancing the model's understanding of text semantics. Secondly, based on the context semantics generated by the large language model, the background context correlation between patents and enterprises is calculated, and the top-k most relevant patents or enterprises are selected according to the correlation sorting, and the background edge relationship in the same field is constructed to complete the patent-patent and enterprise-enterprise correlation edges, thereby improving the structural sparsity problem in heterogeneous patent data networks. Finally, a lightweight graph convolution model is used to perform feature propagation and aggregation in a heterogeneous network structure, aggregate high-order heterogeneous neighbor features, optimize the recommendation relationship in the network, and further integrate the context text and structural features to predict the occurrence of recommendation behavior. This method not only improves the understanding of the core technology of patents, but also provides a new perspective for patent recommendation.

[0073] Example 2

[0074] Step 1: Construction of patent heterogeneous information network.

[0075] Given a patent sample database, a patent recommendation heterogeneous graph network g = (v, ε, x) is constructed, and a patent recommendation framework that integrates large language model topic context generation and structure completion is constructed as follows: Figure 3 shown.

[0076] Constructing a patent heterogeneous information network is one of the key technologies in the field of patent information analysis. It reveals the internal connections between patents by integrating different categories of patent information. In the process of constructing a patent heterogeneous information network, the first task is to clarify the target entity type and its feature type of the network. For example, patents and inventors can be defined as target entity types, while various types of information in patent documents (such as the number of citations, the number of transfers of application rights, technical value, etc.) can be used as features. Figure 2 As shown in Figure (center), the construction of a patent heterogeneous graph network aims to depict the heterogeneous relationships between entities such as patents, companies, inventors, and applicants, thereby forming a patent representation embedding with strong correlation. In order to more accurately model the complex heterogeneous graph pattern of patent transfers, the present invention uses a large language model (LLM) to design an inductive context generation mechanism for complex attributed text to construct a contextual technical background representation of patents / companies. Furthermore, based on the patent subject technical context representation derived from LLM reasoning, the present invention further constructs a large language model-based structure completion to enrich the edge relationships in the heterogeneous network and alleviate the sparsity problem in the text.

[0077] Step 2: Generate technical context of patent attribute text based on LLM.

[0078] In the field of text feature mining, mainstream methods rely on advanced models such as BERT and Transformer to extract text features and then perform downstream tasks. Although the BERT model has demonstrated excellent learning capabilities on large-scale corpora, it still has representation limitations when dealing with discrete attribute short text representation tasks. In addition, short text information such as the name, title, domain name, and technical scope of patents / enterprises is discrete and complex and diverse. Directly applying the BERT embedding method is difficult to effectively capture the main technical structure in the text, resulting in poor application results. Therefore, the present invention designs patent attribute text technical context generation based on a large language model (LLM), aiming to solve the learning limitation problem caused by a large amount of attribute short text, and further optimize the sparsity problem between enterprises and patents, thereby improving the representation quality of patent subject texts.

[0079] Patent attribute text technical context generation method based on LLM: The large language model ChatGPT-3.5 is selected to explore the statistical distribution analysis of long texts with discrete attributes of patents / enterprises. After statistically analyzing the characteristics of patent / enterprise text attributes, a prompt word project is designed to enable the large model to learn the technical background, technical field and core area that the patent / enterprise is facing. Based on the principle of maximizing information entropy, the large model infers the geographical location area and main technical field type of the current patent / enterprise, and then generates the recommended context background under the enterprise-patent relationship. This helps to enrich the embedding representation of patents and enterprises. Therefore, for each patent / enterprise, the present invention can convert its text attributes into background text data containing active areas and main technical fields through LLM to describe it, thereby condensing the technical content of the patent. In order to have a deeper understanding of how text attributes are converted into background text data with active areas and main technical fields through LLM, the present invention uses Figure 3 The LLM reasoning prompting projects for patents and enterprises are presented in

[15] , and a brief explanation is provided. The three-step construction process of the prompting task output is shown in Table 1.

[0080] Table 1 Domain context prompt instruction content based on large language model

[0081]

[0082]

[0083]

[0084]

[0085] like Figure 3 As shown in the figure, the research method process of constructing the identity of ChatGPT-3.5 patent experts is as follows:

[0086] (1) First, the present invention establishes ChatGPT-3.5 as a patent domain expert. This step requires the present invention to set a specific scenario, allowing ChatGPT-3.5 to play the role of a professional natural language processing expert, focusing on extracting core knowledge from the multiple attribute texts of patents and summarizing key technical information.

[0087] (2) Secondly, in the context of ChatGPT-3.5 expert identity, the present invention will perform a small amount of prompt engineering annotation. This includes identifying key information from attribute texts such as patent titles, regions, and technical scopes. Then, using the powerful reasoning ability of large language models, the present invention combines these attribute texts through association relationships to form a contextual description of the active areas, technical scopes, and technical research fields that describe the core content of the patent. This prompt process requires experts to manually construct a small number of patent text samples, and design the patent attribute texts as questions (Q). The technical background and active fields that the experts manually summarize, reason, and conclude are used as answers A. Table 1 summarizes the prompt engineering content designed by the present invention, and Figure 3 The process of generating patent / enterprise domain background context based on patent / enterprise attribute text content is demonstrated.

[0088] (3) Finally, the present invention loops through the attribute text data list of each patent / enterprise into ChatGPT-3.5, and LLM outputs domain background context data based on the prompt engineering examples designed by experts. This means that the present invention will use the created prompt word instructions and prompt content, and ChatGPT-3.5 processes the new patent attribute text according to these instructions to generate technical background context output. This process involves inputting multiple patent attribute texts into a large language model, and expecting the model to be able to identify the key attribute information in the multiple attribute texts, and then summarize the above attribute text forms and output the technical background context. In this way, the present invention can train the model to identify and extract the core content in the patent attribute text, and convert it into context data with domain background for the model to learn enterprise and patent embedding representations.

[0089] Step 3: Structural completion based on the Large Language Model (LLM)

[0090] (1) Contextual representation generation based on BERT

[0091] This section uses BERT to embed the technical background context of patents / enterprises. Note that the patent technical context representation constructed by LLM can effectively reflect the active scope of patents / enterprises, which is not limited to the technical scope and regional scope, so as to further supplement the context features of patents / enterprises. This paper uses the BERT pre-trained model to capture the contextual relationship of the text to extract the context features of the patent representation. Since BERT is a variant of Transformer, d k The BERT model output dimension is represented by constructing a fixed-dimensional token vector and position encoding for patent / enterprise background word segmentation to determine the order of sentence context. Then, the multi-head self-attention mechanism is used to integrate the sentence context information, and the output is:

[0092]

[0093] Where Q, K, and V represent the query, key, and value matrices, respectively. Through a feedforward neural network, layer normalization, and residual connections, the embedded representation of the patent / enterprise background context is output, and the formula is as follows:

[0094] FFN(x)=max(0,xW1+b1)W2+b2,(2)

[0095]

[0096] Where FFN(x) represents the linear transformation of the long text features of the topic, Represents the final patent / enterprise background context features of node i after residual connection and layer normalization.

[0097] (2) Structural completion based on domain context embedding

[0098] Based on historical patent-enterprise transfer relationships and a large number of heterogeneous relationships such as applicants, this paper constructs a complex patent-enterprise network heterogeneous graph G, which contains enterprise information and patent information. However, due to the small number of transfers between enterprises and patents, the network structure is relatively sparse. To this end, this section proposes to evaluate the domain background correlation between patents and enterprises to complement the highly correlated patent and enterprise relationships, thereby alleviating the difficulties brought by network sparsity to patent recommendations, such as Figure 3 shown.

[0099] Specifically, the heterogeneous graph based on enterprise-patent transfers includes the transfer relationships between enterprises and patents, forming a complex heterogeneous network, where nodes represent enterprises or patents, and edges represent the transfer relationships between them. After obtaining the domain context embedding of patents / enterprises, the present invention uses cosine similarity to measure the contextual semantic similarity of nodes of the same type (patents / enterprises) to assess the closeness between patents. The calculation formula for cosine similarity is:

[0100]

[0101] in, and The embedding vectors of the domain context of two patents (the same applies to enterprises) are the sum of the dot products of 716-dimensional vectors, A i represents the set of relevance scores between patent i and other patent field context embeddings.

[0102] After obtaining the correlation between patents and companies, we normalize it and map the correlation to a range between 0 and 1. By calculating the cosine similarity of the contextual semantics, we select the top-k most relevant nodes and complete the edge relationship between the two nodes to achieve the cosine similarity top-k selection of the domain context.

[0103]

[0104] Asorted represents the weighted score of domain-specific relevance. Therefore, the selectable neighborhood range is the top k nodes with the strongest relevance among all nodes of the same type, which helps alleviate the sparsity between patents and companies and improves recommendation performance.

[0105] By completing the patent-patent graph edge structure relationship and enterprise-enterprise edge relationship, the original heterogeneous graph G is transformed into G′. Specifically, let the original heterogeneous graph G = (V, E), where the node set V includes the patent node set, P = {p1, p2, ..., p m} and enterprise node set C = {c1, c2, ..., c n}, that is, V=P∪C; the edge set E contains multiple types of edge relations. If patent p i In patent p j There is a high correlation between the domain backgrounds, so a line from p to i Point to p j Directed edges Among them E p is a subset of patent-patent edge relationships. By mining the domain background context, we can mine potential patent relevance, that is, expand E p This enriches the patent recommendation heterogeneous network to ensure that the knowledge flow paths between patents (or enterprises) are fully displayed.

[0106] The construction of enterprise-enterprise edge relationships relies on the enterprise domain context. If enterprise C s With Company C t If there is a high correlation, add an undirected edge to the graph Characterizing highly correlated companies, E c It is a subset of enterprise-enterprise edge relationships. The newly added edge relationships reflect the mutual connections between enterprises, which helps the propagation of features. After the completion and mining of the Topk structure of the domain background, E c , improve the network structure correlation between enterprises.

[0107] Through the above rigorous and meticulous patent-patent graph edge structure relationship completion operation and enterprise-enterprise edge relationship construction process, the original heterogeneous graph G is successfully transformed into a graph G′=(V, E′) with richer structure and greater information carrying capacity, where the edge set E′=E P ∪Ec , this conversion process is rigorously expressed with the help of mathematical symbols: for any patent pair (p i , p j ),have is the completed Top-k patent correlation edge; for (C s , C t ) and (C u , C v ),have

[0108] At this point, based on precise data mining and relationship building rules, the transformation from G to G′ is achieved, laying a solid foundation for subsequent research such as complex network analysis and patent recommendation modeling.

[0109] Step 4: Information propagation and fusion within the lightweight graph convolution layer

[0110] (1) Lightweight Graph Convolution

[0111] For patent node p i The initial feature vector consists of two parts: one is the contextual semantic feature vector generated by summarizing and summarizing the discrete patent attribute text using the large language model. The second is the basic feature vector extracted based on the inherent attributes of the patent itself (such as technical field, type, etc.), namely Similarly, enterprise node C j The initial eigenvector of They all conform to the standardized normal distribution.

[0112] For patent node p i , its embedding vector update at layer l depends on the information aggregation of neighboring patent nodes and associated enterprise nodes. Let patent p i The patent neighbor node set is N p , the enterprise neighbor node set is N C Based on the patent-patent edge relationship, from the perspective of citation or similar technology, if patent p j ∈N p , its effect on P i The information contribution weight is not only related to the size of the neighboring nodes, but also combined with the context relevance score generated by the large language model. i With p j The large language model generates context semantic vectors, such as cosine similarity, and the edge weight normalization factor is

[0113]

[0114] in It is the patent node p j The embedding vector at layer l, NC Represents the neighbor set of a patent.

[0115] Based on the patent-enterprise edge relationship, the enterprise node acts as a promoter of patent knowledge commercialization.

[0116] C k ∈N C (Pi), Enterprise Node C k P i The information contribution at layer l is:

[0117]

[0118] in is the enterprise node c j The embedding vector at layer l, N P Represents the neighbor set of an enterprise.

[0119] (2) Cross-layer information transmission and feature fusion

[0120] After k layers of LightGCN iterations, the output of each layer is used as the input of the next layer, and the multi-hop neighbor information is gradually integrated. Finally, in order to make patent recommendations, it is necessary to integrate the output features of each layer. A common way is to perform patent node p i and enterprise node c j The weighted summation of the embedding vectors of each layer is performed. Here, the present invention uses the cross-layer mean fusion method, and the final patent node p i The fusion feature vector is:

[0121]

[0122] where ω k is 1 / k, is the feature of patent i after cross-layer fusion, are the features of enterprise j after cross-layer fusion. Based on these fused feature vectors, recommendation models can be further constructed. For example, the matching score between patents and enterprises can be calculated based on inner product similarity to predict whether a recommendation will occur. This helps identify the most suitable enterprise for a target patent, enabling precise patent recommendations and effectively alleviating the recommendation challenges caused by insufficient attribute text utilization and data sparsity. Through the meticulous design of the three-layer Light GCN and its integration with heterogeneous relational networks, combined with the advantages of large language model preprocessing, deep mining of the heterogeneous patent-enterprise graph is achieved, significantly improving the accuracy of patent recommendations and facilitating the efficient transfer of patent technologies.

[0123] Step 5: Contextual Attention Mechanism Fusion

[0124] The present invention further supplements the patent context representation by embedding the patent technology main line through the multi-head attention mechanism. Therefore, the context-aware multi-head attention mechanism fusion module is as follows Figure 6 As shown, the input of this module is the embedded representation of the patent subject text. Next, the present invention designs a context-aware self-attention mechanism encoder that adaptively fuses the patent / company embeddings output by lightweight graph convolution with the domain context embedding. This encoder is composed of L stacked multi-head self-attention (MSA) and feed-forward network (FFN) blocks.

[0125] The patent subject text technical context under LLM is used as the query vector and the patent subject text technical context as the input of key and value, aiming to capture the dependency under the context environment for patent subject technical representation. The calculation process is as follows:

[0126]

[0127] Step 6: Loss function construction

[0128] First, the present invention calculates the inner product between the patent node and the transfer object node, and then uses the softmax function to convert it into the probability of interaction between the two, that is,

[0129]

[0130] Binary Cross Entropy Loss is used for edge 0-1 classification problems. The final optimization loss is the sum of the binary cross entropy loss and the multi-relationship enhanced contrast loss.

[0131]

[0132] Where y represents the positive sample set of the interaction between the patent and the assigned enterprise, which comes from the historical interaction records between the patent and the assigned enterprise, y - is a set of negative samples of interaction between patents and assigned enterprises, which is a set of negative samples randomly drawn from the set of assigned enterprises where patents and assigned enterprises have never interacted.

[0133] r ui is the true label, is the predicted score and ξ is the regularization coefficient.

[0134] Example 3

[0135] Step 1: Build a dataset

[0136] To achieve this goal, we selected 39 985 universities in China as data samples and employed experts in the field of patent technology transfer to develop a university patent transfer search strategy. On July 1, 2024, we searched the patent database HimmPat, obtaining a total of 1.12 million patent data. After data processing such as deduplication, denoising, and indexing, we randomly selected 7,830 patent transfer data items from Section B, 11,042 from Section C, 7,081 from Section D, and 40,665 from Section E, totaling 66,618 patent transfer data items, as training and testing datasets.

[0137] Constructing a scientific and reasonable evaluation index system is the key to achieving patent transformation prediction. In order to comprehensively evaluate the transformation potential and value of patents, the present invention refers to the research of Yi Huifang and Wu Hong

[46] , adopts the fuzzy set qualitative comparative analysis method, and explores the multivariate equivalent path of university patent transformation based on the whole process of patent transformation. In addition, the research of Zheng Wan

[47] also provides the present invention with a patent recommendation technology based on intelligent matching, which can help the present invention better understand and annotate patent data.

[0138] During the annotation process, the present invention uses fuzzy set theory to process qualitative data and converts qualitative evaluations into quantitative data for subsequent analysis and model building. For example, for the evaluation of market potential, the present invention can set a fuzzy set from "low" to "high" to convert the expert's qualitative evaluation into a numerical value between 0 and 1. The present invention adopts the multidimensional value indicator combination shown in Table 2, and uses heterogeneous graph neural networks combined with advanced large language models (LLMs) to model the enterprise-patent heterogeneous information network, thereby realizing the patent transfer recommendation task. In order to realize the encoding of patent features, the present invention uses the query-based approach of the Pytorch framework based on the above-mentioned 27 secondary indicators and 5 types of node relationships to construct the initial vectors of 27 different indicators, and averages and normalizes the 27 initial vectors corresponding to each patent to embed the initial representation of the patent.

[0139] Table 2 Patent recommended heterogeneous network characteristic indicators

[0140]

[0141]

[0142]

[0143] In the semi-supervised patent transfer prediction task, the present invention randomly shuffled the dataset and divided it into the following proportions: 80% for training and 20% for testing. In order to verify the effectiveness of the method proposed in the present invention, the present invention collected patent datasets of various types in various fields, such as authoritative datasets: 985 University B Department Patent Transfer and License Dataset, 985 University C Department Patent Transfer and License Dataset, 985 University D Department Patent Transfer and License Dataset, and 985 University E Department Patent Transfer and License Dataset. In the above-mentioned multi-field authoritative datasets, 35 patent-related features were considered, including patent title, assignee unit name, total number of transfers, number of applicants, patent score, number of citations, number of licenses, legal value, technical value, market value, and strategic value, etc. 35 indicators. Here, the present invention further explains the patent characteristics:

[0144] In the semi-supervised patent transfer prediction task, the present invention randomly shuffles the dataset and divides it into 80% for training and 20% for testing. To verify the effectiveness of the proposed method, the present invention collects patent datasets of various fields, such as the authoritative datasets: the patent transfer and licensing dataset of the 985 University H Department, the patent transfer and licensing dataset of the 985 University F Department, the patent transfer and licensing dataset of the 985 University A Department, and the patent transfer and licensing dataset of the 985 University G Department. In these multi-field authoritative datasets, 27 patent-related features are considered, including the number of IPC classifications, the number of citations, the number of licenses, the strategic emerging industry classification, and the patent-intensive industry classification.

[0145] Table 3 Statistical details of patent recommendation dataset information

[0146]

[0147]

[0148]

[0149] Step 2: Model Evaluation

[0150] To improve the training and generalization capabilities of the patent transfer prediction model, the present invention optimizes the parameter values ​​within each model. During each training session, the present invention randomly selects 80% for training and 20% for testing, splitting the dataset and randomly collecting negative samples to accelerate the model training process. Furthermore, the present invention uses accuracy, precision, and recall to measure model performance. Because accuracy and recall influence each other, the present invention incorporates the F1 score into these two metrics.

[0151] Step 3: Experimental Setup

[0152] (1) Baseline model

[0153] We compare the proposed GAPNRec with seven deep learning baseline models using the same experimental environment as ours, and perform detailed parameter optimization. These include Graph Convolutional Networks (GCN), Graph Sample and Aggregation Neural Networks (GraphSAGE), Graph Attention Networks (GAT), MixHop Graph Neural Networks (MixHop), Simple Graph Convolution Neural Networks (SGC), and Lightweight Graph Convolutional Networks (LightGCN). The baseline models are summarized below:

[0154] GCN: A graph neural network that learns node embedding representations by applying convolution operations on a graph. It leverages the local connectivity patterns of the graph to aggregate features of neighboring nodes, thereby updating the feature representation of each node.

[0155] GraphSAGE: A flexible graph neural network framework designed to efficiently learn features for nodes in graph data. It updates the embedding representation of each node by sampling and aggregating features from neighboring nodes.

[0156] GAT: Introduces the attention mechanism, allowing the model to dynamically assign different weights to different nodes when processing graph data.

[0157] LightGCN: A lightweight graph convolutional network that discards linear transformations and activation functions based on GCN to quickly learn node embedding representations

[0158] MF: Based on matrix decomposition and convolution, it processes sparse data to mine potential features, capture local features, and improve node representation learning or prediction capabilities.

[0159] SGC: A simple graph convolution-based method that simplifies traditional graph convolution calculations, reduces training computations, and can quickly obtain effective node features in restricted scenarios.

[0160] MixHop: A method based on the MixHop architecture that mixes neighbor information of different orders to learn node representations, captures graph structures at multiple scales, and comprehensively characterizes node relationships.

[0161] (2) Basic experiments

[0162] Given that different comparison models each have their own unique advantages and limitations in patent data processing and patent transfer prediction, we carefully selected the following baseline models for comparison. By analyzing and comparing these models, we aim to evaluate the performance and efficiency of our proposed method XXX in patent transfer prediction.

[0163] Table 4 Comparison of baseline models

[0164]

[0165] (3) Model parameters

[0166] The present invention successfully implemented the proposed GAPNRec model using the Pytorch framework, and carefully adjusted the key parameters to optimize its performance. Specifically, the Adam optimizer was used to iteratively optimize all model parameters, and the batch size was set to 1024, the learning rate was set to 0.001, and RTX4090 was used as the training device. Experimental verification was carried out on multiple patented heterogeneous datasets such as science and technology and life, and the embedding dimension of the model input and output was set to 64. In the experimental verification part, the impact of the number of layers of lightweight graph convolution on the model recommendation performance was explored. In addition, the value of the important coefficient λ in the contrastive loss function was adjusted to observe its impact on the model performance, and its range of variation was from 0.1 to 0.9. In order to ensure the fairness of the experiment, the Baseline model was tested under the same experimental conditions to ensure that the batch size and learning rate were consistent with the GAPNRec model.

[0167] (4) Model performance comparison and analysis

[0168] We evaluated the proposed GAPNRec model and compared its performance with baseline models on multiple patent transfer application domain datasets, including those from 985 University H, 985 University F, 985 University A, 985 University A, and 985 University G. Table 5 summarizes the performance of 10 baseline models based on multidimensional patent metrics on four key evaluation metrics: accuracy, precision, recall, and F1 score.

[0169] Table 5 Performance of GAPNRec model and baseline model on patent datasets in different fields

[0170]

[0171] The analysis results in Table 5 demonstrate that the patent recommendation model proposed in this paper demonstrates superior performance across multiple key performance indicators, achieving significant improvements in F1-score, accuracy (Acc), precision (Precision), and recall (Recall) compared to the baseline model. These results fully demonstrate the effectiveness of the patent recommendation framework proposed in this paper, which uses a large language model to generate topic context and complete structure. Specifically, in terms of F1-score, the proposed model achieved an average improvement of approximately 7 percentage points compared to the baseline model, with particularly strong performance when handling complex scenarios. In terms of accuracy, the proposed model also performed well, with an average improvement of approximately 7-8 percentage points, demonstrating its higher accuracy when recommending patents to businesses. The proposed model also achieved significant improvements in precision and recall. This improvement in precision means that the proposed model recommends a higher proportion of patents that are actually transferred, reducing false positives. The improvement in recall indicates that the proposed model more comprehensively covers all patents that match businesses, reducing missed positives.

[0172] Secondly, this paper further analyzes the experimental results to reveal the advantages and principles of the proposed model. First, graph neural networks such as GCN, GraphSAGE, and LightGCN can effectively propagate and aggregate information across heterogeneous patent graph structures, improving the receptive field between patents and companies and generating high-quality heterogeneous embedding representations of companies and patents. Due to its simplified structure, the SGC model has limited ability to handle complex relationships, resulting in poor performance on multiple metrics. Furthermore, these models have limitations in handling complex, heterogeneous types and long text within patents and struggle to address the sparsity issues of heterogeneous patent recommendation networks. GAPNRec achieves optimal experimental results in terms of accuracy, precision, recall, and F1-score. This paper successfully combines the latest large-scale model (LLM) with heterogeneous patent-company networks to construct domain context for patents and companies, enriching the patent embedding representation process and achieving excellent performance across multiple different patent technology fields, providing a new solution for patent recommendation research. After rigorous empirical analysis and validation, the proposed method demonstrates significant advantages in patent recommendation. Compared with traditional patent recommendation methods, this approach effectively improves the accuracy and stability of patent recommendations. Experimental results demonstrate that this approach, which integrates a large language model with a heterogeneous patent-enterprise network, is highly reliable in terms of commercial transformation and technological impact. It provides strong support for flexible patent recommendations for enterprises and promotes the sustainable development of patent technology transfer and innovation.

[0173] Step 4: Impact of different functional modules on patent transfer prediction results (ablation experiment)

[0174] This section analyzes the impact of different functional modules on patent recommendation results. To verify the experimental effectiveness of the contextual summarization of the large language model, the structure completion based on the large language model, and the lightweight graph convolution module designed by this invention, this section conducts experimental verification and analysis of three different modules. The corresponding modules are removed in each of these modules to verify the effectiveness of the modules designed by this invention in improving model performance.

[0175] The analysis results in Table 6 reveal the outstanding performance of the patent recommendation model designed by the present invention in key evaluation indicators such as accuracy, precision, recall and F1-score on data sets from different university departments. Specifically, whether on the data sets of H, F, A or G departments of 985 universities, the model of the present invention achieved the highest accuracy and F1-score, demonstrating the high comprehensive performance of the model in predicting patent transfers. In addition, the model of the present invention also surpassed other functional modules in terms of precision and recall, further confirming its advantages in reducing misjudgments and missed judgments. This advantage may be due to the high independence and importance of GAPNRec in patent documents, which makes the model perform better in various indicators in most cases. Whether it is accuracy, precision, recall or comprehensive F1-score, GAPNRec has demonstrated good performance, and can more accurately and comprehensively mine patents related to corporate needs, and the overall recommendation effect is relatively ideal. When the lightweight graph convolution model (LGC) module is removed, the performance indicators of the model drop significantly, which indicates that the lack of heterogeneous graph structure relationships will limit the overall performance of the model, thereby highlighting the key role of graph structure in accurately recommending patents. The context learning (LLM) and heterogeneous graph structure completion (HGSC) modules based on the large language model perform differently in different indicators. Although they have shown some performance in terms of precision, they also have deficiencies in terms of accuracy and recall, which affects the overall patent recommendation effect. Therefore, different functional modules have a significant impact on the patent recommendation results. In practical applications, appropriate functional modules can be selected according to specific needs and the degree of importance attached to different indicators, or each module can be further optimized to improve the overall performance of the patent recommendation system. The present invention verifies the effectiveness of the model through empirical analysis and provides a new solution for the field of patent recommendation.

[0176] Step 5: The impact of the number of lightweight graph convolution layers on patent recommendation results (ablation experiment)

[0177] Table 7 Impact of the number of lightweight graph convolution layers on recommendation performance

[0178]

[0179]

[0180] Table 7 shows the impact of the number of lightweight graph convolution layers on patent recommendation performance. The analysis results show that increasing the number of graph convolution layers can, in most cases, improve various performance metrics of the recommendation system, including accuracy, precision, recall, and F1-score. Specifically, as the number of layers increases, these metrics generally improve initially and then stabilize or slightly decrease, indicating that optimal recommendation performance is achieved around two layers. This phenomenon may be because an appropriate number of layers helps the model more deeply explore feature relationships in the data, thereby improving recommendation accuracy and comprehensiveness. However, an excessive number of layers may lead to overfitting, negatively impacting recommendation performance. The optimal number of layers and the magnitude of changes in metrics at each layer vary depending on the data characteristics and differences between different university departments, but the overall trends are similar. For example, on the dataset from the H department of a 985 university, increasing the number of layers from 0 to 2 resulted in improvements in all metrics, while increasing to three layers resulted in a slight decrease in accuracy and F1-score. This trend is also reflected in datasets from other departments. Therefore, choosing the right number of lightweight graph convolution layers based on the technical characteristics of patent data and the different priorities for recommendation performance metrics is crucial for improving patent recommendation effectiveness. By carefully adjusting the model structure, we can optimize the model's performance on specific datasets and achieve more accurate patent recommendations.

[0181] Example 4

[0182] Figure 6 A visualization chart for predicting patent recommendation structures shows the correlations between different patents, their characteristics, and recommendations. The chart includes information such as the applicant, inventor, patent title, technical field, patent type, preliminary translations of the IPC (International Patent Classification) and CPC (Cooperative Patent Classification), and technical effects. In patent recommendation systems, the technical field and the target audience are two key factors that directly influence the relevance and accuracy of recommendations. Figure 5 (Part B) shows the recommendation probability relationship between enterprises and patents. For example, the patents matched by Beijing Yutong Technology Co., Ltd. are CN103769997B (a multi-axis linkage processing machine tool based on a rectangular array synchronous stroking mechanism) and patent CN103600128B (a method and device for processing the inner surface of a workpiece using a ring-shaped detachable tool). These patents all involve the processing and manufacturing of mechanical workpieces. The recommended patent CN103722432B (a differential screw mechanism and a multi-axis machine tool based on a differential screw mechanism) also meets the needs of the enterprise and may have a high matching degree, so the recommendation probability is as high as 0.83. If the matching degree between the enterprise and the patent is low, such as Figure 7(Section C) Beijing Natural Sky Technology Development Co., Ltd. specializes in metal smelting and alloy fabrication. The model analyzes the domain contextual information generated by the large language model between the company and patent CN104864067 (a hierarchical face gear and its implementation method), as well as the patent technology neighborhood in which the company is active. The model infers that the correlation between the company and the technology-related fields of patent CN104864067 is too large to meet the company's development needs. Therefore, the recommendation probability for the company is 0.16. Patents are more likely to be recommended if the company's target market is highly correlated with the patent's technical field. For example, a company focused on renewable energy may be interested in patents related to "the use of natural energy sources, such as solar energy and underground energy," as these technologies are directly related to the company's business area. Patents whose technical fields are unrelated to the company's primary business area may not be recommended. For example, a company focused on mobile communications may not be interested in patents related to solar energy technology. Therefore, the patent recommendation system comprehensively considers factors such as the patent's technical field, innovativeness, relevance, the company's target market, technical needs, market positioning, as well as the patent's legal status and commercial potential to determine whether to recommend a specific patent.

[0183] Current research in the field of patent recommendation faces significant challenges, primarily due to the complexity of the patent information landscape and the sparsity of heterogeneous data structures. These factors severely limit the performance of recommendation systems. To address this issue, this paper innovatively proposes a novel patent recommendation method, GAPNRec, which combines domain context generation from a large language model (LLM) with graph structure completion technology. This method aims to overcome bottlenecks in existing recommendation research and reconstruct the patent recommendation graph structure. Specifically, it leverages the advanced reasoning capabilities of a large language model to conduct in-depth analysis of patent attribute text. Through precise concept induction and relational reasoning, it transforms scattered and disorganized patent and company text into logically coherent and rich contextual information. This process provides a solid foundation for the model to accurately parse text semantics, significantly improving the depth of semantic understanding. Second, based on the high-quality contextual semantics generated by the large language model, sophisticated computational methods are used to assess the correlations between patents and companies. This allows for precise edge completion in heterogeneous data networks, effectively addressing the problem of structural sparsity, enhancing the integrity of the data network, and optimizing inter-data correlations. Finally, a lightweight graph convolutional network is introduced, which follows the structure of the patent-enterprise network and realizes the propagation and aggregation of features between high-order ranges to optimize the recommendation relationship. In this process, contextual features are combined with structural features, and through multi-dimensional feature fusion, the model's prediction accuracy of recommendation behavior is improved, achieving precise recommendations. It is worth noting that the GAPNRec method has demonstrated its ability to efficiently integrate complex patent text information, sparse relationships in historical transfer structures, and large language model technology in actual application scenarios. Compared with traditional patent recommendation methods, after rigorous experimental verification and comparative analysis, GAPNRec has shown significant advantages in key evaluation indicators such as accuracy, precision, recall rate, and F1 value, achieving a significant improvement in recommendation performance. This method provides a new direction for the development of patent recommendation technology and is expected to promote the development of this field to a higher level.

[0184] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A patent recommendation method based on LLM-driven semantic association expansion and structure completion, characterized by: include: S100: Based on LLM, the concept of discrete patent attribute text is summarized and completed to generate background context information of the patent or enterprise, and enhance the model's understanding of text semantics; S200: Based on the contextual semantic representation generated by the large language model, the contextual relevance between patents and companies is calculated. The top-k most relevant patents or companies are selected based on the relevance ranking, and the contextual edge relationships in the same field are constructed. The patent-patent and company-company relevance edges are completed to obtain a heterogeneous network structure. S300: Based on the graph convolution model, feature propagation and aggregation are performed on the heterogeneous network structure, high-order heterogeneous neighbor characteristics are aggregated, the recommendation relationship in the heterogeneous network structure is optimized, and the background context information and the structure characteristics of the heterogeneous network are further integrated to predict the probability of occurrence of patent recommendation behavior.

2. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 1 is characterized in that: In step S100, the generation of background context information of the patent or enterprise includes: S201: Use the BERT pre-trained model to capture the contextual relationship of the text, and construct a fixed-dimensional token vector and position encoding for the patent / enterprise background context segmentation to determine the order of the sentence context; S202: Use the multi-head self-attention mechanism to fuse the sentence context information, and its output is: where Q, K and V represent query, key and value matrices respectively, d k Represents the dimension of the BERT model output; S203: Output the embedded representation of the patent / enterprise background context through a feedforward neural network, layer normalization, and residual connections: FFN(x)=max(0,χW1+b1)W2+b2,(2) Where FFN(x) represents the linear transformation of the long text features of the topic, represents the background context information of the patent or enterprise after node i is residually connected and layer normalized, w1 and w2 represent weights, and b1 and b2 represent constants.

3. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 2 is characterized in that: In step S200, the heterogeneous network structure includes: S201: The heterogeneous graph based on enterprise-patent transfer includes the transfer relationships between enterprises and patents, forming a complex heterogeneous relationship network, where nodes represent enterprises or patents and edges represent the transfer relationships between them; S202: Cosine similarity is used to measure the contextual semantic similarity of nodes of the same type to evaluate the closeness between patents. The calculation formula of cosine similarity is: in, and The embedding vectors of the context of the two patent fields are the sum of the dot products of the 716-dimensional vectors, A i represents the set of relevance scores between patent i and other patent field context embeddings.

4. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 3 is characterized in that: In step S200, the heterogeneous network structure includes: S203: By calculating the cosine similarity of the contextual semantics, the most relevant Top-k nodes are selected and the edge relationship between the two nodes is completed to achieve the cosine similarity Top-k selection of the domain context: Among them, Asorted represents the domain background relevance weight score.

5. A patent recommendation method for LLM-driven semantic association expansion and structure completion according to any one of claims 1-4, characterized in that: In step S300, feature propagation and aggregation include: S301: For patent node p i , the embedding vector update at layer l depends on the information aggregation of neighboring patent nodes and associated enterprise nodes; let patent p i The patent neighbor node set is N p , the enterprise neighbor node set is N C Based on the patent-patent edge relationship, from the perspective of citation or similar technology, if patent p j ∈N p , its effect on P i The information contribution weight is not only related to the size of the neighboring nodes, but also to the context relevance score generated by the large language model. i With p j The large language model generates context semantic vectors, such as cosine similarity, and the edge weight normalization factor is: in It is the patent node p j The embedding vector at layer l, N C Represents the neighbor set of a patent.

6. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 5 is characterized in that In step S300, feature propagation and aggregation include: S302: Based on the patent-enterprise edge relationship, the enterprise node acts as a promoter of patent knowledge commercialization. If enterprise C k ∈N C (Pi), Enterprise Node C k P i The information contribution at layer l is: in is the enterprise node c j The embedding vector at layer l, N P Represents the neighbor set of an enterprise.

7. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 6 is characterized in that: In step S300, feature propagation and aggregation include: S303: For patent node p i and enterprise node c j The weighted summation of the embedding vectors of each layer is performed. In this paper, the cross-layer mean fusion method is used, and the final patent node p i The fusion feature vector is: where ω k is 1 / k, is the feature of patent i after cross-layer fusion, are the characteristics of enterprise j after cross-layer fusion.

8. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 7 is characterized in that In step S300, feature propagation and aggregation include: S304: The patent subject text technical context under LLM is used as the query vector and the patent subject text technical context as the input of Key and Value to capture the dependency in the context environment for the patent subject technical representation: in, is the context semantic feature vector; is the basic feature vector.

9. The patent recommendation method of LLM-driven semantic association expansion and structure completion according to claim 8 is characterized in that In step S300, feature propagation and aggregation include: S305: Calculate the inner product between the patent node and the transfer object node, and then use the softmax function to convert it into the probability of interaction between the two, that is, Binary Cross Entropy Loss is used for edge 0-1 classification problems. The final optimization loss is the sum of the binary cross entropy loss function and the multi-relationship enhanced contrast loss. Where y represents the positive sample set of the interaction between the patent and the assigned enterprise, which comes from the historical interaction records between the patent and the assigned enterprise, y - is a set of negative samples of interaction between patents and assigned enterprises, which is a set of negative samples randomly drawn from the set of assigned enterprises where patents and assigned enterprises have never interacted. r ui is the true label, is the predicted score and ξ is the regularization coefficient.

10. A patent recommendation system driven by LLM for semantic association expansion and structural completion, characterized by: The patent recommendation method for implementing the LLM-driven semantic association expansion and structure completion as described in claims 1-9 includes: The context information enhancement module is used to summarize and complete the concepts of discrete patent attribute text based on LLM to generate background context information of the patent or enterprise, thereby enhancing the model's understanding of text semantics; A heterogeneous network structure construction module is used to calculate the contextual relevance between patents and companies based on the contextual semantics generated by the large language model, select the top-k most relevant patents or companies based on the relevance ranking, construct background edge relationships in the same field, and complete the patent-patent and company-company relevance edges to obtain a heterogeneous network structure. The patent recommendation prediction module is used to perform feature propagation and aggregation on the heterogeneous network structure based on the graph convolution model, aggregate high-order heterogeneous neighbor characteristics, optimize the recommendation relationship in the heterogeneous network structure, and further integrate the background context information with the heterogeneous network structure characteristics to predict the probability of occurrence of patent recommendation behavior.

Images