Patent recommendation method and system based on multi-modal heterogeneous graph attribute aggregation

Abstract

The invention discloses a patent recommendation method and system based on multi-modal heterogeneous graph attribute aggregation, and relates to the technical field of patent information services, and the method comprises the steps: obtaining multi-modal data of a patent, carrying out the feature fusion, and generating a multi-modal representation vector; constructing a heterogeneous graph containing various types of nodes, taking multi-modal representation as patent node features, and calling a large language model to infer and generate implicit technology association edges with weights so as to enhance the graph; according to a user recommendation demand, quantifying a precision requirement by using a Fisher information matrix, dynamically determining an aggregation order and range, executing hierarchical attribute aggregation on a target node in the heterogeneous graph, and generating a demand-adaptive patent representation; and finally, user preference representation is constructed based on user historical interaction behaviors, and a personalized patent recommendation list is generated through matching. According to the method, patent deep semantic association mining, demand-adaptive dynamic recommendation and accurate personalized matching are realized, and the accuracy, flexibility and interpretability of patent recommendation are improved.

Description

Technical Field

[0001] This invention relates to the field of patent information service technology, and more specifically, to a patent recommendation method and system based on multimodal heterogeneous graph attribute aggregation. Background Technology

[0002] The patent recommendation method based on multimodal heterogeneous graph attribute aggregation was proposed against the backdrop of numerous challenges facing current patent information retrieval and recommendation technologies. With the accelerating pace of global technological innovation, the number of patents has exploded, and patent data itself is characterized by multimodality, complex structure, and rich semantics, encompassing various information formats such as text, images, classification numbers, applicants, and citation networks. However, most existing patent recommendation systems still rely on traditional information retrieval and recommendation technologies, exhibiting significant limitations.

[0003] Currently, mainstream patent recommendation solutions mainly include keyword matching-based retrieval systems, citation network-based analysis methods, classification-based recommendations, and collaborative filtering techniques. Keyword matching methods typically use algorithms such as TF-IDF and BM25 to calculate text similarity, but are often limited to literal matching and struggle to deeply understand the technical semantics and innovative connotations behind the patents. Citation network analysis constructs a technology association graph through the citation relationships between patents, but patent citation relationships are usually sparse and lagging, making it difficult to fully reflect the deep connections between technologies. Methods based on the International Patent Classification (IPC) or the Joint Patent Classification (CPC), while able to organize technical fields at a macro level, have a coarse classification granularity and insufficient support for subdivided technologies or cross-domain technology integration scenarios. Collaborative filtering methods rely on user-patent interaction history, are easily constrained by data sparsity and cold start problems, and struggle to explain the technical basis of the recommendation results.

[0004] The shortcomings of existing technologies are mainly reflected in the following aspects: First, most systems fail to fully integrate and utilize the multimodal information of patents, especially the rich technical content contained in visual data such as technical drawings, chemical structural formulas, and circuit diagrams, resulting in incomplete information utilization. Second, the semantic understanding of patent technical content is relatively superficial, lacking in-depth analysis and correlation mining of technical principles, innovation points, and application scenarios. Third, existing methods mostly rely on the structured relationships between patents, lacking the ability to automatically discover semantic-level technical connections. Furthermore, the flexibility and personalization of the recommendation process are limited, typically providing a fixed range of recommendation results, unable to dynamically adjust the breadth and depth of recommendations according to the user's specific needs. Finally, how to effectively combine users' historical behavior and real-time intent to achieve truly personalized patent recommendations remains a common challenge faced by current systems. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a patent recommendation method and system based on multimodal heterogeneous graph attribute aggregation. This effectively solves the problems of insufficient utilization of multimodal information, shallow semantic understanding, rigid recommendation granularity, and limited personalization in traditional patent recommendation systems, thereby improving the accuracy, flexibility, and interpretability of patent recommendations.

[0006] The first aspect of this invention provides a patent recommendation method based on multimodal heterogeneous graph attribute aggregation, comprising the following steps: Multimodal data of patents are obtained from the patent database, and features are extracted and fused from the multimodal data to generate a multimodal representation vector for each patent. A heterogeneous graph containing patent nodes, technical keyword nodes, applicant nodes, inventor nodes, and classification nodes is constructed, and the multimodal representation vector is used as the initial feature of the patent nodes; a large language model is invoked to reason about the technical semantic relationships between patent nodes, generating implicit technical association edges containing relationship types and confidence weights to enhance the heterogeneous graph; Based on the user's recommendation needs, the accuracy requirements of the recommendation needs are quantified using the Fisher information matrix. Based on the accuracy requirements, the order and range of attribute aggregation are dynamically determined. Hierarchical attribute aggregation operation is performed on the target patent nodes in the heterogeneous graph to generate a patent representation vector adapted to the requirements. A user preference representation is constructed based on the user's historical interaction behavior. The patent representation vector is then matched with the user preference representation to generate and output a personalized patent recommendation list.

[0007] In this scheme, multimodal data of patents are obtained from a patent database, and feature extraction and fusion are performed on the multimodal data to generate a multimodal representation vector for each patent, including: The process involves obtaining text data, image data, and structured data of relevant patents from a patent database; segmenting the text data into paragraphs and identifying technical chapters; detecting key regions and extracting structural information from the image data; and performing entity alignment and normalization on the structured data. The title and abstract in the preprocessed text data are encoded using a language model pre-trained with patent corpus to obtain a technical topic vector. For the full text of the specification, segmented attention encoding and chapter-level feature aggregation are used to obtain a long text structure vector. Based on the technical topic vector and the long text structure vector, a text representation vector is output. The technical figures in the preprocessed image data are used to extract global and local visual features using a visual Transformer. The structured image is then fused with visual and topological features using a graph neural network to obtain an image semantic vector. Based on the global and local visual features and the image semantic vector, an image representation vector is output. Heterogeneous graph node embedding is used to obtain entity vectors for applicants and inventors in preprocessed structured data, and hierarchical coding network is used to obtain classification hierarchy vectors for IPC classification numbers. Based on the entity vectors and classification hierarchy vectors, structured representation vectors are output. By calculating the semantic association between text representation vectors and image representation vectors through cross-attention, an intermodal multi-head attention network is constructed. The complementary weights between text representation vectors, image representation vectors, and structured representation vectors are dynamically learned to generate a fused comprehensive feature vector. The comprehensive feature vector is then mapped to a multimodal representation vector through a patented semantic space projection layer.

[0008] In this scheme, a heterogeneous graph is constructed, including patent nodes, technical keyword nodes, applicant nodes, inventor nodes, and classification nodes. The multimodal representation vector is used as the initial feature of the patent nodes, including: Define a heterogeneous graph node type that includes patent nodes, technical keyword nodes, applicant nodes, inventor nodes and classification nodes, and use the multimodal representation vector as the initial feature of the patent node; Based on patent metadata, predefined explicit relationship edges are established between nodes, including content ownership relationship, ownership and creation relationship, legal and family relationship, and technology diffusion relationship, and weights based on metadata statistics are assigned to the explicit relationship edges. A dynamic incremental construction strategy is adopted to dynamically insert new patent nodes and related nodes into the heterogeneous graph. The similarity is calculated by using the multimodal representation vector of the patent nodes, providing similarity priors for implicit semantic relationship reasoning based on a large language model, and the structure of the heterogeneous graph is used as contextual information for relationship reasoning.

[0009] In this solution, a large language model is invoked to infer the technical semantic relationships between patent nodes, generating implicit technical association edges that include relationship types and confidence weights, thereby enhancing the heterogeneous graph, including: For the patent node degree to be analyzed, obtain the core content summary of the patent corresponding to the two patent nodes and the prior information of the local graph structure extracted from the heterogeneous graph, and construct the structured context information for relational reasoning. Based on the aforementioned context information, prompt instructions are constructed, and a comparative analysis is conducted from four dimensions: similarity of technical principles, overlap of application scenarios, complementarity of technologies, and relationship of technological evolution. Based on the results of the comparative analysis, the instruction big language model determines whether there are predefined types of technical associations, including technical substitution relationships, technical complementarity relationships, technical evolution relationships, and technical migration relationships, and outputs a confidence score for each type of technical association that is determined to exist. The large language model determines the existing technical relationships, constructs implicit technical association edges with relationship type labels between the corresponding two patent nodes, and uses the confidence score as the initial weight of the implicit technical association edges to perform semantic enhancement on the heterogeneous graph.

[0010] In this solution, based on the user's recommendation needs, the Fisher information matrix is ​​used to quantify the accuracy requirements of the recommendation needs, including: The system obtains the user's recommendation needs, performs semantic parsing on the recommendation needs, and extracts features of preset semantic dimensions, including technology focus, scope openness, and domain cross-domain tolerance. Based on the parsing results, it generates a demand representation vector of the recommendation needs. The recommendation process is modeled as an information processing model. The model parameters of the information processing model include the order of attribute aggregation, distance decay coefficient, and meta-path selection weight. The optimal combination of model parameters is expected to correspond to different recommendation needs. The output of the information processing model is the conditional probability distribution of the recommendation results. Based on the optimal model parameter combination, the Fisher information matrix of the information processing model is calculated; scalar metric values ​​are extracted from the Fisher information matrix as the quantitative recommendation accuracy requirements of the recommendation needs.

[0011] In this scheme, based on the aforementioned accuracy requirements, the order and range of attribute aggregation are dynamically determined, and a hierarchical attribute aggregation operation is performed on the target patent nodes in the heterogeneous graph to generate a demand-adaptive patent representation vector, including: Based on the quantitative recommendation accuracy requirements, the maximum aggregation order, distance decay coefficient, enabled heterogeneous meta-path set, and corresponding meta-path weight vector of attribute aggregation are determined through a predefined decision mapping. Starting from the target patent node, a meta-path-based traversal is performed in the heterogeneous graph according to the maximum aggregation order and the enabled set of heterogeneous meta-paths to sample and obtain the set of heterogeneous multi-hop neighbor nodes of the target node. For the neighboring nodes, an aggregation weight is calculated based on the hop distance to the target node, the type of the meta-path, and the semantic attention score between the nodes. This aggregation weight incorporates a distance decay factor and a meta-path weight vector. Represented as: , in Represents the target node A heterogeneous multi-hop neighbor node set Indicates neighboring nodes, , This represents the unnormalized weight score. The path weight vector represents the meta-path. Indicated based on distance attenuation coefficient The distance decay function, with the number of hops Increase and decrease, the rate of decrease is from control, Represents a non-linear activation function. Let represent the input feature vectors of the target patent node and its neighboring nodes, respectively. Represents the learnable projection matrix; The target patent node and all its neighboring nodes are weighted and summed according to the aggregation weight, and a demand-adaptive patent representation vector is generated through a nonlinear transformation.

[0012] In this solution, a user preference representation is constructed based on the user's historical interaction behavior. The patent representation vector is then matched with the user preference representation to generate and output a personalized patent recommendation list, including: Collect user-patent interaction behavior data, including explicit feedback behavior, implicit feedback behavior, negative and boundary behavior, and query and filtering behavior. For each behavior type, set a corresponding weight value according to the intensity of user preference representation and the occurrence time, and construct a weighted user-patent interaction sequence. Based on the user patent interaction sequence, a three-level user preference representation is constructed, including: long-term static preference representation, short-term dynamic interest representation, and instant conversation intent representation. The three-level user preference representations are weighted and summed through dynamically generated fusion weights to obtain the final user preference representation. A dual-tower deep matching model is constructed, with the patent representation vector and user preference representation as inputs respectively. The model is mapped to the matching space through a nonlinear projection layer, and the context-aware matching score is calculated through dynamic matching. All candidate patents are sorted according to the matching score to generate a personalized patent recommendation list. Based on the matching score analysis and heterogeneous primitive path analysis, an interpretable natural language recommendation reason is generated for each recommendation result.

[0013] The second aspect of this invention provides a patent recommendation system based on multimodal heterogeneous graph attribute aggregation. The system includes: a multimodal data acquisition and preprocessing module, a multimodal representation learning and fusion module, a heterogeneous graph construction and dynamic maintenance module, a relational reasoning module enhanced by a large language model, a user demand analysis and precision quantification module, a dynamic hierarchical attribute aggregation module, and a user preference modeling and matching module. The multimodal data acquisition and preprocessing module acquires multimodal information of patents from a multi-source patent database and preprocesses the acquired data. The multimodal representation learning and fusion module transforms the preprocessed multimodal patent data into semantic vectors, integrates text, images and structured information, and generates multimodal representation vectors. The heterogeneous graph construction and dynamic maintenance module constructs a heterogeneous graph based on patent data, which includes patent nodes, technical keyword nodes, applicant nodes, inventor nodes and classification nodes, and uses the multimodal representation vector as the initial feature of the patent node. The relational reasoning module enhanced by the large language model calls the large language model to reason about the technical semantic relationships between patent nodes, generating implicit technical association edges that include relation types and confidence weights, thereby enhancing the heterogeneous graph; The user demand analysis and precision quantification module quantifies the precision requirements of the recommendation requirements based on the user's recommendation needs using the Fisher information matrix. The dynamic hierarchical attribute aggregation module dynamically determines the order and range of attribute aggregation according to the accuracy requirements, performs hierarchical attribute aggregation operation on the target patent node in the heterogeneous graph, and generates a patent representation vector that adapts to the requirements. The user preference modeling and matching module constructs a user preference representation based on the user's historical interaction behavior, and matches the patent representation vector with the user preference representation to generate and output a personalized patent recommendation list.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention utilizes a large language model to perform structured reasoning on the principles, application scenarios, and innovative points of patented technologies, identifying and quantifying potential semantic relationships between patents, thus greatly enhancing the accuracy and insight of recommendations. Furthermore, it introduces a demand precision quantification mechanism based on the Fisher information matrix, transforming users' fuzzy intentions into computable aggregation control parameters. Based on this, the scope, order, and path preferences of heterogeneous graph aggregation are dynamically adjusted to achieve flexible recommendations, satisfying both high-precision technology benchmarking requirements and supporting heuristic cross-domain innovation discovery.

[0015] This invention constructs a hierarchical user preference representation that organically integrates users' long-term stable interests, short-term dynamic hot topics, and immediate conversational intent, and achieves precise matching through a context-aware deep matching model. This allows recommendation results to not only be based on the patent content itself, but also deeply integrate users' historical behavioral patterns and real-time task goals, enabling personalized recommendations and effectively alleviating data sparsity and cold start problems. It automatically generates natural language recommendation reasons based on technical association paths and semantic analysis. This enhances the system's transparency and user trust, enabling recommendation results not only for information retrieval but also to directly serve scenarios such as R&D decision-making and technology competition analysis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0017] Figure 1 A flowchart of a patent recommendation method based on multimodal heterogeneous graph attribute aggregation is shown; Figure 2 A flowchart illustrating the process of invoking a large language model to enhance a heterogeneous graph is shown. Figure 3 A flowchart is shown to generate and output a personalized list of patent recommendations; Figure 4 A block diagram of a patent recommendation system based on multimodal heterogeneous graph attribute aggregation is shown. Detailed Implementation

[0018] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0020] like Figure 1 As shown, this embodiment provides a patent recommendation method based on multimodal heterogeneous graph attribute aggregation, including: Multimodal data of patents are obtained from the patent database, and features are extracted and fused from the multimodal data to generate a multimodal representation vector for each patent. A heterogeneous graph containing patent nodes, technical keyword nodes, applicant nodes, inventor nodes, and classification nodes is constructed, and the multimodal representation vector is used as the initial feature of the patent nodes; a large language model (LLM) is invoked to infer the technical semantic relationship between patent nodes, generating implicit technical association edges containing relationship type and confidence weight, thereby enhancing the heterogeneous graph; Based on the user's recommendation needs, the accuracy requirements of the recommendation needs are quantified using the Fisher information matrix. Based on the accuracy requirements, the order and range of attribute aggregation are dynamically determined. Hierarchical attribute aggregation operation is performed on the target patent nodes in the heterogeneous graph to generate a patent representation vector adapted to the requirements. A user preference representation is constructed based on the user's historical interaction behavior. The patent representation vector is then matched with the user preference representation to generate and output a personalized patent recommendation list.

[0021] It should be noted that the relevant patent text data, image data, and structured data are obtained from the patent database. The text data includes the title, abstract, claims, and full text of the specification; the image data includes technical drawings, process flow diagrams, chemical structural formulas, circuit diagrams, etc.; and the structured data includes the application number, applicant, inventor, IPC classification number, legal status, etc. In the preprocessing stage, the text data is segmented and technical chapters are identified; the image data undergoes key region detection and structural information extraction; and the structured data undergoes entity alignment and normalization.

[0022] Language models such as PatentBERT and SciBERT, pre-trained on patent corpora, are used to encode the titles and abstracts in the preprocessed text data to obtain technical topic vectors. For the full text of the specification, segmented attention encoding and chapter-level feature aggregation are used to obtain long text structure vectors, preserving the complete logic and detailed information of the technical description. Based on the technical topic vectors and long text structure vectors, text representation vectors are output. For the technical figures in the preprocessed image data, a visual Transformer is used to extract global and local visual features, and key components and labeled areas in the images are identified through object detection. For structured images, a graph neural network is used to fuse visual and topological features, encoding image pixel information and implicit topological structures such as chemical bonds and circuit connections to obtain image semantic vectors. Based on the global and local visual features and image semantic vectors, image representation vectors are output. For entities such as applicants and inventors in the preprocessed structured data, heterogeneous graph node embedding is used to obtain entity vectors. For IPC classification numbers, a hierarchical encoding network is used to obtain classification hierarchy vectors, preserving the hierarchical semantics of major categories, minor categories, main groups, and subgroups. Based on the entity vectors and classification hierarchy vectors, structured representation vectors are output.

[0023] At the local alignment level, the semantic association between text representation vectors and image representation vectors is calculated through cross-attention to enhance the consistency of text and image representation. At the global semantic level, a multi-head attention network between modalities is constructed to dynamically learn the complementary weights between text representation vectors, image representation vectors, and structured representation vectors, generating a fused comprehensive feature vector. Through the patent semantic space projection layer, the comprehensive feature vector is mapped to a unified patent semantic space to output a multimodal representation vector.

[0024] It should be noted that a heterogeneous graph containing patent nodes, technical keyword nodes, applicant nodes, inventor nodes, and classification nodes is constructed. The node types of this heterogeneous graph are defined, and the multimodal representation vectors are used as the initial features of patent nodes, providing a comprehensive semantic foundation. A patent pre-training model is used to identify core noun phrases with technical substance from the claims and specifications as keywords, and the terminology embedding vectors of these keywords in the patent corpus are obtained as the initial features of the technical keyword nodes. Applicant nodes and inventor nodes refer to institutional entities and individual entities, respectively, and the technical theme distribution vectors based on historical patent portfolios reflect the areas of technical focus and cooperation patterns. Classification nodes correspond to patent classification numbers, and the hierarchical semantic embedding vectors of the classification numbers are used as the initial features.

[0025] Based on patent metadata, predefined explicit relationship edges are established between nodes, including content ownership relationships, ownership and creation relationships, legal and family relationships, and technology diffusion relationships. These explicit relationship edges are assigned weights based on metadata statistics. The content ownership relationship directly connects patents with technology content tags and domain affiliations. The ownership and creation relationship constructs a technology entity network primarily based on patent ownership associations. The legal and family relationships represent inherent strong associations within the patent domain. The technology diffusion relationship, based on the patent examination process or transfer records, constructs edges such as divisional applications to capture ownership flow. Each edge is assigned a weight, such as citation count or family patent size, to quantify the relationship strength.

[0026] A dynamic incremental construction strategy is adopted. When processing new patents or user queries, new patent nodes and related nodes are dynamically inserted into the heterogeneous graph and corresponding edges are established. If there is no direct explicit relationship between two patents, the similarity is calculated using the multimodal representation vector of the patent nodes. This provides a similarity prior for implicit semantic relationship reasoning based on a large language model. The structure of the heterogeneous graph is used as contextual information for relationship reasoning to assist in the discovery and generation of deep semantic associations.

[0027] The large language model is invoked to infer the technical semantic relationships between patent nodes, uncovering potential, non-explicit technical connections between patents, generating implicit technical connection edges that include relationship types and confidence weights, and enhancing the heterogeneous graph. For example... Figure 2As shown, for the patent node degree to be analyzed, the core content summary of the patents corresponding to the two patent nodes and the prior information of the local graph structure extracted from the heterogeneous graph are obtained to construct structured context information for relational reasoning. The core content summary of the patents includes the titles, abstracts and core technical features extracted from the claims of the two patents. The prior information of the local graph structure extracted from the heterogeneous graph includes the local structural information of the two patents in the constructed heterogeneous graph, which provides important relational clues and constraints for LLM reasoning.

[0028] For reasoning prompt templates in different technical fields, prompt instructions are constructed based on the context information, and qualitative-semi-quantitative analysis is conducted from four dimensions: similarity of technical principles, overlap of application scenarios, complementarity of technologies, and relationship of technological evolution. The similarity of technical principles judges the degree of commonality of underlying engineering principles, and is divided into high, medium, and low. The overlap of application scenarios assesses the scope of overlap of the actual problems solved by the technology or the application fields, and is divided into high, medium, and low. The complementarity of technologies analyzes whether two patents are complementary in terms of function, components, or processes, and is divided into strong, medium, and weak. The relationship of technological evolution judges whether there is a relationship of foundation and improvement or substitution. Based on the results of the comparative analysis, the instruction big language model determines whether there are predefined types of technical associations. These predefined types include technical substitution relationships, technical complementarity relationships, technical evolution relationships, and technical migration relationships. A confidence score is output for each type of technical association that is determined to exist. A technical substitution relationship refers to two different but interchangeable solutions used to solve the same technical problem. A technical complementarity relationship refers to two solutions that, when combined, produce a better or more complete technical solution. A technical evolution relationship refers to one being an improvement or derivative of the other in its technical path. A technical migration relationship refers to one's core technology being applicable to different fields belonging to the other.

[0029] The large language model determines the existing technical relationships, constructs implicit technical association edges with relationship type labels between the corresponding two patent nodes, and uses the confidence score as the initial weight of the implicit technical association edges to semantically enhance the heterogeneous graph. Higher confidence indicates a greater certainty from the large language model regarding the existence of the current relationship. If the large language model generates multiple relationships for the same pair of nodes, edges with confidence scores below a preset threshold are filtered out. Preferably, to balance computational cost and graph real-time performance, when a user makes a query recommendation, real-time relationship reasoning is prioritized for the core patents and candidate neighbors related to the query, ensuring immediate enhancement of the recommendation results. Responding to the user query, the set of core patent nodes involved in the query is identified. Based on the topology of the heterogeneous graph, candidate neighbor patent nodes that are associated with the core patent node set within a k-hop range are identified. The patent node pairs between the core patent nodes and the candidate neighbor patent nodes are used as local target pairs to be reasoned. Contextual information is generated for these local target pairs to generate prompts, which are then imported into the large language model to perform the implicit technical association edge reasoning task. During periods of low system load, batch relational reasoning is performed on newly added patents or node pairs in historical patents that have not yet been analyzed, continuously enriching the semantic density of the graph.

[0030] It should be noted that, based on the user's recommendation needs, the Fisher information matrix is ​​used to quantify the accuracy requirements of the recommendation needs, and to quantify the fuzziness and uncertainty of the user's recommendation needs, so as to provide a basis for the subsequent dynamic adjustment of the heterogeneous graph aggregation range; the user's expressed needs are transformed from vague language descriptions into precise quantitative requirements.

[0031] The system acquires user recommendation requests, performs semantic parsing on these requests, and extracts features from preset semantic dimensions, including technology focus, scope openness, and domain cross-domain tolerance. Based on the parsing results, a request representation vector is generated for each recommendation request. The technology focus determines whether the request specifies precise technical details, such as specific materials or algorithm names. The scope openness determines whether the request seeks solutions within known paths or explores unknown connections. The domain cross-domain tolerance determines whether the request allows or encourages recommendations across technical fields.

[0032] The recommendation process is modeled as an information processing model. The model parameters of the information processing model include the order of attribute aggregation, distance decay coefficient, and meta-path selection weight. The optimal combination of model parameters is expected to correspond to different recommendation needs, so as to output recommendation results that meet the accuracy requirements. The output of the information processing model is the conditional probability distribution of the recommendation results. Based on the optimal model parameter combination, calculate the Fisher information matrix of the information processing model. , represented as: , in This represents the conditional probability distribution of the recommendation results. The model parameters represent the information processing model. This represents the optimal combination of model parameters.

[0033] The Fisher information matrix measures the change in the amount of information in the model output when model parameters change. The greater the information content, the more significant the impact of parameter changes on the results, and the more precise the control over the parameters. A scalar metric is extracted from the Fisher information matrix as the quantitative recommendation accuracy requirement for the recommended needs. The scalar metric is one of the trace of the matrix, the largest eigenvalue, or the norm ratio to the baseline demand matrix; the trace of the matrix represents the total amount of information in the parameter space. A higher trace indicates a higher overall requirement for parameter accuracy; the largest eigenvalue represents the amount of information in the direction most sensitive to parameter changes. A higher eigenvalue indicates a higher accuracy requirement for parameters in specific dimensions such as aggregation order; a norm ratio to the baseline demand matrix greater than 1 indicates that the current demand has a higher accuracy requirement than the baseline demand.

[0034] Using Fisher Information Matrix (FIM) analysis, when user demand is r, how precisely should the model parameters be adjusted to meet the current demand? The precision requirement calculated by the Fisher information matrix quantifies the precise requirements for parameter adjustment.

[0035] It should be noted that, based on the accuracy requirements of quantization, the specific combination of control parameters for attribute aggregation is selected and determined from the parameter space for the current recommendation request. Based on the aforementioned quantitative recommendation accuracy requirements, the maximum aggregation order of attribute aggregation is determined through a predefined decision mapping. Distance attenuation coefficient Set of enabled heterogeneous metapaths and the corresponding meta-path weight vector The decision mapping satisfies the following: A higher accuracy requirement results in a smaller maximum aggregation order, a larger distance decay coefficient, and a more concentrated meta-path weight vector within directly related short meta-paths; conversely, a lower accuracy requirement results in a larger maximum aggregation order, a smaller distance decay coefficient, and a more dispersed meta-path weight vector across domains and long-range meta-paths. The determined maximum aggregation order... Distance attenuation coefficient Set of enabled heterogeneous metapaths and the corresponding meta-path weight vector This is a specific instance of the model parameters in the preceding steps, and the accuracy requirement for quantization is the criterion for selecting this instance.

[0036] With target patent node Starting from the heterogeneous graph, based on the maximum aggregation order... and the set of enabled heterogeneous metapaths Perform a meta-path-based traversal to sample and obtain the set of heterogeneous multi-hop neighbor nodes of the target node. ; For the neighbor node Based on the hop distance from the target node Metapath type And the semantic attention scores between nodes, calculate the aggregation weight, wherein the aggregation weight calculates the fusion distance decay factor and the meta-path weight vector; wherein the aggregation weight Represented as: , in Represents the target node A heterogeneous multi-hop neighbor node set Indicates neighboring nodes, , This represents the unnormalized weight score. Metapath In the metapath weight vector The corresponding path weight vector, Indicated based on distance attenuation coefficient The distance decay function, with the number of hops Increase and decrease, the rate of decrease is from control, Represents a non-linear activation function. Let represent the input feature vectors of the target patent node and its neighboring nodes, respectively. Represents the learnable projection matrix; The target patent node and all its neighboring nodes are weighted and summed according to the aggregation weights, and a demand-adaptive patent representation vector is generated through a nonlinear transformation. , is represented as: , in , This represents the learnable weight matrix.

[0037] It should be noted that, as Figure 3As shown, user interaction data with patents is collected. This data includes explicit feedback behavior, implicit feedback behavior, negative and boundary behaviors, and query and filtering behavior. For each behavior type, a weighted user patent interaction sequence is constructed based on the intensity of user preference and the timing of occurrence. Explicit feedback behaviors, such as saving, downloading, explicitly rating, and adding to projects, directly express preferences and are assigned high positive weights. Implicit feedback behaviors, such as browsing time, repeated viewing, detailed reading of claims, and viewing drawings, are converted to medium weights through a time decay function and participation quantification. Negative and boundary behaviors, such as quickly skipping, marking as irrelevant, and excluding similar patents after retrieval, are assigned negative weights. Query and filtering behaviors record users' historical query keywords and classification codes used for filtering, reflecting the user's proactive technical intent, and are encoded as intent vectors.

[0038] Based on the user's patent interaction sequence, a three-tiered user preference representation is constructed, including: long-term static preference representation, short-term dynamic interest representation, and instant conversation intent representation. The three-tiered user preference representation is weighted and summed using dynamically generated fusion weights to obtain the final user preference representation. The long-term static preference representation is based on the multimodal representation of all the user's historical interaction patents, learned through an attention pooling mechanism, and uses a user profile determined by the technical field as the query vector. The most relevant technical topics, key technical means, and other information are extracted from the historical interaction patents and aggregated. The short-term dynamic interest representation is based on the recent (e.g., the past week or month) interaction sequence and is modeled using a Transformer-based sequence encoder to capture the user's current research hotspots and interest drift. The instant conversation intent representation is calculated in real time based on the interaction behavior and query input in the current session. A lightweight encoder is used to fuse the keyword vector of the current query and the patent representations that have been viewed or operated.

[0039] Construct a dual-tower deep matching model to integrate the patent representation vector. and user preference representation Each input is mapped to the matching space through a nonlinear projection layer. A context-aware matching score is then calculated using dynamic matching, as follows: , As a dynamic matching matrix, it is used by a context encoding network based on the current recommendation context. The recommended scenarios are generated in real time. This includes recommendation accuracy requirements and information about the user's technical field. During the model training phase, a multi-task learning framework is adopted, with patent recommendation as the primary task, and two auxiliary tasks—patent quality prediction and technology novelty prediction—jointly optimized to improve the generality of the representation and the quality of the recommendations.

[0040] All candidate patents are sorted according to the matching scores to generate a personalized patent recommendation list. Based on matching score analysis and heterogeneous primitive path analysis, an interpretable natural language recommendation reason is generated for each recommendation result. User feedback on the recommendation results is collected in real time, and the parameters of the short-term dynamic interest representation, the immediate conversational intent representation, and the dual-tower deep matching model are incrementally updated using the feedback data to achieve adaptive optimization of the recommendation system.

[0041] like Figure 4 As shown, the second embodiment of the present invention provides a patent recommendation system based on multimodal heterogeneous graph attribute aggregation. The system includes: a multimodal data acquisition and preprocessing module, a multimodal representation learning and fusion module, a heterogeneous graph construction and dynamic maintenance module, a relational reasoning module enhanced by a large language model, a user demand analysis and precision quantification module, a dynamic hierarchical attribute aggregation module, and a user preference modeling and matching module. The multimodal data acquisition and preprocessing module acquires multimodal information of patents from a multi-source patent database and preprocesses the acquired data. The multimodal representation learning and fusion module transforms the preprocessed multimodal patent data into semantic vectors, integrates text, images and structured information, and generates multimodal representation vectors. The heterogeneous graph construction and dynamic maintenance module constructs a heterogeneous graph based on patent data, which includes patent nodes, technical keyword nodes, applicant nodes, inventor nodes and classification nodes, and uses the multimodal representation vector as the initial feature of the patent node. The relational reasoning module enhanced by the large language model calls the large language model to reason about the technical semantic relationships between patent nodes, generating implicit technical association edges that include relation types and confidence weights, thereby enhancing the heterogeneous graph; The user demand analysis and precision quantification module quantifies the precision requirements of the recommendation requirements based on the user's recommendation needs using the Fisher information matrix. The dynamic hierarchical attribute aggregation module dynamically determines the order and range of attribute aggregation according to the accuracy requirements, performs hierarchical attribute aggregation operation on the target patent node in the heterogeneous graph, and generates a patent representation vector that adapts to the requirements. The user preference modeling and matching module constructs a user preference representation based on the user's historical interaction behavior, and matches the patent representation vector with the user preference representation to generate and output a personalized patent recommendation list.

[0042] Preferably, the system further includes an interactive feedback and visualization module, and an online learning and system optimization module. The interactive feedback and visualization module provides a user interface for inputting queries, selecting recommendation modes, and browsing recommendation lists, and graphically displays the technical relationship network between patents, the reasons for recommendation, and the technical evolution path. It collects explicit and implicit user feedback on the recommendation results and feeds it back to the user preference modeling and online learning module in real time. The online learning and system optimization module continuously updates the user model, matching model, and graph relationships through real-time feedback data to address the cold start problem and adapt to user interest drift, thereby achieving system self-iteration and optimization.

[0043] A third embodiment of the present invention provides a computer-readable storage medium including a patent recommendation method program based on multimodal heterogeneous graph attribute aggregation. When the patent recommendation method program based on multimodal heterogeneous graph attribute aggregation is executed by a processor, it implements the steps of the patent recommendation method based on multimodal heterogeneous graph attribute aggregation.

[0044] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms. Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0045] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0046] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A patent recommendation method based on multimodal heterogeneous graph attribute aggregation, characterized in that, Includes the following steps: Multimodal data of patents are obtained from the patent database, and features are extracted and fused from the multimodal data to generate a multimodal representation vector for each patent. Construct a heterogeneous graph containing patent nodes, technical keyword nodes, applicant nodes, inventor nodes, and classification nodes, and use the multimodal representation vector as the initial feature of the patent nodes; The large language model is invoked to infer the technical semantic relationships between patent nodes, generating implicit technical association edges that include relationship types and confidence weights, thereby enhancing the heterogeneous graph; Based on the user's recommendation needs, the accuracy requirements of the recommendation needs are quantified using the Fisher information matrix. Based on the accuracy requirements, the order and range of attribute aggregation are dynamically determined. Hierarchical attribute aggregation operation is performed on the target patent nodes in the heterogeneous graph to generate a patent representation vector adapted to the requirements. A user preference representation is constructed based on the user's historical interaction behavior. The patent representation vector is then matched with the user preference representation to generate and output a personalized patent recommendation list.

2. The patent recommendation method based on multimodal heterogeneous graph attribute aggregation according to claim 1, characterized in that, Multimodal data of patents are obtained from a patent database. Feature extraction and fusion are performed on the multimodal data to generate a multimodal representation vector for each patent, including: The process involves obtaining text data, image data, and structured data of relevant patents from a patent database; segmenting the text data into paragraphs and identifying technical chapters; detecting key regions and extracting structural information from the image data; and performing entity alignment and normalization on the structured data. The title and abstract in the preprocessed text data are encoded using a language model pre-trained with patent corpus to obtain a technical topic vector. For the full text of the specification, segmented attention encoding and chapter-level feature aggregation are used to obtain a long text structure vector. Based on the technical topic vector and the long text structure vector, a text representation vector is output. The technical figures in the preprocessed image data are used to extract global and local visual features using a visual Transformer. The structured image is then fused with visual and topological features using a graph neural network to obtain an image semantic vector. Based on the global and local visual features and the image semantic vector, an image representation vector is output. Heterogeneous graph node embedding is used to obtain entity vectors for applicants and inventors in preprocessed structured data, and hierarchical coding network is used to obtain classification hierarchy vectors for IPC classification numbers. Based on the entity vectors and classification hierarchy vectors, structured representation vectors are output. By calculating the semantic association between text representation vectors and image representation vectors through cross-attention, an intermodal multi-head attention network is constructed. The complementary weights between text representation vectors, image representation vectors, and structured representation vectors are dynamically learned to generate a fused comprehensive feature vector. The comprehensive feature vector is then mapped to a multimodal representation vector through a patented semantic space projection layer.

3. The patent recommendation method based on multimodal heterogeneous graph attribute aggregation according to claim 1, characterized in that, Construct a heterogeneous graph containing patent nodes, technical keyword nodes, applicant nodes, inventor nodes, and classification nodes, and use the multimodal representation vector as the initial feature of the patent nodes, including: Define a heterogeneous graph node type that includes patent nodes, technical keyword nodes, applicant nodes, inventor nodes and classification nodes, and use the multimodal representation vector as the initial feature of the patent node; Based on patent metadata, predefined explicit relationship edges are established between nodes, including content ownership relationship, ownership and creation relationship, legal and family relationship, and technology diffusion relationship, and weights based on metadata statistics are assigned to the explicit relationship edges. A dynamic incremental construction strategy is adopted to dynamically insert new patent nodes and related nodes into the heterogeneous graph. The similarity is calculated by using the multimodal representation vector of the patent nodes, providing similarity priors for implicit semantic relationship reasoning based on a large language model, and the structure of the heterogeneous graph is used as contextual information for relationship reasoning.

4. The patent recommendation method based on multimodal heterogeneous graph attribute aggregation according to claim 1, characterized in that, The large language model is invoked to infer the technical semantic relationships between patent nodes, generating implicit technical association edges containing relationship types and confidence weights, thereby enhancing the heterogeneous graph, including: For the patent node degree to be analyzed, obtain the core content summary of the patent corresponding to the two patent nodes and the prior information of the local graph structure extracted from the heterogeneous graph, and construct the structured context information for relational reasoning. Based on the aforementioned context information, prompt instructions are constructed, and a comparative analysis is conducted from four dimensions: similarity of technical principles, overlap of application scenarios, complementarity of technologies, and relationship of technological evolution. Based on the results of the comparative analysis, the instruction big language model determines whether there are predefined types of technical associations, including technical substitution relationships, technical complementarity relationships, technical evolution relationships, and technical migration relationships, and outputs a confidence score for each type of technical association that is determined to exist. The large language model determines the existing technical relationships, constructs implicit technical association edges with relationship type labels between the corresponding two patent nodes, and uses the confidence score as the initial weight of the implicit technical association edges to perform semantic enhancement on the heterogeneous graph.

5. The patent recommendation method based on multimodal heterogeneous graph attribute aggregation according to claim 1, characterized in that, Based on the user's recommendation needs, the accuracy requirements of the recommendation needs are quantified using the Fisher information matrix, including: The system obtains the user's recommendation needs, performs semantic parsing on the recommendation needs, and extracts features of preset semantic dimensions, including technology focus, scope openness, and domain cross-domain tolerance. Based on the parsing results, it generates a demand representation vector of the recommendation needs. The recommendation process is modeled as an information processing model. The model parameters of the information processing model include the order of attribute aggregation, distance decay coefficient, and meta-path selection weight. The optimal combination of model parameters is expected to correspond to different recommendation needs. The output of the information processing model is the conditional probability distribution of the recommendation results. Based on the optimal model parameter combination, the Fisher information matrix of the information processing model is calculated; scalar metric values ​​are extracted from the Fisher information matrix as the quantitative recommendation accuracy requirements of the recommendation needs.

6. The patent recommendation method based on multimodal heterogeneous graph attribute aggregation according to claim 1, characterized in that, Based on the aforementioned accuracy requirements, the order and range of attribute aggregation are dynamically determined. A hierarchical attribute aggregation operation is then performed on the target patent nodes in the heterogeneous graph to generate a demand-adaptive patent representation vector, including: Based on the quantitative recommendation accuracy requirements, the maximum aggregation order, distance decay coefficient, enabled heterogeneous meta-path set, and corresponding meta-path weight vector of attribute aggregation are determined through a predefined decision mapping. Starting from the target patent node, a meta-path-based traversal is performed in the heterogeneous graph according to the maximum aggregation order and the enabled set of heterogeneous meta-paths to sample and obtain the set of heterogeneous multi-hop neighbor nodes of the target node. For the neighboring nodes, an aggregation weight is calculated based on the hop distance to the target node, the type of the meta-path, and the semantic attention score between the nodes. This aggregation weight incorporates a distance decay factor and a meta-path weight vector. Represented as: ， in Represents the target node A heterogeneous multi-hop neighbor node set Representing neighboring nodes, This represents the unnormalized weight score. The path weight vector represents the meta-path. Indicated based on distance attenuation coefficient The distance decay function, with the number of hops Increase and decrease, the rate of decrease is from control, Represents a non-linear activation function. Let represent the input feature vectors of the target patent node and its neighboring nodes, respectively. The learnable projection matrix is ​​represented; the target patent node and all its neighboring nodes are weighted and summed according to the aggregation weight, and a demand-adaptive patent representation vector is generated through nonlinear transformation.

7. The patent recommendation method based on multimodal heterogeneous graph attribute aggregation according to claim 1, characterized in that, A user preference representation is constructed based on the user's historical interaction behavior. The patent representation vector is then matched with the user preference representation to generate and output a personalized patent recommendation list, including: Collect user-patent interaction behavior data, including explicit feedback behavior, implicit feedback behavior, negative and boundary behavior, and query and filtering behavior. For each behavior type, set a corresponding weight value according to the intensity of user preference representation and the occurrence time, and construct a weighted user-patent interaction sequence. Based on the user patent interaction sequence, a three-level user preference representation is constructed, including: long-term static preference representation, short-term dynamic interest representation, and instant conversation intent representation. The three-level user preference representations are weighted and summed through dynamically generated fusion weights to obtain the final user preference representation. A dual-tower deep matching model is constructed, with the patent representation vector and user preference representation as inputs respectively. The model is mapped to the matching space through a nonlinear projection layer, and the context-aware matching score is calculated through dynamic matching. All candidate patents are sorted according to the matching score to generate a personalized patent recommendation list. Based on the matching score analysis and heterogeneous primitive path analysis, an interpretable natural language recommendation reason is generated for each recommendation result.

8. A patent recommendation system based on multimodal heterogeneous graph attribute aggregation, characterized in that, For implementing the patent recommendation method based on multimodal heterogeneous graph attribute aggregation as described in any one of claims 1-7, the system includes: a multimodal data acquisition and preprocessing module, a multimodal representation learning and fusion module, a heterogeneous graph construction and dynamic maintenance module, a relational reasoning module enhanced by a large language model, a user demand analysis and precision quantification module, a dynamic hierarchical attribute aggregation module, and a user preference modeling and matching module; The multimodal data acquisition and preprocessing module acquires multimodal information of patents from a multi-source patent database and preprocesses the acquired data. The multimodal representation learning and fusion module transforms the preprocessed multimodal patent data into semantic vectors, integrates text, images and structured information, and generates multimodal representation vectors. The heterogeneous graph construction and dynamic maintenance module constructs a heterogeneous graph based on patent data, which includes patent nodes, technical keyword nodes, applicant nodes, inventor nodes and classification nodes, and uses the multimodal representation vector as the initial feature of the patent node. The relational reasoning module enhanced by the large language model calls the large language model to reason about the technical semantic relationships between patent nodes, generating implicit technical association edges that include relation types and confidence weights, thereby enhancing the heterogeneous graph; The user demand analysis and precision quantification module quantifies the precision requirements of the recommendation requirements based on the user's recommendation needs using the Fisher information matrix. The dynamic hierarchical attribute aggregation module dynamically determines the order and range of attribute aggregation according to the accuracy requirements, performs hierarchical attribute aggregation operation on the target patent node in the heterogeneous graph, and generates a patent representation vector that adapts to the requirements. The user preference modeling and matching module constructs a user preference representation based on the user's historical interaction behavior, and matches the patent representation vector with the user preference representation to generate and output a personalized patent recommendation list.

Images