A graph network patent retrieval method based on meta-learning ipc classification

Abstract

The application discloses a graph network patent retrieval method based on meta learning IPC classification, comprising constructing a binary classification task of each IPC category on a patent dataset, using a meta learning mode to enable a meta model to learn shared prior of the task, and enabling the model to adapt to a specific IPC category determination task with only a few samples; through IPC level-down sequential hierarchical classification, the IPC category to be determined is circled layer by layer; a graph is constructed based on a noun phrase, and structural information is fused on the basis of semantics to meet the requirement of accurately judging patent similarity.

Description

Technical Field

[0001] This invention belongs to the field of patent mining technology, specifically relating to a graph network patent retrieval method based on meta-learning IPC classification. Background Technology

[0002] The increase in patent applications has brought significant challenges to the entire patent system. In reality, after an applicant submits a patent application, patent examiners or patent experts need to conduct relevant patent searches and investigations to check whether the patent application has novelty and whether there are any related legal issues such as infringement. Manually performing this process is a complex problem. On the one hand, the patent database is huge, and manual searches may miss some patents; on the other hand, the technical complexity of patents requires examiners to have relevant professional knowledge to understand the technical information in order to select accurate and reasonable keywords for patent searches and determine the novelty of patent applications. IPC classification numbers are very important for patent applications, examinations, searches, and analyses. However, the IPC classification number is a complex five-level hierarchical structure. Determining the field that matches the patent application requires a large amount of cross-disciplinary professional knowledge. Furthermore, the number of categories at each level increases progressively, with 7,590 categories at the "major group" level and 70,788 categories at the "minor group" level. Obviously, determining the IPC classification number is a time-consuming and labor-intensive process. To address these issues, researchers have proposed many models for IPC classification and patent search. In IPC classification, the main method is based on patent text word embeddings and deep neural network learning to map patent representations to IPC codes. In related patent retrieval, the main method simplifies patents into sets of keywords, and different methods are derived from these keyword sets to assess the similarity of the sets.

[0003] The most similar prior art implementation to this invention is as follows:

[0004] 1. Application No.: CN202410069639.9, Patent Title: A Patented IPC Classification Method and System Based on Multimodal Hyperbolic Embedding of Text and Images, and its similarities and differences with this invention:

[0005] Differences: This invention proposes an IPC classification method based on image-text multimodal hyperbolic embedding. It utilizes an image-text multimodal hyperbolic model to obtain embedded representations of images and text, projects these two vectors into a multimodal embedding space, and performs IPC multi-label classification. In contrast, this invention only requires title, abstract, and independent claim text data for the IPC classification part. Furthermore, this invention does not consider the hierarchical structure information of IPCs; this invention employs meta-learning to perform hierarchical classification according to the IPC hierarchy.

[0006] Similarities: Both this invention and this invention use patented embedding representations obtained through deep learning, and both are used to automatically determine IPC classification numbers.

[0007] 2. Application No.: CN202311310422.4, Patent Title: A Post-Interaction Patent Retrieval Method and Apparatus Based on Self-Attention Mechanism, and its similarities and differences with this invention:

[0008] Differences: This invention proposes a post-interaction patent retrieval method based on a self-attention mechanism. It uses BERT encoding to obtain query vectors and patent vectors, concatenates them, and inputs them into a self-attention model. The output is the similarity between the query vector and the patent, performed using supervised learning. The related patent retrieval method proposed in this invention, based on graph networks, constructs a graph of noun phrases and uses GraphSAGE's unsupervised training method to learn patent representations, thereby calculating patent similarity without requiring a labeled dataset.

[0009] Similarities: Both this invention and this invention use BERT to obtain the embedded representation of text and use deep learning to calculate patent similarity in order to retrieve related patents.

[0010] 3. Application No.: CN202311270171.1, Patent Title: A Patent Search Method, Search System and Storage Medium Based on Similarity, and its differences from this invention:

[0011] The difference lies in the approach: The patent retrieval method proposed in this invention extracts keyword groups from the query text and stored patents, calculates the TF-IDF value of each keyword, selects the keywords with the highest weights as patent keyword groups, and calculates the similarity between the search keyword groups and each patent keyword group. This invention focuses on noun phrases and automatically generates keywords based on an entire patent to be examined. It constructs a graph based on noun phrases and calculates the similarity between patents through the graph representation of the patents, rather than the similarity of individual scattered keywords.

[0012] Similarities: Both this invention and this invention focus on extracting keywords, and both use word vectors of keywords in the process of calculating the similarity of patents to retrieve related patents.

[0013] 4. Differences and similarities between the paper "DeepPatent: patent classification with convolutional neural networks and word embedding" and this invention:

[0014] Differences: This paper uses the Skip-gram algorithm to generate word embeddings for each word, while this invention uses BERT. This paper uses CNN to directly learn the mapping between patent text word embeddings and over 600 categories of IPC classification numbers at level three, while this invention uses meta-learning to learn an intermediate model. This intermediate model learns binary classification during training, making it easier to fit the data compared to directly learning over 600 categories.

[0015] Similarities: Both this method and the present invention use deep learning technology to obtain word embeddings of the patented text, and use deep neural networks to learn the mapping between word embeddings and IPC codes based on these word embeddings. Summary of the Invention

[0016] Objective: To address the increasing number of patent applications, this invention proposes a graph network-based patent retrieval method using meta-learning IPC classification. This method employs a meta-learning model to learn the mapping between patent text and IPC classification numbers, predicts possible IPC classification numbers for the patent text, narrows the scope of relevant patent searches using the predicted IPC classification numbers, and captures the similarity between two patents using a graph network approach to retrieve relevant patents as accurately as possible.

[0017] Technical solution:

[0018] The specific technical solution adopted in this invention is a graph network patent retrieval method based on meta-learning IPC classification. The words in patent text that reflect technical characteristics are primarily noun phrases; therefore, the patent text is parsed into a sequence of noun phrases. Due to the five-level tree structure of IPC classification numbers, the lower the level, the more subdivided the fields, making it increasingly difficult to determine the classification. Furthermore, the data volume for each IPC classification number varies significantly. Therefore, the determination of each IPC classification number is treated as a separate 0 / 1 binary classification problem. Through meta-learning, the model learns the task of multiple IPC classification numbers at once, aiming to ensure that the meta-model can learn with a small number of samples regardless of the IPC classification task. The trained meta-model then serves as the shared prior for each IPC classification. Given a patent to be examined, the trained meta-model determines the possible IPC classifications of the patent to be examined, layer by layer, from the first IPC level down. The possible IPC classifications predicted by the meta-model's hierarchical classification can narrow down the scope of relevant patent searches. To accurately determine the relevance of patents, based on the word embedding of noun phrases, the structural association of phrases is combined. Noun phrases are used as nodes, and edges are constructed based on whether noun phrases co-occur in a sentence, thus representing a patent as a graph network of noun phrases. The graph learns node embeddings in an unsupervised manner, aggregating representations of neighboring nodes. The graph representations implicitly contain structural relationships between noun phrases, better reflecting the connections between patents. Specifically, this includes the following steps:

[0019] (1) Pretreatment;

[0020] Data preprocessing is a crucial step in the deep learning process, aiming to clean and transform raw data to facilitate subsequent modeling. This module's main tasks include XML parsing, sentence dependency resolution, data storage, outlier handling, and text-to-feature conversion.

[0021] The data originates from patent data files of the United States Patent and Trademark Office (USPTO). USPTO patent data is stored in XML format, with each XML file containing extensive patent information, including patent number, applicant, inventor, abstract, claims, and drawings. The structured nature of XML makes the data easy to parse and process, providing a rich information resource for researchers, patent search systems, and other stakeholders. This method uses ElementTree to parse the patent XML files, extracting information such as the title, abstract, independent claims, and IPC classification number for each patent. Considering that the first few claims in the collected patent data may be "cancelled," the claims section only stores the text of the first valid claim as the independent claim.

[0022] Because the textual data of the patent needs to be converted into numerical vectors before it can be input into the model for calculation, it is necessary to consider which data from the patent text should represent the entire patent. Patent documents typically use domain-specific terminology and technical jargon to describe inventions, technologies, or methods; these terms and phrases accurately reflect the technical field and its characteristics. Therefore, identifying and understanding noun phrases in a patent is crucial for understanding its core content and technical details. Spacy provides a set of efficient and accurate tools capable of performing lexical and syntactic analysis of text, including part-of-speech tagging, named entity recognition, and dependency parsing. This method utilizes Spacy for syntactic analysis to extract noun phrases from patent documents. The extracted set of noun phrases is then deduplicated to ensure the accuracy and uniqueness of the results. Simultaneously, some misidentified stop words irrelevant to the patent are removed. This set of noun phrases is then used to represent the entire patent.

[0023] The pre-trained BERT model is used to convert each noun phrase into a vector representation. Traditional word vector models (such as Word2Vec and GloVe) typically map each word to a fixed vector representation, ignoring the influence of context. BERT can dynamically generate word vectors based on context, thus better capturing the meaning of words in different contexts and being more effective in handling semantic similarity, word ambiguity, and other issues.

[0024] For a patent document, a set P of noun phrases is obtained through lexical analysis, where P = {p1, p2, ..., p...} n Each noun phrase uses BERT to generate word embeddings. Each noun phrase is a 768-dimensional word vector. The noun phrase word embedding sequence in the patent is denoted as Embed, where Embed = {e1, e2, ..., e...} n}

[0025] (2) IPC classification based on meta-learning;

[0026] Determining the IPC classification number for a patent is a complex task. The IPC classification system has a five-level hierarchical structure, with each level subdividing the domain above it. Due to the intricate nature of these disciplines, the number of categories increases as you approach the bottom levels. Some IPC categories have a large number of samples, while others have fewer, making classification in categories with fewer samples difficult. To address this issue, we treat the IPC classification task as a few-shot learning task and employ meta-learning to classify patents into IPC Level 3.

[0027] The goal of meta-learning is to design a learning algorithm that enables learners to learn from a series of related tasks, extract the commonalities and differences between tasks, and quickly adapt to new tasks by leveraging the experience gained from previous learning, thus exhibiting good generalization ability.

[0028] The tasks generated by the task generator are divided into two types: training-time tasks and testing-time tasks.

[0029] The training task is as follows: For each category in IPC levels one to three, randomly select a category, such as "G06F", and randomly sample K+M patents belonging to this category and K+M patents not belonging to this category. 2K samples are used as the Support Set, and 2M samples are used as the Query Set. The Support Set and Query Set constitute a 0 / 1 binary classification task T to determine whether a patent belongs to the category "G06F". i It contains 2K+2M samples. During training, each batch of tasks sampled is denoted as D. train D train ={T1,T2,…,T B}, where T i ={S,Q}, S stands for Support Set, Q stands for Query Set, and B is the number of tasks in a batch.

[0030] The testing task is as follows: For an IPC category (regardless of level), randomly sample K patents belonging to that category and K patents not belonging to that category. These 2K samples constitute the Support Set, while the Query Set consists only of the test samples (i.e., the Query Set has 1 sample). The Support Set and Query Set constitute a 0 / 1 binary classification task for that category, containing 2K+1 samples. Each sampling task during testing is denoted as D. test D test = {T}, where T = {S, Q}, Q={(x test ,y test )}.

[0031] The metamodel employs a long short-term memory recurrent neural network combined with a fully connected neural network. Metamodel f θ It has a global parameter θ, which is copied and distributed to a batch of tasks to be learned. The model uses samples from the Support Set of each task to perform several inner gradient updates to update the model parameters. The loss function is calculated based on cross-entropy loss. Get adapted to task T i Model parameters θ i ′ The adapted model is used to calculate the cross-entropy loss in the Query Set, denoted as... Results obtained from various missions similar to "G06F" Summing and updating global model parameters f θ .

[0032]

[0033] Through the above task construction, the model can learn multiple similar binary classification problems. This method divides model training into three stages, with the first stage acting on the first level "part" of IPC. Each element represents a classification number at the first level of IPC. Similar tasks are constructed for each category at the first level, and the model is trained, denoted as θ. 1 After the first phase of training, the model has learned the shared priors of each category at the first level, enabling it to adapt to binary classification problems of each category with only a few samples. The second phase operates on the second level of IPC, the "major class". Each element represents a classification number in the second level of IPC. Based on the global model trained in the first level, it can help learn the classification of each category in the second level. Similarly, a second-level task is constructed to train the global model, and the trained model is denoted as θ. 2 In the third stage, the global model trained at level two is applied to the third-level "subclasses" of IPC. Each element represents a classification number at the third level of IPC. A three-level task is constructed, the model is trained, and the trained model is denoted as θ. 3 This training method comprehensively considers the performance of the global model after a few adaptations at various levels and on various binary classification tasks. The goal is for the final trained model to quickly learn a new task through a few adaptations. Furthermore, these global parameters can be viewed as shared features learned by the model across the entire dataset.

[0034] During the testing phase, hierarchical classification is implemented. Given a patent, the patent is processed by the preprocessing module into a sequence of noun phrase embeddings, denoted as sample X. The current level is denoted as l, and the candidate category set Z = {C} l,j |Pred(parent(C l,j ))=1}, initially l=1, Z represents all classes at the first level of IPC, that is When l = 2 or 3, the IPC in the candidate set must require the IPC of the previous layer to be predicted as 1 by the model. The trained model is updated during the task of sampling and testing for each class in the first level of IPC, with X input into each adapted model. in Predict whether X belongs to this category. Record the categories with an output of 1. Sample the test task from the subcategories of the categories with an output of 1. For example, if the first-level category A is predicted as 1, the second-level category will sample from IPCs A01 to A24. Based on the model parameters adapted from the first level. The model is then updated based on these secondary tasks. Inputting X again yields the category in the secondary major class that outputs 1, representing the secondary major class to which the model believes sample X belongs. For example, A01 is predicted as 1. The updated model parameters are then... Finally, for the subcategories of the second-level major category to which X belongs, such as A01G, A01N, etc., a sampling task is performed. Based on the adaptation of the upper-level model, the subcategories to which X belongs are determined. Following the tree structure of IPC, the classification is performed layer by layer from top to bottom to obtain the predicted IPC third-level classification results. Where C 3,i Pred is a certain IPC classification number in the third layer, and Pred is the meta-model's prediction of whether the patent belongs to a certain IPC classification number.

[0035] (3) Relevant patent search based on graph representation;

[0036] Patent searching is a crucial step in the patent application process. By determining the possible IPC classification numbers of the patent to be applied for, the scope of the search can be narrowed down to several subfields. Traditional methods mainly rely on keyword searches, which require a high level of professional knowledge and are also susceptible to subjective bias due to differences in the keywords obtained by different individuals. This paper proposes a graph-based patent similarity judgment method, which can eliminate the influence of subjective factors and considers not only synonyms and ambiguities but also the structural information of the keywords.

[0037] The patent pair selection module pairs patents in the database that fall under the predicted IPC category with the patents to be applied for.

[0038] Graph Construction Module: In patent texts, technical terms are primarily nouns or noun phrases. These noun phrases can be used to determine the similarity between two patents. However, due to the existence of polysemy and synonyms, directly judging whether the nouns appearing are identical is insufficient for accurate similarity determination. Therefore, this method first uses BERT word vectors; two different words with similar word vectors are semantically similar. Secondly, while two patents may contain overlapping technical terms, their technical focus and themes may differ. Methods based on phrase word vector similarity cannot adequately address this issue. Therefore, this method proposes constructing a graph representation of the patent as a whole. Combining word embeddings with phrase structure information allows for more accurate relevance assessment. Specifically, a graph G = (V, E) is constructed, where V is a set of nodes, and noun phrases are the nodes. E is a set of edges. If two noun phrases appear in the same sentence, there is an edge between the two nodes, which is the occurrence function.

[0039] E = {(v i ,v j )|occur(v i ,v j )=1}

[0040]

[0041] Extracting noun phrases requires first performing part-of-speech tagging (e.g., noun, verb, adjective), and then constructing a syntactic dependency tree for the sentence. The dependency tree describes the grammatical structure of the sentence, with each word in the sentence being a node in the tree, and edges between nodes indicating the grammatical relationships between words. Based on the generated syntactic dependency tree, noun phrases can be identified through the part-of-speech tagging of each word and its grammatical relationships with other words. The node features in Graph G are the BERT word embeddings of the noun phrases.

[0042] GraphSAGE: After obtaining a graph constructed using a patent, it is necessary to learn the overall representation of the graph using graph neural networks. GraphSAGE is a framework for graph representation learning, designed to address the problem of learning representations of nodes in graph data. The core idea of ​​GraphSAGE is to generate node representations by sampling and aggregating the features of a node's neighboring nodes, thereby achieving effective feature learning and prediction tasks across the entire graph, without being limited to a specific neural network. This represents a random sample of the neighboring nodes of node v. u represents a neighboring node of node v. This represents the feature of node u at layer (k-1). k This represents an aggregate function. W k These are the trainable parameters of the k-th layer. In this invention, the aggregation function uses LSTM aggregation. The features of some neighboring nodes of node v are grouped into a sequence and input into the LSTM, which outputs the aggregated features of the neighboring nodes. The aggregated features are concatenated with the features of node v, i.e., the CONCAT function. The concatenated features are then input into a fully connected neural network (MLP) for feature transformation, corresponding to W. k This yields the new features of the current node v.

[0043]

[0044] Unsupervised learning is achieved by constructing positive and negative sample pairs. Assume (v, u) is a positive sample pair, meaning there is an edge between v and u. ′ ) are negative sample pairs, i.e., v, u ′ There is no edge between them. Q is the number of negative samples, P n (v) represents the negative sample distribution. The goal is to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs. The node features learned through aggregation, compared to the initial word embeddings, incorporate the connections between noun phrases in the document. The patent graph nodes are composed of... Learning Transformation

[0045] Different noun phrases have varying degrees of relevance to the patent topic; therefore, the graphical representation representing the overall patent should be obtained by weighting the representations of different noun phrases. In this invention, TF-IDF is used to calculate the weights of each noun phrase. TF-IDF is a statistical method used to measure the importance of a word in a document set, commonly used in information retrieval and text mining. t,d It is the frequency of word t in patent d, ∑ k n k,dThis represents the total frequency of all words in patent d. |D| is the total number of patents in the database, and |{d:t∈d}| represents the number of patents containing word t. TF represents the frequency of a word in the current document; the higher the IDF value, the stronger the word's discriminative power. Words with large TF-IDF values ​​are more relevant to the document's topic.

[0046]

[0047] TF-IDF(t,d) = TF(t,d)·IDF(t)

[0048] The overall representation h of the graph is obtained by weighting the TF-IDF of each noun phrase. G .

[0049] h G =∑ v∈V TF-IDF(v)·h v

[0050] For two patents, the cosine similarity between the overall representations of graphs G1 and G2 is calculated to measure the relevance of the two patents. The cosine similarity value ranges from -1 to 1, with values ​​closer to 1 indicating greater similarity between the vectors.

[0051]

[0052] Patents with a similarity greater than a threshold λ are selected as relevant patents. The set of relevant patents is as follows:

[0053] Beneficial effects:

[0054] This invention utilizes a five-layer tree structure of IPC classification numbers and employs a meta-learning model to independently determine each IPC classification number in a 0 / 1 binary classification problem. This allows the meta-model to quickly adapt and learn when faced with different IPC classification tasks, improving classification efficiency and accuracy. By determining the possible IPC classification of the patent under examination layer by layer, this method significantly narrows the scope of relevant patent searches, improving retrieval accuracy and efficiency. Simultaneously, by using noun phrases as nodes and constructing an edge-formed graph network using noun phrase co-occurrence, and learning node embeddings and aggregating neighbor node representations through self-supervised learning, this method fully explores the structural relationships between noun phrases, enhancing the representational power of patent texts. The graph network representation not only better reflects the correlation between patents but also improves the accuracy of relevant patent searches. Attached Figure Description

[0055] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0056] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0057] As shown in the figure, this invention provides a graph network patent retrieval method based on meta-learning IPC classification. It constructs binary classification tasks for each IPC category on a patent dataset, using a meta-learning model to learn the shared priors of these tasks. The model can adapt to the determination task of a specific IPC category using only a few samples. Through sequential hierarchical classification from the first level down through IPCs, the required IPC categories are identified layer by layer. A graph is constructed based on noun phrases to integrate structural information on the semantic level, meeting the need for accurate patent similarity judgment. Specific implementation steps include: preprocessing, training and use of the meta-model, and graph representation of related patent retrieval.

[0058] In this embodiment, the preprocessing specifically includes:

[0059] Step 1.1: Use custom XML parsing rules to extract the title, abstract, claims, and IPC classification data from the original XML patent data;

[0060] Step 2.1: Use Spacy to perform syntactic analysis on the title, abstract, and claims to extract all noun phrases, including neural stimulation and audio.

[0061] Step 3.1: Use BERT to convert the extracted noun phrases into high-dimensional word embedding vectors.

[0062] In this embodiment, ElementTree is used to parse the patent XML file and extract the title, abstract, independent claims, and IPC classification information of each patent. In the claims section, only the text of the first valid claim is stored as an independent claim.

[0063] In this embodiment, the training of the meta-model includes:

[0064] Step 2.1: Construct the training task. Each time, randomly select a category from IPC level 1 to 3, and randomly sample K+M patents belonging to that category and K+M patents not belonging to that category. Use 2K samples as the Support Set and 2M samples as the Query Set. Construct the testing task. For an IPC category regardless of level, randomly sample K patents belonging to that category and K patents not belonging to that category. Use 2K samples to form the Support Set. The Query Set consists only of test samples, i.e., the number of samples in the Query Set is 1.

[0065] Step 2.2: Metamodel f θ The global parameters θ are copied and distributed to a batch of tasks to be learned. The model uses samples from the support sets of each task to perform several inner-layer gradient updates to iterate the model parameters. The loss function is calculated based on cross-entropy loss. Get adapted to task T i Model parameters θ i ′ ;

[0066] Step 2.3: Calculate the cross-entropy loss of the adapted model on the query set, denoted as...

[0067] Step 2.4: Obtain the results from each task Sum the results as the total loss and update the meta-model f. θ ;

[0068] Step 2.5: Continuously execute steps 2.1 to 2.4 until the model converges.

[0069] In this embodiment, during the construction of the training task, the Support Set and Query Set constitute a 0 / 1 binary classification task T to determine whether a class belongs to a certain category. i It contains 2K+2M samples; each batch of tasks sampled during training is denoted as D. train D train ={T1,T2,…,T B}, where T i ={S,Q}, S stands for Support Set, and Q stands for Query Set. In the task construction for testing, the Support Set and Query Set constitute a 0 / 1 binary classification task for that category, containing 2K+1 samples. The task for each sampling during testing is denoted as D. test D test = {T}, where T = {S, Q}, Q={(x test ,y test )};

[0070] The training of the meta-model is divided into three stages. The first stage operates on the first level "part" of IPC. Each element represents a classification number at the first level of the IPC; the second stage operates on the second-level "major category" of the IPC. Each element represents a classification number at the second level of IPC; the third stage operates on the third-level "subclasses" of IPC. Each element represents a classification number at the third level of the IPC.

[0071] In this embodiment, the use of the metamodel includes:

[0072] Step 3.1: For the newly arrived patent, obtain the noun phrase word embedding sequence X of the patent through preprocessing;

[0073] Step 3.2: Let the current level be l, and the candidate category set Z = {C} l,j |Pred(parent(C l,j ))=1}, sampling test task, where C l,j For a certain IPC classification number in the l-th layer: For each IPC category in the candidate category set Z, 2K samples of that category are randomly selected to form the support set, and the test set consists only of X;

[0074] Step 3.3: Update the model using gradient descent on the support set of the task sampled at the current level, based on the updated model from the previous layer, to complete the adaptation;

[0075] Step 3.4: Input X into each adapted model Predict whether X belongs to this category; record the categories with an output of 1; use the IPC classification numbers of the categories with an output of 1 as the candidate category set S of level l+1; modify l, l = l+1;

[0076] Step 3.5: Repeat steps 3.2-3.4 until the IPC level 3 prediction is completed, and obtain the inferred IPC level 3 classification number. Among them, C 3,i It is a certain IPC classification number in the third layer.

[0077] In this embodiment, the graph representation-related patent search includes:

[0078] Step 4.1: Based on the IPC classification number predicted by the meta-model, narrow down the relevant patent search scope, and form patent pairs with the patents belonging to the predicted IPC classification number and the newly arrived patents.

[0079] Step 4.2: Construct a graph G = (V, E) for the patent, where V is the set of nodes, and noun phrases are nodes; E is the set of edges, and there is an edge between two nodes if two noun phrases appear in the same sentence.

[0080] Step 4.3: Use the LSTM aggregation function to aggregate the representations of neighboring nodes, maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs in an unsupervised manner;

[0081] Step 4.4: Calculate the TF-IDF values ​​for the nodes in the graph, and use the TF-IDF weighted average to obtain the overall graph representation h. G h G =∑ v∈V TF-IDF(v)·hv , where h v It is a representation of the node;

[0082] Step 4.5: Calculate the cosine similarity of the graph representations of patent pairs as a relevance index; patent pairs with a relevance greater than a threshold λ are added to the relevant patent set. Among them, Doc i For the candidate potentially relevant patents, Doc self For the current patent, the Sim function represents the similarity value between two patents.

[0083] In addition, the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method.

[0084] The present invention also provides an electronic device, including a memory and one or more processors, wherein the memory is used to store one or more programs; when the one or more programs are executed by the one or more processors, they implement the method described.

[0085] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A graph network patent retrieval method based on meta-learning IPC classification, characterized in that, On the patent dataset, binary classification tasks for each IPC category are constructed. Meta-learning is used to allow the meta-model to learn the shared priors of these tasks. The model can adapt to the decision task of a specific IPC category with only a few samples. The IPC categories to be determined are delineated layer by layer through sequential hierarchical classification from the first level down of IPC. Based on the construction of noun phrase graphs, structural information is integrated on the basis of semantics to meet the need for accurate judgment of patent similarity; The specific implementation steps include: preprocessing, training and use of the meta-model, and searching for relevant patents for graph representation; Training the metamodel includes: Step 2.1: Construct the training task. Each time, randomly select a category from IPC level 1 to 3, and randomly sample K+M patents belonging to that category and K+M patents not belonging to that category. Use 2K samples as the Support Set and 2M samples as the Query Set. Construct the testing task. For an IPC category regardless of level, randomly sample K patents belonging to that category and K patents not belonging to that category. Use 2K samples to form the Support Set. The Query Set consists only of test samples, i.e., the number of samples in the Query Set is 1. Step 2.2: Metamodel global parameters The model is copied and distributed to a batch of tasks to be learned. The model uses samples from the support set of each task to perform several inner-layer gradient updates to iterate the model parameters. The loss function is calculated based on cross-entropy loss. To adapt to the task Model parameters ; Step 2.3: Calculate the cross-entropy loss of the adapted model on the query set, denoted as... ; Step 2.4: Obtain the results from each task Sum the results as the total loss and update the metamodel. ; Step 2.5: Continuously execute steps 2.1 to 2.4 until the model converges; The use of metamodels includes: Step 3.1: For newly arrived patents, obtain the noun phrase embedding sequence of the patent through preprocessing. ; Step 3.2: Record the current level as Candidate category set The sampling test task, among which, For the first One IPC number of the layer, This indicates the classification number of the next higher level of this IPC number. The meta-model predicts whether a patent belongs to a certain classification number: for a candidate category set For each IPC category, 2K samples of that category are randomly selected to form the support set, while the test set consists only of... constitute; Step 3.3: Update the model using gradient descent on the support set of the task sampled at the current level, based on the updated model from the previous layer, to complete the adaptation; Step 3.4: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Input each adapted model ,predict Whether it belongs to this category; record the category with an output of 1; use the IPC classification number of the category with an output of 1 as... The candidate category set S at level; modified , ; Step 3.5: Repeat steps 3.2-3.4 until the IPC level 3 prediction is completed, and obtain the inferred IPC level 3 classification number. ,in, It is a certain IPC classification number in the third layer; Patent searches related to graphic representation include: Step 4.1: Based on the IPC classification number predicted by the meta-model, narrow down the relevant patent search scope, and form patent pairs with the patents belonging to the predicted IPC classification number and the newly arrived patents. Step 4.2: Patent Construction Diagram , It is a set of nodes; noun phrases are nodes. It is a set of edges. If two noun phrases appear in the same sentence, there is an edge between the two nodes. Step 4.3: Use the LSTM aggregation function to aggregate the representations of neighboring nodes, maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs in an unsupervised manner; Step 4.4: Calculate the TF-IDF values ​​for the nodes in the graph, and use the TF-IDF weighted average to obtain the overall representation of the graph. , - ,in It is a representation of the node; Step 4.5: Calculate the cosine similarity of the graph representations of the patent pairs as a relevance indicator; if the relevance is greater than a threshold... The inclusion of related patents ,in, These are potential related patents as candidates. This is the current patent.

2. The graph network patent retrieval method based on meta-learning IPC classification according to claim 1, characterized in that, Preprocessing includes: Step 1.1: Use custom XML parsing rules to extract the title, abstract, claims, and IPC classification data from the original XML patent data; Step 1.2: Use Spacy to perform syntactic analysis on the title, abstract, and claims to extract all noun phrases, including neural stimulation and audio. Step 1.3: Use BERT to convert the extracted noun phrases into high-dimensional word embedding vectors.

3. The graph network patent retrieval method based on meta-learning IPC classification according to claim 2, characterized in that, The patent XML file is parsed using ElementTree, and the title, abstract, independent claims, and IPC classification information of each patent are extracted. In the claims section, only the text of the first valid claim is stored as an independent claim.

4. The graph network patent retrieval method based on meta-learning IPC classification according to claim 3, characterized in that, In the training task described above, the Support Set and Query Set constitute a 0 / 1 binary classification task to determine whether a class belongs to a certain category. It contains 2K+2M samples; each batch of tasks sampled during training is denoted as... , ,in , , , That is, Support Set, and Q is Query Set. That is, a sequence of noun phrases in a patent. This refers to the IPC classification number of this patent; in the task described for constructing the test, the Support Set and Query Set constitute a 0 / 1 binary classification task for this category, containing 2K+1 samples; the task for each sampling during testing is denoted as... , ,in , , ; The training of the meta-model is divided into three stages. The first stage operates on the first level "part" of IPC. Each element represents a classification number at the first level of the IPC; the second stage operates on the second-level "major category" of the IPC. Each element represents a classification number at the second level of IPC; the third stage operates on the third-level "subclasses" of IPC. Each element represents a classification number at the third level of the IPC.

5. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 4.

6. An electronic device, characterized in that, The system includes a memory and one or more processors, the memory being used to store one or more programs; when the one or more programs are executed by the one or more processors, they implement the method as described in any one of claims 1 to 4.

Images