Graph network patent retrieval method based on meta-learning IPC classification

Abstract

The invention discloses a graph network patent retrieval method based on meta-learning IPC classification, which comprises the following steps: constructing dichotomy tasks of each IPC category on a patent data set, using a meta-learning mode to enable a meta-model to learn sharing priori of the tasks, and enabling the model to adapt to a judgment task of a specific IPC category only by using a few samples; the IPC categories needing to be determined are delineated layer by layer through sequential hierarchical classification of the IPCs from one level to the lower level; a graph is constructed based on noun phrases, and structure information is fused on the basis of semantics, so that the requirement for accurately judging patent similarity is met.

Description

Technical Field

[0001] The present invention belongs to the technical field of patent mining, and specifically relates to a graph network patent retrieval method based on meta-learning IPC classification. Background Art

[0002] The growth of patent applications has brought important challenges to the entire patent system. In reality, after an applicant submits a patent application, a patent examiner or patent expert needs to conduct relevant patent searches and research based on the submitted application to check whether the applied patent has novelty and whether there are related legal issues such as infringement. Manually performing this process is a complex problem. On the one hand, the patent database is huge, and manual searches are likely to have problems such as missed detection; on the other hand, the technical complexity of patents requires examiners to have relevant professional knowledge to understand technical information in order to select accurate and reasonable keywords for patent searches and determine the novelty of patent applications. The IPC classification number is very important for patent application, examination, search and analysis. However, the IPC classification number is a complex 5-layer hierarchical structure. To determine the field that corresponds to the patent application, on the one hand, a large amount of cross-domain professional knowledge reserves are required. On the other hand, the number of categories at each layer increases step by step, with a total of 7,590 categories at the "large group" layer and 70,788 categories at the "small group" layer. Obviously, determining the IPC classification number is a time-consuming and labor-intensive process. In the face of the above problems, researchers have proposed many models for IPC classification and patent search. In terms of IPC classification, the main method is to learn the mapping of patent representation to IPC code based on patent text word embedding and deep neural network. In terms of related patent retrieval, the main method is to simplify patents into a set of keywords, and derive different methods based on the keyword set to judge the similarity of the set.

[0003] The prior art implementation scheme most similar to the present invention is as follows:

[0004] 1. Application number: CN202410069639.9, Patent name: A patented IPC classification method and system based on multimodal hyperbolic embedding of images and texts, similarities and differences between this invention and the present invention:

[0005] Differences: This invention proposes an IPC classification method based on multimodal hyperbolic embedding of images and texts. This invention uses a multimodal hyperbolic model of images and texts to obtain embedded representations of images and texts, and projects the two vectors into a multimodal embedding space to perform IPC multi-label classification. However, this invention only needs to use the title, abstract, and independent claim text data in the IPC classification part. This invention does not consider the hierarchical structure information of IPC. This invention uses meta-learning to perform hierarchical classification according to the IPC hierarchical structure.

[0006] Similarities: Both the invention and the present invention use patented embedding representations based on deep learning, and are used to automatically determine IPC classification numbers.

[0007] 2. Application number: CN202311310422.4, Patent name: A post-interaction patent retrieval method and device based on self-attention mechanism, similarities and differences between this invention and the present invention:

[0008] Differences: This invention proposes a post-interactive patent search method based on the self-attention mechanism, which encodes the query text and the patent text to be searched through BERT to obtain the query vector and the patent vector to be searched, and splices the two into the self-attention model to output the similarity between the query vector and the patent to be searched, which is carried out in a supervised learning manner. The graph network-based related patent search proposed by the present invention constructs a graph of noun phrases and uses GraphSAGE's unsupervised training method to learn the representation of patents, thereby calculating patent similarity, without the need for labeled data sets.

[0009] Similarities: Both the invention and the present invention use BERT to obtain the embedded representation of the text and use deep learning to calculate patent similarity to retrieve related patents.

[0010] 3. Application number: CN202311270171.1, Patent name: A patent search method, search system and storage medium based on similarity, similarities and differences between the present invention and the present invention:

[0011] Differences: The patent search method proposed in this invention extracts keyword groups from query texts and stored patents, calculates the TF-IDF value of each keyword, selects several keywords with the highest weight as patent keyword groups, and calculates the similarity between the search keyword groups and each patent keyword group. The present invention focuses on noun phrases, and automatically generates keywords based on an entire patent to be examined, constructs a graph based on noun phrases, and calculates the similarity between patents through the graph representation of the patent, rather than the similarity of each scattered keyword.

[0012] Similarities: Both the invention and the present invention focus on extracting keywords, and both use keyword word vectors in the process of calculating the similarity of patents to retrieve related patents.

[0013] 4. Paper: DeepPatent:patent classification with convolutional neuralnetworks and word embedding, similarities and differences between this paper and the present invention:

[0014] Differences: The paper uses the Skip-gram algorithm to generate word embedding for each word, while the present invention uses BERT. The paper uses CNN to directly learn the mapping between patent text word embedding and more than 600 categories of the third level of IPC classification numbers, while the present invention uses meta-learning to learn an intermediate model, which has been learning two-class classification problems during the training process, which is easier to fit than directly learning more than 600 categories.

[0015] Similarities: Both this method and the present invention use deep learning technology to obtain word embeddings of patent texts, and use deep neural networks to learn the mapping between word embeddings and IPC codes based on these word embeddings. Summary of the invention

[0016] Purpose of the invention: In view of the increasing number of patent applications, this paper proposes a patent search method based on graph network with meta-learning IPC classification. This method uses a meta-learning model to learn the mapping between patent text and IPC classification number, predicts the possible IPC classification number of the patent text, uses the predicted IPC classification number to narrow the scope of related patent search, and uses a graph network to capture the similarity between two patents, so as to retrieve related patents as accurately as possible.

[0017] Technical solution:

[0018] The specific technical solution adopted by the present invention is a graph network patent search method based on meta-learning IPC classification. The words that can reflect the technical characteristics in the patent text are basically noun phrases, so the patent text is parsed into a sequence composed of noun phrases. Due to the five-layer tree structure of the IPC classification number, the more subdivided fields there are at the bottom level, the more difficult it is to judge, and the data volume of each IPC classification number is quite different. The judgment of each IPC classification number is independently regarded as a 0 / 1 binary classification problem. Through the meta-learning model, the tasks of multiple IPC classification numbers are learned at one time, so that the meta-model can learn with a small number of samples no matter which IPC classification task it faces. The trained meta-model is the shared prior of each IPC classification. Given a patent to be examined, the trained meta-model determines the possible IPC classification of the patent to be examined from the first level of IPC layer by layer. The possible IPC classification predicted by the meta-model hierarchical classification can narrow the scope of related patent retrieval. In order to accurately judge the relevance of the patent, on the basis of using the word embedding of noun phrases, the structural association of phrases is combined, and noun phrases are used as nodes. Whether the noun phrases co-occur in a sentence to construct edges, a patent is represented as a graph network of noun phrases. Let the graph learn node embedding in an unsupervised manner and aggregate the representations of neighboring nodes. The representation of the graph implies the structural association between noun phrases and can better reflect the relevance between patents. Specifically, it includes the following steps:

[0019] (1) Pretreatment;

[0020] Data preprocessing is an important step in the deep learning process, which aims to clean and transform raw data for subsequent modeling. The main tasks of this module include XML parsing, sentence dependency parsing, data storage, outlier processing, text conversion to features, etc.

[0021] The data comes from the patent data files of the United States Patent and Trademark Office (USPTO). The patent data of the USPTO is stored in XML format. Each XML file contains a large amount of detailed information about the patent, including patent number, applicant, inventor, abstract, claims, drawings, etc. The structured nature of XML makes the data easy to parse and process, providing a rich information resource for researchers, patent search systems and other stakeholders. This method uses ElementTree to parse the patent XML file and extract information such as the title, abstract, independent claim and IPC classification number of each patent. Considering that the first few claims in the collected patent data may be in a "canceled" state, the claims section only stores the first valid claim text as an independent claim.

[0022] Since the text data of the patent needs to be converted into a numerical vector before it can be input into the model calculation. It is necessary to consider which data of the patent text represents this patent. Patent documents usually use field-specific terms and professional terms to describe inventions, technologies or methods. These terms and noun phrases can accurately reflect the technical field and its characteristics. Therefore, identifying and understanding noun phrases in patents is crucial to understanding the core content and technical details of patents. Spacy provides a set of efficient and accurate tools that can perform lexical and syntactic analysis of texts, including functions such as part-of-speech tagging, named entity recognition, and dependency syntactic analysis. This method uses Spacy for syntactic analysis to extract noun phrases from patent documents. And the extracted noun phrase set is deduplicated to ensure the accuracy and uniqueness of the results. At the same time, some misidentified stop words that are not related to the patent are removed. Use a set of these noun phrases to represent the entire patent.

[0023] Use the pre-trained BERT model to convert each noun phrase into a vector representation. Traditional word vector models (such as Word2Vec and GloVe) usually map each word to a fixed vector representation, ignoring the impact of context. BERT can dynamically generate word vectors based on the context, so it can better capture the meaning of words in different contexts and is more effective in dealing with problems such as semantic similarity and word ambiguity.

[0024] For a patent Doc, the set of noun phrases P of the patent is obtained through lexical analysis, where P = {p1, p2, ..., p n}, each noun phrase uses BERT to generate word embeddings Each noun phrase is a 768-dimensional word vector. The word embedding sequence of the patent noun phrase is recorded as Embed, Embed = {e1, e2..., e n}.

[0025] (2) IPC classification based on meta-learning;

[0026] Determining the IPC classification number of a patent is a complex task. The IPC classification number is a 5-layer hierarchical structure from top to bottom. Each layer from top to bottom is a subdivision of the field of the upper layer. Due to the complexity of the subject field, the closer to the bottom layer, the more categories there are. Some IPC categories have a large number of samples, while other categories have fewer samples, which makes classification on the few-sample category difficult. To solve this problem, the IPC classification task is regarded as a small sample learning task, and meta-learning is used to classify patents into the third level of IPC.

[0027] The goal of meta-learning is to design a learning algorithm that enables the learner to extract the commonalities and differences between tasks by learning from a series of related tasks. When faced with new tasks, it can use the experience gained from previous learning to quickly adapt to new tasks and have good generalization ability.

[0028] The tasks constructed by the task generator are divided into two types: training tasks and testing tasks.

[0029] The training task is as follows: for each category of IPC level 1 to 3, randomly select a category, such as "G06F", randomly sample K+M patents belonging to this category and K+M patents not belonging to this category, 2K samples as Support Set, 2M samples as Query Set, Support Set and Query Set constitute a 0 / 1 binary classification task T of whether it belongs to the category "G06F". i , containing 2K+2M samples. During training, each batch of tasks sampled is denoted as D train , D train ={T1,T2,…,T B}, where T i ={S,Q}, S stands for Support Set, Q stands for Query Set, and B is the number of tasks in a batch.

[0030] The tasks during testing are as follows: for an IPC category (regardless of the level), randomly sample K patents belonging to the category and K patents not belonging to the category. 2K samples constitute the Support Set, and the Query Set consists only of test samples, that is, the number of Query Set samples is 1. The Support Set and the Query Set constitute a 0 / 1 binary classification task for the category, containing 2K+1 samples. The task of each sampling during testing is denoted as D test , D test = {T}, where T = {S, Q}, Q={(x test ,y test )}.

[0031] The meta-model uses a long short-term memory recursive neural network plus a fully connected neural network. Meta-model f θ It has a global parameter θ, which is copied and distributed to a batch of tasks to be learned. The model uses the samples of the Support Set of each task to perform several inner gradient updates to iterate the model parameters. The loss function is calculated based on the cross entropy loss. Get the adaptation task T i The model parameters θ i ′ , the adapted model is used to calculate the cross entropy loss in the Query Set, denoted as Obtained from various tasks like "G06F" Sum and update the global model parameters f θ .

[0032]

[0033] Through the above task construction, the model can learn multiple similar binary classification problems. This method divides the model training into three stages. The first stage acts on the first level "department" of IPC. Each element represents a classification number of the first level of IPC. Similar tasks are constructed in each category of the first level, and the model is trained, denoted as θ 1 After the first stage of training, the model has learned the shared priors of each category in the first level, and given a few samples, it can adapt to the binary classification problem judgment of each category. The second stage acts on the second level "big category" of IPC. Each element represents a classification number of the second level of IPC. Based on the global model trained at the first level, it can help learn the classification of each category at the second level. Similarly, similar secondary tasks are constructed as mentioned above to train the global model. The trained model is recorded as θ 2 In the third stage, the global model trained in the second level is used on the third level “subclass” of IPC. Each element represents a classification number of the third level of IPC. Construct a three-level task and train the model. The trained model is recorded as θ 3 This training method comprehensively considers the performance of the global model after a few adaptations on each binary classification task at each level, hoping that the trained model can quickly learn the task after a few adaptations when given a new task. At the same time, these global parameters can be regarded as shared features learned by the model on the entire dataset.

[0034] In the testing phase, hierarchical classification is implemented. Given a patent, the patent is processed into a sequence of noun phrase word embeddings by the preprocessing module, denoted as sample X. The current level is denoted as l, and the candidate category set Z = {C l,j |Pred(parent(C l,j ))=1}, initially l=1, Z is all categories of IPC level 1, that is When l = 2 or 3, the IPC in the candidate set must require that the IPC of the previous layer is predicted by the model to be 1. The trained model is updated during the sampling test of each category at the first level of IPC, and X is input into each adapted model. in Predict whether X belongs to this category. Record the category with output 1. Sample test tasks in the second-level categories under the category with output 1. For example, if the first-level category A is predicted to be 1, the second-level will sample tasks in IPCs such as A01 to A24. Based on the model parameters after the first-level adaptation Update these secondary tasks again, input X again to get the category with output 1 in the secondary category, that is, the secondary category that the model believes sample X belongs to, such as A01 is predicted to be 1, and the updated model parameters are Finally, for the third-level subcategories under the second-level category to which X belongs, such as A01G, A01N, etc., the sampling task is based on the adaptation of the upper model to determine the third-level subcategory to which X belongs. According to the tree structure of IPC, classification is performed layer by layer from top to bottom to obtain the predicted IPC three-level classification results. Among them C 3,i is an IPC classification number at the third level, and Pred is the meta-model's prediction value of whether the patent belongs to a certain IPC classification number.

[0035] (3) Relevant patent search based on graph representation;

[0036] The search for related patents is a very important step in the patent application process. By determining the possible IPC classification number of the patent to be applied for, the scope of the related patent search can be narrowed down to several sub-fields. The traditional method is mainly based on keyword search, which is a very high requirement for professional knowledge background. At the same time, influenced by subjective consciousness and experience, the keywords obtained by different people have subjective deviations. This method proposes a patent similarity judgment method based on graph representation, which can remove the influence of subjective factors. It not only considers synonyms, ambiguity and other issues, but also combines the structural information of keywords.

[0037] The patent pair selection module combines the patents in the database under the predicted IPC category with the patents to be applied for into patent pairs.

[0038] Graph construction module: In patent texts, technical terms are basically nouns or noun phrases. Based on these noun phrases, the similarity between two patents can be judged. Due to the existence of polysemy and synonyms, directly judging whether the nouns that appear are the same is not enough to accurately determine the similarity. Therefore, this method first uses the BERT word vector. For two different words, if the word vectors are similar, the semantics are similar. Secondly, although the two patents may have overlapping technical terms, the technical focus and themes described are not the same. The method based on the similarity of phrase word vectors cannot solve this problem well. Therefore, this method proposes to construct an overall representation of the patent in the form of a graph. Combining phrase structure information on the basis of word embedding can more accurately judge the relevance. Specifically, a graph G = (V, E) is constructed, where V is a set of nodes, and noun phrases are nodes. E is a set of edges. As long as two noun phrases appear in the same sentence at the same time, there is an edge between the two nodes, that is, the occur function.

[0039] E={(v i ,v j )|occur(v i ,v j )=1}

[0040]

[0041] The extraction of noun phrases requires part-of-speech tagging, such as nouns, verbs, adjectives, etc., and then the syntactic dependency tree of the sentence is established. The dependency tree is used to describe the grammatical structure of the sentence. Each word in the sentence is a node of the tree, and the edges between nodes identify the grammatical relationship between words. Based on the generated syntactic dependency tree, noun phrases can be identified by the part of speech of each word and the grammatical relationship with other words. The node features in Figure G are BERT word embeddings of noun phrases.

[0042] GraphSAGE: After obtaining a graph constructed by a patent, it is necessary to use a graph neural network to learn the overall representation of the graph. GraphSAGE is a framework for graph representation learning, which aims to deal with the representation learning problem of nodes in graph data. The core idea of ​​GraphSAGE is to generate node representations by sampling and aggregating the features of the node's neighbor nodes, thereby achieving effective feature learning and prediction tasks on the entire graph, and is not limited to a specific neural network. Represents some neighbor nodes randomly sampled from node v. u represents a neighbor node of node v. Represents the characteristics of node u at the k-1th layer. k Represents an aggregate function. k is a trainable parameter of the kth layer. In the present invention, the aggregation function adopts LSTM aggregation, and the feature sequence of some neighbor nodes of node v is input into LSTM, and the aggregated features of the neighbor nodes are output. The aggregated features are spliced ​​with the features of node v, that is, the CONCAT function, and the spliced ​​features are input into a layer of fully connected neural network MLP for feature conversion, corresponding to W k , get the new features of the current node v.

[0043]

[0044] Unsupervised learning is achieved by constructing positive and negative sample pairs. Assume that (v,u) is a positive sample pair, that is, there is an edge between v and u, (v,u ′ ) is a negative sample pair, that is, v,u ′ There is no edge between them. Q is the number of negative samples, P n (v) is the negative sample distribution. The goal is to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs. Compared with the initial word embedding, the node features after aggregation learning integrate the connections between noun phrases in the document. The patent graph nodes are composed of Learning to convert

[0045] Different noun phrases have different relevance to the patent subject, so the graph representation representing the overall representation of the patent should be obtained by weighting the representations of different noun phrases. In the present invention, TF-IDF is used to calculate the weight of each noun phrase. TF-IDF is a statistical method used to measure the importance of a word in a document collection, and is commonly used in information retrieval and text mining. t,d is the frequency of word t appearing in patent d, ∑ k n k,dis the sum of the frequencies of all words in patent d. |D| is the total number of patents in the database, and |{d:t∈d}| represents the number of patents containing word t. TF represents the frequency of a word in the current document. The larger the IDF value, the stronger the word's ability to distinguish. Words with large TF-IDF values ​​have higher relevance to the topic of the document.

[0046]

[0047] TF-IDF(t,d)=TF(t,d)·IDF(t)

[0048] The overall representation h of the graph is obtained by weighting the TF-IDF of each noun phrase. G .

[0049] h G =∑ v∈V TF-IDF(v)·h v

[0050] For two patents, the cosine similarity between the overall representations of graphs G1 and G2 is calculated to measure the relevance of the two patents. The value range of cosine similarity is between -1 and 1. The closer to 1, the more similar the vectors are.

[0051]

[0052] The patents with similarity greater than the threshold λ are selected as related patents. The related patent set is

[0053] Beneficial effects:

[0054] The present invention utilizes the five-layer tree structure of the IPC classification number and adopts a meta-learning model to independently determine the 0 / 1 binary classification problem for each IPC classification number, so that the meta-model can quickly adapt and learn when facing different IPC classification tasks, thereby improving classification efficiency and accuracy. By determining the possible IPC classification of the patent to be examined layer by layer, this method significantly narrows the scope of related patent retrieval and improves the accuracy and efficiency of the retrieval. At the same time, noun phrases are used as nodes, and edge-forming graph networks are constructed using the co-occurrence of noun phrases. Node embedding and aggregation of neighboring nodes are learned in a self-supervised manner, fully tapping the structural associations between noun phrases and enhancing the representation capabilities of patent texts. The representation of the graph network can not only better reflect the correlation between patents, but also improve the accuracy of related patent retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0057] As shown in the figure, the present invention provides a graph network patent retrieval method based on meta-learning IPC classification, constructs binary classification tasks for each IPC category on a patent data set, uses a meta-learning model to allow the meta-model to learn the shared priors of these tasks, and the model can adapt to the judgment task of a specific IPC category with only a few samples; through sequential hierarchical classification from the first level of IPC downward, the IPC categories that need to be determined are identified layer by layer; a graph is constructed based on noun phrases to achieve fusion of structural information on the basis of semantics to meet the needs of accurately judging patent similarity; specific implementation steps include: preprocessing, training and use of the meta-model, and graph representation-related patent retrieval.

[0058] In this embodiment, the preprocessing specifically includes:

[0059] Step 1.1: Customize XML parsing rules to parse the title, abstract, claims, and IPC classification number data from the original XML patent data;

[0060] Step 2.1: Use Spacy to perform syntactic analysis on the title, abstract, and claims, and parse out all noun phrases, including neural stimulation and audio;

[0061] Step 3.1: Use BERT to convert the extracted noun phrases into high-dimensional word embedding vectors.

[0062] In this embodiment, ElementTree is used to parse the patent XML file to extract the title, abstract, independent claim and IPC classification number information of each patent, wherein the claims section only stores the first valid claim text as the independent claim.

[0063] In this embodiment, the training of the meta-model includes:

[0064] Step 2.1: Construct the training task. Each time, randomly select a category from IPC level 1 to 3, randomly sample K+M patents belonging to the category and K+M patents not belonging to the category, 2K samples as the support set Support Set, and 2M samples as the query set Query Set; construct the test task. For an IPC category of any level, randomly sample K patents belonging to the category and K patents not belonging to the category. 2K samples constitute the Support Set, and the Query Set consists only of test samples, that is, the number of Query Set samples is 1;

[0065] Step 2.2: Metamodel f θ The global parameter θ is copied and distributed to a batch of tasks to be learned. The model uses the samples of the support set of each task to perform several inner gradient updates on the iterative model parameters. The loss function is calculated based on the cross entropy loss. Get the adaptation task T i The model parameters θ i ′ ;

[0066] Step 2.3: Calculate the cross entropy loss of the adapted model on the query set, denoted as

[0067] Step 2.4: Combine the results obtained from each task Sum, as the total loss, update the meta-model f θ ;

[0068] Step 2.5: Continue to execute steps 2.1 to 2.4 until the model converges.

[0069] In this embodiment, in the training task, the Support Set and the Query Set constitute a 0 / 1 binary classification task T of whether the task belongs to a certain category. i , containing 2K+2M samples; a batch of tasks sampled each time during training is denoted as D train , D train ={T1,T2,…,T B}, where T i ={S,Q}, S stands for Support Set, and Q stands for Query Set. In the test task, Support Set and Query Set constitute a 0 / 1 binary classification task of this category, which contains 2K+1 samples. The task of each sampling during the test is recorded as D test , D test = {T}, where T = {S, Q}, Q={(x test ,y test )};

[0070] The training of the meta-model is divided into three stages. The first stage acts on the first level "department" of IPC. Each element represents a classification number at the first level of the IPC; the second stage acts on the "major categories" at the second level of the IPC. Each element represents a classification number at the second level of the IPC; the third stage acts on the "subclass" at the third level of the IPC. Each element represents a classification code at the third level of the IPC.

[0071] In this embodiment, the use of the metamodel includes:

[0072] Step 3.1: For the newly arrived patent, obtain the noun phrase word embedding sequence X of the patent through preprocessing;

[0073] Step 3.2: Let the current level be l, and the candidate category set Z = {C l,j |Pred(parent(C l,j ))=1}, sampling test task, where C l,j is an IPC classification number of the lth layer: for each IPC category in the candidate category set Z, 2K samples of this category are randomly selected to form the support set, and the test set is composed only of X;

[0074] Step 3.3: Update the model by gradient descent of the support set of the task sampled at the current level using the updated model of the previous layer to complete the adaptation;

[0075] Step 3.4: Feed X into each adapted model Predict whether X belongs to this category; record the category with output 1; use the subordinate IPC classification number of the category with output 1 as the candidate category set S of level l+1; modify l, l=l+1;

[0076] Step 3.5: Loop through steps 3.2-3.4 until the IPC level 3 prediction is complete and the IPC level 3 classification number inference is obtained. Among them, C 3,i An IPC classification number at level 3.

[0077] In this embodiment, the patent search related to the graph representation includes:

[0078] Step 4.1: Based on the IPC classification number predicted by the meta-model, narrow the search scope of related patents and select patents belonging to the predicted IPC classification number and the newly arrived patents to form patent pairs;

[0079] Step 4.2: Construct a graph G = (V, E) for the patent, where V is a set of nodes, and noun phrases are nodes; E is a set of edges, and as long as two noun phrases appear in the same sentence at the same time, there is an edge between the two nodes;

[0080] Step 4.3: Aggregate the representations of neighbor nodes using the LSTM aggregation function to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs in an unsupervised manner;

[0081] Step 4.4: Calculate the TF-IDF value of the nodes in the graph and use TF-IDF weighting to obtain the overall representation h of the graph G ,h G =∑ v∈V TF-IDF(v)·hv , where h v It is the representation of the node;

[0082] Step 4.5: Calculate the cosine similarity of the patent pair graph representation as the correlation index of the patent pair; add the patents with correlation greater than the threshold λ into the related patent set. Among them, Doc i For candidate potentially relevant patents, Doc self For the current patent, the Sim function represents the similarity value between two patents.

[0083] In addition, the present invention also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method are implemented.

[0084] At the same time, the present invention also provides an electronic device, including a memory and one or more processors, wherein the memory is used to store one or more programs; when the one or more programs are executed by the one or more processors, the described method is implemented.

[0085] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A graph network patent retrieval method based on meta-learning IPC classification, characterized in that: We build binary classification tasks for each IPC category on the patent dataset, and use a meta-learning model to let the meta-model learn the shared priors of these tasks. The model can adapt to the judgment task of a specific IPC category with only a few samples. We classify the IPC categories from the first level downward in order, and identify the IPC categories that need to be determined layer by layer. Constructing graphs based on noun phrases to integrate structural information on the basis of semantics to meet the needs of accurately judging patent similarity; The specific implementation steps include: preprocessing, training and use of metamodels, and patent retrieval related to graph representation.

2. According to claim 1, a graph network patent search method based on meta-learning IPC classification is characterized in that: Preprocessing includes: Step 1.1: Customize XML parsing rules to parse the title, abstract, claims, and IPC classification number data from the original XML patent data; Step 2.1: Use Spacy to perform syntactic analysis on the title, abstract, and claims, and parse out all noun phrases, including neural stimulation and audio; Step 3.1: Use BERT to convert the extracted noun phrases into high-dimensional word embedding vectors.

3. According to claim 2, a graph network patent search method based on meta-learning IPC classification is characterized in that: ElementTree is used to parse patent XML files and extract the title, abstract, independent claim and IPC classification number information of each patent. Among them, the claims part only stores the first valid claim text as the independent claim.

4. According to claim 3, a graph network patent search method based on meta-learning IPC classification is characterized in that: The training of the metamodel includes: Step 2.1: Construct the training task. Each time, randomly select a category from IPC level 1 to 3, randomly sample K+M patents belonging to the category and K+M patents not belonging to the category, 2K samples as the support set Support Set, and 2M samples as the query set Query Set; construct the test task. For an IPC category of any level, randomly sample K patents belonging to the category and K patents not belonging to the category. 2K samples constitute the Support Set, and the Query Set consists only of test samples, that is, the number of Query Set samples is 1; Step 2.2: Metamodel f θ The global parameter θ is copied and distributed to a batch of tasks to be learned. The model uses the samples of the support set of each task to perform several inner gradient updates on the iterative model parameters. The loss function is calculated based on the cross entropy loss. Get the adaptation task T i The model parameters θ i ′ ; Step 2.3: Calculate the cross entropy loss of the adapted model on the query set, denoted as Step 2.4: Combine the results obtained from each task Sum, as the total loss, update the meta-model f θ ; Step 2.5: Continue to execute steps 2.1 to 2.4 until the model converges.

5. According to claim 4, a graph network patent search method based on meta-learning IPC classification is characterized in that: In the training task, the Support Set and the Query Set constitute a 0 / 1 binary classification task T of whether the task belongs to a certain category. i , containing 2K+2M samples; a batch of tasks sampled each time during training is denoted as D train , D train ={T1,T2,…,T B }, where T i ={S,Q}, S stands for Support Set, Q stands for QuerySet, x ij That is, the sequence of noun phrases in a patent, y ij That is, the IPC classification number of this patent; in the task of constructing the test, the Support Set and the Query Set constitute the 0 / 1 binary classification task of this category, including 2K+1 samples; the task of each sampling during the test is recorded as D test , D test = {T}, where T = {S, Q}, Q={(x test ,y test )}; The training of the meta-model is divided into three stages. The first stage acts on the first level "department" of IPC. Each element represents a classification number at the first level of the IPC; the second stage acts on the "major categories" at the second level of the IPC. Each element represents a classification number at the second level of the IPC; the third stage acts on the "subclass" at the third level of the IPC. Each element represents a classification code at the third level of the IPC.

6. According to claim 5, a graph network patent search method based on meta-learning IPC classification is characterized in that: Uses of the metamodel include: Step 3.1: For the newly arrived patent, obtain the noun phrase word embedding sequence X of the patent through preprocessing; Step 3.2: Let the current level be l, and the candidate category set Z = {C l,j |Pred(parent(C l,j ))=1}, sampling test task, where C l,j is an IPC number of layer l, parent(C l,j ) represents the classification number of the first level above the IPC number, and Pred is the meta-model's prediction value on whether the patent belongs to a certain classification number: for each IPC category in the candidate category set Z, 2K samples of this category are randomly selected to form the support set, and the test set is composed of X only; Step 3.3: Update the model by gradient descent of the support set of the task sampled at the current level using the updated model of the previous layer to complete the adaptation; Step 3.4: Feed X into each adapted model Predict whether X belongs to this category; record the category with output 1; use the subordinate IPC classification number of the category with output 1 as the candidate category set S of level l+1; modify l, l=l+1; Step 3.5: Loop through steps 3.2-3.4 until the IPC level 3 prediction is complete and the IPC level 3 classification number inference is obtained. Among them, C 3,i An IPC classification number at level 3.

7. The graph network patent search method based on meta-learning IPC classification according to claim 6 is characterized in that: Patent searches related to graph representation include: Step 4.1: Based on the IPC classification number predicted by the meta-model, narrow the search scope of related patents and select patents belonging to the predicted IPC classification number and the newly arrived patents to form patent pairs; Step 4.2: Construct a graph G = (V, E) for the patent, where V is a set of nodes, and noun phrases are nodes; E is a set of edges, and as long as two noun phrases appear in the same sentence at the same time, there is an edge between the two nodes; Step 4.3: Aggregate the representations of neighbor nodes using the LSTM aggregation function to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs in an unsupervised manner; Step 4.4: Calculate the TF-IDF value of the nodes in the graph and use TF-IDF weighting to obtain the overall representation h of the graph G ,h G =∑ v∈ V TF-IDF(v)·h v , where h v It is the representation of the node; Step 4.5: Calculate the cosine similarity of the patent pair graph representation as the correlation index of the patent pair; add the patents with correlation greater than the threshold λ into the related patent set. Among them, Doc i For candidate potentially relevant patents, Doc self It is currently patented.

8. A computer storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

9. An electronic device, characterized in that: It comprises a memory and one or more processors, wherein the memory is used to store one or more programs; when the one or more programs are executed by the one or more processors, the method according to any one of claims 1 to 7 is implemented.

Images