Deep Learning-based Patent Portfolio Recommendation Method and System

Abstract

The present invention provides a patent portfolio recommendation method and system based on deep learning, which relates to the technical field of patent recommendation. This application divides the pre-acquired patent dataset into patent text data, patent relationship data, and patent structured data, and then obtains a patent text similarity matrix and patent characterization vectors from these three types of data. At the same time, the enterprise patent dataset corresponding to each enterprise is divided into real patent portfolios in chronological order. Then, a patent portfolio characterization learning model is used to learn and update the patent characterization vectors. Finally, the above real patent portfolios and updated patent characterization vectors are used to train an LSTM model, and each patent prediction score is obtained based on the optimal LSTM model after training. The patent similarity score is obtained based on the patent text similarity matrix. Finally, the patent prediction score and the patent similarity score are comprehensively considered to recommend patent portfolios for enterprises. Compared with the prior art, the present invention has higher patent portfolio recommendation accuracy and can recommend patents for enterprises in batches.

Description

Technical Field

[0001] The present invention relates to the technical field of patent recommendation, and particularly to a method and system for patent portfolio recommendation based on deep learning. Background Art

[0002] Compared with the traditional patent market circulation which mainly adopts brokerage mechanisms such as patent licensing or patent transfer, the online patent trading platform enables enterprises to conduct more direct demand communication and patent transactions, and the patent recommendation system can provide patent information and suggestions to enterprises, thereby improving the possibility of patent communication, recommendation, and transaction.

[0003] Currently, the methods of patent recommendation mainly include the content-based recommendation method that focuses on exploring the patent text content, the graph and network-based recommendation method that focuses on mining the patent-related relationships, the method of learning patent features based on a model to recommend patents for enterprises, and the comprehensive recommendation method that comprehensively considers the patent text content and patent-related relationships, etc.

[0004] However, on the one hand, the above methods all recommend single patents for enterprises, which does not meet the needs of enterprises to apply for or purchase a group of related patents most of the time; on the other hand, the patent data volume of existing enterprises is small, and the patent recommendation method based on model learning of patent features cannot solve the problem of patent sparsity existing in patent recommendation, which will lead to limited or inaccurate recommendation results. Therefore, the existing technology cannot provide accurate recommendation of patent portfolios for enterprises. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] Aiming at the deficiencies of the existing technology, the present invention provides a method and system for patent portfolio recommendation based on deep learning, which solves the problem that the existing technology cannot accurately recommend patent portfolios for enterprises.

[0007] (2) Technical Solutions

[0008] To achieve the above object, the present invention is realized through the following technical solutions:

[0009] In the first aspect, the present invention first proposes a method for patent portfolio recommendation based on deep learning, and the method includes:

[0010] Obtain a patent data set, and divide the patent data set into patent text data, patent relationship data, and patent structured data;

[0011] Obtain the patent text representation vector and the patent text similarity matrix based on the patent text data, obtain the patent relationship representation vector based on the patent relationship data, and obtain the patent structured representation vector based on the patent structured data, and splice the patent text representation vector, the patent relationship representation vector, and the patent structured representation vector to obtain the patent representation vector;

[0012] Based on the patent dataset, divide the enterprise patent dataset corresponding to each enterprise into real patent portfolios in chronological order;

[0013] After learning the patent representation vector using the patent portfolio representation learning model, obtain the updated patent representation vector, and obtain the patent portfolio representation vector based on the updated patent representation vector;

[0014] Use the real patent portfolio and the patent portfolio representation vector to train the LSTM model, obtain each patent prediction score based on the trained LSTM model, obtain the patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent prediction score and the patent similarity score.

[0015] Preferably, the patent text data includes: patent title text data; the patent relationship data includes: IPC classification number data, inventor data, and agency data; the patent structured data includes: citation times, times cited, number of patent family members, number of inventors, and number of claims.

[0016] Preferably, the obtaining the patent relationship representation vector based on the patent relationship data includes:

[0017] Obtain a triple relationship dataset by taking any three patent relationships in the patent relationship data as a triple;

[0018] Construct a patent-node relationship graph dataset based on the triple relationship dataset;

[0019] Train a multi-relationship graph convolutional neural network model based on the patent-node relationship graph dataset;

[0020] Input the patent-node relationship graph dataset into the trained multi-relationship graph convolutional neural network model to obtain the patent relationship representation vector;

[0021] The obtaining the patent structured representation vector based on the patent structured data includes:

[0022] Convert each patent structured data into a multi-dimensional structured vector, and perform zero-mean normalization on the multi-dimensional structured vector to obtain the patent structured representation vector.

[0023] Preferably, the patent portfolio characterization learning model includes a Lite Transformer layer, and the Lite Transformer layer consists of only one Transformer Encoder.

[0024] Preferably, the patent portfolio recommendation based on each of the patent prediction scores and the patent similarity scores includes:

[0025] Sum the patent prediction score and the patent similarity score to obtain the final score of each patent, and then select a preset number of patents with the top final scores as a patent portfolio for patent portfolio recommendation.

[0026] In a second aspect, the present invention also proposes a patent portfolio recommendation system based on deep learning, and the system includes:

[0027] A data acquisition and partitioning module, configured to acquire a patent dataset and partition the patent dataset into patent text data, patent relationship data, and patent structured data;

[0028] A vector acquisition and splicing module, configured to acquire a patent text characterization vector and a patent text similarity matrix based on the patent text data, acquire a patent relationship characterization vector based on the patent relationship data, and acquire a patent structured characterization vector based on the patent structured data, and splice the patent text characterization vector, the patent relationship characterization vector, and the patent structured characterization vector to obtain a patent characterization vector;

[0029] A true patent portfolio acquisition module, configured to divide the enterprise patent dataset corresponding to each enterprise into true patent portfolios in chronological order based on the patent dataset;

[0030] A patent portfolio characterization learning module, configured to learn the patent characterization vector using a patent portfolio characterization learning model to obtain an updated patent characterization vector, and acquire a patent portfolio characterization vector based on the updated patent characterization vector;

[0031] A patent portfolio recommendation module, configured to train an LSTM model using the true patent portfolio and the patent portfolio characterization vector, acquire each patent prediction score based on the trained LSTM model, acquire a patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each of the patent prediction scores and the patent similarity scores.

[0032] Preferably, the patent text data includes: patent title text data; the patent relationship data includes: IPC classification number data, inventor data, and agency data; the patent structured data includes: citation times, times cited, number of patent family members, number of inventors, and number of claims.

[0033] Preferably, the vector obtaining and splicing module obtains patent relationship representation vectors based on the patent relationship data, including:

[0034] Obtain a triple relationship data set by taking all patent relationships in the patent relationship data as a triple in any combination of three patent relationships;

[0035] Construct a patent-node relationship graph data set based on the triple relationship data set;

[0036] Train a multi-relationship graph convolutional neural network model based on the patent-node relationship graph data set;

[0037] Input the patent-node relationship graph data set into the trained multi-relationship graph convolutional neural network model to obtain patent relationship representation vectors;

[0038] The obtaining of the patent structure representation vector based on the patent structure data includes:

[0039] Convert each piece of the patent structure data into a multi-dimensional structure vector, and perform zero-mean normalization on the multi-dimensional structure vector to obtain a patent structure representation vector.

[0040] Preferably, the patent portfolio representation learning model includes a Lite Transformer layer, and the Lite Transformer layer consists of only one Transformer Encoder.

[0041] Preferably, the patent portfolio recommendation based on each of the patent prediction scores and the patent similarity scores includes:

[0042] Sum the patent prediction scores and the patent similarity scores to obtain the final score of each patent, and then select a preset number of patents with the top final scores as a patent portfolio for patent portfolio recommendation.

[0043] (III) Beneficial Effects

[0044] The present invention provides a method and system for patent portfolio recommendation based on deep learning. Compared with the prior art, the following beneficial effects are achieved:

[0045] The present invention divides the patent dataset into patent text data, patent relationship data, and patent structured data, and then obtains a patent text similarity matrix and patent representation vectors from these three types of data. At the same time, the enterprise patent dataset corresponding to each enterprise is divided into real patent portfolios in chronological order. Then, a patent portfolio representation learning model is used to learn and update the patent representation vectors. Finally, an LSTM model is trained using the above real patent portfolios and patent representation vectors, and based on the optimal LSTM model after training and combined with the patent text similarity matrix, patent portfolio recommendations are made for enterprises. The present invention can accurately recommend patent portfolios for enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0047] Figure 1 It is the overall flowchart of a patent portfolio recommendation method based on deep learning in an embodiment of the present invention;

[0048] Figure 2 It is the embodiment diagram of a patent portfolio recommendation method based on deep learning in an embodiment of the present invention;

[0049] Figure 3 It is the patent-node relationship graph constructed based on triples in an embodiment of the present invention;

[0050] Figure 4 It is the structural schematic diagram of the multi-relational graph convolutional neural network model R-GCN in an embodiment of the present invention;

[0051] Figure 5 It is the structural diagram of the Lite Transformer layer and the linear layer in an embodiment of the present invention;

[0052] Figure 6 It is the schematic diagram of patent portfolio recommendation in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described below. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0054] The embodiments of the present application provide a method and system for patent portfolio recommendation based on deep learning, which solves the problem in the prior art that it is impossible to accurately recommend patent portfolios for enterprises and realizes the purpose of efficient patent transformation.

[0055] The technical solutions in the embodiments of the present application for solving the above technical problems have the following general idea:

[0056] In order to overcome the problem that the prior art can only recommend single patents for enterprises and cannot accurately recommend patents for enterprises in the form of patent portfolios when the enterprise patent dataset is small, the present application divides the pre-obtained patent dataset into patent text data, patent relationship data, and patent structured data, and then obtains a patent text similarity matrix and patent characterization vectors from these three types of data. At the same time, the enterprise patent dataset corresponding to each enterprise is divided into real patent portfolios in chronological order; then, a patent portfolio representation learning model is used to learn and update the patent characterization vectors; finally, the above real patent portfolios and the updated patent characterization vectors are used to train an LSTM model, and each patent prediction score is obtained based on the trained optimal LSTM model, and the patent similarity score is obtained based on the patent text similarity matrix. Finally, the patent prediction score and the patent similarity score are comprehensively considered to recommend patent portfolios for enterprises. Compared with the prior art, the patent portfolio recommendation of the present invention has higher accuracy.

[0057] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0058] Embodiment 1:

[0059] In the first aspect, the present invention first proposes a method for patent portfolio recommendation based on deep learning. Refer to Figure 1-2 , the method includes:

[0060] S1. Obtain a patent dataset and divide the patent dataset into patent text data, patent relationship data, and patent structured data;

[0061] S2. Obtain patent text characterization vectors and a patent text similarity matrix based on the patent text data, obtain patent relationship characterization vectors based on the patent relationship data, and obtain patent structured characterization vectors based on the patent structured data, and splice the patent text characterization vectors, patent relationship characterization vectors, and patent structured characterization vectors to obtain patent characterization vectors;

[0062] S3. Divide the enterprise patent dataset corresponding to each enterprise into real patent portfolios in chronological order based on the patent dataset;

[0063] S4. Use the patent characterization vector to obtain an updated patent characterization vector after learning with the patent portfolio characterization learning model, and obtain a patent portfolio characterization vector based on the updated patent characterization vector;

[0064] S5. Use the real patent portfolio and the updated patent portfolio characterization vector to train an LSTM model, and obtain a predicted score for each patent based on the trained LSTM model; obtain a patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent predicted score and the patent similarity score.

[0065] It can be seen that in this embodiment, the patent dataset is divided into patent text data, patent relationship data, and patent structured data, and then a patent text similarity matrix and a patent characterization vector are obtained from these three types of data. At the same time, the enterprise patent dataset corresponding to each enterprise is divided into real patent portfolios in chronological order; then, the patent portfolio characterization learning model is used to learn and update the patent characterization vector; finally, the above real patent portfolio and the patent portfolio characterization vector are used to train the LSTM model, and based on the optimal LSTM model after training, combined with the patent text similarity matrix, patent portfolio recommendation is performed for the enterprise. This embodiment can accurately recommend patent portfolios for enterprises.

[0066] The following combines the attached Figure 1-6 , as well as the explanations of the specific steps of S1-S5, to elaborate on the implementation process of an embodiment of the present invention. The specific steps of a patent portfolio recommendation method based on deep learning in this embodiment are as follows:

[0067] S1. Obtain a patent dataset, and divide the patent dataset into patent text data, patent relationship data, and patent structured data.

[0068] Obtain a patent dataset, and classify the above dataset according to the data attribute dimension.

[0069] In this implementation, patents historically applied for or purchased by enterprises are collected from the Patsnap Innojoy database to form a patent data collection. The patent dataset has a total of 13,684 patents, that is, the total number of patents N = 13,684. The relevant information of the patents mainly includes patent publication number, title text, inventor name, IPC classification number, agency name, application date, citation times, times cited, number of patent family members, number of inventors, and number of claims, etc. The patent dataset involves a total of 501 enterprise samples, that is, the total number of enterprises U = 501. These enterprises are all listed enterprises, covering multiple industries such as technology, medicine and healthcare, chemical industry, manufacturing, food and cosmetics, transportation, and construction, and also include other enterprise basic information, industry information, and patent data.

[0070] The patent data collected above is divided into text data, relationship data, and structured data according to the attribute dimension. Among them, the patent title text is divided into text data; the IPC classification number, inventor, and agency are divided into relationship data; the citation times, times cited, number of patent family members, number of inventors, and number of claims are divided into structured data.

[0071] S2. Obtain a patent text representation vector and a patent text similarity matrix based on the patent text data, obtain a patent relationship representation vector based on the patent relationship data, and obtain a patent structured representation vector based on the patent structured data, and splice the patent text representation vector, the patent relationship representation vector, and the patent structured representation vector to obtain a patent representation vector.

[0072] S21. Process the above three types of patent data respectively to obtain corresponding representation vectors.

[0073] 1) Process the text data to obtain a patent text representation vector and a patent text similarity matrix.

[0074] 1.1) Perform text preprocessing on each title text, including word segmentation, removing special symbols, removing stop words, etc., and use word2vec to convert each processed text into a set of word vectors within the text. For the patent text data obtained after the above classification, perform text preprocessing operations such as word segmentation, removing special symbols, and removing stop words on each title text, and then use all the words involved in all the preprocessed patent texts as a corpus, and use the word2vec model to obtain the entire set of word vectors where, w i is the i-th word vector, d represents the dimension of the word vector, where d can be selected from a set D. Preferably, d can take values in the set D = {32, 64, 128, 256}, and more preferably, in this embodiment, d = 64. |W| is the number of words.

[0075] 1.2) Use TF-IDF to perform weighted summation on the set of word vectors of each text to obtain each patent text vector, thereby obtaining all patent text representation vectors, with a dimension of k1.

[0076] Calculate the TF-IDF value of each word in each patent text and use this value as the weight of each word for the patent text:

[0077]

[0078]

[0079] tfidf i,j = tfi,j ×idf i

[0080] wherein, tf i,j is the tf value of the i-th word in the j-th patent text, idf i is the idf value of the i-th word, and tfidf i,j is the TF-IDF value of the i-th word in the j-th patent text; w i,j is the number of occurrences of the i-th word in the j-th text; is the sum of the number of occurrences of the i-th word in all texts; N is the total number of patent texts, and in this embodiment, N = 13684, |{j: w i ∈ d j}| is the number of files containing the i-th word.

[0081] Then the representation vector of the j-th patent text can be expressed by the formula:

[0082]

[0083] Furthermore, the set of all patent text representation vectors is obtained N = 13684 is the number of patents, and k1 = d = 128 is the dimension of the patent text representation vector.

[0084] 1.3) Calculate the cosine similarity between any two patent text vectors in the set A of patent text representation vectors to obtain the set of patent text similarity vectors Furthermore, a patent text similarity matrix Q of N×N can be obtained synchronously.

[0085] 2) Process the patent relationship data to obtain the patent relationship representation vector.

[0086] 2.1) For the obtained patent relationship data above, first organize the IPC classification, inventor name, and agency data information of the patent into a triple form such as (Patent, Belongs_to, IPC), (Patent, Invented_by, Inventor), (Patent, Acted_by, Agency), then form a relationship data set containing all three relationship triples, and then construct a patent-node relationship graph based on the relationship data set of triples, as Figure 3 shown.

[0087] 2.2) Divide the above relationship data set into a training set, a validation set, and a test set according to the ratio of 6:2:2, and train a multi-relationship graph convolutional neural network model R-GCN to learn the representation of each patent node in the graph. The R-GCN model is as Figure 4As shown in the figure. Specifically, according to the R-GCN model, the node update formula in the patent-node relationship graph can be expressed as:

[0088]

[0089] Among them, represents the set of neighbor nodes whose relationship with node i is rel; c i,rel is a regularization constant, where c i,rel takes a value of is a linear transformation function that transforms neighbor nodes of the same type of edge using a parameter matrix ; σ is an activation function, and the relu function is adopted in this method. After the R-GCN model is trained, fine-tuned, and tested, all the data sets are input into the trained model to obtain the set of all patent relationship representation vectors Among them, k2 is the dimension of the patent relationship representation vector. Among them, k2 can take a value in a set k2 = {50, 100, 150, 200}. In this embodiment, preferably, k2 = 100.

[0090] 3) Process the structured data to obtain the patent structured representation vector.

[0091] The structured data of each patent obtained above, namely the five attributes of citation times, times cited, number of patent family members, number of inventors, and number of claims, are converted into a 5-dimensional vector, and the value on each dimension is the value of the patent on each attribute. The structured vector of each patent is normalized with zero mean on the dimension of the above attributes:

[0092]

[0093] Among them, s i,att is the result after normalization of the i-th patent on the attribute att, x i,att is the original data, μ att is the mean of the attribute att on all patents, and σ att is the variance of the attribute att on all patents. Thus, the set of patent structured representation vectors

[0094] S22. Concatenate the obtained patent text representation vector, patent relationship representation vector, and patent structured representation vector to obtain the final patent representation vector.

[0095] First, concatenate the obtained patent text representation vector and the patent relationship representation vector to obtain a patent text relationship representation vector of dimension k1 + k2. Then concatenate the patent text relationship representation vector and the structured representation vector to obtain a set of final patent representation vectors of dimension k1 + k2 + 5. In this embodiment, let k = k1 + k2 + 5. Thus, a patent representation vector matrix M of N×k can be obtained.

[0096] S3. Based on the patent dataset, divide the enterprise patent dataset corresponding to each enterprise into real patent portfolios in chronological order.

[0097] Divide all the patents applied for or purchased by each enterprise into real patent portfolios according to the application and purchase time. The data format processing, text comparison, conditional judgment, and data merging in the division process are combined with the pandas library in Python. The specific division logic is as follows: Compare the texts of the original patentee and the current patentee data of each patent in the dataset. If the two texts are judged to be the same, define the application time data of this patent as the time division basis; if the two texts are judged to be different, and only one transaction record is identified in the "matters" data of this patent, then define the time of this transaction record of this patent as the time division basis; if the two texts are judged to be different, and more than 1 transaction record is identified in the "matters" data of this patent, it means that the time of the last transaction record of this patent may not be due to real transaction matters. For example, enterprise A transfers the right of patent patent1 to enterprise B at time time1, and enterprise B transfers the right of this patent to its subsidiary enterprise B1 at time2. In the whole matter involved in this patent, the original patentee is A and the current patentee is B1, but the real patent transaction time should be time1 because the patent right transfer at time2 is due to internal enterprise matters changes and it cannot reflect the real needs of enterprise B1. Based on the above situation, it is necessary to view the subjects involved in each transaction matter from the data source manually to determine the real transaction time, and define the real transaction time as the time division basis. Thus, the patents with the same time division basis among all the patents of each enterprise are regarded as a real patent portfolio.

[0098] S4. Use the patent portfolio representation learning model to learn the patent representation vectors to obtain updated patent representation vectors, and obtain patent portfolio representation vectors based on the updated patent representation vectors, and obtain patent portfolio representation vectors based on the updated patent representation vectors.

[0099] S41. Based on the patent portfolio representation learning model, the patent representation vectors learn the patent portfolio representation to obtain updated patent representation vectors.

[0100] In the final set of patent characterization vectors P obtained above, all the patent characterization vectors corresponding to the historical patents before the next patent application or purchase by each enterprise are input into the patent portfolio characterization learning model for learning, so as to learn the interests and intentions of the enterprise in applying for or purchasing patents from the historical behavior of the enterprise.

[0101] In practice, for each enterprise, the patent transaction data is less, and the training data does not reach the amount of data required for the Transformer to achieve high performance. Therefore, in this embodiment, the Transformer is improved to a Lite Transformer layer more suitable for small data sets. The Lite Transformer layer consists of only one Transformer Encoder. The Transformer Encoder is the same as the traditional Transformer, but since the timing information is not considered, the position encoding is not added to the input vector. Use this Lite Transformer layer to learn the features of all patents before the next purchase of the enterprise, implicitly learn the interests and intentions of the enterprise in purchasing patents, and output the patent characterization vectors of the new entire patent portfolio, that is, update the patent characterization vectors. The Lite Transformer layer involved in this embodiment can first regard all the patents of the enterprise without considering the timing as a whole portfolio, and learn the potential interests and intentions of the enterprise from it.

[0102] When performing the learning of the patent portfolio characterization, let where P u ' represents the set of patent portfolio characterization vectors output by the Lite Transformer layer for the historical patents of a certain enterprise, and P u represents the set of patent characterization vectors input by the enterprise. The set of patent characterization vectors P u is composed of the patent characterization vectors of all the historical patents purchased or applied by the enterprise. n is the total number of historical patents of the enterprise, and k is the vector dimension.

[0103] S42. Obtain the patent portfolio characterization vector of each enterprise based on all the updated patent characterization vectors in each patent portfolio of each enterprise.

[0104] After using the patent portfolio characterization learning model to learn the patent characterization vectors, input all the updated patent characterization vectors in each patent portfolio of each enterprise after learning into a linear layer, multiply the output result by the input, and obtain a k-dimensional patent portfolio characterization vector. Specifically,

[0105] Divide P u ' according to the partitioning method in S3 above, and then input the partitioned vector group into a linear layer In it, a softmax activation function is added:

[0106]

[0107] Among them, m is the number of historical patent portfolios of the enterprise, and l is the number of patents in the historical largest patent portfolio. Use the output P u ″′ to multiply with P u ″ to obtain the representation of all patent portfolios of the enterprise From this, the set of representations of all historical patent portfolios of all enterprises is further obtained Among them, m u is the number of historical patent portfolios of enterprise u, U = 501 is the number of enterprises, and k is the vector dimension. In this embodiment, the Lite Transformer layer and the linear layer structure are as Figure 5 shown.

[0108] S5. Use the real patent portfolio and the patent portfolio representation vector to train the LSTM model, obtain the prediction score of each patent based on the trained LSTM model, obtain the patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent prediction score and the patent similarity score.

[0109] S51. Use the real patent portfolio and the patent portfolio representation vector to train a two-layer LSTM model, and obtain the prediction score of each patent based on the trained two-layer LSTM model.

[0110] Regard the patent portfolio representation vector of each enterprise obtained in S4 above as a time series, input it into the LSTM model to learn the time series information, take values for the number of layers of LSTM in the set F = {1, 2, 3, 4}, and in this embodiment, select F = 2, that is, there are two layers of LSTM in the LSTM model. See Figure 6 , and the last layer of LSTM outputs the hidden vector at each moment:

[0111]

[0112] Among them, is the patent portfolio representation vector of enterprise u at the t i th moment, is the hidden vector output by enterprise u at the t i th moment. Among them, X and Y are the weight parameter matrices that need to be learned in the LSTM unit. f(x) is the activation function, and the sigmoid function is selected as the activation function in this method, that is Finally, the output of enterprise u at the t i+1 th moment is obtained through the following formula, and this result is the output of enterprise u at the t i+1Recommendation probability prediction or scoring for each patent at each moment:

[0113]

[0114] Among them, |t| represents the number of moments, that is, the number of patent portfolios before the next moment; M is the patent characterization vector matrix. To make the expression more intuitive and concise, will be abbreviated as o uniformly in the following text u .

[0115] S52. Obtain the patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent prediction score and the patent similarity score.

[0116] According to the patent text similarity vector set S (corresponding to the patent text similarity matrix Q) obtained in S2, use the following formula to calculate the patent similarity score of the enterprise at the next moment:

[0117]

[0118] Among them, S u is the patent text similarity vector set of enterprise u.

[0119] Then the final score of enterprise u for all patents at the next moment is q u = o u + v u . Use the BPR ranking method to determine the objective function, that is, it is necessary to maximize the following probability:

[0120] p(u, p > p') = σ(q u,p - q u,p' )

[0121] Among them, p is the patent positive example, indicating the patent existing in the true patent portfolio of the enterprise at the next moment; p' is the patent negative example, indicating the patent not existing in the true patent portfolio of the enterprise at the next moment. Therefore, the above formula can be understood as maximizing the difference between the score of the positive example and the score of the negative example in the next patent portfolio of the enterprise. σ(x) is a non-linear function. Let Adding up all its log-likelihood terms and regularization terms, the objective function can be written in the following form:

[0122]

[0123] Among them, Θ is the parameter to be learned, and λ is the parameter to control regularization. Update and train the above formula by backpropagation until it is optimal. The last patent portfolio is used as the test set, and the rest is used as the training set. Use topk as the recommendation result, that is, recommend these topk patents as the patent portfolio that the enterprise may purchase at the next moment for the enterprise.

[0124] So far, the entire process of a patent portfolio recommendation method based on deep learning has been completed.

[0125] To verify that the patent portfolio recommendation method based on deep learning in the embodiments of the present invention has higher accuracy when making patent portfolio recommendations, this experiment selects F1, Recall, and NDCG as the indicators for evaluating the model. Among them, Recall refers to the proportion of positive examples in the original sample that are predicted correctly. F1 is the harmonic mean of accuracy and recall. NDCG is a commonly used evaluation indicator for ranking models. The closer the three indicators are to 1, the better the model performance. We verify the present application from the following three aspects.

[0126] 1) Sensitivity analysis experiment. The sensitivity analysis experiment is mainly used to test the influence of important parameters on the model and determine the final parameter configuration of the model through the experimental results. The following are the results of the sensitivity analysis experiment. Among them, k = 10 patents are selected as the recommended results. The patent text dimension k1 is set to 64, the relationship vector dimension k2 is set to 100, and the number of multi-head attention heads of Lite-Transformer is set to 2. In the sensitivity analysis results of the number of transformer encoder layers, the model performance is better when the number of encoder layers is 1, which also confirms that 1 encoder layer is more suitable for small data sets such as transaction patents.

[0127] Table 1 Influence of the number of LSTM layers on the model accuracy

[0128] Number of LSTM layers 1 2 3 4 F1 0.15521 0.166878 0.16361 0.15482 rec 0.37577 0.402934 0.39466 0.37414 ndcg 0.30907 0.32074 0.31915 0.29968

[0129] According to Table 1, when the LSTM model selects two LSTM layers, its model accuracy is the best. So finally, 2 LSTM layers are selected.

[0130] Table 2 Influence of the patent text characterization vector dimension on the model accuracy

[0131] Patent text dimension 128 64 32 16 F1 0.166878 0.16853 0.16843 0.16498 rec 0.402934 0.40241 0.40704 0.39893 ndcg 0.32074 0.32793 0.32768 0.31945

[0132] According to Table 2, when the patent text characterization vector dimension is selected as 64, the accuracy of the entire model is better.

[0133] Table 3 Influence of the patent relationship characterization vector dimension on the model accuracy

[0134]

[0135] According to Table 3, when the patent relationship characterization vector dimension is selected as 100, the accuracy of the entire model is better.

[0136] Table 4 Influence of the patent relationship characterization vector dimension on the model accuracy

[0137] Number of attention heads 1 2 4 6 8 F1 0.16853 0.17022 0.16643 0.16537 0.16614 rec 0.40241 0.408134 0.40122 0.40061 0.40076 ndcg 0.32793 0.326968 0.3196 0.3191 0.31822

[0138] As can be seen from Table 4, when the number of multi-head attention heads of Lite-Transformer is set to 2, the accuracy of the entire model is better.

[0139] Table 5 Influence of the dimension of the patent relationship representation vector on the model accuracy

[0140]

[0141]

[0142] As can be seen from Table 5, in the sensitivity analysis results of the number of transformer encoder layers, when the number of encoder layers is 1, the model effect is better and the accuracy of the entire model is better, further confirming that the number of encoder layers being 1 in the LSTM model is more suitable for small data sets such as trading patents.

[0143] 2) Ablation experiment and t-test. In order to verify the positive impact of using different categories of patent data and different model structures on the final patent portfolio recommendation results in the present invention, we prove it through ablation experiments and t-tests. See Table 6 below, where different categories of patent data are continuously added to models 1 to 6, and corresponding model parts are added until the final model is formed; "*" indicates that the p-value of this index of this model < 0.05, that is, this index of this model is significantly greater than that of the previous model, and model 4 is compared with models 2 and 3 at the same time.

[0144] Table 6 Results of ablation experiment and t-test

[0145]

[0146] As can be seen from Table 6, each part of the model involved in this embodiment can improve the model effect, is necessary, and has a positive impact on the improvement of the accuracy of the final patent portfolio recommendation.

[0147] 3) Comparison between this model and existing combined recommendation models. To verify the high precision and superiority of patent portfolio recommendation in the embodiments, two models are selected as baselines for comparative experiments in the fields of "next product portfolio" recommendation and single patent recommendation respectively. Among them, both Beacon and CLEA are the latest models in the field of "next product portfolio" recommendation in the past two years. Both of them predict the product portfolio that users may interact with at the next moment based on the user's past product portfolios, and both have good performance; HINforRec is a single patent recommendation method based on heterogeneous information networks, which mainly considers various relationships between patents and between patents and enterprises; HTW is a single patent recommendation method based on patent texts, which mainly considers patent text features. The experimental results are shown in Table 7 below.

[0148] Table 7 Comparison results between this model and existing recommendation models

[0149] Serial number Model F1@10 Recall@10 NDCG@10 1 Beacon 0.1307 0.34419 0.29481 2 CLEA 0.15132 0.27944 0.30578 3 HINforRec 0.14632 0.35402 0.16782 4 HTW 0.13874 0.29467 0.23578 5 ours 0.17022 0.40813 0.32697

[0150] According to the results shown in Table 7, this model performs best in all three indicators. The better performance of the model than Beacon and CLEA indicates that the present invention is more applicable to the portfolio recommendation of patents, a special commodity, compared with the "next portfolio" recommendation model for ordinary commodities in terms of technology; while the higher performance of the model than HINforRec and HTW indicates that traditional single patent recommendation models are difficult to accurately estimate a group of patents that an enterprise may purchase simultaneously at the next moment, which limits the possibility of many transactions. Different from this, the present invention can recommend a batch of patents that are more likely to appear in a portfolio to the enterprise.

[0151] Embodiment 2:

[0152] Second, the present invention also provides a patent portfolio recommendation system based on deep learning, which includes:

[0153] A data acquisition and division module, configured to acquire a patent data set and divide the patent data set into patent text data, patent relationship data, and patent structured data;

[0154] A vector acquisition and splicing module, configured to acquire a patent text representation vector and a patent text similarity matrix based on the patent text data, acquire a patent relationship representation vector based on the patent relationship data, and acquire a patent structured representation vector based on the patent structured data, and splice the patent text representation vector, the patent relationship representation vector, and the patent structured representation vector to obtain a patent representation vector;

[0155] A true patent portfolio acquisition module, configured to divide the enterprise patent data set corresponding to each enterprise into true patent portfolios in chronological order based on the patent data set;

[0156] A patent portfolio characterization learning module, which is used to obtain an updated patent characterization vector after learning the patent characterization vector by using a patent portfolio characterization learning model, and obtain a patent portfolio characterization vector based on the updated patent characterization vector;

[0157] A patent portfolio recommendation module, which is used to train an LSTM model by using the real patent portfolio and the patent portfolio characterization vector, obtain a predicted score for each patent based on the trained LSTM model, obtain a patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent predicted score and the patent similarity score.

[0158] Optionally, the patent text data includes: patent title text data; the patent relationship data includes: IPC classification number data, inventor data, and agency data; the patent structured data includes: citation times, times cited, number of patent family members, number of inventors, and number of claims.

[0159] Optionally, the vector acquisition and splicing module obtaining a patent relationship characterization vector based on the patent relationship data includes:

[0160] Obtaining a triple relationship data set by taking any three patent relationships in all the patent relationships in the patent relationship data as a triple;

[0161] Constructing a patent-node relationship graph data set based on the triple relationship data set;

[0162] Training a multi-relational graph convolutional neural network model based on the patent-node relationship graph data set;

[0163] Inputting the patent-node relationship graph data set into the trained multi-relational graph convolutional neural network model to obtain a patent relationship characterization vector;

[0164] The obtaining a patent structured characterization vector based on the patent structured data includes:

[0165] Converting each piece of the patent structured data into a multi-dimensional structured vector, and performing zero-mean normalization on the multi-dimensional structured vector to obtain a patent structured characterization vector.

[0166] Optionally, the patent portfolio characterization learning model includes a Lite Transformer layer, and the Lite Transformer layer is only composed of one Transformer Encoder.

[0167] Optionally, the performing patent portfolio recommendation based on each patent predicted score and the patent similarity score includes:

[0168] Sum the patent prediction score and the patent similarity score to obtain the final score for each patent, and then select a preset number of patents with the top final scores as a patent portfolio for patent portfolio recommendation.

[0169] It can be understood that the patent portfolio recommendation system based on deep learning provided by the embodiments of the present invention corresponds to the above-mentioned patent portfolio recommendation method based on deep learning. For the explanations, examples, beneficial effects, etc. of the relevant content, reference can be made to the corresponding content in the patent portfolio recommendation method based on deep learning, which will not be elaborated here.

[0170] In summary, compared with the prior art, the following beneficial effects are achieved:

[0171] 1. The present invention divides the patent dataset into patent text data, patent relationship data, and patent structured data, and then obtains the patent text similarity matrix and patent representation vectors from these three types of data. At the same time, the enterprise patent dataset corresponding to each enterprise is divided into real patent portfolios in chronological order; then the patent portfolio representation learning model is used to learn and update the patent representation vectors; finally, the above real patent portfolios and patent portfolio representation vectors are used to train the LSTM model, and based on the optimal LSTM model after training, combined with the patent text similarity matrix, patent portfolio recommendation is performed for the enterprise. The present invention can accurately recommend patent portfolios for enterprises.

[0172] 2. The present invention constructs patent features from multiple perspectives such as text, relationship, and structure, and uses methods suitable for text data, relationship data, and structure data respectively for feature extraction. Compared with the prior art, it can more effectively solve the sparsity problem of patent recommendation and make the recommendation results more accurate.

[0173] 3. The present invention selects the Lite Transformer layer as the patent portfolio representation learning model. First, all the patents of an enterprise without considering the time sequence can be regarded as a whole portfolio, and the potential interests and intentions of the enterprise can be learned from it; then the linear layer can be used to more accurately learn the vectors of each patent portfolio of the enterprise; finally, two layers of LSTM can be used to learn the time sequence information of the enterprise patent portfolio. The resulting patent portfolio recommendation results can implicitly learn the correlation between patents and the relationship between patent portfolios, making the patent portfolio recommendation results more accurate.

[0174] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

[0175] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for patent portfolio recommendation based on deep learning, characterized in that, The method includes: Obtain a patent dataset, and divide the patent dataset into patent text data, patent relationship data, and patent structured data; Based on the patent text data, obtain a patent text representation vector and a patent text similarity matrix, based on the patent relationship data, obtain a patent relationship representation vector, and based on the patent structured data, obtain a patent structured representation vector, and splice the patent text representation vector, the patent relationship representation vector, and the patent structured representation vector to obtain a patent representation vector; Based on the patent dataset, divide the enterprise patent dataset corresponding to each enterprise into real patent portfolios in chronological order; specifically: define the real transaction time as the time division basis, and regard the patents with the same time division basis among all the patents of each enterprise as a real patent portfolio; Use the patent portfolio representation learning model to learn the patent representation vector to obtain an updated patent representation vector, and obtain a patent portfolio representation vector based on the updated patent representation vector; Use the real patent portfolio and the patent portfolio representation vector to train an LSTM model, obtain a patent prediction score for each patent based on the trained LSTM model, obtain a patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent prediction score and the patent similarity score.

2. The method according to claim 1, characterized in that, The patent text data includes: patent title text data; the patent relationship data includes: IPC classification number data, inventor data, and agency data; the patent structured data includes: citation times, times cited, number of patent family members, number of inventors, and number of claims.

3. The method according to claim 1, characterized in that, The obtaining of the patent relationship representation vector based on the patent relationship data includes: Obtain a triple relationship dataset by taking any three patent relationships in all the patent relationships in the patent relationship data as a triple; Construct a patent-node relationship graph dataset based on the triple relationship dataset; Train a multi-relationship graph convolutional neural network model based on the patent-node relationship graph dataset; Input the patent-node relationship graph dataset into the trained multi-relationship graph convolutional neural network model to obtain a patent relationship representation vector; The obtaining of the patent structured representation vector based on the patent structured data includes: Convert each piece of the patent structured data into a multi-dimensional structured vector, and perform zero-mean normalization on the multi-dimensional structured vector to obtain a patent structured representation vector.

4. The method according to claim 1, characterized in that, The patent portfolio representation learning model includes a LiteTransformer layer, and the Lite Transformer layer consists of only one Transformer Encoder.

5. The method according to claim 1, characterized in that, The performing of the patent portfolio recommendation based on each patent prediction score and the patent similarity score includes: Sum the patent prediction score and the patent similarity score to obtain a final score for each patent, and then select a preset number of patents with the top final scores as a patent portfolio for patent portfolio recommendation.

6. A system for patent portfolio recommendation based on deep learning, characterized in that, The system includes: A data acquisition and division module, configured to acquire a patent dataset and divide the patent dataset into patent text data, patent relationship data, and patent structured data; A vector acquisition and splicing module, configured to acquire a patent text representation vector and a patent text similarity matrix based on the patent text data, acquire a patent relationship representation vector based on the patent relationship data, and acquire a patent structured representation vector based on the patent structured data, and splice the patent text representation vector, the patent relationship representation vector, and the patent structured representation vector to obtain a patent representation vector; A true patent portfolio acquisition module, configured to divide the enterprise patent dataset corresponding to each enterprise into true patent portfolios in chronological order based on the patent dataset; specifically: defining the true transaction time as the time division basis, and regarding the patents with the same time division basis among all the patents of each enterprise as a true patent portfolio; A patent portfolio representation learning module, configured to obtain an updated patent representation vector after learning the patent representation vector by using a patent portfolio representation learning model, and obtain a patent portfolio representation vector based on the updated patent representation vector; A patent portfolio recommendation module, configured to train an LSTM model by using the true patent portfolio and the patent portfolio representation vector, obtain a patent prediction score for each patent based on the trained LSTM model, obtain a patent similarity score based on the patent text similarity matrix, and perform patent portfolio recommendation based on each patent prediction score and the patent similarity score.

7. The system according to claim 6, characterized in that, The patent text data includes: patent title text data; the patent relationship data includes: IPC classification number data, inventor data, and agency data; the patent structured data includes: citation times, times cited, number of patent family members, number of inventors, and number of claims.

8. The system according to claim 6, characterized in that, The vector acquisition and splicing module acquiring a patent relationship representation vector based on the patent relationship data includes: Obtaining a triple relationship dataset by taking all the patent relationships in the patent relationship data as a triple in the way of any three patent relationships; Constructing a patent-node relationship graph dataset based on the triple relationship dataset; Training a multi-relationship graph convolutional neural network model based on the patent-node relationship graph dataset; Inputting the patent-node relationship graph dataset into the trained multi-relationship graph convolutional neural network model to obtain a patent relationship representation vector; The obtaining a patent structured representation vector based on the patent structured data includes: Converting each piece of the patent structured data into a multi-dimensional structured vector, and performing zero-mean normalization on the multi-dimensional structured vector to obtain a patent structured representation vector.

9. The system according to claim 6, characterized in that,The patent portfolio representation learning model includes a LiteTransformer layer, and the LiteTransformer layer is only composed of one Transformer Encoder.

10. The system according to claim 6, wherein Performing patent portfolio recommendation based on each patent prediction score and the patent similarity score includes: Sum the patent prediction score and the patent similarity score to obtain the final score of each patent, and then select a preset number of patents with the top final scores as a patent portfolio for patent portfolio recommendation.

Images