A patent technology similarity quantification method and system based on a multi-attention mechanism

Abstract

The application discloses a kind of patent technology similarity quantification method and system based on multiple attention mechanism, the method includes the following steps: (1) based on semantic correlation and distribution correlation, construct attention weight and carry out subject perspective alignment between patent abstract vocabulary and title vocabulary;(2) based on attention mechanism, carry out quantification perspective alignment between associated patent abstract vocabulary and query patent abstract vocabulary;(3) based on theme consistency and quantization consistency, construct fusion prediction model for quantifying the technical similarity between query patent and associated patent.The application constructs pseudo query patent based on theme consistency and quantization consistency, by nonlinearly connecting transformation to the semantic difference between query patent and pseudo query patent, form the semantic similarity between query patent and associated patent, realize the technical similarity quantification under public context, help to improve recall rate on the basis of ensuring accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to data mining, in particular to a patent technology similarity quantification method and system based on a multi-attention mechanism. BACKGROUND

[0002] The technology similarity quantification method is an important basis and means for patent authorization and infringement detection. The traditional technology similarity quantification method mainly includes the following two ways: (1) The first is to use the word distribution weight represented by TF-IDF and its variants between patent documents to quantify the technology similarity of patents, which helps to ensure the accuracy of quantification. However, due to the neglect of the adverse effects of semantic multi-word forms on the quantification of technology similarity based on word frequency, the recall rate of this method is not high. (2) The second is to expand the patent vocabulary based on semantic knowledge bases such as WordNet, together with the original vocabulary to form new patent technology feature words for technology similarity quantification, which helps to improve the recall rate of quantification. However, due to the inevitable semantic drift caused by the introduction of vocabulary, the accuracy rate is sacrificed to some extent. SUMMARY

[0003] The purpose of the present application is to provide a patent technology similarity quantification method and system based on a multi-attention mechanism, so as to realize the quantification of technology similarity in a public context, and help to improve the recall rate on the basis of ensuring the accuracy.

[0004] The technology scheme of the present application is a patent technology similarity quantification method based on a multi-attention mechanism, which extends and expands the traditional word frequency quantification method in two aspects: in terms of theme consistency, not only the distribution association of vocabulary is quantified, but also the semantic association based on word vectors is integrated, that is, the attention weight of the patent abstract is required to be distributed according to the semantic association of the vocabulary and the title vocabulary. Considering the huge difference in technical granularity between different patents, technology similarity should not be quantified based on document granularity, but should be quantified from the perspective of public context. Therefore, in terms of quantification consistency, the associated patent requires attention weight distribution according to the semantic similarity of the query patent vocabulary. Then, based on theme consistency and quantification consistency, a pseudo query patent is constructed, and the semantic similarity between the query patent and the associated patent is formed by nonlinear connection transformation of the semantic difference between the query patent and the pseudo query patent. This method of constructing a pseudo query patent based on the main technical features of the query patent realizes the quantification of technology similarity in a public context, and helps to improve the recall rate on the basis of ensuring the accuracy.

[0005] The method comprises the following steps:

[0006] (1) Semantic and distributional associations are constructed based on the vocabulary of the patent abstract and the vocabulary of the title to quantify the thematic consistency of patent technical features.

[0007] (1.1) A patent text training set is constructed based on different text domains of patent data, and the word vector technology is used to vectorize the vocabulary of the patent text.

[0008] (1.1.1) The patent title field and abstract field are spliced ​​together to form a text field, and data cleaning is performed. Specific methods include punctuation removal, stop word removal, and vocabulary tracing, thereby forming a patent vocabulary text set separated by specific symbols.

[0009] (1.1.2) The patent vocabulary text set is trained by a word vector tool to generate a word vector mapping function. Each word is projected into a high-dimensional vector through the mapping function.

[0010] (1.2) The probability frequency of words in the patent text forms a distribution correlation, and the semantic distance between words forms a semantic correlation. The two correlations are combined to construct the thematic consistency of the patent abstract vocabulary.

[0011] (1.2.1) The vocabulary text set of a patent is divided into a title vocabulary set W = (w1, w2, ..., w...). k ) and a summary vocabulary set U = (u1, u2, ..., u n ).

[0012] (1.2.2) The distribution correlation of patents is quantified based on the abstract vocabulary set of a patent. The quantification method is as follows:

[0013] P α = (α1, α2, ..., α) n )

[0014]

[0015] Where, α i It is a summary vocabulary u i The distribution correlation weight, i.e., the word frequency of the abstract vocabulary.

[0016] (1.2.3) Abstract vocabulary based on a patent i The semantic relevance of the patent to the title vocabulary set W is quantified using the following method:

[0017] P β = (β1, β2, ..., β) n )

[0018]

[0019] Where, β i It is a summary vocabulary ui semantic relevance weight of the abstract word u i The average of the semantic distance accumulation of each title word wj, φ is a word vector mapping function, and sim(*, *) is a semantic calculation function of the word vector.

[0020] (1.2.4) Quantify the thematic consistency of a patent based on the distribution relevance and semantic relevance of the patent, and the quantification method is as follows:

[0021] M = (m1, m2,..., m n ) = softmax(P α * P β )

[0022]

[0023] where m i is the thematic consistency weight of the abstract word, and softmax(*) is a function for normalizing the weight distribution.

[0024] (2) Attention focusing of associated patent technology words based on query patent technology features, used to construct the quantified consistency of patent technology features.

[0025] (2.1) Q = (w1, w2,..., w n ) is the word text set of the query patent, and D = (u1, u2,..., u m ) is the word text set of the associated patent.

[0026] (2.2) The quantified consistency method between the query patent word w i and the associated patent word text set D = (u1, u2,..., u m ) based on the attention mechanism is as follows:

[0027]

[0028] where score(*, *) is an association quantification function based on the attention mechanism.

[0029] (2.3) The quantified consistency matrix of the query patent Q = (w1, w2,..., w n ) and the associated patent D = (u1, u2,..., u m ) is formalized as follows:

[0030]

[0031] where a ij is the query patent word w i and the associated patent word u jQuantitative consistency weights.

[0032] (3) Based on the theme consistency and quantitative consistency, construct pseudo-query patents oriented towards the technical features of the query patent, and realize the quantification of patent technology similarity in the public context.

[0033] (3.1) Construct a semantic fusion matrix based on the topic consistency weight and quantitative consistency weight of the patent to generate pseudo-query patents for querying patents.

[0034] (3.1.1) Search for patent Q = {w1, w2, ..., w n The topic consistency weight is M. Q ={m q1 m q2 , ..., m qn}, and the associated patent D = {u1, u2, ..., u m The topic consistency weight is M. D ={m d1 m d2 , ..., m dm}

[0035] (3.1.2) Construct a semantic fusion matrix based on the topic consistency weight and quantification consistency weight of the patent. The construction method is as follows:

[0036]

[0037]

[0038] c ij =m qi *a ij *m dj

[0039] Among them, t ij It is w i and u j The semantic fusion weights, m qi It is w i Topic consistency weight, m dj is u j Topic consistency weight, a ij It is w i and u j Quantitative consistency weights.

[0040] (3.1.3) For each query patent term w i Using a semantic fusion matrix and associated patent vocabulary D = {u1, u2, ..., u} m The pseudo-query patent terminology is constructed as follows:

[0041]

[0042]

[0043]

[0044] wherein, is the query patent-oriented vocabulary w i The constructed pseudo-query patent vocabulary vector is a linear combination of the word vectors of the K D highest semantic fusion weights of the associated patents, and TOP-K D (*) is a function for selecting the highest K D semantic fusion weights for a specific query patent vocabulary.

[0045] (3.2) Determine the main technical features of the query patent and the pseudo-query patent based on the selection factor, which is used to construct a projection matrix for supervised technical relevance quantification.

[0046] (3.2.1) Construct a technical feature selection factor based on the highest K D semantic fusion weights of the associated patent vocabulary, and the construction method is as follows:

[0047]

[0048]

[0049] wherein, r i is the highest K i semantic fusion weight cumulative sum of the query patent-oriented vocabulary w D

[0050] (3.2.2) Construct a patent technology similarity feature matrix based on the technical feature selection factor, the query patent feature expression and the pseudo-query patent feature expression, and the construction method is as follows:

[0051]

[0052]

[0053]

[0054] wherein, is a technology feature matrix splicing function.

[0055] (3.2.3) The patent technology similarity feature matrix is connected to the technology similarity label value through multiple transformation layers and activation layers, and a patent technology similarity discrimination model is formed through parameter training.

[0056] ​A patent technology similarity quantification system based on a multi-attention mechanism, which adopts the above-mentioned patent technology similarity quantification method based on a multi-attention mechanism, comprises the following modules:

[0057] A technical feature generation module: through the semantic association of the alignment of the abstract vocabulary and the title vocabulary, the distribution association of the word frequency distribution of the patent vocabulary text set, the construction of the subject consistency of the patent technology features by fusing the semantic association and the distribution association, and the realization of the quantitative consistency of the patent technology features by aligning the associated patent vocabulary with the query patent vocabulary;

[0058] A projection feature generation module: the subject consistency and the quantitative consistency form a fused semantic matrix, which is used to generate a patent technology feature selection factor, construct a pseudo query patent for the query patent technology features, and generate a feature projection matrix of the technology similarity;

[0059] A model training and prediction module: the feature projection matrix is connected with multiple connection layers and activation layers, and then connected with an output layer to form an output prediction, which forms a residual with a technology similarity label, adjusts the attention function weight in reverse, trains and forms a corresponding discrimination model, and is used for the prediction of the patent technology similarity.

[0060] A computer storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the above-mentioned patent technology similarity quantification method based on a multi-attention mechanism.

[0061] A computer device comprising a storage, a processor, and a computer program stored on the storage and executable on the processor, wherein the processor implements the above-mentioned patent technology similarity quantification method based on a multi-attention mechanism when executing the computer program.

[0062] Advantages: Compared with the prior art, the present application has the following advantages: the present application constructs a pseudo query patent based on the subject consistency and the quantitative consistency, and forms a semantic similarity discrimination model between the query patent and the associated patent by performing a nonlinear connection transformation on the semantic difference between the query patent and the pseudo query patent. The present application has the characteristics of generating significant correlation features based on the attention mechanism to construct a pseudo query patent and a feature projection matrix, effectively filtering the interference and influence of noise and non-relevant semantics on the technology similarity discrimination model. At the same time, the construction of the pseudo query patent based on the main technical features of the query patent realizes the quantification of the technology similarity in the public context, which helps to improve the recall rate on the basis of ensuring the accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 A step flowchart of the method of the present application;

[0064] Figure 2 A functional module diagram of the system of the present application. Detailed Implementation

[0065] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0066] like Figure 1 As shown, this embodiment combines query patent X and associated patent Y to specifically explain the patent technology similarity quantification method based on the multi-attention mechanism. The specific steps are as follows:

[0067] (1) The abstract and title words of a patent are used to construct the semantic and distributional relevance of the patent, and the weights of the two relevances are combined to form thematic consistency of the patent's technical features.

[0068] (1.1) A patent text dataset is constructed in the patent text domain of the patent database, and the patent text vocabulary is vectorized using word vector technology.

[0069] (1.1.1) The title field and abstract field of a patent are concatenated to form a patent text field, and all patent text fields constitute a patent text dataset.

[0070] (1.1.2) The patent text dataset was cleaned by removing punctuation marks, formulas and stop words, and using Stanford Natural Language Processing tools to trace the origins of the words, forming a patent vocabulary text set with spaces as delimiters.

[0071] (1.1.3) Use word vector tools such as GloVe or word2vec to train the patent vocabulary text set and generate a high-dimensional implicit space. Each word is mapped by a function to become a vector in the high-dimensional implicit space.

[0072] (1.2) The probability frequency of words in the patent text constitutes the distribution correlation, the semantic distance between words constitutes the semantic correlation, and the integration of distribution correlation and semantic correlation forms the thematic consistency of the patent abstract vocabulary.

[0073] (1.2.1) The vocabulary text set of a patent contains a title vocabulary set W = (w1, w2, ..., w k ) and a summary vocabulary set U = (u1, u2, ..., u n ).

[0074] (1.2.2) The distribution correlation of a patent is quantified by the vocabulary set of its abstract. The quantification method is as follows:

[0075] P α = (α1, α2, ..., α) n )

[0076]

[0077] where, α i is the distribution relevance weight of abstract vocabulary u i , i.e. abstract vocabulary word frequency.

[0078] (1.2.3) The semantic relationship between the abstract vocabulary u i of a patent and the title vocabulary set W forms the semantic relevance of the patent, and the quantification method is as follows:

[0079] P β = (β1, β2,..., β n )

[0080]

[0081]

[0082] where, β i is the semantic relevance weight of abstract vocabulary u i , i.e. the average of the semantic distance cumulative sum between abstract vocabulary u i and each title vocabulary w j , φ(*) is a word vector mapping function, and cos_sim(*) is a cosine similarity calculation function.

[0083] (1.2.4) The fusion of distribution relevance and semantic relevance of a patent forms the subject consistency of the patent, and the fusion method is as follows:

[0084] M = (m1, m2,..., m n ) = softmax(P α *P β )

[0085]

[0086] where, m i is the subject consistency weight of abstract vocabulary, is the exponential function with base e.

[0087] (2) Attention focusing of associated patent Y technical vocabulary based on query patent X technical features, used to construct the quantitative consistency of patent technical features.

[0088] (2.1) Q = (w1, w2,..., w n ) is the vocabulary text set of query patent X, and D = (u1, u2,..., u m ) is the vocabulary text set of associated patent Y.

[0089] (2.2) Based on the attention mechanism, the vocabulary w iand the vocabulary text set D = (u1, u2,..., u m ) in the associated patent Y, the quantization consistency method is as follows:

[0090] A i =(a i1 , a i2 ,..., a im )

[0091] wherein a ij represents the quantization consistency weight of the vocabulary w i in the query patent X and the vocabulary u j in the associated patent Y.

[0092]

[0093] wherein score(*, *) is an association quantization function based on attention mechanism.

[0094]

[0095] wherein w and u are query patent vocabulary and associated patent vocabulary respectively, W and U are feature transformation matrices, b is a bias, and v is a scoring matrix, wherein W, U, b and v are all learnable parameters.

[0096] (2.3) The quantization consistency matrix of the query patent X and the associated patent Y is formalized as follows:

[0097]

[0098] (3) Based on the theme consistency and the quantization consistency, the pseudo query patent for the technical features of the query patent X is constructed to realize the quantization of the patent technology similarity in the public context.

[0099] (3.1) Based on the theme consistency weight and the quantization consistency weight of the patent, the semantic fusion matrix is constructed to generate the pseudo query patent for the query patent X.

[0100] (3.1.1) The theme consistency weight of the query patent X is M Q ={m q1 , m q2 ,..., m qn}, and the theme consistency weight of the associated patent Y is M D ={m dd1 , m d2 ,..., m dm}.

[0101] (3.1.2) Based on the theme consistency weight and the quantization consistency weight of the patent, the semantic fusion matrix is constructed, and the construction method is as follows:

[0102]

[0103] wherein t ij is the semantic fusion weight of w i and u j .

[0104]

[0105] c ij = m qi *a ij *m dj

[0106] wherein m qi is the topic consistency weight of w i , m dj is the topic consistency weight of u j , and a ij is the quantization consistency weight of w i and u j .

[0107] (3.1.3) For each word in the query patent X, a pseudo query patent word vector is constructed using the semantic fusion matrix and the word vector in the associated patent Y, in the following manner:

[0108]

[0109]

[0110] wherein, is the pseudo query patent word vector constructed for the word w i in the query patent X, and is the linear combination of the word vectors of the K D highest semantic fusion weight words in the associated patent Y.

[0111]

[0112] wherein TOP-K D (*) is a function for selecting the K i highest semantic fusion weight for the word w D in the query patent X.

[0113] (3.2) Based on the selection factor, the main technical features of the query patent X and the pseudo query patent are determined, which are used to construct a projection matrix for supervised technical relevance quantization.

[0114] (3.2.1) Based on the K D highest semantic fusion weight, the projection matrix is constructed in the following manner:The semantic fusion weight of the associated patent vocabulary is used to construct a technical feature selection factor, and the construction method is as follows:

[0115]

[0116]

[0117] wherein r i is the query patent vocabulary w i The highest K D The cumulative sum of the semantic fusion weight.

[0118] (3.2.2) Based on the technical feature selection factor, the query patent feature expression and the pseudo query patent feature expression, a patent technology similarity feature matrix is constructed, and the construction method is as follows:

[0119]

[0120]

[0121]

[0122] wherein, is a technical feature matrix splicing function.

[0123] (3.2.3) Based on whether there is a technical semantic similarity between the query patent X and the associated patent Y, a two-dimensional label value of the patent technology similarity is constructed.

[0124] (3.2.4) The patent technology similarity feature matrix is sequentially subjected to feature conversion by a transformation layer, an activation layer and a full connection layer, and then connected to the patent technology similarity two-dimensional label for parameter training, to form a patent technology similarity discrimination model.

Claims

1. A patent technology similarity quantification method based on a multi-attention mechanism, characterized in that, Includes the following steps: (1) Semantic and distributional associations are constructed based on patent abstract vocabulary and title vocabulary to quantify the thematic consistency of patent technical features; (2) Focus attention on related patent technology terms based on the query patent technology features to build quantitative consistency of patent technology features; (3) Based on topic consistency and quantitative consistency, construct pseudo-search patents oriented towards the technical features of the searched patents to realize the quantification of patent technology similarity in the public context. Step (1) is as follows: (1.1) Construct a patent text training set based on different text domains of patent data, and use word vector technology to vectorize the vocabulary of the patent text; (1.1.1) The patent title field and abstract field are concatenated to form a text field, and data cleaning is performed. The processing methods include punctuation removal, stop word removal, and vocabulary tracing, thereby forming a patent vocabulary text set separated by specific symbols. (1.1.2) The patent vocabulary text set is trained by a word vector tool to generate a word vector mapping function. Each word is projected into a high-dimensional vector through the mapping function; (1.2) The probability frequency of words in the patent text forms a distribution correlation, the semantic distance between words forms a semantic correlation, and the two correlations are combined to construct the thematic consistency of the patent abstract vocabulary; (1.2.1) The vocabulary text set of a patent is divided into a title vocabulary set W = (w1, w2, ..., w k ) and a summary vocabulary set U = (u1, u2, ..., u n ); (1.2.2) The distribution correlation of patents is quantified based on the abstract vocabulary set of a patent. The quantification method is as follows: P α ＝(α1,α2,…,α n ) Where, α i It is a summary vocabulary u i The distribution correlation weight, i.e., the word frequency of the abstract vocabulary; (1.2.3) Abstract vocabulary based on a patent i The semantic relevance of the patent to the title vocabulary set W is quantified using the following method: P β ＝(β1,β2,…,β n ) Where, β i It is a summary vocabulary u i The semantic relevance weight, i.e., the summarization vocabulary u i With each title word w j The mean of the accumulated semantic distances, φ is a word vector mapping function, and sim(*,*) is a semantic calculation function for word vectors; (1.2.4) The topic consistency of a patent is quantified based on its distributional and semantic relevance. The quantification method is as follows: M=(m1,m2,…,m n )=softmax(P α *P β ) Where, m i It represents the topic consistency weights of the abstract vocabulary, and softmax(*) is a function that normalizes the weight distribution; Step (2) is as follows: (2.1)Q=(w1,w2,…,w n D is the vocabulary text set for querying patents, where D = (u1, u2, ..., u...). m ) is a collection of related patent terminology; (2.2) Calculating patent terminology based on attention mechanism w i And the associated patent terminology text set D = (u1, u2, ..., u m The quantitative consistency method between them is as follows: A i =(a i1 ,a i2 ,…,a im ) Here, score(*,*) is an attention-based correlation quantization function; (2.3) Query patent Q = (w1, w2, ..., w n ) and related patents D=(u1,u2,…,u m The quantization consistency matrix is ​​formalized as follows: Among them, a ij It is a search for patent terminology. i Related patent terminology u j Quantitative consistency weights.

2. The patent technology similarity quantification method based on a multi-attention mechanism according to claim 1, characterized in that, Step (3) specifically involves: (3.1) Construct a semantic fusion matrix based on the topic consistency weight and quantitative consistency weight of the patent to generate pseudo-query patents for querying patents; (3.1.1) Search for patent Q = {w1, w2, ..., w n The topic consistency weight is M. Q ={m q1 ,m q2 ,…,m qn }, and the associated patent D = {u1, u2, ..., u m The topic consistency weight is M. D ={m d1 ,m d2 ,…,m dm }; (3.1.2) Construct a semantic fusion matrix based on the topic consistency weight and quantification consistency weight of the patent. The construction method is as follows: c ij =m qi *a ij *m dj Among them, t ij It is w i and u j The semantic fusion weights, m qi It is w i Topic consistency weight, m dj is u j Topic consistency weight, a ij It is w i and u j Quantitative consistency weights; (3.1.3) For each query patent term w i Using a semantic fusion matrix and associated patent vocabulary D = {u1, u2, ..., u} m The pseudo-query patent terminology is constructed as follows: in, It is for searching patent terminology. i The constructed pseudo-query patent term vector is the K with the highest semantic fusion weight among the associated patents. D Linear combination of word vectors of 10 words, TOP-K D (*) is a function used to select the highest K patent terms for a specific query. D Each semantic fusion weight; (3.2) Based on the selection factor, the main technical features of the query patent and the pseudo query patent are determined and used to construct the projection matrix for supervised technical correlation quantification; (3.2.1) Based on the highest K D The semantic fusion weights of related patent terms are used to construct selection factors for technical features, and the construction method is as follows: Where, r i It is for searching patent terminology. i The highest K D The sum of the cumulative semantic fusion weights; (3.2.2) Based on the technical feature selection factor, the expression of query patent features, and the expression of pseudo-query patent features, a patent technology similarity feature matrix is ​​constructed as follows: in, It is a function for concatenating technical feature matrices; (3.2.3) The patent technology similarity feature matrix is ​​connected to the technology similarity label value through multiple transformation layers and activation layers, and a patent technology similarity discrimination model is formed through parameter training.

3. A patent technology similarity quantification system based on a multi-attention mechanism, wherein the system employs a patent technology similarity quantification method based on a multi-attention mechanism as described in any one of claims 1-2, characterized in that, Includes the following modules: Technical Feature Generation Module: By aligning summary words with title words to focus on semantic associations, the frequency distribution of patent terminology text sets forms distributional associations, and the integration of semantic associations and distributional associations constructs thematic consistency of patent technical features. By associating patent terminology with and querying patent terminology, quantitative consistency of patent technical features is achieved. Projection feature generation module: Thematic consistency and quantification consistency form a fused semantic matrix, which is used to generate patent technology feature selection factors. Pseudo-query patents are constructed for query patent technology features, which are used to generate feature projection matrices for technology similarity. Model training and prediction module: The feature projection matrix is ​​connected to multiple connection layers and activation layers in turn, and then connected to the output layer to form the output prediction. The residual is formed with the technology similarity label. The attention function weights are adjusted in reverse to train and form the corresponding discrimination model for predicting the similarity of patent technologies.

4. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a patent technology similarity quantification method based on a multi-attention mechanism as described in any one of claims 1-2.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a patent technology similarity quantification method based on a multi-attention mechanism as described in any one of claims 1-2.

Images