A technical efficacy matrix construction method for technical literature

Abstract

The application discloses a patent technology function extraction method for technical literature, analyzes the features of patent technology terms and function terms in the field of high-end equipment, and improves the accuracy of technology term and function term extraction in Chinese patents. The application constructs a deep learning model for technology term and function term extraction, combines the sentence pattern rules of patents, constructs the heuristic features of technology terms, positions the function sentences by constructing a function term feature dictionary, accelerates the training speed of the model, and improves the extraction accuracy. In order to reduce the cost of manual sample labeling and avoid the model overfitting phenomenon caused by too small data set, a self-training algorithm is used to realize weak supervision learning of the model. The technology terms most similar to the theme of the patent text are selected from the word clustering, the cosine similarity is used to combine the similar semantic terms, and a technology function matrix is constructed.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and patent mining. Background Technology

[0002] High-end manufacturing equipment is the cornerstone of high-quality development in the manufacturing industry, creating a huge demand for technological innovation. Accurately identifying technological development trends in relevant fields, discovering key technological opportunities, exploring innovation paths, and uncovering core patents are particularly urgent. Patent visualization transforms obscure, unstructured patent information into clearly identifiable structured data, enabling non-experts to understand the immense R&D value hidden behind patents and providing feasible ideas for subsequent research and development. The technology efficacy matrix, as a specific form of qualitative analysis in patent analysis, uniquely presents designers with the degree of technological aggregation and dispersion in the development process of relevant technical fields, helping users identify technological hotspots, weaknesses, and development paths, thus supporting decisions on subsequent innovation directions.

[0003] The main challenge in constructing a patent technology efficacy matrix lies in the extraction of technical and efficacy terms. Traditional methods for extracting these terms include rule-based methods, statistical model-based methods, and methods combining both. However, traditional methods only consider the characteristics of the terms themselves and their frequency in the target corpus, making the extraction results highly dependent on the size and quality of the target corpus. Therefore, scholars have increasingly applied machine learning and deep learning techniques to term extraction.

[0004] Term extraction using machine learning techniques typically treats terms as "named entities" within a specific domain, employing methods related to named entity recognition for term identification. However, for domain-specific technical terms, the accuracy of the training model decreases because these terms are often "out-of-vocabulary" words—comprising multiple words, uncommon, or appearing for the first time. For efficacy terms, firstly, their diverse expressions, sometimes scattered across one or more sentences, make direct extraction difficult; secondly, efficacy terms often reside within parts of the patent text, introducing significant noise and uneven sample distribution when extracting from the full patent text or abstract. Furthermore, for Chinese high-end equipment patents, there is currently no standard dataset, making manual annotation costly.

[0005] Based on the above reasons, this paper proposes the following: For technical terms, the model is trained by constructing entity features of technical terms in high-end equipment patents to improve the model extraction accuracy; For efficacy terms, the statement containing the efficacy term is first located to avoid the influence of noise information and uneven sample on model training, and the efficacy term is decomposed into subject (representing efficacy effect) and object (representing attribute) for annotation and extraction to avoid the influence of the expression form of the efficacy statement on the extraction results.

[0006] This invention proposes a technology for extracting technical efficacy terms, thereby achieving accurate extraction of patent information in the field of high-end equipment. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an automatic method for constructing a technical efficacy matrix, which addresses the shortcomings of the prior art and improves the extraction of technical terms and efficacy terms.

[0008] To address the above problems, this invention proposes a method for extracting the efficacy of patented technologies from technical documents, comprising the following steps:

[0009] Step 1: Use the tool LTP to perform word segmentation and part-of-speech tagging on technical documents. Although the expression of technical efficacy phrases in technical documents is relatively rich, they usually have relatively fixed forms. Words or phrases are the basic units of sentence analysis. Through dependency parsing, the relationships between grammatical units or word units contained in the sentence are replaced with a syntactic parsing tree. The technical efficacy target sentence is decomposed into nested and stacked combinations of phrases, emphasizing the dependency relationship between words in the sentence, and extracting word pairs with dependency relationships, such as: [extend / v, lifespan / n, VOB], etc., transforming complex and lengthy technical and efficacy terms into word combinations. The terms in the patent dataset are manually labeled using the {B,I,O} label scheme, that is, the label {B,I} represents technical and efficacy terms, and the label {O} represents non-technical efficacy terms. Construct the model training dataset and test dataset.

[0010] Step 2: Since the grammatical features of terms differ greatly, their contextual semantics are an important basis for term extraction. The semantic information contained in word vectors can enhance the generalization performance of theoretical terms and improve the extraction effect. Therefore, the skip-gram model is used to pre-train the training set labeled in Step 1 to obtain word vectors.

[0011] Step 3: Construct technical terminology extraction models and efficacy terminology extraction models respectively;

[0012] Step 3.1: Construct entity features of technical terms by using clue words before and after the technical terms and the word features of the technical terms themselves;

[0013] Step 3.2: Based on the corresponding rules existing in the statements expressing efficacy in technical literature, construct an efficacy attribute dictionary, locate efficacy statements, initially filter efficacy statements to construct efficacy terms; break down the efficacy terms of efficacy statements into subject words (such as verbs like "improve" and "enhance") and object words (such as object words and nouns like "failure rate" and "efficiency") and extract them separately; the subject words represent the words of efficacy and effect, and the object words represent the attributes that the subject words affect;

[0014] Step 3.3: Add part-of-speech tags and dependency relations to the word vectors as input to the efficacy term extraction model, and combine the technical term entity features, word vectors, part-of-speech tags, and dependency relations into a vector matrix as input to the technical term extraction model;

[0015] Step 3.4: Construct a technical terminology extraction model and a efficacy terminology extraction model based on a bidirectional long short-term memory neural network;

[0016] Step 4: Implement weakly supervised learning for the technical terminology extraction model and the efficacy terminology extraction model; use a self-training algorithm to train the model using a small amount of labeled and unlabeled data, thus performing weakly supervised learning to reduce the manual cost of labeling sample sets and avoid overfitting of the model; the specific steps are as follows:

[0017] Step 4.1: Initialize the extraction model. Here, the extraction model refers to the technical term extraction model or the efficacy term extraction model. The extraction model is trained using a small amount of labeled data.

[0018] Step 4.2: Label the unlabeled data using the trained extraction model, calculate the confidence level of each label decision using a decision algorithm, and select the reliable data with a confidence level higher than the threshold.

[0019] The standard hard-decision Viterbi algorithm is executed, which takes the state of the previous position as a premise and calculates the state with the highest probability to the current position as the optimal hidden state sequence, i.e. the survival path.

[0020] During execution, the branch metric values ​​of each sub-path are stored, that is, the branch metric values ​​of all possible paths between two adjacent nodes. Storage node x t-1 With node x t The formula for the branch metric of all sub-paths is as follows:

[0021]

[0022] in, This represents the status score of each label at position t-1. This represents the transition score between each label at position t-1 and each label at position t. The score represents the status of each label at position t-1;

[0023] Calculate the probability of each surviving sub-path being selected in path competition, and use it as the confidence score of the corresponding node label. The confidence score of the node label at position t is expressed as:

[0024]

[0025] in, For node x t-1 With node x t The surviving sub-path metric between them. For node x t-1 With node x t Metrics for each sub-path;

[0026] Step 4.3: Incrementally train the model using these reliable data and their predicted labels. After training, perform the same operation on the remaining unlabeled data, and continue this process iterating until there is no data exceeding the threshold.

[0027] Step 5: Based on the extraction model obtained in Step 4, extract technical terms and efficacy terms from the target technical documents. The extraction results are technical term vectors and efficacy term vectors, respectively.

[0028] Using terms appearing in the name as seed words, the cosine similarity is used to calculate the similarity between each technical term and the seed word. The term with the highest similarity value is selected as the technical topic term most similar to the theme. The cosine similarity formula is:

[0029]

[0030] Where A and B are the vectors corresponding to the terms, A i B i For each element in the corresponding vector, n represents the total number of elements;

[0031] Step 6: Determine terminology similarity;

[0032] An entity alignment method based on similarity theory is adopted to calculate the cosine similarity between technical topic terms and efficacy term vectors to determine the semantic similarity between phrases. Based on the magnitude of the similarity, it is determined whether the phrases have the same meaning, and words with the same semantics are merged.

[0033] Step 7: Obtain the unique technical keywords and multiple efficacy terms of the technical solutions through the above steps. Construct a technical efficacy matrix with the technical keywords as the horizontal axis, the efficacy terms as the vertical axis, and the intersection points representing the number of corresponding technical solutions.

[0034] Furthermore, the specific method of step 3.4 is as follows: both the technical term extraction model and the efficacy term extraction model include: an input layer, a hidden layer, and an output layer;

[0035] For the forward sequence of the hidden layer and hidden layer backward sequence The state and output at time t are represented as follows:

[0036]

[0037]

[0038]

[0039] Where, x t This represents the input data at time t. x represents t The weight matrix for forward propagation. express The weight matrix for forward propagation. This indicates the bias during forward propagation. y′ represents the bias during backpropagation. t This indicates that the classification label of the node is predicted through the output layer, b y This represents the bias value during the extraction process;

[0040] Given an input sequence x = {x1, x2, ..., x...} t-1 ,x t When using the term ", ...}, the input sequence for the technical term extraction model is: x = {w, s, d, c}, and the input sequence for the efficacy term extraction model is: x = {w, s, d}, where w represents word vectors, s represents part-of-speech vectors, d represents dependency relation vectors, and c represents technical term entity feature vectors.

[0041] When the labeled sequence is y = {y1, y2, ..., y...} t-1 ,y t When the sequence is labeled, the evaluation score function is expressed by the following formula:

[0042]

[0043] in, Indicates position y at t-1 t-1 Move to current position y t The score, The input sequence x = {x1, x2, ..., x...} represents the input sequence x = {x1, x2, ..., x...}. t-1 ,x t ,...} is y at position t t The score of the label, where T represents the end position of the labeled sequence;

[0044] The probability value of the predicted sequence is expressed as:

[0045]

[0046] Represents all possible sequences, y x Represent all possible labeled sequences of the input sequence x;

[0047] The sequence label output predicted by the output layer is:

[0048]

[0049] The argmax function uses the Viterbi algorithm to solve for the optimal label sequence, thus completing the model output.

[0050] Furthermore, the method for calculating the maximum probability to the current position in step 4.2 is as follows: for the input sequence x = {x1, x2, ..., x...} t-1 ,x t The output sequence is O = {O1, O2, ..., O}. t}, initial sequence state P(x i |S), transition probability sequence P(x) j |x i ) indicates from x i Transfer to x j The probability, P ij This indicates that the output at time i is O. i In the case of state x j The probability of;

[0051]

[0052] The beneficial effects of adopting the above technical solution are as follows: The present invention provides a patent-oriented technical efficacy matrix construction and system implementation, which improves the construction effect of the technical efficacy matrix in two aspects: 1) The present invention analyzes the characteristics of technical terms in patents within the field, summarizes the entity features of technical terms as heuristic features for technical term extraction models, and combines them with word vectors, parts of speech, and dependency relations to form the final vector matrix as input to the model, so as to enhance the model's learning ability and improve the accuracy of technical term extraction; 2) The present invention analyzes the syntactic rules of efficacy sentences in efficacy terms in patents within the field, constructs an efficacy attribute dictionary, locates efficacy sentences, and initially filters efficacy sentences as the dataset for efficacy term extraction, so as to reduce the impact of noisy data on model training. Attached Figure Description

[0053] Figure 1A flowchart of a patent-oriented method for extracting technical efficacy terms is provided for this invention.

[0054] Figure 2 This invention provides a structural diagram of a technical term extraction model. Specific implementation methods

[0055] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings, taking a high-end equipment gas turbine as an example. The following is for illustrative purposes only and is not intended to limit the scope of the invention.

[0056] A method for extracting the technical efficacy of a technical solution is implemented, such as... Figure 1 As shown, it includes the following steps:

[0057] Step 1: Utilize the tool LTP to perform word segmentation and part-of-speech tagging on technical documents. While the expression of technical efficacy phrases in patents is relatively diverse, they usually have relatively fixed forms. Words or phrases serve as the basic units of sentence analysis. Dependency parsing can transform the relationships between grammatical units or word units contained in a sentence into a syntactic parse tree. This allows the technical efficacy target sentence to be decomposed into nested and layered combinations of phrases, emphasizing the dependencies between words in the sentence and extracting word pairs with dependency relationships, such as: [extend / v, lifespan / n, VOB], thereby transforming complex and lengthy technical and efficacy terms into word combinations. The terms in the patent dataset are manually labeled using the {B,I,O} tagging scheme, where {B,I} represent technical and efficacy terms, and {O} represents non-technical efficacy terms. This constructs the model training dataset and test dataset. Table 1 shows an example of patent preprocessing.

[0058]

[0059] Step 2: Since the semantic information contained in word vectors can enhance the generalization performance of term extraction and improve the extraction effect in the absence of a large number of labeled pre-trained words, this invention uses the skip-gram model to pre-train word vectors and uses them as word vector features input to the model.

[0060] Step 3: Construct extraction models for technical terms and efficacy terms respectively.

[0061] Step 3.1: By analyzing a large number of relevant patents, and by using the clue words before and after the technical terms and the word features of the technical terms themselves, some features are shown in Table 2. We construct entity features of technical terms as heuristic features for the model to extract terms.

[0062] Table 2. Some characteristics of technical terms

[0063]

[0064] For example, the entity feature description of "a new energy vehicle safety early warning method and system based on artificial intelligence" is "OOL0L0OOOOOOOOOOOOOOOR0R0OR0R0".

[0065] Step 3.2: By analyzing a large number of patents in the field, and based on the rules governing the statements expressing efficacy in patents, an efficacy attribute dictionary is constructed, part of which is shown in Table 3. Furthermore, statements expressing efficacy often appear in the last sentence of a patent, and sometimes in the first sentence. Weights are assigned to the last sentence, first sentence, and middle sentences of the patent abstract from highest to lowest. Based on these weights, an initial filtering of efficacy statements is performed using the efficacy term dictionary, thus constructing a dataset for extracting efficacy terms.

[0066] Table 3. Partial content of the dictionary of efficacy attributes.

[0067]

[0068] Because the expression of efficacy statements in Chinese patents varies widely, efficacy terms cannot be directly extracted from them. Therefore, efficacy terms are broken down into subject terms and object terms for extraction. The subject term represents the efficacy value, such as "improve" or "enhance," while the object term represents the attribute that modifies the efficacy value, such as "failure rate" or "efficiency." Both are combined into efficacy terms.

[0069] Step 3.3: Add part-of-speech tags and dependency relations to the word vectors as input to the efficacy term extraction model. Combine the entity features, word vectors, part-of-speech tags, and dependency relations of technical terms to form the final vector matrix, which is then used as input to the technical term extraction model. Construct technical term and efficacy term extraction models separately.

[0070] For the sequence before and after the hidden layer and hidden layer backward sequence The state and output at time t are respectively represented as:

[0071]

[0072]

[0073]

[0074] in, It is x t The weight matrix for forward propagation. yes The weight matrix for forward propagation. It is the bias of forward propagation. It is a bias in backpropagation.

[0075] Given an input sequence x = {x1, x2, ..., x...}t-1 ,x t When the input sequence is x = {w, s, d, c}, for the technical term extraction model, the input sequence is x = {w, s, d}, and for the efficacy term extraction model, the input sequence is x = {w, s, d}, where w represents word vectors, s represents part-of-speech vectors, d represents dependency relation vectors, and c represents technical term entity feature vectors.

[0076] When the labeled sequence is y = {y1, y2, ..., y...} t-1 ,y t When , ...}, its evaluation score function is expressed by the following formula.

[0077]

[0078] in, Indicates position y at t-1 t-1 Move to current position y t The score, The input sequence x = {x1, x2, ..., x...} represents the input sequence x = {x1, x2, ..., x...}. t-1 ,x t ,...} is y at position t t The score of the tag.

[0079] The predicted sequence probability value is expressed as

[0080]

[0081] The sequence label output predicted by the output layer is

[0082]

[0083] The argmax function uses the Viterbi algorithm to solve for the optimal label sequence, thus completing the model output.

[0084] Step 4: Implement weakly supervised learning for the technical and efficacy terminology models. Using a self-training algorithm, the model is trained with a small amount of labeled and unlabeled data to perform weakly supervised learning, reducing the manual cost of labeling sample sets and avoiding overfitting. The specific steps are as follows:

[0085] Step 4.1: Initialize the extracted model by training it with a small amount of labeled data.

[0086] Step 4.2: Label the unlabeled data using the trained model, calculate the confidence level of each label decision using a decision algorithm, and select the reliable data with a confidence level higher than the threshold.

[0087] The standard hard-decision Viterbi algorithm is executed to obtain the optimal hidden state sequence, i.e., the surviving path. During execution, the branch metric values ​​of each sub-path are stored, i.e., the branch metric values ​​of all possible paths between two adjacent nodes. Storage node x t-1 With node x t The formula for the branch metric of all sub-paths is as follows:

[0088]

[0089] Calculate the probability of each surviving sub-path being selected in path competition, and use it as the confidence score for the corresponding node label. The confidence score of the node label at position t is expressed as...

[0090]

[0091] in, For node x t-1 With node x t The surviving sub-path metric between them. For node x t-1 With node x t The metrics for each sub-path between them.

[0092] Step 4.3: Incrementally train the model using these reliable data and their predicted labels. After training, perform the same operation on the remaining unlabeled data, and continue iterating until there is no data exceeding the threshold.

[0093] Step 5: Select patent technical keywords for constructing the technical efficacy matrix. A patent often contains multiple technical terms. The term that best fits the patent theme is selected as the patent technical keyword in the technical efficacy matrix. Using terms appearing in the patent title as seed terms, the cosine similarity is used to calculate the similarity between the term and the seed term. The term with the highest similarity value is selected as the patent technical keyword most similar to the patent theme to construct the technical efficacy matrix. The cosine similarity formula is:

[0094]

[0095] Here, A and B are the vectors corresponding to the terms.

[0096] Step 6: Determine terminology similarity. As described in Step 2, all word vectors have been represented. Further, vector representations of patent technology subject terms and efficacy terms are implemented. An entity alignment method based on similarity theory is used to calculate the cosine similarity between the vectors of patent technology subject terms and efficacy terms to determine the semantic similarity between phrases. Based on the magnitude of the similarity, it is determined whether phrases have the same meaning, and words with the same semantic meaning are merged.

[0097] Step 7: Obtain the unique patent technology keywords and multiple efficacy terms of the patent through the above steps. Construct a technology efficacy matrix with the patent technology keywords as the horizontal axis, efficacy terms as the vertical axis, and the intersection points representing the corresponding number of patents.

Claims

1. A method for extracting the efficacy of patented technologies from technical documents, comprising the following steps: Step 1: Use the tool LTP to perform word segmentation and part-of-speech tagging on technical documents. Through dependency parsing, the relationships between grammatical units or word units contained in the sentence are replaced with a syntactic analysis tree. The technical efficacy target sentence is decomposed into nested and stacked combinations of phrases. The dependency relationship between words in the sentence is emphasized, word pairs with dependency relationship are extracted, and complex and lengthy technical and efficacy terms are transformed into word combinations. The terms in the patent dataset are manually labeled using the {B,I,O} labeling scheme, where {B,I} represent technical and efficacy terms and {O} represent non-technical and efficacy terms. This is used to construct the model training dataset and the test dataset. Step 2: Pre-train the labeled training set from Step 1 using the skip-gram model to obtain word vectors; Step 3: Construct technical terminology extraction models and efficacy terminology extraction models respectively; Step 3.1: Construct entity features of technical terms by using clue words before and after the technical terms and the word features of the technical terms themselves; Step 3.2: Based on the corresponding rules in the statements expressing efficacy in technical literature, construct an efficacy attribute dictionary, locate efficacy statements, initially filter efficacy statements to construct efficacy terms; split the efficacy terms of efficacy statements into subject terms and object terms for extraction; the subject term represents the words of efficacy and effect, and the object term represents the attribute of the subject term. Step 3.3: Add part-of-speech tags and dependency relations to the word vectors as input to the efficacy term extraction model, and combine the technical term entity features, word vectors, part-of-speech tags, and dependency relations into a vector matrix as input to the technical term extraction model; Step 3.4: Construct a technical terminology extraction model and a efficacy terminology extraction model based on a bidirectional long short-term memory neural network; Step 4: Implement weakly supervised learning for the technical terminology extraction model and the efficacy terminology extraction model; The specific steps are as follows: Step 4.1: Initialize the extraction model. Here, the extraction model refers to the technical term extraction model or the efficacy term extraction model. The extraction model is trained using a small amount of labeled data. Step 4.2: Label the unlabeled data using the trained extraction model, calculate the confidence level of each label decision using a decision algorithm, and select the reliable data with a confidence level higher than the threshold. The standard hard-decision Viterbi algorithm is executed, which takes the state of the previous position as a premise and calculates the state with the highest probability to the current position as the optimal hidden state sequence, i.e. the survival path. During execution, the branch metric values ​​of each sub-path are stored, that is, the branch metric values ​​of all possible paths between two adjacent nodes. storage nodes With nodes The formula for the branch metric of all sub-paths is as follows: ； in, This represents the status score of each label at position t-1. This represents the transition score between each label at position t-1 and each label at position t. The score represents the status of each label at position t-1; Calculate the probability of each surviving sub-path being selected in path competition, and use it as the confidence score of the corresponding node label. The confidence score of the node label at position t is expressed as: ； in, For nodes With nodes The surviving sub-path metric between them. For nodes With nodes Metrics for each sub-path; Step 4.3: Incrementally train the model using these reliable data and their predicted labels. After training, perform the same operation on the remaining unlabeled data, and continue this process iterating until there is no data exceeding the threshold. Step 5: Based on the extraction model obtained in Step 4, extract technical terms and efficacy terms from the target technical documents. The extraction results are technical term vectors and efficacy term vectors, respectively. Using terms appearing in the name as seed words, the cosine similarity is used to calculate the similarity between each technical term and the seed word. The term with the highest similarity value is selected as the technical topic term most similar to the theme. The cosine similarity formula is: Where A and B are the vectors corresponding to the terms. For each element in the corresponding vector, n represents the total number of elements; Step 6: Determine terminology similarity; An entity alignment method based on similarity theory is adopted to calculate the cosine similarity between technical topic terms and efficacy term vectors to determine the semantic similarity between phrases. Based on the magnitude of the similarity, it is determined whether the phrases have the same meaning, and words with the same semantics are merged. Step 7: Obtain the unique technical keywords and multiple efficacy terms of the technical solutions through the above steps. Construct a technical efficacy matrix with the technical keywords as the horizontal axis, the efficacy terms as the vertical axis, and the intersection points representing the number of corresponding technical solutions.

2. The method for extracting the efficacy of patented technologies from technical documents as described in claim 1, characterized in that, Both the technical term extraction model and the efficacy term extraction model include: an input layer, a hidden layer, and an output layer; For the forward sequence of the hidden layer and hidden layer backward sequence The state and output at time t are represented as follows: ； ； ； in, This represents the input data at time t. express The weight matrix for forward propagation, express The weight matrix for forward propagation, Indicates the bias in forward propagation. Indicates the bias during backpropagation. This indicates that the classification label of the node is predicted through the output layer. This represents the bias value during the extraction process; Given an input sequence For the technical term extraction model, the input sequence is: For the efficacy term extraction model, the input sequence is: ,in, Representing word vectors, Representing part-of-speech vectors, Represents a dependency vector. Represents the feature vector of a technical term entity; When the labeled sequence is When the labeled sequence is evaluated, the evaluation score function is expressed by the following formula: ； in, Indicates position t-1 Move to current location The score, Represents the input sequence At position t Tag scores, Indicates the end position of the labeled sequence; The probability value of the predicted sequence is expressed as: ； Represents all possible sequences. Represents the input sequence All possible labeled sequences; The sequence label output predicted by the output layer is: ； in, The function uses the Viterbi algorithm to solve for the optimal labeled sequence, thus completing the model output.

3. The method for extracting the efficacy of patented technologies from technical documents as described in claim 1, characterized in that, The method for calculating the maximum probability to the current position in step 4.2 is as follows: for the input sequence The output sequence is Initial sequence state Transition probability sequence Indicates from Transferred to The probability, express Output at all times The state occurs in the following circumstances The probability of; 。

Images