A method for identifying opportunities using classification number co-occurrence and link prediction techniques.

Abstract

The application relates to a method for identifying technical opportunities by using classification number co-occurrence and link prediction, comprising the following steps: dividing patents in a certain technical field or application field into three periods, extracting classification number information of a patent set in each period, determining a classification number co-occurrence relationship, and constructing a co-occurrence network with classification numbers as nodes according to the co-occurrence relationship; extracting three kinds of features of a node pair in the co-occurrence network in the three periods, namely, a similarity index, a node attribute related to the generation of a technology link and a node pair relationship index, and splicing them into a feature vector; performing node embedding by using graph representation learning based on a graph convolutional neural network (GCN), splicing the similarity index, the node pair relationship index and the GCN-based node attribute index to form a feature vector for the next step of modeling; and constructing and training a machine learning model based on the feature index and the node connection state of the node pair in the co-occurrence network to obtain a link prediction model.

Description

Technical Field

[0001] This invention relates to a method and system for identifying technological opportunities by utilizing classification number co-occurrence and link prediction, thereby identifying potential future technological development points, and belongs to the field of technological opportunity identification. Background Technology

[0002] Technological opportunities refer to the possible or potential directions of technological progress. Technological opportunity identification is the process of mining potential technological progress from a knowledge database containing various technologies and inventions. With the significantly accelerated pace of technological change, quickly and effectively capturing technological opportunities has become an important factor in achieving technological innovation. Currently, quantitative methods for identifying technological opportunities using objective data have received widespread attention. The emergence and development of technologies such as data mining, natural language processing, and machine learning have supported quantitative research on technological opportunity identification. Quantitative research methods for technological opportunity identification mainly include morphological analysis, patent mapping, outlier detection, and link prediction. In recent years, research on identifying potential technological opportunities through link prediction has received widespread attention because it can predict possible future technological connections to identify potential technological opportunities. Link prediction technology involves using graph networks to predict possible connections between nodes in the network. Given a graph, based on information about existing node pairs, link prediction predicts possible future node connections. The process of finding missing connections in a network with technologies as nodes is the process of finding potential technological opportunities. This method mainly uses patent classification numbers (CPC, IPC, etc.) as nodes to construct a technology network because classification numbers can represent technologies in the technical field, and the edges in the network can be co-occurrence relationships or reference relationships of classification numbers. For predicted CPC links, the technological opportunity can be interpreted as follows: the technologies indicated by two CPC codes may interact in the future, resulting in a new technology applicable to the research field. Researchers can view predicted technology links as potential future technological opportunities.

[0003] Link prediction and opportunity identification methods can be divided into two categories: similarity-based methods and learning-based methods. Similarity-based methods primarily focus on the network topology, calculating the similarity between nodes using various similarity metrics; nodes with high similarity are more likely to form links in the future. Learning-based methods define link prediction as a binary classification problem, mainly by concatenating various similarity metrics into a feature vector and predicting possible future links through supervised learning.

[0004] However, both methods often overlook the impact of node attributes and node-to-node interactions on the formation of links. Whether two technical nodes will form a link depends not only on node similarity but also on the node's own attributes and the problem under study. Therefore, it is necessary to add attributes to nodes from the perspective of influencing the formation of technical links and to consider node attributes in conjunction with network topology information. Furthermore, a technical network is a complex system with intricate connections and mutual influences among various technologies. Therefore, it is also necessary to consider the impact of inter-technology interactions on the formation of technical links.

[0005] Secondly, previous studies have typically represented technological opportunities in the form of "patent classification pairs," but since the meaning of patent classifications is quite broad, the meaning of technological opportunities is not clear. Summary of the Invention

[0006] This invention aims to address the problems of missing node information and unclear meaning of technical opportunities in existing link prediction-based research on identifying technological opportunities. It provides a link prediction method that combines topological and node information, and analyzes technical opportunity solutions by extracting themes and keywords for technical nodes to support research and development. The technical solution is as follows:

[0007] A method for identifying opportunities using classification number co-occurrence and link prediction techniques includes the following steps:

[0008] S1: Divide the patents in a certain technical or application field into three time periods, extract the classification number information of the patent set in each time period, determine the co-occurrence relationship of the classification numbers, construct a co-occurrence network based on the co-occurrence relationship with the classification numbers as nodes, and the co-occurrence relationship of the classification numbers constitutes the links in the co-occurrence network;

[0009] S2: Extract three features of node pairs in the co-occurrence network for the three time periods: similarity index, node attributes related to the generation of technical links, and node pair relationship index, and concatenate them into a feature vector;

[0010] The similarity indices include local similarity, global similarity, and quasi-local similarity;

[0011] Node attributes related to the generation of the technology chain include:

[0012] The growth rate of patents in the node GR, the total number of patents in the CPC node TN, the number of patents in the CPC node in the most recent year NP, and the embedding vector of the CPC node.

[0013] Node-to-relationship metrics include:

[0014] Co-occurrence of CPC node pairs at the subclass level (CS), hierarchical similarity of CPC node pairs (TS), number of CPC node pairs with the same assignee (SA).

[0015] Node embedding is performed using graph representation learning based on graph convolutional neural networks (GCNs). The feature matrix is ​​composed of a vector formed by concatenating the GR, TN, TP, and EV of each node. A two-layer GCN is used, and the GCN model is trained by calculating the cosine similarity of nodes and using binary cross-entropy to calculate the loss. The Hadamard product operation is used to reconstruct the representation vectors of two nodes into node pairs. The similarity indicators, node pair relationship indicators, and GCN-based node attribute indicators are concatenated to form the feature vector for the next step of modeling.

[0016] S3: Based on the feature indicators and node connection states of node pairs in the co-occurrence network, construct and train a machine learning model to obtain a link prediction model. The method is as follows: select the set of node pairs that were not linked in the first time period in step S1 but became linked in the second time period as the positive samples of the training set; the negative samples of the training set consist of those node pairs that were not linked in the second time period in the first time period, and their number is consistent with that of the positive samples; the positive samples of the test set consist of node pairs that were not linked in the second time period but became linked in the third time period, and the negative samples are those node pairs that were not linked in the third time period in the second time period, and their number is consistent with that of the positive samples. By setting experiments and evaluation indicators, the link prediction model is obtained.

[0017] S4: Input the feature vectors of unlinked node pairs into the link prediction model obtained in step S3 to predict possible future links as potential technological opportunities; for the predicted node pairs, analyze the themes and keywords of the two technologies, and use the fusion of themes and keywords as a technological opportunity solution.

[0018] Furthermore, the local similarity indices include the Adamic-Adar Index (AA), Common Neighbors (CN), Hub Depressed Index (HD), and Hub Promoted Index (HP).

[0019] Jaccard Index(JC), Leicht-Holme-Newman Index(LHN), PreferentialAttachment(PA), Resource Allocation Index(RA), Index (SI), Salton index (SL).

[0020] Furthermore, the global similarity metrics include Average Commute Time (ACT), Cosine Similarity on L+ (Cos+), Katz Index (KI), and Random Walk with Restart (RWR);

[0021] Furthermore, the quasi-local similarity index is the Local Path Index.

[0022] Furthermore, the topics and keywords of the two techniques were analyzed using LDA topic modeling.

[0023] This invention constructs a joint patent classification (CPC) co-occurrence network in different time periods and extracts similarity indicators, node attributes, and node pair interaction indicators for each pair of nodes. The node pair indicator vectors and node link states of the co-occurrence network in different time periods are used to train and test a machine learning binary classification model, and the best prediction model is selected to predict possible future classification number associations. Furthermore, technical opportunity solutions are provided by analyzing the technical themes and keywords of the two CPCs in the prediction results. Compared with other methods for identifying technical opportunities based on link prediction, this invention fully integrates network topology information (node ​​similarity), node attributes, and node pair interaction relationships. It uses more complete information analysis to predict the connection patterns of CPC nodes in the network, better uncovers potential associations between technologies, and more effectively supports technology research and development by analyzing technical themes and keywords. Attached Figure Description

[0024] Figure 1 Flowchart of the present invention

[0025] Figure 2 Three types of node pair feature indicators

[0026] Figure 3 Feature Importance of RF Model Detailed Implementation

[0027] As a basic technical solution, this invention includes the following steps: S1: Divide the data into three time periods, extract the classification number information of the patent set in each time period, determine the co-occurrence relationship of the classification numbers, and construct a co-occurrence network based on the co-occurrence relationship; S2: Extract three feature indicators of node pairs in the co-occurrence network of the three time periods respectively: similarity (local similarity, global similarity, quasi-local similarity) indicators, node attributes, and node pair interaction indicators, and concatenate them into a feature vector; S3: Use four machine learning models to establish a link prediction model based on the extracted node pair feature indicators. Specifically, the set of node pairs that were not linked in the first time period in step S1 but became linked in the second time period is selected as the positive samples (label 1) of the training set. The negative samples (label 0) of the training set consist of those node pairs that were not linked in the second time period in the first time period, and their number is consistent with that of the positive samples. The positive samples in the test set consist of node pairs that were not linked in the second time period but became linked in the third time period. The negative samples are those node pairs that were not linked in the third time period in the second time period, and their number is the same as the positive samples. By setting experiments and evaluation indicators, the best prediction model is selected. S4: The feature vectors of all unlinked node pairs in the third time period of step S1 are fed into the best prediction model in step S3 to predict possible future links as potential technological opportunities. For the predicted CPC node pairs, the topics and keywords of the two technologies are analyzed using LDA topic modeling, and the fusion of topics and keywords is used as a technological opportunity solution.

[0028] This embodiment collects patent data in the field of autonomous driving from the Derwent patent database and uses the method proposed in this invention to identify potential technological opportunities in the field of autonomous driving.

[0029] 1. Construction of CPC co-occurrence network

[0030] All patent data was divided into three periods: t1 (2014-2018), t2 (2019-2020), and t3 (2021-2022). CPC co-occurrence information was collected for each period. CPC codes indicate the technical field involved in each patent. If a patent is assigned multiple CPC codes, it means that the patent is based on two or more technologies (CPCs), and the multiple CPCs in this patent constitute a co-occurrence relationship. A co-occurrence network was constructed using CPCs as nodes, and the co-occurrence relationships of CPCs constituted links in the network. The hierarchical structure of CPCs can be divided into departments, classes, subclasses, groups, and subgroups from high to low levels. Group-level CPC codes can already represent the technical field in detail. Therefore, this embodiment constructs a CPC co-occurrence network for each period using group-level CPC codes as nodes.

[0031] 2. Feature index extraction for each time period node

[0032] This embodiment extracts node pair feature indicators from three aspects: similarity (local similarity, global similarity, quasi-local similarity) indicators, node attributes, and node pair interaction indicators, such as... Figure 2 As shown in Table 1, the effectiveness of each similarity index depends on the specific problem and network topology. Therefore, we broadly select these indices as classification features. The calculation formulas for each index are shown in Table 1. Where, v x and v y Represents two distinct nodes; Γ(v x ) represents node v x The set of neighboring nodes; L + β is the pseudo-inverse of the Laplacian matrix of the network; A is the adjacency matrix, I is the identity matrix of appropriate dimension, β < 1 / λ1, where λ1 is the largest eigenvalue of A; γ is a free parameter, which is set to 1 in this embodiment; qxy is regarded as the probability that a random walker starting from vertex x will reach vertex y in steady state.

[0033] Table 1. Calculation formulas for each similarity index

[0034]

[0035]

[0036] Table 2 lists the node attributes and node pair interaction metrics related to the generation of technology links. GR measures the growth rate of patents in a specific CPC node. TN and NP measure the total number of patents and the number of patents in the most recent year for each CPC node, respectively. EV is defined as the embedding vector of each CPC node, a 20-dimensional vector based on DocVec embedding. CS represents the co-occurrence of CPC node pairs at the subclass level. TS refers to the hierarchical structural similarity of CPC pairs; specifically, if two CPCs have the same parts but different classes, the index is marked as 1; if the parts are the same, the classes are also the same, but the subclasses are different, the index is marked as 2, and so on. Finally, NA measures the number of CPC node pairs with the same assignee.

[0037] Table 2 shows the node attributes and node-to-node relationship indicators related to the generation of the technology link.

[0038]

[0039]

[0040] Link prediction is performed on the edges of the network. CS, TS, and NA represent the interaction relationships between node pairs, and they, along with various similarity metrics, can be directly used as node pair feature metrics. GR, TN, TP, and EV are node attributes that need to be reconstructed at the node pair level. To simultaneously consider node attributes and network topology, we use graph representation learning based on graph convolutional neural networks (GCN) for node embedding.

[0041] Graph representation learning, as a method for representing graph-structured data, can efficiently extract graph structure information and effectively capture complex relationships between nodes. Given a graph G = (V, E), where V (|V| = n) and E are the sets of nodes and edges, respectively, let A be the adjacency matrix and X be the feature matrix. The propagation method between layers of GCN is as follows:

[0042]

[0043] in, I n It is an identity matrix. H (l) H is the activation matrix of the l-th layer. (0) =X. W (l) This is a layer-specific trainable weight matrix. σ() represents the activation function. The feature matrix X is composed of a vector concatenated from the GR, TN, TP, and EV of each node. We employ a two-layer GCN, training the GCN model by calculating node cosine similarity and using binary cross-entropy to compute the loss. Each node is represented as a 6-dimensional embedding vector. Finally, the Hadamard product operation is used to reconstruct the representation vectors of two nodes at the node pair level.

[0044] By concatenating the similarity indicators, node-to-node relationship indicators, and GCN-based node attribute indicators, a feature vector for the next step of modeling can be formed. The three types of indicators and their dimensions are shown in Table 2.

[0045] Table 2 Node Pairs Feature Indicators

[0046]

[0047]

[0048] 3. Machine Learning Model Construction and Evaluation

[0049] For link prediction, this invention trains four machine learning models based on the feature metrics of node pairs and node connection states in the co-occurrence network: Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boosting (XGB), and Lightweight Gradient Boosting Machine (LGBM). This invention aims to predict CPC node pairs that are currently unlinked but may become linked in the future. Therefore, the selection of the training and test sets focuses on node pairs that are unlinked in the co-occurrence network in one time period but become linked in the next. Specifically, for the training set, node pairs that are unlinked in time period t1 but become linked in time period t2 are selected as positive samples (label 1). Link prediction problems typically involve an imbalanced dataset, where the number of links (label 1) in the CPC co-occurrence network is much lower than the number of unlinked nodes (label 0), resulting in highly imbalanced data. Therefore, negative sampling is used. The negative samples (label 0) of the training set consist of node pairs that are not linked in time period t1 and are the same number as the positive samples. The positive samples in the test set consist of node pairs that were not linked in time period t2 but became linked in time period t3, while the negative samples are those node pairs that were not linked in time period t2 and were the same number as the positive samples.

[0050] To evaluate the model, five performance metrics were used to measure its predictive performance. Accuracy is the percentage of correctly predicted results out of the total sample; precision measures the probability of correctly predicting a positive result; recall measures the probability that a positive result was predicted as positive; the F1 score is the harmonic mean of precision and recall; and AUC is the area under the ROC curve, with a larger area indicating better predictive performance.

[0051] Table 3 presents the evaluation results of the four machine learning classification models. The RF model achieves a better balance between precision and recall. Furthermore, the RF model has the best accuracy and also performs well in terms of AUC. Therefore, we chose the RF model for further analysis.

[0052] Table 3. Scores of each model's evaluation indicators

[0053]

[0054]

[0055] Figure 3The feature importance of the RF model is listed. It can be seen that similarity methods exhibit high feature importance. Specifically, among the top 6 features, RWR is a global similarity method, RA, PA, AA, and CN are local similarity methods, and LP is a quasi-local similarity method. This means that the network topology is very important for predicting future node associations in the network. Furthermore, CS is also considered an important feature. The importance of node attributes based on GCN is generally in the lower-middle range, but still higher than similarity indices such as JC, HD, and SI. Therefore, when predicting network links, it is necessary to consider appropriate node attributes for specific problems.

[0056] 4. Identification of technological opportunities and analysis of technological solutions

[0057] The selected RF model was used to predict future technology links based on 2,029,595 unlinked CPC pairs from 2021 to 2022. A total of 275,485 new CPC pairs were predicted. The recall rate of the RF model shows that these new CPC pairs contain the majority (84.1%) of the links for the future period. Therefore, the set of newly predicted CPC pairs can be interpreted as representing a large portion of future technology opportunities contained in a new set much smaller than the original set (2,029,595). R&D personnel can support R&D by identifying and selecting new CPC pairs suitable for enterprise development.

[0058] Natural Language Processing (NLP)-based text mining techniques can extract useful information from large-scale datasets. Latent Dirichlet Allocation (LDA) topic modeling is a widely used text mining technique that assumes each document is generated by mixing multiple topics, and each topic is generated by mixing multiple words. For predicted CPC node pairs, the LDA topic model is applied using Python's gensim library to analyze the topics and keywords in patent texts containing two CPCs. The meaning of technological opportunities is analyzed from the perspective of the interaction and fusion of the topics and keywords represented by the two CPCs, and solutions for these technological opportunities are proposed.

[0059] By following the steps above, a list of technological opportunities in a particular field can be obtained (new link CPC node pairs), and the potential for interaction and integration of technologies can be analyzed from two technological themes and keywords, providing more effective decision support for the formulation of technology development strategies.

Claims

1. A method for identifying technology opportunities using classification number co-occurrence and link prediction, comprising the following steps: S1: dividing patents in a certain technical field or application field into three periods, extracting classification number information for each period of patent set, and determining the co-occurrence relationship of classification numbers, and constructing a co-occurrence network with classification numbers as nodes according to the co-occurrence relationship, the co-occurrence relationship of classification numbers constituting the link in the co-occurrence network; S2: extracting three kinds of features of node pairs in the co-occurrence network in the three periods respectively: similarity index, node attribute related to the generation of technology link and node pair relationship index, and splicing into feature vector; The similarity index includes local similarity, global similarity and quasi-local similarity; The node attribute related to the generation of technology link includes: The growth rate GR of patents in the node, the total number TN of patents of the CPC node, the number NP of patents of the CPC node in the last year, and the embedding vector EV of the CPC node; The node pair relationship index includes: The co-occurrence situation CS of the subclass level of the CPC node pair, the hierarchical structure similarity TS of the CPC node pair, and the number SA of the same assignee of the CPC node pair; Node embedding is performed using graph convolutional neural network (GCN) based graph representation learning, and the feature matrix is composed of vectors spliced from GR, TN, TP and EV of each node; two-layer GCN is adopted, the loss is calculated by calculating the cosine similarity of nodes and using binary cross entropy to train the GCN model, and the representation vectors of two nodes are reconstructed to node pair level using Hadamard product operation; the similarity index, node pair relationship index and GCN based node attribute index are spliced to form a feature vector for the next modeling step; S3: constructing and training a machine learning model based on the feature index and node connection state of the node pair in the co-occurrence network to obtain a link prediction model, the method being: selecting a set of node pairs that are not linked in the first period of the step S1 but have generated links in the second period as positive samples of the training set; the negative samples of the training set consist of those node pairs in the first period that have no links in the second period, and the number is consistent with that of the positive samples; the positive samples of the test set consist of node pairs that are not linked in the second period but have generated links in the third period, and the negative samples are node pairs that have no links in the third period in the second period, the number being consistent with that of the positive samples, the link prediction model is obtained by setting experiments and evaluation indexes; S4: sending the feature vector of the unlinked node pair into the link prediction model obtained in the step S3 to predict the possible link in the future as a potential technology opportunity; for the predicted node pair, the themes and keywords of the two technologies are analyzed, and the fusion of the themes and keywords is taken as the technology opportunity scheme. 2.The method of identifying technology opportunities using classification number co-occurrence and link prediction of claim 1, wherein, The local similarity index includes Adamic-Adar Index (AA), Common Neighbors (CN), Hub Depressed Index (HD) and Hub Promoted Index (HP). Jaccard Index (JC), Leicht-Holme-Newman Index (LHN), Preferential Attachment (PA), Resource Allocation Index (RA), Index (SI), Salton index (SL). 3.The method of identifying technology opportunities using classification number co-occurrence and link prediction of claim 1, wherein, The global similarity indicators include Average Commute Time (ACT), Cosine Similarity on L+ (Cos+), Katz Index (KI), and Random Walk with Restart (RWR). 4.The method of identifying technology opportunities using classification number co-occurrence and link prediction of claim 1, wherein, The quasi-local similarity indicator is Local Path Index.

5. The method of identifying technology opportunities using classification number co-occurrence and link prediction of claim 1, wherein, The topics and keywords of the two technologies are analyzed using the LDA topic model.

Images