Methods for predicting future technological knowledge flows

Abstract

The present invention discloses a method for predicting the future flow of technological knowledge, including: step 1, obtaining the authorized patents and their technical classifications in each time period, as well as the cited patent documents of each patent; step 2, aggregating the citations between the patents into the flow between technologies, and constructing the technical knowledge flow TKF network of each year; step 3, calculating the patent data to obtain the initial feature vectors of the diffusion ability and absorptive ability of the technical field in each time period; step 4, inputting the initial feature vectors of the technology in each time period into the technology flow prediction framework, obtaining the feature vectors of the technology diffusion ability and the feature vectors of the absorptive ability in the next time period, and matching the diffusion ability and absorptive ability of different technologies to obtain the final flow trend between technologies. The method can efficiently and accurately predict the interactive relationship and development trend between future technical fields.

Description

Technical Field

[0001] The present invention relates to the field of big data analysis technology, and in particular, to a method for predicting future technical knowledge flows. Background Art

[0002] Technological Knowledge Flow (TKF) refers to the directional flow of knowledge from one technical field to another, reflecting an interaction between two different technical fields. Under the trend of economic globalization, many high-tech enterprises are facing severe innovation pressure. In order to maintain their competitive advantage, they must grasp the trend of technological development and keep pace with the times. In this regard, an effective method is to predict the flow between different technical fields in the future, so as to grasp the trend of technological development and establish an effective research and development strategy.

[0003] TKF analysis is often conducted in large-scale scientific literature, in which patent mining plays an important role. In the field of patents, each patent often cites already published patents, and when each patent application is filed, one or more technical classification codes are assigned according to the patent classification system. Taking the most commonly used Common Patent Classification (CPC) system as an example, each classification code can be regarded as a technical field, and different technologies can be connected in two ways, one is the inherent hierarchical structure of the CPC classification, and the other is through patent citations. Therefore, with the help of the CPC classification system and patent citations, we can construct an annual technical knowledge flow map and predict future technical knowledge flow trends. Summary of the invention

[0004] The purpose of the present invention is to provide a method for predicting the future flow of technological knowledge, which method can efficiently and accurately predict the interactive relationship and development trend between future technological fields.

[0005] In order to achieve the above object, the present invention provides a method for predicting future technical knowledge flow, the method comprising:

[0006] Step 1: Obtain the authorized patents and their technical classifications in each time period, as well as the cited patent documents of each patent;

[0007] Step 2: Aggregate the citations between the patents into technology flows, and construct a technical knowledge flow TKF network for each year;

[0008] Step 3: Calculate the patent data to obtain the initial feature vectors of diffusion capacity and absorption capacity of the technical field in each time period;

[0009] Step 4: Input the initial feature vector of the technology in each of the above time periods into the technology flow prediction framework to obtain the feature vector of technology diffusion capability and the feature vector of absorption capability in the next time period, and match the diffusion capability and absorption capability of different technologies to obtain the final flow trend between technologies.

[0010] Preferably, the initial feature vector d of technology diffusion capability in step 3 includes:

[0011] Technology growth rate, which is the ratio of the number of licenses for a technology to the total number of licenses for that technology in the past few years;

[0012] The amount of technological diffusion, that is, the amount of knowledge contained in a technology that spreads to other technological fields;

[0013] The technology diffusion rate is the ratio of the diffusion of a technology to the total diffusion of all technologies.

[0014] Preferably, the initial feature vector a of the technology absorptive capacity in step 3 includes:

[0015] Technology growth rate, which is the ratio of the number of licenses for a technology to the total number of licenses for that technology in the past few years;

[0016] Technology absorption, that is, the amount of knowledge absorbed by a technology from other technological fields;

[0017] The technology absorption rate is the ratio of the absorption of one technology to the total absorption of all technologies.

[0018] Preferably, step 4 comprises:

[0019] The initial feature vector d of the technology diffusion capability and the initial feature vector a of the technology absorption capability in each time period are input into the high-order interaction module HOI of the technology flow prediction framework to calculate the diffusion capability feature vector d after aggregating the surrounding high-order neighbors in the technology field. o and the absorptive capacity characteristic vector a o ;

[0020] The feature vector d o and a o Input the hierarchical transfer module, and obtain the characteristic vectors of technology diffusion capability and absorptive capability that contain the hierarchical structure information unique to the technology system through calculation;

[0021] Input the characteristic vectors of technology diffusion capacity and absorptive capacity of each time period into the technology flow tracking module to obtain the characteristic vectors of diffusion capacity and absorptive capacity of the technology field in the next time period;

[0022] Match the diffusion capacity and absorptive capacity of the technology in the future time period to obtain the flow probability between two different technological fields;

[0023] The obtained probability and true value are input into the loss function for gradient update.

[0024] Preferably, the calculation formula of the high-order interaction module in step 4 is:

[0025]

[0026]

[0027] in, and Representation Technology i The diffusion and absorption embedding representations of the k-th layer neighbors, and Represents technology U i The outflow and inflow neighbor sets; after aggregating the high-order neighbor features of the K layer, the updated technology diffusion capability representation is obtained and absorptive capacity

[0028]

[0029]

[0030] Among them, α k ≥0 indicates the importance of the k-th layer neighbors, that is, the k-th layer neighbors are important for technology U i impact.

[0031] Preferably, the hierarchical transfer module in step 4 includes:

[0032] The upward convergence module is used to aggregate the information of the lower-level nodes to the upper-level nodes. This module uses the attention mechanism to automatically learn the weights of the lower-level nodes to indicate the importance of different lower-level nodes. The formula is as follows:

[0033]

[0034]

[0035]

[0036] Among them, w ij represents the weight of the lower layer node j relative to the upper layer node i, is the transformation matrix, is the offset vector, |g i | is the upper technology node g iThe number of all underlying technology nodes included, β ij is the standardized weight, is the final representation of the upper-level node i;

[0037] The downward update module is used to update the upper node The information is updated to the lower nodes The calculation formula is as follows:

[0038]

[0039] in, is the updated representation of the lower layer nodes, is the transformation matrix, is the bias vector and σ is the activation function.

[0040] Preferably, in the technical flow tracking module of step 4, a bidirectional long short-term memory model BiLSTM is used for modeling dynamic features, and the specific formula is as follows:

[0041] The diffusion characteristics d in the continuous time period T are d = [d 1 , d 2 , ..., d T ] Input BiLSTM,

[0042]

[0043]

[0044]

[0045] d t★ =h t

[0046] Get the diffusion characteristics in the T+1 time period

[0047] According to the above technical scheme, the present invention first extracts the initial diffusion characteristics and absorption characteristics of each technical field in a continuous time period, including the growth rate of technology, the diffusion amount and absorption amount of technology, and the diffusion rate and absorption rate of technology; then the initial diffusion characteristics and absorption characteristics are input into the preset technology flow framework, and these characteristics are sequentially passed through the high-order interaction module and the hierarchical transmission module of technology to obtain the characteristic vector of technology in diffusion and absorption in each time period, and then the diffusion capacity vector and the absorption capacity vector of technology in each time period are input into the technology flow tracking module to obtain the diffusion capacity and absorption capacity vector of each technical field in the future time period; finally, the diffusion capacity and absorption capacity of technology are matched to obtain the flow probability between two different technical fields. The hierarchical structure properties of the existing patent classification system and the high-order flow between technologies are fully utilized, and the method of graph neural network is used to construct a technology flow network based on patent citations, and the future technology flow direction is predicted according to the dynamic characteristics of technology. The present invention overcomes the shortcomings of the existing methods that mainly rely on expert experience, require a lot of manpower, and cannot make full use of the characteristics of technology flow and the characteristics of the patent hierarchical structure, and can efficiently and accurately predict the future technology flow direction.

[0048] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:

[0050] Figure 1 is a flow chart of a method for predicting future technical knowledge flow provided by the present invention;

[0051] Figure 2 It is a schematic diagram of a model of a method for predicting future technological knowledge flows provided by the present invention. DETAILED DESCRIPTION

[0052] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0053] See also Figure 2 The present invention provides a method for predicting future technological knowledge flows, comprising:

[0054] Step 1: Obtain the authorized patents and their technical classifications in each time period, as well as the cited patent documents of each patent;

[0055] Step 2: Aggregate the citations between the patents into technology flows, and construct a technical knowledge flow TKF network for each year;

[0056] Step 3: Calculate the patent data to obtain the initial feature vectors of diffusion capacity and absorption capacity of the technical field in each time period;

[0057] Step 4: Input the initial feature vector of the technology in each of the above time periods into the technology flow prediction framework to obtain the feature vector of technology diffusion capability and the feature vector of absorption capability in the next time period, and match the diffusion capability and absorption capability of different technologies to obtain the final flow trend between technologies.

[0058] Among them, the initial feature vector d of technology diffusion capability in step 3 includes:

[0059] Technology growth rate, which is the ratio of the number of licenses for a technology to the total number of licenses for that technology in the past few years;

[0060] The amount of technological diffusion is the amount of knowledge contained in a technology that spreads to other technological fields;

[0061] The technology diffusion rate is the ratio of the diffusion of a technology to the total diffusion of all technologies.

[0062] The initial feature vector a of technological absorptive capacity in step 3 includes:

[0063] Technology growth rate, which is the ratio of the number of licenses for a technology to the total number of licenses for that technology in the past few years;

[0064] Technology absorption, that is, the amount of knowledge absorbed by a technology from other technological fields;

[0065] The technology absorption rate is the ratio of the absorption of one technology to the total absorption of all technologies.

[0066] Specifically, step 4 includes:

[0067] The initial feature vector d of the technology diffusion capability and the initial feature vector a of the technology absorption capability in each time period are input into the high-order interaction module HOI of the technology flow prediction framework to calculate the diffusion capability feature vector d after aggregating the surrounding high-order neighbors in the technology field. o and the absorptive capacity characteristic vector a o ;

[0068] The feature vector d o and a o Input the hierarchical transfer module, and obtain the characteristic vectors of technology diffusion capability and absorptive capability that contain the hierarchical structure information unique to the technology system through calculation;

[0069] Input the characteristic vectors of technology diffusion capacity and absorptive capacity of each time period into the technology flow tracking module to obtain the characteristic vectors of diffusion capacity and absorptive capacity of the technology field in the next time period;

[0070] Match the diffusion capacity and absorptive capacity of the technology in the future time period to obtain the flow probability between two different technological fields;

[0071] The obtained probability and true value are input into the loss function for gradient update.

[0072] The calculation formula of the high-order interaction module in step 4 is:

[0073]

[0074]

[0075] in, and Representation Technology i The diffusion and absorption embedding representations of the k-th layer neighbors, and Represents technology U i The outflow and inflow neighbor sets; after aggregating the high-order neighbor features of the K layer, the updated technology diffusion capability representation is obtained and absorptive capacity

[0076]

[0077]

[0078] Among them, α k ≥0 indicates the importance of the k-th layer neighbors, that is, the k-th layer neighbors are important for technology U i impact.

[0079] The hierarchical transfer module in step 4 includes:

[0080] The upward convergence module is used to aggregate the information of the lower-level nodes to the upper-level nodes. This module uses the attention mechanism to automatically learn the weights of the lower-level nodes to indicate the importance of different lower-level nodes. The formula is as follows:

[0081]

[0082]

[0083]

[0084] Among them, w ij represents the weight of the lower layer node j relative to the upper layer node i, is the transformation matrix, is the offset vector, |g i | is the upper technology node g i The number of all underlying technology nodes included, β ij is the standardized weight, is the final representation of the upper-level node i;

[0085] The downward update module is used to update the upper node The information is updated to the lower nodes The calculation formula is as follows:

[0086]

[0087] in, is the updated representation of the lower layer nodes, is the transformation matrix, is the bias vector and σ is the activation function.

[0088] In addition, in the technical flow tracking module of step 4, the bidirectional long short-term memory model BiLSTM is used to model dynamic features. The specific formula is as follows:

[0089] The diffusion characteristics d in the continuous time period T are d = [d 1 , d 2 , ..., d T ] Input BiLSTM,

[0090]

[0091]

[0092]

[0093] d t★ =h t

[0094] Get the diffusion characteristics in the T+1 time period

[0095] Specifically, Figure 1 As shown, in one embodiment of the present invention, the following steps are performed in sequence:

[0096] S101: Obtain the authorized patents in each time period and their technical classifications, as well as the cited patent documents of each patent.

[0097] In the method provided in the embodiment of the present invention, based on the patent data provided by the United States Patent and Trademark Office (USPTO) and the technical classification provided by the Joint Patent Classification System (CPC), the newly added patents each year and their technical classifications and other cited patents are extracted according to the year.

[0098] S102: Aggregate the citations between the patents into technology flows, and construct a technology knowledge flow TKF network for each year.

[0099] In the method provided by the embodiment of the present invention, if the patent included in technology A cites the patent included in technology B n times, it is considered that there may be a knowledge flow from technology B to technology A, where n represents the intensity of the flow. Obviously, the larger n is, the stronger the knowledge flow between technology A and technology B. In addition, in order to remove the influence of noise and limit the size of the technology flow network, a threshold limit is imposed on n: only when n>N, the corresponding knowledge flow can serve as an edge of the technology knowledge flow network. Among them, the value of N is set according to the specific needs in the specific implementation process, and the specific value is determined by the knowledge flow intensity requirement between technology A and technology B.

[0100] S103: Calculate the patent data to obtain initial feature vectors of diffusion capacity and absorption capacity of the technical field in each time period.

[0101] In the method provided by the embodiment of the present invention, the initial characteristic vectors of diffusion capacity and absorption capacity respectively include three values, namely, technology growth rate, technology diffusion amount, technology diffusion rate and technology growth rate, technology absorption amount, and technology absorption rate. and Represents technology U i The diffusion and absorption characteristics at time t will be and As input to subsequent modules.

[0102] S104: Input the initial feature vectors d and a of the technology in each time period into the high-order interaction module HOI of the technology flow prediction framework, and calculate the diffusion capability feature vector d after the technology field aggregates the surrounding high-order neighbors. o and the absorptive capacity characteristic vector a o .

[0103] In the method provided by the embodiment of the present invention, the edges in the TKF network graph can be represented as the interaction between the diffusion capability of the source node and the absorption capability of the target node. Therefore, the idea of ​​an interactive graph neural network is used to update the two characteristics of the technology. Specifically, the diffusion characteristics of the technical field are updated by aggregating the absorption characteristics of adjacent technical fields, and vice versa. A simple weighted sum operation is selected as the operator here, taking into account both efficiency and effect. The aggregation process is as follows:

[0104]

[0105]

[0106] in, and Representation Technology i The diffusion and absorption embedding representations of the k-th layer neighbors, and Respectively represent technology U i After aggregating the high-order neighbor features of K layers, the updated technology diffusion capability representation is obtained: and absorptive capacity

[0107]

[0108]

[0109] Among them, α k ≥0 indicates the importance of the k-th layer neighbors, that is, the k-th neighbors are important for technology U i impact.

[0110] S105: Input the above-mentioned feature vector into the hierarchical transfer module, and obtain the technology diffusion capability and absorptive capability feature vectors containing the hierarchical structure information unique to the technology system through calculation.

[0111] In the method provided in the embodiment of the present invention, since the technologies can be connected through the hierarchical structure of the CPC system. Therefore, a more comprehensive technical representation can be generated by utilizing the hierarchical relationship between technologies, thereby achieving a more accurate TKF prediction. Since the upper category has an inclusion relationship with the lower category, the upper category can be represented by aggregating the information of the lower category. On the other hand, higher category information can also supplement incomplete information in lower categories. For this purpose, a hierarchical transfer module is constructed to describe the mutual influence between upper and lower categories. The hierarchical transfer module consists of two parts, namely an upward transfer module and a downward update module. In this embodiment, each Section of CPC can constitute a data set, such as Figure 2 As shown, we mainly focus on the hierarchical structure between three levels, namely subgroup, group and subsection.

[0112] The upward transfer module aims to aggregate information from a lower level to a higher level, that is, from subgroup to group, and from group to subsection. Here, taking the upward transfer process from subgroup to group as an example, the diffusion features of the group-level nodes are represented by aggregating the diffusion features of its child nodes at the subgroup level. In order to distinguish the importance of different child nodes, the attention mechanism is applied to automatically learn the weights of the child nodes. ij To represent the weight of node j at the subgroup level to node i at the group level, it can be calculated as:

[0113]

[0114]

[0115]

[0116] in, is the transformation matrix, is the offset vector, |g i | is the upper technology node g i The number of all underlying technology nodes included, β ij is the standardized weight, is the updated diffusion feature representation of group layer node i. Similarly, the absorption feature of technology i at the group level can be obtained Then, through the upward transfer process from the group level to the subsection level, the diffusion representation of technology i at the subsection level is obtained. and absorption characterization

[0117] In the downward update phase, the information of the upper nodes needs to be used to update the lower nodes, that is, from subsection to group and from group to subgroup. Here, taking the update process from the subsection level to the group level as an example, in order to update the diffusion representation of the nodes in the group level, the diffusion features of the current node and the diffusion features of its parent node at the subsection level are concatenated, and then a nonlinear transformation is used to obtain an enhanced diffusion representation:

[0118]

[0119] in, is the node representation of the updated group level. is the transformation matrix, is the offset vector, and σ is the activation function. Similarly, by updating information from the group level to the subgroup level, we can obtain the updated diffusion characteristics of technology j at the subgroup level after passing through the hierarchical transfer module. and absorption characteristics

[0120] S106: Input the characteristic vectors of the technology diffusion capacity and absorptive capacity of each of the above time periods into the technology flow tracking module to obtain the characteristic vectors of the diffusion capacity and absorptive capacity of the technology field in the next time period.

[0121] In the embodiment of the present application, the bidirectional long short-term memory model BiLSTM is used to model the dynamic features, and the diffusion feature d=[d 1 , d 2 , ..., d T ] Input BiLSTM, the specific formula is as follows:

[0122]

[0123]

[0124]

[0125] d t★ =h t

[0126] Get the diffusion characteristics in the T+1 time period

[0127] S107: Match the diffusion capacity and absorption capacity of the technology in the future time period to obtain the flow probability between two different technology fields.

[0128] In this embodiment, the diffusion characteristics at time T+1 are and absorption characteristics Perform inner product operation, the formula is: Finally, we can get the probability of technology i flowing to technology j in the T+1 time period.

[0129] S108: Input the obtained probability and true value into the loss function to perform gradient update.

[0130] In this embodiment, since brand new technology flows are difficult to predict and have never appeared before, a well-designed objective function is still required to train the model. In order to more accurately predict new edges without reducing accuracy, a loss function is designed for the edges that appear in each time window, and its formula is as follows:

[0131]

[0132]

[0133]

[0134] in, represents the loss function for predicting new edges, represents the loss function for predicting all the edges that appear, is the total loss function, α represents and exist The proportion of .

[0135] It can be seen that the present invention first extracts the initial diffusion characteristics and absorption characteristics of each technical field in a continuous time period, including the growth rate of technology, the diffusion amount and absorption amount of technology, and the diffusion rate and absorption rate of technology; then the initial diffusion characteristics and absorption characteristics are input into the preset technology flow framework, and these characteristics are sequentially passed through the high-order interaction module and the hierarchical transmission module of technology to obtain the characteristic vector of technology in diffusion and absorption in each time period, and then the diffusion capacity vector and the absorption capacity vector of technology in each time period are input into the technology flow tracking module to obtain the diffusion capacity and absorption capacity vector of each technical field in the future time period; finally, the diffusion capacity and absorption capacity of technology are matched to obtain the flow probability between two different technical fields. The hierarchical structure properties of the existing patent classification system and the high-order flow between technologies are fully utilized, and the method of graph neural network is used to construct a technology flow network based on patent citations, and the future technology flow direction is predicted according to the dynamic characteristics of technology. The present invention overcomes the shortcomings of the existing methods that mainly rely on expert experience, require a lot of manpower, and cannot make full use of the characteristics of technology flow and the characteristics of the patent hierarchical structure, and can efficiently and accurately predict the future technology flow direction.

[0136] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0137] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0138] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A method for predicting future technological knowledge flows, characterized in that: The method comprises: Step 1: Obtain the authorized patents and their technical classifications in each time period, as well as the cited patent documents of each patent; Step 2: Aggregate the citations between the patents into technology flows, and construct a technical knowledge flow TKF network for each year; Step 3: Calculate the patent data to obtain the initial feature vectors of diffusion capacity and absorption capacity of the technical field in each time period; Step 4: Input the initial feature vector d of the technology diffusion capability and the initial feature vector a of the technology absorption capability of each time period into the high-order interaction module HOI of the technology flow prediction framework, and calculate the diffusion capability feature vector d after aggregating the surrounding high-order neighbors in the technology field. o and the absorptive capacity characteristic vector a o ; The calculation formula of the high-order interaction module is: in, and Representation Technology i The diffusion and absorption embedding representations of the k-th layer neighbors, and Represents technology U i The outflow and inflow neighbor sets; after aggregating the high-order neighbor features of the K layer, the updated technology diffusion capability representation is obtained and absorptive capacity Among them, α k ≥0 indicates the importance of the k-th layer neighbors, that is, the k-th layer neighbors are important for technology U i The impact of The feature vector d o and a o Input the hierarchical transfer module, and obtain the characteristic vectors of technology diffusion capability and absorptive capability that contain the hierarchical structure information unique to the technology system through calculation; Input the characteristic vectors of technology diffusion capacity and absorptive capacity of each time period into the technology flow tracking module to obtain the characteristic vectors of diffusion capacity and absorptive capacity of the technology field in the next time period; Match the diffusion capacity and absorption capacity of technology in the future time period to obtain the flow probability between two different technological fields; The obtained probability and true value are input into the loss function for gradient update.

2. The method for predicting future technological knowledge flow according to claim 1, characterized in that: The initial feature vector d of technology diffusion capability in step 3 includes: Technology growth rate, which is the ratio of the number of licenses for a technology to the total number of licenses for that technology in the past few years; The amount of technological diffusion, that is, the amount of knowledge contained in a technology that spreads to other technological fields; The technology diffusion rate is the ratio of the diffusion of a technology to the total diffusion of all technologies.

3. The method for predicting future technological knowledge flow according to claim 1, characterized in that: The initial feature vector a of technological absorptive capacity in step 3 includes: Technology growth rate, which is the ratio of the number of licenses for a technology to the total number of licenses for that technology in the past few years; Technology absorption, that is, the amount of knowledge absorbed by a technology from other technological fields; The technology absorption rate is the ratio of the absorption of one technology to the total absorption of all technologies.

4. The method for predicting future technological knowledge flow according to claim 1, characterized in that: The hierarchical transfer module in step 4 includes: The upward convergence module is used to aggregate the information of the lower-level nodes to the upper-level nodes. This module uses the attention mechanism to automatically learn the weights of the lower-level nodes to indicate the importance of different lower-level nodes. The formula is as follows: Among them, w ij represents the weight of the lower layer node j relative to the upper layer node i, is the transformation matrix, is the offset vector, |g i | is the upper technology node g i The number of all underlying technology nodes included, β ij is the standardized weight, is the final representation of the upper-level node i; The downward update module is used to update the upper node The information is updated to the lower nodes The calculation formula is as follows: in, is the updated representation of the lower layer nodes, is the transformation matrix, is the bias vector and σ is the activation function.

5. The method for predicting future technological knowledge flow according to claim 1, characterized in that: In the technical flow tracking module of step 4, the bidirectional long short-term memory model BiLSTM is used to model dynamic features. The specific formula is as follows: The diffusion characteristics d in the continuous time period T are d = [d 1 , d 2 , ..., d T ] Input BiLSTM, d t★ =h t Get the diffusion characteristics in the T+1 time period

Images