Technical knowledge flow prediction method and related device

Abstract

The invention discloses a technical knowledge flow prediction method, which comprises the following steps: firstly, generating text description for technical knowledge through a large language model to make up for a semantic gap of the technical knowledge among different fields, and then, carrying out feature extraction on the technical knowledge through a graph neural network and Transform from two perspectives of a flow relationship and a co-occurrence relationship, structural information and time sequence evolution information between technical knowledge under two perspectives are captured, then fusion features are obtained by fusing features of the technical knowledge under a co-occurrence relation and a flow relation, feature expression of the technical knowledge is effectively enhanced, finally, the flow trend of the technical knowledge can be predicted based on the fusion features, and the flow trend of the technical knowledge can be predicted. Therefore, the problem of weak future flow trend prediction capability caused by factors such as large semantic gap, strong graph structure modeling isomerism and weak time sequence evolution information capture capability among different fields is solved, and the prediction accuracy of cross-domain technical knowledge flow is improved.

Description

Technical Field

[0001] This application involves interdisciplinary fields such as artificial intelligence, graph neural networks, and natural language processing, and specifically relates to a method for predicting the flow of technical knowledge and related devices. Background Art

[0002] Technological Knowledge Flow (TKF) refers to the process of technological knowledge dissemination, which plays a vital role in promoting innovation and cross-disciplinary collaboration.

[0003] Traditional TKF prediction methods mainly focus on knowledge dissemination within the same field and adopt link prediction models based on graph neural networks, such as graph convolutional neural networks and graph attention networks. These models rely on the structural homogeneity assumption and emphasize the similarity between nodes.

[0004] However, with the intensification of interdisciplinary research and technological integration, cross-domain technological knowledge flows are becoming increasingly frequent, and traditional methods face challenges in predicting the accuracy of cross-domain technological knowledge flows. Summary of the Invention

[0005] In view of this, the present application provides a technology knowledge flow prediction method and related devices, which improve the modeling capability and prediction accuracy of cross-domain technology knowledge flow.

[0006] To solve the above problems, the technical solutions provided by this application are as follows:

[0007] In one aspect, the present application provides a method for predicting technology knowledge flow, the method comprising:

[0008] Obtaining basic data, including documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document, wherein the technical classification information is used to identify the category of technical knowledge involved in the document;

[0009] Based on the basic data, a description text of the technical knowledge is generated by a large language model, wherein the description text is used to describe the flow relationship and co-occurrence relationship between technical knowledge;

[0010] Encoding the technical knowledge based on the description text to obtain initial flow features and initial co-occurrence features;

[0011] The initial flow features and the initial co-occurrence features are used as node representations in a directed graph sequence and an undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure. The directed graph sequence is used to identify the flow relationship of the technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of the technical knowledge in different time periods.

[0012] Inputting the directed graph structure and the undirected graph structure into a feature extraction module for feature extraction to obtain technology flow features and technology co-occurrence features, the feature extraction module including a graph neural network (GNN) and a Transformer architecture;

[0013] By using non-negative contrastive learning, the technology flow features and the technology co-occurrence features are aligned to obtain the fusion features of the technology knowledge;

[0014] Based on the fusion features, calculating the similarity of the technical knowledge and determining the dissemination weight of the technical knowledge;

[0015] Based on the similarity and the propagation weight, the future flow result of the technical knowledge is predicted.

[0016] In a possible implementation, the description text includes a definition, a flow relationship description, and a co-occurrence relationship description, and encoding the technical knowledge based on the description text to obtain the initial flow features and initial co-occurrence features includes:

[0017] Based on the definition and the flow relationship description, encoding the technical features using Sentence-BERT technology to obtain the initial flow features;

[0018] Based on the definition and the flow relationship description, the technical features are encoded using Sentence-BERT technology to obtain the initial co-occurrence features.

[0019] In a possible implementation, calculating the similarity of the technical knowledge based on the fusion feature includes:

[0020] Based on the fusion features, the similarity of the technical knowledge is calculated by a bilinear function.

[0021] In a possible implementation, the aligning of the technology flow features and the technology co-occurrence features through non-negative contrast learning to obtain the fusion features of the technology knowledge includes:

[0022] Through the non-negative contrast learning, the technology flow feature is updated based on the same technology knowledge in the technology flow feature and the technology collinearity feature to obtain the fusion feature of the technology knowledge.

[0023] In a possible implementation, during the feature alignment process, the method further includes:

[0024] Model parameters are smoothed by the exponential moving average technique.

[0025] In a possible implementation, predicting the future flow result of the technical knowledge based on the similarity and the propagation weight includes:

[0026] Fusing the similarity and the propagation weight to obtain an initial flow result of the technical knowledge;

[0027] The initial flow result is optimized by a decoupling loss function to obtain the future flow result. The decoupling loss function includes a loss function designed for the flow of technical knowledge within a field and a loss function designed for the flow of cross-field technical knowledge.

[0028] In another aspect, the present application provides a technology knowledge flow prediction device, comprising an acquisition unit, a generation unit, an encoding unit, a construction unit, a feature extraction unit, a feature alignment unit, a determination unit, and a prediction unit:

[0029] The acquisition unit is used to acquire basic data, wherein the basic data includes documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document, wherein the technical classification information is used to identify the category of technical knowledge involved in the document;

[0030] The generating unit is configured to generate a description text of the technical knowledge based on the basic data using a large language model, wherein the description text is used to describe the flow relationship and co-occurrence relationship between technical knowledge;

[0031] The encoding unit is used to encode the technical knowledge based on the description text to obtain initial flow features and initial co-occurrence features;

[0032] The construction unit is configured to use the initial flow features and the initial co-occurrence features as node representations in a directed graph sequence and an undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure, wherein the directed graph sequence is used to identify the flow relationship of the technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of the technical knowledge in different time periods;

[0033] The feature extraction unit is used to input the directed graph structure and the undirected graph structure into a feature extraction module for feature extraction to obtain technology flow features and technology co-occurrence features. The feature extraction module includes a graph neural network (GNN) and a Transformer architecture.

[0034] The feature alignment unit is used to align the technology flow feature and the technology co-occurrence feature through non-negative contrast learning to obtain the fusion feature of the technology knowledge;

[0035] The determining unit is configured to calculate the similarity of the technical knowledge based on the fusion features and determine the dissemination weight of the technical knowledge;

[0036] The prediction unit is used to predict the future flow results of the technical knowledge based on the similarity and the propagation weight.

[0037] In a possible implementation, the description text includes a definition, a flow relationship description, and a co-occurrence relationship description, and the encoding unit is specifically configured to:

[0038] Based on the definition and the flow relationship description, encoding the technical features using Sentence-BERT technology to obtain the initial flow features;

[0039] Based on the definition and the flow relationship description, the technical features are encoded using Sentence-BERT technology to obtain the initial co-occurrence features.

[0040] In a possible implementation manner, the determining unit is configured to:

[0041] Based on the fusion features, the similarity of the technical knowledge is calculated by a bilinear function.

[0042] In a possible implementation, the feature alignment unit is configured to:

[0043] Through the non-negative contrast learning, the technology flow feature is updated based on the same technology knowledge in the technology flow feature and the technology collinearity feature to obtain the fusion feature of the technology knowledge.

[0044] In a possible implementation, the apparatus further includes a smoothing unit, and the smoothing unit is configured to:

[0045] Model parameters are smoothed by the exponential moving average technique.

[0046] In a possible implementation, the prediction unit is configured to:

[0047] Fusing the similarity and the propagation weight to obtain an initial flow result of the technical knowledge;

[0048] The initial flow result is optimized by a decoupling loss function to obtain the future flow result. The decoupling loss function includes a loss function designed for the flow of technical knowledge within a field and a loss function designed for the flow of cross-field technical knowledge.

[0049] In another aspect, the present application provides a computer device, comprising a processor and a memory:

[0050] The memory is used to store computer programs;

[0051] The processor is configured to execute any one of the above methods according to the computer program.

[0052] On the other hand, the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a computer device, it implements any of the methods described above.

[0053] On the other hand, the present application provides a computer program product comprising a computer program, which, when executed on a computer device, enables the computer device to execute any one of the methods described above.

[0054] It can be seen from the above technical solution that the technical solution first obtains basic data, including documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document, wherein the technical classification information of each document can represent the technical knowledge involved in the document, and then based on the basic data, through the large language model, obtains a descriptive text for describing the flow relationship and co-occurrence relationship of technical knowledge to make up for the problem of insufficient description of the technical knowledge and break the semantic gap between different fields, thereby encoding the technical knowledge based on the descriptive text, which can effectively improve the semantic expression ability of technical knowledge, and obtain the initial flow features based on the flow relationship and the initial co-occurrence features based on the co-occurrence relationship, and then use the initial flow features and the initial co-occurrence features as node representations in the directed graph sequence and the undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure, wherein the directed graph sequence is used to identify the flow relationship of technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of technical knowledge in different time periods, and then the directed graph structure and the undirected graph structure are subjected to feature extraction respectively, wherein the graph Neural networks can fuse structural information and avoid the limitations of single-domain structural assumptions. The Transformer architecture can capture the evolution of technical knowledge over time, thereby enhancing the feature expression of technical knowledge and obtaining technical flow features and technical co-occurrence features. Then, through non-negative contrastive learning, the technical flow features and technical co-occurrence features are aligned to obtain the fusion features of technical knowledge, achieving consistency in the expression of the same technical knowledge from different perspectives. Finally, based on the fusion features, the similarity and propagation weight between each piece of technical knowledge are calculated to achieve modeling of complex relationships between technical knowledge. The similarity can measure the possibility of technical knowledge propagation, thereby accurately predicting the future flow results of each piece of technical knowledge. This technical solution has significant advantages in many aspects such as semantic enhancement of technical knowledge, graph structure and temporal relationship modeling, and flow prediction mechanism. When technical knowledge flows across domains, it effectively overcomes the problem of weak future flow trend prediction ability due to factors such as large semantic gaps, strong heterogeneity of graph structure modeling, and weak ability to capture temporal evolution information between different domains. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 A flowchart of a method for predicting technical knowledge flow provided in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of the structure of a non-negative contrastive learning method based on large model enhancement provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of a technology knowledge flow prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0060] As described in the background, traditional TKF prediction methods rely on the assumption of structural homogeneity within the same domain and are primarily applied to knowledge transfer within the same domain. However, the semantic differences and complexity of technical knowledge across different domains, as well as the structural heterogeneity, exacerbate the difficulty of representation learning, making it difficult for the model to capture the underlying connections between domains. Furthermore, traditional TKF methods lack limited modeling of semantic gaps, limiting the model's generalization ability in predicting cross-domain knowledge flows. Therefore, a comprehensive approach that integrates structural and semantic information is urgently needed to improve the modeling capabilities and prediction accuracy of cross-domain technical knowledge flows.

[0061] To address the key issues of large semantic gaps, strong structural heterogeneity, and modeling difficulty in cross-domain technology flows, this application proposes a technology knowledge flow prediction method suitable for predicting future cross-domain technology knowledge flow trends. This technical solution uses a large language model to generate text descriptions for each piece of technology knowledge, thereby bridging the semantic gap between different domains and enhancing the semantic information of technology knowledge. The technical knowledge is then encoded from two perspectives: flow relationships and co-occurrence relationships. Features of technology knowledge from different perspectives are obtained and used as node representations to construct a graph data structure representing the relevance of technology knowledge from different perspectives over different time periods. The graph data structure is then extracted using graph neural networks (GNNs) and transformers. This not only fuses structural information, avoiding the limitations of single-domain structural assumptions, but also captures the temporal evolution of technology knowledge flows, thereby obtaining feature representations of technology knowledge from two perspectives. The corresponding feature representations from different perspectives are then aligned to obtain fused features of the technology knowledge. Finally, based on the fused features, the similarity and diffusion weight between the technology knowledge are determined to predict the flow trend of technology knowledge, thereby effectively improving the accuracy of cross-domain technology knowledge flow predictions.

[0062] The solutions provided in the embodiments of this application involve intersecting fields such as artificial intelligence, graph neural networks, and natural language processing, and are specifically illustrated through the following embodiments.

[0063] See also Figure 1 As shown, it is a flow chart of a method for predicting the flow of technical knowledge provided in an embodiment of the present application. In this embodiment, it can be executed by a computer device as an example.

[0064] S101: Obtain basic data, which includes documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document.

[0065] Among them, technical classification information is used to identify the category of technical knowledge involved in the document. Each document involves one or more technical knowledge, and a single piece of technical knowledge can be disseminated in different fields. Technical classification information will also be iteratively updated over time, such as adding new category labels or refining category labels. The embodiments of this application do not impose specific restrictions on technical knowledge. For example, technical knowledge can be a patent classification label or simplified text features.

[0066] This application can use technical means to obtain documents published in different time periods and their corresponding technical classification information, as well as the documents cited by each document, thereby helping computer equipment capture the citation relationship between technical knowledge in different time periods, that is, the flow relationship and the co-occurrence relationship between technical knowledge in different time periods.

[0067] S102: Generate descriptive text of technical knowledge through a large language model based on basic data.

[0068] The description text is used to describe the flow and co-occurrence relationships of technical knowledge. Based on the technical classification information corresponding to the authorized patents, the co-occurrence relationship of technical knowledge can be determined. Combined with the flow relationship between patents, the flow relationship of technical knowledge can be determined.

[0069] This application can capture the flow relationship and co-occurrence relationship between technical knowledge based on basic data through a large model, thereby generating a descriptive text for each technical knowledge. The flow relationship and co-occurrence relationship of technical knowledge can be described through the application of technical knowledge in the same or different fields, thereby enhancing the semantic information of technical knowledge, generating semantically rich and context-related text descriptions, making up for the problem of insufficient description of technical knowledge, and breaking the semantic gap between different fields.

[0070] S103: Encode the technical knowledge based on the description text to obtain initial flow features and initial co-occurrence features.

[0071] Based on the text content describing the flow relationship and co-occurrence relationship of technical knowledge, the technical knowledge can be encoded using the Sentence-BERT technology to obtain the initial flow features based on the flow relationship and the initial co-occurrence features based on the co-occurrence relationship. This process can be expressed by the following formula:

[0072] (1)

[0073] (2)

[0074] in, is the initial flow characteristic, is the initial co-occurrence feature, SBERT refers to Sentence-BERT technology, Refers to the text content of the flow relationship, Refers to the text content of the co-occurrence relationship.

[0075] In order to further enhance the semantic information of technical knowledge, description texts can be generated from multiple perspectives.

[0076] In a possible implementation, the description text includes a definition, a flow relationship description, and a co-occurrence relationship description. Step S103 includes:

[0077] S1031: Based on the definition and flow relationship description, the technical features are encoded using Sentence-BERT technology to obtain initial flow features;

[0078] S1032: Based on the definition and flow relationship description, each technical feature is encoded using the Sentence-BERT technology to obtain the initial co-occurrence feature.

[0079] Definition refers to the basic introduction of technical knowledge, flow relationship description is used to describe the flow relationship of technical knowledge, and technical co-occurrence is used to describe the co-occurrence relationship of technical knowledge.

[0080] Taking medical informatics as an example, the text description generated by the big model is as follows:

[0081] 1) Basic Introduction: Healthcare informatics applies information and communication technologies specifically for the management and processing of medical or health data. It encompasses a variety of techniques and methods to improve the efficiency, accuracy, and accessibility of medical information. Core areas include electronic health records, telemedicine, data analytics, and patient monitoring systems, aiming to improve patient care and optimize healthcare operations.

[0082] 2) Fluid Relationships: The technological evolution of medical informatics began with the digitization of medical records—the systematic collection and storage of patient data through electronic health record systems. As electronic health record functionality continues to grow, real-time monitoring capabilities are gradually being introduced. These health metrics are typically collected through IoT devices and wearable technology and transmitted via the secure communication networks used for telemedicine, enabling remote consultations and ongoing health tracking. Based on this data, machine learning-driven predictive analytics models are constructed to predict potential health risks based on historical and real-time information, providing reliable support for patient care and medical decision-making.

[0083] 3) Co-occurrence: In medical informatics, multiple technologies converge to create complete solutions. For example, electronic health records and telemedicine platforms are deeply integrated to enable cross-regional patient data sharing; IoT devices and wearable devices continuously feed real-time health data into analytics systems; and advanced data analytics tools work in tandem with artificial intelligence and machine learning algorithms to extract insights from massive data sets. Predictive analytics, at the core of modern medical informatics, builds on this synergy, enabling early intervention and precision medicine.

[0084] Using Sentence-BERT technology, we can combine the definition and flow relationship description to generate the initial flow features of technical knowledge under the flow relationship, and combine the definition and co-occurrence relationship description to generate the initial co-occurrence features under the co-occurrence relationship.

[0085] Therefore, generating descriptive texts of technical knowledge from multiple perspectives can be based on the definition of technical knowledge and combined with the correlation information between technical knowledge under flow relationships and co-occurrence relationships, covering more details and background information, helping to understand technical knowledge more comprehensively and provide richer semantic information, thereby strengthening the semantic expression of technical knowledge and effectively avoiding the sparsity and heterogeneity of technical knowledge descriptions.

[0086] S104: Using the initial flow features and the initial co-occurrence features as node representations in a directed graph sequence and an undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure.

[0087] The directed graph sequence is used to identify the flow relationship of technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of technical knowledge in different time periods. Therefore, the evolution information of technical knowledge over time can be represented by the directed graph sequence and the undirected graph sequence.

[0088] By taking the initial flow features as the feature expression of the nodes of a directed graph sequence, a directed graph structure can be constructed. By taking the initial co-occurrence features as the feature expression of the nodes of an undirected graph sequence, an undirected graph structure can be constructed, thereby modeling the flow relationship and collinear relationship of technical knowledge based on the graph structure.

[0089] S105: Inputting the directed graph structure and the undirected graph structure into a feature extraction module respectively for feature extraction to obtain technology flow features and technology co-occurrence features.

[0090] The feature extraction module includes GNN and Transformer architectures. Directed and undirected graph structures are input into the GNN network to extract structural information, which can be expressed by the following formula:

[0091] (3)

[0092] (4)

[0093] in, and are the normalized adjacency matrices of directed and undirected graph structures in different time stages, represents the activation function, is a trainable transformation matrix.

[0094] Then, the Transformer architecture is used to capture time series information and model the evolution of technical knowledge over time, thereby obtaining the characteristics of technical flow and technical co-occurrence, which can be expressed by the following formula:

[0095] (5)

[0096] (6)

[0097] in, and They are technology flow characteristics and technology co-occurrence characteristics respectively.

[0098] Therefore, GNN can integrate structural information to avoid the limitations of single-domain structural assumptions. The Transformer architecture can capture the evolution of technical knowledge over time and enhance the feature expression of technical knowledge.

[0099] S106: Through non-negative contrast learning, the technology flow features and technology co-occurrence features are aligned to obtain the fusion features of technology knowledge.

[0100] The flow of technical knowledge across fields has complex interactions, and it is difficult to make effective predictions based solely on historical data of technical knowledge from a flow perspective. The embodiments of the present application model the historical data of technical knowledge from a flow perspective and a co-occurrence perspective respectively, so feature alignment is required to obtain a consistent expression of technical knowledge from different perspectives.

[0101] Non-negative contrastive learning refers to aligning only the features of the same technical knowledge in the technology flow features and the technology co-occurrence features, without aligning the features of different technical knowledge. In the embodiment of the present application, feature alignment can be performed through contrast loss.

[0102] The contrastive loss function for non-negative contrastive learning is as follows:

[0103] (7)

[0104] in, represents the optimization objective of non-negative contrastive learning, Represents the characteristics of technology flow, represents the technical collinearity feature, represents the dot product, express norm, and m is the number of characteristics of technical knowledge.

[0105] In order to ensure that the aligned features retain the directional information of the technical knowledge flow, in a possible implementation, step S106 includes:

[0106] Through non-negative contrast learning, based on the same technical knowledge in the technical flow characteristics and technical collinearity characteristics, the technical flow characteristics are updated to obtain the fusion characteristics of technical knowledge;

[0107] The process of minimizing the optimization objective of non-negative contrastive learning is the process of feature alignment. Based on non-negative contrastive learning, by optimizing the features of identical technical knowledge in technology flow features and technology co-occurrence features, we can achieve consistent expression of the same technical knowledge.

[0108] In the embodiment of the present application, the corresponding features in the technical flow features are updated based on the consistency expression results to obtain the fusion features of technical knowledge from different perspectives.

[0109] Therefore, the fusion features obtained by updating the features of the corresponding technical knowledge in the technology flow features based on the consistency expression results not only integrate the correlation information of technical knowledge from different perspectives, but also retain the directional information of the technology knowledge flow, thereby improving the accuracy of the prediction of the direction of technology knowledge flow.

[0110] In order to improve the stability of the model, in a possible implementation, during the feature alignment process, the method further includes:

[0111] Model parameters are smoothed by the exponential moving average technique.

[0112] The exponential moving average technique is described by the following expression:

[0113] (8)

[0114] in, is the smoothing factor, are the relevant parameters of technology flow characteristics, are the relevant parameters of technology co-occurrence features.

[0115] Therefore, adjusting the relevant parameters of the technology co-occurrence feature through the relevant parameters of the technology flow feature can effectively improve the stability of the model during the feature update process and ensure the stable convergence of the model.

[0116] S107: Based on the fusion features, the similarity of the technical knowledge is calculated, and the dissemination weight of the technical knowledge is determined.

[0117] By fusion features, we can calculate the similarity based on the feature vectors of any two pieces of technical knowledge and measure the potential possibility of the dissemination of technical knowledge. By fusing the splicing results of the feature vectors of two pieces of technical knowledge through the Multilayer Perceptron (MLP), we can measure the relative importance of the flow of technical knowledge between the two and thus determine the dissemination weight of technical knowledge.

[0118] In order to capture the directional information of technical knowledge flow, in one possible implementation, the similarity of technical knowledge is calculated based on the fusion features, including:

[0119] Based on the fused features, the similarity of technical knowledge is calculated through a bilinear function.

[0120] The bilinear interaction function is determined as follows:

[0121] (9)

[0122] in, represents the similarity between node i and node j, represents the fusion feature of node i, yes The transpose of is a trainable transformation matrix.

[0123] Therefore, by using bilinear functions to calculate the similarity between technical knowledge, we can obtain the directed interactive relationship between technical knowledge, thereby helping to explore the potential flow direction of technical knowledge and improve the accuracy of predicting the flow direction of technical knowledge.

[0124] At this point, the propagation weight can be determined by the following expression:

[0125] (10)

[0126] in, represents the propagation weight of the flow from node i to node j, represents the fusion feature of node j, Represents a feature concatenation operation.

[0127] Therefore, the diffusion weight reflects the interactive relationship between technical knowledge and captures its relative contribution in the directed flow of technical knowledge.

[0128] S108: Based on similarity and diffusion weight, predict the future flow results of technical knowledge.

[0129] By fusing similarity and propagation weight through the Sigmoid function, we can obtain the prediction score of the flow of technical knowledge to other technical knowledge. Based on the prediction score, we can determine the future flow of technical knowledge, achieve accurate modeling of the directionality and complexity of technical knowledge flow, and obtain a technical knowledge flow prediction model. The expression is as follows:

[0130] (11)

[0131] in, is the Sigmoid function.

[0132] In order to adapt to the patterns and preferences of technical knowledge flow within the same field or different fields, in one possible implementation, step S108 includes:

[0133] S1081: Fusing similarity and propagation weight to obtain the initial flow result of technical knowledge;

[0134] S1082: Optimize the initial flow result by decoupling the loss function to obtain the future flow result.

[0135] Among them, the decoupling loss function includes the loss function designed for the flow of technical knowledge within the field and the loss function designed for the flow of technical knowledge across fields.

[0136] The flow of technical knowledge within a domain tends to be homogeneous, meaning that two pieces of flowing technical knowledge typically have high similarity. The flow of technical knowledge across domains tends to be heterogeneous, meaning that the pieces of flowing technical knowledge typically have low similarity. Therefore, to accommodate edges of different natures, a decoupled loss function is used to model the characteristics of homogeneous and heterogeneous edges separately:

[0137] (12)

[0138] (13)

[0139] in, is a loss function designed for the flow of technical knowledge within the field, A loss function designed for cross-domain technical knowledge flow, and denote the sets of homogamous and heterogamous edges, respectively. Representative Node and nodes There is flow between them.

[0140] Furthermore, the number of positive edges (actual technology flows) in technology flows is far less than the number of negative edges (non-existent technology flows), resulting in a class imbalance. Traditional contrastive learning relies on a negative sample generation mechanism, which can easily misclassify potential technology flows that have not yet occurred as irrelevant edges, leading to training bias.

[0141] In order to alleviate the problem of category imbalance, structured negative sampling is introduced, and the sampled negative sample set is recorded as Since negative edges indicate non-existent technology flows, they are treated uniformly during training regardless of whether they are cross-domain or not, and the loss function is:

[0142] (14)

[0143] in, , indicating that there is no technology flow between node i and node k.

[0144] As a result, the model can better adapt to different flow patterns, whether it is the flow of technical knowledge within the same field or across fields, and can more accurately identify the flow paths and trends between technical knowledge, thereby significantly improving the accuracy of technology flow predictions.

[0145] The total loss function during model training in the embodiment of the present application is as follows:

[0146] (15)

[0147] in, Indicates the ratio of the same edges in the directed graph structure at the previous moment, Indicates the ratio of heterogeneous edges in the directed graph structure at the previous moment, is the weight coefficient, is the contrast loss.

[0148] This weighting strategy balances the contributions of homogamous and heterogamous edges according to their relative importance in the directed graph. The setting enables the model to strike a balance between feature alignment and flow prediction.

[0149] Therefore, the technical solution of the present application can be effectively applied to the task of predicting the future cross-domain technical knowledge flow trend. First, a text description is generated for each technical knowledge through a large language model to break the semantic gap between different fields and enhance the semantic information of technical knowledge. Then, the technical knowledge is encoded from the two perspectives of flow relationship and co-occurrence relationship to obtain the characteristics of technical knowledge under different perspectives, and use them as node representations to construct a graph data structure used to represent the relevance of technical knowledge under different perspectives in different time periods. Then, the graph data structure is subjected to feature extraction through GNN and Transformer, which can not only fuse structural information to avoid the limitations of single-domain structural assumptions, but also capture the evolution information of technical knowledge flow over time, thereby obtaining the feature expression of technical knowledge from two perspectives. Then, the corresponding feature expressions from different perspectives are aligned to obtain the fusion features of technical knowledge. Finally, based on the fusion features, the similarity and propagation weight between technical knowledge are determined, and the flow trend of technical knowledge is predicted, thereby effectively improving the prediction accuracy of cross-domain technical knowledge flow.

[0150] In order to describe this technology knowledge flow prediction method more clearly, refer to Figure 2 The figure shows a schematic diagram of the structure of a non-negative contrastive learning method based on large model enhancement provided by the embodiment of the present application. The structure diagram of the method includes a technology representation module enhanced by big data, a non-negative graph contrastive learning module, a technology flow prediction module, and a decoupling loss function module. Its task goal is to give a set of historical technology flow graphs. (i.e. directed graph sequence), historical technology co-occurrence graph collection (i.e. undirected graph sequence), as well as basic data Text, prediction time The result of the flow of technological knowledge.

[0151] This task can be formulated as learning a mapping function:

[0152] (16)

[0153] in, Indicates prediction through historical technology flow diagram, historical technology co-occurrence diagram and basic data The edge set of Representative time The flow results.

[0154] The method may specifically include the following steps:

[0155] 1) The technology representation module based on large model enhancement obtains the initial features of technology knowledge.

[0156] First, basic data is obtained, which includes patents authorized in different time periods and their corresponding technical classification information, as well as the patents cited by each patent.

[0157] Secondly, based on the basic data, the large language model LLM is used to generate descriptive text, including text content used to describe the flow relationship and text content describing the co-occurrence relationship .

[0158] Then, the technical knowledge is feature-encoded using SBERT technology to obtain the initial flow features from the flow perspective. Initial co-occurrence features from the co-occurrence perspective .

[0159] 2) Feature extraction through non-negative image contrast learning module.

[0160] First, and As directed graph sequences and undirected graph sequences Based on the node representation, we construct directed graph structures (TKF Graphs) and undirected graph structures (TC Graphs). The directed graph sequence is used to identify the flow relationship of technical knowledge in different time periods, while the undirected graph sequence is used to identify the co-occurrence relationship of technical knowledge in different time periods.

[0161] Then, TKF Graphs and TC Graphs are input into the feature extraction submodule of the non-negative graph contrast learning module to extract features and obtain the technical flow features. and technical co-occurrence features ,This sub-module includes GNN and Transformer architecture, thus integrating structural ,information and temporal evolution information, effectively improving the ,feature expression capability of technical knowledge.

[0162] Then, the consistency of the same features is expressed through the contrast loss function to obtain the fusion features that integrate information from different perspectives. .

[0163] 4) Technology flow forecast.

[0164] Calculate fusion features through bilinear function operation middle and The similarity between the two is determined by MLP, so as to predict the future flow results and obtain a prediction model based on the large model enhanced non-negative contrastive learning method.

[0165] 5) Optimize the prediction model by decoupling the loss function.

[0166] This method designs decoupling loss functions for different modes of technical knowledge flow, including the above-mentioned contrast loss function and the loss function designed for technical knowledge flow within the field. , and loss functions designed for cross-domain technical knowledge flow , the weights of each loss function can be adjusted based on the prediction performance of the model, thereby optimizing the prediction model and improving the accuracy of the prediction model.

[0167] This method not only avoids the limitations of traditional methods that rely on expert experience and single structural domain assumptions, but can also model potential cross-domain technology integration relationships that have not yet occurred, thereby accurately predicting the future knowledge flow paths between domains, and making up for the shortcomings of existing methods in handling heterogeneous technology relationships, capturing cross-domain flow laws, and automated prediction capabilities.

[0168] Its innovative advantages are as follows: First, by introducing a large language model to generate semantically rich multi-perspective text descriptions, the semantic representation ability of technical knowledge is effectively enhanced; second, by jointly constructing a directed graph based on flow relationships and an undirected graph based on co-occurrence relationships, the modeling ability of cross-domain collaboration and evolution is structurally improved; third, by using a non-negative graph contrast learning method to avoid misjudgment of potential connections, the robustness of the model in heterogeneous environments is enhanced; further, by using Transformer to model graph structure sequences, the model has the ability to model time evolution trends; finally, by determining the similarity of technical knowledge through directional interaction functions, and determining the dissemination weight of technical knowledge, the complex directed flow relationship between technical knowledge is accurately modeled. In addition, the decoupling loss function is combined to perform differentiated optimization of same-domain edges and cross-domain edges, significantly improving prediction accuracy and generalization ability.

[0169] Based on the above embodiments, the present application provides a technology knowledge flow prediction device, referring to Figure 3 FIG. 3 is a schematic diagram of a technology knowledge flow prediction device provided by an embodiment of the present application. The device 300 includes an acquisition unit 301, a generation unit 302, an encoding unit 303, a construction unit 304, a feature extraction unit 305, a feature alignment unit 306, a determination unit 307, and a prediction unit 308.

[0170] The acquisition unit is used to acquire basic data, wherein the basic data includes documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document, wherein the technical classification information is used to identify the category of technical knowledge involved in the document;

[0171] The generating unit is configured to generate a description text based on the basic data using a large language model, wherein the description text is used to describe the flow relationship and co-occurrence relationship of the technical knowledge;

[0172] The encoding unit is used to encode the technical knowledge based on the description text to obtain initial flow features and initial co-occurrence features;

[0173] The construction unit is configured to use the initial flow features and the initial co-occurrence features as node representations in a directed graph sequence and an undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure, wherein the directed graph sequence is used to identify the flow relationship of the technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of the technical knowledge in different time periods;

[0174] The feature extraction unit is used to input the directed graph structure and the undirected graph structure into a feature extraction module for feature extraction to obtain technology flow features and technology co-occurrence features. The feature extraction module includes a graph neural network (GNN) and a Transformer architecture.

[0175] The feature alignment unit is used to align the technology flow feature and the technology co-occurrence feature through non-negative contrast learning to obtain the fusion feature of the technology knowledge;

[0176] The determining unit is configured to calculate the similarity of the technical knowledge based on the fusion features and determine the dissemination weight of the technical knowledge;

[0177] The prediction unit is used to predict the future flow results of the technical knowledge based on the similarity and the propagation weight.

[0178] Therefore, the technical solution of the present application can be effectively applied to the task of predicting the future cross-domain technical knowledge flow trend. First, a text description is generated for each technical knowledge through a large language model to break the semantic gap between different fields and enhance the semantic information of technical knowledge. Then, the technical knowledge is encoded from the two perspectives of flow relationship and co-occurrence relationship to obtain the characteristics of technical knowledge under different perspectives, and use them as node representations to construct a graph data structure used to represent the relevance of technical knowledge under different perspectives in different time periods. Then, the graph data structure is subjected to feature extraction through GNN and Transformer, which can not only fuse structural information to avoid the limitations of single-domain structural assumptions, but also capture the evolution information of technical knowledge flow over time, thereby obtaining the feature expression of technical knowledge from two perspectives. Then, the corresponding feature expressions from different perspectives are aligned to obtain the fusion features of technical knowledge. Finally, based on the fusion features, the similarity and propagation weight between technical knowledge are determined, and the flow trend of technical knowledge is predicted, thereby effectively improving the prediction accuracy of cross-domain technical knowledge flow.

[0179] In a possible implementation, the description text includes a definition, a flow relationship description, and a co-occurrence relationship description, and the encoding unit is specifically configured to:

[0180] Based on the definition and the flow relationship description, encoding the technical features using Sentence-BERT technology to obtain the initial flow features;

[0181] Based on the definition and the flow relationship description, the technical features are encoded using Sentence-BERT technology to obtain the initial co-occurrence features.

[0182] Therefore, generating descriptive texts of technical knowledge from multiple perspectives can be based on the definition of technical knowledge and combined with the correlation information between technical knowledge under flow relationships and co-occurrence relationships, covering more details and background information, helping to understand technical knowledge more comprehensively and provide richer semantic information, thereby strengthening the semantic expression of technical knowledge and effectively avoiding the sparsity and heterogeneity of technical knowledge descriptions.

[0183] In a possible implementation manner, the determining unit is configured to:

[0184] Based on the fusion features, the similarity of the technical knowledge is calculated by a bilinear function.

[0185] Therefore, by using bilinear functions to calculate the similarity between technical knowledge, we can obtain the directed interactive relationship between technical knowledge, thereby helping to explore the potential flow direction of technical knowledge and improve the accuracy of predicting the flow direction of technical knowledge.

[0186] In a possible implementation, the feature alignment unit is configured to:

[0187] Through the non-negative contrast learning, the technology flow feature is updated based on the same technology knowledge in the technology flow feature and the technology collinearity feature to obtain the fusion feature of the technology knowledge.

[0188] Therefore, the fusion features obtained by updating the features of the corresponding technical knowledge in the technology flow features based on the consistency expression results not only integrate the correlation information of technical knowledge from different perspectives, but also retain the directional information of the technology knowledge flow, thereby improving the accuracy of the prediction of the direction of technology knowledge flow.

[0189] In a possible implementation, the apparatus further includes a smoothing unit, and the smoothing unit is configured to:

[0190] Model parameters are smoothed by the exponential moving average technique.

[0191] Therefore, adjusting the relevant parameters of the technology co-occurrence feature through the relevant parameters of the technology flow feature can effectively improve the stability of the model during the feature update process and ensure the stable convergence of the model.

[0192] In a possible implementation, the prediction unit is configured to:

[0193] Fusing the similarity and the propagation weight to obtain an initial flow result of the technical knowledge;

[0194] The initial flow result is optimized by a decoupling loss function to obtain the future flow result. The decoupling loss function includes a loss function designed for the flow of technical knowledge within a field and a loss function designed for the flow of cross-field technical knowledge.

[0195] As a result, the model can better adapt to different flow patterns, whether it is the flow of technical knowledge within the same field or across fields, and can more accurately identify the flow paths and trends between technical knowledge, thereby significantly improving the accuracy of technology flow predictions.

[0196] Based on the above embodiments, an embodiment of the present application provides a computer device, which includes a processor and a memory:

[0197] The memory is used to store computer programs;

[0198] The processor is configured to execute the above-mentioned technical knowledge flow prediction method according to the computer program.

[0199] Based on the above embodiments, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a computer device, it implements the above-mentioned technical knowledge flow prediction method.

[0200] On the basis of the above embodiments, an embodiment of the present application provides a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the above-mentioned technical knowledge flow prediction method.

[0201] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0202] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for predicting technical knowledge flow, characterized in that: The method comprises: Obtaining basic data, including documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document, wherein the technical classification information is used to identify the category of technical knowledge involved in the document; Based on the basic data, a description text of the technical knowledge is generated by a large language model, wherein the description text is used to describe the flow relationship and co-occurrence relationship between technical knowledge; Encoding the technical knowledge based on the description text to obtain initial flow features and initial co-occurrence features; The initial flow features and the initial co-occurrence features are used as node representations in a directed graph sequence and an undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure. The directed graph sequence is used to identify the flow relationship of the technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of the technical knowledge in different time periods. Inputting the directed graph structure and the undirected graph structure into a feature extraction module for feature extraction to obtain technology flow features and technology co-occurrence features, the feature extraction module including a graph neural network (GNN) and a Transformer architecture; By using non-negative contrastive learning, the technology flow features and the technology co-occurrence features are aligned to obtain the fusion features of the technology knowledge; Based on the fusion features, calculating the similarity of the technical knowledge and determining the dissemination weight of the technical knowledge; Based on the similarity and the propagation weight, the future flow result of the technical knowledge is predicted.

2. The method according to claim 1, characterized in that The description text includes a definition, a flow relationship description, and a co-occurrence relationship description. The encoding of the technical knowledge based on the description text to obtain the initial flow features and initial co-occurrence features includes: Based on the definition and the flow relationship description, encoding the technical features using Sentence-BERT technology to obtain the initial flow features; Based on the definition and the flow relationship description, the technical features are encoded using Sentence-BERT technology to obtain the initial co-occurrence features.

3. The method according to claim 1, characterized in that The calculating the similarity of the technical knowledge based on the fusion features includes: Based on the fusion features, the similarity of the technical knowledge is calculated by a bilinear function.

4. The method according to claim 1, wherein The technology flow feature and the technology co-occurrence feature are aligned by non-negative contrast learning to obtain the fusion feature of the technology knowledge, including: Through the non-negative contrast learning, the technology flow feature is updated based on the same technology knowledge in the technology flow feature and the technology collinearity feature to obtain the fusion feature of the technology knowledge.

5. The method according to claim 4, characterized in that During the feature alignment process, the method further includes: Model parameters are smoothed by the exponential moving average technique.

6. The method according to claim 1, characterized in that The predicting of future flow results of the technical knowledge based on the similarity and the propagation weight includes: Fusing the similarity and the propagation weight to obtain an initial flow result of the technical knowledge; The initial flow result is optimized by a decoupling loss function to obtain the future flow result. The decoupling loss function includes a loss function designed for the flow of technical knowledge within a field and a loss function designed for the flow of cross-field technical knowledge.

7. A technology knowledge flow prediction device, characterized in that: The device includes an acquisition unit, a generation unit, an encoding unit, a construction unit, a feature extraction unit, a feature alignment unit, a determination unit, and a prediction unit: The acquisition unit is used to acquire basic data, wherein the basic data includes documents published in different time periods and their corresponding technical classification information, as well as documents cited by each document, wherein the technical classification information is used to identify the category of technical knowledge involved in the document; The generating unit is configured to generate a description text based on the basic data and the technical knowledge using a large language model, wherein the description text is used to describe the flow relationship and co-occurrence relationship between the technical knowledge; The encoding unit is used to encode the technical knowledge based on the description text to obtain initial flow features and initial co-occurrence features; The construction unit is configured to use the initial flow features and the initial co-occurrence features as node representations in a directed graph sequence and an undirected graph sequence, respectively, to construct a directed graph structure and an undirected graph structure, wherein the directed graph sequence is used to identify the flow relationship of the technical knowledge in different time periods, and the undirected graph sequence is used to identify the co-occurrence relationship of the technical knowledge in different time periods; The feature extraction unit is used to input the directed graph structure and the undirected graph structure into a feature extraction module for feature extraction to obtain technology flow features and technology co-occurrence features. The feature extraction module includes a graph neural network (GNN) and a Transformer architecture. The feature alignment unit is used to align the technology flow feature and the technology co-occurrence feature through non-negative contrast learning to obtain the fusion feature of the technology knowledge; The determining unit is configured to calculate the similarity of the technical knowledge based on the fusion features and determine the dissemination weight of the technical knowledge; The prediction unit is used to predict the future flow results of the technical knowledge based on the similarity and the propagation weight.

8. A computer device, characterized in that: The computer device includes a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 6 according to the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a computer device, the computer program implements the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the method is executed on a computer device, the computer device is enabled to execute the method according to any one of claims 1 to 6.

Images