Recommendation method and system based on large language model and time sequence link prediction

Abstract

The invention discloses a recommendation method and system based on a large language model and time sequence link prediction, and the method comprises the steps: obtaining related patent data of an industry field where a target company is located, inputting the related patent data into a trained large language model, and outputting a patent division result of each stage of a patent industry chain in the industry field; constructing a time sequence network from the patent data of each industry chain stage by adopting a sliding window strategy, and extracting time sequence features; inputting the extracted time sequence features into a CNN-GRU model with a multi-head attention mechanism, which is optimized by a rabbet optimization algorithm, outputting a prediction result, and carrying out potential technology fusion opportunity screening and verification; and according to the social influence and the technology focusing degree of the selected technology fusion opportunity, recommending a potential partner of the next stage. The invention aims to accurately identify and recommend potential technical partners from the perspective of time dimension and dynamic change by combining methods of deep learning text mining, a knowledge graph, a multi-dimensional neighbor attribute network and the like.

Description

Technical Field

[0001] The present invention relates to the field of matching recommendation of potential technical partners of an enterprise, and in particular to a method and system for recommending potential technical partners of a target enterprise based on a large language model and temporal link prediction. Background Art

[0002] In the current economic environment, technological innovation is the core driving force for industrial development. However, with the increasing complexity of technology and the strengthening of market diversification, the resources controlled by a single enterprise are relatively limited, making technological innovation increasingly difficult. In order to effectively respond to this challenge, enterprises must strengthen cooperation, share resources, and promote the transfer and utilization of knowledge to accelerate technological innovation. How to explore potential cooperation possibilities and improve the accuracy of mutually beneficial and win-win partner recommendations, thereby innovating technology and expanding the market, has become an important issue in inter-enterprise cooperation.

[0003] The paper "Technology Opportunity Discovery using Deep Learning-based TextMining and a Knowledge Graph" proposes a technology opportunity discovery framework that combines deep learning text mining and knowledge graphs. It uses deep learning algorithms to extract valuable information from large amounts of text data, and uses knowledge graphs to perform association analysis to identify potential technology opportunities. The study emphasizes the importance of new technology base companies (NTBFs) in technology evaluation and uses the investment information of these companies as an important criterion for technology opportunity discovery. Although this method can effectively extract text information and perform association analysis, it does not fully consider the time dimension and dynamic changes of technology integration, and lacks fine-grained analysis of patent co-occurrence relationships.

[0004] The paper proposes a two-stage technology opportunity discovery method based on graph convolutional networks (GCN), combined with machine learning for patent link prediction to identify possible technology fusion opportunities, and verifies its feasibility through technology and market similarity analysis. Although this method has made significant progress in technology opportunity prediction, it does not consider the connection between patents at different levels of the industrial chain, and does not explore in depth how to convert these opportunities into actual partner recommendations, lacking a partner recommendation mechanism for market and technology verification.

[0005] The technology "Two-stage technology opportunity discovery for firm-level decision making: GCN-based link-prediction approach" provides a method for predicting cooperative network links based on a multi-dimensional proximity attribute network. The autoencoder model, joint probability model and attribute Skip-Gram model are used to retain multi-dimensional proximity features, local network features and global network features respectively, and the cooperative network link prediction is performed through the cosine similarity of the vectors corresponding to the network nodes. This method comprehensively considers cognitive proximity features, geographical proximity features and institutional proximity features, but lacks a partner recommendation mechanism for market and technology verification.

[0006] Patent application CN201910568895.1 published on January 17, 2020 provides a patent-based school-enterprise cooperation recommendation algorithm, establishes a UIC network, and makes school-enterprise cooperation recommendations by calculating sampling path sets and path sampling sets. This method uses patent data to establish a network model and combines the path sampling method to make cooperation recommendations, thereby improving the accuracy of school-enterprise cooperation recommendations. This method provides a way of thinking for enterprise cooperation and recommendation, but does not take into account the time dimension and dynamic changes, and ignores the time dynamic factors of cooperative relationships and technological integration.

[0007] In summary, current research has proposed a variety of frameworks for technology opportunity discovery and partner recommendation based on methods such as deep learning, graph convolutional networks, and multi-dimensional proximity attribute networks, but each has certain limitations, such as lack of consideration of the time dimension and dynamic changes, insufficient exploration of the connection between patents at different levels of the industrial chain, and the dynamic factors of cooperative relationships and technological integration. On the one hand, this requires future research to further integrate the advantages of these methods and focus on multi-dimensional and multi-level comprehensive analysis; on the other hand, it is necessary to strengthen the consideration of time dynamic factors and the partner recommendation mechanism of actual market technology verification to improve the effectiveness of technological innovation and market expansion. Summary of the invention

[0008] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a recommendation method and system based on a large language model and temporal link prediction, which aims to accurately identify and recommend potential technology partners from the perspective of time dimension and dynamic changes by combining methods such as deep learning text mining, knowledge graphs and multi-dimensional proximity attribute networks.

[0009] According to one aspect of the present invention, a recommendation method based on a large language model and temporal link prediction is provided, comprising:

[0010] Obtain relevant patent data in the industry where the target company is located and input it into the trained large language model to output the patent division results of each stage of the patent industry chain in the industry;

[0011] A sliding window strategy is used to construct a time series network from the patent data of each industrial chain stage and extract time series features;

[0012] The extracted time series features are input into the CNN-GRU model with multi-head attention mechanism optimized by the transit optimization algorithm, the prediction results are output, and potential technology fusion opportunities are screened and verified;

[0013] Recommend potential partners for the next stage based on the social impact and technical focus of the selected technology convergence opportunities.

[0014] As a further technical solution, the training of the large language model includes:

[0015] Randomly extract a portion of data from the patent dataset for annotation, and divide the annotated dataset into a training set and a test set;

[0016] Use the initially designed Prompt template on the training set, input the patent title and abstract, and output the industry chain stage label;

[0017] Based on the differences between the training set output results and the annotation results, gradually adjust the role description, instruction instructions, and question settings in the Prompt template;

[0018] Multiple tests are performed on the adjusted Prompt template, and the Prompt template with the best annotation effect is selected for batch processing of the entire data set.

[0019] As a further technical solution, a sliding window strategy is used to construct a time series network from the patent data of each industrial chain stage and extract time series features, including:

[0020] The patent data in each industrial chain stage are divided into non-overlapping time intervals in chronological order, and a snapshot of the patent co-occurrence network in each time interval is constructed;

[0021] The sliding window size is set so that the co-occurrence relationship of node pairs in the time snapshot is calculated in each sliding window;

[0022] For each network snapshot in a sliding window, the number of common neighbors, Jaccard coefficient, Adamic / Adar coefficient, local path index, and average commuting time of the node pair in the past sliding window are calculated.

[0023] As a further technical solution, the extracted time series features are input into a CNN-GRU model with a multi-head attention mechanism optimized by the transit optimization algorithm, and the prediction results are output, including: using a convolutional neural network CNN to extract local time series features, using a gated recurrent unit GRU to capture the time dependencies of long sequences, introducing a multi-head attention mechanism to enhance the attention to important parts of the time series features, and multiple groups of weighted outputs from the multi-head attention mechanism are activated through a fully connected layer, and the output represents the probability value of whether the predicted node pair will generate a link in the future time step.

[0024] As a further technical solution, the transit optimization algorithm is used to optimize the CNN-GRU model with a multi-head attention mechanism, which also includes: using the transit search optimization algorithm to explore and search for the best parameter combination by simulating the transit phenomenon of celestial bodies. In each search iteration, the improvement of model performance is measured as the degree of light attenuation, and finally the global optimal solution is found.

[0025] As a further technical solution, potential technology fusion opportunities are screened and verified, including: market novelty calculation, technology stability assessment and technology impact measurement, feature normalization and weighted calculation of the three indicators of market novelty, technology stability and technology impact, to obtain potential technology fusion opportunities.

[0026] As a further technical solution, potential partners for the next stage are recommended based on the social impact and technical focus of the selected technology integration opportunities, including:

[0027] Determine social influence based on the number of patents the company has in the target technology field, the number of times its patents have been cited, the types of technologies covered by the patents, and the geographical scope of patent authorization;

[0028] Determine the degree of technological focus based on the proportion of patents in the target technology field and the weight of these patents in its overall technology portfolio;

[0029] Potential partners are ranked based on a combined score of social impact and technology focus.

[0030] According to one aspect of the present invention, a recommendation system based on a large language model and temporal link prediction is provided, comprising:

[0031] The data collection module is used to obtain relevant patent data in the industry field where the target company is located and input it into the trained large language model to output the patent division results of each stage of the patent industry chain in the industry field;

[0032] The network construction module is used to construct a time series network from the patent data of each industrial chain stage using a sliding window strategy and extract time series features;

[0033] The prediction module is used to input the extracted time series features into the CNN-GRU model with multi-head attention mechanism optimized by the transit optimization algorithm, output the prediction results, and screen and verify potential technology fusion opportunities;

[0034] The recommendation module is used to recommend potential partners for the next stage based on the social impact and technology focus of the selected technology convergence opportunities.

[0035] According to one aspect of the present specification, there is provided an electronic device, comprising: at least one processor, at least one memory and a communication interface; wherein the processor, memory and communication interface communicate with each other; the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the described method.

[0036] According to one aspect of the present specification, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the method described above.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention integrates multiple technologies, combines the semantic analysis of large language models with temporal networks, and uses a link prediction method that considers technical associations to improve the efficiency and accuracy of enterprise cooperation prediction. The method of the present invention can help enterprises understand their influence in the industry to optimize resource allocation, recommend suitable partners for enterprises, reduce the cost and time of enterprises in finding partners, promote cooperation and communication between enterprises, and promote innovation and development of the industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction is given below to the drawings used in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A flowchart of a recommendation method based on a large language model and timing link prediction provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the TCGM prediction algorithm provided in an embodiment of the present invention;

[0042] Figure 3 A flowchart of a TSOA optimization algorithm provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a recommendation system based on a large language model and temporal link prediction provided by an embodiment of the present invention.

[0044] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] It should be noted that:

[0046] The terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to the steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0047] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices. The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all the contents and operations / steps, nor do they necessarily have to be executed in the order described. For example, some operations / steps may be further decomposed, while some operations / steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention are arbitrarily combined with each other to form a new technical solution. This combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that this combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0049] The embodiment of the present invention discloses a recommendation method based on a large language model and time-series link prediction, which is used to recommend matching potential technical partners of a target enterprise. The method specifically includes:

[0050] S1: Select relevant patent data in the industry where the target company is located, use the large language model (LLM) to annotate the patent data in this field by analyzing the semantic content of the patent text, and classify the patents according to the upstream, midstream and downstream of the industrial chain;

[0051] S2: A sliding window strategy is used to construct a time series network from the patents of each stage, and the input data for link prediction is constructed by extracting five time series features: local information, path information, random walk information, node information, and community information;

[0052] S3: Use the improved CNN-GRU combined with the multi-head attention mechanism (MHAE) and the transit optimization algorithm (TSOA) to optimize the link prediction to effectively improve the prediction accuracy of technology fusion opportunities. Screen and verify potential technology fusion opportunities in different periods through factors such as market novelty, technology stability and technology influence;

[0053] S4: Recommend potential partners for the next stage based on the social impact and technology focus of the selected technology convergence opportunities.

[0054] As a preferred embodiment, Figure 1 As shown, the embodiment of the present invention is described from four aspects: data collection and industrial chain division, time series network construction and feature extraction, enterprise-level technology integration opportunity discovery, and potential cooperative enterprise recommendation.

[0055] In an embodiment of the present invention, the data collection and industrial chain division link includes data collection and industrial chain division, which are used to semantically annotate patent data and divide the industrial chain stages. Specifically, this link selects relevant patent data in the industry field where the target company is located, and uses a large language model (LLM) to process the semantic content of the patent text. By analyzing the title and abstract of the patent, it is classified into the upstream, midstream or downstream stages of the industrial chain. The LLM is fine-tuned using the preset "role-instruction" prompt strategy, and some patents are manually annotated, and these annotation results are used to train the LLM model to improve its classification accuracy. The final model is used to automatically classify the remaining patent data in batches to achieve patent division at various stages of the industry chain.

[0056] In the data collection step, the embodiment of the present invention collects and analyzes data based on patent data in a certain industry. Patent data is selected according to the semiconductor technology classification of the World Intellectual Property Organization. The data comes from the Orbis database, does not include design and utility model patents, and is limited to the U.S. Patent Office. The content of patent data includes information such as publication number, IPC classification number, priority date, title, abstract, and the name of the current direct right holder.

[0057] In the industrial chain division step of the embodiment of the present invention, the large language model (LLM) used is a natural language processing technology based on deep learning, which can understand and generate natural language text. In order to accurately match the collected specific industry patents with the corresponding industrial chain stages, this link uses ChatGPT combined with the industrial chain segmentation standards to construct prompts. By performing semantic analysis on the title and abstract of each patent, it is determined whether the patent belongs to the upstream, midstream or downstream link of the industrial chain, thereby completing the division of the industrial chain.

[0058] When designing the prompt project, the embodiment of the present invention adopts the "role indication prompt" project strategy. In order to further improve the accuracy of the industry chain interpretation, a fine-tuning strategy is also adopted to optimize the prompt involvement, thereby improving the accuracy of reasoning. Specifically, a portion of the data is randomly selected for manual labeling, and the classification and terminology in the industry chain segmentation standard are continuously improved to ensure that the optimized prompt improves the accuracy of the manually labeled data as much as possible.

[0059] Specifically, the Prompt fine-tuning method includes: randomly extracting a part of the data from the patent data set for manual annotation, and dividing the annotated data set into a training set and a test set; using the initially designed Prompt template on the training set, inputting the patent title and abstract, and outputting the industrial chain stage label; based on the difference between the output results of the training set and the manual annotation results, gradually adjusting the role description, instruction instructions and question settings in the Prompt template to improve the annotation accuracy of the model; performing multiple tests on the adjusted Prompt, and selecting the Prompt with the best annotation effect for batch processing of the entire data set.

[0060] After consulting a large number of literature on the division of the semiconductor industry chain, the embodiment of the present invention integrates the existing research division standards as the basic rules for the division of the semiconductor industry chain stages. The upstream of the industry chain includes raw materials and equipment, specifically the extraction and preparation of semiconductor raw materials and the manufacture of semiconductor equipment. The midstream includes semiconductor devices, including the design, material preparation technology and methods, manufacturing, packaging and testing of semiconductor devices. The downstream involves specific solutions or applications related to semiconductors. In addition, each specific technical category mentioned in the literature is also listed under the basic rules. When designing ChatGPT prompts, the embodiment of the present invention selected the "role-instruction" strategy, which includes four parts: role, instruction, content and question. The role describes the identity of the task performer; the instruction outlines the specific task, the division criteria of the industry chain and the requirements for ChatGPT output; the content refers to the specific content that needs to be processed, in this case the title and abstract of the patent; the question raises the question that needs to be answered. Finally, the division of the semiconductor industry chain is completed using a large model.

[0061] In an embodiment of the present invention, time series network construction and feature extraction are used to construct a time series network and extract features based on the data obtained from the data collection and industrial chain division links. Specifically, the patent data in each industrial chain stage are divided into non-overlapping time intervals (Δt) in chronological order. In each time interval, a patent co-occurrence network snapshot is constructed based on the first six IPC classification numbers. The nodes represent the technical fields, and the weights of the edges represent the number of patents between the two technical fields. All snapshots are arranged in chronological order to form a time series network. Using the sliding window method, the features of node pairs are extracted from continuous time snapshots. The features include: the number of common neighbors (CN), Jaccard coefficient (JC), Adamic / Adar coefficient (AA), local path index (LP), and average commuting time (ACT). Time series features are constructed from five aspects: local information, path information, random walk information, node information, and community information.

[0062] This link processes the patent data in each industry chain separately, and cuts them continuously and non-overlappingly according to the time sequence of the priority data within a certain time interval Δt. And all patents within each Δt are constructed as group-level technology co-occurrence network snapshots based on the first six IPCs. The nodes in represent technical fields, and the weights of the edges represent the number of patents belonging to the same two technical fields. Merge in chronological order to obtain the constructed temporal network G, as shown below:

[0063]

[0064] In order to obtain input and output features from G that take into account the temporal technology convergence relationship for link prediction, we use a sliding window method. The window size ω solves the problem of network sparseness and unclear features in a single snapshot. The window slides backward in a sliding manner to extract features in sequence. Find and The common nodes N between i .from Select in There are edges but There are no node pairs n in which there is no edge i (n i ∈N i ). Using indicators such as common neighbors (CN), Jaccard coefficient (JC), Adamic / Adar coefficient (AA), local path index (LP), and average commuting time (ACT), n is calculated from five aspects: local information, path, random walk, node attributes, and community. iIn order to reduce the bias of the classifier to the majority class (negative samples) and improve the recognition ability of the minority class (positive samples), undersampling is used to balance the distribution of classes when constructing the feature data of negative samples.

[0065] In the embodiment of the present invention, the enterprise-level technology fusion opportunity discovery is used to predict technology fusion opportunities based on the temporal features extracted in the previous link. That is, after extracting the time features, in order to better capture the temporal pattern and learning features, the technology fusion opportunities are trained and tested using the CNN-GRU (TCGM) optimized with TSOA and multi-head attention mechanism. In order to find technology fusion opportunities at the enterprise level, the market novelty, technology stability and technology influence of a specific enterprise are calculated to determine the opportunities for technology fusion.

[0066] Specifically, a model (TCGM) combining a convolutional neural network (CNN) and a gated recurrent unit (GRU) is constructed. Multiple convolution kernels are used in the CNN part to extract local temporal features, and the temporal dependencies of long sequences are processed in the GRU part. A multi-head attention mechanism (MHAM) is introduced to enhance the focus on important parts of temporal features, and finally feature fusion and output are completed in the fully connected layer. In the prediction process, the transit search optimization algorithm (TSOA) is used to adjust model parameters such as convolution kernel size, number of GRU units, and learning rate to ensure the generalization ability of the model across different technical fields. In the verification of technology fusion opportunities for target enterprises, by evaluating indicators such as market novelty, technical stability, and technical influence, combined with patent data within a specific time window, technology fusion opportunities are dynamically screened and verified to ensure the sustainability and actual cooperation value of these opportunities in different time periods.

[0067] Convolutional Neural Network (CNN) Feature Extraction. The input features are the feature matrix of time series node pairs extracted from the time series network. The CNN part first convolves the input sequence features through multiple one-dimensional convolution kernels. The size and step size of the convolution kernel are carefully designed to adapt to the time series features of different scales. Through the weight sharing mechanism of the convolution layer, CNN is able to capture local patterns and short-term dependency characteristics in the time series. After the convolution operation, the batch normalization layer is used to standardize the output to prevent the gradient from disappearing or exploding. Subsequently, the pooling layer is used to downsample the features to extract significant features with translation invariance. The pooling operation helps to reduce the dimension of the features and reduce the risk of overfitting.

[0068] Gated Recurrent Unit (GRU) captures temporal dependencies. The features extracted from the CNN part are input to the GRU layer, which is a recurrent neural network (RNN) variant for time series data processing, with an update gate and a reset gate structure. The update gate determines how much previous information the current state will retain, while the reset gate controls the degree of integration of the current input and the previous state. These gating mechanisms of GRU enable the model to effectively capture long-term dependencies and sequence patterns, especially when processing longer time series, and can avoid the gradient vanishing problem of traditional RNNs. The output of the GRU layer represents the dynamic change pattern in the time series features.

[0069] The Multi-Head Attention Mechanism (MHAM) enhances feature capture capabilities. The output of the GRU layer is passed to the Multi-Head Attention Mechanism layer. MHAM focuses on features at different locations through multiple independent attention heads (AttentionHead) operating in parallel. Each attention head first performs a linear transformation on the input sequence to generate a query, key, and value matrix. Then, the dot product similarity between the query and the key is calculated, and after scaling and softmax normalization, the attention weights are obtained. These weights are used to weight the value matrix to generate a weighted output. The parallel structure of the Multi-Head Attention Mechanism allows the model to view input features from different perspectives, improving the ability to learn complex patterns.

[0070] Feature fusion and fully connected layer output. Multiple groups of weighted outputs from the multi-head attention mechanism are concatenated to form a comprehensive feature vector, which contains multiple pattern information from the time series. The comprehensive features are input into the fully connected layer, activated by a nonlinear activation function (such as ReLU), and the high-dimensional features are mapped to the prediction space. The final output layer uses the sigmoid activation function to generate a probability value between 0 and 1, indicating whether the predicted node pair will have a link in the future time step (1 means there is a link, 0 means there is no link).

[0071] Transit Search Optimization Algorithm (TSOA) parameter optimization. The hyperparameters of the entire TCGM model, including convolution kernel size, pooling window size, number of GRU neurons, number of attention heads, and learning rate, are optimized using the Transit Search Optimization Algorithm. TSOA explores and searches for the best parameter combination by simulating the transit of celestial bodies. In each search iteration, the improvement in model performance is measured as the degree of light attenuation, and the global optimal solution is finally found. The optimization process involves multi-stage search and local fine-tuning to ensure that the model has the best generalization ability and prediction performance in complex time series networks.

[0072] In an embodiment of the present invention, the CNN-GRU prediction with multi-head attention mechanism optimized by TSOA includes a TCGM overall prediction process and a TSOA optimization process.

[0073] In the overall prediction process of TCGM, the prediction principle diagram of CNN-GRU with multi-head attention mechanism after TSOA optimization is as follows Figure 2 First, we use the positive and negative sample data extracted from the temporal network in turn, that is, from [G ti , G ti+ω ] is input into the CNN layer, local spatial features are extracted through convolution operations, and downscaling and overfitting are prevented through pooling layers and elimination layers. Secondly, the processed data is passed to the GRU layer, where time series information is obtained. Then, the data enters the MHAM layer, and the hidden state output by the GRU layer is mapped to key, value, and query vectors through linear transformation and attention mechanism. Then, the attention weight is calculated through the mask scaling dot multiplication attention mechanism to strengthen the focus on important features. Finally, after further processing in the fully connected layer, the data enters the output layer to generate the prediction result, that is, the output is G ti+ω+1 Whether the corresponding node pair in is linked (1 means linked, 0 means not linked). The result of link prediction can be expressed as whether the two technical fields will be fused in the next step when the input feature is the time feature between the two technical fields in the past ω step.

[0074] In the TSOA optimization process, the entire link prediction process is optimized by TSOA, adjusting the model parameters to maximize the performance. The TSOA optimization process is mainly divided into five stages, such as Figure 3As shown. In the galaxy phase, the algorithm randomly selects locations in the search space, called "galaxies," to identify potential optimal regions. In the transit phase, the algorithm remeasures the light received from the star to evaluate potential solutions. In the planet phase, the algorithm determines the initial positions of potential solutions (planets) and refines these positions through multiple observations. In the neighbor phase, if no transit is detected, the algorithm checks the neighbors of previously detected planets and replaces them if better conditions are found. In the development phase, the algorithm repeatedly optimizes the best planetary characteristics for each star to find the global optimal solution. These phases enable TSOA to systematically explore and optimize potential solutions and efficiently solve complex optimization problems.

[0075] The specific optimization parameters of TSOA include: first, the size of the convolution kernel. The main purpose of the optimization is to enhance the effect of spatial feature extraction in CNN; second, the number of neurons in GRU. The purpose of the optimization is to improve the model's ability to capture time series information; third, the learning rate of the overall model. The purpose of the optimization is to ensure the stability of the model during training and improve the convergence speed.

[0076] In the verification of technology convergence opportunities at the enterprise level in the embodiment of the present invention, in order to find technology convergence opportunities at the enterprise level, it is necessary to screen the six-digit IPC to which the target enterprise's existing technology belongs in the prediction results of the test set. In the screening process, the time when the target enterprise has owned the technology will be considered to ensure that potential convergence opportunities are only possible after the target enterprise has owned the technology. However, not all pairings generated with technical documents owned by the target company are suitable. Therefore, the embodiment of the present invention proposes market novelty (MN), technology stability (TR) and technology influence (TI) for further verification.

[0077] Market Novelty (MN): This indicator is used to measure the degree of innovation in a certain technology field within a specific time window. Market novelty is calculated by counting the number of patents that appear for the first time in the time window, focusing on the frequency of new inventions or innovative technologies in the technology field. This indicator reflects the activity of the technology field and the rate of technological innovation, indicating the forward-looking nature of potential technological opportunities in the field.

[0078] Furthermore, market novelty calculation refers to the number of new patents that appear in the target technology field within a given time window, which is used to measure the innovation activity in the field. First, a fixed-length time window is selected (for example, the past 12 months), and the number of all patents belonging to the target technology field within the time window is counted. Then, these patents are sorted by priority date, and the number of patents that first appeared in the time window is extracted. By comparing the number of new patents in different periods, the market novelty index is calculated to reflect the innovation dynamics in the current technology field.

[0079] The formula for market novelty is as follows:

[0080]

[0081] Among them, MN represents the number of patents belonging to the same two technical fields within M months before the fusion occurs, P(u) represents the patent set in technical field u, M represents the number of months before the fusion occurs, and T represents the time when the fusion occurs.

[0082] Technological Reliability (TR): This indicator evaluates the stability and continuity of a technology field over multiple time periods. This indicator reflects the continuity of technology cooperation and technology implementation by analyzing the changes in edge weights of node pairs within a given time window. Technological stability emphasizes the frequency and intensity of long-term cooperation and repeated technology interactions, and provides a quantitative analysis of the stability of long-term technology cooperation relationships in the technology field, avoiding the interference of short-term fluctuations.

[0083] Furthermore, technology stability assessment refers to defining technology stability as the average value of edge weights of node pairs in the past time period in a time series network. In the specific calculation process, all relevant patents in the target technology field are first selected to construct a technology co-occurrence network, in which nodes represent specific technology fields and edge weights represent the co-occurrence frequency of patents belonging to that field. Then, a time window is set (for example, the past 24 months), and the average value of edge weights of each node pair in the time window is calculated. In this way, the technology stability indicator can reflect the continuity and stability of the technology field in that time period.

[0084] The formula for technical stability is as follows:

[0085]

[0086] Among them, TR represents the average edge weight of paired nodes in the past N months, N represents the number of months before the fusion occurs, and F u,t Represents the edge weight of node u at time t.

[0087] Technological Influence (TI): This indicator is used to evaluate the importance and influence of a technology field in the entire technology network. Technological influence is achieved by calculating the eigenvector centrality of the node, reflecting the connectivity and radiation capacity of the node in the technology network. Highly influential nodes are usually directly connected to multiple key technology fields, and these fields themselves are also highly important. Therefore, the technological influence indicator captures the ability of a specific technology field to serve as a core hub in the entire technology ecosystem, influencing the diffusion and development of new technologies.

[0088] Furthermore, the measurement of technological influence refers to the use of eigenvector centrality to measure the importance of nodes in a temporal network. The calculation of technological influence involves the following steps: First, a global technological network is constructed, in which the nodes represent various technological fields, and the weights of the edges represent the co-occurrence relationship between them. Then, the eigenvector centrality of each node is calculated. This centrality value is obtained by iteratively solving the influence distribution of the node, reflecting the importance of a node in the entire technological network. A higher centrality means that the technology is connected to multiple high-impact technological fields in the network, thus having a greater technological influence.

[0089] The formula for technology influence is as follows:

[0090]

[0091] Among them, TI represents the eigenvector centrality of paired nodes in the network in the past N months, V u [t] represents the eigenvector centrality of node u at time t.

[0092] The three indicators of market novelty, technological stability, and technological influence are normalized to eliminate the differences between different feature dimensions. The normalization method uses Min-Max normalization to convert the value of each indicator to the range of [0,1]. The normalized feature values ​​are linearly weighted according to the preset weights (for example, w1, w2, and w3 correspond to MN, TR, and TI, respectively) to generate a comprehensive score. The choice of weights can be based on specific application scenarios, and the weight combination can be optimized through multiple experiments and verifications to ensure that the final score reflects the credibility of actual technology integration opportunities.

[0093] Specifically, and After normalization, the Technology Recommendation Index (TRI) of each potential technology opportunity is calculated, and the top 10 values ​​corresponding to the target company's technology opportunities are selected. The specific calculation method is as follows:

[0094]

[0095] In an embodiment of the present invention, the recommendation of potential cooperative enterprises is used to recommend potential partners of the enterprise based on the technology integration opportunities obtained in the previous link. Specifically, based on the results of technology integration prediction, potential technology cooperation opportunities are identified. By calculating the degree of matching of technical content, possible partners are screened and sorted according to the technical fields they own within the prediction time window. By quantifying the social influence and technical focus of potential partners, the partner recommendation index (PRI) is calculated in combination with a preset weight coefficient, and finally a list of potential partners for the target enterprise is generated.

[0096] This step matches the corresponding potential partners according to the target company's technology convergence opportunities. Each technology convergence opportunity has a time of convergence with the technology owned by the target company. First, find a list of partners that includes all partners who have the corresponding technology before the technology convergence occurs. Secondly, in order to accurately identify the best candidate partners from the above list of companies, two measurement criteria are proposed: social influence (SI) and technology focus (TF). Social influence considers the technology maturity of the company in a specific field, while technology focus evaluates the company's focus on related technologies.

[0097] Social influence is used to measure a potential partner's technological accumulation and market performance in a specific technological field. It is determined by calculating the number of patents the company has in the target technological field, the number of citations of its patents, the types of technologies covered by the patents, and the geographical scope of patent authorization. The level of social influence reflects the company's technological depth and market influence in the field, and these factors help evaluate its value contribution in technological cooperation.

[0098] The quantification of social influence is expressed by the number of matching technology fields v that potential partner x has before technology convergence occurs at time T, which reflects the technology accumulation and innovation capabilities of partners in specific fields. Partners with high SI values ​​can provide more resources in the process of technology convergence and increase the possibility of successful cooperation.

[0099]

[0100] Among them, t i represents the time point before technological convergence occurs, P(x,v,t i ) indicates that partner x is in t i The patent collection in technology field v owned at a point in time.

[0101] Technological Focus (TF) indicates the degree of concentration of an enterprise in a specific technology field. By analyzing the proportion of patents in a specific technology field among all the patents of an enterprise, the concentration and specialization level of its technology strategy are evaluated. Specifically, technological focus considers the proportion of the number of patents of the enterprise in this technology field and the weight of these patents in its overall technology portfolio. High technological focus indicates that the enterprise has strong technological advantages and continuous innovation capabilities in this field, and is suitable for in-depth cooperation with it.

[0102] The quantitative indicator of technology focus is the ratio of the number of patents owned by x before T to the number of all patents. The higher the TF, the more likely the company's technology development strategy is to focus on a highly concentrated and specialized field, and will provide more professional and in-depth technical support in cooperation.

[0103]

[0104] Among them, P(x,t i ) indicates that partner x is in t i The set of all patents owned at time, |·| represents the number of elements in the set.

[0105] Finally, in order to comprehensively consider the above two indicators, the results are normalized and then W SI and W TF Weighted, we get the partner recommendation index, which is calculated as follows:

[0106]

[0107] Based on the ranking of the partner recommendation index, partners that are most likely to cooperate in areas of technology convergence within a fine-grained time frame in the future are recommended to the target company.

[0108] It should be noted that the Partner Recommendation Index (PRI) is a comprehensive score based on market influence and technology focus, which is used to rank potential partners. The index forms a comprehensive indicator for evaluating the potential for corporate cooperation by weighted integration of market influence and technology focus. The weighting factors are adjusted according to actual application scenarios and strategic goals to ensure that the recommended partners meet the cooperation needs of target companies in terms of technical capabilities and market coverage. The level of PRI directly affects the final partner recommendation list, aiming to optimize cooperation effects and innovation results.

[0109] The implementation basis of each embodiment of the present invention is to implement programmed processing through a device with a processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this reality, on the basis of the above embodiments, an embodiment of the present invention provides a recommendation system based on a large language model and timing link prediction, which is used to execute the recommendation method based on a large language model and timing link prediction in the above method embodiment.

[0110] See also Figure 4 The system includes: a data collection module, which is used to obtain relevant patent data in the industry field where the target company is located and input it into the trained large language model, and output the patent division results of each stage of the patent industry chain in the industry field; a network construction module, which is used to construct a time series network from the patent data of each stage of the industry chain using a sliding window strategy, and extract time series features; a prediction module, which is used to input the extracted time series features into a CNN-GRU model with a multi-head attention mechanism optimized by the transit optimization algorithm, output the prediction results, and screen and verify potential technology fusion opportunities; a recommendation module, which is used to recommend potential partners for the next stage based on the social influence and technology focus of the selected technology fusion opportunities.

[0111] The recommendation system based on large language model and time series link prediction provided by the embodiment of the present invention adopts Figure 4 It integrates several modules in the project, integrates multiple technologies, combines the semantic analysis of large language models with temporal networks, and uses link prediction methods that consider technical associations to improve the efficiency and accuracy of enterprise cooperation predictions.

[0112] It should be noted that the system embodiment provided by the present invention is used to implement the methods in the above method embodiment as well as the methods in other method embodiments provided by the present invention. The only difference is that the corresponding functional modules are set. The principle is basically the same as the principle of the above system embodiment provided by the present invention. As long as the technical personnel in this field refer to the specific technical solutions in other method embodiments on the basis of the above system embodiment, obtain the corresponding technical means and the technical solutions composed of these technical means by combining the technical features, and on the premise of ensuring the practicality of the technical solutions, improve the modules in the above system embodiment to obtain the corresponding system class embodiments, which are used to implement the methods in other method class embodiments. For example:

[0113] Based on the content of the above system embodiment, as a preferred embodiment, in the recommendation system based on large language model and timing link prediction provided in the embodiment of the present invention, the data collection module is further used to execute the following instructions:

[0114] Randomly extract a portion of data from the patent dataset for annotation, and divide the annotated dataset into a training set and a test set;

[0115] Use the initially designed Prompt template on the training set, input the patent title and abstract, and output the industry chain stage label;

[0116] Based on the differences between the training set output results and the annotation results, gradually adjust the role description, instruction instructions, and question settings in the Prompt template;

[0117] Multiple tests are performed on the adjusted Prompt template, and the Prompt template with the best annotation effect is selected for batch processing of the entire data set.

[0118] Based on the content of the above system embodiment, as a preferred embodiment, in the recommendation system based on large language model and timing link prediction provided in the embodiment of the present invention, the network construction module is also used to execute the following instructions:

[0119] The patent data in each industrial chain stage are divided into non-overlapping time intervals in chronological order, and a snapshot of the patent co-occurrence network in each time interval is constructed;

[0120] The sliding window size is set so that the co-occurrence relationship of node pairs in the time snapshot is calculated in each sliding window;

[0121] For each network snapshot in a sliding window, the number of common neighbors, Jaccard coefficient, Adamic / Adar coefficient, local path index, and average commuting time of the node pair in the past sliding window are calculated.

[0122] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and timing link prediction provided in the embodiment of the present invention, the prediction module is also used to execute the following instructions:

[0123] Convolutional neural network (CNN) is used to extract local temporal features, gated recurrent unit (GRU) is used to capture the temporal dependencies of long sequences, and a multi-head attention mechanism is introduced to enhance the attention to important parts of temporal features. Multiple groups of weighted outputs from the multi-head attention mechanism are activated through a fully connected layer, and the output represents the probability value of whether the predicted node pair will generate a link in the future time step.

[0124] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and timing link prediction provided in the embodiment of the present invention, the prediction module is also used to execute the following instructions:

[0125] The transit search optimization algorithm is used to explore and search for the best parameter combination by simulating the transit phenomenon of celestial bodies. In each search iteration, the improvement of model performance is measured as the degree of light attenuation, and finally the global optimal solution is found.

[0126] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and timing link prediction provided in the embodiment of the present invention, the prediction module is also used to execute the following instructions:

[0127] Calculate market novelty, evaluate technology stability and measure technology impact, normalize and weight the three indicators of market novelty, technology stability and technology impact, and obtain potential technology integration opportunities.

[0128] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and timing link prediction provided in the embodiment of the present invention, the recommendation module is also used to execute the following instructions:

[0129] Determine social influence based on the number of patents the company has in the target technology field, the number of times its patents have been cited, the types of technologies covered by the patents, and the geographical scope of patent authorization;

[0130] Determine the degree of technological focus based on the proportion of patents in the target technology field and the weight of these patents in its overall technology portfolio;

[0131] Potential partners are ranked based on a combined score of social impact and technology focus.

[0132] The method of the embodiment of the present invention is implemented by relying on electronic devices, so it is necessary to introduce the relevant electronic devices. Based on this purpose, the embodiment of the present invention provides an electronic device, such as Figure 5 As shown, the electronic device includes: at least one processor, a communication interface, at least one memory, and a communication bus, wherein at least one processor, the communication interface, and at least one memory communicate with each other through the communication bus. At least one processor calls the logic instructions in at least one memory to execute all or part of the steps of the method provided by the aforementioned various method embodiments.

[0133] In addition, when the logic instructions in the above-mentioned at least one memory are implemented in the form of a software functional unit and sold or used as an independent product, they are stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or partly in other words, the part that contributes to the prior art or part of the technical solution is embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for enabling a computer device (a personal computer, a server, or a network device) to perform all or part of the steps of the method described in each method embodiment of the present invention. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and various media for storing program codes.

[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] These computer usable program codes may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A recommendation method based on a large language model and temporal link prediction, characterized in that: include: Obtain relevant patent data in the industry where the target company is located and input it into the trained large language model to output the patent division results of each stage of the patent industry chain in the industry; A sliding window strategy is used to construct a time series network from the patent data of each industrial chain stage and extract time series features; The extracted time series features are input into the CNN-GRU model with multi-head attention mechanism optimized by the transit optimization algorithm, the prediction results are output, and potential technology fusion opportunities are screened and verified; Recommend potential partners for the next stage based on the social impact and technical focus of the selected technology convergence opportunities.

2. The recommendation method based on large language model and temporal link prediction according to claim 1, characterized in that: The training of the large language model includes: Randomly extract a portion of data from the patent dataset for annotation, and divide the annotated dataset into a training set and a test set; Use the initially designed Prompt template on the training set, input the patent title and abstract, and output the industry chain stage label; Based on the differences between the training set output results and the annotation results, gradually adjust the role description, instruction instructions, and question settings in the Prompt template; Multiple tests are performed on the adjusted Prompt template, and the Prompt template with the best annotation effect is selected for batch processing of the entire data set.

3. The recommendation method based on large language model and temporal link prediction according to claim 1, characterized in that: A sliding window strategy is used to construct a time series network from the patent data of each industrial chain stage, and extract time series features, including: The patent data in each industrial chain stage are divided into non-overlapping time intervals in chronological order, and a snapshot of the patent co-occurrence network in each time interval is constructed; The sliding window size is set so that the co-occurrence relationship of node pairs in the time snapshot is calculated in each sliding window; For each network snapshot in a sliding window, the number of common neighbors, Jaccard coefficient, Adamic / Adar coefficient, local path index, and average commuting time of the node pair in the past sliding window are calculated.

4. The recommendation method based on large language model and temporal link prediction according to claim 1, characterized in that: The extracted time series features are input into the CNN-GRU model with a multi-head attention mechanism optimized by the transit optimization algorithm, and the prediction results are output, including: using the convolutional neural network CNN to extract local time series features, using the gated recurrent unit GRU to capture the time dependency of long sequences, introducing the multi-head attention mechanism to enhance the attention to the important parts of the time series features, and multiple groups of weighted outputs from the multi-head attention mechanism are activated through the fully connected layer, and the output represents the probability value of whether the predicted node pair will generate a link in the future time step.

5. The recommendation method based on large language model and temporal link prediction according to claim 4, characterized in that: The transit optimization algorithm is used to optimize the CNN-GRU model with a multi-head attention mechanism, which also includes: using the transit search optimization algorithm to explore and search for the best parameter combination by simulating the transit phenomenon of celestial bodies. In each search iteration, the improvement of model performance is measured as the degree of light attenuation, and finally the global optimal solution is found.

6. The recommendation method based on large language model and temporal link prediction according to claim 1, characterized in that: Screening and verification of potential technology integration opportunities, including: market novelty calculation, technology stability assessment and technology impact measurement, feature normalization and weighted calculation of the three indicators of market novelty, technology stability and technology impact, to obtain potential technology integration opportunities.

7. The recommendation method based on large language model and temporal link prediction according to claim 1, characterized in that: Based on the social impact and technical focus of the selected technology integration opportunities, potential partners for the next stage are recommended, including: Determine social influence based on the number of patents the company has in the target technology field, the number of citations of its patents, the types of technologies covered by the patents, and the geographical scope of patent authorization; Determine the degree of technological focus based on the proportion of patents in the target technology field and the weight of these patents in its overall technology portfolio; Potential partners are ranked based on a combined score of social impact and technology focus.

8. A recommendation system based on a large language model and temporal link prediction, characterized in that: include: The data collection module is used to obtain relevant patent data in the industry field where the target company is located and input it into the trained large language model to output the patent division results of each stage of the patent industry chain in the industry field; The network construction module is used to construct a time series network from the patent data of each industrial chain stage using a sliding window strategy and extract time series features; The prediction module is used to input the extracted time series features into the CNN-GRU model with multi-head attention mechanism optimized by the transit optimization algorithm, output the prediction results, and screen and verify potential technology fusion opportunities; The recommendation module is used to recommend potential partners for the next stage based on the social impact and technology focus of the selected technology convergence opportunities.

9. An electronic device, characterized in that: include: At least one processor, at least one memory and a communication interface; wherein the processor, memory and communication interface communicate with each other; the memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the method described in any one of claims 1 to 7.

10. A non-transitory computer readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, which cause the computer to execute the method of any one of claims 1 to 7.

Images