Recommendation method and system based on large language model and time series link prediction

Abstract

The application discloses a recommendation method and system based on a large language model and time sequence link prediction, and the method comprises the following steps: obtaining relevant patent data of an industry field where a target company is located and inputting the large language model after training, and outputting patent division results of each stage of the industry field patent industry chain; adopting a sliding window strategy to construct a time sequence network from patent data of each stage of the industry chain, and extracting time sequence features; inputting the extracted time sequence features into a CNN-GRU model with a multi-head attention mechanism optimized by a Lingri optimization algorithm, outputting a prediction result, and performing potential technical fusion opportunity screening and verification; and recommending a potential partner of a next stage according to social influence and technical focus degree of the selected technical fusion opportunity. The application aims to accurately identify and recommend potential technical partners from the time dimension and dynamic change by combining deep learning text mining, a knowledge graph, a multi-dimensional adjacent attribute network and other methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of matching and recommending potential technology partners for enterprises, and in particular to a method and system for recommending potential technology partners for target enterprises based on large language models and time-series link prediction. BACKGROUND

[0002] In the current economic environment, technological innovation is the core driving force for industrial development. However, with the increasing complexity of technology and the increasing trend of market diversification, the resources controlled by a single enterprise are relatively limited, making technological innovation increasingly difficult. In order to effectively respond to this challenge, enterprises must strengthen cooperation, share resources, and promote the transfer and utilization of knowledge, thereby accelerating technological innovation. How to explore potential cooperation possibilities and improve the accuracy of win-win cooperation partner recommendations has become an important problem in inter-enterprise cooperation.

[0003] The document "Technology Opportunity Discovery using Deep Learning-based TextMining and a Knowledge Graph" proposes a framework for technology opportunity discovery combining deep learning text mining and a knowledge graph. It uses deep learning algorithms to extract valuable information from large amounts of text data and conducts correlation analysis through a knowledge graph to identify potential technology opportunities. The study emphasizes the importance of new technology-based firms (NTBFs) in technology evaluation and uses investment information from these firms as an important criterion for technology opportunity discovery. Although this method can effectively extract text information and conduct correlation analysis, it does not fully consider the time dimension and dynamic changes of technology fusion, and lacks a fine-grained analysis of patent co-occurrence relationships.

[0004] The document proposes a two-stage technology opportunity discovery method based on graph convolutional networks (GCN), combining machine learning for patent link prediction to identify potential technology fusion opportunities and verifying their feasibility through technology and market similarity analysis. Although this method has made significant progress in technology opportunity prediction, it does not consider the connections between patents at different industry chain levels and does not delve into how to convert these opportunities into actual partner recommendations, lacking a mechanism for market and technology-validated partner recommendations.

[0005] The technology "Two-stage technology opportunity discovery for firm-level decision making: GCN-based link-prediction approach" provides a method for predicting links in a collaboration network based on a multi-dimensional proximity attribute network. The self-encoder model, joint probability model and attribute Skip-Gram model are used to retain multi-dimensional proximity features, local network features and global network features, respectively, and the cosine similarity of the vectors corresponding to the nodes in the network is used to predict the links in the collaboration network. This method takes into account cognitive proximity, geographical proximity and institutional proximity, but lacks a mechanism for recommending partners for market and technology validation.

[0006] Patent application CN201910568895.1, published on January 17, 2020, provides a school-enterprise cooperation recommendation algorithm based on patents, establishes a UIC network, and performs school-enterprise cooperation recommendation by calculating a sample path set and a path sampling set. This method establishes a network model based on patent data and combines a path sampling method to perform cooperation recommendation, improving the accuracy of school-enterprise cooperation recommendation. This method provides an approach for enterprise cooperation and recommendation, but does not consider the time dimension and dynamic changes, ignoring the time dynamic factors of cooperation relationships and technology integration.

[0007] In summary, current research has proposed various methods for technology opportunity discovery and partner recommendation based on deep learning, graph convolution networks and multi-dimensional proximity attribute networks, but each has certain limitations, such as lack of consideration of time dimension and dynamic changes, insufficient exploration of patent connections at different industry chain levels, and dynamic factors of cooperation relationships and technology integration. On the one hand, this requires future research to further integrate the advantages of these methods and focus on multi-dimensional and multi-level comprehensive analysis; on the other hand, it is necessary to strengthen the consideration of time dynamic factors and the partner recommendation mechanism for actual market and technology validation to improve the effectiveness of technology innovation and market expansion. SUMMARY

[0008] To overcome the shortcomings of the above-mentioned prior art, the present application provides a recommendation method and system based on large language models and time series link prediction, aiming to accurately identify and recommend potential technology partners by combining deep learning text mining, knowledge graphs and multi-dimensional proximity attribute networks, from the perspective of time dimension and dynamic changes.

[0009] According to one aspect of the present application, a recommendation method based on large language models and time series link prediction is provided, comprising:

[0010] Obtaining relevant patent data of the target company in the industry field and inputting the trained large language model outputs the patent division results of each stage of the industry field patent industrial chain;

[0011] A sliding window strategy is adopted to construct a time series network from the patent data of each stage of the industrial chain and extract time series features;

[0012] The extracted time series features are input into the CNN-GRU model with multi-head attention mechanism optimized by Lingri optimization algorithm, and the prediction results are output, and the potential technical fusion opportunities are screened and verified;

[0013] According to the social influence and technical focus degree of the selected technical fusion opportunity, the potential partners of the next stage are recommended.

[0014] As a further technical solution, the training of the large language model includes:

[0015] A part of data is randomly extracted from the patent data set for labeling, and the labeled data set is divided into training set and test set;

[0016] The initial designed Prompt template is used on the training set, the patent title and abstract are input, and the industrial chain stage label is output;

[0017] Based on the difference between the training set output result and the labeled result, the role description, instruction description and question setting in the Prompt template are gradually adjusted;

[0018] Multiple tests are performed on the adjusted Prompt template, and the Prompt template with the best labeling effect is selected for batch processing of the entire data set.

[0019] As a further technical solution, a sliding window strategy is adopted to construct a time series network from the patent data of each stage of the industrial chain and extract time series features, including:

[0020] For patent data in each stage of the industrial chain, divide it into non-overlapping time intervals in chronological order, and construct a patent co-occurrence network snapshot in each time interval;

[0021] Set the size of the sliding window so that in each sliding window, calculate the co-occurrence relationship of node pairs in the time snapshot;

[0022] For each sliding window network snapshot, calculate the number of common neighbors, Jaccard coefficient, Adamic / Adar coefficient, local path index and average commuting time of node pairs in the past sliding window.

[0023] As a further technical solution, the extracted temporal features are input into a CNN-GRU model with a multi-head attention mechanism optimized by the transit optimization algorithm to output the prediction results, including: using a convolutional neural network (CNN) to extract local temporal features, using a gated recurrent unit (GRU) to capture the temporal dependencies of long sequences, introducing a multi-head attention mechanism to enhance attention to important parts of the temporal features, and multiple groups of weighted outputs from the multi-head attention mechanism are activated through a fully connected layer to output a probability value indicating whether the predicted node pair will generate a link in the future time step.

[0024] As a further technical solution, the transit optimization algorithm is used to optimize the CNN-GRU model with a multi-head attention mechanism. It also includes: using the transit search optimization algorithm to explore and search for the optimal parameter combination by simulating the transit phenomenon of celestial bodies. In each search iteration, the improvement of model performance is measured as the degree of light attenuation, and finally the global optimal solution is found.

[0025] As a further technical solution, potential technology fusion opportunities are screened and verified, including: market novelty calculation, technology stability assessment and technology influence measurement, and feature normalization and weighted calculation of the three indicators of market novelty, technology stability and technology influence to obtain potential technology fusion opportunities.

[0026] As a further technical solution, potential partners for the next phase are recommended based on the social impact and technical focus of the selected technology integration opportunities, including:

[0027] Determine social influence based on the number of patents held by the enterprise in the target technology field, the number of citations of the patents, the types of technologies covered by the patents, and the geographical scope of patent authorization;

[0028] Determine the degree of technological focus based on the proportion of patents held in the target technology field and the weight of these patents in its overall technology portfolio;

[0029] Potential partners are ranked based on a combined score of social impact and technical focus.

[0030] According to one aspect of the present invention, a recommendation system based on a large language model and temporal link prediction is provided, comprising:

[0031] The data collection module is used to obtain relevant patent data in the target company's industry and input it into the trained large language model to output the patent classification results at each stage of the patent industry chain in the industry;

[0032] The network construction module is used to construct a time series network from the patent data of each industry chain stage using a sliding window strategy and extract time series features;

[0033] A prediction module is configured to input the extracted time sequence features into a CNN-GRU model with multi-head attention mechanism optimized by a transit optimization algorithm, output a prediction result, and perform potential technical fusion opportunity screening and verification.

[0034] A recommendation module is configured to recommend a potential partner for the next stage according to the social influence and technical focus degree of the selected technical fusion opportunity.

[0035] According to an aspect of the present application, an electronic device is provided, comprising at least one processor, at least one memory and a communication interface; wherein the processor, memory and communication interface communicate with each other; the memory stores program instructions executed by the processor, and the processor invokes the program instructions to execute the method.

[0036] According to an aspect of the present application, a non-transitory computer readable storage medium is provided, which stores computer instructions, and the computer instructions make the computer execute the method.

[0037] Compared with the prior art, the present application has the following beneficial effects:

[0038] The present application combines multiple technologies, combines semantic analysis of large language models with time sequence networks, and uses a link prediction method considering technical correlation to improve the efficiency and accuracy of enterprise cooperation prediction. The method of the present application can help enterprises understand their influence in the industry, optimize resource allocation, recommend suitable partners for enterprises, reduce the cost and time of enterprises to find partners, promote cooperation and communication between enterprises, and promote innovation and development of the industry. BRIEF DESCRIPTION OF DRAWINGS

[0039] To make the technical solutions in the embodiments or the prior art clearer, the drawings used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0040] Figure 1 A flowchart of the recommendation method based on large language model and time sequence link prediction provided by the embodiment of the present application is shown in the figure;

[0041] Figure 2 A TCGM prediction algorithm principle diagram provided by the embodiment of the present application is shown in the figure;

[0042] Figure 3 A flowchart of the TSOA optimization algorithm provided by the embodiment of the present application is shown in the figure;

[0043] Figure 4 A schematic diagram of a recommendation system based on a large language model and time series link prediction is provided for an embodiment of the present application.

[0044] Figure 5 A schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0045] It should be noted that:

[0046] The terms "comprise" and "have" and any variations thereof in the specification and in the claims and the above mentioned attached drawings, are intended to cover not exclusively inclusive, for example, a process, method, system, product or device comprising a series of steps or units, not necessarily limited to which steps or units are clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] The block diagrams shown in the drawings are only functional entities, which do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices. The flowcharts shown in the drawings are only exemplary descriptions, which do not necessarily include all contents and operations / steps, and are not necessarily executed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0048] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. In addition, the technical features in each embodiment or single embodiment provided by the present application are combined with each other to form new technical solutions, which are not restricted by the order of steps and / or structure composition mode, but must be based on the implementation by those skilled in the art. When the combination of technical solutions contradicts each other or cannot be implemented, it should be considered that the combination of technical solutions does not exist, and is not within the scope of protection required by the present application.

[0049] The embodiments of the present application disclose a recommendation method based on a large language model and time series link prediction, which is used for matching and recommending potential technical cooperation partners of target enterprises. The method specifically comprises:

[0050] S1: Select relevant patent data in the industry field of the target company, use a large language model (LLM) to analyze the semantic content of the patent data in this field, and label the patents according to the upstream, midstream, and downstream stages of the industry chain;

[0051] S2: Adopt a sliding window strategy to construct a time series network from patents in each stage, and construct input data for link prediction by extracting five kinds of time series features: local information, path information, random walk information, node information, and community information;

[0052] S3: Use an improved CNN-GRU combined with a multi-head attention mechanism (MHAE), and optimize the link prediction algorithm through a TSOA optimization algorithm to effectively improve the prediction accuracy of technology fusion opportunities. Screen and verify potential technology fusion opportunities in different periods by factors such as market novelty, technical stability, and technical influence;

[0053] S4: According to the social influence and technical focus of the selected technology fusion opportunities, recommend potential partners for the next stage.

[0054] As a preferred embodiment, as shown in Figure 1 The embodiment of the present application is described from four aspects: data collection and industry chain division, time series network construction and feature extraction, technology fusion opportunity discovery at the enterprise level, and potential cooperation enterprise recommendation.

[0055] In the embodiment of the present application, the data collection and industry chain division includes data collection and industry chain division, which is used for semantic labeling and industry chain stage division of patent data. Specifically, this step selects relevant patent data in the industry field of the target company, and uses a large language model (LLM) to process the semantic content of the patent text. By analyzing the title and abstract of the patent, it is classified into the upstream, midstream, or downstream stage of the industry chain. Use the pre-set "role-instruction" prompt strategy to fine-tune the LLM, and use the manual annotation of part of the patents to train the LLM model to improve its classification accuracy. The final model is used for automatic batch classification of the remaining patent data, realizing the division of patents in each stage of the industry chain

[0056] In the data collection step of the embodiment of the present application, the data is collected and analyzed based on the patent data in a certain industry. The patent data is selected according to the semiconductor technology classification of the World Intellectual Property Organization, the data comes from the Orbis database, does not include design and utility model patents, and is limited to the United States Patent Office. The patent data content includes disclosure number, IPC classification number, priority date, title, abstract, and the name of the current direct right holder, etc.

[0057] In the industry chain division step, the large language model (LLM) used by the embodiment of the present application is a natural language processing technology based on deep learning, which can understand and generate natural language text. In order to accurately match the collected specific industry patents with the corresponding industry chain stage, this link uses ChatGPT combined with industry chain subdivision standards to construct prompts. Through semantic analysis of the title and abstract of each patent, it is determined that the patent belongs to the upstream, midstream or downstream link of the industry chain, thereby completing the division of the industry chain.

[0058] In designing the prompt engineering, the embodiment of the present application adopts the "role indication prompt" engineering strategy. In order to further improve the accuracy of industry chain interpretation, a fine-tuning strategy is also used to optimize the prompt involvement, thereby improving the accuracy of reasoning. Specifically, a part of the data is randomly extracted for manual annotation, and the classification and terminology in the industry chain subdivision standard are constantly improved to ensure that the optimized Prompt improves the accuracy of the manually annotated data as much as possible.

[0059] Specifically, the Prompt fine-tuning method includes: randomly extracting a part of data from the patent data set for manual annotation, dividing the annotated data set into a training set and a test set; using the initially designed Prompt template on the training set, inputting the patent title and abstract, and outputting the industry chain stage label; based on the difference between the training set output result and the manual annotation result, gradually adjusting the role description, instruction description and question setting in the Prompt template to improve the annotation accuracy of the model; multiple tests are performed on the adjusted Prompt, and the Prompt with the best annotation effect is selected for batch processing of the entire data set.

[0060] After consulting a large number of literature on the division of the semiconductor industry chain, the embodiment of the present application synthesizes the division standards of existing research as the basic rules for the division of the semiconductor industry chain. The upstream of the industry chain includes raw materials and equipment, specifically the extraction and preparation of semiconductor raw materials and the manufacturing of semiconductor equipment. The midstream includes semiconductor devices, including the design of semiconductor devices, material preparation technology and methods, manufacturing, packaging and testing. The downstream relates to specific solutions or applications related to semiconductors. In addition, the basic rules also list each specific technical category mentioned in the literature. When designing the ChatGPT prompt, the embodiment of the present application selects the "role-instruction" strategy, which includes four parts: role, instruction, content and question. The role describes the identity of the task performer; the instruction outlines the specific task, the division standard of the industry chain and the requirement for the ChatGPT output; the content refers to the specific content that needs to be processed, which is the title and abstract of the patent in this case; the question proposes the question that needs to be answered. Finally, the large model is used to complete the division of the semiconductor industry chain.

[0061] In an embodiment of the present invention, time series network construction and feature extraction are used to construct a time series network and extract features based on the data obtained from the data collection and industrial chain division links. Specifically, the patent data in each industrial chain stage are divided into non-overlapping time intervals (Δt) in chronological order. In each time interval, a patent co-occurrence network snapshot is constructed based on the first six digits of the IPC classification number. The nodes represent the technical fields, and the weights of the edges represent the number of patents between the two technical fields. All snapshots are arranged in chronological order to form a time series network. Using the sliding window method, the features of the node pairs are extracted from continuous time snapshots. The features include: the number of common neighbors (CN), the Jaccard coefficient (JC), the Adamic / Adar coefficient (AA), the local path index (LP), and the average commuting time (ACT). Time series features are constructed from five aspects: local information, path information, random walk information, node information, and community information.

[0062] This step processes the patent data of each industry chain separately and cuts them into continuous and non-overlapping sections according to the time sequence of the priority data within a certain time interval Δt. And all patents within each Δt are constructed as a group-level technology co-occurrence network snapshot based on the first six IPCs. The nodes in represent technical fields, and the weights of the edges represent the number of patents belonging to the same two technical fields. Merge in chronological order to obtain the constructed temporal network G, as shown below:

[0063]

[0064] In order to obtain input and output features from G that take into account the temporal technology convergence relationship and thus perform link prediction, we use a sliding window method. The window size ω solves the problem of network sparseness and unclear features in a single snapshot. The window slides backward in a sliding manner to extract features in sequence. Find and The common nodes N between i .from Select in There are edges but There are no node pairs n in which there is no edge i (n i ∈N i Using common neighbors (CN), Jaccard coefficient (JC), Adamic / Adar coefficient (AA), local path index (LP), average commuting time (ACT) and other indicators, n is calculated from five aspects: local information, path, random walk, node attributes and community. iIn order to reduce the classifier’s bias towards the majority class (negative samples) and improve its recognition ability towards the minority class (positive samples), undersampling is used to balance the class distribution when constructing the feature data of negative samples.

[0065] In this embodiment of the present invention, enterprise-level technology convergence opportunity discovery is used to predict technology convergence opportunities based on the temporal features extracted in the previous step. Specifically, after extracting temporal features, a CNN-GRU (TCGM) optimized with a TSOA-based multi-head attention mechanism is used to train and test technology convergence opportunities to better capture temporal patterns and learning features. Furthermore, to identify enterprise-level technology convergence opportunities, the market novelty, technological stability, and technological influence of a specific enterprise are calculated to determine technology convergence opportunities.

[0066] Specifically, a model (TCGM) combining a convolutional neural network (CNN) and a gated recurrent unit (GRU) is constructed. Multiple convolution kernels are used in the CNN part to extract local temporal features, and the temporal dependencies of long sequences are processed in the GRU part. A multi-headed attention mechanism (MHAM) is introduced to enhance the focus on important parts of temporal features, and finally feature fusion and output are completed in the fully connected layer. During the prediction process, the transit search optimization algorithm (TSOA) is used to adjust model parameters such as the convolution kernel size, the number of GRU units, and the learning rate to ensure the generalization ability of the model across different technical fields. In the verification of technology fusion opportunities for target enterprises, by evaluating indicators such as market novelty, technological stability, and technological influence, combined with patent data within a specific time window, technology fusion opportunities are dynamically screened and verified to ensure the sustainability of these opportunities in different time periods and their actual cooperation value.

[0067] Convolutional Neural Network (CNN) feature extraction. The input features are the time series node pair feature matrix extracted from the time series network. The CNN part first performs a convolution operation on the input sequence features through multiple one-dimensional convolution kernels. The size and step size of the convolution kernel are carefully designed to adapt to time series features of different scales. Through the weight sharing mechanism of the convolution layer, CNN is able to capture local patterns and short-term dependency characteristics in the time series. After the convolution operation, the batch normalization layer (BatchNormalization) is used to standardize the output to prevent the gradient from disappearing or exploding. Subsequently, the pooling layer (PoolingLayer) is used to downsample the features and extract significant features with translation invariance. The pooling operation helps to reduce the dimension of the features and reduce the risk of overfitting.

[0068] Gated recurrent unit (GRU) time dependency capture. The features extracted from the CNN part are inputted into a GRU layer, which is a variant of recurrent neural network (RNN) for time series data processing with the structure of update gate and reset gate. The update gate decides how much previous information will be retained in the current state, while the reset gate controls the degree of combination between the current input and the previous state. These gating mechanisms of GRU enable the model to effectively capture long-term dependencies and sequential patterns, especially when dealing with longer time series, which can avoid the gradient vanishing problem of traditional RNN. The output of the GRU layer represents the dynamic change patterns in the time series features.

[0069] Multi-head attention mechanism (MHAM) enhances feature capture ability. The output of the GRU layer is passed to the multi-head attention mechanism layer, which operates in parallel through multiple independent attention heads to focus on features at different positions. Each attention head first performs linear transformation on the input sequence to generate query, key, and value matrices. Then, the dot product similarity between the query and the key is calculated, and after scaling and softmax normalization, the attention weights are obtained, which are used to weight the value matrix to generate the weighted output. The parallel structure of multi-head attention mechanism allows the model to view the input features from different angles, improving the learning ability of complex patterns.

[0070] Feature fusion and fully connected layer output. The multiple sets of weighted output from the multi-head attention mechanism are concatenated to form a comprehensive feature vector, which contains multiple pattern information from the time series. The comprehensive features are inputted into the fully connected layer, which is activated by a nonlinear activation function (such as ReLU) to map high-dimensional features to the prediction space. The final output layer uses a sigmoid activation function to generate a probability value between 0 and 1, representing whether the predicted node pair will produce a link in the future time step (1 represents a link, 0 represents no link).

[0071] The TSOA (Transit Solarization Optimization Algorithm) parameter optimization. The hyperparameters of the entire TCGM model, including the convolution kernel size, the pooling window size, the GRU neuron number, the attention head number, and the learning rate, are optimized by the TSOA. The TSOA explores and searches for the best parameter combination by simulating the transit of celestial bodies, and in each search iteration, the improvement of the model performance is measured as the degree of light attenuation, and the global optimal solution is finally found. The optimization process involves multi-stage search and local fine-tuning, ensuring that the model has the best generalization ability and prediction performance in complex time series networks.

[0072] In the embodiments of the present application, the CNN-GRU prediction with multi-head attention mechanism optimized by TSOA includes the TCGM overall prediction process and the TSOA optimization process.

[0073] In the TCGM overall prediction process, the principle diagram of the CNN-GRU prediction with multi-head attention mechanism optimized by TSOA is as shown in Figure 2 First, the positive and negative sample data extracted from the time series network are used in sequence, that is, the time features extracted from [G ti , G ti+ω ] are input into the CNN layer, local spatial features are extracted through convolution operation, and the pooling layer and the rejection layer are used to prevent downscaling and overfitting. Second, the processed data is passed to the GRU layer to obtain time series information. Then, the data enters the MHAM layer, which maps the hidden state output by the GRU layer into key, value, and query vectors through linear transformation and attention mechanism, and then calculates the attention weight through the mask scaling dot product attention mechanism to strengthen the attention to important features. Finally, after further processing in the fully connected layer, the data enters the output layer to generate the prediction result, that is, the output is the corresponding node pair of G ti+ω+1 Linking (1 represents linking, and 0 represents not linking). The result of link prediction can be represented as: when the input feature is the time feature between two technical fields in the past ω steps, whether the two technical fields will produce fusion in the next step.

[0074] In the TSOA optimization process, the entire link prediction process is optimized by TSOA to adjust the model parameters to maximize the performance. The optimization process of TSOA mainly includes five stages, as shown in Figure 3The algorithm randomly selects positions in the search space, called "galaxies", to determine potential optimal regions. In the transit phase, the algorithm re-measures the light received from the stars to evaluate potential solutions. In the planet phase, the algorithm determines the initial positions of potential solutions (planets) and refines these positions through multiple observations. In the neighbor phase, if no transit is detected, the algorithm checks the neighbors of previously detected planets and replaces them if better conditions are found. In the development phase, the algorithm repeatedly optimizes the best planet features for each star to find the globally optimal solution. These phases enable TSOA to systematically explore and optimize potential solutions, effectively solving complex optimization problems.

[0075] The specific optimization parameters of TSOA include: first, the size of the convolution kernel, the main purpose of optimization is to enhance the effect of spatial feature extraction in CNN; second, the number of neurons in GRU, the purpose of optimization is to improve the ability of the model to capture time series information; third, the learning rate of the overall model, the purpose of optimization is to ensure the stability of the model during training and improve the convergence speed.

[0076] In the enterprise-level technology fusion opportunity verification, in order to find the enterprise-level technology convergence opportunity, the six-digit IPC to which the existing technology of the target enterprise belongs needs to be screened in the prediction results of the test set. In the screening process, the time when the target enterprise owns the technology will be considered to ensure that the potential convergence opportunity is likely to appear after the target enterprise owns the technology. However, not all pairs generated with the technology documents owned by the target company are suitable. Therefore, the market novelty (MN), technology stability (TR) and technology influence (TI) are proposed to further verify.

[0077] Market Novelty (Market Novelty, MN): This indicator is used to measure the degree of innovation in a certain technology field within a specific time window. Market Novelty is calculated by counting the number of patents that first appear within the time window, focusing on the frequency of new inventions or innovative technologies in the technology field. This indicator reflects the activity and emerging rate of technological innovation in the field, indicating the forward-looking nature of potential technology opportunities in the field.

[0078] Further, the market novelty calculation refers to the definition of the number of newly appeared patents in a given time window within the target technical field, used to measure the innovation activity in this field. First, a fixed length time window (e.g., the past 12 months) is selected, and the number of all patents belonging to the target technical field within the time window is counted. Then, these patents are sorted by priority date, and the number of patents that first appear in the time window is extracted. By comparing the number of new patents in different periods, the market novelty index is calculated to reflect the innovation dynamics of the current technical field.

[0079] The formula of market novelty is as follows:

[0080]

[0081] Where MN represents the number of patents belonging to the same two technical fields in the M months before the fusion occurred, P(u) represents the patent set of technical field u, M represents the number of months before the fusion occurred, and T represents the time of the fusion.

[0082] Technological Reliability (TR): This index evaluates the stability and continuity of a technical field over multiple time periods. This index reflects the continuity of technical cooperation and technology implementation by analyzing the changes in the edge weight of node pairs within a given time window. Technological reliability emphasizes the frequency and intensity of long-term cooperation and repeated technical interactions, providing a quantitative analysis of the stability of long-term technical cooperation relationships in a technical field, avoiding the interference of short-term fluctuations.

[0083] Further, the technological stability evaluation refers to the definition of the average edge weight of node pairs in the past time period in the time series network. In the specific calculation process, first, all relevant patents of the target technical field are selected to construct a technology co-occurrence network, where nodes represent specific technical fields, and the weight of the edge represents the co-occurrence frequency of patents belonging to this field. Then, a time window (e.g., the past 24 months) is set, and the average value of the edge weight of each node pair in the time window is calculated. In this way, the technological stability index can reflect the continuity and stability of the technical field in this time period.

[0084] The formula of technological stability is as follows:

[0085]

[0086] Where TR represents the average edge weight of paired nodes in the past N months, N represents the number of months before the fusion occurred, F u,t represents the edge weight of node u at time t.

[0087] Technological Influence (TI): This indicator is used to assess the importance and influence of a technology field in the overall technology network. Technological influence is achieved by calculating the eigenvector centrality of nodes, reflecting the connectivity and radiation ability of nodes in the technology network. Nodes with high influence are usually directly connected to multiple key technology fields, and these fields themselves also have high importance. Therefore, the technological influence indicator captures the ability of a specific technology field to serve as a core hub in the overall technology ecosystem, affecting the diffusion and development of new technologies.

[0088] Further, the technological influence measurement refers to the use of eigenvector centrality to measure the importance of nodes in the time series network. The calculation of technological influence involves the following steps: First, a global technology network is constructed, with nodes representing various technology fields and edge weights representing their co-occurrence relationships. Then, the eigenvector centrality of each node is calculated, which is obtained by iteratively solving the influence distribution of the node, reflecting the importance of a node in the overall technology network. Higher centrality means that the technology is connected to multiple high-influence technology fields in the network, thus having greater technological influence.

[0089] The formula for technological influence is as follows:

[0090]

[0091] where TI represents the eigenvector centrality of paired nodes in the network over the past N months, V u [t] represents the eigenvector centrality of node u at time t.

[0092] The market novelty, technology stability, and technology influence indicators are normalized to eliminate differences between different feature dimensions. The normalization method uses Min-Max normalization to convert the value of each indicator to the range [0, 1]. The normalized feature values are linearly weighted according to the preset weights (e.g., w1, w2, w3 correspond to MN, TR, TI), generating a comprehensive score. The selection of weights can be based on specific application scenarios, and the weight combination can be optimized through multiple experiments and validation to ensure that the final score reflects the credibility of actual technology fusion opportunities.

[0093] Specifically, the normalized and are calculated, and the technology recommendation index (TRI) of each potential technology opportunity is calculated, from which the top 10 values corresponding to the target company's technology opportunities are selected. The specific calculation method is as follows:

[0094]

[0095] In the embodiment of the present application, potential cooperation enterprise recommendation is used for recommending potential cooperation partners of enterprises based on the technology fusion opportunity obtained in the previous link. Specifically, based on the result of technology fusion prediction, potential technology cooperation opportunities are identified. By calculating the matching degree of technical content, possible cooperation partners are screened and sorted according to the technical fields they have within the prediction time window. By quantifying the social influence and technical focus of potential cooperation partners, the cooperation partner recommendation index (PRI) is calculated by combining the preset weight coefficient, and finally the list of potential cooperation partners of the target enterprise is generated.

[0096] This link matches the corresponding potential cooperation partners according to the technology fusion opportunities of the target company. Each technology convergence opportunity has a time of occurrence of convergence with the technology owned by the target company. First, a list of cooperation partners is found, which includes all cooperation partners that own the corresponding technology before the technology fusion occurs. Second, in order to accurately identify the best candidate cooperation partners from the above list of enterprises, two measurement standards, social influence (SI) and technical focus (TF), are proposed. Social influence considers the technical maturity of the enterprise in a specific field, while technical focus evaluates the degree of focus of the enterprise on the relevant technology.

[0097] Social influence is used to measure the technical accumulation and market performance of a potential cooperation partner in a specific technical field. It is determined by calculating the number of patents of the enterprise in the target technical field, the number of citations of the patents, the types of technology covered by the patents, and the geographical scope of the patent authorization. The level of social influence reflects the technical depth and market influence of the enterprise in the field, which helps to evaluate its value contribution in technology cooperation.

[0098] The quantification of social influence is represented by the number of matching technical fields v owned by potential cooperation partner x before the occurrence of technology fusion at time T, which reflects the technical accumulation and innovation ability of the cooperation partner in a specific field. Cooperation partners with high SI value can provide more resources in the process of technology fusion and increase the possibility of cooperation success.

[0099]

[0100] where t i represents the time point before the occurrence of technology convergence, P(x,v,t i ) represents the patent set of technical field v owned by cooperation partner x at time t i .

[0101] Technological Focus (TF) represents the degree of focus of an enterprise in a specific technical field. By analyzing the proportion of patents in a certain technical field among all patents of an enterprise, the concentration and specialization level of its technical strategy are evaluated. Specifically, technological focus considers the proportion of the number of patents in this technical field and the weight of these patents in the overall technology portfolio. High technological focus indicates that the enterprise has strong technical advantages and continuous innovation capabilities in this field, which is suitable for deep cooperation.

[0102] The quantitative indicator of technological focus is the ratio of the number of patents owned by T to the number of all patents. The higher the TF, the more likely it is that the enterprise's technology development strategy is focused on a highly concentrated and specialized field, and it will provide more professional and in-depth technical support in cooperation.

[0103]

[0104] where P(x, t i ) represents the set of all patents owned by partner x at time t i , and |·| represents the number of elements in the set.

[0105] Finally, in order to consider the above two indicators comprehensively, the results are normalized, and then weighted with W SI and W TF to obtain the Partner Recommendation Index (PRI). The calculation method is as follows:

[0106]

[0107] According to the ranking of the Partner Recommendation Index (PRI), the target company is recommended to cooperate with partners in the field that is most likely to carry out technical integration in the future within a specific time range.

[0108] It should be noted that the Partner Recommendation Index (PRI) is a comprehensive score based on market influence and technological focus, used to rank potential partners. This index combines market influence and technological focus through weighted merging to form a comprehensive indicator for evaluating the cooperation potential of enterprises. The weighting factors are adjusted according to actual application scenarios and strategic goals to ensure that the recommended partners meet the cooperation needs of the target enterprise in terms of technical capabilities and market coverage. The higher or lower the PRI directly affects the final list of recommended partners, aiming to optimize cooperation results and innovation achievements.

[0109] The implementation basis of each embodiment of the present application is achieved by programmed processing of a device with processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present application are packaged into various modules. Based on this actual situation, on the basis of each of the above embodiments, the embodiments of the present application provide a recommendation system based on large language model and time series link prediction, which is used to execute the recommendation method based on large language model and time series link prediction in the above method embodiment.

[0110] Referring to Figure 4 , the system comprises: a data collection module configured to obtain relevant patent data of an industry field in which a target company is located and input a large language model after training, and output patent division results of each stage of a patent industrial chain in the industry field; a network construction module configured to construct a time series network from patent data of each stage of the industrial chain using a sliding window strategy, and extract time series features; a prediction module configured to input the extracted time series features into a CNN-GRU model with a multi-head attention mechanism optimized by a Lingri optimization algorithm, output a prediction result, and perform screening and verification of potential technology fusion opportunities; and a recommendation module configured to recommend potential partners in the next stage according to social influence and technology focus of the selected technology fusion opportunities.

[0111] The recommendation system based on large language model and time series link prediction provided by the embodiments of the present application adopts several modules in Figure 4 , integrates multiple technologies, combines semantic analysis of a large language model with a time series network, and uses a link prediction method considering technology association to improve the efficiency and accuracy of enterprise cooperation prediction.

[0112] It should be noted that the system embodiments provided by the present application are used to implement the methods in the above method embodiments, and are also used to implement the methods in other method embodiments provided by the present application. The difference is only that the corresponding function modules are set, and the principle is basically the same as that of the above system embodiments provided by the present application. As long as the person skilled in the art, on the basis of the above system embodiments, refers to the specific technical solutions in other method embodiments, obtains the corresponding technical means by combining technical features, and the technical solutions composed of these technical means, as long as the technical solutions have practicality, the modules in the above system embodiments are improved to obtain the corresponding system class embodiments, which are used to implement the methods in other method class embodiments. For example:

[0113] Based on the content of the above system embodiments, as a preferred embodiment, the recommendation system based on large language model and time series link prediction provided in the embodiments of the present application, the data collection module is also used to execute the following instructions:

[0114] Randomly extract a part of data from the patent data set for labeling, and divide the labeled data set into a training set and a test set;

[0115] Use the initially designed Prompt template on the training set, input the patent title and abstract, and output the industry chain stage label;

[0116] Based on the difference between the output results of the training set and the labeled results, gradually adjust the role description, instruction description and question setting in the Prompt template;

[0117] Test the adjusted Prompt template multiple times, and select the Prompt template with the best labeling effect for batch processing of the entire data set.

[0118] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and time series link prediction provided in the embodiment of the application, the network construction module is further used to execute the following instructions:

[0119] For patent data in each industry chain stage, divide it into non-overlapping time intervals in chronological order, and construct a patent co-occurrence network snapshot in each time interval;

[0120] Set the size of the sliding window so that in each sliding window, the co-occurrence relationship of node pairs in the time snapshot is calculated;

[0121] For each network snapshot of the sliding window, calculate the number of common neighbors, Jaccard coefficient, Adamic / Adar coefficient, local path index and average commuting time of node pairs in the past sliding window.

[0122] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and time series link prediction provided in the embodiment of the application, the prediction module is further used to execute the following instructions:

[0123] Local time series features are extracted using convolutional neural network CNN, long sequence time dependencies are captured using gated recurrent unit GRU, multi-head attention mechanism is introduced to enhance the attention degree to important parts of time series features, and multiple sets of weighted outputs from the multi-head attention mechanism are activated via a fully connected layer to output probability values representing the prediction of whether node pairs will form links in future time steps.

[0124] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and time series link prediction provided in the embodiment of the application, the prediction module is further used to execute the following instructions:

[0125] The transit search optimization algorithm is used to explore and search for the best parameter combination by simulating the transit phenomenon of celestial bodies. In each search iteration, the improvement of model performance is measured as the degree of light attenuation, and finally the global optimal solution is found.

[0126] Based on the content of the above system embodiment, as a preferred embodiment, in the recommendation system based on large language model and temporal link prediction provided in the embodiment of the present invention, the prediction module is further used to execute the following instructions:

[0127] Market novelty calculation, technology stability assessment and technology influence measurement are carried out, and the three indicators of market novelty, technology stability and technology influence are normalized and weighted to obtain potential technology integration opportunities.

[0128] Based on the content of the above system embodiment, as a preferred embodiment, the recommendation system based on large language model and temporal link prediction provided in the embodiment of the present invention, the recommendation module is further used to execute the following instructions:

[0129] Determine social influence based on the number of patents held by the enterprise in the target technology field, the number of citations of the patents, the types of technologies covered by the patents, and the geographical scope of patent authorization;

[0130] Determine the degree of technological focus based on the proportion of patents held in the target technology field and the weight of these patents in its overall technology portfolio;

[0131] Potential partners are ranked based on a combined score of social impact and technical focus.

[0132] The method of the embodiment of the present invention is implemented by electronic devices, so it is necessary to introduce the relevant electronic devices. Based on this purpose, the embodiment of the present invention provides an electronic device, such as Figure 5 As shown, the electronic device includes: at least one processor, a communications interface, at least one memory, and a communications bus, wherein the at least one processor, the communications interface, and the at least one memory communicate with each other via the communications bus. The at least one processor invokes logic instructions in the at least one memory to execute all or part of the steps of the methods provided in the aforementioned method embodiments.

[0133] Moreover, the logic instructions in the at least one memory described above are implemented by means of software functional units and sold or used as an independent product, and are stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially, or parts contributing to the prior art, or part of the technical solutions are embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (personal computer, server, or network device) to execute all or part of the steps of the methods described in the various method embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various storage program codes.

[0134] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] These computer-usable program codes can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction means, which implement the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0136] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0137] It should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A recommendation method based on a large language model and temporal link prediction, characterized in that: include: Obtain relevant patent data in the target company's industry and input it into the trained large language model to output the patent classification results for each stage of the patent industry chain in the industry; A sliding window strategy is used to construct a time series network from the patent data of each industry chain stage and extract time series features; The extracted time series features are input into the CNN-GRU model with a multi-head attention mechanism optimized by the transit optimization algorithm, and the prediction results are output. Potential technology fusion opportunities are screened and verified. Among them, the convolutional neural network CNN is used to extract local time series features. The time features extracted from the time series network are input into the CNN layer, and local spatial features are extracted through convolution operations. The pooling layer and the elimination layer are used to prevent downscaling and overfitting. The processed data is passed to the GRU layer, and the GRU is used to capture the time dependency of long sequences and obtain time series information. The multi-head attention mechanism is introduced to enhance the focus on important parts of the time series features. The acquired time series information enters the MHAM layer. Through linear transformation and attention mechanisms, the hidden state output by the GRU layer is mapped into key, value, and query vectors. The attention weights are calculated using the masked scaled dot product attention mechanism. After being activated by the fully connected layer, the multiple weighted outputs from the multi-head attention mechanism enter the output layer, which outputs the probability value of whether the predicted node pair will be linked in the future time step, with 1 indicating a link and 0 indicating no link. The link prediction result indicates whether the two technical fields will merge in the next step when the input feature is the time feature between two technical fields within a set period of time in the past. Screening and verifying potential technology convergence opportunities, including market novelty calculation, technology stability assessment, and technology impact measurement. The three indicators of market novelty, technology stability, and technology impact are normalized and weighted to identify potential technology convergence opportunities. Based on the social influence and technological focus of the selected technology integration opportunities, potential partners for the next stage are recommended, including: determining social influence based on the number of patents the company holds in the target technology field, the number of citations of their patents, the types of technologies covered by the patents, and the geographical scope of patent authorization; determining technological focus based on the proportion of the company's patents in the target technology field and the weight of these patents in its overall technology portfolio; and ranking potential partners based on the comprehensive score of social influence and technological focus.

2. The recommendation method based on a large language model and temporal link prediction according to claim 1, characterized in that: The training of the large language model includes: Randomly extract a portion of data from the patent dataset for annotation, and divide the annotated dataset into a training set and a test set; Use the initially designed Prompt template on the training set, input the patent title and abstract, and output the industry chain stage label; Based on the differences between the training set output and the annotation results, gradually adjust the role description, instruction instructions, and question settings in the Prompt template; Multiple tests are performed on the adjusted Prompt template, and the Prompt template with the best annotation effect is selected for batch processing of the entire dataset.

3. The recommendation method based on a large language model and temporal link prediction according to claim 1, characterized in that: A sliding window strategy is used to construct a time series network from the patent data of each industry chain stage and extract time series features, including: The patent data within each industry chain stage are divided into non-overlapping time intervals in chronological order, and a snapshot of the patent co-occurrence network within each time interval is constructed; Set the sliding window size so that within each sliding window, the co-occurrence relationship of node pairs in the time snapshot is calculated; For each network snapshot in a sliding window, the number of common neighbors, Jaccard coefficient, Adamic / Adar coefficient, local path index, and average commuting time of the node pair in the past sliding window are calculated.

4. The recommendation method based on a large language model and temporal link prediction according to claim 1, characterized in that: Using the transit optimization algorithm to optimize the CNN-GRU model with a multi-head attention mechanism also includes: using the transit search optimization algorithm to explore and search for the best parameter combination by simulating the transit phenomenon of celestial bodies. In each search iteration, the improvement in model performance is measured as the degree of light attenuation, and ultimately the global optimal solution is found.

5. A recommendation system based on a large language model and temporal link prediction, used to implement the method according to any one of claims 1 to 4, characterized in that: include: The data collection module is used to obtain relevant patent data in the target company's industry and input it into the trained large language model to output the patent classification results at each stage of the patent industry chain in the industry; The network construction module is used to construct a time series network from the patent data of each industry chain stage using a sliding window strategy and extract time series features; The prediction module is used to input the extracted time series features into the CNN-GRU model with a multi-head attention mechanism optimized by the transit optimization algorithm, output the prediction results, and screen and verify potential technology fusion opportunities; The recommendation module is used to recommend potential partners for the next stage based on the social influence and technical focus of the selected technology integration opportunities.

6. An electronic device, characterized in that: include: At least one processor, at least one memory, and a communication interface; wherein the processor, memory, and communication interface communicate with each other; The memory stores program instructions executed by the processor, and the processor calls the program instructions to execute the method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the method of any one of claims 1 to 4.

Images