Research trend prediction method, device, and computer-readable storage medium based on LSTM network

Abstract

This application relates to a research trend prediction method, apparatus, and computer-readable storage medium based on an LSTM network. The method includes the following steps: S1, using a first LSTM network as an encoder, taking the true values ​​of research trend data in a certain field over the past t years as the input x to each of the t LSTM units of the encoder. i And the output vector h of each LSTM unit of the encoder i Input the next LSTM unit to obtain the output vector h of the last LSTM unit. t S2. Using the second LSTM network as the decoder, the output vector h of the last LSTM unit of the encoder is... t Input the first LSTM unit of the decoder, and output the vector h of each LSTM unit of the decoder. t+j Input its next LSTM unit, and output the vector h of each LSTM unit of the decoder. t+j This is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value is used as the input x of the next LSTM unit. t+j This application provides more accurate predictions and greater flexibility.

Description

Technical Field

[0001] This application relates to trend analysis and prediction techniques, and more specifically, to a research trend prediction method, apparatus, and computer-readable storage medium based on LSTM (Long Short-Term Memory) networks. Background Technology

[0002] With the development of various technological fields, the number of published papers is also increasing. From the evaluation of research, the formulation of funding and reward policies, to commercial investment, the prediction of research trends is an essential task.

[0003] The analysis and prediction of research trends mainly employs bibliometric methods, content-based methods, and methods combining bibliometric information and content. Bibliometric methods primarily utilize bibliometric information related to literature to predict technological development trends. This information may include the number of papers or patents published, citation relationships between different works, or the number of citations. Content-based methods mainly utilize content information, such as keyword information, and can analyze changes in the similarity ranking of different keywords over different years to identify popular trends. There are also predictive methods that combine bibliometric and content information, such as predicting which articles will become journal cover articles.

[0004] Trend analysis can be divided into two categories: analyzing current hot fields and predicting which fields will become more popular in the future. One way to measure the development trend of a field is by using the number of papers published in that field. They use Long Short-Term Memory (LSTM) networks to predict the number of papers published in the next five years based on the number of papers published annually over the past ten years. This task is a regression task of sequence prediction. Recurrent Neural Networks (RNNs) are a classic type of neural network used for time-related tasks. RNNs can suffer from gradient vanishing or gradient exploding problems, so LSM networks were developed. LSM networks use a special gating mechanism to avoid these problems. Currently, many time-related prediction tasks use LSM networks. For example, some researchers use LSM networks to predict stock values, others use them to analyze and predict water quality, and still others stack two LSM networks together, using the output of each step of the first LSM network as the input of the second LSM network, and then using the output of each step of the second LSM network as the final output to demonstrate the advantages of stacking two LSM networks, and so on.

[0005] To predict the future number of papers published in a specific research field, a Long Short-Term Memory (LSTM) network is used as the prediction model, based on the number of papers published in that field over the past ten years. Predicting the number of papers to be published in a certain field each year over the next five years is currently the most advanced method, where f represents the field, y is the starting year, and p is the number of papers. Figure 1 The diagram shown is a simplified schematic of a unit in a Long Short-Term Memory (LSTM) network. (See also...) Figure 1 As shown, each LSTM unit consists of three gates: the forget gate f. t Input gate i t Output gate o t .

[0006] The Long Short-Term Memory (LSTM) network first needs to decide which information to discard from the previous state. This is achieved through a sigmoid function of the forget gate, where σ represents the sigmoid function.

[0007] f t =σ(W f ·[h t-1 ,x t ]+b f (1)

[0008] Secondly, it is necessary to decide which information from the current step needs to be stored in the state unit. This refers to new information that needs to be stored.

[0009] i t =σ(W i ·[h t-1 ,x t ]+b i (2)

[0010]

[0011]

[0012] The output gate controls which content the Long Short-Term Memory network outputs at the current step, also implemented using a sigmoid function:

[0013] o t =σ(W o ·[h t-1 ,x t ]+b o (5)

[0014] h t =o t *tanh(C t (6)

[0015] Formulas (1) to (6) above represent the calculation process of one LSTM unit in a Long Short-Term Memory (LSTM) network, where f t For the Gate of Oblivion, it For the input gate, o t For output gate, h t-1 and C t-1 The output value and memory state at time t-1 are respectively, x t h t and C t These represent the input value, output value, and memory state at time t, respectively. Let be the candidate value at time t, W be the weight, b be the compensation parameter, tanh be the activation function, and σ be the sigmoid function.

[0016] The method of using Long Short-Term Memory networks to predict the number of papers in the next five years is as follows: Figure 2 As shown, where x1, x2, ... x 10 This is the input to each step of the Long Short-Term Memory (LSTM) network. The number of papers published each year undergoes a linear transformation, mapping it to a 100-dimensional vector, which serves as the input x to the LSTM unit at each step. i Here, the linear transformation can be a linear mapping of a dense layer (fully connected layer), that is:

[0017]

[0018] Figure 2 In the example, h0 is a randomly initialized hidden state vector, h 10 This is the vector output by the last LSTM unit, and h has a dimension of 100. This model needs to predict the number of papers published each year for the next five years. The 100-dimensional vector h 10 After a linear transformation, it is transformed into a 5-dimensional vector. These five numbers represent the results to be predicted for the next five years:

[0019] model output = dense output (h 10 (8)

[0020]

[0021] There are five values ​​in model_output, from model_output0 to model_output4, representing the prediction results for the next five years.

[0022] The prediction method based on long short-term memory networks has two drawbacks. The first drawback is that the prediction error is still relatively large. The second drawback is that it can only predict the number of papers for the next five years at a time, and cannot flexibly predict data for several years in the future. Summary of the Invention

[0023] The technical problem to be solved by this application is to provide an LSTM-based research trend prediction method, apparatus and computer-readable storage medium that can improve prediction error and prediction flexibility, in view of the above-mentioned defects of the prior art.

[0024] To address its technical problem, this application proposes a research trend prediction method based on LSTM networks in its first aspect, the method comprising the following steps:

[0025] S1. Using the first LSTM network as the encoder, the true value of each year in the research trend data of a certain field over the past t years is used as the input x of each corresponding LSTM unit in the t LSTM units of the encoder. i And the output vector h of each of the t LSTM units in the encoder. i Input the next LSTM unit to obtain the output vector h of the last LSTM unit. t where i∈[1,t];

[0026] S2. Using the second LSTM network as the decoder, the output vector h of the last LSTM unit of the encoder is... t Input the first LSTM unit of the decoder, and output the vector h from each of the n LSTM units of the decoder. t+j Input its next LSTM unit, and the output vector h of each of the n LSTM units of the decoder will be used. t+j This is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value for each year is used as the input x of the next LSTM unit in the decoder's n LSTM units. t+j , where j∈[1,n].

[0027] In one embodiment of the research trend prediction method based on LSTM networks according to the first aspect of this application, the first LSTM network and the second LSTM network are two LSTM networks using different parameters, or the same LSTM network using the same parameters.

[0028] In one embodiment of the research trend prediction method based on LSTM networks according to the first aspect of this application, the method uses the true value of each year as the input x of the next LSTM unit of the LSTM unit corresponding to that year during the decoder training phase. t+j .

[0029] In one embodiment of the research trend prediction method based on LSTM networks according to the first aspect of this application, if the current input year in the training set overlaps with the test set during the decoder training phase, the predicted value of the current input year is used as the input x of the next LSTM unit of the LSTM unit corresponding to the current input year. t+j .

[0030] In one embodiment of the research trend prediction method based on LSTM networks according to the first aspect of this application, the method sets a mask m during the training phase of the decoder, with a default value of 0. When the current input year y+t+j-1 overlaps with the test set, m... j =1, then the inputs of the n LSTM units of the decoder are as follows:

[0031]

[0032] Where f represents the domain, y represents the start year, and p is the actual value. These are predicted values.

[0033] In one embodiment of the research trend prediction method based on LSTM networks according to the first aspect of this application, the research trend data is the number of published papers or the number of patents.

[0034] To address its technical problem, this application proposes a research trend prediction device based on an LSTM network in a second aspect, the device comprising:

[0035] The encoder using the first LSTM network has t LSTM units, which are used to take the true value of each year from the research trend data of a certain field over the past t years as the input x of each of the corresponding LSTM units in the encoder. i And the output vector h of each of the t LSTM units in the encoder. i Input the next LSTM unit to obtain the output vector h of the last LSTM unit. t where i∈[1,t];

[0036] The decoder uses a second LSTM network with n LSTM units to process the output vector h of the last LSTM unit of the encoder. t Input the first LSTM unit of the decoder, and output the vector h from each of the n LSTM units of the decoder. t+j Input its next LSTM unit, and the output vector h of each of the n LSTM units of the decoder will be used. t+jThis is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value for each year is used as the input x of the next LSTM unit in the decoder's n LSTM units. t+j , where j∈[1,n].

[0037] In one embodiment of the research trend prediction device based on LSTM network according to the second aspect of this application, the first LSTM network and the second LSTM network are two LSTM networks using different parameters, or the same LSTM network using the same parameters.

[0038] To address its technical problem, this application proposes a research trend prediction device based on LSTM networks in a third aspect, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the research trend prediction method based on LSTM networks as described above.

[0039] In order to solve its technical problem, this application proposes a computer-readable storage medium in a fourth aspect, which stores a computer program that, when executed by a processor, implements the steps of the research trend prediction method based on LSTM networks as described above.

[0040] The research trend prediction method, apparatus, and computer-readable storage medium based on LSTM networks of this application have the following beneficial effects: The research trend prediction method and apparatus based on LSTM networks according to the embodiments of this application use an LSTM network as a decoder concatenated after the encoder to perform chained prediction. The first LSTM unit of the decoder is connected to the last LSTM unit of the encoder. There is no need to limit the sequence length of the input data, nor is there a need to limit the data of the predicted future fixed years. The data of the next n years can be predicted step by step according to the needs, making the prediction more accurate and greatly improving the flexibility. Attached Figure Description

[0041] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0042] Figure 1 This is a simplified schematic diagram of the principle of a unit in the Long Short-Term Memory (LSTM) network.

[0043] Figure 2 This is a schematic diagram of a long short-term memory network prediction model that uses the number of papers published in a certain field over the past ten years to predict the number of papers published in the next five years.

[0044] Figure 3 This is a flowchart of a research trend prediction method based on LSTM networks according to an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of the logical structure of a research trend prediction device based on an LSTM network according to an embodiment of this application;

[0046] Figure 5 This is a schematic diagram of the logical structure of a research trend prediction device based on an LSTM network according to another embodiment of this application;

[0047] Figure 6 This is a schematic diagram showing the root mean square error of the chain prediction model and the mask-trained chain prediction model of this application compared with the benchmark model on four different datasets.

[0048] Figure 7 The R-values ​​of the chain prediction model and the mask-trained chain prediction model of this application, compared with the benchmark model, are tested on four different datasets. 2 Result diagram;

[0049] Figure 8 This is a schematic diagram showing the root mean square error of the chain prediction model and the mask-trained chain prediction model of this application when predicting the next five years with the benchmark model.

[0050] Figure 9 The chain prediction model and the mask-trained chain prediction model of this application predict the annual R-values ​​for the next five years compared to the baseline model. 2 Result comparison diagram. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Furthermore, the embodiments and features described herein can be combined with each other unless otherwise specified.

[0052] This application aims to predict future development trends in a specific research field by using research trend data from the past few years to predict future research trends. This research trend data can be the number of published papers or patents. After obtaining the predicted data, targeted analyses can be conducted for different regions and fields, providing relevant institutions with a reference when formulating funding or incentive policies.

[0053] Therefore, this application proposes a research trend prediction method based on LSTM networks. Figure 3 A flowchart of a research trend prediction method 100 based on an LSTM network according to an embodiment of this application is shown. Figure 3 As shown, the method 100 includes the following steps:

[0054] In step S110, the first LSTM network is used as the encoder, and the true value of each year in the research trend data of a certain field over the past t years is used as the input x of each corresponding LSTM unit in the t LSTM units of the encoder. i And the output vector h of each of the t LSTM units in the encoder. i Input the next LSTM unit to obtain the output vector h of the last LSTM unit. t where i∈[1,t];

[0055] In step S120, the second LSTM network is used as the decoder, and the output vector h of the last LSTM unit of the encoder is... t Input the first LSTM unit of the decoder, and output the vector h from each of the n LSTM units of the decoder. t+j Input its next LSTM unit, and the output vector h of each of the n LSTM units of the decoder will be used. t+j This is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value for each year is used as the input x of the next LSTM unit in the decoder's n LSTM units. t+j , where j∈[1,n].

[0056] For a specific field f, the number of papers published or patents in that field in year k is denoted as f(x). Its predicted value is denoted as This requires using data from the past t years to predict data for the next n years, i.e., the input is... The output is Where y is the starting year.

[0057] The research trend prediction method 100 based on LSTM network in the above embodiments of this application uses an encoder and a decoder to complete this task. Both the encoder and the decoder are implemented using LSTM network prediction models. These two LSTM network prediction models can be two LSTM network prediction models with different parameters, or they can be the same LSTM network with the same parameters. That is, the same LSTM network acts as both the encoder and the decoder.

[0058] In step S110 above, the input to each LSTM unit of the encoder is the true value of the year y+i-1 represented in this step. This true value is transformed into a 100-dimensional vector through a linear mapping, i.e.:

[0059]

[0060] In step S120 above, the input to each LSTM unit of the decoder is different during the training and testing phases. During the training phase, the decoder uses the real data of each year as the input x of the next LSTM unit of the LSTM unit corresponding to that real value. t+j In other words, the input for each step is the actual value of the year represented by the previous step. However, during the testing phase, the decoder uses the predicted value from the previous step as the input for this step, meaning the input for each step is the predicted value from the previous step.

[0061]

[0062]

[0063] In step S120 above, the output vector h of each LSTM unit of the decoder t+j It will be transformed into a one-dimensional vector through a dense layer, serving as the predicted value for each year in the next n years:

[0064]

[0065] According to the method 100 described above in this application, each time step of the decoder will have an output of an LSTM network. Instead of predicting the data for the next n years all at once, it first predicts the year y+t, and then uses the actual value of year y+t (which is the predicted value during the testing phase) as the input of the decoder for the next time step to predict the data for year y+t+1, until the data for year y+t+n-1 is predicted.

[0066] The Long Short-Term Memory (LSTM) network has 100 hidden units, and the output of the LSM network undergoes a dropout (random deactivation) with a probability of 0.1. Normally, the activation function inside the LSM network is tanh. According to the model in the above embodiment of this application, ReLU is used because, based on experiments, tanh is difficult to converge to a good level. Therefore, tanh in the aforementioned original LSM network formulas (3) and (6) is replaced with ReLU as follows:

[0067]

[0068] h t =o t *Relu(C t (15)

[0069] In the LSTM-based trend prediction method 100 according to the above embodiments of this application, the chained trend prediction model of the encoder and decoder may have the problem of overlapping data for certain years in the training and testing phases. For example, if a training dataset is from 1998 to 2012 (assuming 1998 to 2007 are used as input data and 2008 to 2012 as prediction data), and a test dataset is from 1999 to 2013, then in the chained trend prediction model of the encoder and decoder, the decoder will use the data of 2011 as input to predict the data of 2012. This situation will also occur in the testing phase. To some extent, this training method allows the model to know what the test set looks like during the training phase. In a certain step of the decoder's calculation process, the same input year and prediction year appear in both the training and testing phases. For example, the actual data of 2011 may be used as the input of the decoder to predict the data of 2012.

[0070]

[0071] h 14 =LSTMcell(x 14 ,h 13 ,c 13 (17)

[0072]

[0073] As shown above, the 2011 data was used to calculate the input to the LSTM unit in step 14. This input was then used to generate the output of step 14, which was transformed into a one-dimensional result representing the prediction result for 2012. Here, the input for 2011 used real data.

[0074] To prevent this from happening, a further embodiment of this application sets an additional mask during the decoder training phase. During decoder training, if the current input year in the training set overlaps with the test set, the true value will be masked, and the predicted value of the current input year will be used as the input x of the next LSTM unit corresponding to the current input year. t+j .

[0075] Let m represent the mask, with a default value of 0. When the current input year y+t+j-1 is the same as a year in the test set, m... j =1, then the inputs of the n LSTM units of the decoder are as follows:

[0076]

[0077] This application proposes a chain-based trend prediction model (TMED) based on encoders and decoders, building upon the LSTM network-based trend prediction method described above. The first Long Short-Term Memory (LSTM) network acts solely as an encoder, not directly predicting data for a fixed year. Instead, a second LSTM network is added after the encoder as a decoder. The decoder uses the output of the first LSTM network and the current year's data as input to predict the next year's data. This step-by-step prediction of future year data significantly improves flexibility. Existing prediction methods output a 5-dimensional vector representing the prediction results for the next five years at once, while this application uses the number of LSTM units in the decoder to predict the number of future years, allowing for easy adjustment and improving model flexibility and scalability. Furthermore, each step of this chain-based prediction model directly uses the previous year's data as input, combined with the output of past LSTM networks, to predict the next year's result. This results in smaller prediction errors and better performance, whereas existing methods that output five years' worth of prediction results at once have relatively large errors when predicting the second to fifth years.

[0078] This application further proposes a mask-trained chain prediction model (MTMED) based on the LSTM network-based research trend prediction method described above. This is because the training and test sets overlap in certain windows, meaning data from a particular year might be used as input to the decoder during both training and testing. This creates an "overlapping" problem in the datasets, potentially rendering the test results ineffective. To prevent this, the mask-trained chain prediction model (MTMED) completely separates training and testing at the decoder level. When a year appears in the test set, the predicted value is used during training; when a year does not appear in the test set, the true value is used as input. This operation is achieved through a masking mechanism, ensuring the independence of training and testing while maximizing the use of real training data, making the trained model more comparable and the test results more effective.

[0079] Based on the research trend prediction method 100 based on LSTM network introduced above, this application also proposes a research trend prediction device based on LSTM network. Figure 4 A logic block diagram of a research trend prediction device 200 based on an LSTM network according to an embodiment of this application is shown. Figure 4As shown, the LSTM network-based research trend prediction device 200 includes an encoder 210 using a first LSTM network and a decoder 220 using a second LSTM network. The encoder 210 has t LSTM units 211, and the decoder 220 has n LSTM units 221. The encoder 210 is used to take the true value of each year in the research trend data of a certain field over the past t years as the input x of each corresponding LSTM unit in the t LSTM units 211 of the encoder 210. i And the output vector h of each LSTM unit in the t LSTM units 211 of the encoder 210 i Input the next LSTM unit to obtain the output vector h of the last LSTM unit. t Where i∈[1,t]. Decoder 220 is used to convert the output vector h of the last LSTM unit of encoder 210 into a single vector. t Input the first LSTM unit of the decoder 220, and output the vector h of each of the n LSTM units 221 of the decoder 220. t+j Input its next LSTM unit, and the output vector h of each of the n LSTM units 221 of the decoder 220 will be used. t+j This is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value for each year is used as the input x of the next LSTM unit in the n LSTM units 221 of the decoder. t+j Where jv[1,n]. According to different embodiments of this application, the encoder 210 using a first LSTM network and the decoder 220 using a second LSTM network can be two LSTM networks using different parameters, or they can be the same LSTM network using the same parameters.

[0080] The LSTM-based research trend prediction device 200 according to the above embodiments of this application is used to implement the aforementioned LSTM-based research trend prediction method. For further specific implementations of the encoder 210 and decoder 220 of the LSTM-based research trend prediction device 200, please refer to the detailed description of each step of the LSTM-based research trend prediction method 100 described above.

[0081] Based on the research trend prediction method 100 based on LSTM networks in the foregoing embodiments of this application, this application also proposes a research trend prediction device 300 based on LSTM networks. See also Figure 5As shown, the LSTM network-based research trend prediction device 300 includes a processor 310 and a memory 320, which are communicatively connected. The memory 320 stores a computer program, which, when executed by the processor 310, causes the processor 310 to implement the steps of the LSTM network-based research trend prediction method 100 of the foregoing embodiments of this application.

[0082] Processor 310 can be a Central Processing Unit (CPU). Processor 310 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. The general-purpose processor can be a microprocessor or any conventional processor.

[0083] The memory 320, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the LSTM network-based research trend prediction method 100 in this embodiment of the invention. The processor 310 executes various server functions and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 320, thereby realizing the LSTM network-based research trend prediction method 100 in the aforementioned embodiment.

[0084] The memory 320 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the LSTM network-based research trend prediction method 100, etc. Furthermore, the memory 320 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 320 may optionally include memory remotely located relative to the processor 310, and these remote memories may be connected to the processor 310 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0085] This application also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the LSTM-based research trend prediction method 100 of the foregoing embodiments of this application. The computer-readable storage medium may be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc. The computer-readable storage medium may also include combinations of the above-mentioned types of memory.

[0086] The encoder-decoder-based chained trend prediction model (TMED) and the mask-trained chained prediction model (MTMED) of this application were validated on four datasets. These four datasets are as follows:

[0087] DBLP Global: This dataset is the same as the one used by the baseline model. The baseline model used version 12 of the DDBLP dataset and validated its performance using data from 34 computer-related fields. This application uses the updated version 13 dataset. For comparison, the selection of years and fields for training and testing data is consistent with the baseline model.

[0088] DBLP China: Based on DBLP Global, a simple filtering rule is used to select papers whose first author's name is in Chinese, count the number of papers, and consider them to be papers from China, forming a specific dataset called DBLP China.

[0089] Aminer Global and Aminer China: The data source is the Zhiyin Fenny dataset on the Aminer website. This dataset was crawled from different domains on the website https: / / vip.aminer.cn / analysis / analysis / workplace, and 757 domains from 2001 to 2020 were selected as training and testing data. Data from 2001 to 2018 was used as the training set, and data from 2004 to 2020 was used as the testing set.

[0090] This application uses root mean square error and R on these datasets. 2The model's error and consistency with real data were measured. Experimental results demonstrate that the proposed model exhibits smaller prediction errors compared to the best current models across different tasks, domains, and datasets, while also achieving better consistency in predictions. Furthermore, the proposed model supports more flexible prediction modes through its prediction mechanism.

[0091] Figure 6 The figure shows the root mean square error (RMSE) results of the Chained Solution Prediction Model (TMED) and the Mask-Trained Chained Prediction Model (MTMED) from this application compared with the baseline model on four different datasets. It can be seen that, regardless of the dataset, this application achieves a lower prediction error compared to the previous best model. Figure 7 R is the results of testing different models on four datasets. 2 In terms of predictive relevance, the results predicted by the model in this application are more consistent with the trends of the actual data. Figure 8 This diagram illustrates the root mean square error for each year over the next five years. It shows that, except for the first year when all models have relatively similar errors, the differences between different models widen after the second year. The model in this application exhibits smaller errors after the second year. Figure 9 It is the annual forecast of R for the next five years. 2 The comparison shows that the three models performed similarly in the first year, but from the second year onwards, the model in this application performed significantly better, while the baseline model became increasingly inconsistent with the trend of the real data as the years progressed.

[0092] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A research trend prediction method based on LSTM networks, characterized in that, The method includes the following steps: S1. Using the first LSTM network as the encoder, the true value of each year in the research trend data of a certain field over the past t years is used as the input of each corresponding LSTM unit in the t LSTM units of the encoder. And the output vector of each LSTM unit in the t LSTM units of the encoder. Input the next LSTM unit to obtain the output vector of the last LSTM unit. ,in, ; S2. Using the second LSTM network as the decoder, the output vector of the last LSTM unit of the encoder is... Input the first LSTM unit of the decoder, and input the output vector of each of the n LSTM units of the decoder. Input its next LSTM unit, and the output vector of each of the n LSTM units of the decoder will be used. This is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value for each year is used as the input to the next LSTM unit in the decoder's n LSTM units. ,in, ; The research trend data refers to the number of published papers or patents. The first LSTM network and the second LSTM network are two LSTM networks using different parameters, or the same LSTM network using the same parameters. The method uses the true value of each year as the input of the next LSTM unit of the LSTM unit corresponding to that year during the decoder training phase. ; In the decoder training phase, if the current input year in the training set overlaps with the test set, the method uses the predicted value of the current input year as the input of the next LSTM unit of the LSTM unit corresponding to the current input year. ; The method sets a mask m during the decoder training phase, with a default value of 0. When the current input year y+t+j-1 overlaps with the test set, Then the inputs of the n LSTM units of the decoder are as follows: in, f Representative fields, y Indicates the start year. p For the true value, These are predicted values.

2. A research trend prediction device based on LSTM network, characterized in that, The device includes: The encoder using the first LSTM network has t LSTM units, which are used to take the true value of each year from the research trend data of a certain field over the past t years as the input to each of the corresponding t LSTM units of the encoder. And the output vector of each LSTM unit in the t LSTM units of the encoder. Input the next LSTM unit to obtain the output vector of the last LSTM unit. ,in, ; The decoder uses a second LSTM network with n LSTM units to process the output vector of the last LSTM unit of the encoder. Input the first LSTM unit of the decoder, and input the output vector of each of the n LSTM units of the decoder. Input its next LSTM unit, and the output vector of each of the n LSTM units of the decoder will be used. This is transformed into a one-dimensional vector as the predicted value for each of the next n years, and the predicted value for each year is used as the input to the next LSTM unit in the decoder's n LSTM units. ,in, ; The research trend data refers to the number of published papers or patents. The first LSTM network and the second LSTM network are either two LSTM networks using different parameters or the same LSTM network using the same parameters. During the training phase of the decoder, the true value of each year is used as the input to the next LSTM unit of the LSTM unit corresponding to that year. ; During the training phase of the decoder, if the current input year in the training set overlaps with the test set, the predicted value of the current input year is used as the input of the next LSTM unit of the LSTM unit corresponding to the current input year. ; The decoder's training phase sets a mask m, with a default value of 0. When the current input year y+t+j-1 overlaps with the test set, Then the inputs of the n LSTM units of the decoder are as follows: in, f Representative fields, y Indicates the start year. p For the true value, These are predicted values.

3. A research trend prediction device based on LSTM network, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the steps of the research trend prediction method based on LSTM networks as described in claim 1.

4. A computer-readable storage medium, characterized in that, The system contains a computer program that, when executed by a processor, implements the steps of the research trend prediction method based on LSTM networks as described in claim 1.

Images