Scientific research project effectiveness evaluation method based on LSTM network

Abstract

The invention provides a scientific research project effectiveness evaluation method based on an LSTM network, and the method comprises the steps: firstly constructing a scientific research project effectiveness evaluation index system which covers key factors such as project innovation, scientific research output, fund use efficiency and social influence; then, historical data, including index data, historical weights and evaluation scores after project implementation, of previous scientific research projects are collected, and training is carried out based on the LSTM network; and for a to-be-evaluated scientific research project, obtaining validity evaluation index data of the to-be-evaluated scientific research project and an index weight of a previous scientific research project, inputting the data into the trained LSTM network, predicting the weight of each index, and calculating an evaluation score in combination with the weights. The loss function comprehensively considers a prediction evaluation error, a prediction weight error and a next time node weight influence error, and ensures that a prediction result is more accurate. Scientific and dynamic evaluation can be provided before implementation of a scientific research project, decision reasonability is improved, and powerful support is provided for scientific research resource allocation and management.

Description

Technical Field

[0001] The present invention belongs to the field of evaluation technology, and specifically relates to a scientific research project effectiveness evaluation method based on LSTM network. Background Art

[0002] Evaluating the effectiveness of scientific research projects plays a crucial role in scientific research management and decision-making. A scientifically sound evaluation system is crucial for assessing whether a project should be implemented and how effectively it should be implemented. Traditional research project evaluation methods rely on fixed criteria and weightings, such as research output, technological innovation, funding efficiency, and social impact. However, these traditional methods fail to fully account for the changing priorities and external environments of research projects over time, resulting in an inability to accurately reflect the dynamic needs of different stages of project implementation.

[0003] After a research project is completed, decision-makers face the question of whether to implement it and how vigorously to implement it. At this point, changes in external factors such as market demand, policy support, and the social environment have a significant impact on the implementation of the research project. Therefore, the emphasis on various research project indicators can vary significantly at different points in time. For example, in one period, policy support and market demand may make the conversion rate and social impact of research results the focus of evaluation; in another period, the innovation and technological breakthroughs of research outputs may be more critical.

[0004] However, traditional research project evaluation methods fail to dynamically consider the impact of these external factors. They typically assume that each evaluation indicator has the same weight and influence at all stages. This fixed-weight evaluation approach is unsuitable for the changing needs of research projects, as the value of research project implementation is not static and is dynamically influenced by multiple factors, including market conditions, policies, and the social environment.

[0005] Therefore, the evaluation methods for scientific research projects need to flexibly adjust the emphasis at different time points. Specifically, the evaluation system should be able to dynamically adjust the weights of various evaluation indicators based on factors such as the stage of the scientific research project, the market environment, and policy changes. This means that as the project is implemented, the weights of the evaluation indicators and their impact on the final results of the project will also change. How to reasonably predict the weights of various indicators at different time points and combine these weight changes with the implementation results of the scientific research project has become a technical problem that needs to be solved in the field of scientific research project evaluation.

[0006] In the prior art, Chinese patent application CN110751355A discloses a method and apparatus for evaluating scientific and technological achievements. The method includes: collecting historical information on scientific research achievements, analyzing the historical information to obtain characteristics of the achievements; constructing an adaptive evaluation index system for scientific research achievements based on the characteristics of the achievements; quantifying the adaptive evaluation index system to construct an adaptive evaluation index system model; adjusting the weights of various evaluation indicators in the adaptive evaluation index system model; comparing and analyzing the weights of the same indicator across different historical information sets of scientific research achievements, and optimizing the adaptive evaluation index system model based on the relationship between the weight settings and reward characteristics and the attributes of the indicators; and evaluating multiple projects using the adaptive evaluation index system model to obtain evaluation results. This method relies on historical scientific research project data for indicator weight adjustment and utilizes an inverse coupling algorithm for optimization. However, this optimization approach is based on adjustments for completed projects and fails to reflect the dynamic changes in indicator weights across scientific research projects at different time points due to changes in policies and market environments. In other words, this method is only applicable to evaluating the effectiveness of implemented scientific research projects and cannot adapt to the changing trends in scientific research project evaluation criteria over time. Summary of the Invention

[0007] The purpose of the present invention is to overcome the above-mentioned defects of the existing technology and provide a scientific research project effectiveness evaluation method based on LSTM network to solve the problems that the existing scientific research project evaluation methods are difficult to adapt to the dynamically changing scientific research environment and the indicator weights cannot be accurately predicted.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] On the one hand, the present invention provides a method for evaluating the effectiveness of scientific research projects based on an LSTM network, comprising the following steps:

[0010] Constructing evaluation indicators for the effectiveness of scientific research projects;

[0011] Collect historical data of past scientific research projects, including indicator data, historical weight data and evaluation scores of each scientific research project after its implementation;

[0012] Train the LSTM network based on historical data from past scientific research projects;

[0013] Obtain the effectiveness evaluation index data of the scientific research project to be evaluated and the index weight of the previous scientific research project, input the effectiveness evaluation index data of the scientific research project to be evaluated and the index weight of the previous scientific research project into the trained LSTM network, and obtain the predicted index weight of the effectiveness evaluation index of the scientific research project to be evaluated;

[0014] The evaluation score of the scientific research project to be evaluated is obtained based on the predicted indicator weights and the effectiveness evaluation indicator data of the scientific research project to be evaluated.

[0015] Furthermore, the evaluation indicators for the effectiveness of scientific research projects include project innovation, scientific research output, funding efficiency, and social impact, among which project innovation includes the degree of technological innovation, the novelty of research methods, and breakthroughs in key technologies; scientific research output includes the number of papers published and their impact factors, patent applications and authorizations, and the conversion rate of scientific and technological achievements; funding efficiency includes the rationality of the use of scientific research funds, budget execution, and the ratio of capital input to output; and social impact includes the market application prospects of research results and the degree of policy support.

[0016] Furthermore, the indicator data of each scientific research project are the effectiveness evaluation index scores of the scientific research projects at multiple time points, wherein the effectiveness evaluation indicators include quantitative indicators and non-quantitative indicators. The quantitative indicators include: the number of papers published and the impact factor, patent application and authorization status, the conversion rate of scientific and technological achievements, the ratio of capital input to output, and the non-quantitative indicators include: the degree of technological innovation, the novelty of research methods, key technological breakthroughs, the rationality of the use of scientific research funds, budget execution, the market application prospects of research results, and the degree of policy support. The non-quantitative indicator scores are scored by experts and converted into numerical scores after standardization.

[0017] Furthermore, the historical weight data includes the weights of various effectiveness evaluation indicators.

[0018] Furthermore, the evaluation score after the implementation of the scientific research project is a score given based on the benefits generated by the scientific research project after a period of time has passed since the implementation of the scientific research project results.

[0019] Furthermore, the evaluation score after the implementation of the scientific research project is calculated by the following steps:

[0020] Collect benefit data after the implementation of the scientific research project, including economic benefits, social benefits and academic impact;

[0021] Convert the income data into quantitative scoring indicators, where economic benefits include income from the transformation of scientific and technological achievements, sales revenue, and return on investment; social benefits include policy support, industry impact, and social contribution; and academic impact includes the number of scientific research papers published, impact factors, and citations;

[0022] The benefit data after the implementation of the scientific research project is weighted and summed to obtain the evaluation score after the implementation of the scientific research project.

[0023] Furthermore, the LSTM network is trained based on historical data from past scientific research projects, specifically including:

[0024] Sort the historical scientific research project data at each time node in chronological order to construct time series data;

[0025] Construct input data for each time node, including the scientific research project effectiveness evaluation index data and corresponding historical weights at that time node;

[0026] The input data of each time node is input into the LSTM network in sequence, and the LSTM network outputs the prediction weight of the time node;

[0027] Calculate the predicted evaluation score for that time node using the scientific research project effectiveness evaluation index score at that time node and the predicted weight;

[0028] Calculate the error between the predicted evaluation score and the actual evaluation score after a period of implementation of the scientific research project to obtain the predicted evaluation score error;

[0029] Calculate the error between the predicted weight and the historical weight at the next time node to obtain the predicted weight error;

[0030] Calculate the difference between the historical weight and indicator score of the next time node and the actual evaluation score after a period of implementation of the scientific research project at the next time node to obtain the weight impact error of the next time node;

[0031] The loss function is constructed by comprehensively considering the prediction evaluation score error, prediction weight error, and the weight impact error of the next time node;

[0032] The parameters of the LSTM network are adjusted using the gradient descent method or other optimization algorithms to minimize the loss function and obtain the trained LSTM network.

[0033] Furthermore, the loss function is:

[0034] L1=(S pred,t -S true,t ) 2

[0035]

[0036] L3=(S hist,t+1 -S true,t+1 ) 2

[0037]

[0038] Among them, L1 is the prediction evaluation score error at time node t, S pred,t is the evaluation score of the prediction at time node t, S true,tis the actual evaluation score of the scientific research project at time node t after a period of implementation, n is the total number of effectiveness evaluation indicators, X i,t is the score of the i-th effectiveness evaluation index at time node t, W pred,i,t is the weight of the effectiveness evaluation index of the i-th scientific research project at the time node t predicted by the LSTM network output, L2 is the prediction weight error at the time node t, and W true,i,t+1 is the historical weight of the effectiveness evaluation index of the scientific research project at time node t+1, L3 is the weight influence error of the next time node at time node t, S hist,t+1 is the historical evaluation score at time node t+1, S true,t+1 is the actual evaluation score of the scientific research project at time node t+1 after a period of implementation, L is the loss function, and α is the hyperparameter.

[0039] Furthermore, the effectiveness evaluation index data of the scientific research project to be evaluated is the effectiveness evaluation index data obtained in the pre-implementation stage after the completion of the scientific research project to be evaluated.

[0040] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for evaluating the effectiveness of a scientific research project based on an LSTM network as described above.

[0041] Compared with the prior art, the present invention has the following advantages:

[0042] (1) This paper uses time series modeling of historical data from past scientific research projects and uses an LSTM network for training, enabling the model to learn the changing trends of indicator weights of historical scientific research projects at different time points. This method not only takes into account the historical indicator data and weight data of scientific research projects, but also predicts future weight changes, allowing the evaluation system to adapt to the dynamic changes of external factors such as the market environment and policy orientation, thereby improving the flexibility and applicability of scientific research project evaluation.

[0043] (2) The present invention constructs a loss function by introducing the relationship between the prediction weight error and the weight influence error of the next time node, and optimizes the parameters of the LSTM network to ensure that the model can accurately predict the weight change trend of each evaluation indicator of the scientific research project. Through this technical means, the present invention can predict the possible future benefits of a newly completed scientific research project before it is implemented, thereby providing decision makers with a scientific and reasonable reference basis to assist in judging whether the scientific research project is worth implementing and the scale of resources that should be invested.

[0044] (3) This invention effectively improves the accuracy of scientific research project evaluation by using a time series analysis method, combining historical indicator data, historical weight data, and post-implementation evaluation scores of scientific research projects. Compared with existing technologies, this invention not only focuses on the current evaluation of scientific research projects, but also uses an LSTM network to predict the future revenue trends of scientific research projects, thus avoiding the lag problem caused by existing evaluation methods that rely on static weight adjustment.

[0045] (4) The present invention optimizes the evaluation system of scientific research projects by combining the actual benefit data after the implementation of scientific research projects, so that the weights of evaluation indicators can be dynamically adjusted according to historical data and forecast data, thereby more realistically reflecting the actual value of scientific research projects in different time contexts. Compared with existing methods, the dynamic weight prediction mechanism of the present invention can more effectively adapt to the focus of scientific research management departments at different time stages, thereby improving the timeliness and adaptability of scientific research project evaluation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the method of the present invention;

[0047] Figure 2 Schematic diagram of the evaluation index of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0049] Example 1:

[0050] This embodiment provides a method for evaluating the effectiveness of scientific research projects based on LSTM networks. Figure 1 As shown, the following steps are included:

[0051] Constructing evaluation indicators for the effectiveness of scientific research projects;

[0052] Collect historical data of past scientific research projects, including indicator data, historical weight data and evaluation scores of each scientific research project after its implementation;

[0053] Train the LSTM network based on historical data from past scientific research projects;

[0054] Obtain the effectiveness evaluation index data of the scientific research project to be evaluated and the index weight of the previous scientific research project, input the effectiveness evaluation index data of the scientific research project to be evaluated and the index weight of the previous scientific research project into the trained LSTM network, and obtain the predicted index weight of the effectiveness evaluation index of the scientific research project to be evaluated;

[0055] The evaluation score of the scientific research project to be evaluated is obtained based on the predicted indicator weights and the effectiveness evaluation indicator data of the scientific research project to be evaluated.

[0056] This paper constructs a scientific research project effectiveness evaluation index system and collects historical data from past scientific research projects, including index data, historical weight data, and evaluation scores after implementation. The system then uses an LSTM network for training to learn the dynamic changes in the weights of scientific research project indicators. Through time series modeling, the LSTM network parameters are optimized by combining the error in predicted evaluation scores, predicted weights, and the error in the weight impact of the next time node, achieving accurate predictions of the weights of scientific research project evaluation indicators.

[0057] During the research project evaluation phase, the effectiveness evaluation index data for the research project to be evaluated, along with the index weights of the previous research project, are fed into a trained LSTM network to obtain the predicted index weights. This system then combines the index data to calculate the evaluation score. This system dynamically adapts to the focus of research management departments at different time stages, making evaluation results more scientific and reasonable, improving the accuracy of research project implementation decisions, and providing intelligent support for research management.

[0058] Further, such as Figure 2 As shown in the figure, the evaluation indicators for the effectiveness of scientific research projects include project innovation, scientific research output, efficiency of fund use, and social impact. Project innovation includes the degree of technological innovation, the novelty of research methods, and breakthroughs in key technologies; scientific research output includes the number of papers published and their impact factors, patent applications and authorizations, and the conversion rate of scientific and technological achievements; fund use efficiency includes the rationality of scientific research fund use, budget execution, and the ratio of capital input to output; and social impact includes the market application prospects of research results and the degree of policy support.

[0059] The construction of scientific research project effectiveness evaluation indicators requires comprehensive consideration of multiple dimensions to fully reflect the comprehensive value of scientific research projects. Therefore, the present invention sets four major categories of evaluation indicators, including project innovation, scientific research output, funding efficiency, and social impact. Among them, project innovation measures the degree of breakthrough in scientific research projects in technological innovation and research methods, specifically including the degree of technological innovation, the novelty of research methods, and key technological breakthroughs. By introducing these indicators, it can be ensured that the innovation of scientific research projects is objectively evaluated, which helps to screen out cutting-edge and breakthrough scientific research projects and improve the scientific nature of scientific research resource allocation.

[0060] Furthermore, the indicator data of each scientific research project are the effectiveness evaluation index scores of the scientific research projects at multiple time points, wherein the effectiveness evaluation indicators include quantitative indicators and non-quantitative indicators. The quantitative indicators include: the number of papers published and the impact factor, patent application and authorization status, the conversion rate of scientific and technological achievements, the ratio of capital input to output, and the non-quantitative indicators include: the degree of technological innovation, the novelty of research methods, key technological breakthroughs, the rationality of the use of scientific research funds, budget execution, the market application prospects of research results, and the degree of policy support. The non-quantitative indicator scores are scored by experts and converted into numerical scores after standardization.

[0061] The evaluation of scientific research projects needs to be based on sufficient historical data. Therefore, this method uses the effectiveness evaluation index scores of scientific research projects at multiple time points as input data to construct a scientific and reasonable time series model. By introducing data from multiple time points, it is possible to capture the performance trends of scientific research projects at different stages, thereby improving the accuracy and stability of the evaluation. Compared with using data from only a single time point, this method can more comprehensively reflect the long-term impact of scientific research projects and help improve the reliability of prediction results.

[0062] The effectiveness evaluation indicators for scientific research projects are divided into quantitative and non-quantitative indicators to ensure comprehensive and scientific evaluation. Quantitative indicators include the number of published papers and their impact factors, patent applications and authorizations, the conversion rate of scientific and technological achievements, and the ratio of capital input to output. These indicators are objective and quantifiable, and can be directly used in modeling and analysis. The introduction of quantitative indicators helps ensure that the evaluation results of scientific research projects are based on objective data, thereby enhancing the credibility of the evaluation.

[0063] Non-quantitative indicators include the degree of technological innovation, the novelty of research methods, key technological breakthroughs, the rationality of the use of scientific research funds, budget execution, the market application prospects of research results, policy support, etc. These indicators are difficult to measure directly by objective data, so the present invention adopts the method of expert scoring and converts it into a numerical score by standardization so as to be included in calculation and analysis. By standardization, the deviation of different experts in the scoring scale can be reduced, the consistency of data can be improved, and non-quantitative indicators can be effectively participated in the overall evaluation of scientific research projects. Compared with existing methods that simply rely on quantitative data or subjective judgment, this method can fully consider the actual situation of scientific research projects while ensuring objectivity, make the evaluation results more comprehensive and reasonable, and provide more powerful support for scientific research management and decision-making.

[0064] Furthermore, the historical weight data includes the weights of various effectiveness evaluation indicators.

[0065] Furthermore, the evaluation score after the implementation of the scientific research project is a score given based on the benefits generated by the scientific research project after a period of time has passed since the implementation of the scientific research project results.

[0066] Furthermore, the evaluation score after the implementation of the scientific research project is calculated by the following steps:

[0067] Collect benefit data after the implementation of the scientific research project, including economic benefits, social benefits and academic impact;

[0068] Convert the income data into quantitative scoring indicators, where economic benefits include income from the transformation of scientific and technological achievements, sales revenue, and return on investment; social benefits include policy support, industry impact, and social contribution; and academic impact includes the number of scientific research papers published, impact factors, and citations;

[0069] The benefit data after the implementation of the scientific research project is weighted and summed to obtain the evaluation score after the implementation of the scientific research project.

[0070] The effectiveness of a scientific research project is not only reflected in its short-term performance after completion, but also requires a comprehensive assessment of its long-term benefits after implementation. Therefore, the present invention collects benefit data after the implementation of scientific research projects, including economic benefits, social benefits, and academic impact, to construct a comprehensive project evaluation system. Economic benefits reflect the market conversion ability and actual benefits of scientific research projects, social benefits measure the role of scientific research results in promoting industry and social development, and academic impact reflects the recognition and dissemination value of scientific research projects in the academic community. By integrating data from these three aspects, the actual contribution of scientific research projects can be evaluated more comprehensively and objectively.

[0071] Since the types of income data of different scientific research projects are different, it is difficult to obtain reasonable evaluation results by direct comparison. Therefore, the present invention converts the income data into quantitative scoring indicators to ensure that different types of data can be calculated in the same evaluation system. In terms of economic benefits, the income from the transformation of scientific and technological achievements, sales revenue and return on investment can intuitively reflect the market value of scientific research results. In terms of social benefits, the degree of policy support, industry impact and social contribution can measure the role of scientific research projects in policy guidance and industry applications. In terms of academic influence, the number of publications, impact factors and citations of scientific research papers can characterize the academic value and dissemination of scientific research projects. By quantifying income data of different dimensions into computable indicators, not only the objectivity of the evaluation is improved, but also standardized data input is provided for subsequent calculations.

[0072] After obtaining quantified benefit data, the present invention uses a weighted summation approach to calculate the evaluation score after the implementation of the scientific research project. This weighted summation can assign different weights based on the importance of different evaluation indicators, making the final evaluation results more consistent with actual conditions. Compared to simple average calculation methods, the weighted summation can flexibly adjust the contribution of each indicator to meet the evaluation requirements of different types of scientific research projects. Through this method, the present invention can accurately reflect the overall value of scientific research projects, provide reliable data support for scientific research management departments, and thus optimize resource allocation and improve the efficiency of scientific research funding.

[0073] Furthermore, the LSTM network is trained based on historical data from past scientific research projects, specifically including:

[0074] Sort the historical scientific research project data at each time node in chronological order to construct time series data;

[0075] Construct input data for each time node, including the scientific research project effectiveness evaluation index score and the corresponding historical weight at that time node;

[0076] The input data of each time node is input into the LSTM network in sequence, and the LSTM network outputs the prediction weight of the time node;

[0077] Calculate the predicted evaluation score for that time node using the scientific research project effectiveness evaluation index score at that time node and the predicted weight;

[0078] Calculate the error between the predicted evaluation score and the actual evaluation score after a period of implementation of the scientific research project to obtain the predicted evaluation score error;

[0079] Calculate the error between the predicted weight and the historical weight at the next time node to obtain the predicted weight error;

[0080] Calculate the difference between the historical weight and indicator score of the next time node and the actual evaluation score after a period of implementation of the scientific research project at the next time node to obtain the weight impact error of the next time node;

[0081] The loss function is constructed by comprehensively considering the prediction evaluation score error, prediction weight error, and the weight impact error of the next time node;

[0082] The parameters of the LSTM network are adjusted using the gradient descent method or other optimization algorithms to minimize the loss function and obtain the trained LSTM network.

[0083] The effectiveness evaluation of scientific research projects relies on a wealth of historical data, and the evaluation criteria for scientific research projects at different time stages are dynamically adjusted as policy guidance, market demand, and technological development trends change. Therefore, to build a model that can adapt to the evaluation needs of scientific research projects at different time stages, this paper uses an LSTM network for training to learn the dynamic weight changes of each evaluation indicator at different time points and predict the optimal indicator weights for future scientific research projects.

[0084] First, historical research project data at each time point is sorted chronologically to construct time series data. Since the evaluation of research projects relies on long-term trend analysis, and LSTM networks are well-suited to processing time series data, inputting sorted data into the LSTM network allows the model to learn from historical patterns of change, improving the temporal consistency and accuracy of predictions.

[0085] Next, we construct input data for each time point, including the project effectiveness evaluation indicator scores and corresponding historical weights at that time point. This step aims to feed the LSTM network with the characteristic information of the research project at a specific point in time, enabling it to learn the changes in the importance of each indicator over different time periods. By incorporating historical weights, the model can focus on the impact of each evaluation indicator at different stages on the final implementation of the research project, making the prediction results more valuable.

[0086] Next, the input data for each time point is sequentially fed into the LSTM network, which then outputs the predicted weight for that time point. The LSTM network extracts effective features from long-term dependencies and learns implicit patterns in time series data, thereby predicting the weight trends of various evaluation indicators for the research project at the current time point. Compared to traditional static weight assignment methods, the LSTM network can adjust the predicted weights based on the dynamic trends of historical data, making the evaluation of research projects more realistic.

[0087] Subsequently, the predicted evaluation score for that time point is calculated using the research project effectiveness evaluation index score at that time point and the predicted weight. This calculation step can obtain the predicted score of the research project at the current time point, thereby evaluating the accuracy of the LSTM network's evaluation of the research project's effectiveness at the current stage.

[0088] To optimize the LSTM network's predictions, error calculation is necessary. First, the error between the predicted evaluation score and the actual evaluation score after a period of implementation is calculated to obtain the predicted evaluation score error. This error reflects the accuracy of the LSTM network's prediction of the project's final returns. The smaller the error, the closer the prediction is to the actual situation.

[0089] At the same time, the error between the predicted weight and the historical weight at the next time point is calculated to obtain the predicted weight error. This error measures the reliability of the LSTM network's prediction of the weight of future scientific research project indicators. A smaller error indicates that the model can better predict the changing trend of the importance of each indicator.

[0090] Furthermore, to further optimize the prediction results, the difference between the historical weight and indicator score at the next time point and the actual evaluation score after a period of implementation of the research project at the next time point is calculated to obtain the weight impact error at the next time point. This error is used to measure the impact of historical weights on the actual benefits of the research project and is used to adjust the training objectives of the LSTM network, so that the model pays more attention to the evaluation indicators with higher importance.

[0091] Finally, the predicted evaluation score error, predicted weight error, and the error in the weight impact at the next time node are combined to construct a loss function. Gradient descent or other optimization algorithms are then used to adjust the LSTM network parameters to minimize the loss function, ultimately yielding a trained LSTM network. This optimization process enables the LSTM network to continuously learn the weight variation patterns in historical data, improving the accuracy and adaptability of scientific research project evaluations and providing more reliable decision-making basis for scientific research management departments.

[0092] Furthermore, the loss function is:

[0093] L1=(S pred,t -S true,t ) 2

[0094]

[0095] L3=(S hist,t+1 -S true,t+1 ) 2

[0096]

[0097] Among them, L1 is the prediction evaluation score error at time node t, S pred,t is the evaluation score of the prediction at time node t, S true,t is the actual evaluation score of the scientific research project at time node t after a period of implementation, n is the total number of effectiveness evaluation indicators, X i,t is the score of the i-th effectiveness evaluation index at time node t, W pred,i,t is the weight of the effectiveness evaluation index of the i-th scientific research project at time node t predicted by the LSTM network output, L2 is the prediction weight error at time node t, and W true,i,t+1is the historical weight of the effectiveness evaluation index of the scientific research project at time node t+1, L3 is the weight influence error of the next time node at time node t, S hist,t+1 is the historical evaluation score at time node t+1, S true,t+1 is the actual evaluation score of the scientific research project at time node t+1 after a period of implementation, L is the loss function, and α is the hyperparameter.

[0098] By designing the L2 / L3 ratio, the loss function dynamically adjusts the weight error. By dividing L2 by L3, the loss function implements a dynamic balancing mechanism. When the weight impact error L3 at the next time point is large, the impact of L2 is relatively reduced, and vice versa. This design allows the loss function to more sensitively adjust the optimization process, focusing on the changing trend of the weight error. If the impact of the next time point is large, the loss function will place greater emphasis on reducing the weight error, allowing the model to flexibly adjust weights when evaluating scientific research projects, making the prediction results more in line with actual needs. In addition, α, as a hyperparameter, is used to balance the weight between evaluation error and weight error, ensuring that the loss function can reasonably optimize the LSTM network, giving it stronger generalization capabilities and improving the accuracy and timeliness of scientific research project evaluations.

[0099] Furthermore, the effectiveness evaluation index data of the scientific research project to be evaluated is the effectiveness evaluation index data obtained in the pre-implementation stage after the completion of the scientific research project to be evaluated.

[0100] By using the effectiveness evaluation index data from the pre-implementation phase of the research project to be evaluated, the present invention can make accurate predictions before the project is officially implemented, avoiding interference from external factors on the evaluation results and ensuring that the evaluation better reflects the intrinsic scientific research value of the project. This approach identifies potential and risks in advance, helps decision makers develop more scientific and reasonable implementation strategies, improves resource allocation efficiency, and optimizes the success rate of research projects. At the same time, loss function calculations based on data from the pre-implementation phase make model predictions more accurate, providing a reliable basis for subsequent project implementation.

[0101] Example 2:

[0102] This embodiment provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0103] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method described above is implemented.

[0104] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A scientific research project effectiveness evaluation method based on LSTM network, characterized in that: The following steps are involved: Constructing evaluation indicators for the effectiveness of scientific research projects; Collect historical data of past scientific research projects, including indicator data, historical weight data and evaluation scores of each scientific research project after its implementation; Train the LSTM network based on historical data from past scientific research projects; Obtain the effectiveness evaluation index data of the scientific research project to be evaluated and the index weight of the previous scientific research project, input the effectiveness evaluation index data of the scientific research project to be evaluated and the index weight of the previous scientific research project into the trained LSTM network, and obtain the predicted index weight of the effectiveness evaluation index of the scientific research project to be evaluated; The evaluation score of the scientific research project to be evaluated is obtained based on the predicted indicator weights and the effectiveness evaluation indicator data of the scientific research project to be evaluated.

2. A scientific research project effectiveness evaluation method based on LSTM network according to claim 1, characterized in that: The evaluation indicators for the effectiveness of scientific research projects include project innovation, scientific research output, funding efficiency, and social impact. Project innovation includes the degree of technological innovation, the novelty of research methods, and breakthroughs in key technologies; scientific research output includes the number of papers published and their impact factors, patent applications and authorizations, and the conversion rate of scientific and technological achievements; funding efficiency includes the rationality of scientific research fund use, budget execution, and the ratio of capital input to output; and social impact includes the market application prospects of research results and the degree of policy support.

3. The method for evaluating the effectiveness of scientific research projects based on LSTM network according to claim 1, characterized in that: The indicator data of each scientific research project are the effectiveness evaluation index scores of the scientific research projects at multiple time points, where the effectiveness evaluation indicators include quantitative indicators and non-quantitative indicators. The quantitative indicators include: the number of papers published and the impact factor, patent applications and authorizations, the conversion rate of scientific and technological achievements, and the ratio of capital input to output. The non-quantitative indicators include: the degree of technological innovation, the novelty of research methods, breakthroughs in key technologies, the rationality of the use of scientific research funds, budget execution, the market application prospects of research results, and the degree of policy support. The non-quantitative indicator scores are scored by experts and converted into numerical scores after standardization.

4. The method for evaluating the effectiveness of a scientific research project based on an LSTM network according to claim 1, wherein: The historical weight data includes the weights of various effectiveness evaluation indicators.

5. The method for evaluating the effectiveness of scientific research projects based on LSTM network according to claim 1, characterized in that: The evaluation score after the implementation of the scientific research project is a score given based on the benefits generated by the scientific research project after a period of implementation of the scientific research project results.

6. The method for evaluating the effectiveness of scientific research projects based on LSTM networks according to claim 5, characterized in that: The evaluation score after the implementation of the scientific research project is calculated through the following steps: Collect benefit data after the implementation of the scientific research project, including economic benefits, social benefits and academic impact; Convert the income data into quantitative scoring indicators, where economic benefits include income from the transformation of scientific and technological achievements, sales revenue, and return on investment; social benefits include policy support, industry impact, and social contribution; and academic impact includes the number of scientific research papers published, impact factors, and citations; The benefit data after the implementation of the scientific research project is weighted and summed to obtain the evaluation score after the implementation of the scientific research project.

7. The method for evaluating the effectiveness of scientific research projects based on LSTM network according to claim 1, characterized in that: The training of the LSTM network based on historical data from past scientific research projects specifically includes: Sort the historical scientific research project data at each time node in chronological order to construct time series data; Construct input data for each time node, including the scientific research project effectiveness evaluation index data and corresponding historical weights at that time node; The input data of each time node is input into the LSTM network in sequence, and the LSTM network outputs the prediction weight of the time node; Calculate the predicted evaluation score for that time node using the scientific research project effectiveness evaluation index score at that time node and the predicted weight; Calculate the error between the predicted evaluation score and the actual evaluation score after a period of implementation of the scientific research project to obtain the predicted evaluation score error; Calculate the error between the predicted weight and the historical weight at the next time node to obtain the predicted weight error; Calculate the difference between the historical weight and indicator score of the next time node and the actual evaluation score after a period of implementation of the scientific research project at the next time node to obtain the weight impact error of the next time node; The loss function is constructed by comprehensively considering the prediction evaluation score error, prediction weight error, and the weight impact error of the next time node; The parameters of the LSTM network are adjusted using the gradient descent method or other optimization algorithms to minimize the loss function and obtain the trained LSTM network.

8. The method for evaluating the effectiveness of scientific research projects based on LSTM networks according to claim 7, characterized in that: The loss function is: L1=(S pred,t -S true,t ) 2 Among them, L1 is the prediction evaluation score error at time node t, S pred,t is the evaluation score of the prediction at time node t, S true,t is the actual evaluation score of the scientific research project at time node t after a period of implementation, n is the total number of effectiveness evaluation indicators, X i,t is the score of the i-th effectiveness evaluation index at time node t, W pred,i,t is the weight of the effectiveness evaluation index of the i-th scientific research project at the time node t predicted by the LSTM network output, L2 is the prediction weight error at the time node t, and W true,i,t+1 is the historical weight of the effectiveness evaluation index of the scientific research project at time node t+1, L3 is the weight influence error of the next time node at time node t, S hist,t+1 is the historical evaluation score at time node t+1, S true,t+1 is the actual evaluation score of the scientific research project at time node t+1 after a period of implementation, L is the loss function, and α is the hyperparameter.

9. The method for evaluating the effectiveness of a scientific research project based on an LSTM network according to claim 1, wherein: The effectiveness evaluation index data of the scientific research project to be evaluated is the effectiveness evaluation index data obtained in the pre-implementation stage after the completion of the scientific research project to be evaluated.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images