Construction method of influence analysis and evaluation model

Abstract

The invention discloses a construction method of an influence analysis and evaluation model. The method comprises the following steps: collecting literature input variables; carrying out modeling analysis by respectively taking an international influence alternative index Altmetics and academic reference as output variables; selecting the algorithm with the highest reference coefficient as an input algorithm for modeling to carry out grid parameter adjustment for modeling; a feature importance evaluation model is adopted, and the contribution degree of each variable to the result influence is compared; and displaying the trend of the influence of each variable on the achievement by using a partial correlation graph, comprehensively evaluating the influence of the achievement by combining the importance of the use features and the partial correlation graph, and outputting a result. According to the method disclosed by the invention, by comparing the factors which have the greatest influence on the academic reference with the factors which have the greatest influence on the Almetics, the main source of the international influence of the research result and the influence mode of the international influence of the research result can be effectively analyzed, and a more scientific and traceable analysis tool is provided for the international influence evaluation of the research result.

Description

Technical Field

[0001] The present invention relates to the field of industrial 5G, and in particular to a method for constructing an impact analysis and evaluation model. Background Art

[0002] Evaluation technology is increasingly being used in the field of industrial 5G, which can help interactive systems evaluate networks in a timely and efficient manner, and realize network analysis and evaluation. At present, due to the characteristics of 5G, such as huge amount of information, diverse types, and high information density, there are many uncertain factors in the evaluation method, resulting in greater uncertainty in the evaluation method. Although some methods for constructing evaluation models for 5G networks have been invented, they still cannot effectively solve the uncertainty problem of the evaluation method. Summary of the invention

[0003] The purpose of the present invention is to provide a method for constructing an impact analysis and evaluation model.

[0004] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0005] The present invention comprises the following steps:

[0006] A. Collect literature input variables;

[0007] B uses Altmetrics and academic citations as output variables for modeling and analysis;

[0008] C adds academic application volume and international influence proxy Altmetrics to the data set, performs 10-fold cross validation based on the inherited algorithm pool of the decision tree, evaluates the important performance of the algorithm on the data set in turn, and uses 10-fold cross validation to evaluate the benchmark performance of each algorithm on the data set in turn;

[0009] D. Select the algorithm with the highest benchmark coefficient as the input algorithm for modeling and adjust the grid parameters for modeling;

[0010] E uses the feature importance assessment model and compares the contribution of each variable to the impact of the outcome;

[0011] F uses partial correlation diagrams to show the trend of the impact of each variable on the results, and combines the use of feature importance and partial correlation diagrams to comprehensively evaluate the impact of the results and output the results.

[0012] Furthermore, the preprocessing method includes missing value processing, outlier detection and processing, and data type conversion.

[0013] Furthermore, the document input variables include the publication year, author, research institution, research field, abstract content, and keywords of the paper.

[0014] Furthermore, the succession of decision tree-based algorithms include Adaptive Boosting, Extremely Randomized Trees, Random Forest, and Gradient Boosting Machine.

[0015] 5. The method for constructing an impact analysis and evaluation model according to claim 1 is characterized in that a 10-fold cross-validation is performed to evaluate the performance of the algorithm on the data set, including accuracy, recall, and F1 score indicators, to estimate whether the algorithm performs best among all algorithms. If not, return to select other algorithms to repeat the steps.

[0016] Furthermore, the specific standardized equation of the partial dependence graph is calculated as follows:

[0017]

[0018] Where f is the algorithm, i is the sample instance, i is the variable, n is the number of samples, and j is the sample number.

[0019] Furthermore, the partial dependence graph is calculated as follows:

[0020]

[0021] Where X is the input data set, Xs is the variable data set to be analyzed, and Xc is the other variable data set. xs is a single instance in Xs, and xc is a single instance in Xc. The analysis of a single variable includes the following:

[0022]

[0023] Where n is the sample size.

[0024] In a second aspect, an embodiment of the present application further provides an electronic device, including:

[0025] Processor; and

[0026] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method steps described in the first aspect.

[0027] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs.

[0028] The beneficial effects of the present invention are:

[0029] The present invention is a method for constructing an impact analysis and evaluation model. Compared with the prior art, the present invention has the following technical effects:

[0030] By comparing the factors that have the greatest impact on academic citations and the factors that have the greatest impact on Almetrics, the present invention can effectively analyze the main sources of the international impact of research results and their impact methods, providing a more scientific and traceable analysis tool for the evaluation of the international impact of research results. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flowchart of the steps of a method for constructing an impact analysis and evaluation model of the present invention;

[0032] Figure 2 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The present invention is further described below by means of specific embodiments. The illustrative embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0034] The present invention provides a method for constructing an impact analysis and evaluation model, comprising the following steps:

[0035] like Figure 1 As shown, in this embodiment, the following steps are included:

[0036] This paper uses the research results funded by the second regional comprehensive scientific investigation research project as an example to demonstrate the model establishment and evaluation process. The research data comes from the Dimensions database (https: / / app.dimensions.ai / discover / publication).

[0037] Based on the STEP project funding numbers as of November 11, 2022, a total of 4,679 publication records were retrieved. During the modeling process, 3,221 records with matching journal names on Resurchify were selected. The entry dimensions used for modeling include: research topics, number of research citations, altmetrics indicators of the research, years since the research was published, number of authors, professional science communication actions, and the impact factor, h-index, SCImago Journal Ranking (SJR) and overall ranking of the journal in which the research was published. Among them, professional science communication actions refer to promotional press releases issued by professional science communicators of scientific research institutions on international science communication platforms. If the press release of related research can be searched, it is a professional science communication action. Except for professional science communication actions, all data are obtained from the Resurchify website.

[0038] Collect literature input variables; use Altmetrics, a proxy indicator of international influence, and academic citations as output variables for modeling and analysis; add academic citations and Altmetrics to the data set, perform 10-fold cross-validation based on the inherited algorithm pool of the decision tree, evaluate the important performance of the algorithm on the data set in turn, and use 10-fold cross-validation to evaluate the benchmark performance of each algorithm on the data set in turn; select the algorithm with the highest benchmark coefficient as the input algorithm for modeling to adjust the grid parameters of modeling; use the feature importance evaluation model and compare the contribution of each variable to the impact of the results. Use PDP to show the trend of the impact of each variable on the results, and finally combine FIS and PDP to comprehensively evaluate the impact of the results.

[0039] Among them, Gradient Boosting Machine (GBM):

[0040] GBM is a popular gradient boosting algorithm that improves the performance of a model by iteratively training weak learners (usually decision trees), each time trying to correct the mistakes of the previous model.

[0041] It starts with an initial model and then adds new models to correct the residual errors of the previous model. This process is repeated until a predetermined number of iterations is reached or the model performance no longer improves significantly.

[0042] Random Forest(RF):Gradient Boosting Machine

[0043] RF is an ensemble learning method that builds multiple decision trees and combines their prediction results to improve the accuracy and robustness of the model.

[0044] Each decision tree is trained using a randomly selected subset of features, which increases diversity between trees and reduces the risk of overfitting.

[0045] Extremely Randomized Trees(ET):Random Forest(RF):Gradient BoostingMachine

[0046] ET, also known as Extra-Trees, is an ensemble method similar to Random Forest, but it uses extreme randomness when building decision trees.

[0047] In ET, the split of each node is not based on the best split point, but on randomly selected features and randomly selected thresholds, which helps to increase the diversity of the model.

[0048] AB, also known as AdaBoost, is an adaptive ensemble method that iteratively trains weak learners, with each new learner focusing on samples that were misclassified by the previous learner.

[0049] AdaBoost works by assigning weights to each training example, with each new model placing more emphasis on examples that were misclassified by the previous model.

[0050] The core innovation of this invention is to establish dual models for the same group of data and to compare and interpret the models. The specific interpretation is achieved through the analysis of feature importance FIS and partial dependency graph PDP. FIS mainly determines the variables that are important for impact assessment, while PDP reflects the way in which they affect. When comparing, the FIS and key variable PDP of the academic citation model and the international impact model are mainly compared. FIS can achieve cross-model comparison by adding standardization in the calculation process, while PDP is mainly for intra-model comparison and cross-model trend comparison.

[0051] Eliminate features that appear flat or jitter irregularly in the PDP analysis, as these features do not contribute much to the model; for features with very steep PDP curves, it means that they have a greater impact on model predictions, and these features should be retained and further analyzed; Use PDP to show the average impact of a single feature on model predictions, and determine the approximate linear relationship, monotonic relationship, or more complex relationship between the feature and the prediction result;

[0052] For categorical features, PDP can be calculated for each category to estimate partial dependence.

[0053] Comparative analysis of FIS and key variable PDP for academic citation model and international impact model to identify key features of impact prediction in different models and their influence modes;

[0054] By standardizing FIS, cross-model comparison is achieved, while PDP is mainly used for comparison within a model and trend comparison across models.

[0055] According to the analysis results of FIS and PDP, the model is continuously optimized iteratively, useless features are eliminated, the model's ability to capture important features is enhanced, and the accuracy and interpretability of the model are improved.

[0056] Through these steps, the FIS and PDP analysis results can be effectively used to optimize model performance and improve the prediction accuracy and interpretability of the model.

[0057] The specific standardization methods are as follows:

[0058]

[0059] PDP is calculated as follows:

[0060]

[0061] Where X is the input data set, Xs is the variable data set to be analyzed, and Xc is the other variable data set, xs is a single instance in Xs, and xc is a single instance in Xc.

[0062] Analyses of individual variables included the following:

[0063]

[0064] Where n is the sample size.

[0065] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 2 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0066] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0067] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0068] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an information security risk warning device at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the aforementioned methods for constructing an impact analysis and assessment model.

[0069] The above application Figure 1 The method for constructing an impact analysis and evaluation model disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0070] The electronic device may also perform Figure 1 A method for constructing an impact analysis and evaluation model is proposed and implemented Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0071] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple applications, any one of the aforementioned methods for constructing an impact analysis and evaluation model is executed.

[0072] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0073] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0076] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0078] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0080] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing an impact analysis and evaluation model, characterized in that: The following steps are involved: A. Collect literature input variables; B uses Altmetrics and academic citations as output variables for modeling and analysis; C adds academic application volume and international influence proxy Altmetrics to the data set, performs 10-fold cross validation based on the inherited algorithm pool of the decision tree, evaluates the important performance of the algorithm on the data set in turn, and uses 10-fold cross validation to evaluate the benchmark performance of each algorithm on the data set in turn; D. Select the algorithm with the highest benchmark coefficient as the input algorithm for modeling and adjust the grid parameters for modeling; E uses the feature importance assessment model and compares the contribution of each variable to the impact of the outcome; F uses partial correlation diagrams to show the trend of the impact of each variable on the results, and combines the use of feature importance and partial correlation diagrams to comprehensively evaluate the impact of the results and output the results.

2. The method for constructing an impact analysis and evaluation model according to claim 1, characterized in that: The preprocessing methods include missing value processing, outlier detection and processing, and data type conversion.

3. The method for constructing an impact analysis and evaluation model according to claim 1, characterized in that: The literature input variables include the publication year, author, research institution, research field, abstract content, and keywords of the paper.

4. The method for constructing an impact analysis and evaluation model according to claim 1, characterized in that: The succession of decision tree based algorithms include Adaptive Boosting, Extremely Randomized Trees, Random Forest, and Gradient Boosting Machine.

5. The method for constructing an impact analysis and evaluation model according to claim 1, characterized in that: Perform 10-fold cross validation to evaluate the performance of the algorithm on the data set, including accuracy, recall, and F1 score indicators, to estimate whether the algorithm performs best among all algorithms. If not, return to select other algorithms and repeat the steps.

6. The method for constructing an impact analysis and evaluation model according to claim 1, characterized in that: The specific normalization equation for the partial dependence plot is calculated as follows: Where f is the algorithm, i is the sample instance, i is the variable, n is the number of samples, and j is the sample number.

7. The method for constructing an impact analysis and evaluation model according to claim 1, characterized in that: The partial dependence plot is calculated as follows: Where X is the input data set, Xs is the variable data set to be analyzed, and Xc is the other variable data set. xs is a single instance in Xs, and xc is a single instance in Xc. The analysis of a single variable includes the following: Where n is the sample size.

8. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 7.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute any one of the methods of claims 1 to 7.

Images