A dynamic evaluation method for journals based on multidimensional indicator analysis

Abstract

The present invention provides a method for dynamic journal evaluation based on multidimensional indicator analysis, comprising the steps of obtaining a journal indicator dataset, constructing a journal time series dataset, analyzing evaluation indicators, performing importance and correlation analysis on the journal evaluation indicators, constructing a journal indicator time series dataset, and constructing a dynamic journal evaluation prediction model based on the dataset. The present invention constructs a journal time series dataset, which has the characteristics of large data volume and long time span, making it well suited for analyzing the contribution of various indicators to journal evaluation. Its inherent time series characteristics can also help researchers dynamically analyze the development of journals. This method introduces time series data prediction into the journal evaluation scenario and obtains certain usable results through experiments.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and natural language processing, and in particular to a journal dynamic evaluation method based on multidimensional indicator analysis. Background Art

[0002] In recent years, artificial intelligence technology has continued to develop, producing numerous achievements in the fields of image, audio, text, etc. All walks of life are beginning to be or have already been influenced by artificial intelligence technology.

[0003] In terms of journal evaluation, there are two main evaluation methods for foreign academic journals: qualitative evaluation and quantitative evaluation.

[0004] Qualitative evaluation based on peer review makes the journal evaluation process subjective. Currently, the United States UTD24, Australia ABDC, the United Kingdom ABS and FT50, and France CNRS are relatively well-known peer-reviewed journal lists.

[0005] In quantitative evaluation, single indicators related to journal publishing behavior are the most important objective assessment methods. Common categories include citation metrics such as the Journal Impact Factor (JIF), the h-index, PageRank, and altmetrics. Currently, the first category is the most common, with key evaluation metrics including the Journal Impact Factor, the five-year impact factor, total citations, and the journal's citation half-life.

[0006] However, although natural language processing technology has been applied in the field of journal evaluation, such as journal research hotspot analysis, journal indicator evaluation, etc., although these applications have their own strengths, they still have problems such as narrow application of individual technologies, shallow text data mining, and lack of dynamic evaluation of multi-dimensional indicators. Therefore, journal evaluation based on manual sorting and single indicator sorting still has certain limitations. Summary of the Invention

[0007] The purpose of this invention is to provide a dynamic evaluation method for journals based on multidimensional indicator analysis, which can solve the limitations of single indicator ranking and evaluate the influence of journals by dynamically integrating multidimensional indicator data.

[0008] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0009] A journal dynamic evaluation method based on multidimensional indicator analysis includes the following steps:

[0010] (1) Collect, record, and sort journal indicators from the journal platform to obtain a journal indicator dataset;

[0011] (2) Preprocess the structured data and construct a journal time series dataset;

[0012] (3) Normalize the journal index data set and analyze the evaluation indicators using grey correlation analysis and regression analysis methods;

[0013] (4) Based on the journal indicator dataset, traditional machine learning, simple neural network, and deep neural network methods are selected for the regression task of journal evaluation direction to conduct importance and correlation analysis of journal evaluation indicators;

[0014] (5) Based on the journal time series data set and the evaluation index analysis results, a journal index time series data set is constructed. Based on this data set, a dynamic journal evaluation prediction model is constructed to conduct dynamic evaluation of scientific and technological journals.

[0015] Each journal indicator includes 3 types of journal overview attributes and 18 types of journal evaluation indicators. The journal overview attributes are journal name, journal type, and indicator recording year. The journal evaluation indicators are regional distribution number, funded paper ratio, average number of authors, average number of citations, citation half-life, extended H index, extended citation rate, extended annual index, extended discipline impact index, extended discipline diffusion index, extended number of citing journals, extended total citation frequency, extended citation half-life, document selection rate, institutional distribution number, source document volume, overseas paper ratio, and extended impact factor.

[0016] In order to support the time series prediction methods of both machine learning and deep learning, and to facilitate the comparison and weighting of indicators of different units or magnitudes, various indicators in the time series data set are converted from dimensional expressions to dimensionless expressions and normalized.

[0017] The method for constructing a journal time series dataset is as follows: complete the extraction of the feature matrix and dependent variables of the data, deal with missing data by deletion or calculation of the average value, and perform feature scaling on the data. In feature scaling, the journal indicators are in the same range, and the distribution of each indicator data remains consistent with that before feature scaling. The dataset is standardized, and the formula is as follows:

[0018]

[0019] Among them, x is the original value, mean is the mean, a is the standard deviation, and x' is the standardized value;

[0020] Finally, the dataset is divided into training set and test set.

[0021] The grey relational analysis method is:

[0022] Define n journal indicators as X′1, X′2…, X′ n , collect the literature index data of m journal samples over the years to obtain the following matrix:

[0023]

[0024] Determine the reference data column and use the averaging method to make the indicator data dimensionless:

[0025]

[0026] The dimensionless data matrix is ​​as follows:

[0027]

[0028] Calculate the absolute difference between the corresponding elements of the indicator column to be analyzed and the reference column one by one:

[0029] |x0(k)-x1(k)| (i=0,1,...,n; k=1,...,m)

[0030] Calculate the correlation coefficient, and calculate the correlation coefficient of the corresponding elements of each index column to be analyzed and the reference column respectively.

[0031]

[0032] Where ρ is the resolution coefficient, 0<ρ<1;

[0033] Find the correlation:

[0034]

[0035] Among them, R is the grey relational degree;

[0036] Finally, the correlation between each literature indicator is compared.

[0037] The regression analysis method is: select one or more machine learning methods and neural networks for the regression task of journal evaluation, and obtain the contribution of each journal indicator to the impact factor based on the model trained by the regression task.

[0038] The machine learning methods used are multiple linear regression, random forest and XGBoost;

[0039] (1) The implementation method of multiple linear regression is as follows: select the impact factor as the dependent variable and the other journal indicators as the independent variables. After standardizing the data, start the training of the multiple linear regression model and fit the following multiple linear function:

[0040]

[0041] in, is the predicted value of the impact factor of the i-th journal, θ0 is the function bias, θ n is the weight of the nth evaluation index of the journal, is the nth evaluation index value of the i-th journal;

[0042] Euclidean distance As the objective function, when the objective function is as small as possible, the multivariate linear function fitting is completed, and the training of the multivariate linear regression model is also completed. At this time, the weights of each evaluation index, namely θ, are obtained. n , which is used as the contribution of each journal evaluation indicator to the impact factor. Among these weights, positive numbers indicate positive correlation, and negative numbers indicate negative correlation. The contribution calculation formula is as follows:

[0043] importance i =θ i

[0044] Among them, importance i is the predicted contribution of the i-th journal evaluation index to the journal evaluation impact factor, θ i is the weight of the i-th journal evaluation index in the multiple linear regression model;

[0045] (2) The implementation of random forest is as follows: CART decision tree is used as a weak learner. When generating each tree, each tree uses bootstrap sampling to randomly collect a random sub-data set from the training set, and randomly selects a small number of journal indicators as the input of the decision tree, and the number of journal indicators selected is the square root of the total number of indicators. During the training process, each tree in the random forest algorithm generates a root node and determines whether the number of training samples under the node is less than the predetermined threshold or the node impurity is less than the predetermined threshold. If it is satisfied, the generation is stopped. If not, the selected journal evaluation indicators and their values ​​are traversed and used as the splitting variables and splitting points respectively, and the weighted sum of the absolute average errors of each child node after the split A(x i ,v ij ) to judge the quality of the segmentation, select the best segmentation effect as the segmentation variable and segmentation point of the node, and generate new left and right subtrees based on the segmentation variable and segmentation point, and perform a new round of division on the left and right word counts or stop generating. A(x i ,v ij ) is calculated as follows:

[0046]

[0047] Among them, x i is a segmentation variable, i.e., journal evaluation index, v ij is a cutoff value of the cutoff variable, n left 、n right , N S are the number of training samples of the left child node after segmentation, the number of training samples of the right child node, and the number of all training samples of the current node, respectively.left 、X right are the training sample sets of the left and right child nodes respectively, H(X) is the function to measure the node impurity, and is calculated using the absolute mean error H(Ω) in the regression task. The formula is as follows:

[0048]

[0049] Among them, Ω is the sample set on the node, N is the total number of samples in the sample set Ω, is the average value of the impact factor of the training sample of the current node, y i is the impact factor value of the i-th sample in the sample set;

[0050] When a regression decision tree generates leaf nodes, the tree training is completed. Next, the method provided by sklearn will be used to rank the contribution of journal evaluation indicators to journal impact factors. First, for a node k, its importance is calculated as follows:

[0051] I k =w k ×A k -w left ×A left -w right ×A right

[0052] Among them, w k , w left , w right are the ratio of the number of training samples in node k and its left and right child nodes to the total number of training samples, A k , A left , A right are the impurities of node k and its left and right child nodes respectively. After the importance of a certain node, the importance of a certain journal evaluation indicator is obtained by the following formula.

[0053]

[0054] Among them, Ψ is the node with journal evaluation index i as the segmentation variable, and all nodes refers to all nodes in the random forest;

[0055] (3) The implementation of XGBoost is as follows: Since both XGBoost and Random Forest use CART as weak classifiers, Random Forest averages the independent prediction scores of each weak classifier as the prediction score of the strong classifier, while XGBoost adds the prediction scores of each weak classifier as the prediction score of the strong classifier, so the influencing factor during training is y i The true value of the impact factor of the nth tree in The prediction result of journal impact factor for the n-1th tree;

[0056] For the i-th journal evaluation index, the contribution calculation formula is as follows:

[0057]

[0058] Among them, c ti is the number of times the i-th evaluation index is used as a segmentation variable on the t-th tree, c t is the summary point of the t-th tree, and Φ is the classification and regression tree with the i-th evaluation index as the segmentation variable.

[0059] Neural networks are used for regression tasks in the journal evaluation direction using simple neural networks and deep neural networks;

[0060] During the model training phase, the simple neural network uses the journal impact factor as the true value and other journal evaluation indicators as the model's input feature matrix. The simple neural network model structure has four hidden layers: one input layer and one output layer. The number of neurons in each hidden layer is 200, 100, 50, and 25, respectively. The number of neurons in the last output layer is 1, and no softmax layer is required. The model is optimized using adaptive moment estimation. After model training, feature importance is calculated based on input perturbation, correlation coefficient, weight analysis, or a combination of the first three methods.

[0061] (1) The formula for calculating feature importance based on input perturbation is:

[0062]

[0063] Among them, importance i is the contribution of the i-th journal evaluation index to the prediction of the journal evaluation impact factor, u is the total number of journal evaluation indicators, MAE i is the absolute average error of the i-th journal evaluation index after perturbation;

[0064] (2) To calculate feature importance based on correlation coefficient, you need to first calculate the Pearson correlation coefficient, which is calculated as follows:

[0065]

[0066] Among them, correlation is the Pearson correlation coefficient, q is the total number of journals, and x it is the value of the tth journal indicator in the ith journal, y i is the value of the journal impact factor in the i-th journal, is the average value of the t-th journal index of all journals, is the average value of the journal impact factors of all journals. After calculating the Pearson correlation coefficient between all journal evaluation indicators and journal impact factors, the contribution of journal evaluation indicators to the prediction of journal impact factors is obtained by the following formula:

[0067]

[0068] Among them, correlation i is the Pearson correlation coefficient between the i-th journal evaluation index and the journal impact factor;

[0069] (3) The method of calculating feature importance based on weight analysis is:

[0070]

[0071] Among them, weight i is the total weight of the i-th journal evaluation index to the hidden layer;

[0072] (4) The hybrid method proposes a parameter d:

[0073]

[0074] Among them, m is the total number of journal evaluation indicators, I i Calculate the contribution of the i-th journal evaluation index to the prediction of the journal impact factor under the input perturbation algorithm, Calculate the mean contribution of all journal evaluation indicators to the prediction of journal impact factors under the input perturbation algorithm; the final contribution importance i The calculation method is:

[0075] importance i =weight_im i +d*inputpertubation_im i +(1-d)*correlation_im i

[0076] Among them, weight_im i Calculate the contribution of the i-th journal evaluation index to the journal impact factor prediction under the weight analysis algorithm, inputpertubation_im i Calculate the contribution of the i-th journal evaluation index to the prediction of the journal impact factor under the input perturbation algorithm, correlation_im i Calculate the contribution of the i-th journal evaluation index to the prediction of the journal impact factor under the correlation coefficient algorithm;

[0077] During the training phase of the deep neural network, the number of neurons in the hidden layer was 50, 1024, 2048, 4096, 2048, 1024, and 50, respectively. The number of neurons in the final output layer was 1, and no softmax activation function was required. Each fully connected layer was followed by a batch normalization layer, and dropout was implemented in all layers. The model optimizer used a stochastic gradient descent optimizer. After model training, the VIANN and Garson algorithms were used to obtain and rank the contribution of each journal evaluation indicator to the journal evaluation impact factor.

[0078] The VIANN algorithm updates the current motion variance at the end of each iteration, and first calculates the motion variance VAR of the weight after the nth iteration. n and mean M n , the calculation formula is as follows:

[0079]

[0080]

[0081] Among them, V n-1 and M n-1 are the sum of squares and the average of all updated values ​​of a weight after the n-1th iteration, x n is the weight updated after the nth iteration; after all iterations are completed, all weights between the input layer and the first hidden layer after the last update are obtained to calculate the contribution of each journal evaluation indicator to the journal impact factor prediction. The calculation formula is as follows:

[0082]

[0083] Among them, T is the first hidden layer neuron connected to the i-th input journal evaluation index, VAR it is the variance of the connection weight between the i-th journal evaluation index and the t-th neuron, last_weight it is the value of the connection weight between the i-th journal evaluation index and the t-th neuron after the last update;

[0084] The Garson algorithm requires the weight matrix W1 between the input layer and the first hidden layer in the deep neural network, and the weight matrix W2 between the last hidden layer and the output layer. The following formula can be used to calculate the vector of the contribution of journal evaluation indicators to the prediction of journal impact factors:

[0085] importance = stu(W1×W2)

[0086] Among them, stu() is the normalization function.

[0087] The dynamic journal evaluation prediction model is constructed by selecting machine learning and deep learning methods, including: (1) processing the journal indicator time series data set into a one-dimensional input vector and a single target supervised learning form to train the prediction model constructed by the machine learning method, and inputting the data of the data set for training; (2) processing the journal indicator time series data set into a two-dimensional vector of the size of time step * eigenvalue number and a single target to train the prediction model constructed by the deep learning method, normalizing the data and inputting it into the model for training;

[0088] The machine learning methods include multivariate linear regression, random forest, XGBoost, and LightGBM, and the deep learning methods include LSTM, GRU, Conv-1D, WaveNet, and ANN.

[0089] Furthermore, the journal dynamic evaluation method based on multidimensional indicator analysis disclosed in this scheme also includes the steps of generating keywords based on the journal indicator data set, visualizing the indicators, and constructing a portrait of the scientific and technological journal.

[0090] The method for constructing a portrait of a scientific journal is as follows: using part-of-speech tags to define the grammar of subject terms, setting nouns and their single variations as the grammatical rules of the required keywords, analyzing the subject sentences to obtain the grammatical tree corresponding to the subject sentences, and then segmenting the subject sentences to extract the keywords as the subject terms of the paper; using the keyword cloud method to standardize the journal's own information and create a portrait.

[0091] The advantages of the present invention are:

[0092] This paper constructs a journal time series dataset, which has the characteristics of large data volume and long time span, making it very suitable for analyzing the contribution of various indicators to journal evaluation. Its inherent time series characteristics can also help relevant researchers dynamically analyze the development of journals. It introduces time series data prediction into the journal evaluation scenario and obtains certain usable results through experiments.

[0093] The grey correlation analysis method adopted in this paper can perform strength and weakness analysis and correlation analysis between different indicators. It plans to use the state space equation based on time-varying parameters to analyze the changes in the strength of the relationship between evaluation indicators from a dynamic perspective. The XGBoost algorithm uses each indicator as a parameter input, and obtains the importance of the parameter as the contribution of the indicator through parameter adjustment, from which the weight is obtained, and the importance of different evaluation indicators is compared, which greatly improves the calculation speed, simplifies the model, and avoids overfitting. It confirms that a total of 9 mainstream machine learning and deep learning methods in the field of time series data analysis have certain generalization properties in the comprehensive dynamic evaluation task in the journal evaluation scenario, laying a foundation for the subsequent algorithm optimization of this task and playing a certain direction guidance role.

[0094] The present invention extracts journal keywords through text analysis technology, and visualizes them to generate a keyword cloud. At the same time, the evaluation attributes of the journal are clearly displayed. It can intuitively display the development dynamics of the journal, grasp the changes in the journal structure, understand the competitive situation of the circular and the journal market demand, and thus provide data support for the planning of academic journal columns. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 This is a flowchart of the journal dynamic evaluation method based on multidimensional indicator analysis;

[0096] Figure 2 This is a flow chart for obtaining the contribution of evaluation indicators to journal impact factors using a dynamic journal evaluation method based on multidimensional indicator analysis. DETAILED DESCRIPTION

[0097] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0098] This embodiment discloses a journal dynamic evaluation method based on multi-dimensional indicator analysis, comprising the steps of:

[0099] (1) Based on the journal index datasets of HowNet and Wanfang, 18 evaluation indicators were collected for each journal to obtain the journal index dataset;

[0100] (2) Preprocess the structured data by performing feature matrix and dependent variable differentiation to construct a journal time series dataset;

[0101] (3) Based on the journal indicator dataset, artificial intelligence and natural language processing technologies are used to generate subject terms, visualize the indicators, and construct a portrait of scientific journals;

[0102] (4) Perform dimensionless normalization on the journal index data set, and use grey correlation analysis and regression analysis to analyze the evaluation indicators;

[0103] (5) Based on the journal indicator dataset, traditional machine learning, simple neural network, and deep neural network methods are selected for the regression task of journal evaluation direction to conduct importance and correlation analysis of journal evaluation indicators;

[0104] (6) Based on the analysis results of the journal time series data set and evaluation indicators, a journal indicator time series data set was constructed. Based on this data set, a dynamic journal evaluation prediction model was constructed using 9 methods including machine learning and deep learning to conduct dynamic evaluation of scientific and technological journals.

[0105] The technical details of the above steps are as follows, please refer to Figure 1 For comparison:

[0106] Step 1: Build a journal time series dataset

[0107] The dataset used in this example was compiled by collecting historical journal indicator data from the China Wanfang Data Knowledge Service Platform, a renowned Chinese academic database covering journals, conference proceedings, papers, and academic research findings. The journal time series dataset contains 27,125 journal indicator records, each composed of 21 categories of elements. To ensure diversity, this example collected journal indicator records across 12 fields. The dataset includes 5,425 journals, each with 21 indicator records for each year from 2017 to 2021.

[0108] Each journal indicator includes 3 types of journal overview attributes and 18 types of journal evaluation indicators. The journal overview attributes are journal name, journal type, and indicator recording year. The journal evaluation indicators are regional distribution number, funded paper ratio, average number of authors, average number of citations, citation half-life, extended H index, extended citation rate, extended annual index, extended discipline impact index, extended discipline diffusion index, extended number of citing journals, extended total citation frequency, extended citation half-life, document selection rate, institutional distribution number, source document volume, overseas paper ratio, and extended impact factor.

[0109] A large number of journal indicator records were collected by year and recorded using Excel. Journals containing five consecutive years of records were then selected based on journal name and unique identifiers. These records were then spliced ​​together by journal name and sorted by journal name and record year. Finally, the spliced ​​data was cleaned by operations such as missing value filling and text conversion. For missing values, the following two methods were used to handle missing data: the first was to count the specific rows of missing data in the statistical dataset and then delete these specific rows. Since the sample size of the dataset was large enough, deleting these rows did not affect the accuracy of the experiment, and a relatively accurate answer was still obtained. The second method handled missing data by calculating the average value. If all five years were missing, the data was directly filled with 0. For text, it was converted into numerical elements as model input, ultimately forming the journal time series dataset for this article.

[0110] The time series dataset contains multiple multivariate, short-term time series. Each journal in the dataset contains annual metric records for the past five years, resulting in 5,425 time series samples with five time steps and 18 dimensions, enabling experiments using various time series forecasting methods. To support both machine learning and deep learning time series forecasting methods and facilitate comparison and weighting of metrics of different units or magnitudes, various metrics in the time series dataset were normalized from dimensional expressions to dimensionless expressions.

[0111] The method for constructing the journal time series dataset is as follows: the iloc method in the pandas library is used to complete the extraction of the feature matrix and dependent variables, complete the missing processing of structured data, and perform feature scaling on the data. This is particularly important in multiple linear regression, but this step is not required in random forest and XGBoost. In feature scaling, the 17 indicators are all in the same range, and the distribution of each indicator data is consistent with that before feature scaling. This ensures that when performing Euclidean distance calculations in multiple linear regression experiments, while keeping the characteristics of each indicator unaffected, there will be no problem of one variable dominating other variables between indicators. The data set is standardized, and the formula is as follows:

[0112]

[0113] Where x is the original value, mean is the mean, a is the standard deviation, and x' is the normalized value. Finally, the dataset is divided into a training set and a test set. 70% of the dataset is used as the training set, and the remaining 30% is used as the test set.

[0114] Conduct correlation analysis on indicators and apply the most widely used analysis method in grey theory - grey correlation analysis. Grey correlation analysis is a method for quantitative analysis of the development and changes of system trends. In essence, it compares the degree of proximity between the evaluation object and a specific object. The closer the two are, the greater the correlation between the evaluation objects. The correlation degree is a measure of the degree of correlation between the indicators of two objects. It can quantitatively describe the relative changes between objects. It mainly solves the problems of which factors are primary and secondary, which factors have a greater impact and which factors have a smaller impact in a system containing multiple factors. In view of the characteristics of small samples of journal indicator data, grey correlation analysis can overcome the shortcomings of large samples and large amount of calculation required by traditional mathematical analysis, and can be better applied to dynamic process analysis.

[0115] Taking the relationship analysis of the impact factor and other indicators as an example, the variable symbols are explained as follows: the impact factor is defined as X′0, the total citation frequency is defined as X′1, the annual index is defined as X′2, the citation rate is defined as X′3, the number of citing journals is defined as X′4, the open factor is defined as X′5, the diffusion factor is defined as X′6, the authority factor is defined as X′7, the cited half-life is defined as X′8, etc.

[0116] The following matrix is ​​obtained by collecting the literature index data of journals over the years:

[0117]

[0118] Where m is the number of journal samples collected, n is the number of literature indicators, the impact factor is determined as the reference data column, and the indicator data is dimensionless using the averaging method. There are many methods for dimensionless processing. This invention uses averaging processing:

[0119]

[0120] The dimensionless data matrix is ​​as follows:

[0121]

[0122] Calculate the absolute difference between the corresponding elements of the indicator column to be analyzed and the reference column one by one:

[0123] |x0(k)-x1(k)| (i=0, 1,..., n; k=1,..., m)

[0124] Calculate the correlation coefficient, and calculate the correlation coefficient of the corresponding elements of each index column to be analyzed and the reference column respectively.

[0125]

[0126] Where ρ is the discrimination coefficient, 0<ρ<1; the smaller ρ is, the greater the difference between the correlation coefficients and the stronger the discrimination ability, and ρ is usually taken as 0.5.

[0127] Find the correlation:

[0128]

[0129] Among them, R is the grey correlation degree. The grey correlation degree value reflects the degree of correlation between the relevant influencing factors and the influencing factors. The larger the value, the greater the degree of influence of the relevant evaluation indicators on the influencing factors.

[0130] Finally, the correlation between each literature indicator is compared.

[0131] This paper selects three machine learning methods, namely multivariate linear regression, random forest and XGBoost, as well as simple neural network and deep neural network methods for the regression task of journal evaluation, and obtains the contribution of each index of the journal to the impact factor based on the model trained by the regression task. Please refer to the process Figure 2 .

[0132] (1) The implementation method of multiple linear regression is as follows: take the impact factor of each journal as the dependent variable and the other 17 journal indicators as the independent variables. After standardizing the data, start the training of the multiple linear regression model and fit the following multiple linear function:

[0133]

[0134] in, is the predicted value of the impact factor of the i-th journal, θ0 is the function bias, θ n is the weight of the nth evaluation index of the journal, is the nth evaluation index value of the i-th journal;

[0135] Euclidean distance As the objective function, when the objective function is as small as possible, the multivariate linear function fitting is completed, and the training of the multivariate linear regression model is also completed. At this time, the weights of each evaluation index, namely θ, are obtained. n , which is used as the contribution of each journal evaluation indicator to the impact factor. Among these weights, positive numbers indicate positive correlation, and negative numbers indicate negative correlation. The contribution calculation formula is as follows:

[0136] importance i =θ i

[0137] Among them, importance i is the predicted contribution of the i-th journal evaluation index to the journal evaluation impact factor, θ i is the weight of the i-th journal evaluation index in the multiple linear regression model.

[0138] (2) The implementation of random forest is as follows: CART decision tree is used as a weak learner. When generating each tree, each tree uses bootstrap sampling to randomly collect a random sub-data set from the training set, and randomly selects a small number of journal indicators as the input of the decision tree. The number of journal indicators selected is the square root of the total number of indicators, which is 4. This ensures the randomness of the features. At this time, the input data does not need to be standardized. During the training process, each tree in the random forest algorithm generates a root node to determine whether the number of training samples under the node is less than a predetermined threshold or the node impurity is less than a predetermined threshold. If it is satisfied, the generation is stopped. If not, the selected journal evaluation indicators and their values ​​are traversed and used as splitting variables and splitting points respectively, and the weighted sum of the absolute average errors of each child node after splitting A(x i , v ij ) to judge the quality of the segmentation, select the best segmentation effect as the segmentation variable and segmentation point of the node, and generate new left and right subtrees based on the segmentation variable and segmentation point, and perform a new round of division on the left and right word counts or stop generating. A(x i , v ij ) is calculated as follows:

[0139]

[0140] Among them, x i is a segmentation variable, i.e., journal evaluation index, v ijis a cutoff value of the cutoff variable, n left 、n right , N S are the number of training samples of the left child node after segmentation, the number of training samples of the right child node, and the number of all training samples of the current node, respectively. left 、X right are the training sample sets of the left and right child nodes respectively, H(X) is the function to measure the node impurity, and is calculated using the absolute mean error H(Ω) in the regression task. The formula is as follows:

[0141]

[0142] Among them, Ω is the sample set on the node, N is the total number of samples in the sample set Ω, is the average value of the impact factor of the training sample of the current node, y i is the impact factor value of the i-th sample in the sample set;

[0143] When a regression decision tree generates a leaf node, the tree training is completed. Next, the method provided by skleam will be used to rank the contribution of journal evaluation indicators to journal impact factors. First, for a node k, its importance is calculated as follows:

[0144] I k =w k ×A k -w left ×A left -w right ×A rig

[0145] Among them, w k , w left , w right are the ratio of the number of training samples in node k and its left and right child nodes to the total number of training samples, A k , A left , A right are the impurities of node k and its left and right child nodes respectively. After the importance of a certain node, the importance of a certain journal evaluation indicator is obtained by the following formula.

[0146]

[0147] Among them, Ψ is the node with journal evaluation index i as the segmentation variable, and all nodes refers to all nodes in the random forest.

[0148] (3) The implementation of XGBoost is as follows: Since both XGBoost and Random Forest use CART as weak classifiers, Random Forest averages the independent prediction scores of each weak classifier as the prediction score of the strong classifier, while XGBoost adds the prediction scores of each weak classifier as the prediction score of the strong classifier, so the influencing factor during training is y i The true value of the impact factor of the nth tree in The prediction result of journal impact factor for the n-1th tree;

[0149] For the i-th journal evaluation index, the contribution calculation formula is as follows:

[0150]

[0151] Among them, c ti is the number of times the i-th evaluation index is used as a segmentation variable on the t-th tree, c t is the summary point of the t-th tree, and Φ is the classification and regression tree with the i-th evaluation index as the segmentation variable.

[0152] According to the above method, the top 5 contribution rankings are as follows:

[0153] Methods Rank 1 Rank 2 Rank3 Rank 4 Rank 5 XGBoost Extended H-Index Current year indicators Number of source documents Number of Institutional Distribution Average number of citations Random Forest Extended H-Index Current year indicators Average number of citations Number of source documents Number of Institutional Distribution Linear Regression Current year indicators Extended H-Index Discipline Impact Indicators Number of source documents Ratio of funded papers

[0154] Neural networks are used for regression tasks in journal evaluation using simple neural networks and deep neural networks:

[0155] (1) Simple Neural Network

[0156] During model training, we used the journal impact factor as ground truth and the other 17 journal evaluation metrics as the model's input feature matrix. The model structure consists of four hidden layers: one input layer and one output layer. The number of neurons in each hidden layer is 200, 100, 50, and 25, respectively. The last output layer has 1 neuron and does not require a softmax layer, as this is a regression task that only outputs the predicted impact factor. The model is optimized using Adaptive Moment Estimation (ADAM), an optimization algorithm that introduces momentum and adaptively changes the learning rate, effectively preventing problems such as gradient oscillation and sparse gradients. The model's initial learning rate is set to 0.01, and the batch size is set to 32. As recommended by the authors, 75% of the training set is used for training and 25% for validation.

[0157] After completing the model training, we used the following four methods to obtain the contribution of each journal evaluation indicator to the journal evaluation impact factor and ranked the top 5:

[0158] ① Calculate feature importance based on input perturbations

[0159]

[0160] Among them, importance i is the contribution of the i-th journal evaluation index to the prediction of the journal evaluation impact factor, u is the total number of journal evaluation indicators, MAE i is the absolute mean error of the i-th journal evaluation index after perturbation.

[0161] ②Calculate feature importance based on correlation coefficient

[0162] The Pearson correlation coefficient must be calculated first, and the calculation formula is as follows:

[0163]

[0164] Among them, correlation is the Pearson correlation coefficient, q is the total number of journals, and x it is the value of the tth journal indicator in the ith journal, y i is the value of the journal impact factor in the i-th journal, is the average value of the t-th journal index of all journals, is the average value of the journal impact factors of all journals. After calculating the Pearson correlation coefficient between all journal evaluation indicators and journal impact factors, the contribution of journal evaluation indicators to the prediction of journal impact factors is obtained by the following formula:

[0165]

[0166] Among them, correlation i is the Pearson correlation coefficient between the i-th journal evaluation index and the journal impact factor.

[0167] ③Calculate feature importance based on weight analysis:

[0168]

[0169] Among them, weight i is the total weight of the i-th journal evaluation index to the hidden layer.

[0170] ④ Hybrid method

[0171] Some of the first three methods rely only on the dataset or only on the trained model. Therefore, a hybrid method is implemented here, combining the first three methods to propose a parameter d:

[0172]

[0173] Among them, m is the total number of journal evaluation indicators, I i Calculate the contribution of the i-th journal evaluation index to the prediction of the journal impact factor under the input perturbation algorithm, Calculate the mean contribution of all journal evaluation indicators to the prediction of journal impact factors under the input perturbation algorithm; the final contribution importance i The calculation method is:

[0174] importance i =weight_im i +d*inputpertubation_im i +(1-d)*correlation_im i

[0175] Among them, weight_im i Calculate the contribution of the i-th journal evaluation index to the journal impact factor prediction under the weight analysis algorithm, inputpertubation_im i Calculate the contribution of the i-th journal evaluation index to the prediction of the journal impact factor under the input perturbation algorithm, correlation_im i Calculate the contribution of the i-th journal evaluation index to the prediction of the journal impact factor under the correlation coefficient algorithm.

[0176] The top 5 contributions of the simple neural network method are:

[0177]

[0178] (2) Deep Neural Networks

[0179] During model training, the number of neurons in the hidden layer was 50, 1024, 2048, 4096, 2048, 1024, and 50, respectively. The number of neurons in the final output layer was 1, and no softmax activation function was required. Each fully connected layer was followed by a batch normalization layer, and dropout was implemented in all layers. The model optimizer used a stochastic gradient descent optimizer. After model training, the VIANN and Garson algorithms were used to obtain and rank the contribution of each journal evaluation indicator to the journal evaluation impact factor.

[0180] ①VIANN algorithm

[0181] At the end of each iteration, when the weight is updated, the current motion variance is updated. First, the motion variance VAR of the weight after the nth iteration is calculated. n and mean M n , the calculation formula is as follows:

[0182]

[0183]

[0184] Among them, V n-1 and M n-1 are the sum of squares and the average of all updated values ​​of a weight after the n-1th iteration, x n is the weight updated after the nth iteration; after all iterations are completed, all weights between the input layer and the first hidden layer after the last update are obtained to calculate the contribution of each journal evaluation indicator to the journal impact factor prediction. The calculation formula is as follows:

[0185]

[0186] Among them, T is the first hidden layer neuron connected to the i-th input journal evaluation index, VAR it is the variance of the connection weight between the i-th journal evaluation index and the t-th neuron, last_weight it is the value of the connection weight between the i-th journal evaluation index and the t-th neuron after the last update;

[0187] ②Garson algorithm

[0188] It is necessary to obtain the weight matrix W1 between the input layer and the first hidden layer and the weight matrix W2 between the last hidden layer and the output layer in the deep neural network. The vector consisting of the contribution of journal evaluation indicators to the prediction of journal impact factors can be calculated according to the following formula:

[0189] importance = stu(W1×W2)

[0190] Among them, stu() is the normalization function.

[0191] The top 5 contributions of deep neural network methods are:

[0192] Methods Rank 1 Rank 2 Rank3 Rank 4 Rank 5 VIANN Cited half-life Current year indicators Extended H-Index Expanded citation rate Literature selection rate Garson Current year indicators Cited half-life Extended H-Index Expanded citation rate Literature selection rate

[0193] Since the contribution ranking of journal impact factors by input perturbation in a simple neural network is significantly different from that of other methods, the Input Perturbation method was used again in a deeper neural network to recalculate the contribution and compare it with the results before perturbation. The mean square error (MSE) and mean absolute error (MAE) were calculated. The comparison of the loss of journal impact factor prediction before and after perturbation of journal evaluation indicators is shown in the following table:

[0194] Factor Mse Mae Null 0.2993 0.2941 Regional distribution 0.43305 0.534394 Ratio of funded papers 0.43330 0.534522 Average number of authors 0.43288 0.534569 Average number of citations 0.43093 0.533971 Citation half-life 0.43286 0.534738 Extended H-Index 0.42829 0.533240 Expanded citation rate 0.43302 0.534505 Current year indicators 0.43288 0.534471 Discipline Impact Indicators 0.43304 0.534521 Discipline Diffusion Index 0.43256 0.534354 Extended citation count 0.42343 0.534203 Total citations 0.43234 0.553242 Cited half-life 0.43291 0.534099 Literature selection rate 0.43302 0.534546 Number of Institutional Distribution 0.43838 0.541738 Number of source documents 0.43440 0.555592 Overseas papers ratio 0.43311 0.534530

[0195] The dynamic journal evaluation prediction model is constructed by selecting machine learning and deep learning methods, including: (1) processing the journal indicator time series data set into a one-dimensional input vector and a single target supervised learning form to train the prediction model constructed by the machine learning method, and inputting the data of the data set for training; (2) processing the journal indicator time series data set into a two-dimensional vector of the size of time step * eigenvalue number and a single target to train the prediction model constructed by the deep learning method, normalizing the data and inputting it into the model for training;

[0196] The machine learning methods include multivariate linear regression, random forest, XGBoost, and LightGBM, and the deep learning methods include LSTM, GRU, Conv-1D, WaveNet, and ANN.

[0197] The prediction results of the dynamic journal evaluation prediction model constructed by the nine methods are shown in the following table:

[0198]

[0199] Example 2

[0200] Based on Example 1, the present invention also discloses a specific implementation method for constructing an evaluation model for scientific journal portraits, including three parts: data collection, data processing, and visualization. The construction of journal portraits requires the selection of data sources that are reasonably sized and easy to extract, and can accurately display the characteristics and dynamic development of journal attributes. In order to match the journal model construction method, the research process requires processing the collected data, including normalization and data cleaning, and finally visualizing the journal portrait.

[0201] (1) Journal portrait composition information

[0202] The journal portrait is a vivid and three-dimensional journal model that is extracted from information highly relevant to the journal and reflects the journal's attributes. It constructs and combines the entity information and abstract information in the attribute characteristics. By combining the journal's own information with quantitative evaluation, a comprehensive evaluation of academic journals is carried out. The journal portrait is divided into two parts: the entity journal portrait and the abstract journal portrait. The entity journal portrait mainly includes the information of the journal's core papers, and the abstract journal portrait mainly includes the journal's historical indicator data, as shown in the following table:

[0203]

[0204] (2) Generation of keywords

[0205] For the text data of relevant journals, the topic sentences of journal articles need to be processed into keyword terms. Since the abstract text is too long and it is difficult to determine the core expression, it is possible to consider using keyword terms instead of abstracts. The "HightLight" part of the topic sentence of the paper is mainly the author's description of the core part of his or her paper. It is often also the expression data of the core points and has important research significance. First, the topic word block grammar is defined using part-of-speech tags, and nouns and their single deformations (singular noun blocks with a single modification or deformations of the noun blocks themselves) are set as the grammatical rules of the required keywords. Then, the topic sentence is analyzed to obtain the grammatical tree corresponding to the topic sentence, and then the topic sentence is divided into blocks to extract the keywords as the topic words of the paper. See the table below for examples:

[0206]

[0207] (3) Visualization of keywords and indicators

[0208] Based on the data collection and processing methods described above, we used a keyword cloud approach to standardize and profile journal information. The collected journal indicator data and corresponding years were used as components of a journal profile to construct a profile for each academic journal. Using drawing tools, we visualized the data in the table file.

[0209] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A journal dynamic evaluation method based on multidimensional indicator analysis, characterized by: Including steps: (1) Collect, record, and sort journal indicators from the journal platform to obtain a journal indicator dataset; (2) Preprocess the structured data and construct a journal time series dataset; (3) Normalize the journal index data set and analyze the evaluation indicators using grey correlation analysis and regression analysis methods; (4) Based on the journal indicator dataset, traditional machine learning, simple neural network, and deep neural network methods are selected for the regression task of journal evaluation, and the importance and correlation analysis of journal evaluation indicators are performed; (5) Based on the journal time series data set and the evaluation index analysis results, a journal index time series data set is constructed. Based on this data set, a dynamic journal evaluation prediction model is constructed to conduct dynamic evaluation of scientific and technological journals; (6) Generate subject words based on journal indicator datasets, visualize indicators, and construct a portrait of scientific journals; The method for constructing a scientific journal portrait is as follows: using part-of-speech tags to define the grammar of subject words, setting nouns and their single variations as the grammatical rules of the required keywords, analyzing the subject sentences to obtain the grammatical trees corresponding to the subject sentences, and then segmenting the subject sentences to extract the keywords therein as the subject words of the paper; using the keyword cloud method to standardize the journal's own information and create a portrait; The journal indicators include 3 types of journal overview attributes and 18 types of journal evaluation indicators. The journal overview attributes are journal name, journal type, and indicator recording year. The journal evaluation indicators are regional distribution number, funded paper ratio, average number of authors, average number of citations, citation half-life, extended H index, extended citation rate, extended annual index, extended discipline impact index, extended discipline diffusion index, extended number of citing journals, extended total citation frequency, extended citation half-life, document selection rate, institutional distribution number, source document volume, overseas paper ratio, and extended impact factor. The dynamic journal evaluation prediction model is constructed by selecting machine learning and deep learning methods, including: (1) processing the journal indicator time series data set into a supervised learning form of a one-dimensional input vector and a single target to train the prediction model constructed by the machine learning method, and inputting the data of the data set for training; (2) processing the journal indicator time series data set into a form of a two-dimensional vector of the size of time step*characteristic value number and a single target to train the prediction model constructed by the deep learning method, normalizing the data and inputting it into the model for training; The machine learning methods include multiple linear regression, random forest, XGBoost, and LightGBM, and the deep learning methods include LSTM, GRU, Conv-1D, WaveNet, and ANN.

2. The journal dynamic evaluation method based on multidimensional indicator analysis according to claim 1 is characterized in that: The normalization process is to convert various indicators from dimensional expressions into dimensionless expressions.

3. The method for dynamic evaluation of journals based on multidimensional indicator analysis according to claim 1 is characterized in that: The method for constructing a journal time series dataset is as follows: extracting the feature matrix and dependent variables from the data, processing missing data by deletion or filling in the average value, and scaling the data. In the scaling, the journal indicators are all in the same range, and the distribution of each indicator data is consistent with that before scaling. The dataset is standardized, and the formula is as follows: in, is the original value, is the average value, is the standard deviation, is the standardized value; Finally, the dataset is divided into training set and test set.

4. The method for dynamic evaluation of journals based on multidimensional indicator analysis according to claim 1 is characterized in that: The grey relational analysis method is: Define n journal indicators as , collect the literature index data of m journal samples over the years to obtain the following matrix: Determine the reference data column and use the averaging method to make the indicator data dimensionless: The dimensionless data matrix is ​​as follows: Calculate the absolute difference between the corresponding elements of the indicator column to be analyzed and the reference column one by one: Calculate the correlation coefficient, and calculate the correlation coefficient of the corresponding elements of each index column to be analyzed and the reference column respectively; Where ρ is the resolution coefficient, 0<ρ<1; Find the correlation: Among them, R is the grey relational degree; Finally, the correlation between each literature indicator is compared.

5. The method for dynamic evaluation of journals based on multidimensional indicator analysis according to claim 1 is characterized in that: The regression analysis method is as follows: one or more machine learning methods and neural networks are selected for the regression task of journal evaluation, and the contribution of each journal indicator to the impact factor is obtained based on the model obtained by regression task training.

6. The method for dynamic evaluation of journals based on multidimensional indicator analysis according to claim 5 is characterized in that: The machine learning methods used are multiple linear regression, random forest and XGBoost; (1) The implementation method of multiple linear regression is as follows: select the impact factor as the dependent variable and the other journal indicators as independent variables. After standardizing the data, start the training of the multiple linear regression model and fit the following multiple linear function: in, is the predicted value of the impact factor of the i-th journal, is the function bias, is the weight of the nth evaluation index of the journal, is the nth evaluation index value of the i-th journal; Euclidean distance As the objective function, when the objective function is as small as possible, the multivariate linear function fitting is completed, and the training of the multivariate linear regression model is also completed. At this time, the weights of each evaluation index are obtained, that is, , which is used as the contribution of each journal evaluation indicator to the impact factor. Among these weights, positive numbers indicate positive correlation, and negative numbers indicate negative correlation. The contribution calculation formula is as follows: in, is the predicted contribution of the i-th journal evaluation index to the journal evaluation impact factor, is the weight of the i-th journal evaluation index in the multiple linear regression model; (2) The implementation method of random forest is as follows: CART decision tree is used as a weak learner. When generating each tree, each tree uses bootstrap sampling to randomly collect a random sub-data set from the training set, and randomly selects a small number of journal indicators as the input of the decision tree, and the number of journal indicators selected is the square root of the total number of indicators. During the training process, each tree in the random forest algorithm generates a root node to determine whether the number of training samples under the node is less than the predetermined threshold or the node impurity is less than the predetermined threshold. If it is satisfied, the generation is stopped. If not, the selected journal evaluation indicators and their values ​​are traversed and used as the splitting variables and splitting points respectively, and the weighted sum of the absolute average errors of each sub-node after the split is obtained. To judge the quality of the segmentation, select the one with the best segmentation effect as the segmentation variable and segmentation point of the node, and generate new left and right subtrees based on the segmentation variable and segmentation point, and perform a new round of division on the left and right word counts or stop generating. The calculation formula is as follows: in, is a certain segmentation variable, namely the journal evaluation index, is a cutoff value of the cutoff variable, 、 , are the number of training samples of the left child node after segmentation, the number of training samples of the right child node, and the number of all training samples of the current node. 、 are the training sample sets of the left and right child nodes respectively, H(X) is the function to measure the node impurity, and the absolute mean error is used in the regression task. The calculation formula is as follows: in, is the sample set on the node, N is the sample set The total number of samples, is the average value of the impact factor of the training sample of the current node, is the impact factor value of the i-th sample in the sample set; When a regression decision tree generates a leaf node, the training of the tree is completed. Next, the method provided by sklearn will be used to rank the contribution of journal evaluation indicators to journal impact factors. First, for a node k, its importance is calculated as follows: in, , , are the ratio of the number of training samples in node k and its left and right child nodes to the total number of training samples, , , The impurities of node k and its left and right child nodes are respectively. After the importance of a certain node, the importance of a journal evaluation indicator is obtained by the following formula; in, is the node with journal evaluation index i as the segmentation variable, and all nodes refers to all nodes in the random forest; (3) The implementation of XGBoost is as follows: Since both XGBoost and Random Forest use CART as weak classifiers, Random Forest averages the independent prediction scores of each weak classifier as the prediction score of the strong classifier, while XGBoost adds the prediction scores of each weak classifier as the prediction score of the strong classifier, so the influencing factor during training is The true value of the impact factor of the p-th tree ,in The prediction result of journal impact factor for the p-1th tree; For the i-th journal evaluation index, the contribution calculation formula is as follows: in, is the number of times the i-th evaluation index is used as a segmentation variable on the t-th tree, is the summary number of the t-th tree, is a classification and regression tree with the i-th evaluation index as the segmentation variable.

7. The method for dynamic evaluation of journals based on multidimensional indicator analysis according to claim 5 is characterized in that: The neural network is used for the regression task of journal evaluation direction using simple neural network and deep neural network; During the model training phase, the simple neural network uses the journal impact factor as the true value and other journal evaluation indicators as the model's input feature matrix. The simple neural network model structure has four hidden layers: one input layer and one output layer. The number of neurons in each hidden layer is 200, 100, 50, and 25, respectively. The number of neurons in the last output layer is 1, and no softmax layer is required. The model is optimized using adaptive moment estimation. After model training, feature importance is calculated based on input perturbation, correlation coefficient, weight analysis, or a combination of the first three methods. The formula for calculating feature importance based on input perturbation is: in, is the contribution of the i-th journal evaluation index to the prediction of the journal evaluation impact factor, is the total number of journal evaluation indicators, is the absolute average error of the i-th journal evaluation index after perturbation; The calculation of feature importance based on correlation coefficient requires first calculating the Pearson correlation coefficient, which is calculated as follows: Among them, correlation is the Pearson correlation coefficient, q is the total number of journals, is the value of the t-th journal indicator in the ith journal, is the value of the journal impact factor in the i-th journal, is the average value of the t-th journal index of all journals, is the average value of the journal impact factors of all journals. After calculating the Pearson correlation coefficient between all journal evaluation indicators and journal impact factors, the contribution of journal evaluation indicators to the prediction of journal impact factors is obtained by the following formula: in, is the Pearson correlation coefficient between the i-th journal evaluation index and the journal impact factor; The method of calculating feature importance based on weight analysis is: in, is the total weight of the i-th journal evaluation index to the hidden layer; The hybrid method proposes a parameter d: in, is the total number of journal evaluation indicators, For the The contribution of each journal evaluation index to the prediction of journal impact factor is calculated under the input perturbation algorithm. Calculate the mean contribution of all journal evaluation indicators to the prediction of journal impact factors under the input perturbation algorithm; the final contribution The calculation method is: in, For the The contribution of each journal evaluation index to the prediction of journal impact factor is calculated under the weight analysis algorithm. For the The contribution of each journal evaluation index to the prediction of journal impact factor is calculated under the input perturbation algorithm. For the The contribution of each journal evaluation index to the prediction of journal impact factor is calculated using the correlation coefficient algorithm; During the training phase of the deep neural network, the number of neurons in the hidden layer was 50, 1024, 2048, 4096, 2048, 1024, and 50, respectively. The number of neurons in the final output layer was 1, and no softmax activation function was required. Each fully connected layer was followed by a batch normalization layer, and dropout was implemented in all layers. The model optimizer used a stochastic gradient descent optimizer. After model training, the VIANN and Garson algorithms were used to obtain and rank the contribution of each journal evaluation indicator to the journal evaluation impact factor. The VIANN algorithm updates the current motion variance at the end of each iteration when the weight is updated. The motion variance of the weight after the nth iteration is first calculated. and mean , the calculation formula is as follows: in, and Respectively The sum of squares and average values ​​of all updated values ​​of a weight after iterations, For the After all iterations, all weights between the input layer and the first hidden layer after the last update are obtained to calculate the contribution of each journal evaluation indicator to the journal impact factor prediction. The calculation formula is as follows: in, is the first hidden layer neuron connected to the i-th input journal evaluation index, is the motion variance of the connection weight between the i-th journal evaluation index and the t-th neuron, The value of the connection weight between the i-th journal evaluation indicator and the t-th neuron after the last update; The Garson algorithm requires obtaining the weight matrix between the input layer and the first hidden layer in the deep neural network and the weight matrix between the last hidden layer and the output layer , the vector of the contribution of journal evaluation indicators to the prediction of journal impact factors can be obtained by calculating according to the following formula: in, is the normalization function.

Images