Comprehensive measurement method for reusability of data publication

Through natural language processing technology and semantic enhancement models, the citation function and polarity of data publications are automatically classified, and the citation intensity, heat and breadth index are calculated, which solves the shortcomings of citation intensity evaluation of data publications in the prior art and realizes data publications Comprehensive evaluation and tracking of reusability of objects.

CN118897986APending Publication Date: 2024-11-05COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410971793.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

The existing data reusability measurement and evaluation methods cannot comprehensively and objectively reflect the citation strength and reusability of data publications, and lack low manual and efficient semantic citation automatic classification methods, and data reusability tracking Affected by time periods and subject areas, there is a lack of comprehensive measurement methods.

Method used

Using natural language processing technology, through data cleaning and semantic enhancement, a classification model of generalization capabilities is constructed, and the citation function and polarity are automatically classified, and the citation intensity, heat and breadth index are calculated to form a comprehensive index of reusability of data publications. Implement automatic classification and impact factor evaluation of citation functions and polarity.

Benefits of technology

It realizes low-artistic and efficient data publication citation function and automatic polarity classification, provides an objective and complete data publication reusability assessment, revealing the potential scientific value and development potential of the data set.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897986A_ABST
    Figure CN118897986A_ABST
Patent Text Reader

Abstract

The invention discloses a comprehensive measurement method for reusability of a data publication. The method comprises the following steps: acquiring a full cited document text of the data publication for constructing a sample set; through data cleaning, chapter positions of data publications in the sample set in the full text of all cited literatures are identified, and paragraph texts where the cited positions are located are extracted to serve as total samples input by subsequent models; defining a reference function and a reference polarity classification standard, and training a classification model with generalization ability based on semantic enhancement; obtaining a model tuning parameter set through continuous iteration, and forming a result sample set of reference function and reference polarity automatic classification based on a total sample; constructing and calculating data publication reusability influence factor indexes, wherein the influence factor indexes comprise a reference strength index, a reference popularity index and a reference breadth index; and constructing and calculating a reusability comprehensive index of the data publication. According to the method, the automatic classification of the reference function and the polarity can be realized, the reference strength after the reference function and the polarity are corrected is formed, and on the basis, a metering framework consisting of the reference strength, the reference popularity and the reference breadth is constructed so as to objectively and completely reveal the real situation and the development potential of the reusability of the data publication and the associated data set thereof.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the cross-field of scientific research achievement impact measurement and evaluation and natural language processing, and in particular to a quantitative measurement method for the depth of reusability of data publications. Background Art

[0002] Scientific data is an important input and output of scientific research activities, and is of great significance to scientific and technological innovation, economic and social development. With the rapid development of open data, objectively and accurately evaluating the reusability of open data sets will help to standardize and encourage the production and sharing of high-quality data, and further accelerate the efficiency and level of data reuse. However, mainstream data reusability measurement and evaluation methods, such as the number of visits, downloads and their derivative indicators of data sets, only partially reveal the users' attention to data, and there is still a lack of tracking and measurement mechanisms for the complete chain of data reusability. To solve this problem, data citations based on data publications, that is, data papers, provide ideas for deep measurement of data reusability. However, traditional measurement methods based on citation frequency do not consider the differences between different citation functions and citation polarity, and therefore cannot objectively and completely reflect the true citation intensity of data publications. Machine learning methods have proposed new ideas. By mining the semantic layer of the full text of citing documents, automatic classification of citation functions and polarity can be achieved, thereby revealing the true citation intensity. However, data publications often have high disciplinary specialization, and semantic-level analysis of citing documents is accompanied by characteristics such as high throughput. Therefore, in conventional machine learning methods, model training relies on manual annotation to create high-quality training sample sets, which faces challenges such as timeliness, accuracy, and reliability. Therefore, it is urgent to design and build a low-manual, high-efficiency model training method to achieve automatic classification of semantic-level citations. In addition, based on the research on citation scale, citation function, and polarity, we still need to realize that complete data reusability tracking is inherently subject to the evolution of time periods and is significantly related to the subject area, author cooperation network, etc. There is still a lack of comprehensive measurement methods for the reusability of data publications that fully cover all aspects of influencing factors. Summary of the invention

[0003] The purpose of the present invention is to develop a semantic layer depth measurement method for large-scale citation data in data publications, and to track the reusability of data in data publications. To achieve the above purpose, the present invention provides a comprehensive measurement method for the reusability of data publications, which includes the following steps:

[0004] Obtain the full text of the citing articles of the data publications used to construct the sample set; through data cleaning, identify the chapter position of the data publications in the sample set in the full text of all citing articles, and extract the paragraph text of the citation position as the total sample for subsequent model input;

[0005] Define the classification standards for citation functions and citation polarity, and train a classification model with generalization capabilities based on semantic enhancement; obtain the model tuning parameter set through continuous iteration, and form a sample set of automatic classification results of citation functions and citation polarity based on the total samples;

[0006] Construct and calculate the impact factor index of data publication reusability, which includes: citation intensity index, citation heat index and citation breadth index;

[0007] Construct and calculate the comprehensive index of reusability of data publications. This invention proposes a new measurement method for the reusability of data publications, which uses natural language processing optimization methods to realize the automatic classification of citation functions and polarities, and forms citation strength after correction of citation functions and polarities. On this basis, a measurement framework consisting of citation strength, citation heat, and citation breadth is constructed to objectively and completely reveal the real situation and development potential of the reusability of data publications and their associated data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 The complete system of a comprehensive measurement method for reusability of data publications provided by an embodiment of the present invention is shown;

[0009] Figure 2 Show Figure 1 The training method and process of the semantic-level automatic classification model of citations in the method shown;

[0010] Figure 3 Show Figure 1 Illustration of the calculation of citation breadth-discipline distance in the method shown;

[0011] Figure 4 Shows an example of a citation intensity index for a data publication. DETAILED DESCRIPTION

[0012] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the above description. Although the present embodiment shows exemplary embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to enable the scope of the present disclosure to be fully communicated to those skilled in the art.

[0013] The embodiment of the present invention relies on scientometrics and natural language processing technology to construct a comprehensive measurement method for the reusability of data publications based on the citation intensity, citation time heat and spatial breadth elements of the literature cited by data publications. Among them, relying on the citation position of the data publication and its context, the content-level and semantic key information of the literature cited by the data publication is intelligently identified, and the citation intensity of the data publication based on the citation function and citation polarity is quantified. On this basis, the correlation elements of citation time heat and spatial breadth are introduced to construct a comprehensive index of the reusability of data publications. The present invention can help researchers to deeply understand the reusability of data publications, thereby revealing the potential scientific value of data publications and their data sets.

[0014] Figure 1 The complete system of a comprehensive measurement method for the reusability of data publications provided by an embodiment of the present invention is shown. Figure 1 As shown, the steps include:

[0015] (1) Sample structuring and citation classification definition

[0016] Obtain the full text of the citing articles of the data publications used to construct the sample set. Through data cleaning (data cleaning, i.e., data integrity, accuracy, and normative review and verification), identify the chapter position of the data publications in the sample set in the full text of all citing articles, i.e., the citation position information, and extract the paragraph text where the citation position is located as the total sample for subsequent model input. Among them, when the data publication has multiple citations in a single citing article, extract the complete information of all citation position information and the paragraph text where it is located.

[0017] Furthermore, according to the behavioral characteristics of data reusability, the classification standards of reference function F and reference polarity P are designed (see Table 1). According to the reference function and its reference polarity, weight values ​​α are assigned respectively. i and β j , to reflect the relative weights of different citation functions and citation polarities.

[0018] Table 1 Citation categories and weights

[0019]

[0020] (2) Automatic classification of citation function and citation polarity

[0021] The embodiment of the present invention selects the BERT model (the BERT model is a deep bidirectional language representation model based on Transformer, which can use the Transformer structure to construct a multi-layer bidirectional Encoder network, and pre-train with the context of the sample set to generate a pre-trained language model), and introduces active learning and semi-supervised learning methods, relying on semantic enhancement, to train a classification model with generalization ability. Through continuous iteration, the model tuning parameter set is finally obtained to realize the automatic classification of reference function and reference polarity.

[0022] ① Specifically, some samples from the total sample set obtained in step (1) are randomly selected and manually labeled to form a labeled sample set L, and the remaining samples form an unlabeled sample set D.

[0023] Furthermore, the labeled sample set L is further divided into a training set and a validation set. The validation set is used for model performance evaluation during the training process and does not participate in the model parameter update process.

[0024] For each sample d in the unlabeled sample set i ∈D, both coarse-grained text enhancement and fine-grained text enhancement are performed, and the corresponding coarse-grained enhancement sample set and the corresponding fine-grained enhancement sample set are formed.

[0025] ② Furthermore, multiple rounds of semi-supervised training are performed on the labeled sample set and the enhanced unlabeled sample set to gradually complete the iterative update of the model parameters. Specifically, the specific steps of each round of training are as follows (see Figure 2 ):

[0026] a. Use the current round of labeled sample set to train the model and update the parameters;

[0027] b. For unlabeled samples, a translation method that can generate similar samples and a generative pre-trained language model method that can generate derivative text are used to perform coarse-grained text enhancement;

[0028] c. Import the unlabeled samples and the samples after coarse-grained text enhancement in b into the model for inference at the same time to obtain the predicted probability distribution set of sample classification. The probability distribution set is weighted averaged to obtain the approximate true probability distribution of the unlabeled sample classification, which is used for the subsequent iterative update of the model parameters;

[0029] d. For the coarse-grained enhanced samples, fine-grained enhancement is performed, that is, local noise is added to the coarse-grained enhanced samples to further improve the generalization ability of the model;

[0030] e. Import the fine-grained enhanced samples into the model for training, use the divergence mean of the predicted probability distribution of the training samples and the approximate true probability distribution obtained in step c as the loss function, and use the output of the loss function to update the model parameters;

[0031] f. After each round of training, the model performance is evaluated using the reserved validation set;

[0032] g. If the performance does not meet expectations, comprehensively evaluate the inconsistency reflected in the predicted probability distribution of the coarse-grained enhancement samples and the fine-grained enhancement samples; based on the inconsistency evaluation results, select the top M samples with higher inconsistency for uncertainty evaluation; based on the uncertainty evaluation results, select the top N samples with higher uncertainty as samples to be labeled, manually label their reference functions and reference polarities, and use them as new labeled samples, migrate from the unlabeled sample set to the labeled sample set, and start the next round of training;

[0033] h. When the performance of the model reaches the set threshold, the training is terminated and the model tuning parameter set is output.

[0034] ③ After model optimization, the structured sample of cited documents to be classified in step (1) is used as input for model reasoning, and finally a sample set of automatic classification results of citation function and citation polarity is obtained.

[0035] (3) Construct and calculate the impact factor index of data publication reusability

[0036] ① Citation intensity index C int

[0037] Assuming the citation frequency of a data publication is n, that is, the sample size of citing documents of a single data publication is n, the Boolean matrices F and P of the citation categories are constructed.

[0038] Where F is a Boolean matrix representing the type of citation function of data publications, F = (f ki ); for each element f ki , k is the sequence number of the kth citing document sample in the total sample n, k = {1, 2, ..., n}; i is the citation function type parameter, i = {1, 2, ..., 5}, corresponding to the five citation functions of citing but not using, background introduction, method design, data analysis and conclusion comparison, that is:

[0039]

[0040] In which, for each element f in the Boolean matrix F ki ,have:

[0041]

[0042] Furthermore, P is a Boolean matrix representing the type of citation polarity of data publications, P = (p kj ); for each element p kj , k is the sequence number of the k-th citing document sample in the total sample n, k = {1, 2, ..., n}; j is the citation polarity type parameter, j = {1, 2, 3}, corresponding to positive citation, neutral citation and negative citation respectively, that is:

[0043]

[0044] Among them, for each element pkj in the Boolean matrix P, there is:

[0045]

[0046] Thus, we get the citation intensity index C int ,Right now:

[0047] C int =α T F T Pβ

[0048] Where α is the citation function weight, β is the citation polarity weight, and α = (α1, α2, ..., α5) T , β=(β1,β2,β3) T .

[0049] Furthermore, when a data publication has multiple citations in a single citing document, the function and polarity of the multiple citations in the single citing document need to be weighted averaged. Suppose the citation frequency of the data publication in the case of multiple citations is n′, then (n′=n if and only if the data publication has only a single citation in the entire sample of citing articles). The citation frequency n′ of the kth citing article k , a citation function matrix F′ containing multiple citations can be constructed k and the reference polarity matrix P′ k ,Right now:

[0050]

[0051] Thus, the citation intensity index C in the case of multiple citations is obtained int ,Right now:

[0052]

[0053] ② Citation heat index C pop

[0054] The time span of the earlier half of citations and the time span of the newer half of citations (citation half-life) are used to characterize the decay process of the immediacy and sustainability potential of citation popularity, namely:

[0055]

[0056] and:

[0057]

[0058] in:

[0059] C pop is the citation heat index;

[0060] λ is the correction coefficient, and t * Map from [0, +∞) to (0, 1];

[0061] t * is the time difference of the first citation of the data publication (days);

[0062] φ is the time factor, which measures the time index of the persistence and immediacy of the popularity of data publications;

[0063] t is the publication cycle of the data publication, that is, the span from the publication time of the data publication to the publication time of the latest citing literature collected at the time of the survey (days);

[0064] t′ is the citation half-life of the data publication, that is, the longest time span (days) between the publication of the newest half of the citing documents.

[0065] ③ Citation breadth index C b r h

[0066] The relative distance between data publications and cited literature and the relative distance between author groups are used to characterize the diffusion of data reusability in space, namely:

[0067]

[0068] and:

[0069]

[0070] in:

[0071] C brh is the citation breadth index;

[0072] is the relative distance between the data publication and the kth citing article;

[0073] is the relative distance between the author of the data publication and the kth citing article;

[0074] x′ is the subject coordinate axis. There is a one-to-one mapping f between x′ and the 22 integer coordinates on the one-dimensional axis x, which is used to represent the relative positions of the 22 major subject categories in the China Library Classification System (see Figure 3 ),

[0075] x′0 is the position of the data publication on the subject coordinate axis x′;

[0076] x′ k is the position of the k-th citing article on the subject coordinate axis x′, k = {1, 2, ..., n};

[0077] A0 represents the set of authors of data publications, I0, N0, C0 represent the sets of institutions, countries and continents that form a surjection with A0;

[0078] A k represents the author set of the k-th citing article, I k , N k , C k Indicates that k The set of institutions, countries and continents that constitute the surjection.

[0079] (4) Construct and calculate the comprehensive index C of data publication reusability

[0080]

[0081] and:

[0082]

[0083] in:

[0084] C is the comprehensive index of reusability of data publications;

[0085] A citation heat-corrected index to characterize the reusability of data publications;

[0086] A citation-breadth-corrected index to characterize the reusability of data publications.

[0087] Compared with the prior art, the embodiments of the present invention are beneficial in that:

[0088] 1. Optimize machine learning methods to achieve semantic level measurement of citation context content. Focus on using coarse-grained text enhancement and fine-grained text enhancement methods to improve the robustness of the model's automatic classification of citations, reduce the cost of constructing sample sets for model training, and complete the automatic classification of citation functions and polarity of data publications with low labor and high quality.

[0089] 2. Introduce quantitative time and space dimensions to construct a comprehensive measurement method for the reusability of data publications, including citation function, weighted citation intensity of citation polarity, citation heat and citation breadth, so as to fully and objectively reveal the reusability of data publications.

[0090] 3. The comprehensive index of data publication reusability can be used to track the data reusability characteristics based on data publications, data sets, and data platforms. The citation intensity index after citation function and polarity correction, as well as the citation heat correction index and citation breadth correction index can also be used separately to characterize the real citation intensity of data publications and their data sets, and their reusability evolution within the scope of time heat and spatial breadth.

[0091] The features and implementation of the embodiments of the present invention are further described below through examples.

[0092] The embodiment of the present invention selects 2088 citing articles from three data publications with high citation volume in the data journal "Earth System Science Data" as samples (Table 2), and the data collection time is from August to November 2023. The implementation examples of this patent are as follows:

[0093] Table 2 Sample data publications

[0094]

[0095] (1) Sample structuring and citation classification definition

[0096] Obtain the full text of the citing articles of the data publications used to construct the sample set. Through data cleaning, identify the chapter position of the data publications in the sample set in the full text of the citing articles, that is, the citation position information, and extract the paragraph text where the citation position is located as the subsequent model input. Among them, there are multiple citations in the citing articles of the three data publications in this example sample, so all the citation position information and the complete information of the paragraph text where they are located are extracted.

[0097] Furthermore, according to the behavioral characteristics of data reusability, the classification standards of reference function F and reference polarity P are designed (see Table 1), and the corresponding weight value α is defined i and β j , to reflect the relative weights of different citation functions and citation polarities.

[0098] (2) Automatic classification of citation function and citation polarity

[0099] The embodiment of the present invention selects the BERT model, introduces active learning and semi-supervised learning methods, iteratively constructs a sample set, gradually defines the samples that need to be labeled according to the results of the model prediction output, and updates the model parameters.

[0100] Specifically, 15% of the total sample volume obtained in step (1) is randomly selected and manually labeled to form a sample set L of labeled samples.

[0101] Furthermore, 2 / 3 of the labeled sample set L is used as the labeled training set, and the remaining 1 / 3 is used as the validation set during the training process. The validation set is used to evaluate the model performance and does not participate in the model parameter update process.

[0102] Furthermore, the remaining 85% of the total sample volume is used as the unlabeled sample set D. For each sample d in the unlabeled sample set i ∈D, two-stage data enhancement is performed to form a coarse-grained enhanced sample set D coa And the corresponding fine-grained enhanced sample set D fin .

[0103] Among them, coarse-grained text enhancement is text-level enhancement. The embodiment of the present invention adopts a translation method that can generate similar samples and a generative pre-trained language model method that can generate derivative text to complete the coarse-grained enhancement of the quoted text. Assume that the coarse-grained enhancement function is coa j (d i )(j represents the jth coarse-grained enhancement method, there are J of them), then the coarse-grained enhancement sample set D coa satisfy:

[0104] D coa ={coa j ((d i )|j=0,1,…,J}

[0105] Furthermore, fine-grained sample enhancement is a machine-readable vector-level enhancement, which needs to maximize the diversity and representativeness of coarse-grained enhancement samples within a certain range. To this end, the fine-grained enhancement process is achieved by adding local noise to the coarse-grained enhancement samples. Assume that the word vector of the current unlabeled coarse enhancement sample is The local noise is rp, then the word vector enhanced by fine-grained samples is Define the fine-grained enhancement function as fin j (d i ), then the fine-grained enhanced sample set D fin satisfy:

[0106] D fin ={fin j (d i )|j=0,1,…,J}

[0107] Furthermore, the BERT model is trained semi-supervised. For labeled samples, supervised training is performed to update the model parameters; for unlabeled samples, unsupervised training is performed to compare the predicted probability distribution of fine-grained enhanced samples with the approximate true probability distribution. The divergence mean of is defined as the loss function, and the output of the loss function is used to update the model parameters.

[0108] Specifically, for unsupervised training, each original sample d i And its corresponding coarse-grained enhanced sample set D coa Input the model for inference. The predicted probability distribution y of the original sample will be output i And the predicted probability distribution of coarse-grained enhanced samples The weighted average As an approximation of the true probability distribution of the sample Right now:

[0109]

[0110] Furthermore, the fine-grained enhanced sample set D fin Import the model and train it. The predicted probability distribution of the fine-grained enhanced samples is similar to the approximate true probability distribution. The divergence mean of is defined as the loss function Loss, and the output of the loss function can be used to update the model parameters, namely:

[0111]

[0112] Furthermore, the reserved validation set is used to evaluate the model performance to determine whether the precision p and F1 value under the current model have reached the final expected thresholds. and Here, according to actual needs, set

[0113] Specifically, if the performance does not meet expectations, the divergence Inc of the probability distribution set of the coarse-grained enhancement samples and the fine-grained enhancement samples obtained by model inference is comprehensively evaluated. coa and Inc fin , and take the weighted average of the two divergence values ​​as sample d i The inconsistency index Inc(d i ),Right now:

[0114]

[0115] and:

[0116]

[0117] Furthermore, the inconsistency index of all cited documents is sorted from high to low, and the top M samples with higher inconsistency index are selected. Here, the value of M is determined by the accuracy of the model obtained by the current round of training on the validation set, that is, (where [·] is the rounding function), γ represents the parameter for updating the number of samples, and p t represents the final expected accuracy threshold of the model, and p represents the accuracy of the validation set under the current model.

[0118] Furthermore, the uncertainty of M samples is evaluated, that is, the sample d is calculated i After model inference, the entropy of the predicted probability distribution is obtained as the sample uncertainty index Unc(d i ), that is:

[0119] Unc(d i )=-y i T log(y i )

[0120] Specifically, the samples are sorted from high to low according to the uncertainty index, and the first N samples with higher uncertainty index (here N=2 / 3M) are selected as samples to be marked.

[0121] Furthermore, the samples to be labeled are manually labeled in terms of reference function and reference polarity, and then these samples are migrated from the unlabeled sample set D to the labeled sample set L to start the next round of training.

[0122] If the model performance reaches the threshold, the training is terminated and the model tuning parameter set is output.

[0123] Furthermore, the structured sample of citing documents to be classified in step (1) is used as the input of the tuned model, and after model reasoning, a sample set of automatic classification results of citation function and citation polarity is finally obtained.

[0124] (3) Construct and calculate the impact factor index of data publication reusability

[0125] ① Citation intensity index C int

[0126] Construct the sample category Boolean matrices F and P. Combined with the importance of the citation function of the data publication, assign the citation function weight α = (0.5, 0.7, 1.2, 1.0, 1.2) T 、Citation polarity weight β = (1.2, 1, 0.8) T Since the sample has multiple citations, the multiple citation functions and polarities of a single citing document in the three sample data publications are weighted averaged. Thus, the citation intensity index of the sample is calculated (see Figure 4).

[0127] ② Citation heat index C pop

[0128] The time span of the earlier half of citations and the time span of the newer half of citations (citation half-life) are used to characterize the decay process of the immediacy and sustainability potential of citation popularity, namely:

[0129]

[0130] and:

[0131]

[0132] in,

[0133] C pop is the citation heat index;

[0134] λ is the correction coefficient, and t * Map from [0, +∞) to (0, 1];

[0135] t * is the time difference of the first citation of the data publication (days);

[0136] φ is the time factor, which measures the time index of the persistence and immediacy of the popularity of data publications;

[0137] t is the publication cycle of the data publication, that is, the span from the publication time of the data publication to the publication time of the latest citing literature collected at the time of the survey (days);

[0138] t′ is the citation half-life of the data publication, that is, the longest time span (days) between the publication of the newest half of the citing documents.

[0139] Thus, the citation heat index C of the sample is calculated. pop (See Table 3).

[0140] Table 3 Examples of citation heat indexes for data publications

[0141]

[0142] ③ Citation breadth index C b r h

[0143] The relative distance between data publications and cited literature and the relative distance between author groups are used to characterize the diffusion of data reusability in space, namely:

[0144]

[0145] and:

[0146]

[0147] in:

[0148] C brh is the citation breadth index;

[0149] is the relative distance between the data publication and the kth citing article;

[0150] is the relative distance between the author of the data publication and the kth citing article;

[0151] x′ is the subject coordinate axis. There is a one-to-one mapping f between x′ and the 22 integer coordinates on the one-dimensional axis x, which is used to represent the relative positions of the 22 major subject categories in the China Library Classification System (see Figure 3 ),

[0152] x′0 is the position of the data publication on the subject coordinate axis x′;

[0153] x′ k is the position of the k-th citing article on the subject coordinate axis x′, k = {1, 2, ..., n};

[0154] A0 represents the set of authors of data publications, I0, N0, C0 represent the sets of institutions, countries and continents that form a surjection with A0;

[0155] A k represents the author set of the k-th citing article, I k , N k , C k Indicates that k The set of institutions, countries and continents that constitute the surjection.

[0156] Thus, the citation breadth index C of the sample is calculated. brh (See Table 4).

[0157] Table 4 Examples of citation breadth index of data publications

[0158]

[0159] (6) Construct and calculate the comprehensive index C of data publication reusability

[0160]

[0161] and:

[0162]

[0163] in:

[0164] C is the comprehensive index of reusability of data publications;

[0165] A citation heat-corrected index to characterize the reusability of data publications;

[0166] A citation-breadth-corrected index to characterize the reusability of data publications.

[0167] Thus, the comprehensive reusability index C of the example sample is calculated (see Table 5).

[0168] Table 5 Examples of comprehensive reusability index C of data publications

[0169]

[0170] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A comprehensive measurement method for the reusability of data publications, characterized in that: The following steps are involved: Obtain the full text of the citing articles of the data publications used to construct the sample set; through data cleaning, identify the chapter position of the data publications in the sample set in the full text of all citing articles, and extract the paragraph text of the citation position as the total sample for subsequent model input; Define the classification standards for citation functions and citation polarity, and train a classification model with generalization capabilities based on semantic enhancement; obtain the model tuning parameter set through continuous iteration, and form a sample set of automatic classification results of citation functions and citation polarity based on the total samples; Construct and calculate the impact factor index of data publication reusability, which includes: citation intensity index, citation heat index and citation breadth index; Construct and calculate a comprehensive index of reusability of data publications.

2. The method according to claim 1, characterized in that When a data publication has multiple citations in a single citing document, extract all citation location information and complete information of the paragraph text in which it is located.

3. The method according to claim 1, characterized in that According to the behavioral characteristics of data reusability, the weight values ​​α are assigned according to the classification standards and actual conditions of the reference function and reference polarity. i and β j , to reflect the relative weights of different citation functions and citation polarities.

4. The method according to claim 1, characterized in that: The steps of constructing a reference function and a reference polarity classification sample set, and training a classification model through semantic enhancement, include: Extract some samples from the citation position and its citation text sample set, manually mark them to form a marked sample set L, and the remaining samples form an unmarked sample set D; The labeled sample set is further divided into a training set and a validation set. The validation set is used to evaluate the model performance during the training process and does not participate in the model parameter update process. For each sample d in the unlabeled sample set i ∈D, perform coarse-grained text enhancement and fine-grained text enhancement, and form corresponding coarse-grained enhancement sample sets and corresponding fine-grained enhancement sample sets; Multiple rounds of semi-supervised training are performed on the labeled sample set and the enhanced unlabeled sample set to gradually complete the iterative update of the model parameters.

5. The method according to claim 4, characterized in that The steps of each training round include: a. Use the current round of labeled sample set to train the model and update the parameters; b. For unlabeled samples, a translation method that can generate similar samples and a generative pre-trained language model method that can generate derivative text are used to perform coarse-grained text enhancement; c. Import the unlabeled samples and the samples after coarse-grained text enhancement in step b into the model for inference at the same time to obtain a set of predicted probability distributions for sample classification; perform weighted averaging on the set of probability distributions to obtain an approximate true probability distribution for the classification of unlabeled samples, which is used for subsequent iterative updates of model parameters; d. Implement fine-grained enhancement on coarse-grained enhancement samples to further improve the generalization ability of the model; e. Import the fine-grained enhanced samples into the model for training, use the divergence mean of the predicted probability distribution of the training samples and the approximate true probability distribution obtained in step c as the loss function, and use the output of the loss function to update the model parameters; f. After each round of training, the model performance is evaluated using the reserved validation set; g. If the performance does not meet expectations, comprehensively evaluate the inconsistency reflected in the predicted probability distribution of the coarse-grained enhancement samples and the fine-grained enhancement samples; based on the inconsistency evaluation results, select the top M samples with higher inconsistency for uncertainty evaluation; based on the uncertainty evaluation results, select the top N samples with higher uncertainty as samples to be labeled, manually label their reference functions and reference polarities, and use them as new labeled samples, migrate from the unlabeled sample set to the labeled sample set, and start the next round of training; h. When the performance of the model reaches the set threshold, the training is terminated and the model tuning parameter set is output; i. After model optimization, the structured samples of cited documents to be classified are used as input for model reasoning, and finally a sample set of automatic classification results of citation function and citation polarity is obtained.

6. The method according to claim 4, characterized in that The citation strength index is obtained by the following steps: Construct the Boolean matrices F and P of citation categories, where n is the citation frequency of data publications, that is, the total sample size of citing literature of a single data publication, and F is the Boolean matrix representing the citation function type of data publications, F = (f ki ); for each element f ki , k is the sequence number of the k-th citing document sample in the total sample n, k = {1, 2, ..., n}; i is the citation function type parameter, i = {1, 2, ..., 5}, corresponding to the five citation functions of citing but not using, background introduction, method design, data analysis and conclusion comparison, that is: In which, for each element f in the Boolean matrix F ki ,have: P is a Boolean matrix that characterizes the type of citation polarity of data publications, P = (p kj ); for each element p kj , k is the sequence number of the k-th citing document sample in the total sample n, k = {1, 2, ..., n}; j is the citation polarity type parameter, j = {1, 2, 3}, corresponding to positive citation, neutral citation and negative citation respectively, that is: Where, for each element p in the Boolean matrix P kj ,have: Thus, we get the citation intensity index C int ,Right now: C int =a T F T Pb Among them, α is the citation function weight, β is the citation polarity weight, and α=(α1,α2,…,α5) T , β=(β1,β2,β3) T .

7. The method according to claim 6, characterized in that When a data publication has multiple citations in a single citing document, the function and polarity of the multiple citations in the single citing document need to be weighted averaged; where n' is the citation frequency of the data publication in the case of multiple citations, then If and only if the data publication has only a single citation in the entire sample of citing articles, n′=n; the citation frequency of the k-th citing article is n′ k , construct the citation function matrix F′ containing multiple citations k and the reference polarity matrix P′ k ,Right now: Thus, the citation intensity index C in the case of multiple citations is obtained int ,Right now:

8. The method according to claim 1, characterized in that: The time span of the earlier half of the citations and the time span of the newest half of the citations, i.e., the citation half-life, are used to characterize the decay process of the immediacy and sustainability potential of the citation heat, respectively. The calculation formula of the citation heat index is: and: in: C pop is the citation heat index; λ is the correction coefficient, and t * Map from [0, +∞) to (0, 1]; t * is the time difference of the first citation of the data publication; φ is the time factor, which measures the time index of the persistence and immediacy of the popularity of data publications; t is the publication cycle of the data publication, that is, the span from the publication time of the data publication to the publication time of the latest citing literature collected at the time of the survey; t′ is the citation half-life of the data publication, that is, the longest time span of the publication time of the newest half of the citing documents.

9. The method according to claim 1, characterized in that: The relative distance between the disciplines of data publications and citing documents and the relative distance between author groups are used to characterize the diffusion of data reusability in space. The calculation formula of the citation breadth index is: and: in: C brh is the citation breadth index; is the relative distance between the data publication and the kth citing article; is the relative distance between the author of the data publication and the kth citing article; x′ is the subject coordinate axis. There is a one-to-one mapping f between x′ and the 22 integer coordinates on the one-dimensional axis x, which is used to represent the relative distance between the 22 major subjects in the China Library Classification System, f: x→x′. x = {0, ..., 21}; x′0 is the position of the data publication on the subject coordinate axis x′; x′ k is the position of the k-th citing article on the subject coordinate axis x′, k = {1, 2, ..., n}; A0 represents the set of authors of data publications, I0, N0, C0 represent the sets of institutions, countries and continents that form a surjection with A0; A k represents the author set of the k-th citing article, I k , N k , C k Indicates that k The set of institutions, countries and continents that constitute the surjection.

10. The method according to claim 1, characterized in that The calculation formula of the publication reusability comprehensive index is: and: in: C is the comprehensive index of reusability of data publications; A citation heat-corrected index to characterize the reusability of data publications; A citation-breadth-corrected index to characterize the reusability of data publications.

Citation Information

Cited By

  • Credit labeling method

    CN120046607A