A knowledge point-oriented scientific and technological resource recommendation method and system

Abstract

The application discloses a kind of knowledge point-oriented scientific and technological resource recommendation method and system, the present application includes using paper vector representation model respectively calculate the final vector of candidate paper in submission paper and candidate paper set as target paper;The similarity of the final vector of submission paper and each candidate paper is calculated respectively to select the highest similarity of multiple candidate papers;Paper vector representation model includes text encoder, for mining knowledge points from the title T and abstract A of input target paper and the title T of n references of target paper and encoding into hidden state;Linear layer is used to linearly process hidden state to obtain vector representation;Citation parser is used to mine potential citation preference and possible missing knowledge points;Splicing layer is used to splice vector representation output.The present application aims to combine paper content text and existing citations to improve the accuracy of missing citation recommendation for submission papers, improve recommendation accuracy and citation recommendation efficiency.

Description

Technical Field

[0001] This invention relates to artificial intelligence processing technology for science and education case studies and literature analysis, specifically to a method and system for recommending science and technology resources oriented towards knowledge points. Background Technology

[0002] Currently, learners can access a wealth of scientific and technological resources through the internet and digital technologies, such as online courses, academic papers, and instructional videos. However, faced with such a vast amount of resources, learners often struggle with how to select those that best suit their learning needs and knowledge points. Scientific papers, as records of scientific research and carriers of research results, represent the sum of knowledge points and are among the most core types of scientific and technological resources. However, with the explosive growth of scientific publications, efficiently and comprehensively finding scientific papers that meet one's needs and fit one's knowledge points has become increasingly difficult. Current citation recommendation systems aim to recommend lists of scientific papers for a given text context or paper content. However, existing citation recommendation systems only use the papers in the reference list of the input paper as the standard, but the reference list of the input paper is usually imperfect and requires the recommendation of other suitable citations as supplements. A common example is in academic publishing scenarios where submitted papers are deemed by peer reviewers to lack citations of important papers during the peer review process.

[0003] Citation recommendation models are generally divided into two categories: overall citation recommendation, which recommends a list of references for a given paper manuscript; and local citation recommendation, which recommends citations for a specific contextual passage within the paper where a citation occurs. In general, the citation recommendation task aims to automatically recommend appropriate citations for a given textual context or a draft paper lacking references. Existing research focuses solely on the paper's content, neglecting potential imperfections in the input paper's references and the negative impact of missing important citations. Consequently, it is common for submitted papers to be deemed lacking in important citations by reviewers for various reasons. Therefore, how to integrate knowledge points to improve the recommendation effect in existing scientific and technological resource recommendation methods has become a critical technical problem that urgently needs to be solved. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method and system for recommending scientific and technological resources based on knowledge points, addressing the aforementioned problems in the prior art. This invention aims to improve the accuracy of recommending missing citations in submitted papers by combining the text of the paper content with existing citations, thereby improving the accuracy and efficiency of citation recommendation.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A knowledge-point-oriented method for recommending scientific and technological resources includes:

[0007] S101, taking the submitted papers and candidate papers in the paper candidate set as target papers, the final vector of the target paper is calculated using a paper vector representation model; the paper vector representation model includes: a text encoder, used to mine knowledge points from the title T and abstract A of the input target paper, and the titles T of n references of the target paper, and encode them into corresponding hidden states; a linear layer, used to linearly process the hidden states to obtain the vector representation v of the target paper. content and the title vector representation of n references A citation parser is used to convert the vector representation v of the target paper. content and the title vector representation of n references Based on the cross-attention mechanism, potential citation preferences and potentially overlooked knowledge points are mined to obtain the reference vector representation v. R The splicing layer is used to concatenate the vector representation v of the target paper. content and reference vector representation v R The concatenated vector serves as the final vector of the target paper.

[0008] S102, calculate the similarity between the final vectors of the submitted paper and each candidate paper respectively;

[0009] S103: Sort the candidate papers according to similarity, and select a specified number of candidate papers as the recommended results for submission in order of similarity from high to low.

[0010] Optionally, the text encoder mines knowledge points from the title T and abstract A of the input target paper and encodes them into corresponding hidden states. This includes: first, concatenating the title T and abstract A of the target paper, inserting sentence separator markers between them, and inserting a preorder marker at the beginning before inputting them into the BERT encoder to obtain a series of hidden states:

[0011]

[0012] In the above formula, h [C] The hidden state corresponding to the preceding marker [CLS] for The corresponding hidden state, for The corresponding hidden state, BERT refers to the BERT encoder, and [C] is the preorder tag [CLS] added by the BERT encoder to the beginning of the sequence. These represent the first to i-th single occurrences in the title T, respectively. [S] adds the sentence separator marker [SEP] to the BERT encoder. Let each word in the abstract A be the j-th word. The BERT encoder generates a corresponding hidden state for each single word or token in the input; then, it generates the hidden state h corresponding to the preceding token [CLS].[C] Input a linear layer to obtain the vector representation v of the target paper. content .

[0013] Optionally, when the text encoder mines knowledge points from the titles T of the n references of the input target paper and encodes them into corresponding hidden states, the encoding for any x-th reference of the target paper includes: first, inserting a sentence separator marker at the end and a preorder marker at the beginning of the title T of the x-th reference before inputting it into the BERT encoder to obtain a series of hidden states:

[0014]

[0015] In the above formula, The hidden state corresponding to the pre-reference tag [CLS] of the xth reference. These are references 1 to m of the xth reference. x The hidden states corresponding to each word, BERT refers to the BERT encoder, and [C] represents the preorder suffix [CLS] added by the BERT encoder to the beginning of the sequence. These are references 1 to m of the xth reference. x For each word, [S] adds a sentence separator marker [SEP] to the BERT encoder; then, the hidden state corresponding to the preceding marker [CLS] is... Input a linear layer to obtain the title vector representation of the x-th reference.

[0016] Optionally, the citation parser will use the vector representation v of the target paper. content and the title vector representation of n references Based on the cross-attention mechanism, potential citation preferences and potentially overlooked knowledge points are mined to obtain the reference vector representation v. R include:

[0017] S201, calculate the cross-attention weight vector α of the attention layer according to the following formula:

[0018]

[0019] In the above formula, softmax is the softmax activation function, and any element α in the x-th dimension of the cross-attention weight vector α is... x The title vector representation of the x-th reference. The weights;

[0020] S202, Represent the title vectors of the n references according to the following formula. By introducing an attention layer, the reference vector representation v can be obtained. R :

[0021]

[0022] In the above formula, α x Let x be any element in the x-th dimension of the cross-attention weight vector α. Let x be the vector representation of the title of the x-th reference, and n be the number of references.

[0023] Optionally, the splicing layer will concatenate the vector representation v of the target paper. content and reference vector representation v R The expression for the final vector of the target paper after concatenation is:

[0024]

[0025] In the above formula, v is the final vector of the target paper, and T represents the transpose operation of the vector.

[0026] Optionally, before step S101, a paper vector representation model is trained using sample paper p, positive sample paper p+, and negative sample paper p-, where positive sample paper p+ is a paper that fits the knowledge points of sample paper p, negative sample paper p- is a paper that has not been cited by sample paper p or recommended for citation by reviewers, and the loss function used to train the paper vector representation model is the triplet marginal loss function.

[0027] Optionally, the functional expression of the triplet marginal loss function is:

[0028] L = max{s(v p ,v - )-s(v p ,v + )+m,0},

[0029] In the above formula, L is the marginal loss function of the triple, max is the maximum value, and v p Let v be the final vector obtained by using the paper vector representation model for sample paper p. + For the positive sample paper p+, the final vector obtained using the paper's vector representation model is v. - p represents the negative sample paper, which is the final vector obtained by using the paper vector representation model. s is used to calculate the similarity, and m is the marginal hyperparameter of the loss.

[0030] Optionally, when using sample paper p, positive sample paper p+, and negative sample paper p- to train the paper vector representation model, the sources of negative sample paper p- include: (1) papers randomly selected from the corpus; (2) for sample paper p and its candidate negative sample papers, the similarity between the final vectors of sample paper p and each candidate negative sample paper is calculated respectively, and the candidate papers are sorted according to the similarity, excluding a specified number of candidate negative sample papers with the highest similarity, and selecting one or more candidate negative sample papers with the highest similarity from the remaining candidate negative sample papers.

[0031] Furthermore, the present invention also provides a knowledge-point-oriented science and technology resource recommendation system, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the knowledge-point-oriented science and technology resource recommendation method.

[0032] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program that is programmed or configured by a microprocessor to execute the knowledge-point-oriented technology resource recommendation method.

[0033] Compared with the prior art, the present invention has the following main advantages:

[0034] 1. This invention utilizes a citation parser to perform representation learning on the papers in the reference list of the input paper and integrates it with the paper content to uncover potential citation preferences and possible omissions in the papers, which can help improve recommendation performance.

[0035] 2. This invention can not only handle traditional citation recommendation tasks, that is, recommend relevant citations for paper content; it can also complete the existing citation list of an input paper, thus expanding the application scenarios. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram of the structure of the vector representation model in the embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram illustrating the specific structure of the vector representation model in this embodiment of the invention.

[0039] Figure 4 This is a schematic diagram illustrating the training principle of the vector representation model in this embodiment of the invention.

[0040] Figure 5 The bar chart shows the experimental results of different sampling strategies in the embodiments of the present invention. Detailed Implementation

[0041] like Figure 1 As shown, the method for recommending scientific and technological resources based on knowledge points in this embodiment includes:

[0042] S101, taking the submitted papers and candidate papers in the paper candidate set as target papers, calculate the final vector of the target papers using the paper vector representation model respectively;

[0043] S102, calculate the similarity between the final vectors of the submitted paper and each candidate paper respectively;

[0044] S103: Sort the candidate papers according to similarity, and select a specified number of candidate papers as the recommended results for submission in order of similarity from high to low.

[0045] like Figure 2 As shown, the vector representation model in this embodiment includes:

[0046] A text encoder is used to extract knowledge points from the title T and abstract A of the input target paper, as well as the titles T of n references of the target paper, and encode them into corresponding hidden states.

[0047] A linear layer is used to linearly process the hidden states to obtain the vector representation v of the target paper. content and the title vector representation of n references

[0048] A citation parser is used to convert the vector representation v of the target paper. content and the title vector representation of n references Based on the cross-attention mechanism, potential citation preferences and potentially overlooked knowledge points are mined to obtain the reference vector representation v. R ;

[0049] The concatenation layer is used to concatenate the vector representation v of the target paper. content and reference vector representation v R The concatenated vector serves as the final vector of the target paper.

[0050] like Figure 3 As shown, in this embodiment, the text encoder uses a pre-trained model—the BERT encoder. However, other types of text encoders can be used as needed, and are not limited to pre-trained models; non-pre-trained models can also be used. In this embodiment, the text encoder mines knowledge points from the input target paper's title T and abstract A and encodes them into corresponding hidden states. This includes: first, concatenating the target paper's title T and abstract A, inserting sentence separators between them, and inserting a preorder suffix at the beginning before inputting it into the BERT encoder to obtain a series of hidden states:

[0051]

[0052] In the above formula, h [C] The hidden state corresponding to the preceding marker [CLS] for The corresponding hidden state, for The corresponding hidden state, BERT refers to the BERT encoder, and [C] is the preorder tag [CLS] added by the BERT encoder to the beginning of the sequence. These represent the first to i-th single occurrences in the title T, respectively. [S] adds the sentence separator marker [SEP] to the BERT encoder. Let each word in the abstract A be the j-th word. The BERT encoder generates a corresponding hidden state for each single word or token in the input; then, it generates the hidden state h corresponding to the preceding token [CLS]. [C] Input a linear layer to obtain the vector representation v of the target paper. content .

[0053] like Figure 3 As shown, in this embodiment, the text encoder will input the titles T( of n references in the target paper) Figure 3 The middle is represented as T r1 ~T rn When mining knowledge points and encoding them into corresponding hidden states, the encoding of any x-th reference in the target paper includes: first, inserting a sentence separator marker at the end of the title T of the x-th reference and a preorder marker at the beginning, and then inputting it into the BERT encoder to obtain a series of hidden states:

[0054]

[0055] In the above formula, The hidden state corresponding to the pre-reference tag [CLS] of the xth reference. These are references 1 to m of the xth reference. x The hidden states corresponding to each word, BERT refers to the BERT encoder, and [C] represents the preorder suffix [CLS] added by the BERT encoder to the beginning of the sequence. These are references 1 to m of the xth reference. x For each word, [S] adds a sentence separator marker [SEP] to the BERT encoder; then, the hidden state corresponding to the preceding marker [CLS] is... Input a linear layer to obtain the title vector representation of the x-th reference.

[0056] In this embodiment, the linear layer linearly processes the hidden state of the target paper to obtain the vector representation v of the target paper. contentIt can be represented as:

[0057] v content =Linear(h [C] ),

[0058] In the above formula, Linear represents linear processing.

[0059] In this embodiment, the linear layer linearly processes the hidden states of n references to obtain the title vector representations of n references. It can be represented as:

[0060]

[0061] In the above formula, Let x be the vector representation of the titles of the references, where Linear indicates linear processing. This is the hidden state corresponding to the pre-reference marker [CLS] of the xth reference.

[0062] A citation parser is used to uncover potential citation preferences, resulting in vector representations of the references in the input scientific paper. For example... Figure 3 As shown, in this embodiment, the citation parser will convert the vector representation v of the target paper... content and the title vector representation of n references Based on the cross-attention mechanism, potential citation preferences and potentially overlooked knowledge points are mined to obtain the reference vector representation v. R include:

[0063] S201, calculate the cross-attention weight vector α of the attention layer according to the following formula:

[0064]

[0065] In the above formula, softmax is the softmax activation function, and any element α in the x-th dimension of the cross-attention weight vector α is... x The title vector representation of the x-th reference. The weights;

[0066] S202, Represent the title vectors of the n references according to the following formula. By introducing an attention layer, the reference vector representation v can be obtained. R :

[0067]

[0068] In the above formula, α x Let x be any element in the x-th dimension of the cross-attention weight vector α. Let be the title vector representation of the x-th reference, and n be the number of references. Intuitively, a paper's citation patterns or preferences can be derived from its relationships with cited papers. To model the existing citations of the input paper, this embodiment inputs all vectors of cited papers into the attention layer and calculates the cross-attention weight vector of the references using the formula above.

[0069] To recommend omitted citations relevant to content and citation preferences and avoid meaningless repetition, this embodiment concatenates the content vector and reference vector to obtain the final vector of the input paper. The concatenation layer represents the vector v of the target paper. content and reference vector representation v R The expression for the final vector of the target paper after concatenation is:

[0070]

[0071] In the above formula, v is the final vector of the target paper, and T represents the transpose operation of the vector.

[0072] like Figure 4 As shown, before step S101, a paper vector representation model is trained using sample paper p, positive sample paper p+, and negative sample paper p-. Positive sample paper p+ refers to papers that match the knowledge points of sample paper p (e.g., papers recommended for citation by reviewers), and negative sample paper p- refers to papers that are not cited by sample paper p or recommended for citation by reviewers. The loss function used to train the paper vector representation model is the triplet marginal loss function. The functional expression of the triplet marginal loss function is:

[0073] L = max{s(v p ,v - )-s(v p ,v + )+m,0},

[0074] In the above formula, L is the marginal loss function of the triple, max is the maximum value, and v p Let v be the final vector obtained by using the paper vector representation model for sample paper p. + For the positive sample paper p+, the final vector obtained using the paper's vector representation model is v. - Let s be the final vector obtained from the negative sample paper p- using the paper vector representation model, s be the similarity calculation, and m be the marginal loss hyperparameter. The value of m determines the sensitivity of the triplet marginal loss function L to the negative sample paper p-. In this embodiment, the similarity s is defined as the cosine similarity between the two final vectors. However, other similarity algorithms can also be used.

[0075] The definition of positive sample paper p+ is simple; it is a paper recommended and cited by the reviewers of sample paper p. However, careful selection of negative sample paper p- is crucial to the performance of the paper vector representation model. In this embodiment, when training the paper vector representation model using sample paper p, positive sample paper p+, and negative sample paper p-, the sources of negative sample paper p- include: (1) papers randomly selected from the corpus; (2) negative nearest neighbor sampling (e.g., ...). Figure 4 For sample paper p and its candidate negative sample papers, the similarity between the final vectors of sample paper p and each candidate negative sample paper is calculated. The candidate papers are then sorted according to the similarity, and a specified number of candidate negative sample papers with the highest similarity are excluded. The remaining candidate negative sample papers are then selected from those with the highest similarity. For source (1), the paper vector representation model may easily distinguish between random negative sample paper p- and positive sample paper p+. Source (2) excludes a specified number of candidate negative sample papers with the highest similarity (equivalent to the standard answer), whose vectors are highly similar to the vector of the input paper. Therefore, we call the paper selected from these candidates "hard negative examples". Given the input paper, we use the current checkpoint of the model to obtain the top K papers other than the standard answer paper. n The most recent candidates; training the paper vector representation model to distinguish difficult negative sample papers p- may improve overall performance. iter Each training iteration updates the model's checkpoints, and at this time, the negative sample p-sampling is also updated.

[0076] In this embodiment, a base version of the BERT encoder is used as the text encoder, with only the last three layers trainable, and it is initialized with weights from SciNCL (M. Ostendorff, N. Rethmeier, I. Augenstein, B. Gipp, and G. Rehm, “Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings.”). We train our model using the AdamW optimizer with an initial learning rate of 2e-5 and a weight decay rate of 1e-2. The marginal loss hyperparameter m is set to 0.05. The model is trained for five epochs on a single NVIDIA GeForce V100 (32GB) GPU, and its checkpoints are at every N... iier =Updated after 5000 training iterations. Each cycle contains 5000 iterations, and each iteration contains three triplets. Each triplet contains one input paper, one positive paper, and one paper from K. n=100 recently selected hard-to-negative papers and one randomly selected easy-to-negative paper from the most recent candidates. In this embodiment, the best-performing node is selected from several trained models using validation set data for testing. During model testing, the model parameters are fixed and not updated. Recall, F1 score, mean reciprocal rank (MRR), and normalized depreciation cumulative gain (nDCG) are used as evaluation metrics. Three existing methods are compared, including traditional sparse retrieval models, citation recommendation models, and pre-trained scientific document models: 1. Traditional sparse retrieval model: BM25: is an efficient and strong baseline model representing traditional sparse retrieval models. 2. Citation recommendation model: Citeomatic: uses a shallow feedforward network to learn the representation of paper content and metadata, and introduces a contrastive learning objective for training. Transformer: uses a hierarchical Transformer layer as a paper encoder in the prefetching stage, and provides the concatenation of the query paper and candidate papers to SciBERT for re-ranking. The number of candidate papers in the re-ranking stage is limited to 2,000. III. Pre-trained Scientific Paper Models: BERT: The dominant pre-trained model, achieving great success on various language understanding tasks. BERT is pre-trained using a Transformer encoder through Mask Language Model (MLM) and Next Sentence Prediction (NSP) tasks. SciBERT: A variant of BERT trained on a corpus of scientific articles, aiming at masked language modeling. SPECTER: A SciBERT-based scientific paper encoder trained with contrastive learning as its objective, minimizing the L2 distance between citation-cited paper pair embeddings. CiteBERT: A variant of SciBERT fine-tuned for citation value detection. LinkBERT: A variant of BERT fine-tuned for document relation (linked, random, continuous) prediction. SciNCL: A SciBERT-based encoder that uses paper citation graph embeddings to select more informative negative examples, with the same contrastive learning objective as SPECTER. Finally, the evaluation results are shown in the table below:

[0077]

[0078] As shown in the table above, the method in this embodiment outperforms all other baseline methods across all metrics in both versions of CitationR. These improvements are attributed to the Transformer-based pre-trained language model pre-trained on a large corpus, the citation parser's partial mining of relationships between input papers and their cited papers, and the contrastive learning objective for effectively training the model. These results validate the effectiveness of the method in this embodiment compared to other state-of-the-art methods.

[0079] In addition, ablation experiments were conducted on the paper vector representation model of the method in this embodiment, and the results are shown in the table below:

[0080]

[0081] In the table above, Ours refers to the paper vector representation model in the method of this embodiment.

[0082] In the ablation experiments, this embodiment investigates the effectiveness of different components of the method by removing them. First, the effect is demonstrated by removing the citation parser. In fact, regardless of the text encoder chosen, the citation parser is a key structural difference between the method of this embodiment and traditional citation recommendation models. The results in the table show that the citation parser plays a crucial role in the method of this embodiment, indicating the necessity of modeling cited papers to mine citation patterns and preferences in the task of this embodiment. During model training, this embodiment also removes hard negative examples, using only randomly selected papers as easy negative examples. The performance degradation validates the benefit of training the model to distinguish hard negative examples. However, removing both components simultaneously does not significantly worsen the performance compared to the model that only removes hard negative examples. A potential reason is that these hard negative examples are addressed to some extent by the citation parser. To further examine the effectiveness of the citation parser, this embodiment adds additional information about cited papers in the reference section to two baselines: BM25 (the only non-embedded model) and SciNCL. For BM25, adding citations improves performance across all metrics, simply because the titles in the citations provide more text for keyword extraction and matching. However, for SciNCL, directly appending titles from the references after the paper content leads to a performance degradation across all metrics. This is because simple appending fails to uncover the relationships between the input paper, cited papers, and the standard paper, a problem partially addressed by the citation parser proposed in this embodiment.

[0083] Experimental results of different sampling strategies are as follows Figure 5As shown. The strategy here refers to the number of sample papers p, positive sample papers p+, and negative sample papers p- in the training instance. Considering the total number of papers and the average number of positive examples per paper (the average number of positive sample papers p+) of approximately 2.5, this embodiment sets the total number of papers in an instance at a relatively low level during the experiments. The best results were achieved across all metrics when all paper counts were set to 1. Increasing the number of sample papers of any type in the training instance slightly degrades performance, which may be related to the current size of the CitationR dataset.

[0084] In summary, the knowledge-point-oriented scientific and technological resource recommendation method in this embodiment first uses a paper encoder to encode the input scientific and technological paper and the papers in its references, obtaining vector representations of the paper content and the references. Then, a citation parser is used to mine potential citation preferences, obtaining vector representations of the references of the input scientific and technological paper. Next, the content vector and the reference vector are concatenated to obtain the vector representation of the scientific and technological paper. Finally, based on the obtained paper vectors, the similarity between papers is calculated, the loss function during training is calculated, negative sample sampling is performed, and the paper most similar to the input paper is selected as the recommendation result. Addressing the shortcomings of existing citation recommendation technologies, such as insufficient accuracy and inability to effectively handle the existing reference lists of the input paper, this knowledge-point-oriented scientific and technological resource recommendation method can improve the accuracy of recommending missing citations in submitted papers by combining the paper content text and existing citations. It has the advantages of high recommendation accuracy and high citation recommendation efficiency, and can recommend important scientific and technological papers that are missing but need to be cited for a given scientific and technological paper, providing researchers with comprehensive scientific and technological resource recommendations that meet their learning needs and knowledge points.

[0085] Furthermore, this embodiment also provides a knowledge-point-oriented technology resource recommendation system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the knowledge-point-oriented technology resource recommendation method. This embodiment also provides a computer-readable storage medium storing a computer program for being programmed or configured by the microprocessor to execute the knowledge-point-oriented technology resource recommendation method.

[0086] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for recommending science and technology resources based on knowledge points, characterized in that, include: S101, taking the submitted papers and candidate papers in the paper candidate set as target papers, calculate the final vector of the target papers using the paper vector representation model respectively; The paper vector representation model includes: A text encoder is used to extract knowledge points from the title T and abstract A of the input target paper, as well as the titles T of n references of the target paper, and encode them into corresponding hidden states. Linear layers are used to linearly process the hidden states to obtain a vector representation of the target paper. and the title vector representation of n references ~ ; A citation parser is used to convert the target paper into a vector representation. and the title vector representation of n references ~ Based on the cross-attention mechanism, potential citation preferences and potentially overlooked knowledge points are mined to obtain the reference vector representation. This includes: S201, calculating the cross-attention weight vector of the attention layer according to the following formula. : ， In the above formula, The softmax activation function and the cross-attention weight vector are... Any of the following dimensional elements For the first Vector representation of the title of a reference The weights; S202, the title vectors of the n references are represented according to the following formula. ~ By introducing an attention layer, a reference vector representation can be obtained. : ， In the above formula, Cross-attention weight vector Any of the following dimensional elements, For the first The title vector representation of each reference, where n is the number of references; The concatenation layer is used to embed the vector representation of the target paper. and reference vector representation The concatenated vector serves as the final vector of the target paper. S102, calculate the similarity between the final vectors of the submitted paper and each candidate paper respectively; S103, Sort the candidate papers according to similarity, and select a specified number of candidate papers as the recommended results for submission in order of similarity from high to low. Before step S101, a paper vector representation model is trained using sample paper p, positive sample paper p+, and negative sample paper p-. Positive sample paper p+ refers to papers that match the knowledge points of sample paper p, and negative sample paper p- refers to papers that are not cited by sample paper p or recommended for citation by reviewers. The loss function used to train the paper vector representation model is the triplet marginal loss function, the expression of which is: ， In the above formula, Let be the marginal loss function of the triple. To obtain the maximum value, The final vector obtained by using the paper vector representation model for sample paper p. The final vector obtained by using the vector representation model of the paper is the positive sample paper p+. For the negative sample paper p, the final vector obtained using the paper's vector representation model is... To calculate similarity, This is the loss margin hyperparameter.

2. The method for recommending knowledge-point-oriented science and technology resources according to claim 1, characterized in that, The text encoder extracts knowledge points from the title T and abstract A of the input target paper and encodes them into corresponding hidden states. This process involves: first, concatenating the title T and abstract A of the target paper, inserting sentence separators between them, and then inserting a preorder tagged tag at the beginning before inputting the data into the BERT encoder to obtain a series of hidden states. ， In the above formula, Preorder tag The corresponding hidden state, for The corresponding hidden state, for The corresponding hidden state, For BERT encoder, Adding preorder tags to the front of the sequence for the BERT encoder , ~ These are the first to i words in title T, respectively. Add sentence separator tags to the BERT encoder , ~ Let A be the j-th word of the abstract A. The BERT encoder generates a corresponding hidden state for each input word or token; then, it generates a hidden state for the preceding token. Corresponding hidden state Input a linear layer to obtain a vector representation of the target paper. .

3. The method for recommending knowledge-point-oriented science and technology resources according to claim 1, characterized in that, When a text encoder extracts knowledge points from the titles T of n references in an input target paper and encodes them into corresponding hidden states, it can perform a hidden state operation for any nth reference in the target paper. The coding of the references includes: first, the first... The title T of each reference is inserted with a sentence separator at the end and a preorder tagged at the beginning before being input into the BERT encoder to obtain a series of hidden states: ， In the above formula, For the first Preface markers of the references The corresponding hidden state, ~ The first The first to last reference The hidden states corresponding to each word For BERT encoder, Adding preorder tags to the front of the sequence for the BERT encoder , ~ The first The first to last reference One word, Add sentence separator tags to the BERT encoder Then the preceding tag Corresponding hidden state Input a linear layer and get the first... Vector representation of the title of a reference .

4. The method for recommending knowledge-point-oriented science and technology resources according to claim 1, characterized in that, The splicing layer represents the vector of the target paper. and reference vector representation The expression for the final vector of the target paper after concatenation is: ， In the above formula, For the final vector of the target paper, This represents the transpose operation of a vector.

5. The method for recommending science and technology resources based on knowledge points according to claim 1, characterized in that, When using sample paper p, positive sample paper p+ and negative sample paper p- to train the paper vector representation model, the sources of negative sample paper p- include: (1) papers randomly selected from the corpus; (2) for sample paper p and its candidate negative sample papers, the similarity between the final vectors of sample paper p and each candidate negative sample paper is calculated respectively, and the candidate papers are sorted according to the similarity, excluding a specified number of candidate negative sample papers with the highest similarity, and selecting one or more candidate negative sample papers with the highest similarity from the remaining candidate negative sample papers.

6. A knowledge-point-oriented technology resource recommendation system, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the knowledge-point-oriented technology resource recommendation method according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, The computer program is used to be programmed or configured by a microprocessor to execute the knowledge-point-oriented scientific and technological resource recommendation method according to any one of claims 1 to 5.

Images