Scientific researcher literature information retrieval and evaluation method and device and storage medium
By constructing an associated database and evaluating citation information using large language models, the problem of low statistical efficiency of scientific researchers' literature citation status is solved, automated literature citation analysis and evaluation is realized, and the statistical efficiency of scientific research results is improved.
Patent Information
- Application Number
- CN202410882765.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The statistical efficiency of scientific researchers' literature citations is low. With the increase in the number of documents and the complexity of citation relationships, it is difficult for existing methods to conduct efficient statistics.
By obtaining the information of scientific researchers, building an associated database, using a large language model to evaluate the citation status of cited documents based on cited information, generate evaluation results, and avoid manual statistics.
It has improved the statistical efficiency of scientific research results, realized automated analysis and evaluation of literature citations, and improved the efficiency of literature evaluation.
Smart Images

Figure CN119088762A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device and storage medium for searching and evaluating document information of scientific researchers. Background Art
[0002] The citation of a researcher's academic achievements in his or her field is an important indicator for evaluating the researcher's work.
[0003] At present, the citation status of scientific researchers' literature is generally obtained through manual search and statistics. However, as the number of scientific researchers' literature increases and / or the complexity of the citation relationship of the literature increases, the statistical difficulty will also increase. At this time, the statistical efficiency of scientific researchers' citation status is low. Summary of the invention
[0004] In view of this, the present invention proposes a method, device and storage medium for retrieving and evaluating the literature information of scientific researchers, which can automatically analyze the citation status of the cited literature of the scientific researcher based on the personnel information of the scientific researcher without the need for manual search and statistics, thereby improving the statistical efficiency of the scientific research results of the scientific researcher.
[0005] According to one aspect of the present disclosure, a method for searching and evaluating scientific researcher literature information is provided, the method comprising:
[0006] Obtaining personnel information of target researchers;
[0007] Based on the personnel information, an associated database of the target scientific researcher is obtained; wherein the associated database includes first document information of at least one cited document and second document information of a citing document corresponding to each cited document, wherein the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document;
[0008] Acquiring citation information of each cited document in a corresponding citing document based on the associated database; wherein the citation information includes a citation mark and / or a citing text segment;
[0009] The citation information is input into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain citation situation information; the large language model learns in advance the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and a citation situation information label corresponding to the citation information template;
[0010] An evaluation result of the cited document is generated based on the citation information.
[0011] In a possible implementation, the inputting the citation information into a preset large language model so that the large language model evaluates the citation status of the cited document based on the citation information to obtain the citation status information includes:
[0012] The reference information and the preset prompt word are input into the large language model, so that the large language model first learns the ability to evaluate the reference information according to the preset prompt word, and then evaluates the reference information according to the learned ability to obtain the reference situation information.
[0013] In a possible implementation, the method further includes:
[0014] Obtaining a document evaluation requirement; wherein the document evaluation requirement includes the type of the citation situation, and correspondingly, the citation situation information includes the type of the citation situation;
[0015] The preset prompt word is constructed based on the type of the reference situation.
[0016] In a possible implementation, obtaining the citation information of each cited document in the corresponding citing document based on the associated database includes:
[0017] Based on each second document information in the associated database, directing to a web page of the cited document, the web page is used to display the full text of the cited document;
[0018] Searching the web page for first document information corresponding to the second document information;
[0019] Based on the first document information, citation information of the cited document in the citing document is obtained.
[0020] In a possible implementation, the first document information includes the title of the cited document; accordingly,
[0021] The searching the web page for the first document information corresponding to the second document information includes:
[0022] Searching the web page element where the title of the cited document is located on the web page based on an accurate search method;
[0023] In the case where the web page element is not found, searching the web page element where the title of the cited document is located based on a fuzzy search method to obtain a fuzzy result;
[0024] The fuzzy result is used as the title of the cited document, and the web page element where the fuzzy result is located is searched in the web page based on the precise search method to obtain the first document information in the citing document.
[0025] In a possible implementation, obtaining the citation information of the cited document in the citing document based on the first document information includes:
[0026] In the citing document, identifying a reference mark corresponding to the title of the cited document;
[0027] Searching the citing document for the first citing paragraph including the citation mark;
[0028] In the citation segment, n citation segments before the first citation segment and m citation segments after the first citation segment are obtained to obtain a second citation segment; wherein the reference information includes the reference mark, the first citation segment and the second citation segment, and m and n are both natural numbers.
[0029] In a possible implementation, generating the evaluation result of the cited document based on the citation information includes:
[0030] Based on the output requirement of the evaluation result in the literature evaluation requirement, the citation information is modified to obtain the evaluation result.
[0031] In a possible implementation, the citation information includes the type of citation and the analysis process of the type of citation; accordingly, the output requirement includes: outputting the type of citation without outputting the analysis process of the citation information; and modifying the citation information according to the output requirement to obtain the evaluation result includes:
[0032] The analysis process in the citation information is deleted to obtain the evaluation result.
[0033] According to another aspect of the present disclosure, another method for searching and evaluating literature information of scientific researchers is provided, the method comprising:
[0034] Display the information retrieval interface;
[0035] Receive the personnel information of the target scientific researcher through the information retrieval interface to obtain the associated database of the target scientific researcher based on the personnel information; and obtain the citation information of each cited document in the corresponding citing document based on the associated database; input the citation information into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain the citation situation information; generate the evaluation result of the cited document based on the citation situation information; wherein the large language model pre-learns the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and the citation situation information label corresponding to the citation information template; the associated database includes first document information of at least one cited document and second document information of the citing document corresponding to each cited document, the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document; the citation information includes citation marks and / or citing paragraphs;
[0036] The evaluation result is displayed on the information retrieval interface.
[0037] According to another aspect of the present disclosure, there is provided a device for retrieving and evaluating literature information for scientific researchers, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0038] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.
[0039] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0040] By obtaining the personnel information of the target scientific researcher; obtaining the associated database of the target scientific researcher based on the personnel information, the associated database includes the first document information of at least one cited document and the second document information of the citing document corresponding to each cited document; obtaining the citation information of each cited document in the corresponding citing document based on the associated database; inputting the citation information into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain citation situation information; the large language model pre-learns the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; generating the evaluation results of the cited document based on the citation situation information; allowing the user to obtain the evaluation results of each cited document corresponding to the personnel information by only inputting the personnel information, without the need for manual statistics by the user, thereby avoiding the problem of low efficiency of manual statistics; improving the efficiency of document evaluation for scientific researchers.
[0041] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0043] Figure 1 A flow chart showing a method for retrieving and evaluating scientific research personnel's literature information provided according to one embodiment of the present application;
[0044] Figure 2 A schematic diagram showing the titles of cited documents in citing documents provided according to one embodiment of the present application;
[0045] Figure 3 A schematic diagram showing the titles of cited documents in citing documents provided according to another embodiment of the present application;
[0046] Figure 4 A schematic diagram showing a reference mark of a cited document in a citing document provided according to an embodiment of the present application;
[0047] Figure 5 A schematic diagram showing a first citation paragraph provided according to an embodiment of the present application;
[0048] Figure 6 A flow chart showing a method for retrieving and evaluating scientific research personnel's literature information provided in accordance with another embodiment of the present application;
[0049] Figure 7 A schematic diagram showing an information retrieval interface provided according to an embodiment of the present application;
[0050] Figure 8A block diagram of a scientific researcher document information retrieval and evaluation device provided according to an embodiment of the present application is shown;
[0051] Fig. 9 A block diagram of a device for retrieving and evaluating literature information for researchers provided according to one embodiment of the present application is shown. DETAILED DESCRIPTION
[0052] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0053] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0054] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.
[0055] Figure 1 The flowchart of the method for retrieving and evaluating scientific research personnel's literature information according to one embodiment of the present disclosure is shown. This embodiment takes the method used in an electronic device with processing and computing capabilities as an example for explanation. The electronic device can be a user terminal or a server, wherein the user terminal includes but is not limited to: a computer, a tablet computer, a mobile phone, etc. This embodiment does not limit the type of electronic device. Figure 1 As shown, the method includes:
[0056] Step 101, obtaining the personnel information of the target scientific researcher.
[0057] Personnel information is used to indicate the identity of the target scientific researcher. In one example, personnel information includes the name of the target scientific researcher. Due to the possibility of other scientific researchers with the same name as the target scientific researcher, based on this, optionally, personnel information includes other information of the target scientific researcher in addition to the name of the target scientific researcher. At this time, the combination of multiple personnel information can uniquely indicate the target scientific researcher, and other information includes but is not limited to the affiliated institution of the target scientific researcher, and / or the affiliated field, etc., and the present embodiment does not limit the implementation of other information. Or, personnel information includes the identification code of the target scientific researcher, and the identification code is used to uniquely indicate the target scientific researcher. For example: the identification code is: the personnel database identification code (ID) of the target scientific researcher, and / or the identity card number, etc., and the present embodiment does not limit the implementation of the identification code. In other embodiments, personnel information can also be a combination of the name of the target scientific researcher and the identification code of the target scientific researcher, and the present embodiment does not limit the implementation of personnel information.
[0058] In an example, the personnel information may be received through a human-computer interaction control, or may be sent by other devices. This embodiment does not limit the method for obtaining the personnel information.
[0059] Step 102, obtaining a related database of target researchers based on the personnel information; wherein the related database includes first document information of at least one cited document and second document information of a citing document corresponding to each cited document.
[0060] The cited document refers to the scientific research document of the target researcher, and the citing document refers to other scientific research documents that cite the cited document. In this embodiment, the citing document corresponding to the cited document refers to the citing document that cites the cited document.
[0061] Optionally, the scientific research literature of the target researcher refers to literature whose authors include the target researcher, or literature whose first author is the target researcher.
[0062] In one example, obtaining an associated database of a target scientific researcher based on personnel information includes: searching for cited document information of the personnel information in a preset document citation database to obtain first document information; searching for citing document information corresponding to the cited document information in the document citation database to obtain second document information; and constructing an associated database based on the first document information and the second document information. In the associated database, each first document information corresponds to at least one second document information to indicate that the citing document corresponding to the second document information cites the cited document corresponding to the first document information. Among them, the method of searching for the first document information in the document citation database may be: automatically matching the author information stored in the document citation database with the personnel information one by one; reading the first document information corresponding to the author information matching the personnel information. The method of searching for the second document information in the document citation database may be: after determining the first document information, obtaining the second document information corresponding to the first document information through the citation index provided by the document citation database. That is, searching for the first document information and the second document information in the document citation database does not require manual search, but is automatically searched by the electronic device.
[0063] Wherein, the document citation database can be an existing database (such as Web of Science, and / or Scopus, etc.), or it can also be a document data set obtained by crawler technology. This embodiment does not limit the acquisition method of the document citation database. The document citation database has relevant information of multiple documents in the target field, as well as relevant information of other documents cited by the document. Wherein, the relevant information includes but is not limited to: the title of the document, personal information, and / or digital object identifier (DOI). The document citation database may also include other information, such as publication date, etc. This embodiment does not limit the content of the relevant information of the document in the document citation database. The number of document citation databases can be one or two, and this embodiment does not limit the selection of the document citation database.
[0064] The target field at least includes the scientific research field to which the target researcher belongs. In other embodiments, the target field may also include other fields, for example, the target field may also be all fields. The type of target field is related to the type of literature citation database. This embodiment does not limit the implementation method of the target field.
[0065] Step 103, obtaining the citation information of each cited document in the corresponding citing document based on the associated database.
[0066] The citation information includes citation marks and / or citation paragraphs. A citation mark is a mark used to indicate a cited document in a citing document, for example, a citation mark is an identification code obtained by numbering each document cited in the citing document. A citation paragraph is a paragraph in a citing document that is related to the cited document.
[0067] Obtain the citation information of each cited document in the corresponding citing document based on the linked database, including but not limited to one of the following methods:
[0068] The first method is: based on each second document information in the associated database, direct to the web page of the citing document, which is used to display the full text of the citing document; search for the first document information corresponding to the second document information in the web page; based on the first document information, obtain the citation information of the cited document in the citing document.
[0069] In a first implementation, the first document information is used to search for cited documents in citing documents. Schematically, the first document information includes the title of the cited document. In other embodiments, the first document information may also include the publication date and / or author name of the cited document, etc. This embodiment does not limit the implementation of the first document information.
[0070] The second document information corresponds to the document resource of the citing document one by one, that is, the document of the citing document can be accessed through the second document information. Schematically, the second document information includes the DOI of the citing document. In other embodiments, the second document information can also be the uniform resource locator (URL) of the citing document. This embodiment does not limit the implementation method of the second document information.
[0071] Accordingly, the linked database includes the titles of all cited documents of the target researcher, the title of the citing document corresponding to each cited document, and the DOI of each citing document.
[0072] Schematically, the electronic device uses a loop structure to traverse the associated database, and each loop body performs the following steps: by querying the DOI in the (cited_title, DOI) pair, it is directed to the web page of the citing document, and the first document information of the cited document indicated by cited_title is found on the web page; based on the first document information, the citation information of the cited document in the citing document is obtained. At this time, DOI is the second document information of the citing document, and cited_title is the first document information of the cited document (i.e., the title of the cited document).
[0073] In one example, when the first document information includes the title of the cited document, searching for the first document information corresponding to the second document information in a web page includes: searching for the web page element where the title of the cited document is located based on a precise search method; when the web page element is not found, searching for the web page element where the title of the cited document is located based on a fuzzy search method to obtain a fuzzy result; using the fuzzy result as the title of the cited document, searching for the web page element where the fuzzy result is located based on a precise search method in the web page to obtain the first document information in the citing document.
[0074] Optionally, when a web page element is found based on the precise search method, the electronic device can directly obtain the first document information in the citing document indicated by the web page element.
[0075] In this embodiment, the first document information in the citing document is searched by combining the precise search method with the fuzzy search method, which can avoid the problem of failure to find the first document information due to network loading delays, inconsistent web page layout formats, and / or text errors, thereby ensuring the success rate of the reference information search.
[0076] The precise search method includes a full character matching method, that is, searching for web page elements that match all characters in the first document information in the citing document. In other embodiments, the precise search method may also be other methods such as regular expression matching, and this embodiment does not limit the implementation method of the precise search method.
[0077] Optionally, the web page element where the title of the cited document is located is searched based on a fuzzy search method to obtain a fuzzy result, including: obtaining a citation title group consisting of candidate titles of all documents cited by the citing document; determining the similarity between each candidate title in the citation title group and the title of the cited document, and the candidate title with the highest similarity is the fuzzy result.
[0078] The candidate titles of all documents cited by the citing document are generally located at the end of the full text of the citing document. Accordingly, all titles at the end of the document are read to obtain the candidate titles.
[0079] Optionally, the method of determining the similarity between the candidate title and the title of the cited document includes but is not limited to: determining based on Levenshtein distance or determining based on Hamming distance. This embodiment does not limit the similarity calculation method.
[0080] For example, the title of the cited document in the first document information is: "Analytical model for axially compressed circular concrete-filled double skin steel tubes (CFDSTs): Insights from concrete non-uniformly confined states". The web page reference of the citing document Figure 2 As shown, according to Figure 2 From the box content in the reference, we can see that the "I" in Insights is lowercase "i". At this time, the web page has capitalization errors due to page layout errors, which will cause the precise search method to fail to find web page elements. In this embodiment, after changing to fuzzy search, we can get Figure 2 The similarity of the part shown in is the highest with the first document information above, then Figure 2 The title content shown is used as the search target, and the search target is searched in the full text of the citing literature:
[0081] "Analytical model for axially compressed circular concrete-filled double skin steel tubes (CFDSTs): insights from concrete non-uniformly confined states".
[0082] For example, the title of the cited document in the first document information is: "Innovations in a submarinepiggyback pipeline project in the East China Sea." The web page reference of the citing document Figure 3 As shown, according to Figure 3 The blank content in the box indicates that the title of the cited document in the citing document lacks a period "." at the end of the sentence. In this case, the precise search method will fail to find the web page element. In this embodiment, after changing to fuzzy search, we can get Figure 3 The similarity of the part shown in is the highest with the first document information above, then Figure 3 Use the title content in as the search target and search for the search target in the full text of the citing document:
[0083] “Innovations in a submarine piggyback pipeline project in the EastChina Sea”.
[0084] In other embodiments, the electronic device may also directly retrieve web page elements based on a fuzzy search method, that is, the electronic device obtains a citation title group consisting of candidate titles of all documents cited by the citing document; determines the similarity between each candidate title in the citation title group and the title of the cited document; searches the web page for the web page element where the candidate title with the highest similarity is located based on a precise search method to obtain the first document information in the citing document; or, only uses a precise search method to retrieve web page elements. This embodiment does not limit the search method for web page elements.
[0085] Optionally, the electronic device also provides a search method selection control for the user to select whether to disable the fuzzy search method and / or whether to disable the precise search method. Accordingly, the electronic device retrieves web page elements based on the search method selected by the user.
[0086] In one example, based on the first document information, the citation information of the cited document in the citing document is obtained, including: in the citing document, identifying the citation mark corresponding to the title of the cited document; searching for the first citing text segment including the citation mark in the citing document; in the citing text segment, obtaining n citing text segments before the first citing text segment and m citing text segments after the first citing text segment to obtain a second citing text segment.
[0087] The citation information includes a citation mark, a first citation paragraph and a second citation paragraph, and m and n are both natural numbers.
[0088] Generally, the citation mark is placed at the beginning or end of the title of the cited document, such as: Figure 2 The citation mark is located at the beginning of the title of the cited document, and the citation mark is
[37] .
[0089] refer to Figure 4 After the electronic device finds the first document information in the citing document, it obtains the preset string at the beginning of the paragraph of the first document information to obtain the reference mark "[1]", and then searches the citing document for the first citing paragraph where the reference mark "[1]" is located to obtain Figure 5 The first citation paragraph is shown.
[0090] Among them, the preset form is pre-stored in the electronic device, and the preset form is determined based on the setting form of the reference mark. For example, the reference mark is generally implemented by "[]" and Arabic numerals located in the "[]". The electronic device obtains a character string that conforms to the preset form to obtain the reference mark.
[0091] Optionally, searching for the first citing paragraph including the citation mark in the citing document includes searching for the first citing paragraph including the citation mark in the content of the citing document other than the reference document. In this way, it is possible to avoid analyzing the title of the cited document as the first citing paragraph.
[0092] Optionally, the values of m and n are stored in the electronic device by default, for example: m=n=1; or, the values of m and n are set by the user, and this embodiment does not limit the values of m and n and the way of setting the values.
[0093] The second method is to read detailed information including the first document information and the second document information from the detailed information of the reference relationship of each document stored in the associated database, and obtain the reference information in the detailed information. In this case, the associated database can provide detailed information of the reference relationship of each document, and the detailed information includes the document information of the document, the document information of other documents cited by the document, and the reference information of other documents cited by the document. Accordingly, the electronic device reads the detailed information including the first document information and the second document information from the detailed information, and obtains the reference information in the detailed information.
[0094] At this time, the first document information is used to uniquely indicate the cited document, and the second document information is used to uniquely indicate the citing document. Schematically, the first document information may be a combination of the title, author, and publication date of the cited document, and the second document information may be a combination of the title, author, and publication date of the citing document. This embodiment does not limit the implementation of the first document information and the second document information.
[0095] Step 104, input the citation information into a preset large language model, so that the large language model (LLMs) evaluates the citation status of the cited document based on the citation information to obtain citation status information.
[0096] The large language model learns in advance the ability to evaluate the reference information and obtain the reference situation information based on preset prompt words; the preset prompt words include at least one reference information template and a reference situation information label corresponding to each reference information template.
[0097] In this embodiment, the large language model is a type of artificial intelligence model trained with a large amount of text data. The model can capture and understand complex patterns and structures in language and generate language required by humans. The large language model has the ability of few-shot learning, that is, the ability to effectively learn even with only a small number of labeled samples. Based on this ability, the large language model can learn to evaluate citation information and obtain citation information based on preset prompt words.
[0098] Optionally, the large language model is a trained general language model, such as: CHATGLM3 model, GPT-3, etc.; or, the large language model can also be a new model trained using a large amount of text data. This embodiment does not limit the method for obtaining the large language model.
[0099] In one example, citation information is input into a preset large language model so that the large language model evaluates the citation status of the cited document based on the citation information to obtain citation status information, including: inputting the citation information and preset prompt words into the large language model so that the large language model first learns the ability to evaluate citation information based on the preset prompt words, and then evaluates the citation information based on the learned ability to obtain citation status information.
[0100] In other words, the preset prompt words are used to provide a small sample for the large language model so that the large language model can learn the ability to evaluate the citation information and obtain the citation situation information.
[0101] Optionally, the preset prompt words are constructed based on the user's document evaluation requirements. In this case, before the citation information and the preset prompt words are input into the large language model, the method further includes: obtaining the document evaluation requirements; wherein the document evaluation requirements include the type of citation situation, and correspondingly, the citation situation information includes the type of citation situation; and constructing the preset prompt words based on the type of citation situation, a citation information template related to the type of citation situation, and a citation situation information label corresponding to the citation information template.
[0102] The type of citation refers to the type of citation of the cited document by the citing document, which can be set by the user according to the document evaluation requirements. For example, the types of citation include simple citation and in-depth evaluation; another example is that the types of citation include positive evaluation and negative evaluation. In actual implementation, the type of citation can be set to other types or more types according to user requirements. This embodiment does not list the types of citations one by one.
[0103] Optionally, the preset prompt words can all be constructed manually by the user, or the type of citation situation, the citation information template related to the type of citation situation, and the citation situation information tag corresponding to the citation information template can be input into a preset prompt word template to obtain the preset prompt words.
[0104] The reference information template is the sample data input into the large language model when training the large language model based on the small sample learning ability, and the reference situation information label is the label of the sample data. In this embodiment, the type marked by the reference situation information label is consistent with the type of the reference situation. The prompt word template refers to the text content that can be reused when constructing different preset prompt words. The prompt word template generally includes fixed text that is required to be used when constructing different preset prompt words and a position reserved for variable text, wherein the variable text changes with different preset prompt words, such as: the variable text includes the type of the reference situation.
[0105] For example, taking the types of citations as simple citations and in-depth evaluations, the preset prompt words constructed are as follows:
[0106] “I need you to answer a question now, but before that, I will first provide you with two sets of examples. These two examples describe in detail the different citation states of a particular citation (composed of a number + square brackets, or a round bracket + author's surname + publication year in the text): simple mention and in-depth evaluation.
[0107] First of all, please carefully understand the examples I provide you. Please note that the content of the examples is not directly related to the problem. The examples exist to help you understand the problem.
[0108] Then please answer a question. Please refer to the two sets of examples I gave you to answer this question:
[0109] example:
[0110] 1. Examples briefly mentioned:
[0111] Consequently, innovative designs have emerged, including concrete-encased concrete-filled steel tube (CECFST) members [5], [6], concrete-filleddouble-skin steel tube (CFDST) members [7], [8], [9], and hollow steel-reinforcedconcrete (HSRC) members
[10] ,
[11] . Here the literature [9] is briefly mentioned;
[0112] Concrete-filled double skin steel tube (CFDST) consists of inner and outer concentrically placed steel tubes, with concrete infilled in the sandwiched cavity [1], [2]. Reference [1] is briefly mentioned here;
[0113] Since the heave motion of floating structure is comparatively significant for all seas (Wang et al., 2017; Xu et al., 2020) and would increase the stress concentration in the sag bend and hog bend regions of lazy waveriser (Szczepanski et al., 2018), a simple harmonic motion along y-axis direction is imposed on the top of the riser to simulate the heave motion offfloating structure in this study. The literature (Wang et al., 2017) is briefly mentioned here.
[0114] 2. Examples of in-depth reviews:
[0115] Recently, their possible applications in submarine pipeline structures have also been explored[3],[4]. Here the literature[3] is reviewed in depth;
[0116] Wang et al.
[10] investigated the compressive and flexural behaviors of carbon steel–concrete-stainless steel (CCS) composite pipeline subjected to internal and external pressures. Here, reference
[10] is evaluated in depth;
[0117] In addition, as an alternative solution to pipeline installation planning and decision making, a real-time offshore pipeline installation monitoring system was developed (Wang et al., 2017). Here the literature (Wang et al., 2017) was evaluated in depth.
[0118] question:
[0119] In the following passage, is the reference to literature [{number}] simply mentioned or evaluated in depth? (Please note that you only need to answer “briefly mentioned” or “in-depth evaluation”)
[0120] {text}
[0121] Your answer:
[0122] The citations for document [{number}] are:".
[0123] Among them, number refers to the quotation mark, and text refers to the quoting text.
[0124] The citation information template includes the example 1 “Consequently, innovative designs have emerged, including concrete-encased concrete-filled steel tube (CECFST) members [5], [6], concrete-filled double-skin steel tube (CFDST) members [7], [8], [9], and hollow steel-reinforced concrete (HSRC) members
[10] ,
[11] .” Accordingly, the citation information tag corresponding to the citation information template is “Here document [9] is briefly mentioned” located after the example.
[0125] Similarly, the citation information template also includes "Concrete-filled double skin steeltube(CFDST)consists of inner and outer concentrically placed steel tubes,withconcrete infilled in the sandwiched cavity[1],[2]" in Example 1. Accordingly, the citation information tag corresponding to the citation information template is "The document [1] is briefly mentioned here" after the example.
[0126] The reference information templates in other examples and the reference information tags corresponding to each reference information template will not be described separately one by one in this embodiment.
[0127] Illustratively, when the preset prompt word is constructed based on a prompt word template, the prompt word template corresponding to the preset prompt word can be implemented in the following manner:
[0128] "I need you to answer a question now, but before that, I will first provide you with two sets of examples. These two examples describe in detail the different citation states of a certain cited document (composed of a serial number + square brackets, or a round bracket + author's surname + publication year in the text): Add the type of reference here .
[0129] First of all, please carefully understand the examples I provide you. Please note that the content of the examples is not directly related to the problem. The examples exist to help you understand the problem.
[0130] Then please answer a question. Please refer to the answer I gave you. Add the number of examples here Example answer:
[0131] example:
[0132] Add the reference information template related to the reference situation type and the reference situation corresponding to the reference information template. Status information label.
[0133] question:
[0134] In the following passage, is the reference to literature [{number}] simply mentioned or evaluated in depth? (Please note that you only need to answer “briefly mentioned” or “in-depth evaluation”)
[0135] {text}
[0136] Your answer:
[0137] The citations for document [{number}] are:".
[0138] Among them, the underline indicates the information to be filled in the prompt word template, and this part of information is obtained through the human-computer interaction control; number refers to the reference mark, and text refers to the citing paragraph. These contents are the variable text. The reusable text other than the variable text is the fixed text.
[0139] After obtaining the preset prompt word, the reference information and the preset prompt word are input into the large language model, including: filling the reference information into the preset position of the preset prompt word to obtain input information, and inputting the input information into the large language model. The preset position is a position reserved for the reference information in the preset prompt word, such as {number} and {text} in the above example.
[0140] For example, the citation information includes the citation mark and the citation paragraph. The citation mark is: 31; the citation paragraph is "Unlike the numerous experimental and numerical investigations reported earlier, only two analytical studies (Cheng et al.
[31] and Li et al.)". When evaluating the citation information, the citation information will automatically replace the preset prompt words number and text in the above example, and the input information obtained is as follows:
[0141] “I need you to answer a question now, but before that, I will first provide you with two sets of examples. These two examples describe in detail the different citation states of a particular citation (composed of a number + square brackets, or a round bracket + author's surname + publication year in the text): simple mention and in-depth evaluation.
[0142] First of all, please carefully understand the examples I provide you. Please note that the content of the examples is not directly related to the problem. The examples exist to help you understand the problem.
[0143] Then please answer a question. Please refer to the two sets of examples I gave you to answer this question:
[0144] example:
[0145] 1. Examples briefly mentioned:
[0146] Consequently, innovative designs have emerged, including concrete-encased concrete-filled steel tube (CECFST) members [5], [6], concrete-filleddouble-skin steel tube (CFDST) members [7], [8], [9], and hollow steel-reinforcedconcrete (HSRC) members
[10] ,
[11] . Here the literature [9] is briefly mentioned;
[0147] Concrete-filled double skin steel tube (CFDST) consists of inner and outer concentrically placed steel tubes, with concrete infilled in the sandwiched cavity [1], [2]. Reference [1] is briefly mentioned here;
[0148] Since the heave motion of floating structure is comparatively significant for all seas (Wang et al., 2017; Xu et al., 2020) and would increase the stress concentration in the sag bend and hog bend regions of lazy waveriser (Szczepanski et al., 2018), a simple harmonic motion along y-axis direction is imposed on the top of the riser to simulate the heave motion offfloating structure in this study. The literature (Wang et al., 2017) is briefly mentioned here.
[0149] 2. Examples of in-depth reviews:
[0150] Recently, their possible applications in submarine pipeline structures have also been explored[3],[4]. Here the literature[3] is reviewed in depth;
[0151] Wang et al.
[10] investigated the compressive and flexural behaviors of carbon steel–concrete-stainless steel (CCS) composite pipeline subjected to internal and external pressures. Here, reference
[10] is evaluated in depth;
[0152] In addition, as an alternative solution to pipeline installation planning and decision making, a real-time offshore pipeline installation monitoring system was developed (Wang et al., 2017). Here the literature (Wang et al., 2017) was evaluated in depth.
[0153] question:
[0154] In the following passage, is the reference to the literature
[31] simply mentioned or evaluated in depth? (Please note that you only need to answer “briefly mentioned” or “in-depth evaluation”)
[0155] Unlike the numerous experimental and numerical investigations reported earlier, only two analytical studies (Cheng et al.
[31] and Li et al.
[0156] Your answer:
[0157] The citation status of document
[31] is: ”.
[0159] Optionally, each time the electronic device evaluates the citation information, the citation information and the preset prompt words are input into the large language model for evaluation. Alternatively, when the electronic device obtains a new document evaluation requirement, after constructing the preset prompt words according to the document evaluation requirement, the citation information and the preset prompt words are input into the large language model during the first evaluation; for the same document evaluation requirement, in the evaluation process after the first evaluation, only the citation information is input into the large language model.
[0160] For example, the citation information includes the citation mark and the citation paragraph. The citation mark is: 31; the citation paragraph is "Unlike the numerous experimental and numerical investigations reported earlier, only two analytical studies (Cheng et al.
[31] and Li et al.), then in the evaluation process after the first evaluation, the information input into the large language model is:
[0161] question:
[0162] In the following passage, is the reference to the literature
[31] simply mentioned or evaluated in depth? (Please note that you only need to answer “briefly mentioned” or “in-depth evaluation”)
[0163] Unlike the numerous experimental and numerical investigations reported earlier, only two analytical studies (Cheng et al.
[31] and Li et al.
[0164] Your answer:
[0165] The citation status of document
[31] is: ”.
[0167] Optionally, if preset prompt words are input for each evaluation for the same document evaluation requirement, the citation information templates and citation information labels for the preset prompt words input in different evaluation processes are the same or different, that is, the small samples input into the large language model in different evaluation processes are the same or different.
[0168] In addition, the reference information template in the preset prompt word and the reference information label corresponding to the reference information template can be one group or at least two groups. It can be understood that the more groups there are, the more accurate the reference information will be output by the large language model. This embodiment does not limit the number of groups.
[0169] Step 105: Generate evaluation results of cited documents based on the citation information.
[0170] In one example, generating an evaluation result of a cited document based on the citation information includes: modifying the citation information based on the output requirement of the evaluation result in the document evaluation requirement to obtain the evaluation result. In this case, the document evaluation requirement includes the output requirement of the evaluation result.
[0171] For example: the citation information includes the type of citation and the analysis process of the citation type; accordingly, the output requirements include: outputting the type of citation without outputting the analysis process of the citation information; changing the citation information according to the output requirements to obtain the evaluation result, including: deleting the analysis process in the citation information to obtain the evaluation result.
[0172] For some large language models, the citation information that may be output may also include the reason for selecting a certain citation type, for example: GPT-3 can generate a text and give the reason for selecting a certain word. However, the user may only need to output the type of citation and does not need to know the reason, that is, the above analysis process. In this embodiment, the citation information output by the large language model can be changed based on the user's output requirements to obtain an evaluation result that meets the user's requirements and improve the accuracy of the evaluation result.
[0173] In other embodiments, the electronic device may also directly output the citation information as the evaluation result. For example, in the absence of an output requirement, the electronic device directly outputs the citation information as the evaluation result.
[0174] For example, based on the example in step 104, after the input information in the example is input into the large language model, the citation information obtained is: simple mention. The electronic device outputs the citation information as the evaluation result.
[0175] In summary, the method for retrieving and evaluating the literature information of scientific researchers provided in this embodiment obtains the personnel information of the target scientific researchers; obtains the associated database of the target scientific researchers based on the personnel information, and the associated database includes the first document information of at least one cited document and the second document information of the citing document corresponding to each cited document; obtains the citation information of each cited document in the corresponding citing document based on the associated database; inputs the citation information into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain citation situation information; the large language model learns in advance the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; generates the evaluation results of the cited documents based on the citation situation information; enables the user to obtain the evaluation results of each cited document corresponding to the personnel information by only inputting the personnel information, without the need for manual statistics by the user, thereby avoiding the problem of low efficiency of manual statistics; improves the efficiency of literature evaluation for scientific researchers.
[0176] In addition, the small sample learning ability based on the large language model is used to evaluate the citation information, which eliminates the need for complex large language model training and improves the efficiency of literature evaluation.
[0177] In addition, by combining precise search and fuzzy search to search for the titles of cited documents in citing documents, it is possible to find the titles of cited documents even when there are delays in network loading, inconsistent web page layout formats, or text errors, thereby ensuring the success rate of obtaining citation information.
[0178] In addition, by processing output characters that may not meet user requirements according to the output requirements in the document evaluation needs, an evaluation result that meets the user requirements can be obtained, thereby improving the accuracy of the evaluation result.
[0179] Figure 6 A flowchart of a method for retrieving and evaluating scientific research personnel's literature information according to another embodiment of the present disclosure is shown. This embodiment is described by taking the method as an example of an electronic device with processing, computing and human-computer interaction capabilities. Figure 1 The electronic devices in the illustrated embodiment are the same electronic device or different electronic devices; if they are different electronic devices, the electronic devices in this embodiment are different from Figure 1 The electronic devices in the illustrated embodiment are connected to each other by communication. Optionally, the electronic devices in this embodiment include but are not limited to: computers, tablet computers, mobile phones, etc., and this embodiment does not limit the type of the electronic devices. Figure 6 As shown, the method includes:
[0180] Step 601, displaying the information retrieval interface.
[0181] In this embodiment, the information retrieval interface is used to retrieve the evaluation results of the scientific research literature (ie, cited literature) of a certain target scientific researcher.
[0182] Optionally, the electronic device displays the information retrieval interface after it is turned on; or, the electronic device runs an information retrieval program after it is turned on, and displays the information retrieval interface in response to a startup instruction of the information retrieval program. The information retrieval program may be a browser, a specially developed application, or a program module running in other applications. This embodiment does not limit the information retrieval program and the way to display the information retrieval interface.
[0183] Step 602, receiving the personnel information of the target scientific researcher through the information retrieval interface.
[0184] Personnel information is used to trigger the acquisition of the associated database of the target scientific researcher; and based on the associated database, the citation information of each cited document in the corresponding citing document is obtained; the citation information is input into the preset large language model so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain the citation situation information; and the evaluation result of the cited document is generated based on the citation situation information. Among them, the large language model pre-learns the ability to evaluate the citation information and obtain the citation situation information based on the preset prompt words; the preset prompt words include at least one citation information template and the citation situation information label corresponding to the citation information template; the associated database includes the first document information of at least one cited document and the second document information of the citing document corresponding to each cited document, the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document; the citation information includes the citation mark and / or the citing paragraph.
[0185] refer to Figure 7The information retrieval interface includes a first input box 701 for personnel information, through which the electronic device receives personnel information. The personnel information is used to indicate the identity of the target scientific researcher. The relevant description of the personnel information is detailed in step 101 in the above embodiment, and this embodiment will not be repeated here.
[0186] Optionally, the research literature of the target researcher is the research literature with the target researcher as the first author, or the research literature with the target researcher as one of the authors. The specific type of cited literature can be selected by the user, such as: reference Figure 7 The information search interface also includes a first selection box 702 for the type of cited document, through which the cited document is selected as a scientific research document with the target researcher as the first author, or a scientific research document with the target researcher as one of the authors. A black circle represents that the type of document after the circle is selected.
[0187] Optionally, the information retrieval interface also supports the user to input literature evaluation requirements, which include the type of citations and / or the output requirements for the evaluation results. Figure 7 The information retrieval interface also includes a second input box 703 for document evaluation requirements, and the electronic device obtains the document evaluation requirements through the second input box 703. Figure 7 As shown, the user can select the type of citation by clicking the triangle control in the input box where the type of citation is located; and can select the output requirement of only outputting the type of citation by clicking the circle on the left side of the type of citation only.
[0188] Step 603: Display the evaluation result on the information retrieval interface.
[0189] The information retrieval interface includes an evaluation result display area, in which the evaluation results of each cited document of the target researcher are displayed.
[0190] For example: Reference Figure 7 The display area 704 shown shows the evaluation result of a cited document of the target researcher: brief mention.
[0191] In summary, the method for retrieving and evaluating the literature information of scientific researchers provided in this embodiment displays an information retrieval interface; receives the personnel information of the target scientific researcher through the information retrieval interface to obtain the associated database of the target scientific researcher based on the personnel information; and obtains the citation information of each cited document in the corresponding citing document based on the associated database; inputs the citation information into a preset large language model so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain the citation situation information; generates the evaluation result of the cited document based on the citation situation information; wherein the large language model pre-learns the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and the citation situation information label corresponding to the citation information template; the associated database includes the first document information of at least one cited document and the second document information of the citing document corresponding to each cited document; displays the evaluation result on the information retrieval interface; enables the user to obtain the evaluation results of each cited document of the target scientific researcher by performing simple operations on the information retrieval interface, without the need for the user to manually count the cited documents, thereby avoiding the problem of low efficiency of manual statistics; and improves the efficiency of literature evaluation for scientific researchers.
[0192] Figure 8 It is a block diagram of a device for searching and evaluating literature information for researchers provided in one embodiment of the present application, which includes: an information acquisition module 810, a data acquisition module 820, a citation acquisition module 830, a citation evaluation module 840 and a result generation module 850.
[0193] The information acquisition module 810 is used to acquire the personnel information of the target scientific researcher;
[0194] A data acquisition module 820 is used to acquire a related database of the target scientific researcher based on the personnel information; wherein the related database includes first document information of at least one cited document and second document information of a citing document corresponding to each cited document, wherein the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document;
[0195] A citation acquisition module 830 is used to acquire the citation information of each cited document in the corresponding citing document based on the associated database; wherein the citation information includes a citation mark and / or a citing text segment;
[0196] The citation evaluation module 840 is used to input the citation information into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain the citation situation information; the large language model learns in advance the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and a citation situation information label corresponding to the citation information template;
[0197] The result generating module 850 is used to generate the evaluation result of the cited document based on the citation information.
[0198] For detailed description, please refer to the above method embodiment.
[0199] Fig. 9 It is a block diagram of a device for document information retrieval and evaluation for scientific researchers provided in another embodiment of the present application. The device includes: an interface display module 910, an information receiving module 920 and a result display module 930.
[0200] An interface display module 910 is used to display an information retrieval interface;
[0201] The information receiving module 920 is used to receive the personnel information of the target scientific researcher through the information retrieval interface, so as to obtain the associated database of the target scientific researcher based on the personnel information; and obtain the citation information of each cited document in the corresponding citing document based on the associated database; input the citation information into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain the citation situation information; generate the evaluation result of the cited document based on the citation situation information; wherein the large language model preliminarily learns the ability to evaluate the citation information and obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and the citation situation information label corresponding to the citation information template; the associated database includes the first document information of at least one cited document and the second document information of the citing document corresponding to each cited document, the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document; the citation information includes citation marks and / or citing paragraphs;
[0202] The result display module 930 is used to display the evaluation result on the information retrieval interface.
[0203] For detailed description, please refer to the above method embodiment.
[0204] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0205] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.
[0206] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0207] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0208] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for searching and evaluating scientific research personnel's literature information, characterized in that: The method comprises: Obtaining personnel information of target researchers; Based on the personnel information, an associated database of the target scientific researcher is obtained; wherein the associated database includes first document information of at least one cited document and second document information of a citing document corresponding to each cited document, wherein the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document; Acquiring citation information of each cited document in a corresponding citing document based on the associated database; wherein the citation information includes a citation mark and / or a citing text segment; The citation information is input into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain citation situation information; the large language model learns in advance the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and a citation situation information label corresponding to the citation information template; An evaluation result of the cited document is generated based on the citation information.
2. The method according to claim 1, characterized in that The step of inputting the citation information into a preset large language model so that the large language model evaluates the citation status of the cited document based on the citation information to obtain citation status information includes: The reference information and the preset prompt word are input into the large language model, so that the large language model first learns the ability to evaluate the reference information according to the preset prompt word, and then evaluates the reference information according to the learned ability to obtain the reference situation information.
3. The method according to claim 2, characterized in that The method further comprises: Obtaining a document evaluation requirement; wherein the document evaluation requirement includes the type of the citation situation, and correspondingly, the citation situation information includes the type of the citation situation; The preset prompt word is constructed based on the type of the reference situation.
4. The method according to claim 1, characterized in that: The obtaining, based on the associated database, the citation information of each cited document in the corresponding citing document comprises: Based on each second document information in the associated database, directing to a web page of the cited document, the web page is used to display the full text of the cited document; Searching the web page for first document information corresponding to the second document information; Based on the first document information, citation information of the cited document in the citing document is obtained.
5. The method according to claim 4, characterized in that The first document information includes the title of the cited document; accordingly, The searching the web page for the first document information corresponding to the second document information includes: Searching the web page element where the title of the cited document is located on the web page based on an accurate search method; In the case where the web page element is not found, searching the web page element where the title of the cited document is located based on a fuzzy search method to obtain a fuzzy result; The fuzzy result is used as the title of the cited document, and the web page element where the fuzzy result is located is searched in the web page based on the precise search method to obtain the first document information in the citing document.
6. The method according to claim 4, characterized in that The obtaining, based on the first document information, citation information of the cited document in the citing document comprises: In the citing document, identifying a reference mark corresponding to the title of the cited document; Searching the citing document for the first citing paragraph including the citation mark; In the citation segment, n citation segments before the first citation segment and m citation segments after the first citation segment are obtained to obtain a second citation segment; wherein the reference information includes the reference mark, the first citation segment and the second citation segment, and m and n are both natural numbers.
7. The method according to any one of claims 1 to 6, characterized in that: Generating the evaluation result of the cited document based on the citation information includes: Based on the output requirement of the evaluation result in the literature evaluation requirement, the citation information is modified to obtain the evaluation result.
8. The method according to claim 7, characterized in that The citation information includes the type of citation and the analysis process of the type of citation; accordingly, the output requirement includes: outputting the type of citation without outputting the analysis process of the citation information; the citation information is modified according to the output requirement to obtain the evaluation result, including: The analysis process in the citation information is deleted to obtain the evaluation result.
9. A method for searching and evaluating scientific research personnel's literature information, characterized in that: The method comprises: Display the information retrieval interface; Receive the personnel information of the target scientific researcher through the information retrieval interface to obtain the associated database of the target scientific researcher based on the personnel information; and obtain the citation information of each cited document in the corresponding citing document based on the associated database; input the citation information into a preset large language model, so that the large language model evaluates the citation situation of the cited document based on the citation information to obtain the citation situation information; generate the evaluation result of the cited document based on the citation situation information; wherein the large language model pre-learns the ability to evaluate the citation information to obtain the citation situation information based on preset prompt words; the preset prompt words include at least one citation information template and the citation situation information label corresponding to the citation information template; the associated database includes first document information of at least one cited document and second document information of the citing document corresponding to each cited document, the cited document refers to the scientific research document of the target scientific researcher, and the citing document refers to other scientific research documents that cite the cited document; the citation information includes citation marks and / or citing paragraphs; The evaluation result is displayed on the information retrieval interface.
10. A scientific researcher document information retrieval and evaluation device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 9 when executing the instructions stored in the memory.
11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.
Citation Information
Patent Citations
citation importance recognition method and device, electronic equipment and computer storage medium
CN111782827A
Information identification method and device, equipment, storage medium and program product
CN118194131A
Cited By
Creference type analysis method and system based on joint multi-task learning
CN120929984A
A reference type analysis method and system based on joint multi-task learning
CN120929984B
