Program and bibliographic identification method

Abstract

Identify the bibliographic information correctly. [Solution] The bibliographic identification server 10 includes a token extraction unit 13 that extracts multiple first tokens from a query entered by a user to search for bibliographic data, a similarity calculation unit 14 that calculates the similarity of multiple second tokens extracted from the bibliographic data to multiple first tokens for each of the multiple bibliographic data, and a determination unit 15 that determines whether the bibliographic data corresponding to the similarity with the highest rank among the calculated similarities for each bibliographic data matches the query.

Description

Technical Field

[0001] This disclosure relates to a program and bibliographic identification method.

Background Art

[0002] Techniques for identifying the bibliographic information of documents such as papers and books are known. For example, as a technique for identifying the bibliographic information of English documents, a bibliographic identification method by full-text search (Search-Based Matching with Validation, SBMV) as shown in Non-Patent Document 1 is known. Also, for example, as a technique for identifying the bibliographic information of Japanese documents, a bibliographic identification method by syntax analysis (Parse-Based Matching, PBM) as shown in Non-Patent Document 2 is known.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, bibliographic identification methods such as those shown in Non-Patent Documents 1 and 2 may not be able to properly identify bibliographies. The bibliographic identification method in Non-Patent Document 1 has the problem that it is suitable for English-language documents but not for Japanese-language documents. The bibliographic identification method in Non-Patent Document 2 is suitable for Japanese-language documents, but the bibliographic identification method using PBM as shown in Non-Patent Document 2 has the problem that it is less accurate than the bibliographic identification method using SBMV as shown in Non-Patent Document 1.

[0005] In view of the above circumstances, the purpose of this disclosure is to provide a program and a bibliographic identification method that can appropriately identify bibliographic records. [Means for solving the problem]

[0006] To achieve the above objectives, the program relating to the first aspect of this disclosure is: Computers, A token extraction unit that extracts multiple first tokens from the query entered by the user to search for bibliographic data. A similarity calculation unit calculates the similarity of a plurality of second tokens extracted from a plurality of bibliographic data to a plurality of first tokens for each of the plurality of bibliographic data. A determination unit that determines whether the bibliographic data corresponding to the similarity ranked 1st among the calculated similarity scores for each bibliographic data matches the query. To make it function as such.

[0007] To achieve the above objectives, the program relating to the second aspect of this disclosure is: Computers, A token extraction unit that extracts multiple first tokens from the query entered by the user to search for bibliographic data. A similarity calculation unit calculates the similarity of a plurality of second tokens extracted from a plurality of bibliographic data to a plurality of first tokens for each of the plurality of bibliographic data. For each of the aforementioned bibliographic data, a precision calculation unit calculates the precision of the set of multiple first tokens against the set of multiple second tokens. A determination unit determines whether the bibliographic data matches the query based on the calculated similarity and the document matching accuracy for each of the aforementioned bibliographic data. To make it function as such. [Effects of the Invention]

[0008] According to this disclosure, bibliographic information can be appropriately identified. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the overall configuration of the bibliographic identification system according to Embodiment 1 of this disclosure. [Figure 2] This figure shows the functional configuration of the bibliographic identification server according to Embodiment 1 of this disclosure. [Figure 3] A box plot showing an example of an outlier in Embodiment 1 of this disclosure. [Figure 4] This figure shows an example of a dataset used when building a trained model in the bibliographic identification server according to Embodiment 1 of this disclosure. [Figure 5] This figure shows an example of the hardware configuration of a bibliographic identification server according to Embodiment 1 of this disclosure. [Figure 6] A flowchart illustrating an example of bibliographic identification processing by a bibliographic identification server according to Embodiment 1 of this disclosure. [Figure 7] A diagram showing an example of bibliographic data in Embodiment 2 of this disclosure. [Figure 8] A diagram showing an example of a subset in Embodiment 2 of this disclosure. [Figure 9] This figure shows the overall configuration of the bibliographic identification system according to Embodiment 2 of this disclosure. [Figure 10] A flowchart illustrating an example of bibliographic identification processing by a bibliographic identification server according to Embodiment 2 of this disclosure. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments in which the bibliographic identification method according to this disclosure is applied to a bibliographic identification system will be described while referring to the drawings. In each drawing, the same or equivalent parts will be denoted by the same reference numerals.

[0011] (Embodiment 1) The bibliographic identification system 1 will be described while referring to FIG. 1. The bibliographic identification system 1 includes a bibliographic DB (database) server 20 that stores bibliographic data of documents, a user terminal 30 that is used to search for bibliographic data desired by a user from a plurality of bibliographic data stored in the bibliographic DB server 20, and a bibliographic identification server 10 that identifies one bibliographic data desired by the user from a plurality of bibliographic data stored in the bibliographic DB server 20 based on a character string (hereinafter, this is referred to as a "query") input by the user for searching for bibliographic data. The bibliographic identification server 10, the bibliographic DB server 20, and the user terminal 30 are communicably connected via, for example, the Internet.

[0012] The bibliographic DB server 20 stores bibliographic data of a plurality of documents such as papers and books. The bibliographic DB server 20 provides an API (Application Programming Interface) for the bibliographic identification server 10 to perform full-text search, acquisition, etc. of bibliographic data. However, it is not always possible to search only for the bibliographic data desired by the user by full-text search alone, and bibliographic data not desired by the user may also be searched. For example, in full-text search, bibliographic data containing only a part of the query may be searched.

[0013] The bibliographic identification server 10 provides a user interface to the user terminal 30, and identifies bibliographic information from a plurality of bibliographic data stored in the bibliographic DB server 20 based on a query input to the user interface. The bibliographic identification server 10 provides a user interface to the user terminal 30 by providing a web service with a web server function. When the user inputs a query through the web service, the bibliographic identification server 10 performs a full-text search and obtains bibliographic data from the bibliographic DB server 20 via the above API. As described above, since bibliographic information that the user does not desire may also be retrieved, the bibliographic identification server 10 further identifies the bibliographic information desired by the user based on the query input by the user by a method described later. Note that the identification here also includes determining that the bibliographic data of the bibliographic information desired by the user does not exist in the bibliographic DB server 20. That is, the bibliographic identification server 10 enables the user to specify the desired bibliographic information or to know that the desired bibliographic information does not exist in the bibliographic DB server 20.

[0014] The user terminal 30 is a terminal device equipped with a web browser, such as a personal computer or a smartphone. The user can operate the user terminal 30 to access the bibliographic identification server 10 and input a query via the web service provided by the bibliographic identification server 10. Although details will be described later, when inputting a query, the user does not need to accurately input all bibliographic information. For example, even when the user inputs only the author and the title of the document and does not input other bibliographic information such as the journal in which the document is published, the page on which it is published, and the year of publication, in many cases, the bibliographic information can be correctly identified from the bibliographic data stored in the bibliographic DB server 20 from the input query.

[0015] In the bibliographic identification system 1, the user, for example, refers to the list of references listed in a paper and enters the bibliographic information of the desired reference as a query. For example, the user enters a string such as "○○ Taro (2018) Elucidation of the mechanism concerning ×× of △△. Journal of the Japan ■■ Society 68:298-302." as a query. The numbers at the end, 68, represent the volume and issue number of the journal in which the paper is published, and 298-302 represent the page numbers. However, the query entered by the user and the bibliographic data stored in the bibliographic DB server 20 may not always be a perfect match. For example, the bibliographic information listed in the list of references may include the page number of the journal, while the bibliographic data may not be registered in that much detail. In addition, there may be cases where some co-author names are missing or there are inconsistencies in the title notation. Therefore, in the bibliographic identification system 1, simply searching for bibliographic data that perfectly matches the query entered by the user is unlikely to identify the bibliographic information desired by the user. Therefore, as will be described later, the bibliographic identification system 1 identifies bibliographies based on the similarity between the query and the bibliographic data.

[0016] Furthermore, the bibliographic data registered for academic papers and books differs in some respects. Bibliographic data for academic papers includes, for example, the title, author name, journal, page numbers, and publication date. Bibliographic data for books includes, for example, the title, author name, publisher, edition (first edition, second edition, hardcover edition, paperback edition, etc.), publication date, and contents of the table of contents (chapter titles, co-authors of the chapter, etc.). In the fields of humanities and social sciences, books co-authored by multiple authors are often cited chapter by chapter, so the contents of the table of contents are also registered as bibliographic data and can be searched using queries.

[0017] The functional configuration of the bibliographic identification server 10 will be explained with reference to Figure 2. The bibliographic identification server 10 comprises a web server unit 11, a bibliographic data acquisition unit 12, a token extraction unit 13, a similarity calculation unit 14, and a determination unit 15.

[0018] The web server unit 11 provides the user with a web service that implements a user interface for the user terminal 30 to input queries and a user interface for presenting the bibliographic identification results to the user. The web server unit 11 acquires the queries entered by the user and outputs them to the bibliographic data acquisition unit 12 and the token extraction unit 13, which will be described later. The web server unit 11 presents the results of the bibliographic identification performed by the determination unit 15, which will be described later, to the user.

[0019] The bibliographic data acquisition unit 12 acquires bibliographic data from the bibliographic database server 20 based on the query entered by the user, which is output by the web server unit 11, and outputs the acquired bibliographic data to the token extraction unit 13. The bibliographic data acquisition unit 12 performs a full-text search based on the query entered by the user, for example, by using the full-text search function of the bibliographic database server 20, and acquires the bibliographic data that the bibliographic database server 20 outputs as search results. However, the acquired bibliographic data may include bibliographic data of bibliographies that the user did not request.

[0020] The token extraction unit 13 extracts multiple tokens from the user-entered query output by the web server unit 11. The token extracted from the user-entered query will be referred to as the first token below.

[0021] The token extraction unit 13 extracts multiple tokens from the bibliographic data output by the bibliographic data acquisition unit 12. The tokens extracted from the bibliographic data output by the bibliographic data acquisition unit 12 are hereafter referred to as second tokens. When the bibliographic data acquisition unit 12 outputs multiple bibliographic data (for example, when bibliographic data for three documents is obtained as a result of a full-text search), the token extraction unit 13 extracts multiple second tokens for each bibliographic data.

[0022] The token extraction unit 13 divides the string into parts based on punctuation marks (a general term for symbols and codes such as punctuation marks, parentheses, and mathematical symbols), performs morphological analysis on each of the divided strings, and then applies n-grams to each of the strings separated by morphological analysis on a word-by-word basis to extract tokens. An n-gram is the extraction of as many consecutive n items as possible from any consecutive sequence of words, characters, etc. For example, applying a 2-gram to a sequence of four characters, A, B, C, and D, will extract "AB", "BC", and "CD".

[0023] For example, suppose the string from which token extraction is performed is "○○ Taro (2018) Elucidation of the mechanism concerning ×× of △△. Journal of the Japan Society 68:298-302.". In this case, if we first split the string by punctuation marks, it will be split as follows: "○○ Taro|2018|Elucidation of the mechanism concerning ×× of △△|Journal of the Japan Society|68|298|302" (vertical lines indicate the parts separated by punctuation marks).

[0024] Next, by performing morphological analysis on each of these divided strings, these strings are segmented into fixed word units, such as "[○○][Taro]|

[2018] |[△△][of][××][related][mechanism][of][clarification]|[Japan][■■][academic society][journal]|

[68] |

[0298] |

[0302] " (each enclosed in square brackets represents a single word identified by morphological analysis). Note that morphological analysis can be applied not only to Japanese strings but also to English strings.

[0025] Then, for each string divided by punctuation marks, n-grams are applied to each word that has been separated to extract tokens. For example, when 2-grams are applied to the above separated string on a word-by-word basis, the extracted tokens ignore strings with one or fewer words (each digit in this example) and become "{○○ Taro}|{△△'s}{of ××}{××'s}{related}{mechanism}{of}{elucidation}|{Japan■■}{■■ Society}{Journal}" (each enclosed in curly braces represents a single token. Note that here, the parts separated by punctuation marks are represented as vertical lines, and the word boundaries identified by morphological analysis are omitted).

[0026] The token extraction unit 13 outputs the extracted first tokens and the extracted second tokens for each bibliographic data to the similarity calculation unit 14.

[0027] The similarity calculation unit 14 calculates the similarity of multiple second tokens to multiple first tokens for each bibliographic data. The similarity calculation unit 14 outputs the calculated similarity for each bibliographic data to the determination unit 15. Since the first tokens are extracted from queries entered by the user, and the second tokens are extracted from each bibliographic data obtained by the bibliographic data acquisition unit 12 from the bibliographic DB server 20, the similarity calculation unit 14 calculates a similarity that indicates how similar each acquired bibliographic data is to the query. In the following explanation, the set of multiple first tokens will be referred to as A, and the set of multiple second tokens as B. Set A corresponds to the query, and set B corresponds to the bibliographic data. The number of elements in set X will be represented as |X|.

[0028] The similarity calculation unit 14 calculates the similarity using the index "literature recall rate" described below. The literature recall rate R(A,B) of set B for set A can be calculated using the following formula (1.1). R(A,B)=|A∩B| / |A| (1.1)

[0029] This reference recall rate R is an indicator of how well the common portion of set B (second token, bibliographic data) with set A (first token, query) covers the first token. The terms in parentheses indicate what set A and set B correspond to, respectively. In other words, the reference recall rate R represents the proportion of tokens in set A that are also present in set B. By using this reference recall rate R as the similarity metric, it is possible to calculate similarity by largely ignoring irrelevant bibliographies, even when the bibliographic data contains many irrelevant bibliographies, or conversely, when the query contains many irrelevant bibliographies. For example, when the query concerns a specific chapter of a book and the bibliographic data contains the contents of numerous table of contents entries, it is possible to calculate similarity by largely ignoring bibliographies related to chapters irrelevant to the query. Therefore, this reference recall rate R is particularly suitable when identifying bibliographic data of books.

[0030] The determination unit 15 determines whether the bibliographic data corresponding to the highest-ranking similarity, based on the similarity scores for each bibliographic data output by the similarity calculation unit 14, matches the query. "The bibliographic data matches the query" means that the bibliographic information to be identified in the query matches the bibliographic information in the bibliographic data. When the bibliographic data matches the query, that bibliographic data is the bibliographic information desired by the user.

[0031] The inventors found that the following three features relating to queries and bibliographic data correlate with the match rate of the bibliographic data to the query (the likelihood that the bibliographic data matches the query). (1) Similarity (document recall) of set B (bibliographic data) to set A (queries) (2) Outlier, which indicates how much the similarity of one bibliographic data is different (how much it stands out) from the overall similarity calculated for all bibliographic data related to the same query. (3) The number of elements in the intersection of set A and set B

[0032] Regarding (1), it indicates how well the bibliographic data matches the query, and therefore correlates with the match rate. Regarding (2), for example, the more prominent the similarity of an entry ranked 1st, the more unique it is and the less likely it is to be confused with another bibliographic entry, and therefore correlates with the match rate. Regarding (3), for example, if (1) and (2) are of similarity for 1st and 2nd place, the entry with more matching tokens between the bibliographic data and the query is more likely to be a match, and therefore correlates with the match rate.

[0033] The outlier in (2) above is calculated, for example, based on the IQR (Interquartile Range) method. For example, consider the case where the distribution of similarity is represented by the box plot shown in Figure 3. P is the similarity that ranks 1st, Q1 is the first quartile, and Q3 is the third quartile. In this case, the outlier L of the similarity P that ranks 1st can be calculated by the following formula (1.2). L = (P - Q3) / (Q3 - Q1) (1.2)

[0034] In addition to calculating the outlier using the IQR (Interquartile Range) method, the outlier may also be calculated using methods such as the LOF (Local Outlier Factor) method. Furthermore, the difference between the similarity of the 1st ranked element and the similarity of the 2nd ranked element may be used to calculate the outlier.

[0035] Furthermore, viewing this relationship from a different perspective, it can be said that the combination of the first indicator, similarity (literature recall rate), and the second indicator, "the pair of outliers and the number of common elements," correlates with the match rate. This perspective is related to Embodiment 2, which will be described later.

[0036] In Embodiment 1, first, a trained model is constructed using machine learning to determine whether or not the bibliographic data matches the query (hereinafter referred to as "matching") when the above (1), (2), and (3) of the bibliographic data are input. For example, various queries are prepared in advance, and a full-text search is performed on the bibliographic DB server 20 based on each query. Based on the bibliographic data obtained by the full-text search, a trained model is constructed using supervised learning, with the similarity of set B (bibliographic data) to set A (query), the outlier of the similarity, and the number of elements in the common part of set A and set B as inputs, and whether or not it matches the query as a correct / incorrect label. For example, a trained model is constructed based on a dataset as shown in Figure 4. This dataset is obtained by performing a full-text search on a single query, calculating the similarity, outlier, and number of elements for each bibliographic data, and having the user label whether or not it matches the query. Using the trained model constructed in this way, it is possible to determine whether or not the bibliographic data matches the query based on the above (1), (2), and (3) of the bibliographic data.

[0037] The determination unit 15 performs a match determination using this trained model. Note that the similarity (literature recall) mentioned in (1) above has already been calculated by the similarity calculation unit 14. First, the determination unit 15 calculates the outlier (mentioned in (2) above) of the similarity that ranks 1st relative to the entire calculated similarity.

[0038] Next, the determination unit 15 calculates the number of elements in the intersection of set A and set B, that is, |A∩B| (as in (3) above).

[0039] The determination unit 15 then uses the trained model described above to determine whether the bibliographic data matches the query based on (1), (2), and (3). In other words, the determination unit 15 uses the trained model described above to determine whether the bibliographic data matches the query by inferring the degree of matching from (1), (2), and (3).

[0040] The determination unit 15 outputs the bibliographic data with the highest similarity score to the web server unit 11 when it determines that the bibliographic data with the highest similarity score matches the query, and outputs data indicating that no matching bibliographic data was found when it determines that no matching data was found. As a result, when the bibliographic data with the highest similarity score matches the query, the web server unit 11 presents the bibliographic data with the user as the result of bibliographic identification, and when there is no match, it presents the user as the result of bibliographic identification that the bibliographic data the user desired was not found.

[0041] Next, an example of the hardware configuration of the bibliographic identification server 10 will be described with reference to Figure 5. The bibliographic identification server 10 shown in Figure 5 is implemented using a computer such as a personal computer or a server computer.

[0042] The bibliographic identification server 10 comprises a processor 1001, memory 1002, interface 1003, and secondary storage device 1004, all connected to each other via a bus 1000.

[0043] The processor 1001 is, for example, a CPU (Central Processing Unit). The various functions of the bibliographic identification server 10 are realized when the processor 1001 reads the operation program stored in the secondary storage device 1004 into the memory 1002 and executes it.

[0044] Memory 1002 is a main memory device, for example, composed of RAM (Random Access Memory). Memory 1002 stores the operational program read by the processor 1001 from the secondary memory device 1004. Memory 1002 also functions as working memory when the processor 1001 executes the operational program.

[0045] Interface 1003 is an I / O (Input / Output) interface such as a serial port, USB (Universal Serial Bus) port, or network interface.

[0046] The secondary storage device 1004 is, for example, flash memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive). The secondary storage device 1004 stores the operational programs that the processor 1001 executes.

[0047] Next, an example of the bibliographic identification process performed by the bibliographic identification server 10 will be explained with reference to Figure 6. The process shown in Figure 6 starts when the bibliographic identification server 10 is started up and becomes capable of providing web services.

[0048] The web server unit 11 of the bibliographic identification server 10 waits for query input from the user (step S101). When the user inputs a query, the web server unit 11 outputs the query to the bibliographic data acquisition unit 12 and the token extraction unit 13.

[0049] The bibliographic data acquisition unit 12 of the bibliographic identification server 10 acquires bibliographic data from the bibliographic DB server 20 based on the query (step S102). The bibliographic data acquisition unit 12 outputs the acquired bibliographic data to the token extraction unit 13.

[0050] The token extraction unit 13 of the bibliographic identification server 10 extracts multiple first tokens from the query entered in step S101 (step S103). The token extraction unit 13 outputs the extracted multiple first tokens to the similarity calculation unit 14.

[0051] The token extraction unit 13 extracts multiple second tokens for each bibliographic data obtained in step S102 (step S104). The token extraction unit 13 outputs multiple second tokens for each bibliographic data to the similarity calculation unit 14.

[0052] The similarity calculation unit 14 of the bibliographic identification server 10 calculates the similarity of multiple second tokens to multiple first tokens for each bibliographic data (step S105). The similarity calculation unit 14 outputs the calculated similarity for each bibliographic data to the determination unit 15.

[0053] The determination unit 15 of the bibliographic identification server 10 determines whether the bibliographic data with the highest similarity score matches the query (step S106).

[0054] When it is determined that the bibliographic data with the highest similarity matches the query (Step S107: Yes), the web server unit 11 presents the bibliographic data with the highest similarity as the result of bibliographic identification (Step S108). The bibliographic identification server 10 then repeats the process from Step S101.

[0055] When it is determined that the bibliographic data with the highest similarity does not match the query (step S107: No), the web server unit 11 presents the result of the bibliographic identification as indicating that the bibliographic data desired by the user was not found (step S109). The bibliographic identification server 10 then repeats the process from step S101.

[0056] The bibliographic identification system 1 according to Embodiment 1 has been described above. The bibliographic identification server 10 of the bibliographic identification system 1 extracts multiple first tokens from a query and multiple second tokens from bibliographic data. This token extraction can be applied to both English and Japanese. The bibliographic identification server 10 then performs bibliographic identification based on the similarity of the tokens. Therefore, the bibliographic identification server 10 can appropriately identify bibliographies.

[0057] (Embodiment 2) The overall configuration of the bibliographic identification system 1 according to Embodiment 2 is generally the same as that of the bibliographic identification system 1 according to Embodiment 1 shown in Figure 1. However, as will be explained below, Embodiment 1 determines the degree of matching based on the literature matching recall rate, the degree of outlier, and the number of common elements, whereas Embodiment 2 determines the degree of matching based on the literature matching recall rate and the literature matching precision rate, which will be described later. In other words, Embodiment 1 determines the degree of matching based on a combination of a first indicator, the literature matching recall rate, and a second indicator, the degree of outlier and the number of common elements, whereas Embodiment 2 determines the degree of matching based on a first indicator, the literature matching recall rate, and a second indicator, the literature matching precision rate, meaning that the second indicator is the literature matching precision rate.

[0058] In the following description, as in Embodiment 1, set A corresponds to the query and the first token, and set B corresponds to the bibliographic data and the second token. Set B also includes subsets described below, depending on whether the bibliographic data pertains to an article or to a book. The literature matching accuracy, described later, is expressed using these subsets.

[0059] If the bibliographic data pertains to a publication, set B further includes subsets Bc, Bt, Bi, Bj, Bv, and Bpg. Each subset represents the following: Set Bc: Set of first author names (creators). Set Bt: A collection of titles. Set Bi: Set of publication years (issued) Set Bj: The set of journal names in which the papers are published. Set Bv: A set of volume and issue numbers of the magazine in which the publication was made. Set Bpg: A set of page numbers (page_range) in which the paper is published.

[0060] Furthermore, if the bibliographic data pertains to a research paper, the literature matching precision P(B,A) of set A to set B can be calculated as a set of six values ​​(a vector value with six components) using the following formula (2.1).

[0061]

number

[0062] If the bibliographic data pertains to books, set B further includes subsets Bc, Bt, Bi, and Bpb. Each subset represents the following: Set Bc: Set of first author names (creators). Set Bt: A collection of titles. Set Bi: Set of publication years (issued) Set Bpb: Set of publishers

[0063] Furthermore, if the bibliographic data pertains to a research paper, the literature matching precision P(B,A) of set B for set A can be calculated as a set of four values ​​(a vector value with four components) using the following formula (2.2).

[0064]

number

[0065] Hereafter, when referring to each subset in general terms, they may be expressed as subset Bx, set Bx, etc. If each component of the literature matching precision P(B,A) expressed by equation (2.1) or equation (2.2) is px, then px can be expressed using this notation in the following equation (2.3). px = |A∩Bx| / |Bx| (2.3) When specifically identifying the pixels corresponding to each subset, they are denoted as pt, pc, etc.

[0066] Each component of this literature matching precision P(B,A) represents the proportion of tokens in set A (first token, query) that are included in each subset of set B (second token, bibliographic data).

[0067] For example, consider the case where the bibliographic data is as shown in Figure 7. This bibliographic data corresponds to the query "○○ Taro (2018) Elucidation of the mechanism concerning ×× of △△. Journal of the Japan ■■ Society 68:298-302." However, the title includes the subtitle "~Towards improving the efficiency of ●● operations~". As mentioned above, among the numbers at the end, 68 represents the volume and issue number of the journal in which the paper is published, and 298-302 represents the page numbers.

[0068] First, for each item in the bibliographic data, we determine the sets Bc, Bt, Bi, Bj, Bv, and Bpg. To determine these, we extract tokens for each item shown in Figure 7. However, for the sake of explanation, unlike the example in Embodiment 1, we do not apply n-grams, and instead use the words identified by morphological analysis after being separated by punctuation marks as tokens. The tokens extracted for each item become elements of their respective subsets. Hereinafter, these elements will be referred to as "bibliographic elements."

[0069] For example, extracting the token for the first author's name, "○○ Taro," results in {○○}{Taro}, and each of these tokens becomes a bibliographic element in set Bc (as before, items enclosed in curly braces represent a single token). Similarly, obtaining the bibliographic elements for each subset yields the results shown in Figure 8. Underlined tokens represent tokens that match those similarly extracted from the aforementioned query, "○○ Taro (2018) Elucidation of the mechanism concerning ×× of △△. Journal of the Japan Society 68:298-302." As shown in Figure 8, tokens are also extracted for the subtitle of the title. Note that the characters "." and "~" are punctuation marks and are therefore not extracted as tokens. Also, the title string contains the word "no" three times, but these "no" represent a single bibliographic element in the subset.

[0070] For each subset except set Bt, all bibliographic elements of set Bx are also included in set A. Therefore, for all subsets except Bt, |A∩Bx|=|Bx|, and from equation (2.3), px=1. On the other hand, for set Bt, only 7 of the bibliographic elements of set Bx are included in set A, and the 5 bibliographic elements based on the subtitle are not included in set A. Therefore, |A∩Bt|=7 and |Bt|=12, and from equation (2.3), pt=7 / 12. Thus, in this example, from equation (2.1), P(B,A)=(1,7 / 12,1,1,1,1).

[0071] In the example above, the first author's name, title, and journal name in the bibliographic data were in Japanese. However, these items may be registered in multiple languages. For example, the first author's name may be registered in both kanji and Roman letters, and the title and journal name may be registered in both Japanese and English. In this case, the component px is calculated for all languages, and the highest value is adopted as the px. For example, if the query is in English, and the bibliographic data used to calculate the component px is only in Japanese, the px will be unfairly low.

[0072] Furthermore, the reason n-grams are not applied in the above explanation is for illustrative purposes only; therefore, when extracting tokens to determine the literature relevance P(B,A), n-grams may be applied to each word.

[0073] The inventor focused on the fact that, as a characteristic of metadata in papers and books, the frequency of tokens used for each bibliographic element, such as author name, title, and publication year, differs significantly, and that it should be possible to predict the bibliographic elements that will appear for each token. For example, it is assumed that tokens such as "Smith" and "Chen" are not used in titles, tokens such as "elementary particle" and "virus" are not used in author names, numbers are used for page numbers and volume issues, and some tokens are used for both journal names and titles. The inventor believed that by utilizing this characteristic, it would be possible to perform bibliographic identification from queries without having to specifically parse the queries and estimate which tokens correspond to which bibliographic elements.

[0074] Based on the above characteristics, the inventors found that the pair of the first indicator, similarity (particularly the literature recall rate R(A,B)), and the second indicator, literature precision rate P(B,A), correlates with the matching rate of bibliographic data to queries. Below, assuming that similarity is the literature recall rate R(A,B), two examples of queries matching bibliographic data will be given to illustrate this.

[0075] As the first example, consider the case where the bibliographic data is as shown in Figure 7, and the query is "○○ Taro (2018) Elucidation of the mechanism concerning ×× of △△. Journal of the Japan ■■ Society 68:298-302." However, we will extract tokens without applying n-grams. In this case, the token obtained from the query will be "{○○}{Taro}{2018}{△△}{of}{××}{concerning}{mechanism}{elucidation}{Japan}{■■}{society}{journal}{68}{298}{302}" (duplicate tokens {of} are combined into one). On the other hand, the token obtained from the bibliographic data will be a combination of each element of each subset shown in Figure 8.

[0076] Since all tokens obtained from the query are also included in the tokens obtained from the bibliographic data, |A∩B|=|A|, and from equation (1.1), the literature recall R(A,B)=1. On the other hand, for the literature precision P(B,A), as mentioned above, P(B,A)=(1,7 / 12,1,1,1,1). In other words, for the literature precision P(B,A), some components are slightly less than 1, while the other components are 1.

[0077] As a second example, consider the opposite case: the query contains a string related to the subtitle, but the bibliographic data does not. That is, the query is "○○ Taro (2018) Elucidation of the mechanism concerning ×× of △△. ~Towards the efficiency of ●● operations~ Journal of the Japan ■■ Society 68:298-302." and the bibliographic data is the same as shown in Figure 7 except for the title, and the title is "Elucidation of the mechanism concerning ×× of △△." without a subtitle.

[0078] In this case, tokens derived from the subtitle among the tokens obtained from the query are not included in the tokens obtained from the bibliographic data, so |A∩B| is slightly smaller than |A|, and the literature recall R(A,B) is also slightly less than 1. On the other hand, all tokens obtained from the bibliographic data are included in the tokens obtained from the query, so for each subset, |A∩Bx|=|Bx|. Therefore, the literature precision P(B,A)=(1,1,1,1,1,1), and all components of the literature precision P(B,A) are 1.

[0079] As a third example, consider the case where the bibliographic data contains an appropriate amount of information necessary for identification, and the query entered by the user also contains an appropriate amount of information necessary for identification. In this case, it can be expected that for each set Bx, |A∩Bx|≈|Bx| and |A∩B|≈|A|. In this case, the literature recall R(A,B) will be 1 or close to 1, and all components of the literature precision P(B,A) will also be 1 or close to 1.

[0080] From the above examples, it can be inferred that when a query matches bibliographic data, it tends to follow one of the following patterns: (1) Although some components of the literature matching precision P(B,A) are slightly smaller, the other components of the literature matching precision P(B,A) and the literature matching recall R(A,B) are 1 or close to 1. (2) Although the literature recall R(A,B) is slightly smaller, all components of the literature precision P(B,A) are 1 or close to 1. (3) All components of the literature precision P(B,A) and the literature recall R(A,B) are all 1 or close to 1.

[0081] In particular, when the literature recall rate R(A,B) is somewhat small, as in (2), the outliers in Embodiment 1 also decrease, and it is conceivable that a query may be judged as not matching even though it actually matches the bibliographic data. If the judgment is made using the literature precision rate P(B,A), the likelihood of correctly determining a match increases even in cases like (2).

[0082] Furthermore, we will explain an example in Embodiment 1 where a query may be incorrectly determined to match bibliographic data even though it does not actually match. For example, consider the case where we want to identify a book with the query "Hanako ○○ (2020) 'A New Form of ●●ism' ×× Bookstore." On the other hand, consider the case where a book review (paper) about this book exists and that review is registered as bibliographic data. For example, suppose the title of that book review data is "<Book Review> Hanako ○○ 'A New Form of ●●ism' ×× Bookstore, 2020."

[0083] In this case, the token obtained from the query is "{○○}{Hanako}{2020}{●●}{ideology}{of}{new}{form}{××}{bookstore}". On the other hand, the token obtained from the book review title is "{book review}{○○}{Hanako}{●●}{ideology}{of}{new}{form}{××}{bookstore}{2020}{year}", which includes all the tokens obtained from the query. Therefore, the literature recall ratio R(A,B) becomes 1. As a result, depending on the degree of outlier, it may be incorrectly determined to be a match.

[0084] On the other hand, considering the literature matching precision P(B,A), the tokens obtained from the title have a high match rate with the tokens obtained from the query, so the component pt related to the title also has a high value. On the other hand, the author, publication year, journal, page number, etc. of the book review are different from the book being reviewed, so the px values ​​for each component other than the title are low. Therefore, the literature matching precision P(B,A) has a high value for only one component, and the remaining components have low values. This characteristic does not fall under either (1) or (2) above, so it is determined that the query does not match the bibliographic data of the book review.

[0085] As described above, the pair of similarity (recall rate R(A,B)) and precision rate P(B,A) correlates with the match rate for bibliographic data queries.

[0086] Based on the above explanation regarding the literature matching accuracy P(B,A), the differences between the bibliographic identification server 10 according to Embodiment 2 and that in Embodiment 1 shown in Figure 2 will be explained with reference to Figure 9.

[0087] First, this embodiment differs from Embodiment 1 in that it further includes a precision calculation unit 16. The precision calculation unit 16 calculates the document matching precision P(B,A) for each bibliographic data for each set of first tokens (set A) against a set of second tokens (set B). The precision calculation unit 16 outputs the calculated document matching precision P(B,A) to the determination unit 15.

[0088] Furthermore, the determination unit 15 differs from Embodiment 1 in the following respects. First, while Embodiment 1 only performed a matching determination for similarity values ​​that ranked first, Embodiment 2 differs in that it performs a matching determination for all similarity values.

[0089] Next, the matching method differs as explained below. First, a pre-trained model is built using machine learning to determine the degree of matching when a pair of similarity and literature matching precision (B,A) is input. For example, various queries are prepared in advance, and a full-text search is performed on the bibliographic database server 20 based on each query. Based on the bibliographic data obtained from the full-text search, a pre-trained model is built using supervised learning, where the input is a pair of similarity between set B (bibliographic data) and set A (query), and the input is the literature matching precision (B,A) between set A and set B, and whether or not it matches the query is used as a correct / incorrect label.

[0090] Then, the determination unit 15 uses this trained model to determine whether the bibliographic data matches the query by inferring the degree of matching for each bibliographic data from the similarity calculated by the similarity calculation unit 14 and the precision calculation unit 16 (B,A).

[0091] Referring to Figure 10, an example of the bibliographic identification process by the bibliographic identification server 10 according to Embodiment 2 will be explained, highlighting the differences from Embodiment 1 shown in Figure 6. Note that the processes in steps S101 to S105 shown in Figure 10 are exactly the same as those in steps S101 to S105 shown in Figure 6, so their explanation will be omitted.

[0092] The precision calculation unit 16 of the bibliographic identification server 10 calculates the document matching precision for multiple first tokens against multiple second tokens for each bibliographic data (step S206). The precision calculation unit 16 outputs the calculated document matching precision for each bibliographic data to the determination unit 15.

[0093] The determination unit 15 determines whether the bibliographic data matches the query based on the similarity calculated in step S105 and the document matching accuracy calculated in step S206 for each bibliographic data (step S207).

[0094] The web server unit 11 presents the bibliographic data determined to match the query in step S207 as the result of bibliographic identification (step S208). Note that there may be multiple bibliographic data entries that match. In this case, all bibliographic data are presented as the result of bibliographic data. Also, there may be no bibliographic data that matches. In this case, similar to step S109 shown in Figure 6, the result of bibliographic identification is that the bibliographic data desired by the user was not found. The bibliographic identification server 10 then repeats the process from step S101.

[0095] The bibliographic identification system 1 according to Embodiment 2 has been described above. Similar to Embodiment 1, the bibliographic identification server 10 of the bibliographic identification system 1 extracts multiple first tokens from a query and multiple second tokens from bibliographic data. This token extraction can be applied to both English and Japanese. The bibliographic identification server 10 then performs bibliographic identification based on the similarity of the tokens. Therefore, the bibliographic identification server 10 can appropriately identify bibliographies.

[0096] Furthermore, in Embodiment 2, since the matching accuracy is determined using the literature matching accuracy rate, it is possible to make appropriate determinations even in cases where there is a possibility of misjudgment in Embodiment 1. In addition, when bibliographic data is registered in multiple languages, the highest value among the components px of the literature matching accuracy rate calculated for each language is adopted. Therefore, Embodiment 2 can appropriately process cases where bibliographic data is registered in multiple languages.

[0097] (modified version) In Embodiments 1 and 2, the bibliographic data acquisition unit 12 of the bibliographic identification server 10 acquired bibliographic data from the bibliographic DB server 20 each time a query was entered. Alternatively, the bibliographic data acquisition unit 12 may, in advance, use the bibliographic data acquisition function provided by the bibliographic DB server 20 to acquire all bibliographic data stored by the bibliographic DB server 20 and save it to a storage device not shown. In this case, the bibliographic data acquisition unit 12 performs a full-text search on all bibliographic data stored in the storage device and outputs the bibliographic data obtained by the full-text search to the token extraction unit 13.

[0098] In Embodiments 1 and 2, the bibliographic identification system 1 was equipped with only one bibliographic DB server 20, but there may be multiple bibliographic DB servers 20. For example, the bibliographic identification system 1 may be equipped with two bibliographic DB servers 20, one for storing bibliographic data of academic papers and another for storing bibliographic data of books.

[0099] In Embodiments 1 and 2, the reference recall rate R was used as an index to represent similarity. Alternatively, similarity coefficients such as the Jaccard coefficient, Simpson coefficient, or Dice coefficient may be used as an index to represent similarity. However, the Jaccard coefficient and Dice coefficient are greatly influenced by irrelevant bibliographies when the bibliographic data includes many irrelevant bibliographies. Similarly, the Simpson coefficient is greatly influenced by irrelevant bibliographies when the query includes many irrelevant bibliographies. Therefore, it is preferable to use the reference recall rate R as the similarity index.

[0100] In Embodiment 1, the determination unit 15 determined whether the bibliographic data matched the query based on the three factors shown in (1), (2), and (3) above: similarity, outlier score, and number of elements. However, it is not necessary to use all of these factors; for example, matching may be determined based only on similarity and outlier score. Alternatively, matching may be determined based only on similarity and number of elements, or based on either similarity or outlier score. As mentioned above, since (1) and (2) are correlated with the matching rate, it is sufficient to use at least one of similarity and outlier score to determine matching. However, using all of similarity, outlier score, and number of elements will increase the accuracy of the matching determination.

[0101] In Embodiments 1 and 2, a trained model is used to determine the matchingness by the determination unit 15. However, instead of using a trained model, matchingness may be determined by a threshold-based determination. For example, matchingness may be determined based on conditions such as the similarity being above a predetermined threshold and the outlier being above a predetermined threshold. This threshold may be determined, for example, empirically, or it may be obtained by analyzing the trained model.

[0102] In embodiments 1 and 2, word-level n-grams were applied for token extraction, but instead, character-level n-grams may be applied, for example. In this case, morphological analysis can be omitted, thus reducing the processing burden during token extraction. However, applying word-level n-grams is likely to result in higher accuracy in matching determination.

[0103] In Embodiments 1 and 2, the bibliographic identification server 10 communicated with one bibliographic DB server 20. However, the bibliographic identification server 10 may communicate with multiple bibliographic DB servers 20 and perform a full-text search on multiple bibliographic DB servers 20 collectively based on one query to obtain bibliographic data. In this case, bibliographic data for the same document may be obtained from multiple bibliographic DB servers 20. Furthermore, even for the same document, the bibliographic data may differ depending on the bibliographic DB server 20. In particular, for books, there is a high possibility that the bibliographic data will not perfectly match. To address such cases, for example, if the ISBN (International Standard Book Number) included in the bibliographic data is the same, it is preferable to treat them as the same book and adopt only the one with the higher similarity.

[0104] In Embodiment 2, the literature matching accuracy P(B,A) was defined as a vector value consisting of multiple components px, but the average value of each component px may also be used as the literature matching accuracy P(B,A). For example, when the bibliographic data pertains to an article, there are 6 subsets, so the literature matching accuracy P(B,A) is the sum of the 6 components px divided by 6.

[0105] In the field of academic papers, when citing other papers, the location of the paper is sometimes described by author name, journal, page number, etc., without including the title. Let's consider a case where such a description is entered as a query. In Embodiment 2, if it is clear that the query does not include the title, it is possible to consider not considering the title component pt when calculating the literature matching precision. Whether or not a query does not include the title can be determined, for example, by the submission guidelines of the journal in which the paper is published, query syntax analysis, or options set by the user when entering the query. In this way, if it is known in advance that certain items will not be included, underestimation can be prevented by not considering the component px corresponding to those items. For example, it is possible to prepare a separate trained model that is built without using the title, and use that separate model when the title is not considered. Alternatively, as described above, the average value of each component px may be defined as the literature matching precision P(B,A), a trained model may be prepared based on this literature matching precision P(B,A), and the title may be excluded from the calculation of the average value when the title is not considered.

[0106] In the field of academic papers, English journal titles are often abbreviated (for example, abbreviations defined by ISO (International Organization for Standardization) 4). Therefore, in Embodiment 2, when calculating the literature matching accuracy, in addition to calculating each component px for each language, it is conceivable to identify what the abbreviation is, calculate each component px for the identified abbreviation, and adopt the highest value among the calculated values. This allows for appropriate processing even when, for example, the bibliographic data is registered in its full form, but the query is entered in its abbreviated form.

[0107] In the hardware configuration shown in Figure 5, the bibliographic identification server 10 is equipped with a secondary storage device 1004. However, the configuration is not limited to this; the secondary storage device 1004 may be located outside the bibliographic identification server 10, and the bibliographic identification server 10 and the secondary storage device 1004 may be connected via an interface 1003. In this configuration, removable media such as USB flash drives and memory cards can also be used as the secondary storage device 1004. [Explanation of symbols]

[0108] 1. Bibliographic Identification System 10 Bibliographic Identification Server 11 Web Server Section 12. Bibliographic Data Acquisition Unit 13 Token Extraction Unit 14 Similarity calculation unit 15 Judgment section 16. Precision Calculation Unit 20 Bibliographic Database Server 30 User terminals

Claims

1. Computers, A token extraction unit that extracts multiple first tokens from the query entered by the user to search for bibliographic data. A similarity calculation unit calculates the similarity of a plurality of second tokens extracted from a plurality of bibliographic data to a plurality of first tokens for each of the plurality of bibliographic data. A determination unit determines whether the bibliographic data corresponding to the similarity ranked 1st among the calculated similarity scores for each bibliographic data matches the query. A program that makes it function as such.

2. Computers, A token extraction unit that extracts multiple first tokens from the query entered by the user to search for bibliographic data. A similarity calculation unit calculates the similarity of a plurality of second tokens extracted from a plurality of bibliographic data to a plurality of first tokens for each of the plurality of bibliographic data. For each of the aforementioned bibliographic data, a precision calculation unit calculates the precision of the set of multiple first tokens against the set of multiple second tokens. A determination unit determines whether the bibliographic data matches the query based on the calculated similarity and the document matching accuracy for each of the aforementioned bibliographic data. A program that makes it function as such.

3. The similarity calculation unit calculates the similarity by determining the literature recall rate of the set of the plurality of second tokens with respect to the set of the plurality of first tokens. The program according to claim 1 or 2.

4. The determination unit determines, based on the degree of outlier of the similarity ranked 1st, whether the bibliographic data corresponding to the similarity ranked 1st matches the query. The program according to claim 1.

5. Computers Extract multiple primary tokens from the query entered by the user to search for bibliographic data. For each of the multiple bibliographic data, the similarity of the multiple second tokens extracted from the bibliographic data to the multiple first tokens is calculated. It is determined whether the bibliographic data corresponding to the closest similarity among the calculated similarity scores for each bibliographic data matches the query. Bibliographic identification methods.

6. Computers Extract multiple primary tokens from the query entered by the user to search for bibliographic data. For each of the multiple bibliographic data, the similarity of the multiple second tokens extracted from the bibliographic data to the multiple first tokens is calculated. For each of the aforementioned bibliographic data, the document matching accuracy of the set of the plurality of first tokens with respect to the set of the plurality of second tokens is calculated. For each of the aforementioned bibliographic data, it is determined whether the bibliographic data matches the query based on the calculated similarity and the document matching accuracy. Bibliographic identification methods.

Images