Document analysis system, document analysis method, and program
Patent Information
- Application Number
- JP2022152232
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2022-09-26
- Publication Date
- 2025-08-04
AI Technical Summary
Existing document analysis methods, particularly for technical documents like patent publications, struggle with improving analysis accuracy due to the use of specific expression methods in each field, and researchers face challenges in understanding the disclosed technology and confirming intellectual property rights without manual reading and analysis.
A document analysis system that includes an annotation unit for setting named entity tags, a named entity extraction model, and a structure analysis section, utilizing pre-trained models like BERT for fine-tuning and re-training on specific technical fields, with delimiter tags to enhance accuracy and understanding.
Facilitates improved understanding and analysis of technical documents by accurately extracting and structuring named entities, enabling efficient automation of document classification and reducing search omissions.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a document analysis system for technical documents. [Background technology]
[0002] Extracting the necessary technical elements from a huge collection of technical documents is a big challenge. For example, there are many intellectual property-related documents, including patent publications. These intellectual property-related documents are increasingly being digitized and classified, making it easier to extract the desired documents by machine search using such classification and free keywords.
[0003] In recent years, analysis methods have been developed that use machine learning to organize and output the necessary information for documents. For example, BERT (Bidirectional Encoder Representations The development of a method called "Data mining from Transformers" has attracted attention as a method that can learn various word placement positions and similar expressions used in sentences and assign appropriate named entity tags to phrases (Non-Patent Document 1). There is also a report on PatentBERT, which focuses on patent data as an intellectual property-related document (Non-Patent Document 2). [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Gururangan, S., Marasovic, A., Swayamdipta, S., Lo, K., Beltagy,I., Downey, D., & Smith, NA (2020, July). Don'tStop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp.8342-8360). [Non-Patent Document 2] Lee, J.-S. and Hsiang, J.: Patentbert: Patent classificationwithfine-tuning a pre-trained bert model, arXiv preprint arXiv:1906.02124 (2019). Summary of the Invention [Problem to be solved by the invention]
[0005] The above-mentioned analysis method is capable of extracting appropriate named entities, which is expected to reduce the number of missed searches caused by simple keyword searches. Furthermore, it is expected to automate the classification of intellectual property-related documents that was previously performed manually. In this way, recent machine learning methods seem to be versatile for analyzing technical documents. However, there are many cases in which technical documents use specific expressions for each field, and there was a challenge to further improve the accuracy of analysis.
[0006] Furthermore, for researchers, the goal is not achieved simply by extracting related documents, but they must read and understand the documents themselves in order to understand what technologies are disclosed. Also, those who wish to confirm intellectual property rights must quickly understand the scope of rights disclosed in intellectual property-related documents such as patent documents in order to prevent patent infringement and to define new research areas.
[0007] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide a document analysis system etc. that makes it easier to understand technical documents. [Means for solving the problem]
[0008] The first invention for achieving the above-mentioned object is a document analysis system comprising: an annotation unit that accepts annotation settings for named entity tags for a limited document group, which is a group of documents in a specific technical field, and generates annotated teacher data, wherein the annotation unit is capable of setting delimiter tags to be assigned to character information that delimits a sequence of named entity tags that form a certain semantic relationship as the named entity tag; a named entity extraction model learning unit that uses the teacher data to generate a named entity extraction model; a document input unit that inputs documents in the technical field as targets for analysis; and a named entity extraction unit that uses the named entity extraction model to extract information of the delimiter tags from the documents to be analyzed together with information of other named entity tags.
[0009] According to the first invention, an annotation setting of named entity tags is accepted for a limited document group, which is a document group in a specific technical field, and a named entity extraction model is generated using annotated training data. In particular, in the present invention, when setting annotations, it is possible to set delimiter tags that are added to character information that delimits a named entity tag string that forms a certain semantic relationship as a named entity tag, and the named entity extraction model extracts information on the delimiter tags together with information on other named entity tags from the document to be analyzed. Semantic clusters (example named entity tags) can be identified from the document based on the extracted delimiter tags, making it easier to understand and analyze technical documents.
[0010] The document structure may further include a structure analysis unit that performs a structure analysis of the document by referring to the delimitation tags extracted by the named entity extraction unit. The structure analysis of the document to be analyzed can be easily performed by the delimitation tags.
[0011] The annotation unit may also receive annotation settings for named entity tags having a master-slave relationship, thereby making it possible to set annotations in which the named entity tags have a master-slave relationship.
[0012] The annotation unit may also receive annotation settings of related named entity tags for clue words that associate named entity tags with each other. By setting tags for words that associate named entity tags with each other, it is possible to improve the extraction performance of the named entity tags.
[0013] The named entity extraction model learning unit may also fine-tune a pre-trained model, which has been pre-trained using a large-scale document set, with the teacher data to generate the named entity extraction model. By fine-tuning the pre-trained model, which has been pre-trained using a large-scale document set, with annotated teacher data, it is possible to efficiently generate a named entity extraction model that is suitable for documents to be analyzed with high accuracy. Note that fine tuning refers to a technique in which a language model is trained using unsupervised data, and then the parameters of the language model are adjusted using teacher data for a specific task to be solved (in this invention, named entity extraction).
[0014] The system may further include a re-pre-training unit that re-pre-trains the pre-trained model using a restricted document group that is a document group in a specific technical field to generate a re-pre-trained model, and the named entity extraction model training unit generates the named entity extraction model using the re-pre-trained model and the teacher data. By using a re-pre-trained model that is obtained by re-pre-training a pre-trained model using a restricted document group limited to a specific technical field as the basis of the named entity extraction model, rather than a general-purpose pre-training model, it is possible to generate a named entity extraction model more efficiently and with higher accuracy.
[0015] The system may further include a re-annotation unit that receives a tagging correction operation for named entity tags extracted by inputting a limited document group, which is a document group in a specific technical field, into the named entity extraction model, and a named entity extraction model re-fine-tuning unit that re-fine-tunes the named entity extraction model using the limited document group in which the corrected named entity tags are set. Since errors in the tagging of named entities are corrected and the named entity extraction model is re-trained, the performance of the named entity extraction model is further improved.
[0016] In addition, it is preferable that the limited document group and the document to be analyzed are a document group and document related to intellectual property. Intellectual property-related documents are generally difficult to understand, but by applying the named entity extraction model that has been trained with the delimiter tags of the present invention, the documents can be easily understood.
[0017] A second invention is a document analysis method characterized in that a computer executes the following steps: an annotation process in which a computer accepts annotation settings for named entity tags for a limited document group, which is a group of documents in a specific technical field, and generates annotated teacher data, wherein the annotation process can set delimiter tags to be assigned to character information that delimits a sequence of named entity tags that form a certain semantic relationship as the named entity tag; a named entity extraction model learning process in which a named entity extraction model is generated using the teacher data; a document input process in which a document in the technical field is input as an analysis target; and a named entity extraction process in which information of the delimiter tags is extracted from the document to be analyzed together with information of other named entity tags using the named entity extraction model.
[0018] A third invention is a program that causes a computer to function as an annotation unit that accepts annotation settings for named entity tags for a limited document group, which is a group of documents in a specific technical field, and generates annotated teacher data, wherein the annotation unit is capable of setting delimiter tags to be assigned to character information that delimits a sequence of named entity tags that form a certain semantic relationship as the named entity tag; a named entity extraction model learning unit that uses the teacher data to generate a named entity extraction model; a document input unit that inputs documents in the technical field as targets for analysis; and a named entity extraction unit that uses the named entity extraction model to extract information of the delimiter tags from the documents to be analyzed together with information of other named entity tags. Effect of the Invention
[0019] The present invention provides a document analysis system and the like that makes it easier to understand technical documents. [Brief description of the drawings]
[0020] [Figure 1] FIG. 2 is a diagram illustrating a learning function of the document analysis system 1. [Diagram 2] 13 is a diagram showing an example of annotation settings for each named entity tag including a delimiter tag. FIG. [Diagram 3] A figure showing an example of annotation settings for main named entity tags, subordinate named entity tags, and related named entity tags. [Figure 4] FIG. 13 is a diagram illustrating a re-learning function of the document analysis system 1. [Diagram 5] FIG. 2 is a diagram illustrating an analysis function of the document analysis system 1. [Figure 6] FIG. 13 is a diagram illustrating an example of named entity extraction. [Figure 7] FIG. 13 is a diagram showing an example of a tag string pattern table 90. [Figure 8] FIG. 13 is a diagram showing an example of a structure analysis. [Figure 9] FIG. 2 is a diagram showing an example of a data structure of a document extraction DB 100. [Figure 10] FIG. 13 is a diagram illustrating an example of named entity extraction when delimiter tags are not set. [Figure 11] 1A and 1B are diagrams illustrating examples of structure analysis when no delimiter tags are set, in which (a) shows an example of correct structure analysis, and (b) shows an example of incorrect structure analysis. [Figure 12] FIG. 2 is a diagram illustrating a hardware configuration of a computer 30. [Figure 13] 13 is a flowchart showing the flow of a learning process. [Figure 14] 13 is a flowchart showing a flow of a re-learning process. [Figure 15] 13 is a flowchart showing the flow of an analysis process. [Figure 16] FIG. 2 is a diagram illustrating a search function of the document analysis system 1. [Figure 17] 13 is a flowchart showing the flow of a search process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0021] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
[0022] (1. Learning function) First, the learning function for learning the named entity extraction model 8 used in document analysis will be described with reference to Fig. 1. The learning function consists of "pre-learning", "re-pre-learning", and "actual learning", and the named entity extraction model 8 is generated by carrying out these learnings. Each learning function will be described.
[0023] (1-1. Preliminary learning) The pre-learning is executed by a pre-learning unit 14 included in the document analysis system 1. Specifically, as shown in FIG. 1, the pre-learning unit 14 performs unsupervised pre-learning using a large-scale document set 3 to generate a pre-learned model 6.
[0024] The large-scale document group 3 is a database (large-scale general-purpose corpus) in which sentences written on the Web, in books, etc. are structured and accumulated on a large scale, and serves as training data for training the pre-training model 6. In this embodiment, a database (large-scale corpus) in which articles on the Japanese version of Wikipedia are structured and accumulated on a large scale is used as the large-scale document group 3.
[0025] The learning of the pre-learning model 6 performed by the pre-learning unit 14 is preferably deep learning based on a neural network. In this case, if a transformer is used as the deep learning model, parallel processing is possible and the learning time can be shortened. Such deep learning can be performed, for example, by using a known natural language processing technology known as BERT. In this embodiment, the pre-learning unit 14 uses the BERT technology to learn the pre-learning model 6 using a large amount of document data obtained from Japanese Wikipedia.
[0026] Since pre-learning is a process that involves a high computational load using a large amount of data, it is desirable to perform it in advance. That is, it is desirable to prepare the pre-learning model 6 in advance and store it in the storage device 32 (FIG. 12) of the computer 30. In this case, the function and step of learning the pre-learning model 6 may be omitted from this embodiment.
[0027] (1-2. Re-learning in advance) The re-pre-learning is executed by the re-pre-learning unit 15 included in the document analysis system 1. Specifically, as shown in Fig. 1, the re-pre-learning unit 15 re-pre-learns the pre-learning model 6 using a limited document group 4A, which is a document group in a specific technical field, to generate the re-pre-learning model 7. Here, the documents in the technical field are, for example, documents such as technical papers and technical reports, and documents related to intellectual property.
[0028] The restricted document group 4A is a database (special purpose corpus) that structures and accumulates documents in a specific technical field to be analyzed, and serves as training data for pre-training the pre-training model 6. The restricted document group 4A is a group of documents in the same technical field as the target document 2 to be analyzed (see FIG. 5).
[0029] Moreover, it is desirable that the restricted document group 4A be intellectual property-related documents. Intellectual property-related documents are documents that publish the contents of patent applications and utility model registration applications, such as published patent gazettes, patent gazettes (patent publication gazettes), registered utility model gazettes, etc. When the restricted document group 4A is an intellectual property-related document, the specific technical field is, for example, a technical field classified by a patent classification symbol consisting of IPC, FI, F-term, etc.
[0030] The re-pre-training can be performed by a known technique known as BERT, similar to the pre-training. By re-pre-training the pre-trained model 6 using the restricted document set 4A, a learning model (re-pre-trained model 7) suitable for analysis of a specific technical field can be obtained. In addition, the use of the restricted document set 4A efficiently improves the learning accuracy by large-scale pre-training.
[0031] Although it is possible to train a pre-training model by BERT using the limited document group 4A from the beginning without using the large-scale document group 3, this is inefficient because it requires preparation of many training samples within a limited range. In this respect, it is effective in achieving both accuracy and efficiency to further improve accuracy by re-pre-training based on a universally available pre-training model 6, as in this embodiment.
[0032] Note that the re-pre-learning does not have to be performed. That is, the pre-learning model 6 may be prepared in advance and stored in the storage device 32 (FIG. 12) of the computer 30. In this case, the function and step of learning the re-pre-learning model 7 may be omitted from this embodiment.
[0033] (1-3. Practical learning) The actual learning includes a process of generating teacher data (annotation data) for learning the named entity extraction model 8, and a process of fine-tuning the pre-trained model 6 or the re-pre-trained model 7 using the teacher data (annotation data) to generate the named entity extraction model 8. Specifically, the actual learning is performed by the annotation unit 16 and the named entity extraction model learning unit 17 included in the document analysis system 1.
[0034] The annotation unit 16 receives annotation settings of various named entity tags from a user for the limited document group 4B, which is a document group in a specific technical field, and generates annotated teacher data (hereinafter, referred to as "annotation data 5").
[0035] The restricted document group 4B is a document group in the same technical field as the target document 2 to be analyzed and the restricted document group 4A. The restricted document group 4B is preferably intellectual property-related documents. The documents in the restricted document group 4B may be the same as or have some in common with the documents in the restricted document group 4A.
[0036] In this embodiment, the annotation unit 16 can set, as a named entity tag, a delimiter tag to be assigned to character information (characters, symbols, or a combination thereof) that delimits a named entity tag string that forms a certain semantic relationship.
[0037] Fig. 2 is a diagram for explaining an example of annotation setting for each named entity tag including a delimiter tag. Fig. 2 shows an example of annotation setting for an example claim sentence in the "Claims" section of an intellectual property-related document, "A steel material having a composition containing, by mass%, 0.03-1.5% C, 5-10% Ni or 0.7-2% Cu, 0.1-4% Co, and 5% or less Si."
[0038] In the example in Figure 2, the user sets five types of named entity tags for the sentence. "E" stands for the "element tag" set to the element name, "LF" stands for the "lower limit numeric tag" set to the lower limit of the numeric range, "UF" stands for the "upper limit numeric tag" set to the upper limit of the numeric range, "U" stands for the "unit tag" set to the unit expression of the numeric value, and "S" stands for the "delimiter tag."
[0039] The delimiter tag S is set to character information that delimits a sequence of named entity tags that form a certain semantic relationship. In the case of Figure 2, (a) "C is 0.03 to 1.5%" (Named entity tag sequence: "E", "LF", "UF", "U") (b) "Ni is 5 to 10%" (Named entity tag sequence: "E", "LF", "UF", "U") (c) "Cu 0.7-2%" (Named entity tag sequence: "E", "LF", "UF", "U") (d) "Co is 0.1 to 4%" (Named entity tag sequence: "E", "LF", "UF", "U") (e) "Si content of 5% or less" (Named entity tag string: "E", "UF", "U") is a description that specifies the content of each element contained in the steel material, and each of the above (a) to (e) forms a semantic relationship. The separator tag S is set to character information that separates each of the named entity tag strings of the above (a) to (e).
[0040] 2, for example, delimiter tags S are set for the character information "in," "," "or," "including," "further," "," and "consisting of," which separate the named entity tag strings (a) to (e) above. Note that the character information for setting delimiter tags S is not limited to specific characters or symbols, and may be any characters or symbols that separate named entity tag strings that form a semantic relationship.
[0041] The annotation unit 16 may also accept annotation settings for named entity tags having a master-slave relationship. Specifically, the annotation unit 16 accepts annotation settings that associate a master named entity tag T1 with a slave named entity tag T2 that is in a slave relationship with the master named entity tag T1.
[0042] The master named entity tag T1 is a named entity tag that is set for a named entity in a document of the restricted document group 4B. The subordinate named entity tag T2 is one or more named entity tags that are set subordinate to the master named entity tag T1 for a named entity in a document of the restricted document group 4B. The set master named entity tag T1 and subordinate named entity tag T2 are associated with each other and stored in the annotation data 5.
[0043] Specifically, in the case where the limited document group 4B is the "claims," which are intellectual property-related documents, a master named entity tag T1 can be set for a named entity relating to a specific invention-specific matter described in the claims, and a slave named entity tag T2 can be set for a named entity that more specifically expresses the nature, structure, etc. of the invention-specific matter to which the master named entity tag T1 is set.
[0044] For example, in the case of the field of chemistry, a master named entity tag T1 is set for a named entity (invention-specific matter) that represents a specific element or specific property, and a slave named entity tag T2 is set for a named entity that represents the range or degree of the specific element or specific property in association with the master named entity tag T1. This allows information to be learned about the range or degree (slave named entity) that is disclosed for a specific element or specific property (master named entity), making it possible to perform a detailed analysis of the target document 2. This type of tagging is particularly effective in intellectual property-related documents, where the scope of rights is often expressed in a master-slave relationship.
[0045] The annotation unit 16 may also accept the setting of a related named entity tag Tr from a user for a clue word that associates named entity tags with each other. For example, assume that a master named entity tag T1 is set for a named entity related to a specific element, and a slave named entity tag T2 is set for a named entity related to a numerical range of the specific element. In this case, a related named entity tag Tr is set for a named entity (cue word) that supplements the range or degree of a symbol or phrase expression that indicates the upper or lower limit of a numerical range. By setting the related named entity tag Tr in this manner, a further improvement in the performance of the named entity extraction model 8 can be expected.
[0046] Fig. 3 is a diagram for explaining an example of annotation setting of a main named entity tag T1, a subordinate named entity tag T2, and a related named entity tag Tr. Fig. 3 shows an example of annotation setting for the same claim example sentence as Fig. 2, in which the element name is set as the main named entity, and the lower limit numerical value, the upper limit numerical value, and the unit expression are set as subordinate named entities.
[0047] Specifically, an element tag T1_E, which is a master named entity tag T1, is set for the element name, and a lower limit tag T2_LF, an upper limit tag T2_UF, and a unit tag T2_U, which are slave named entity tags T2, are set for the lower limit and upper limit tags and unit expressions that specify the content of the element. In addition, a range tag Tr_R, which is a related named entity tag Tr, is set for "~", which indicates a range, as a clue word that relates the lower limit tag T2_LF and the upper limit tag T2_UF.
[0048] The named entity extraction model learning unit 17 generates a named entity extraction model 8 by fine-tuning the pre-trained model 6 or the re-pre-trained model 7 with the annotation data 5 (teacher data in which named entity tags are set in the limited document group 4B). Fine-tuning can be performed using a well-known technique known as BERT.
[0049] In this embodiment, a language model (pre-training model 6) is pre-trained using unsupervised data from Japanese Wikipedia, and the language model (re-pre-training model 7) is re-pre-trained using a limited document group 4A. Then, annotation data 5 (supervisor data), which is a dataset for the named entity extraction task, is used to adjust the parameters of the re-pre-training model 7 through fine tuning, and a named entity extraction model 8 is generated.
[0050] The learning function described above generates a named entity extraction model 8 that extracts named entity information from documents. Note that the named entity extraction model 8 may be generated without using the large-scale document set 3, the pre-training model 6, and the re-pre-training model 7. In other words, the named entity extraction model 8 may be directly trained and generated by machine learning based on the annotation data 5 that has been annotated for the limited document set 4B.
[0051] (1-4. Re-learning) Next, re-learning for further improving the learning accuracy of named entity extraction model 8 will be described with reference to Fig. 4. As shown in Fig. 4, re-learning is executed by document input unit 11, named entity extraction unit 12, re-annotation unit 18, and named entity extraction model re-fine-tuning unit 19 provided in document analysis system 1.
[0052] The document input unit 11 accepts input of a restricted document group 4C, which is a document group in a specific technical field. The restricted document group 4C is a document group in the same technical field as the target document 2 and the restricted document groups 4A and 4B. In addition, it is preferable that the restricted document group 4C is an intellectual property-related document.
[0053] The named entity extraction unit 12 extracts named entity information from each document of the limited document group 4C using the named entity extraction model 8, and outputs extracted data 9. In this embodiment, the extracted named entity information includes information on each named entity tag including the delimiter tag S.
[0054] The re-annotation unit 18 displays the extracted data 9 on the display device 34, accepts tagging correction operations (correction operations for each named entity tag including the delimiter tag S) for each extracted named entity tag, and re-generates a limited document group 4C (hereinafter referred to as "re-annotation data 20") in which the corrected named entity tags are set as training data.
[0055] The named entity extraction model re-fine tuning unit 19 re-fine-tunes the named entity extraction model 8 using the re-annotation data 20. The re-fine tuning can be performed using a known technique known as BERT. The re-fine tuning can be performed repeatedly until a predetermined learning performance is obtained.
[0056] Although a model suitable for analyzing the target document 2 is constructed through the re-pre-learning, actual learning, and re-actual learning, over-learning may result in a decrease in the accuracy of extraction from the target document 2. For this reason, it is desirable to appropriately perform generalization performance evaluation by cross-validation in each learning process of the re-pre-learning, actual learning, and re-actual learning.
[0057] (2. Analysis Function) Next, the analysis function of the document analysis system 1 will be described with reference to Fig. 5. The analysis function is mainly composed of a document input unit 11, a named entity extraction unit 12, and a structure analysis unit 13.
[0058] The document input unit 11 accepts input of one or more target documents 2 that are documents to be analyzed. The target documents 2 are documents in a specific technical field to be analyzed, and are documents in the same technical field as the limited document groups 4A, 4B, and 4C. In addition, it is preferable that the target documents 2 are intellectual property-related documents.
[0059] The named entity extraction unit 12 extracts named entity information from the target document 2 using the named entity extraction model 8, and outputs extracted data 10 from which the named entity information has been extracted to the structure analysis unit 13. In this embodiment, the extracted named entity information includes information on each named entity tag including the delimiter tag S.
[0060] Fig. 6 is a diagram showing an example of named entity extraction. Fig. 6 shows an example of named entity extraction performed on an example sentence of a claim in the "Claims" of an intellectual property-related document, "A steel material having a composition containing, by mass%, 0.01-2.0% C, 3-10% Ni or 0.5-1% Cu, 0.1-3% Co, and 3% or less Si." As shown in the figure, extracted data 10 is obtained as a result of named entity extraction, from which information on each named entity tag including the delimiter tag S is extracted.
[0061] The structure analysis unit 13 performs a structure analysis of the target document 2 based on the extracted data 10 extracted by the named entity extraction unit 12, and generates structured data 40. Specifically, the structure analysis unit 13 acquires each named entity tag string separated by the delimiter tag S from the extracted data 10 as a named entity tag string (a group of meanings) forming one semantic relationship. Then, the structure analysis unit 13 refers to the tag string pattern table 90 (FIG. 7), identifies the pattern of each acquired named entity tag string by pattern matching, and converts the information of each named entity tag string into structured data 40 in a data format corresponding to each pattern.
[0062] Fig. 7 is a diagram showing an example of a tag string pattern table 90. The tag string pattern table 90 is a table that holds patterns of named entity tag strings and data formats of structured data in advance in correspondence with each other. As shown in Fig. 7, the data format of structured data is determined in advance for each pattern of named entity tag strings. For example, pattern 1 is a named entity tag string pattern consisting of "element tag E" -> "lower limit numeric tag LF" -> "upper limit numeric tag UF" -> "unit tag U", and the data format of the structured data for this pattern is (E: LF, UF, U).
[0063] In addition, when specifying the content of an element in a claim, there are cases where the content is described in the order of element → numerical value, such as "Contains 0.3% to 1% Cu," or the numerical value → element, such as "Contains 0.3% to 1% Cu." In either case, it is necessary to be able to recognize the content by pattern matching.
[0064] Therefore, the tag string pattern table 90 shown in FIG. 7 provides patterns in which element tag E appears at the beginning of the tag string, such as pattern 1 ("element tag E → lower limit numeric tag LF → upper limit numeric tag UF → unit tag U") and pattern 3 ("element tag E → upper limit numeric tag UF → unit tag U"), and patterns in which element tag E appears at the end of the tag string, such as pattern 2 ("lower limit numeric tag LF → upper limit numeric tag UF → unit tag U → element tag E") and pattern 4 ("upper limit numeric tag UF → unit tag U → element tag E").
[0065] The patterns in the tag string pattern table 90 are not limited to the example in FIG. 7, and any pattern of a named entity tag string can be stored in association with the data format of the structured data.
[0066] Fig. 8 is a diagram showing an example of structure analysis of a document. Fig. 8 shows an example of structure analysis performed based on the extracted data 10 of Fig. 6. As shown in Fig. 8, the structure analysis unit 13 first acquires each of the named entity tag strings Tc1 to Tc5 separated by a delimiter tag S from the named entity tag string included in the extracted data 10 as named entity tag strings (groups of meanings) forming one semantic relationship. Then, the structure analysis unit 13 refers to the tag string pattern table 90 (Fig. 7), identifies the pattern of each of the named entity tag strings Tc1 to Tc5 by pattern matching, and converts the information of each of the named entity tag strings Tc1 to Tc5 into structured data 40 in a data format corresponding to each pattern.
[0067] For example, the named entity tag sequence Tc1 to Tc4 is a tag sequence consisting of "element tag E" -> "lower limit numeric tag LF" -> "upper limit numeric tag UF" -> "unit tag U", and corresponds to pattern 1 in the tag sequence pattern table 90 (FIG. 7). Therefore, the information of the named entity tag sequence Tc1 to Tc4 is written in accordance with the data format (E: LF, UF, U) corresponding to pattern 1, as follows: (C: The data is converted to structured data 40: (Ni: 0.01, 2.0, mass%), (Ni: 3, 10, mass%), (Cu: 0.5, 1, mass%), (Co: 0.1, 3, mass%).
[0068] Furthermore, the named entity tag string Tc5 is a tag string consisting of "element tag E" -> "upper limit numeric tag UF" -> "unit tag U", and corresponds to pattern 3 of the tag string pattern table 90. Therefore, the information of the named entity tag string Tc5 is written as (Si: , 3, mass%) is converted into structured data 40.
[0069] The structure analysis unit 13 stores the generated structured data 40 in the document extraction DB 100 in association with a document ID that uniquely identifies the document, document attributes, and the like.
[0070] Fig. 9 is a diagram showing an example of the data configuration of the document extraction DB 100 that stores the structured data 40. As shown in Fig. 9, the structured data 40 is stored for each document in association with the document ID 41, the document name 43, the document attributes 45, etc. For example, in the case of a document with the document ID 41 "T001" (document name 43: claims, document attributes 45: patent publication), (C: 0.01, 2.0, mass%), (Cu: 0.5, 1, mass%), (Co: 0.1, 3, mass%), (Si: , 3, mass%)... are stored as the structured data 40.
[0071] For comparison, a document structure analysis will be described below in detail when named entity extraction is performed without setting the delimiter tag S, which is different from the present embodiment. Fig. 10 shows an example of extracted data 10 from which named entity information has been extracted. The example sentences to be extracted are the same as those in Fig. 6. As shown in Fig. 10, since delimiter tag S has not been set, it is natural that delimiter tag S is not extracted. Named entity tags other than delimiter tag S are extracted in the same way as in the extraction result in Fig. 6.
[0072] FIG. 11 shows an example of structural analysis performed on the extracted data 10 in FIG. 10 with reference to the tag string pattern table 90 (FIG. 7). In the absence of a delimiter tag S, there are many combinations of patterns that match the named entity tag string, so there is a possibility that the structural analysis may be performed erroneously. For example, as shown in FIG. 11(a), when the pattern matching recognizes "Pattern 1 → Pattern 1 → Pattern 1 → Pattern 1 → Pattern 3," correct structured data 40 is obtained, as in FIG. 8 (an example of a structural analysis result of this embodiment). However, as shown in FIG. 11(b), when the pattern matching recognizes "Pattern 2 → Pattern 2 → Pattern 2 → Pattern 2," erroneous structured data 40 that does not fit the context of the claim is obtained.
[0073] In this way, without the delimiter tag S, the number of combinations of patterns that match with the named entity tag string increases, making the pattern matching complicated and potentially resulting in erroneous structural analysis. By introducing the delimiter tag S as in this embodiment, however, it is only necessary to consider matching of patterns of the named entity tag string delimited by the delimiter tag S (see FIG. 8), thereby reducing the complexity of pattern matching and enabling easy and accurate structural analysis of the document.
[0074] (3. Hardware Configuration) Next, the hardware configuration of a computer 30 to which the document analysis system 1 is applied will be described with reference to Fig. 12. As shown in Fig. 12, the computer 30 is configured by connecting a control unit 31, a storage device 32, an input device 33, a display device 34, a media input / output device 35, a communication I / F unit 36, a peripheral device I / F unit 37, etc. via a bus 39. However, the present invention is not limited to this, and various other configurations may be used as appropriate. Furthermore, the computer 30 may be configured by one or more computers.
[0075] The control unit 31 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The control unit 31 loads a program stored in the storage device 32, the ROM, a recording medium, etc., into a work memory area on the RAM, executes the program, and drives and controls each unit connected via a bus 39.
[0076] The ROM permanently stores the boot program, BIOS, and other programs, data, etc. of the computer 30. The RAM temporarily stores the loaded programs and data, and also includes a work area used by the control unit 31 to perform various processes described later.
[0077] Moreover, the control unit 31 executes the learning process shown in Figures 13 and 14, the analysis process shown in Figure 15, the search process shown in Figure 17, and the like, in accordance with the processing program stored in the storage device 32. The program for executing each process may be stored in advance in the storage device 32 or ROM of the computer 30, or may be downloaded via a network or the like and stored in the storage device 32, etc.
[0078] The storage device 32 is an HDD (hard disk drive) or the like, and stores the programs executed by the control unit 31, data necessary for executing the programs (pre-training model 6, re-pre-training model 7, named entity extraction model 8, document extraction DB 100, etc.), an OS (operating system), etc. These program codes are read out by the control unit 31 as necessary, transferred to the RAM, and read out by the CPU for execution.
[0079] The input device 33 is, for example, a keyboard, a mouse, a touch panel, a pointing device such as a tablet, a numeric keypad, etc., and outputs input data to the control unit 31.
[0080] The display device 34 is composed of a display device such as a liquid crystal panel or a CRT monitor, and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display device, and displays the display information inputted under the control of the control unit 31 on the display device. Note that the input device 33 and the display device 34 may be integrated into a touch panel type input / output unit.
[0081] The media input / output device 35 is an input / output device for various recording media such as a CD / DVD drive, and performs input / output of data. The communication I / F unit 36 has a communication control device, a communication port, etc., and is an interface that mediates communication with an external device connected for communication via a network, and performs communication control.
[0082] The peripheral device I / F unit 37 is a port for connecting peripheral devices to the computer 30, and the computer 30 transmits and receives data to and from the peripheral devices via the peripheral device I / F unit 37. The peripheral device I / F unit 37 is configured with USB, IEEE1394, or the like. The connection with the peripheral devices may be wired or wireless. The bus 39 is a path that mediates the transmission and reception of control signals, data signals, and the like between the devices.
[0083] (4. Processing of Document Analysis System 1) Next, a description will be given of the processing of the document analysis system 1. In the document analysis system 1, the control unit 31 of the computer 30 executes a learning process (FIGS. 13 and 14) for learning the pre-training model 6, the re-pre-training model 7, and the named entity extraction model 8. The control unit 31 also uses the named entity extraction model 8 learned by the learning process to extract information on named entities from the target document 2 to be analyzed, and executes an analysis process (FIG. 15) of the target document 2 based on the extracted data 10.
[0084] (4-1. Learning process) First, the learning process will be described with reference to Fig. 13. The control unit 31 (pre-learning unit 14) of the computer 30 performs unsupervised pre-learning using the large-scale document set 3 to generate a pre-learning model 6 (step S11, pre-learning process). The generated pre-learning model 6 is stored in the storage device 32.
[0085] Next, the control unit 31 (re-pre-training unit 15) of the computer 30 re-pre-trains the pre-training model 6 generated in step S11 using the limited document group 4A, which is a document group in a specific technical field, to generate the re-pre-training model 7 (step S12, re-pre-training process). The generated re-pre-training model 7 is stored in the storage device 32.
[0086] Next, we will explain the actual training process of generating teacher data (annotation data) for training the named entity extraction model 8, and fine-tuning the pre-training model 6 or re-pre-training model 7 using the teacher data (annotation data) to generate the named entity extraction model 8.
[0087] First, the control unit 31 (annotation unit 16) of the computer 30 accepts annotation settings of various named entity tags from a user for a limited document group 4B, which is a group of documents in a specific technical field, and generates annotated teacher data (annotation data 5) (step S13).
[0088] In particular, in this embodiment, the control unit 31 (annotation unit 16) of the computer 30 accepts the setting of a separator tag S that is assigned to character information that separates a sequence of named entity tags that form a certain semantic relationship as a named entity tag (see Figure 2).
[0089] The control unit 31 (annotation unit 16) may also receive from the user an annotation setting that associates a main named entity tag T1 with a subordinate named entity tag T2 that is subordinate to the main named entity tag T1 (see FIG. 3). The control unit 31 (annotation unit 16) may also receive from the user an associated named entity tag Tr setting for a clue word that associates named entity tags with each other (see FIG. 3).
[0090] Next, the control unit 31 (named entity extraction model learning unit 17) of the computer 30 generates a named entity extraction model 8 by fine-tuning the pre-training model 6 generated in step S11 or the re-pre-training model 7 generated in step S12 using the annotation data 5 (step S14, fine-tuning process). The above process generates the named entity extraction model 8. The generated named entity extraction model 8 is stored in the storage device 32.
[0091] 14 is a flowchart illustrating a re-learning process for further improving the learning performance of the named entity extraction model 8. The control unit 31 (document input unit 11) of the computer 30 accepts input of a limited document group 4C, which is a document group in a specific technical field (step S31). Next, the control unit 31 (named entity extraction unit 12) of the computer 30 extracts named entity information from each document of the limited document group 4C using the named entity extraction model 8, and outputs extracted data 9. In this embodiment, the extracted named entity information includes information on each named entity tag including the delimiter tag S.
[0092] Next, the control unit 31 (re-annotation unit 18) of the computer 30 displays the extracted data 9 on the display device 34, accepts tagging correction operations (correction operations for each named entity tag including the delimiter tag S) for each extracted named entity tag, and regenerates a limited document group 4C (re-annotation data 20) in which the corrected named entity tags are set as training data (step S33).
[0093] Then, the control unit 31 (named entity extraction model re-fine-tuning unit 19) of the computer 30 re-fine-tunes the named entity extraction model 8 using the re-annotation data 20 (step S34). Steps S31 to S34 can be repeatedly executed until a desired learning performance is obtained.
[0094] (4-2. Analysis processing) Next, the analysis process will be described with reference to Fig. 15. First, the control unit 31 (document input unit 11) of the computer 30 accepts input of one or more target documents 2 which are documents to be analyzed (step S51).
[0095] Next, the control unit 31 (named entity extraction unit 12) of the computer 30 extracts named entity information from the target document 2 using the named entity extraction model 8, and outputs extracted data 10 (step S52). In this embodiment, the extracted named entity information includes information on each named entity tag including the delimiter tag S.
[0096] Next, the control unit 31 (structure analysis unit 13) of the computer 30 performs a structure analysis of the target document 2 based on the extracted data 10, and generates structured data 40 (step S53). Specifically, the control unit 31 (structure analysis unit 13) acquires each named entity tag string separated by the delimiter tag S from the delimiter tag S included in the extracted data 10 as a named entity tag string (a group of meanings) forming one semantic relationship. The control unit 31 (structure analysis unit 13) then refers to the tag string pattern table 90 (FIG. 7) to identify the pattern of each acquired named entity tag string, and converts the information of each named entity tag string into structured data 40 in a data format corresponding to each pattern (see FIG. 8).
[0097] Then, the control unit 31 (structure analysis unit 13) of the computer 30 stores the generated structured data 40 in the document extraction DB 100 in association with a document ID that uniquely identifies the document, document attributes, and the like (step S54).
[0098] As described above, in this embodiment, annotation settings of named entity tags are accepted for the limited document group 4B, which is a document group in a specific technical field, and a named entity extraction model 8 is generated using annotated teacher data. Particularly in this embodiment, when setting annotations, it is possible to set a delimiter tag S that is assigned to character information that delimits a named entity tag string that forms a certain semantic relationship as a named entity tag, and the named entity extraction model 8 extracts information of the delimiter tag S together with information of other named entity tags from the target document 2 to be analyzed. This makes it possible to identify a named entity tag string (a group of meanings) that forms a single meaning from the target document 2 based on the delimiter tag S, making it easier to understand technical documents and analyze their structure.
[0099] (5. Document Search) 15 is executed for many target documents 2, a large amount of structured data 40 is accumulated for each document in the document extraction DB 100 (see FIG. 9). Here, document retrieval will be explained as an example of how the document extraction DB 100 is used.
[0100] 16 is a functional block diagram showing the document search function of the document analysis system 1. The document search function is mainly composed of a search condition setting section 21, a search section 22, and a search result display section .
[0101] The search condition setting unit 21 accepts inputs of search keywords, numerical conditions, unit conditions, etc. from the user and sets search conditions. The search unit 22 searches the document extraction DB 100 for documents containing structured data that satisfy the search conditions. The search result display unit 23 displays information on the searched documents.
[0102] The flow of the search process will be described with reference to Fig. 17. First, the control unit 31 (search condition setting unit 21) of the computer 30 accepts input of search keywords, numerical conditions, unit conditions, etc. from the user and sets search conditions (step S71). Next, the control unit 31 (search unit 22) searches the document extraction DB 100 for documents containing structured data that satisfy the search conditions set in step S71 (step S72). Then, the control unit 31 (search result display unit 23) displays the document search results on the display device 34 (step S73).
[0103] Although the preferred embodiments of the document analysis system 1 and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally fall within the technical scope of the present invention. [Explanation of symbols]
[0104] 1. Document analysis system 2. Target document 3. Large-scale document collection 4A~4C... Limited Document Group 5. Annotation data 6. Pre-learning model 7. Re-pre-training model 8. Named entity extraction model 9, 10………Extracted data 11.....Document input section 12……………… Named entity extraction unit 13…………Structural analysis department 14.…………Preparatory Study Section 15. Re-learning section 16...Annotation section 17……………Named entity extraction model training section 18. Re-annotation section 19. Named entity extraction model re-fine-tuning part 20. Re-annotated data 21...Search condition setting section 22...Search section 23...Search result display section 30……………Computer 40. Structured data 90....Tag string pattern table 100....Document extraction DB S………………Separator tag T1: Primary named entity tag T2: Sub-named entity tag Tr…Related named entity tag
Claims
1. An annotation unit that accepts annotation settings for specific expression tags for a limited document group that is a document group in a specific technical field and generates annotated teacher data, wherein the annotation unit can set a delimiter tag to be attached to character information that delimits a specific expression tag sequence that forms one semantic relationship as the specific expression tag, and A specific expression extraction model learning unit that generates a specific expression extraction model using the teacher data including the delimiter tag as a specific expression tag, A document input unit that inputs a document in the technical field as an analysis target, A specific expression extraction unit that extracts information of a delimiter tag together with information of other specific expression tags from the document to be analyzed using the specific expression extraction model, A document analysis system characterized by comprising the above.
2. A structure analysis unit that refers to a tag sequence pattern table that pre-associates and holds a pattern of a specific expression tag sequence and a data format of structured data, and identifies the pattern of each specific expression tag sequence by pattern matching, and converts the information of each specific expression tag sequence into structured data in a corresponding data format, further comprising The document analysis system according to claim 1, characterized in that.
3. The annotation unit accepts annotation settings for specific expression tags having a master-slave relationship, and includes the specific expression tags having the master-slave relationship in the teacher data for learning and extraction of specific expression extraction The document analysis system according to claim 1, characterized in that.
4. The annotation unit accepts annotation settings for related specific expression tags for a clue word that associates specific expression tags, and includes the related specific expression tags in the teacher data for learning and extraction of specific expression extraction The document analysis system according to claim 1, characterized in that.
5. The structure analysis unit refers to a tag sequence pattern table that pre-associates and holds a pattern of a specific expression tag sequence and a data format of structured data, identifies the pattern of each specific expression tag sequence by pattern matching, and converts the information of each specific expression tag sequence into structured data in a corresponding data format The document analysis system according to claim 2, characterized in that.
6. The tag sequence pattern table holds a plurality of patterns with different word orders for a specific expression tag sequence representing the same meaning The document analysis system according to claim 5, characterized in that.
7. The specific expression tags include element tags, numerical tags, and unit tags. The document analysis system according to claim 1, characterized in that.
8. The limited document group and the document to be analyzed are document groups and documents related to intellectual property. The document analysis system according to claim 1, characterized in that.
9. A computer, An annotation process in which, for a limited document group that is a document group in a specific technical field, an annotation setting of specific expression tags is accepted, and annotated teacher data is generated. In the annotation process, a delimiter tag that can be set for character information that delimits a specific expression tag sequence forming one semantic relationship as the specific expression tag is provided. A specific expression extraction model learning process of generating a specific expression extraction model using the teacher data including the delimiter tag as a specific expression tag. A document input process of inputting a document in the technical field as an analysis target. A specific expression extraction process of extracting information of the delimiter tag together with information of other specific expression tags from the document to be analyzed using the specific expression extraction model. A document analysis method characterized by executing.
10. A computer, An annotation unit that accepts an annotation setting of specific expression tags for a limited document group that is a document group in a specific technical field and generates annotated teacher data. In the annotation unit, a delimiter tag that can be set for character information that delimits a specific expression tag sequence forming one semantic relationship as the specific expression tag is provided. A specific expression extraction model learning unit that generates a specific expression extraction model using the teacher data including the delimiter tag as a specific expression tag. A document input unit that inputs a document in the technical field as an analysis target. A specific expression extraction unit that extracts information of the delimiter tag together with information of other specific expression tags from the document to be analyzed using the specific expression extraction model. A program characterized by causing it to function.