Document processing program, document processing device, and document processing method

Abstract

To provide a document processing program, a document processing apparatus, and a document processing method for presenting appropriate text in accordance with a viewpoint to be reflected in annotation specifications and easily create highly appropriate annotation specifications.SOLUTION: A document processing apparatus 30 comprises: a sentence analysis unit 31 for analyzing an input sentence to generate the analysis result including a target expression corresponding to each predefined unique expression classification included in the sentence and an expression type indicating a grammatical role of each target expression; a search rule determining unit 32 for determining, based on viewpoints of the expression type, a search rule for examples corresponding to the viewpoints to be reflected in annotation specifications, for the expression type included in the analysis result; an example search unit 33 for searching the examples corresponding to the search rule for each viewpoint from an example set including the plurality of examples; and an output unit 36 for outputting the examples searched from the example set.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a document processing program, a document processing device, and a document processing method that are suitable for creating annotation specifications. [Background technology]

[0002] Machine learning in AI (Artificial Intelligence) requires large amounts of training data with appropriate content to be used with practical accuracy. Creating training data usually requires manual annotation. In annotation, metadata is added to the data to be learned in order to identify the content of the data. When the data to be learned is text data (sentences), for example, metadata is added to extract expressions of parts and events from the text. In the case of the text "oil leaking from hydraulic unit," metadata is added indicating that the part is "hydraulic unit" and the event is "oil leak."

[0003] Annotation is performed by multiple workers, so annotation specifications are established to prevent variations in annotation results. Even when annotation is performed by a single worker, variations in annotation results may occur over time, making it beneficial to establish annotation specifications. Workers perform annotation according to the annotation specifications. Therefore, the quality of the annotation specifications affects the quality of the training data. One of the factors that reduces the quality of annotation specifications is ambiguity in the specifications. Ambiguity causes variations in annotation results, reducing the quality of the training data.

[0004] In the following, we will assume that annotation is performed on text data, particularly when extracting named entities from text.

[0005] In order to create annotation specifications that are less ambiguous, it is desirable to search the text to be annotated from the perspective of the causes of ambiguity and check whether or not the relevant text is found.

[0006] However, when creating annotation specifications, it is difficult to comprehensively consider the perspectives that cause ambiguity. Even if the perspectives that cause ambiguity are known, it is time-consuming and laborious to find relevant examples from the text to be annotated. Furthermore, it is difficult to confirm the number of relevant examples.

[0007] Conventionally, inventions have been disclosed that extend training data by applying various rules to existing training data (see, for example, Patent Document 1). In these inventions, a method is used in which internally stored rules and data are applied to extend the training data.

[0008] However, with conventional methods, it is unclear whether the training data expanded by applying internally stored rules is actually included in the text to be annotated, and even if it is included, the proportion of the text to be annotated that is included is unknown.

[0009] When creating annotation specifications, it is best to base your decisions on expressions that are actually contained in the text to be annotated, especially expressions that make up a large proportion of the text. It is a waste of time to create annotation specifications for expressions that are not contained in the text.

[0010] In particular, with regard to vocabulary, unless annotation specifications are determined based on the expressions actually contained in the text to be annotated, it will not lead to the elimination of ambiguity in the specifications that arise from the annotation specifications. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-004399 Summary of the Invention [Problem to be solved by the invention]

[0012] In order to create annotation specifications that are highly valid, it is necessary to reduce the ambiguity in the annotation specifications that arises from the expression of the actual text to be annotated. However, checking the text to be annotated from various perspectives and reflecting this in the annotation specifications requires a lot of effort, time, and cost.

[0013] Simply expanding the example sentences in the training data based on rules that are unrelated to the text to be annotated cannot resolve the ambiguity of the annotation specifications.

[0014] The problem that this invention aims to solve is to provide a document processing program, document processing device, and document processing method that present appropriate text based on the perspective that should be reflected in the annotation specifications, and make it easy to create highly valid annotation specifications. [Means for solving the problem]

[0015] According to an embodiment, the document processing program causes a computer to function as: a sentence analysis unit that analyzes an input sentence and generates an analysis result including target expressions corresponding to each of predefined named entity classifications contained in the sentence, and expression types indicating the grammatical roles of each of the target expressions; a search rule determination unit that determines, for the expression types contained in the analysis result, search rules for cases corresponding to perspectives to be reflected in the annotation specifications based on the perspective for the expression types; a case search unit that searches for cases that match the search rules for each perspective from a case set containing multiple cases; and an output unit that outputs the cases searched from the case set. [Brief explanation of the drawings]

[0016] [Figure 1]1 is a block diagram showing the configuration of a system using a document processing apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of a document processing function in the document processing apparatus according to the embodiment. [Figure 3] 4 is a flowchart showing the operation of the document processing apparatus according to the present embodiment. [Figure 4] FIG. 10 is a diagram showing an example of an input sentence. [Figure 5] FIG. 4 is a diagram showing an example of a search rule according to the present embodiment. [Figure 6] FIG. 4 is a diagram for explaining determination of a search rule in the present embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a search result of a case set in this embodiment. [Figure 8] FIG. 10 is a diagram showing an example of a search result of a case set in this embodiment. [Figure 9] FIG. 4 is a diagram showing an example of annotation information according to the present embodiment. [Figure 10] FIG. 10 is a diagram showing an example of annotation information including information in which a target expression has been changed according to the present embodiment. [Figure 11] FIG. 10 is a diagram showing an example of annotation information including information in which a target expression has been changed according to the present embodiment. [Figure 12] FIG. 10 is a diagram showing an example of annotation information including information in which a target expression has been changed according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments will be described with reference to the drawings.

[0018] Fig. 1 is a block diagram showing the configuration of a system using a document processing device 10 according to this embodiment. In the system shown in Fig. 1, the document processing device 10 can communicate with a server 14 and various electronic devices via a network 12 such as the Internet to send and receive various types of data.

[0019] The document processing device 10 in this embodiment is realized by a computer such as a personal computer. As shown in FIG. 1, the document processing device 10 includes a processor 20, a memory 21, a storage device 24, an input unit 25, a display unit 26, and a communication unit 29.

[0020] The processor 20 realizes various functions by executing various programs (software) read from the storage device 24 to the memory 21. For example, the processor 20 realizes various functions by executing various programs (software), such as an OS (Operating System) and application programs, stored in the memory 21. For example, the processor 20 executes the document processing program 21a to realize a document processing function for outputting information that can grasp ambiguity when creating annotation specifications based on the text to be annotated. That is, the document processing function inputs example sentences included in the annotation specifications and, based on these example sentences, presents appropriate texts (case studies) that can serve as references for creating the annotation specifications in accordance with the perspectives to be reflected in the annotation specifications. This reduces the effort required to create annotation specifications and enables efficient consideration of annotation specifications.

[0021] The memory 21 stores programs and data executed by the processor 20 .

[0022] The storage device 24 stores various programs (software) such as an OS (Operating System) and application programs, as well as data required to execute the programs, in a non-volatile storage medium.

[0023] The data stored in the storage device 24 includes, for example, input sentence data 24a, analysis result data 24b, search rule data 24c, expression type viewpoint data 24d, decision rule data 24e, case set data 24f, case search result data 24g, annotation information data 24h, frequency data 24j, search result data 24k, and reference history data 24m. Details of each data will be described later.

[0024] Under the control of the processor 20, the input unit 25 controls input from an input device (for example, a keyboard, a mouse, a tablet, etc.) operated by a user.

[0025] The display unit 26, under the control of the processor 20, controls the display on a display such as an LCD (Liquid Crystal Display).

[0026] The communication unit 29 controls communication with the server 14 and electronic devices via the network 12 .

[0027] The document processing device 10 can be implemented either as a hardware configuration or as a combination of hardware resources and software (programs). The software is installed in advance on a computer from the network 12 or a non-transitory computer-readable storage medium, and executed by the processor 20 of the computer, causing the computer to perform the functions of each device.

[0028] In this embodiment, the document processing device 10 executes a document processing function based on the document processing program 21a to output information suitable for creating annotation specifications, but the document processing function may also be executed in the server 14.

[0029] In this case, the server 14 realizes the document processing function by executing, by a processor, a document processing program 14a that realizes the same document processing function as the document processing program 21a. The document processing device 10 requests the server 14 to execute processing using the document processing function, receives information as the processing result, and outputs it (displays it on the display unit 26).

[0030] The data to be processed by the server 14 may be provided by the document processing device 10, or may be acquired from a website (text database) or the like connected via the network 12.

[0031] 2 is a block diagram showing the configuration of a document processing function 30 that is realized when the processor 20 executes a document processing program 21a in the document processing device 10 of this embodiment. The document processing program 21a enables the document processing device 10 to function as a sentence analysis unit 31, a search rule determination unit 32, a case search unit 33, an annotation information generation unit 34, a frequency calculation unit 35, an output unit 36, a reference history registration unit 37, a search rule storage unit 41, a case set storage unit 42, and a reference history storage unit 43.

[0032] The sentence analysis unit 31 performs analysis such as morphological analysis, syntactic analysis, and semantic analysis on the input sentence data 24a (example sentence 51) that has been input, and outputs analysis result data 24b. For example, the sentence analysis unit 31 generates an analysis result including parts included in the text indicated by the input sentence data 24a (example sentence 51), target expressions corresponding to each of the events, and expression types indicating the grammatical roles of each of the target expressions, and outputs the analysis result data 24b (see FIG. 4(B)).

[0033] The input sentence data 24a is, for example, text data of an example sentence included in the annotation specification. The input sentence data 24a includes, for example, text showing the example sentence 51, a target expression to be extracted from the text, and information on the semantic class (location, event) of this target expression (see FIG. 4(A)), which are determined by analysis by the sentence analysis unit 31. Here, the "target expression" means an expression that is annotated in the sample text of the input annotation specification.

[0034] The analysis result data 24b includes, for example, target expressions corresponding to at least the parts and events indicated by the semantic classes contained in the input sentence data 24a (example sentence 51), and data indicating the expression type of each target expression (see Figure 4(B)).

[0035] The search rule determination unit 32 determines search rules for searching for cases corresponding to the viewpoints to be reflected in the annotation specifications based on the analysis results of the sentence analysis by the sentence analysis unit 31. That is, the search rule determination unit 32 determines search rules for searching for cases from a case set (case set data 24f) including a plurality of cases (texts) stored in the case set storage unit 42 from search rule data 24c including a plurality of pre-prepared search rules, and generates determination rule data 24e indicating the determined search rules. For each expression type corresponding to a target expression included in the example sentence 51 obtained by analysis by the sentence analysis unit 31, the search rule determination unit 32 refers to expression type viewpoint data 24d (see FIG. 6(B)) indicating the related viewpoint, and determines search rules for each viewpoint indicated by the expression type viewpoint data 24d. The search rules determined corresponding to the viewpoints for each target expression are defined as determination rule data 24e.

[0036] The search rule storage unit 41 stores search rule data 24c in which search rules corresponding to a plurality of viewpoints to be reflected in the annotation specifications are defined (see FIG. 5).

[0037] The case search unit 33 uses the search rule determined by the search rule determination unit 32 to search for cases that match the search rule indicated by the determination rule data 24e from the case set (case set data 24f) stored in the case set memory unit 42, and stores the cases as case search result data 24g (see Figures 7(C) and 8(C)).

[0038] The case set (case set data 24f) stored in the case set storage unit 42 includes text to be annotated.

[0039] Furthermore, the case search unit 33 limits the cases to be searched from the case set based on the reference history stored in the reference history storage unit 43. The reference history stored in the reference history storage unit 43 includes information indicating cases from the case set stored in the case set storage unit 42 that are not to be output as search results by the case search unit 33.

[0040] The annotation information generation unit 34 generates annotation information (annotation information data 24h) for the cases (case search result data 24g) searched by the case search unit 33 according to a viewpoint. The annotation information includes, for example, information indicating the cases searched by the case search unit 33, candidate expressions for the parts included in the cases, and candidate expressions for the events. Furthermore, the annotation information generation unit 34 can generate annotation information for cases searched by the case search unit 33 from the case set (case set data 24f) using a method according to a viewpoint corresponding to the target expression (expression type) included in the case, including information obtained by changing the target expression (part, event) (see FIGS. 10 to 12). The annotation information is output as a search result and can be used as a reference when reflecting the searched cases in the annotation specifications.

[0041] The frequency calculation unit 35 calculates the frequency of the cases retrieved by the case search unit 33 for each viewpoint corresponding to the search rule used in the search, and outputs the frequency data 24j.

[0042] The output unit 36 ​​outputs search results (search result data 24k) including all or some of the cases (case search result data 24g) searched by the case search unit 33, the annotation information (annotation information data 24h) generated by the annotation information generation unit 34, and the frequencies (frequency data 24j) calculated by the frequency calculation unit 35. The output unit 36 ​​outputs the data to be output after appropriately sorting and filtering. The output unit 36 ​​may output the search results in various ways, such as recording them in a data file in a predetermined format in the storage device 24, displaying them on the display unit 26, or transmitting them to an external device via the communication unit 29.

[0043] The reference history registration unit 37 stores reference history data 24m, which indicates cases included in the case set that are not to be output, in the reference history storage unit 43 in response to a user's operation on the input unit 25. The reference history data 24m stored in the reference history storage unit 43 includes reference history 53, which indicates cases that were included in search results 52 previously output by the document processing function 30 and that the user specified as having been reflected in the annotation specifications. The reference history data 24m also includes cases that the user specified as not requiring reflection in the annotation specifications. The case search unit 33 performs processing to remove the cases indicated by the reference history data 24m stored in the reference history storage unit 43 by the reference history registration unit 37 from the cases searched for in the case set.

[0044] Next, the operation of the document processing device 10 in this embodiment will be described with reference to the flowchart shown in FIG.

[0045] The following describes an example of outputting sample sentences suitable for creating annotation specifications, assuming annotation for named entity extraction related to two semantic classes, parts and events. Note that in this example, two types of semantic classes, parts and events, are assumed, but the types and number of semantic classes are not limited to these, and any type and number of semantic classes can be used.

[0046] The processor 20 of the document processing device 10 starts the document processing function 30 by executing the document processing program 21a.

[0047] First, the processor 20 (sentence analysis unit 31) receives input sentence data 24a (example sentence 51) via the input unit 25, for example, by a user operating an input device (step A1). Here, for example, the example sentence 51 included in the annotation specification is the subject of analysis.

[0048] The processor 20 also performs a sentence analysis on the input sentence data 24a (example sentence 51) and acquires an expression type corresponding to a target expression included in the example sentence 51 (step A2). For the sentence analysis, annotated expressions (target expressions) are used in addition to the text string.

[0049] 4(A) shows an example of an input sentence indicated by the input sentence data 24a (example sentence 51). By performing sentence analysis on the input sentence "The handle does not move" shown in FIG. 4(A), an expression type indicating a grammatical role is determined for each of the target expressions "handle" and "does not move."

[0050] FIG. 4B is a diagram showing an example of the expression type for each target expression as the analysis result of the example sentence 51.

[0051] For example, it shows that the expression type of the target expression "handle" is determined to be "noun phrase" and the expression type of the target expression "does not move" is determined to be "verb."

[0052] Next, based on the results of the sentence analysis, the processor 20 (search rule determination unit 32) determines a search rule for the case corresponding to the viewpoint to be reflected in the annotation specification (step A3). The search rule storage unit 41 stores search rule data 24c in which search rules corresponding to each of the viewpoints to be reflected in the annotation specification are defined, and selects a search rule from this in accordance with the analysis result for the example sentence 51.

[0053] FIG. 5 is a diagram showing an example of a search rule (search rule data 24c) stored in the search rule storage unit 41. As shown in FIG.

[0054] As shown in Figure 5, the search rule data 24c defines multiple perspectives to be reflected in the annotation specifications, such as "common vocabulary," "partially common vocabulary," "word order changed," etc., and defines search rules corresponding to each perspective. For example, for the perspective of "common vocabulary," a search rule for "examples containing the target expression" is set.

[0055] First, in order to select a search rule from the search rule data 24c according to the analysis result of the example sentence 51, the processor 20 generates expression type viewpoint data 24d indicating a related viewpoint for each expression type corresponding to a target expression included in the example sentence 51 obtained by analysis by the sentence analysis unit 31. Note that the viewpoint related to an expression type is identified according to each of the multiple expression types analyzed by the sentence analysis.

[0056] FIG. 6 is a diagram for explaining the determination of a search rule.

[0057] Fig. 6(A) shows the analysis result (Fig. 4(B)) for the example sentence 51 shown in Fig. 4(A), and Fig. 6(B) shows an example of expression type viewpoint data 24d corresponding to the analysis result shown in Fig. 6(A). As shown in Fig. 6(B), the expression type viewpoint data 24d has viewpoints set corresponding to each expression type for the target expression in the analysis result shown in Fig. 6(A).

[0058] For example, for the expression type "noun phrase" for the target expression "handle", multiple perspectives related to "noun phrase" are set, such as "common vocabulary", "partially common vocabulary", "paraphrases of compound words", "compound words", ... and "quantity". Similarly, related perspectives are set for the expression type "predicate" of the target expression "does not move" and all expression types.

[0059] The processor 20 selects a search rule for each perspective indicated by the expression type perspective data 24d from the search rule data 24c (Figure 5), determines a specific search rule for searching for cases from a set of cases, and sets the search rule determined corresponding to the perspective for each target expression as the determination rule data 24e.

[0060] For example, for the perspective "common vocabulary" related to the expression type "noun phrase" indicated by the expression type perspective data 24d, the search rule "examples containing the target expression" shown in Figure 5 is selected, and based on this search rule "examples containing the target expression", a specific search rule "examples containing handle" is determined in which the "target expression" is changed to "handle".

[0061] FIG. 6(C) shows an example of specific search rules for each viewpoint corresponding to each of the multiple target expressions included in the example sentence 51.

[0062] As shown in Figure 6(C), for each of the multiple target expressions, the processor 20 selects a search rule from the search rule storage unit 41 (search rule data 24c) according to the perspective corresponding to the display type, determines a specific search rule, and generates determination rule data 24e.

[0063] Next, the processor 20 searches for cases corresponding to the search rule from the case set (case set data 24f) stored in the case set memory unit 42 based on the specific decision rules determined for each of the multiple target expressions and the reference history data 24m pre-stored in the reference history memory unit 43 (step A4).

[0064] FIG. 7 is a diagram showing an example of the search results of the case set for the target expressions "handle" and "handle, not moving."

[0065] The processor 20 uses specific search rules for each perspective of the target expressions "handle" and "handle, not moving" shown in FIG. 7(B) to search for cases that match the search rules from the set of cases shown in FIG. 7(A).

[0066] Figure 7(C) shows the search results for each perspective of the target expressions "handle" and "handle, not moving."

[0067] Figure 7 (C1) shows an example of a search result based on the search rule "Examples containing handle" from the perspective of "Vocabulary common" for the target expression "handle," showing that examples such as "The handle is broken," "The handle is stuck and cannot be operated," and "The handle is broken" were found.

[0068] As shown in Figure 7(C1), examples containing multiple "handles" found using the search rule for the perspective "common vocabulary" can be used as a reference for creating annotation specifications to enable appropriate annotation of texts that contain multiple events corresponding to the target expression "handle."

[0069] Figure 7 (C2) shows an example of a search result based on the search rule "Examples in which the order of appearance of steering wheel and steering wheel is changed" from the perspective of "word order changed" for the target expression "steering wheel, doesn't move." For example, it shows that the example "The car won't move, the steering wheel doesn't work" was searched.

[0070] As shown in Figure 7 (C2), the examples retrieved using the search rule for the perspective "word order changed" can be used as a reference for creating annotation specifications to enable appropriate annotation of text in which the words corresponding to the target expression appear in an order different from that of the input example sentence.

[0071] Figure 7 (C3) shows an example of a search result based on the search rule "Examples containing compounds containing "handle"" for the perspective "compound word" of the target expression "handle." The perspective "compound word" can be "front," "back," or "before and after," and for each of these, examples such as "The door handle won't move," "The handle lever won't move," and "The front wheel handle lever won't move" were found.

[0072] As shown in Figure 7 (C3), examples retrieved using the search rule for the perspective "compound word" can be used as a reference for creating annotation specifications to determine whether the entire compound word should be treated as a single expression, or whether "before," "after," or "before and after" should each be included or not, enabling appropriate annotation.

[0073] Figure 7 (C4) shows an example of a search result based on the search rule "Examples of adding prefixes and suffixes to handles" for the perspective "suffix" of the target expression "handle." The perspective "suffix" can be either a "prefix" or a "suffix," and for each, examples (prefix) "The handle in question is broken," "Each handle is broken," and example (suffix) "The handle, etc. is damaged" are shown to have been searched.

[0074] As shown in Figure 7 (C4), the examples retrieved using the search rule for the perspective "Affix" can be used as a reference for creating annotation specifications to determine whether the entire affix, including the "prefix" or "suffix," should be treated as a single expression for text containing an affix, or whether the "prefix" and "suffix" are included or not, and to enable appropriate annotation.

[0075] Figure 7 (C5) shows an example of a search result based on the search rule "Examples containing expressions with modifiers for handles" for the perspective "modification" of the target expression "handle." For example, it shows that the examples "The handle on the right side doesn't work" and "The repaired handle doesn't work" were retrieved.

[0076] As shown in Figure 7 (C5), the examples retrieved using the search rule for the perspective "modification" can be used as a reference for creating annotation specifications to determine whether text with a modifier attached to the target expression contains or does not contain a modifier, enabling appropriate annotation.

[0077] Figure 7 (C6) shows an example of search results based on the search rule "Examples containing expressions with different character types for handle" for the target expression "handle" from the viewpoint of "Difference in character type." The viewpoint of "Difference in character type" can be "full-width hiragana" or "half-width katakana," and examples including "handle" or "handle" in half-width katakana were found.

[0078] As shown in Figure 7 (C6), the examples retrieved using the search rule for the perspective "difference in character types" can be used as a reference for creating annotation specifications to enable appropriate annotation of expressions that have the same meaning but are written in different character types.

[0079] Similarly, the processor 20 uses specific search rules for each perspective of the target expressions "handle" and "handle, not moving" to search for cases that match the search rules from the case set. Detailed explanations will be omitted.

[0080] FIG. 8 is a diagram showing an example of the search results of the case set for the target expression "does not move."

[0081] The processor 20 uses specific search rules for each perspective of the target expression "not moving" shown in FIG. 8(B) to search for cases that match the search rules from the case set shown in FIG. 8(A).

[0082] FIG. 8(C) shows the search results for each perspective of the target expression "does not move."

[0083] Figure 8 (C1) shows an example of a search result based on the search rule "Examples containing 'doesn't move'" from the perspective of "common vocabulary" for the target expression "doesn't move," showing that the examples "the car doesn't move," "the vehicle doesn't move," "the wipers don't work," and "the meter doesn't work" were retrieved.

[0084] Figure 8 (C2) shows an example of a search result based on the search rule "Examples containing expressions with modifiers for 'doesn't move'" for the perspective "modification" of the target expression "doesn't move." For the perspective "modification," for example, the examples "The handle doesn't move at all" and "Sometimes the handle doesn't move" were found.

[0085] Figure 8 (C3) shows an example of a search result based on the search rule "Examples including expressions with a modified ending of 'do not move'" for the perspective of 'verbal transformation' for the target expression 'do not move'. For the perspective of 'verbal transformation', examples such as 'the handle moves', 'the handle does not seem to move', 'the handle cannot be moved', 'the handle is not moving', and 'the handle did not move' were found.

[0086] Figure 8 (C4) shows an example of a search result based on the search rule "Examples containing expressions using different character types for 'do not move'" from the viewpoint of "Difference in character type" for the target expression "does not move." The viewpoint of "Difference in character type" can be "Full-width hiragana," and for example, the example "The handle does not move" was found.

[0087] Similarly, the processor 20 uses specific search rules for each perspective of the target expression "does not move" to search for cases that match the search rules from the case set. Detailed explanations will be omitted.

[0088] In this way, by using specific search rules for each perspective to search for cases that match the search rules from the case set for the target expression ``does not move,'' the results can be used as a reference for creating annotation specifications for each perspective, just like the cases searched for each perspective corresponding to the target expression ``handle'' mentioned above.

[0089] In addition, the case search unit 33 uses search rules to limit the cases to be searched from the case set stored in the case set memory unit 42 based on the reference history data 24m stored in the reference history memory unit 43.

[0090] The reference history storage unit 43 stores reference history data 24m indicating cases that have been previously specified by the user as having been reflected in the annotation specifications, or cases that have been specified by the user as not requiring reflection in the annotation specifications.

[0091] In this way, by limiting the cases retrieved from the case set by the case retrieval unit 33, it is possible to avoid creating annotation information that is not required for creating annotation specifications, which is executed in the next process.

[0092] Next, the processor 20 (annotation information generation unit 34) generates annotation information (annotation information data 24h) for the cases (case search result data 24g) searched by the case search unit 33, based on rules determined according to the viewpoint of each target expression (step A5). The annotation information includes, for example, information indicating the cases searched by the case search unit 33, and expressions of parts and events included in the cases. The processor 20 estimates target expressions that are candidates for parts and events using a method according to the viewpoint of the search rule used to search the cases.

[0093] The case set (case set data 24f) stored in the case set storage unit 42 includes, for each case, a case (text) and information on the viewpoint of the search rule. Based on the case and the information on the viewpoint of the search rule, the processor 20 estimates target expressions to be candidates for the "site" and "event" of the annotation information.

[0094] For example, in the case of the viewpoint "common vocabulary," a target expression included in the case that is common to the target expression in example sentence 51 (input sentence data 24a) is associated with the semantic class ("body part" or "event") of the target expression in example sentence 51. In addition, other independent words included in the case are associated with a semantic class ("event" or "body part") different from the target expression in example sentence 51.

[0095] Fig. 9 shows an example of annotation information generated for a searched case. Fig. 9(A) shows an example of a case searched for using the search rule for the target expression "handle" with the viewpoint "common vocabulary," and Fig. 9(B) shows annotation information for the case shown in Fig. 9(A).

[0096] For example, the annotation information for the searched case "The handle is broken" shown in Figure 9(A) includes the case "The handle is broken" searched by the case search unit 33, information indicating the part "handle" of the target expression contained in the case, and the event candidate "broken", as shown in Figure 9(B).

[0097] Furthermore, for the example shown in Figure 9(A) "The handle is stuck and cannot be operated," the part "handle" and multiple event candidates "stuck" and "inoperable" are estimated as shown in Figure 9(B).

[0098] In this way, even if a case includes multiple events, the target expression for each event can be estimated and annotation information can be obtained, making it easier to consider when creating annotation specifications.

[0099] In addition, the processor 20 (annotation information generation unit 34) can generate annotation information for cases searched from the case set by the case search unit 33, including information that changes the target expression ("part", "event") based on the viewpoint.

[0100] 10, 11, and 12 are diagrams showing examples of annotation information including information on a modified target expression.

[0101] FIG. 10(A) shows an example of a search result obtained by using the search rule for the perspective "verbal transformation" for the target expression "does not move."

[0102] In this case, as shown in Figure 10(B), multiple candidate target expressions corresponding to the event are presented according to the viewpoint "verbal transformation." For example, in the example "The handle didn't move," two candidate target expressions corresponding to the event are presented: "didn't move" and "doesn't move."

[0103] This process can be performed, for example, by morphologically analyzing the expression "didn't move," which is the result of transforming a verb, and using a method that outputs two versions: one that includes the inflectional ending, and one that does not.

[0104] In this way, in the case of the perspective of "verbal transformation," since the words to be expressed have conjugations (morphological changes), annotation information can be generated that estimates the target of expression, including or excluding parts that change due to conjugation, for cases searched using search rules for a perspective that indicates that the target of expression has transformations.

[0105] FIG. 11(A) shows an example of a search result obtained by using the search rule with the viewpoint "common vocabulary" for the target expression "does not move."

[0106] In this case, as shown in Figure 11(B), the target expression corresponding to the body part is inferred from each example, and examples include "car," "vehicle," "wiper," "meter," etc. In this case, if there is an existing ontology, it is possible to add information on whether it is a higher-level concept than "steering wheel" and output it.

[0107] Figure 12(A) shows an example of a search result based on the search rule for the perspective "compound word" for the target expression "handle." As shown in Figure 12(A), examples corresponding to each of the perspectives "before," "after," and "before and after" of the perspective "compound word" are searched for.

[0108] In this case, as shown in Figure 12(B), target expressions corresponding to parts are inferred according to the "before," "after," and "before and after" aspects of the "compound word" so that all morphemes that make up the compound word are treated as a single target expression, and the morphemes before and after the target expression used during the search are inferred as individual target expressions. For example, for the "before" aspect of the "compound word," "door handle," which includes the "door" in "before" of "handle," is inferred as the target expression, and the "door" and "handle" in front are inferred as individual target expressions.

[0109] In this way, in the case of the perspective "compound word," since the word to be expressed contains multiple morphemes, annotation information can be generated that estimates the expression target for the entire compound word or for each morpheme contained in the compound word.

[0110] In this way, multiple different annotation information can be generated for cases retrieved from a case set, thereby increasing the variety of information that can be referenced when creating annotation specifications.

[0111] Next, the processor 20 (frequency calculation unit 35) calculates the frequency (number of cases) of cases retrieved using the search rule for each viewpoint of each target expression, and stores the calculated frequency data 24j (step A6).

[0112] Next, the processor 20 (output unit 36) outputs the annotation information (annotation information data 24h) generated by the annotation information generation unit 34 and the frequency calculated by the frequency calculation unit 35 (frequency data 24j) as search results.

[0113] In this way, the document processing device 10 in this embodiment searches for appropriate examples (text) from a set of examples containing the text to be annotated, according to the perspective to be reflected in the annotation specifications, and presents them as annotation information.By referring to these examples, the burden of considering annotation specifications is reduced and highly valid annotation specifications can be easily created.

[0114] Furthermore, by referring to the frequency data 24j, it is possible to determine whether or not something should be added to the annotation specifications, which has the effect of efficiently identifying the perspectives that should be given priority for consideration.

[0115] In addition, since ambiguities in annotation specifications can be specifically understood based on actual examples of annotation targets, the effort required to create annotation specifications is reduced.

[0116] Note that the above explanation assumes annotation of Japanese text, but it can also be used to create annotation specifications assuming annotation of text in other languages, such as English.

[0117] In addition, in this embodiment, the semantic classes of parts and events have been used as specific examples of named entity extraction, but the present invention is not limited to these and can be used to create annotation specifications for any semantic class. Furthermore, the number of semantic classes is not limited to two, and can be used to create annotation specifications for any number of semantic classes greater than or equal to one.

[0118] In addition, the techniques described in the embodiments can also be stored and distributed as a program that can be executed by a computer on a storage medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a magneto-optical disk (MO), or a semiconductor memory.

[0119] Furthermore, the storage medium may be in any storage format as long as it can store the program and is computer-readable.

[0120] In addition, an OS (operating system), database management software, network software, or other MW (middleware) running on a computer may execute some of the processes required to realize the above embodiments based on instructions from a program installed on the computer from a storage medium.

[0121] Furthermore, the storage medium in the embodiments is not limited to a medium independent of a computer, but also includes a storage medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like.

[0122] Furthermore, the storage medium is not limited to one, and the case where the processes in each of the above embodiments are executed from a plurality of media is also included in the storage medium of the present invention, and the media configuration may be any configuration.

[0123] In addition, the computer in the embodiment executes each process in the embodiment based on a program stored in a storage medium, and may be configured as either a single device such as a personal computer, or a system in which multiple devices are connected to a network.

[0124] Furthermore, the term "computer" in the embodiments includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment and devices that can realize the functions of the present invention by a program.

[0125] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0126] 10...document processing device, 12...network, 14...server, 20...processor, 21...memory, 21a...document processing program, 24...storage device, 24a...input sentence data, 24b...analysis result data, 24c...search rule data, 24d...expression type viewpoint data, 24e...decision rule data, 24f...case set data, 24g...case search result data, 24h...annotation information data, 24j...frequency data, 24k...search result data, 24m...reference history data, 25...input unit, 26...display unit, 29...communication unit, 31...sentence analysis unit, 32...search rule determination unit, 33...case search unit, 34...annotation information generation unit, 35...frequency calculation unit, 36...output unit, 37...reference history registration unit, 41...search rule memory unit, 42...case set memory unit, 43...reference history memory unit.

Claims

1. Computer, a sentence analysis unit that analyzes an input sentence and generates an analysis result including target expressions corresponding to each of predefined named entity classifications included in the sentence and expression types indicating the grammatical roles of each of the target expressions; a search rule determination unit that determines, for the expression type included in the analysis result, a search rule for cases corresponding to a viewpoint to be reflected in the annotation specification based on the viewpoint for the expression type; a case search unit that searches a case set including a plurality of cases for cases that match the search rule for each of the viewpoints; a document processing program for causing the program to function as an output unit that outputs cases retrieved from the case set;

2. 2. The document processing program according to claim 1, wherein the named entity classification is a part and an event.

3. Computer, a frequency calculation unit that calculates the frequency of the cases retrieved by the case search unit for each of the viewpoints; 3. The document processing program according to claim 1, wherein the output unit is caused to output the frequency together with the case.

4. Computer, further functioning as an annotation information generation unit that generates annotation information including information indicating the cases retrieved by the case search unit, a portion of a target expression included in the case, and an event associated with the portion; 4. The document processing program according to claim 1, further comprising causing the output unit to function as outputting the annotation information.

5. Computer, further functioning as a registration unit that stores data indicating cases that are not to be output among the cases included in the case set; 5. The document processing program according to claim 1, further comprising causing the case search unit to function to remove cases indicated by data recorded by the registration unit from cases searched for in the case collection.

6. Computer, The document processing program according to claim 4, further comprising causing the annotation information generation unit to generate annotation information for the cases searched for by the case search unit, the annotation information including information that modifies the target expression based on the viewpoint.

7. a sentence analysis unit that analyzes an input sentence and generates an analysis result including target expressions corresponding to each of predefined named entity classifications included in the sentence and expression types indicating the grammatical roles of each of the target expressions; a search rule determination unit that determines, for the expression type included in the analysis result, a search rule for cases corresponding to a viewpoint to be reflected in the annotation specification based on the viewpoint for the expression type; a case search unit that searches a case set including a plurality of cases for cases that match the search rule for each of the viewpoints; an output unit that outputs cases retrieved from the case set; A document processing device having:

8. A computer comprising: a sentence analysis step of analyzing an input sentence to generate an analysis result including target expressions corresponding to each of the predefined named entity classes included in the sentence and expression types indicating the grammatical roles of each of the target expressions; a search rule determination step of determining, for the expression types included in the analysis results, search rules for cases corresponding to viewpoints to be reflected in annotation specifications based on the viewpoints for the expression types; a case retrieval step of retrieving cases that meet the search rules for each of the viewpoints from a case set including a plurality of cases; and an output step of outputting the cases retrieved from the case set.

Images