Document check system and document check program
The hybrid document checking system addresses inefficiencies in existing systems by using a check tool and machine learning for contextual correction, ensuring accurate and efficient defect detection with reduced user rechecking and computational load.
Patent Information
- Application Number
- JP2023075720
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-14
AI Technical Summary
Existing document checking systems suffer from high false positive rates due to superficial analysis and lack of contextual understanding, leading to inefficient proofreading and conflicting defect indications, while AI systems require extensive training data and computational resources.
A hybrid system combining a check tool for superficial defect detection and a machine learning model for contextual correction, with a user-intention-based re-determination process to validate defect warnings.
Reduces false positives and enhances proofreading efficiency by ensuring only valid defect warnings are displayed, minimizing user rechecking and reducing hardware load for model training.
Smart Images

Figure 2024160574000001_ABST
Abstract
Description
[Technical field]
[0001] An embodiment of the present invention relates to a document check system that detects defects in input document data and outputs a warning message, and a document check program for implementing the document check system. [Background technology]
[0002] 2. Description of the Related Art In recent years, document check systems have been developed that use information processing devices (computers) to analyze documents written in natural language and detect defects in the documents.
[0003] Document checking systems can be broadly divided into systems that check based on pre-set checking rules and artificial intelligence (AI) systems that use machine learning models.
[0004] A text checking system based on check rules points out all text that matches the rules, which can result in a high number of false positives and over-detections.
[0005] In other words, since the system points out all defects that fall under the rules without considering the context of the text (the text before and after), the position in the text structure (chapter / section, bullet points, within a table, etc.), the type of document (design document, contract, blog, paper, patent, etc.), etc., false positives occur frequently. In other words, document checking systems using checking tools only analyze the superficial parts of the text, resulting in an increase in false positives.
[0006] This resulted in users having to recheck many of the deficiencies pointed out by the checking system, which could reduce the efficiency of proofreading work.
[0007] In addition, depending on the specifications of multiple tools, the detected issues may conflict with each other. For example, if you correct an issue detected by a first tool, the corrected text may become an issue detected by a second tool.
[0008] On the other hand, in an AI document checking system, building an appropriate machine learning model requires the creation of a huge amount of training data, and model training takes a long time. Also, comprehensively checking an entire document places a large burden on the hardware and takes a long time. Furthermore, in an AI system, it is unclear why a detected defect was identified as a defect. This requires the user to infer the reason for the defect and judge the validity of the detection result, which can reduce the efficiency of proofreading work. [Prior art documents] [Patent documents]
[0009] [Patent Document 1] JP 2012-155645 A Summary of the Invention [Problem to be solved by the invention]
[0010] An object of the embodiment of the present invention is to provide an efficient document check system and a document check program for realizing the efficient document check system. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a configuration diagram of a document check system according to an embodiment. [Diagram 2] FIG. 2 is a configuration diagram of a document check system according to the embodiment. [Diagram 3] FIG. 3 is a flowchart of the document check system according to the embodiment. [Figure 4] FIG. 4 is a configuration diagram of the first determination unit according to the embodiment. [Diagram 5] FIG. 5 is a configuration diagram for explaining a learning method of the first determination unit according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] The document checking system of this embodiment includes a text checking unit that uses a checking tool to check input document data for specified deficiencies and outputs a warning message, a first judgment unit that uses a machine learning model to judge whether the warning message is correct, and a second judgment unit that re-judges the warning message that the first judgment unit has judged to be correct based on the user's intention.
[0013] The document check program of this embodiment is a program for a processor to execute a document check system that includes a text checking unit that uses a checking tool to detect specified deficiencies in input document data and output a warning message, a first judgment unit that uses a machine learning model to determine whether the warning message is correct, and a second judgment unit that re-judges the warning message that the first judgment unit has determined to be correct, based on the user's intention.
[0014] Hereinafter, the document check system of the embodiment will be described in detail with reference to the drawings.
[0015] As shown in Figures 1 and 2, a document check system 1 of the embodiment is a computer including a processor 2, a display 3, a keyboard 4, and a storage device 5. The processor 2, which is made up of a CPU or the like, loads a program 6 stored in the storage device 5 and constitutes a plurality of functional units, each of which executes a predetermined function. At least one of the plurality of functional units of the processor may be constituted by a dedicated hardware circuit. The display 3 displays the output result of the processor 2. A user inputs to the processor 2 using the keyboard 4.
[0016] The display 3 is a liquid crystal display, a CRT, or the like. The display 3 capable of touch input may also function as the keyboard 4. The storage device 5 is a hard disk, a silicon memory disk, or the like. The program 6 may be stored in a server 7.
[0017] 2 includes a display 3, a keyboard 4, and the like for a user to operate the computer. A document data reading unit 11, a text checking unit 13, a first judgment unit 15, a judgment model learning unit 18, a judgment result output unit 19, a second judgment unit 20, and a text checking tool management unit 22 are functional units of the processor 2.
[0018] The document data storage unit 12, the feature extraction rule storage unit 16, the judgment learning data storage unit 17, the second judgment result storage unit 21, and the text check tool storage unit 23 are configured by the memory of the storage device 5 or the server 7.
[0019] In document check system 1 shown in Fig. 2, writing check tool management unit 22 and writing check tool storage unit 23 are managed collectively by an administrator separate from the user. Writing check tool management unit 22 and writing check tool storage unit 23 may be configured by a server 7 separate from the hardware used by the user. Multiple users can access server 7. Note that the user and the administrator may be the same person, and document check system 1 may be a system used by a single user.
[0020] <Processing by Document Check System 1> The process performed by the document check system 1 will be described below with reference to the flowchart in FIG.
[0021] <Step S10> Reading document data A user inputs document data to be checked into a document data reading unit 11 via an interface 10. The natural language document data may be input using a keyboard 4, or may be pre-created document data stored in a storage device 5 or a document data storage unit 12.
[0022] <Step S15> Select a check tool The text check tool storage unit 23 stores a plurality of check tools corresponding to the type of document data, for example. The document types are letters, designs, contracts, blogs, papers, patents, etc. The processor 2 selects a check tool from the plurality of check tools based on the document type input in advance by the user.
[0023] Each checking tool includes tools that detect different kinds of defects: for example, each checking tool includes an error detection tool, an ambiguity detection tool, a spelling variation detection tool, and a grammar detection tool.
[0024] The error detection tool detects unregistered words, words with homonyms, specific word strings, obsolete kana spellings, etc. The ambiguity detection tool detects, for example, ambiguous sentences with multiple dependencies within a sentence. The spelling variation detection tool uses a pre-registered word dictionary to detect, for example, the unification of "window," "window," and "window." The grammar detection tool detects, for example, places where elements constituting a sentence are not correctly arranged. The elements constituting a sentence include subjects, predicates, objects, adjectives, adverbs, etc.
[0025] In particular, by using at least two or more tools, the document checking system 1 can comprehensively point out various defects. The tool is selected based on the user's selection. The processor 2 may automatically select a tool selected by the user in the previous check.
[0026] <Step S20> Check the text The text check unit 13 uses the selected check tool to detect imperfect descriptions in the input document data. The text check unit 13 detects all imperfections that meet the predetermined rules.
[0027] <Step S25> All check tools used? The process of step S20 is repeated until the checks by all the selected checking tools are completed (YES).
[0028] <Step S30> Check result output The text check unit 13 outputs warning text data, which is data on all detected defective descriptions. As shown in the following example, the warning text data includes a rule ID and a warning text written in natural language. The warning text data only needs to include at least a portion of the text containing the defect and the reason for the defect (rule ID). An example of warning text data is shown below.
[0029] (Warning message data 001) Rule ID: A1 A1: Duplication of particles Warning: "It may take a long time to restore service." The particle "ga" appears more than once in one sentence. It is preferable to split the sentence.
[0030] (Warning message data 006) Rule ID: B2 B2: Variations in spelling Warning: There are similar nouns "window" and "window." It is preferable to standardize the terminology.
[0031] The check performed by the text checker 13 is based only on the structure defined in the tool, and is a superficial check that does not take into account the context, the position of the text, etc. For this reason, the text checker 13 may make many false positives.
[0032] <Step S35> First judgment The first determination unit 15 determines whether the defect detection by the sentence check unit 13 is correct or not, using a learning model for determination, which is a machine learning model.
[0033] The first judgment unit 15 represents the natural language as a high-dimensional feature vector based on the extraction rules stored in the feature extraction rule storage unit 16, and performs processing.
[0034] 4, the first determination unit 15 is configured with a neural network (NN) that receives warning sentence data and sentence feature data as input. The first determination unit 15 is an AI (trained model) that receives document data stored in the document data storage unit 12 and warning sentence data as input, converts them into feature vectors stored in the feature extraction rule storage unit 16, and determines whether the defect detection result by the sentence check unit 13 is correct or not.
[0035] The first judgment unit 15, which is a learning model for judgment, is configured, for example, as follows.
[0036] (1) Vector conversion Similar to how general categorical variables are handled, the rule ID is converted into a multidimensional vector based on the extraction rules stored in the feature extraction rule storage unit 16. For example, the rule ID is converted into a vector whose dimension is the number of categories (here, the number of rules) by the one-hot encoding method. That is, in the one-hot encoding method, the rule ID is converted into a vector in which the value of the element corresponding to the rule ID is "1.0" and the other elements are "0.0". For example, a rule ID with 5 rules and a rule ID of "2" is converted into a "0.0, 1.0, 0.0, 0.0, 0.0" vector.
[0037] (2) Natural Language Processing Model (NN) The input natural language sentence is converted into a feature vector. For example, sentences such as "content of the criticism," "target sentence (sentence containing an error)," "chapter, section, title," and "surrounding sentences" are converted into feature vectors. One example is the use of BERT (Bidirectional Encoder Representations from Transformers), a natural language pre-training model.
[0038] The extraction rules are, for example, rules for extracting adjacent vectors, context vectors, and topic vectors of target words as feature vectors.
[0039] An adjacent vector is a word that appears immediately before or after a target word. Specifically, the elements of the vector are the appearance form and part of speech of the words before and after the target word. A context vector is a vector that characterizes the words that appear in the vicinity of the target word. A topic vector is a vector that characterizes the target word by topic.
[0040] (3) Recurrent NN The features of the surrounding text are obtained using a neural network (recurrent neural network: NN) that can process multiple input sequences. For example, LSTM (Long short-term memory) can be used.
[0041] (4) Joining This layer concatenates the output vectors of (1)-(3) above.
[0042] (5) Judgment For example, a decision is made by stacking one or more fully connected layers and combining them with an appropriate activation function.
[0043] As shown in FIG. 5, the first judgment unit 15 performs learning in advance by a judgment model learning unit 18 using judgment learning data, which is teacher data stored in a judgment learning data storage unit 17.
[0044] For example, the warning message data 001 mentioned above was determined to be defective because it contains the particle "ga" more than twice. However, the first particle "ga" is a particle used in the compound sentence "The time until recovery will be longer" that modifies "possibility", and is therefore not considered to be a problem.
[0045] The learning data for judgment storage unit 17 stores training data in which many pieces of warning message data are judged by human judgement and judged as "right" or "wrong."
[0046] Since the first determination unit 15 learns using teacher data similar to the warning message data 001, it determines that the detection of the warning message data 001 is an error.
[0047] The first judgment unit 15 judges the accuracy of the warning text data, instead of detecting defects from the entire input text as in conventional AI document check systems. The document check system 1 requires less time to create the teacher data required to build an appropriate machine learning model and to learn the model than conventional AI document check systems. Therefore, the document check system 1 not only places a small load on hardware, but also has good efficiency in building models.
[0048] <Step S40> Output of first judgment result Only the warning message that the first judgment section 15 judges to be correct is output to the judgment result output section 19. The judgment result output section 19 outputs the warning message data to the display 3 which is the interface 10.
[0049] <Step S45> Display of first judgment result The judgment result output unit 19 displays the warning message data on the display 3 which is the interface 10.
[0050] As already explained, the text check unit 13 may make many false positives. However, in the document check system 1, only warning message data that the first judgment unit 15 judges to be correct is displayed. The document check system 1 reduces the number of warning message data that the user must recheck, making proofreading more efficient.
[0051] <Step S50> Second Determination The user judges whether the warning displayed on the display 3 is correct, and inputs the judgment result to the second judgment unit 20 via an interface 10 such as a keyboard 4. That is, the second judgment unit 20 re-judges whether the warning is correct based on the user's intention. The first judgment result judged by the user as "correct" is re-judged as "correct" by the second judgment unit 20. The first judgment result judged by the user as "incorrect" is re-judged as "incorrect" by the second judgment unit 20.
[0052] The warning message presented to the user includes information about the cause of the detected defect.
[0053] For example, (Warning message data 006) is displayed on display 3 as follows: Rule ID: B2 B2: Variations in spelling Warning: There are similar nouns "window" and "window." It is preferable to standardize the terminology.
[0054] The user can easily reassess the document because the reasons for the deficiencies are displayed. Therefore, the document check system 1 makes proofreading more efficient.
[0055] <Step S60> Output of second judgment result As shown in FIG. 5, it is preferable that the judgment result of the second judgment section 20 is stored in a second judgment result storage section 21, and the judgment model learning section 18 learns the second judgment result as training data.
[0056] The second judgment result is learned as teacher data, thereby improving the reliability of the re-judgment result of the second judgment unit 20. The second judgment result storage unit 21 may be included in the judgment learning data storage unit 17.
[0057] When the text check tool management unit 22 and the text check tool storage unit 23 are managed collectively by an administrator other than the users, the second judgment results of a plurality of users are learned as teacher data, and therefore the reliability of the re-judgment result of the second judgment unit 20 is improved in a short period of time.
[0058] The second judgment result of each user may be used as training data and learned by the first judgment unit 15 dedicated to each user. The document check system 1 may also learn the second judgment result of a user selected from the multiple users as training data.
[0059] As described above, the document checking method of this embodiment uses a checking tool to detect specified deficiencies in input document data, outputs a warning message, determines whether the warning message is correct using a machine learning model, and re-evaluates the warning message that is determined to be correct based on the user's intention.
[0060] The document check program can also be distributed by storing it on a non-transitory computer-readable storage medium such as a magnetic disk (e.g., a floppy disk, a hard disk), an optical disk (e.g., a CD-ROM, a DVD), a magneto-optical disk (MO), a semiconductor memory, etc. The storage medium may be in any storage format as long as it is capable of storing the program and is computer-readable.
[0061] In addition, an OS (operating system), database management software, network software, or other MW (middleware) running on a computer based on instructions from a program installed on the computer from a storage medium may execute some of the processes for realizing the above embodiments.
[0062] The storage medium is not limited to a medium independent of a computer, but also includes a storage medium that stores or temporarily stores a program downloaded from a LAN, the Internet, etc. Also, the storage medium is not limited to one, and the storage medium of the present invention may include a case where each of the above processes is executed from multiple media, and the media configuration may be any configuration.
[0063] In addition, the computer in each embodiment executes each process in each of the above embodiments based on a program stored in a storage medium, and may be configured as either a single device such as a personal computer, or a system in which multiple devices are connected to a network.
[0064] Further, the term "computer" is not limited to a personal computer but also includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions of the present invention by a program.
[0065] Although several embodiments of the invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]
[0066] 1. Document Check System 2… Processor 3. Display 4… Keyboard 5… Storage device 6. Program 7… Server 10. Interface 11... Document data reading section 12... Document data storage section 13. Sentence Check Section 16... Feature extraction rule storage section 17… Learning data storage unit for judgment 18... Decision model learning section 19... Judgment result output section 22... Text Check Tool Management Department 23... Text Check Tool Storage
Claims
1. a text checking unit that uses a check tool to check for predetermined defects in the input document data and outputs a warning message; a first determination unit that determines whether the warning message is correct by using a machine learning model; a second judgment unit that re-judges the warning message that the first judgment unit judges to be correct, based on a user's intention.
2. The document check system according to claim 1 , wherein the warning message includes a reason for the deficiency.
3. 3. The document check system according to claim 2, wherein the text check section detects defects using two or more check tools selected from a plurality of check tools that detect different defects.
4. The document checking system according to claim 1 , wherein the machine learning model learns using a re-determination result of the second determination unit.
5. 2. The document check system according to claim 1, wherein the first determination unit receives the warning message and the document data as input and performs determination using a recurrent neural network.
6. a text checking unit that uses a checking tool to detect predetermined defects in the input document data and output a warning message; a first determination unit that determines whether the warning message is correct by using a machine learning model; A document check program for causing a processor to execute a document check system comprising: a second judgment unit that re-judges the warning text that the first judgment unit judges to be correct based on the user's intention.
Citation Information
Patent Citations
Document processing apparatus and program
JP2012155645A
Cited By
Document creation support device and document creation support method
JP2026182539A
Document analysis device, document analysis method, and document analysis program
JP7897686B1
