Information processing method

Abstract

To provide an information processing method for efficiently extracting a prescribed kind of page from data.SOLUTION: The information processing method includes extracting a page matching a rule for analyzing text included in a material including text and a chart and determining that the page is highly likely to be a predetermined type of page, converting the extracted page into an image, inputting the converted image into a classification model to classify the predetermined type of page, and storing the classified predetermined type of page.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method for classifying pages of a document into predetermined types. [Background technology]

[0002] There is a known device that makes it easy to create or search for Fault Tree Analysis (FTA) for any diagnostic target by registering all FTAs ​​expressed in a diagnostic tree format in a database and further creating a database of target ontologies that make up the FTA and diagnostic ontologies to which multiple target ontologies are associated (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-322125 Summary of the Invention [Problem to be solved by the invention]

[0004] It is not easy to extract only specific pages, such as FTAs, from a vast amount of material.

[0005] In view of the above circumstances, the purpose of the present disclosure is to efficiently extract predetermined types of pages from documents using a model such as a deep learning model. [Means for solving the problem]

[0006] An information processing method according to one embodiment of the present disclosure includes analyzing text contained in a document including text and diagrams, extracting pages that match rules that determine that the pages are likely to be of a predetermined type, converting the extracted pages into images, inputting the converted images into a classification model to classify the predetermined type of pages, and saving the classified predetermined type of pages. [Effects of the Invention]

[0007] According to an information processing method according to an embodiment of the present disclosure, predetermined types of pages are efficiently extracted from materials. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of the configuration of an information processing device according to an embodiment. [Figure 2] 1 is a flowchart illustrating an example of a procedure of an information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Configuration example of information processing device 10) As shown in FIG. 1, an information processing device 10 according to an embodiment includes a control unit 12, a storage unit 14, a communication unit 16, and a display unit 18.

[0010] The control unit 12 controls at least one component of the information processing device 10. The control unit 12 may be configured to include one or more general-purpose processors or dedicated circuits. The general-purpose processor may include a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The dedicated circuit may include an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0011] The memory unit 14 may be configured to include, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited to these. The memory unit 14 may function as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 14 may be configured to include an electromagnetic storage medium such as a magnetic disk. The memory unit 14 may be configured to include a non-transitory computer-readable medium. The memory unit 14 stores any information or program used in the operation of the information processing device 10. The memory unit 14 may store, for example, a system program or an application program. The memory unit 14 may be included in the control unit 12.

[0012] The communication unit 16 may include a communication module configured to be able to communicate with an external database or the like. The communication module may be compatible with a mobile communication standard such as 4G or 5G. The communication module may be compatible with a communication standard such as LAN (Local Area Network). The communication module may be compatible with a wired or wireless communication standard. The communication module is not limited to these and may be compatible with various communication standards.

[0013] The display unit 18 outputs visual information such as images, characters, or graphics. The display unit 18 may be configured to include an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, an inorganic EL display, or the like. The display unit 18 is not limited to these displays and may be configured to include various other types of displays. The display unit 18 may be configured to include a light-emitting device such as an LED (Light Emitting Diode). The display unit 18 may be configured to include various other devices.

[0014] The information processing device 10 may be configured to include an input device that accepts input of information, data, etc. from a user. The input device may be configured to include, for example, a touch panel or touch sensor, or a pointing device such as a mouse. The input device may be configured to include physical keys. The input device may be configured to include an audio input device such as a microphone. The information processing device 10 may include an audio output device such as a speaker that outputs auditory information such as voice, or may include various other output devices.

[0015] The information processing device 10 may be configured to include one or more server devices that can communicate with each other, and may be realized as a cloud server.

[0016] (Example of operation of information processing device 10) The control unit 12 of the information processing device 10 extracts a predetermined type of page, such as a Fault Tree Analysis (FTA), from a document containing text and a diagram. The text may be included in the diagram. The document containing text and a diagram may be non-image data, such as in a Portable Document Format (PDF) format.

[0017] The predetermined type of page is not limited to an FTA page, but may be a page including graphical elements such as a tree or a table. The control unit 12 may store the extracted page in the memory unit 14, or may store it in an external storage device or output it to an external device via the communication unit 16, or may display it on the display unit 18. The control unit 12 may store the extracted page as a predetermined type of page in the form of a database so that the user can search for it. If the predetermined type of page is an FTA page for a vehicle defect, the predetermined type of page may be stored so that the user can search for the page by the defect event or cause.

[0018] The control unit 12 may extract predetermined types of pages from the document using the classification model 20. The classification model 20 is an image classification model, and outputs a determination result as to whether an input image is an image of a predetermined type of page. The control unit 12 converts pages included in the document into images, inputs the images to the classification model 20, obtains the determination result output from the classification model 20, and extracts predetermined types of pages based on the determination result. The images obtained by converting the pages are also called converted images.

[0019] The classification model 20 may be a rule-based model. The classification model 20 may be a trained model generated by executing machine learning. The machine learning may be performed using training data in which converted images are associated with correct answer information indicating whether the converted images are images of a predetermined type of page. The machine learning may include deep learning. The machine learning may be performed by the control unit 12 or an external device.

[0020] A Vision Transformer (ViT) may be used as the classification model 20. A Swin Transformer may be used as the classification model 20. When a Swin Transformer is used as the classification model 20, the rate at which pages determined by the classification model 20 to be of a predetermined type include pages of a type other than the predetermined type, i.e., the garbage rate, is reduced.

[0021] The control unit 12 may input materials as input data 21 to the classification model 20. The classification model 20 may output output data 22 in response to the input of the input data 21. The control unit 12 may display the output data 22 on the display unit 18, or may output the output data 22 to an external device via the communication unit 16. The control unit 12 may extract pages of a predetermined type based on the output data 22 and store them in the memory unit 14 or an external storage device. The output data 22 may include pages that have been determined to be pages of a predetermined type from the input data 21 and extracted.

[0022] The output data 22 may include a determination result display 24 indicating whether the input data 21 is a predetermined type of page. The determination result display 24 may include either a circle indicating that the input data 21 is a predetermined type of page, or an cross indicating that the input data 21 is not a predetermined type of page. The determination result display 24 may include a numerical value indicating the probability that the determination of whether the input data 21 is a predetermined type of page is correct. The determination result display 24 of the output data 22 illustrated in FIG. 1 indicates the determination result of the classification model 20 that the input data 21 is a predetermined type of page with a 99% probability. The output data 22 may also include a heat map 23 indicating the basis for the determination. The heat map 23 indicates the distribution of parts that the classification model 20 values ​​and parts that it does not value among the pages to be determined.

[0023] The control unit 12 may extract pages that match the extraction rules from the pages included in the document, convert the extracted pages into images, and input the images to the classification model 20. The extraction rule may be that the pages contain search text. As preprocessing, the control unit 12 may set the search text, analyze the text included in the document, and extract pages that contain the search text as pages that are likely to be pages of a predetermined type. By inputting the converted images of pages that match the extraction rules to the classification model 20, the control unit 12 can improve the accuracy of determining pages of a predetermined type in the determination results output by the classification model 20.

[0024] The process of extracting pages that match the extraction rules is also referred to as preprocessing. The control unit 12 may realize the preprocessing using a trained model. The trained model that realizes the preprocessing is also referred to as a preprocessing model. The preprocessing model may output pages that match a predetermined rule when a page of a document is input, or may output a determination result as to whether the input page matches the predetermined rule. The preprocessing model may be included in the classification model 20. When included in the classification model 20, the preprocessing model may extract pages that match a predetermined rule from the pages of the document and convert them into images, and output the converted images to a part of the classification model 20 that determines a predetermined type of page from the converted images.

[0025] The search text may be a character string that is likely to be included in a predetermined type of page. The search text may include character strings that are expressed in different ways from character strings that are likely to be included in a predetermined type of page. For example, the search text may include a character string in uppercase or lowercase, a character string in a mixture of uppercase and lowercase, a character string containing spaces or punctuation marks, an abbreviation of a character string, or a character string that is an expansion of an abbreviation. Enriching the search text makes it less likely that a predetermined type of page will be overlooked in preprocessing.

[0026] To extract pages of a predetermined type, the control unit 12 may execute an information processing method including the steps of the flowchart illustrated in Fig. 2. The information processing method may be realized as an information processing program executed by a processor constituting the control unit 12. The information processing program may be stored in a non-transitory computer-readable medium.

[0027] The control unit 12 extracts pages that match the extraction rules from the document (step S1). The control unit 12 converts the extracted pages into images (step S2). The control unit 12 inputs the converted images into the classification model 20 and classifies the pages into predetermined types based on the classification results output from the classification model 20 (step S3). The control unit 12 stores the classified pages into the predetermined types in the memory unit 14 (step S4). The control unit 12 may store the classified pages into an external database via the communication unit 16. After performing the procedure of step S4, the control unit 12 ends the execution of the procedure of the flowchart in FIG. 2.

[0028] The control unit 12 may execute a series of steps from S1 to S3 by inputting the material into the classification model 20 including the preprocessing model.

[0029] (Example) As an example, a case will be described in which a trained model generated by deep learning is used as the classification model 20 to extract pages of a predetermined type from a document. Specifically, the classification model 20 was generated by performing deep learning using, as a training dataset, some pages of a sample in which pages of the predetermined type and pages other than the predetermined type are mixed at a predetermined ratio. Then, the remaining pages of the sample were input to the classification model 20 as an evaluation dataset, and the determination results output by the classification model 20 were evaluated in terms of the percentage of pages of the predetermined type that were correctly determined to be pages other than the predetermined type, i.e., the accuracy rate.

[0030] When 177 pages of data were input as an evaluation dataset to a classification model 20 generated using 708 pages of data as a training dataset, the classification model 20 correctly identified 50 pages as pages of a predetermined type and correctly identified 119 pages as pages other than the predetermined type. In other words, the classification model 20 was able to correctly determine whether a page was of a predetermined type with an accuracy rate of (50 + 119) / 177 = 95.5%. The percentage of pages other than the predetermined type that the classification model 20 identified as the predetermined type, i.e., the garbage rate, was 3 / (50 + 3) = 5.7%. The percentage of pages of a predetermined type that the classification model 20 identified as pages other than the predetermined type, i.e., the oversight rate, was 5 / (5 + 50) = 9.0%.

[0031] (summary) As described above, the information processing device 10 can extract predetermined types of pages from materials by converting materials containing text and diagrams into images and inputting the images into the classification model 20, which is an image classification model. When converting materials into images and using an image classification model, predetermined types of pages can be extracted more efficiently from materials containing text and diagrams than when using a model that analyzes text.

[0032] Furthermore, the information processing device 10 can reduce the waste rate by preprocessing by extracting pages that are likely to be pages of a predetermined type, converting them into images, and inputting them into the classification model 20. The purpose of extracting FTA pages as pages of a predetermined type from documents is to allow users who create FTAs ​​to refer to past FTA pages and proceed with their work efficiently. If pages other than FTAs ​​are included among the pages extracted as FTA pages, the greater the number of pages other than FTAs, the lower the user's work efficiency. Reducing the waste rate improves user work efficiency.

[0033] The information processing device 10 is not limited to FTA pages as the predetermined type of pages, but can efficiently extract other types of pages including characteristic charts and figures from the materials.

[0034] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means or step can be rearranged so as not to be logically inconsistent, and multiple means or steps can be combined or divided into one. [Explanation of symbols]

[0035] 10 information processing device (12: control unit, 14: storage unit, 16: communication unit, 18: display unit) 20 Classification Models 21 Input Data 22 Output data (23: Heat map, 24: Judgment result display)

Claims

1. Analyzing text included in a document including text and diagrams and extracting pages that match rules that determine that the pages are likely to be of a predetermined type; Converting the extracted pages into images; and inputting the converted image into a classification model generated by performing machine learning using training data in which the converted page is associated with correct answer information indicating whether the page is a page of the predetermined type, and classifying the page of the predetermined type; storing the classified pages of the predetermined type; An information processing method, including:

2. Analyzing text included in a document including text and diagrams and extracting pages that match rules that determine that the pages are likely to be of a predetermined type; Converting the extracted pages into images; and inputting the transformed images into a classification model to classify the predetermined type of pages; storing the classified pages of the predetermined type; An information processing method, including:

3. The information processing method according to claim 2, wherein the classification model is a trained model generated by performing learning using training data in which pages converted into images are associated with correct answer information indicating whether the pages are of the specified type.

4. 3. The information processing method according to claim 2, wherein in extracting the pages that match the rule, a character string that is likely to be included in the predetermined type of page is set as a search text, and pages that include the search text are extracted.

5. The classification model includes a preprocessing model that extracts pages that match rules from the document; 3. The information processing method according to claim 2, wherein a series of processes are executed by inputting materials including text and diagrams into the classification model, extracting pages that match the rules, converting the extracted pages into images, and classifying the predetermined types of pages.

6. 3. The information processing method according to claim 2, wherein the predetermined type of page is an FTA page.

Images