Structured data processing methods

Abstract

Processing structured data to improve the accuracy of generative models. [Solution] The structured data processing method is characterized by including an acquisition step of acquiring a document, a conversion step of converting the acquired document into structured data, a formatting step of shaping the converted structured data into data that can be read by a generative model, and an output step of outputting the formatted structured data. For example, if the structured data includes image data, the conversion step incorporates text data converted by OCR processing into the structured data.

Description

Technical Field

[0001] The present disclosure relates to a method for processing structured data for use by a generative model.

Background Art

[0002] In recent years, in the field of data processing, digitization of work by using a generative model for natural language processing has been progressing. A generative model for natural language processing is used to analyze the text in a sentence and extract desired information.

[0003] For example, based on the analyzed data, the generative model generates an answer to a question from a user. The generating step searches for relevant documents from the data, determines which information is reported from the retrieved relevant documents, and responds based on the determination and the graph schema associated with the relevant documents (Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By using the prior art, the generative model can generate an answer to a question from a user based on the data. However, although the generative model generates output data for input data, depending on the type of input data and the program code, it may not be able to accurately read the input data. In this case, since the generative model cannot accurately analyze the intention of the input question, the answer accuracy may decrease.

[0006] Therefore, the present disclosure proposes a method for processing structured data that improves the answer accuracy of a generative model. [Means for solving the problem]

[0007] To solve the above problems, one form of structured data processing device according to this disclosure is characterized by including an acquisition step of acquiring a document, a conversion step of converting the acquired document into structured data, a formatting step of shaping the converted structured data into data that can be read by a generative model, and an output step of outputting the formatted post-formatted structured data. [Effects of the Invention]

[0008] According to one embodiment, structured data can be processed to improve the response accuracy of the generative model. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an overview of the structured data processing system according to the embodiment. [Figure 2] This figure shows an overview of structured data processing according to the embodiment. [Figure 3] This figure shows an example configuration of a structured data processing device according to the embodiment. [Figure 4] This figure shows an example of a structured data information storage unit according to the embodiment. [Figure 5] This is a flowchart (1) showing the procedure for structured data processing according to the embodiment. [Figure 6] This is a flowchart (2) showing the procedure for structured data processing according to the embodiment. [Figure 7] This is a flowchart (3) showing the procedure for structured data processing according to the embodiment. [Figure 8] This figure illustrates an example of data acquired by the acquisition unit according to the embodiment. [Figure 9] This figure illustrates an example of a user interface for structured data processing according to the embodiment. [Figure 10]This is a hardware configuration diagram showing an example of a computer that implements the functions of a structured data processing device. [Modes for carrying out the invention]

[0010] Embodiments of the present disclosure will be described in detail below with reference to the drawings. In the following embodiments, the same parts will be denoted by the same reference numerals to avoid redundant descriptions.

[0011] (1. Embodiments) (1-1. Overview of the structured data system according to the embodiment) The structured data processing described herein is performed by the structured data processing system 1 shown in Figure 1. Figure 1 will be used to illustrate the general configuration of the structured data processing system 1 according to this embodiment. Figure 1 is a diagram illustrating the general configuration of the structured data processing system 1 according to this embodiment.

[0012] As shown in Figure 1, the structured data processing system 1 includes a structured data processing device 100, a customer terminal 10, and a generation model 20. Note that the number of structured data processing devices 100, customer terminals 10, and generation models 20 is not limited to those shown in Figure 1.

[0013] The structured data processing device 100 is an information processing device that converts documents sent from the customer terminal 10 into structured data. The structured data processing device 100 can be, for example, a server device, a PC (Personal Computer), or a cloud server.

[0014] Customer terminal 10 is a terminal device used by a customer. A customer can be an individual or a legal entity. In the following explanation, the customer will be assumed to be a legal entity. Customer terminal 10 is an information processing device such as a desktop PC, tablet PC, notebook PC, mobile phone, smartphone, or PDA (Personal Digital Assistant).

[0015] The generation model 20 is a model that generates an answer sentence in natural language for an instruction sentence set in natural language. The generation model 20 generates an answer to the received question based on, for example, the input of text data indicating the question.

[0016] As described above, the generation model 20 is a model that generates output data for input data. However, depending on the type of input data and the program code, the generation model 20 may not be able to read it. For example, when the document contains figures and tables, the generation model 20 may not be able to accurately read the described content. As a result, it becomes difficult for the generation model 20 to read the entire document. Therefore, there was a problem of converting the document into data that the generation model 20 can read in the reading of the generation model 20.

[0017] Therefore, the structured data processing device 100 according to the embodiment solves the above problems with the following configuration. The structured data processing device 100 converts the acquired document data into structured data. Subsequently, the structured data processing device 100 deletes or corrects elements that reduce the reading accuracy in the structured data, and shapes the structured data into a form that the generation model 20 can read. Thereby, the structured data processing device 100 can improve the reading accuracy of the generation model 20.

[0018] (1-2. Outline of Structured Data Processing) Regarding the above processing, the outline of the structured data processing by the structured data processing system 1 will be described using FIG. 2. FIG. 2 is a diagram showing the outline of the structured data processing according to the embodiment.

[0019] First, the structured data processing device 100 retrieves a document from the customer terminal 10 (step S1). A document is a document consisting of text, tables, images, etc., such as an internal company manual. The structured data processing device 100 converts the data of the retrieved document into structured data (step S2). Structured data is data that is written and arranged according to a structure in which the order and other elements are clearly defined. For example, structured data is text written in a markup language such as XML data (Extensible Markup Language) or HTML data (HyperText Markup Language).

[0020] The structured data processing device 100 analyzes the data converted to HTML, etc., when converting it to structured data, and reflects the results of the analysis in the structured data. This process is called data structure-related processing (step S3). For example, the structured data processing device 100 determines whether or not there are images in the structured data.

[0021] If the structured data contains images, the structured data processing device 100 performs OCR processing on the structured data (step S4). This processing is performed because the generation model 20 may not be able to read the figures, so the figures are represented as text data and made readable.

[0022] The structured data processing device 100 compares the structured data that has undergone OCR processing, that is, the structured data in which images and tables have been converted into text data, with the original document and determines whether there are any discrepancies in content. If there are discrepancies in content, the structured data processing device 100 uses the document as the correct data and corrects the content of the structured data converted into text data (step S5), and then combines the corrected parts into the structured data (step S6).

[0023] Next, the structured data processing device 100 performs a formatting process on the structured data to format it into post-formatted structured data (step S7). Formatting refers to the process of preparing the structured data in a format that the generation model 20 can read. Post-formatted structured data refers to structured data after all formatting processes have been completed. In other words, the structured data processing device 100 formats the data converted to HTML, etc., in order to improve the reading accuracy of the generation model 20, and reflects the formatting results in the structured data. For example, if the structured data includes a flowchart, the structured data processing device 100 changes the arrows that show the relationships between the components of the flowchart to a different representation and formats the structured data. Note that if the structured data processing device 100 determines that OCR processing should not be performed on the structured data, it performs the formatting process in step S7.

[0024] Then, once the molding process is complete, the structured data processing device 100 performs a full calibration of the post-molding structured data (step S8). For example, the structured data processing device 100 determines whether or not there are any errors in the post-molding structured data and performs calibration.

[0025] Once all molding processes and calibrations are complete, the structured data processing device 100 processes the post-molded structured data to input it into the generated model 20, creating post-processed structured data (step S9).

[0026] The generative model 20 accepts input of the processed structured data (step S10). The generative model 20 also accepts questions from users of the documents, such as internal company documents, that served as the basis for the processed structured data (step S11). Based on the content and intent of the processed structured data, the generative model 20 generates answers to the received questions (step S12).

[0027] In this way, the structured data processing device 100 acquires documents from the customer terminal 10, converts the documents into structured data, and shapes the converted structured data. As a result, the structured data processing device 100 creates shaped structured data that can improve the response accuracy of the generative model 20.

[0028] (1-3. Configuration of the structured data processing device according to the embodiment) Next, the configuration of the structured data processing device 100 that performs structured data processing according to the embodiment will be described. Figure 3 is a diagram showing an example of the configuration of the structured data processing device 100 according to the embodiment.

[0029] As shown in Figure 3, the structured data processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The communication unit 110 is implemented, for example, by a NIC (Network Interface Card) or a network interface controller. The communication unit 110 is connected to a network (e.g., the Internet) by wire or wireless connection and transmits and receives information with the customer terminal 10, the generation model 20, etc. via the Internet. For example, the communication unit 110 may transmit and receive information using any communication standard or technology such as Wi-Fi (registered trademark), Bluetooth (registered trademark), SIM (Subscriber Identity Module), or LPWA (Low Power Wide Area).

[0030] The memory unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs.

[0031] The memory unit 120 includes a structured data information storage unit 121. The information stored in the memory unit 120 will be explained below using Figure 4. Note that in the example shown in Figure 4, the information stored in the memory unit 120 may be conceptually represented as "A01," but in reality, each of the pieces of information described later will be stored in the memory unit 120.

[0032] The structured data information storage unit 121 stores information used in the structured data processing according to the embodiment, such as documents transmitted from the customer terminal 10 and the types of structured data.

[0033] Figure 4 shows an example of a structured data information storage unit 121 according to the embodiment. As shown in Figure 4, the structured data information storage unit 121 stores the following items: "Customer ID", "Document ID", "Document Type", and "Type of Structured Data".

[0034] "Customer ID" is an identification number assigned to each customer. "Document ID" is an identification number assigned to each document. "Document Type" is the type of document sent from the customer terminal 10. "Structured Data Type" is the type of structured data converted by the structured data processing device 100.

[0035] As illustrated in the example in Figure 4, the structured data processing device 100 stores that the customer ID is "A01", the document ID is "X01", the type of document sent from the customer terminal 10 is "PDF", and the type of structured data is "HTML". For example, the document may also contain image data. Furthermore, the document is not limited to PDF.

[0036] Returning to Figure 3, let's continue the explanation of the control unit 130. The control unit 130 is implemented by, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), etc., executing a program stored inside the structured data processing device 100 using RAM or the like as a working area. The control unit 130 is also a controller and is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0037] As shown in Figure 3, the control unit 130 includes an acquisition unit 131, a conversion unit 132, a molding unit 133, and an output unit 134.

[0038] The acquisition unit 131 acquires documents sent from the customer terminal 10. For example, the acquisition unit 131 acquires documents such as document data, image data, and table data. Specifically, the acquisition unit 131 acquires internal manuals, reports, etc.

[0039] The conversion unit 132 converts the document acquired by the acquisition unit 131 into structured data. The conversion unit 132 converts the acquired document into PDF data. The structured data processing device 100 determines whether the acquired document is PDF data or not. If the document is not PDF data, the conversion unit 132 performs RIP (Raster Image Processor) processing and converts it into PDF data. RIP processing is a process that converts vector data into raster data, which is an image composed of a collection of dots, and then converts it into halftone data that can be printed by a printer.

[0040] The conversion unit 132 converts PDF data into XML data. For example, the conversion unit 132 converts PDF data into XML data using existing programs or applications. The conversion unit 132 also converts XML data into HTML data. Hereafter, the converted HTML data will be referred to as structured data. The reason for converting documents into XML data or HTML is to facilitate data editing and management by converting them to XML data, etc.

[0041] Furthermore, if the structured data contains image data, the conversion unit 132 performs a conversion that incorporates the text data converted by OCR processing into the structured data. For example, if the structured data contains photographs, figures, tables, etc., the conversion unit 132 converts the photographs, figures, tables, etc. into text data using OCR processing and incorporates the converted text data into the original structured data. Photographs, figures, tables, etc. specifically include JPEG files, image files, data tables, images composed of a collection of dots, etc. Also, for example, if the document does not contain image data, the conversion unit 132 does not perform OCR processing. Specifically, the conversion unit 132 does not perform OCR processing on original data such as captured images that are described by text data.

[0042] The formatting unit 133 formats the converted structured data into data that the generative model 20 can read. The formatting unit 133 formats the HTML data converted by the conversion unit 132, and formats the post-formatted structured data. "Data that the generative model 20 can read" means data that the generative model 20 does not misinterpret the intent of the text and figures written in the original document. Also, "data that the generative model 20 can read" means data that, when processed by the generative model 20 as input data, does not generate unintended responses.

[0043] The molding unit 133 analyzes the relationships between constituent elements within the structured data and formats the data to represent those relationships. For example, the molding unit 133 analyzes the relationships between sentences within the structured data and formats the data to represent those relationships. Also, for example, the molding unit 133 analyzes the relationships between images and between images and between images and between tables within the structured data and formats the data to represent those relationships. Specifically, if the structured data includes a flowchart, the molding unit 133 adds new tags to arrows and callouts that represent relationships between sentences, images, tables, etc., within the flowchart, and formats the data to represent those relationships.

[0044] Furthermore, if the structured data includes a table, the molding unit 133 unlinks the cells within the structured data to form the post-molded structured data. For example, if the cells before unlinking contain text or sentences, the molding unit 133 unlinks the cells and then enters the same text or sentences into each of the unlinked cells to form the post-molded structured data.

[0045] Furthermore, the molding unit 133 replaces demonstrative pronouns included in the structured data with other words. Specifically, the molding unit 133 replaces demonstrative pronouns such as "same as above" and "see reference" with other words so that the generation model 20 can accurately understand the meaning of the sentence.

[0046] Furthermore, the molding unit 133 compares the modified parts of the structured data with the document to determine whether or not there are any modifications, and modifies the post-molded structured data based on the determination result before molding. For example, the molding unit 133 compares hierarchically structured text, figures, etc., with the document as modified parts of the structured data, modifies the post-molded structured data, and then molds it.

[0047] The output unit 134 outputs the molded post-molded structured data. Post-molded structured data is data after all molding processing has been completed on the structured data in the molding unit 133. For example, the output unit 134 outputs the molded post-molded structured data to the customer terminal 10. Alternatively, for example, the output unit 134 outputs the molded post-molded structured data to a server device that performs processing to create input data to be input into the generation model 20.

[0048] (1-4. Procedures for processing structured data) Here, the procedure for structured data processing will be explained using Figures 5 to 7. Figure 5 is a flowchart (1) showing the procedure for structured data processing according to the embodiment. As shown in Figure 5, the structured data processing device 100 determines whether or not it has obtained a document from the customer terminal 10 (step S101). If the document has not been obtained (step S101; No), the structured data processing device 100 waits until the document is obtained. On the other hand, if the document has been obtained (step S101; Yes), the structured data processing device 100 determines whether or not the document is PDF data (step S102).

[0049] If the document is not in PDF format (Step S102; No), the structured data processing device 100 converts the document to PDF format as a preprocessing step (Step S103). If the document is already in PDF format (Step S102; Yes), the structured data processing device 100 may skip Step S103. After that, the structured data processing device 100 converts the document to XML format (Step S104). Furthermore, the structured data processing device 100 converts the XML format to HTML format (Step S105).

[0050] Next, the structured data processing device 100 determines whether or not the converted HTML data contains image data (step S106). If image data is included (step S106; Yes), the structured data processing device 100 performs OCR processing on the image data (step S107).

[0051] The structured data processing device 100 inspects the content of the structured data that has undergone OCR processing (step S108). Specifically, the structured data processing device 100 compares the structured data that has undergone OCR processing with the document.

[0052] Based on the comparison, the structured data processing device 100 determines whether there is a difference between the OCR-processed structured data and the document content (step S109). If there is a difference in content (step S109; Yes), the structured data processing device 100 corrects the content of the structured data using the document as the correct data and combines them (step S110).

[0053] Furthermore, if the structured data does not contain image data (step S106; No), or if there is no discrepancy between the structured data and the document (step S109; No), the structured data processing device 100 can skip the OCR processing in step S107 or the correction processing in step S110.

[0054] In the HTML data obtained in this way, the structured data processing device 100 removes unnecessary image tags (step S111). Then, the structured data processing device 100 performs preprocessing (step S112) and proceeds to step S201. Preprocessing includes, for example, removing unnecessary tags and line breaks contained in the structured data. Preprocessing also includes, for example, replacing XML data tags contained in the structured data with HTML data tags.

[0055] Next, the flow of the formatting process will be explained using Figure 6. Figure 6 is a flowchart (2) showing the procedure for structured data processing according to the embodiment. The structured data processing device 100 determines whether or not there are figures that indicate the relationships between elements in the structured data (step S201). If the structured data processing device 100 finds that there are figures that indicate the relationships between elements in the structured data (step S201; Yes), it modifies the data using HTML tags (step S202) and proceeds to the next stage of processing. Figures that indicate the relationships between elements refer to displays or symbols used to express the relationships between each element with respect to multiple elements contained in the data. For example, a figure that indicates the relationships between elements is a figure composed of elements that indicate the processing content and arrows that indicate the procedure for performing that element, as in a flowchart. Furthermore, figures that indicate the relationships between elements are not limited to flowcharts, but also include situations where sentences are connected by arrows or where callouts are attached to sentences. Specifically, the structured data processing device 100, when there is a predetermined relationship between elements in the structured data, represents that relationship in the structured data using text data (for example, a symbol showing a double arrow). For example, suppose the elements are a superior and a subordinate. In this case, a predetermined relationship between the elements could be that the superior provides education and guidance to the subordinate, and the subordinate reports results and consults with the superior. In this case, assuming that there is a predetermined relationship between the elements, the superior and subordinate elements are represented by a double arrow. Note that the relationship is not limited to this case. If there is no graphic to indicate the relationship between the elements (step S201; No), the structured data processing device 100 skips step S202 and proceeds to the next stage of processing.

[0056] Next, the structured data processing device 100 determines whether or not there are merged cells in the structured data (step S203). If there are merged cells in the structured data (step S203; Yes), the structured data processing device 100 unmerges the cells (step S204). If there was text or a sentence in the merged cells before unmerging, the structured data processing device 100 enters the same text or sentence into the cells after unmerging.

[0057] On the other hand, if there are no merged cells in the structured data (step S203; No), the structured data processing device 100 skips step S204. Once the processing related to cell merging is complete, the structured data processing device 100 corrects the table in the structured data (step S205). For example, the structured data processing device 100 determines whether the tags used to represent the table included in the structured data are correct, and corrects the tags if they are incorrect. When determining the success or failure of the table tags, the structured data processing device 100 checks for errors on a block-by-block basis of the table, for example.

[0058] Next, the structured data processing device 100 determines whether or not the structured data contains a demonstrative pronoun (step S206). If the structured data contains a demonstrative pronoun (step S206; Yes), the structured data processing device 100 converts the demonstrative pronoun into another word (step S207). This is because if the structured data contains a demonstrative pronoun, the generative model 20 may misinterpret the intent of the demonstrative pronoun. For example, the structured data processing device 100 modifies the demonstrative pronoun into another word or deletes the demonstrative pronoun so that the intent it indicates becomes clear. On the other hand, if the structured data does not contain a demonstrative pronoun (step S206; No), the structured data processing device 100 skips step S207 and proceeds to the next stage. Through the formatting process up to step S207, the structured data processing device 100 obtains formatted data (formatted structured data) from the original HTML data.

[0059] Next, the structured data processing device 100 determines whether the content of the original structured data matches the content of the post-molded structured data obtained through the molding process up to step S207 (step S208). If the content of the structured data and the post-molded structured data do not match (step S208; No), the structured data processing device 100 corrects the parts that do not match (step S209). On the other hand, if the content of the structured data and the post-molded structured data match (step S208; Yes), or if the content mismatch was corrected in step S209, the structured data processing device 100 checks the entire process up to that point (step S210) and proceeds to step S301.

[0060] Next, the processing flow after obtaining the post-formed structured data will be explained using Figure 7. Figure 7 is a flowchart (3) showing the procedure for processing structured data according to the embodiment. The structured data processing device 100 compares the document with the post-formed structured data and determines whether or not there are any modifications to the post-formed structured data when proofreading is performed (step S301). If there are modifications (step S301; Yes), the structured data processing device 100 modifies the characters, tags, images, tables, figures, and hierarchical structure of the post-formed structured data (step S302). Note that if there are significant modifications, for example, the structured data processing device 100 returns the process to step S201. After that, the structured data processing device 100 checks the difference between the post-formed structured data and the proofread structured data (step S303). For example, the structured data processing device 100 uses existing programs or applications to check whether any unnecessary changes have occurred between the pre-proofreading and post-proofreading versions in areas other than those to be modified. Unwanted changes refer to alterations to text, tags, images, tables, figures, and hierarchical structures within the post-proofread, formatted structured data, where corrections have already been made or remain unchanged.

[0061] On the other hand, if there are no corrections needed (Step S301; No), the structured data processing device 100 prepares the data for output to the customer terminal 10 (Step S304). Then, once the output preparation is complete, the structured data processing device 100 outputs the HTML data to the customer terminal 10 (Step S305).

[0062] (1-5. Specific Examples) Here, we will explain a specific example of structured data processing using Figures 8 and 9. Figure 8 is a diagram illustrating an example of data acquired by the acquisition unit 131 according to the embodiment. As shown in Figure 8, the acquisition unit 131 acquires document 300.

[0063] Document 300 is, for example, an internal company document that outlines the criteria and procedures for determining the importance of a business operation. As shown in Figure 8, Document 300 has components such as a first-level structure 301, an arrow 302, and a second-level structure 303. Typically, someone viewing Document 300 will grasp the content of the text described in the first-level structure 301, and then, following the direction indicated by the arrow 302, will read on to the procedures shown in the second-level structure 303. In other words, Document 300 is a document in which the relationships between elements are indicated by the arrow 302, showing the relationship between the first-level structure 301 and the second-level structure 303.

[0064] As shown in Figure 8, the first hierarchical structure 301 contains a title 301a and a description 301b. Arrow 302 indicates a transition from the first hierarchical structure 301 to the second hierarchical structure 303. The second hierarchical structure 303 includes a title 303a, a description 303b, a description 303c, arrows 304, 305, and 307. If the judgment result in the second hierarchical structure 303 is No, it transitions from description 303b to description 303c via arrow 304. Also, if the judgment result in description 303b is Yes, the second hierarchical structure 303 transitions to the most important case 306 via arrow 305. Furthermore, if the judgment result in description 303c is Yes, the second hierarchical structure 303 transitions to important case 308 via arrow 307.

[0065] As described above, when the structured data processing device 100 obtains a document 300 from the customer terminal 10, it performs processing to convert the document 300 into formatted structured data. In this conversion process, the structured data processing device 100 provides a user interface for the user who manages the conversion process. The user interface will be explained using Figure 9.

[0066] Figure 9 illustrates an example of a user interface for structured data processing according to an embodiment. As shown in Figure 9, screen 400 has a work screen 400a, a document screen 400b, a button 400c, and an editing tool window 400d. The work screen 400a is a screen for performing formatting processing. For example, the work screen 400a is a screen for modifying HTML data tags. The document screen 400b is a preview screen that allows users to see how the program written on the work screen 400a will be displayed.

[0067] Button 400c is a button that the user presses to instruct conversion or formatting. Button 400c includes an HTML load button 401, a save button 402, a preview button 403, a list hierarchy display button 404, an unnecessary tag delete button 405, a conversion button within the hierarchy 406, a closing tag conversion button 407, a structure display button 408, an editing tool button 409, a search and replace button 410, a preview hide button 411, and a guideline button 412. The editing tool window 400d is an additional window for the editing tools used when correcting or changing tags one tag at a time in the process of step S210 in Figure 6. The editing tool window 400d lists buttons for the tags that can be used in structured data processing, and pressing each button adds the tag of the pressed button to the selected area.

[0068] The HTML Load button 401 is a button for opening the data to be displayed on the work screen 400a. The Save button 402 is a button for saving the work. The Preview button 403 is a button for displaying the document screen 400b. The List Hierarchy Display button 404 is a button for color-coding the hierarchy displayed on the work screen 400a. The Delete Unnecessary Tag button 405 is a button for deleting unnecessary tags included in the structured data. The Conversion button 406 in the hierarchy is a button for the paragraph tags of the data converted in step S104 of Figure 5. If the " is incorrect, the selected part will be treated as a bulleted list tag The button converts to "". The closing tag conversion button 407 is a button that, when there is no closing tag or incorrect information, adds the same closing tag as the selected opening tag in the structured data block. The structure display button 408 is a button that adds hierarchical indentation to the text displayed on the work screen 400a. The editing tool button 409 is a button to open the editing tool window 400d. The search and replace button 410 is a button that searches for words or sentences contained in the structured data or document screen 400b and replaces specific words with other words. The preview hide button 411 is a button that hides the document screen 400b. The guideline button 412, when pressed, displays HTML indentation-specific guidelines on the work screen 400a, improving the visual visibility of hierarchical structural relationships based on indentation.

[0069] The structured data processing device 100 performs structured data processing by selecting and pressing button 400c on screen 400. Furthermore, the structured data processing device 100 can improve work efficiency by displaying the work screen 400a, the document screen 400b, button 400c, and, if applicable, the editing tool window 400d on the same screen.

[0070] The processes described above may be performed automatically or manually. For example, in the conversion process described above, if it is to automatically determine whether or not an image in the HTML data contains characters that should be recognized (characters requiring OCR processing), the structured data processing device 100 uses an image recognition model to determine the presence or absence of characters. Subsequently, if the structured data processing device 100 determines that the image contains characters, it can extract the text data to be incorporated into the HTML data by performing OCR processing. As a result, the structured data processing device 100 can automatically incorporate characters in the image without a human having to visually recognize the presence of characters in the image and decide whether to perform OCR processing.

[0071] Furthermore, for example, in a molding process, if it is necessary to automatically determine whether the structured data contains figures that indicate the relationships between elements, the structured data processing device 100 uses an image recognition model to extract information indicating the direction of arrows and the connection relationships between elements. Subsequently, the structured data processing device 100 replaces the recognized figures with the tag information described above. This allows the structured data processing device 100 to automatically add information indicating relationships within the structured data. For example, in a molding process, if it is necessary to manually determine whether the structured data contains figures that indicate the relationships between elements, a person can visually determine the relationships between elements.

[0072] On the other hand, for example, in a molding process, if it is necessary to manually determine whether or not a demonstrative pronoun is included in the structured data, a person visually inspects the structured data to determine whether or not the demonstrative pronoun is included. In this case, the structured data processing device 100 can convert the demonstrative pronoun into a different expression based on the information entered by the person.

[0073] Furthermore, for example, in a molding process, if it is necessary to manually determine whether the tables and figures included in the structured data have a hierarchical structure, a person visually inspects the structured data to determine if the tables and figures have a hierarchical structure. In this case, the structured data processing device 100 can compare the hierarchical structure with the original document based on the information determined by the person and make corrections.

[0074] Furthermore, for example, in the proofreading process, if the structured data after formatting is compared with the original document manually, a person can perform the comparison visually. In this case, the structured data processing device 100 can correct the parts that need correction based on the results of the comparison performed by the person.

[0075] (2. Other Embodiments) The processing according to the above-described embodiment may be carried out in various other forms besides those described above.

[0076] For example, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0077] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0078] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0079] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0080] (3. Effects of the structured data processing device related to this disclosure) As described above, the structured data processing device (structured data processing device 100 in the embodiment) executes a structured data processing method that includes an acquisition step, a conversion step, a formatting step, and an output step. The acquisition unit (acquisition unit 131 in the embodiment) acquires a document as the acquisition step. The conversion unit (conversion unit 132 in the embodiment) converts the acquired document into structured data. The formatting unit (formatting unit 133 in the embodiment) formats the converted structured data into data that can be read by a generative model. The output unit (output unit 134 in the embodiment) outputs the formatted structured data.

[0081] Thus, a structured data processing device can process structured data that improves the accuracy of a generative model's responses by converting documents into structured data and then shaping the structured data to improve the generative model's ability to read it.

[0082] Furthermore, the structured data processing method includes a process of incorporating text data converted by OCR processing into the structured data if the structured data contains image data. This allows the structured data processing device to perform processing regardless of the file type by converting the image data contained in the structured data into text data.

[0083] Furthermore, the structured data processing method analyzes the relationships between constituent elements within the structured data and shapes those relationships into a representational form. This allows the structured data processing device to convert diagrams representing relationships between elements, such as flowcharts, into other representations, enabling the generative model to understand relationships between elements that previously could only be correctly grasped by humans. Additionally, for example, the structured data processing device can represent relationships between sentences using HTML tags, thereby enabling the generative model to understand those relationships.

[0084] Furthermore, if the structured data includes tables, the structured data processing method unlinks the cells within the structured data and forms the post-structured data. This allows the structured data processing device to reshape the structured data by unlinking cells, creating data that can be read by the generative model.

[0085] Furthermore, the structured data processing method replaces demonstrative pronouns in the structured data with other words. This allows the structured data processing device to reshape the structured data by replacing demonstrative pronouns with other words, preventing the generative model from outputting incorrect answers.

[0086] Furthermore, the structured data processing method compares the modified parts of the structured data with the document, determines whether or not there are any modifications, and modifies and reshapes the post-formed structured data based on the determination result. As a result, the structured data processing device uses the document as the correct data for the modified parts of the structured data, determines whether or not there are any modifications, and modifies them, thereby shaping the structured data and enabling error-free data to be used as input data for the generation model.

[0087] (4. Hardware Configuration) The structured data processing device 100 and other information devices according to the embodiments described above are realized by a computer 1000 having a configuration such as that shown in Figure 10. The structured data processing device 100 according to the embodiments will be described below as an example. Figure 10 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the structured data processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050.

[0088] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, and controls various parts. For example, the CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0089] ROM1300 stores boot programs such as the BIOS (Basic Input Output System) executed by CPU1100 when computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0090] HDD1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU1100 and data used by such programs. Specifically, HDD1400 is a recording medium that records a program that performs the calculation processing related to this disclosure, which is an example of program data 1450.

[0091] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (e.g., the Internet). For example, the CPU 1100 can receive data from other devices or transmit data it has generated to other devices via the communication interface 1500.

[0092] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, speaker, or printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0093] For example, when the computer 1000 functions as a structured data processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes functions such as the control unit 130 by executing a management program loaded on the RAM 1200. The HDD 1400 stores the program that executes the management processing according to this disclosure and the data in the storage unit 120. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0094] Although embodiments of the present application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention. [Explanation of symbols]

[0095] 1. Structured Data Processing System 10 Customer terminals 100 Structured Data Processing Devices 110 Communications Department 120 Storage section 121 Structured Data Information Storage Unit 130 Control Unit 131 Acquisition Department 132 Conversion section 133 Molding section 134 Output section

Claims

1. A method for processing structured data performed by a computer, The process of obtaining the document, A conversion process that converts the acquired documents into structured data, The above-mentioned transformed structured data is formatted into data that can be read by the generative model, The output process outputs the molded post-molding structured data, A structured data processing method characterized by including [a certain element].

2. The aforementioned conversion process is, If the structured data includes image data, the text data converted by OCR processing is incorporated into the structured data. The structured data processing method according to feature 1.

3. The molding process described above is: The relationships between the constituent elements within the aforementioned structured data are analyzed, and the relationships between those constituent elements are formatted to represent them. The structured data processing method according to feature 1.

4. The molding process described above is: If the structured data includes a table, the cells within the structured data are unlinked to form the post-structured data. The structured data processing method according to feature 1.

5. The molding process described above is: Replace the demonstrative pronouns in the aforementioned structured data with other words. The structured data processing method according to feature 1.

6. The molding process described above is: The modified parts of the structured data are compared with the document, it is determined whether or not there are any modifications, and the post-formed structured data is modified and formed based on the determination result. The structured data processing method according to feature 1.

Images