Information processing system, information processing device, database construction method, and program

Abstract

One aspect of the present disclosure relates to an information processing system comprising: a first information processing device managed by a user using a generative model; a second information processing device managed by a generative model operator; and a third information processing device managed by a structuring business operator who structures document data. The first information processing device transmits the document data to the third information processing device. Structured data generated from the document data is stored in a database. The first information processing device transmits a question to the second information processing device and receives an answer generated by using the generative model.

Description

Information processing system, information processing device, database construction method and program

[0001] The present disclosure relates to an information processing system, an information processing device, a database construction method, and a program.

[0002] With the recent advances in deep learning, machine learning techniques have been applied to a wide range of technical fields. For example, in the technical field of natural language processing, research and development of machine learning models used for information extraction, information retrieval, summary generation, machine translation, speech recognition, dialogue processing, etc. Natural language processing using generative models, such as the recently released ChatGPT (registered trademark), has attracted attention.

[0003] Several approaches have been proposed for dialogue processing using generative models to obtain more appropriate answers to questions from generative models. One such approach is retrieval-augmented generation (RAG). RAG involves building a knowledge database in advance, extracting information related to the question from the knowledge database, and creating prompt data to be input to the generative model from the extracted related information and the question.

[0004] Ori Ram, 6 others, “In-Context Retrieval-Augmented Language Models”, [online], January 31, 2020, Internet <URL: https: / / arxiv.org / abs / 2302.00083>

[0005] Knowledge databases are often constructed using document data such as business manuals stored as knowledge on the user side of generative models, such as companies and government agencies. However, from the viewpoint of searchability, it is not desirable to use document data in its original form in a knowledge database, and it is necessary to convert the document data into structured data by redefining it with some kind of structure (regularity) before constructing a knowledge database.

[0006] However, the task of converting document data into structured data requires handling a huge amount of data, which requires a lot of time, effort, and costs.

[0007] The objective of the present disclosure is to provide an information processing system, an information processing device, a database construction method, and a program that can provide an efficient environment for generative AI.

[0008] One aspect of the present disclosure relates to an information processing system having a first information processing device managed by a user who uses a generative model, a second information processing device managed by a generative model operator, and a third information processing device managed by a structuring business operator that structurizes document data, wherein the first information processing device transmits the document data to the third information processing device, the structured data generated from the document data is stored in a database, and the first information processing device transmits a question to the second information processing device and receives an answer generated using the generative model.

[0009] Another aspect of the present disclosure relates to an information processing device managed by a user who uses a generative model, the information processing device having a processor, the processor transmitting document data to an information processing device for a structuring company managed by a structuring company that structures the document data, the structured data generated from the document data being stored in a database, the processor transmitting questions to an information processing device for a generative model operator managed by the generative model operator, and receiving answers generated using the generative model.

[0010] Another aspect of the present disclosure relates to a database construction method executed on an information processing device managed by a user who uses a generative model, the database construction method including a step in which a processor provided in the information processing device transmits document data to an information processing device for a structuring company managed by a structuring company that structures the document data, the structured data generated from the document data is stored in a database, and the processor transmits a question to an information processing device for a generative model operator managed by the generative model operator, and receives an answer generated using the generative model.

[0011] Another aspect of the present disclosure relates to a program executed on an information processing device managed by a user who uses a generative model, which causes the computer to execute a step of sending document data to an information processing device for a structuring business managed by a structuring business that structures the document data, and the structured data generated from the document data is stored in a database, and the program further causes the computer to execute a step of sending a question to an information processing device for a generative model operator managed by the generative model operator, and a step of receiving an answer generated using the generative model.

[0012] According to the present disclosure, it is possible to provide an information processing system, an information processing device, a database construction method, and a program that can provide an efficient environment for generative AI.

[0013] FIG. 1 is a block diagram showing an example of a system configuration of an information processing system according to a first embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of hardware configurations of a user device, a structured data generation device, and a generative model operation device according to the first embodiment of the present disclosure. FIG. 3 is a sequence diagram showing an example of a knowledge database construction process performed in the information processing system according to the first embodiment of the present disclosure, and an example of an interaction process for a generative model using the knowledge database. FIG. 4 is a diagram showing an example of a document data transmission screen displayed on a user device in the first embodiment of the present disclosure. FIG. 5 is a diagram showing an example of a standby screen displayed after document data is transmitted by a user device in the first embodiment of the present disclosure. FIG. 6 is an explanatory diagram showing an example of an outline of document structure recognition processing by a structured data generation device in the first embodiment of the present disclosure. FIG. 7 is a diagram showing an example of document data in the first embodiment of the present disclosure. FIG. 8 is a diagram showing an example of document data after layout recognition processing (structured data in the structuring process) in the first embodiment of the present disclosure. FIG. 9 is a diagram showing an example of document data after semantic block formation (structured data in the structuring process) in the first embodiment of the present disclosure. FIG. 10 is a flowchart showing an example of a process for generating structured data in a first generation mode (document structure recognition process) performed by a structured data generation device in a first embodiment of the present disclosure. FIG. 11A is a diagram showing an example of a UI screen used in a process for generating structured data in a second generation mode (document structure recognition process) performed by a structured data generation device in a first embodiment of the present disclosure. FIG. 11B is a flowchart showing an example of a process for generating structured data in a second generation mode (document structure recognition process) performed by a structured data generation device in a first embodiment of the present disclosure. FIG. 12 is an explanatory diagram showing an example of an outline of chunk division by a user device in a first embodiment of the present disclosure. FIG. 13 is an explanatory diagram of an example of chunks in a first embodiment of the present disclosure. FIG. 14 is a block diagram showing an example of a system configuration of an information processing system in a second embodiment of the present disclosure. FIG. 15 is a sequence diagram showing an example of a process for building a knowledge database performed in an information processing system in a second embodiment of the present disclosure, and an example of an interaction process for a generation model using the knowledge database.FIG. 16 is a block diagram illustrating an example of a system configuration of an information processing system according to a third embodiment of the present disclosure.

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0015] In the following embodiments, a method for constructing a knowledge database used for interactive processing of a generative model and a method for interactive processing of a generative model using the knowledge database are disclosed.

[0016] First Embodiment First, an information processing system that realizes a method for building a knowledge database and a method for interactive processing for a generative model according to a first embodiment of the present disclosure will be described.

[0017] 1 is a block diagram showing an example of a system configuration of an information processing system 10 according to a first embodiment of the present disclosure. As shown in FIG. 1, the information processing system 10 includes a user device 20, a structured data generation device 30, a knowledge database 40, and a generative model operation device 50.

[0018] The user device 20 and the structured data generation device 30 are connected to an external network 2. In addition, the knowledge database 40 and the generative model operation device 50 are connected to an internal network 3, and are connected to the external network 2 via the internal network 3.

[0019] The external network 2 may be, for example, at least one of the Internet and a WAN (Wide Area Network), but is not limited to these. The internal network 3 may be, for example, a LAN (Local Area Network), but is not limited to these. The external network 2 and the internal network 3 may be wired networks, wireless networks, or a mixture of wired and wireless networks. The network configuration is not limited to the configuration shown in FIG. 1, and may be any configuration in which devices can communicate with each other.

[0020] The generative model 60 is also connected to the external network 2. The generative model 60 may or may not be included in the information processing system 10.

[0021] The user device 20 (an example of a first information processing device) is an information processing device managed by a user (customer) who uses the generative model 60. The user may be a corporation such as a company, a public official such as a government agency, or an individual. The user device 20 requests the structured data generation device 30 to convert document data into structured data, stores the structured data as reference data in the knowledge database 40, and performs interactive processing with the generative model 60 via the generative model operation device 50.

[0022] Examples of document data include, but are not limited to, various document data such as in-house documents, business manuals, product manuals, catalogs, test materials, weekly magazines, and economic magazines.

[0023] The structured data may be data in which document data is defined by some structure (regularity), such as data in which the layout of document data is defined by a predetermined structure. For example, the structured data may be data in a format in which each element constituting the structure of the document data is assigned a label. Furthermore, for example, the structured data may be data in which the above-mentioned data is written in a markup language such as XML (Extensible Markup Language), or in a format such as JSON (JavaScript Object Notation). Furthermore, the structured data may be data in which the above-mentioned data is expressed in a tabular format.

[0024] The structured data generation device 30 (an example of a third information processing device) is an information processing device managed by a structuring business that structures document data. Examples of structuring businesses include, but are not limited to, AI (artificial intelligence) vendors. The structured data generation device 30 structures document data requested for conversion by the user device 20. As a result, when the user device 20 performs dialogue processing with the generative model 60 via the generative model operation device 50, the structured data generation device 30 generates structured data to be used as reference data to be referenced when the generative model 60 generates an answer, and provides the structured data to the user device 20.

[0025] The structured data generation device 30 may provide a service for converting document data into structured data to a user as a web service such as SaaS (Software as a Service) or Web API (Web Application Programming Interface). Here, the structured data generation device 30 may include, as a structured data generation mode, at least a first generation mode in which structured data is generated from document data at a first quality. Furthermore, the structured data generation device 30 may include, as a structured data generation mode, at least a second generation mode in which structured data is generated from document data at a second quality higher than the first quality. In the first embodiment of the present disclosure, the structured data generation device 30 includes, as a structured data generation mode, at least the first generation mode and the second generation mode, but is not limited thereto. Note that the generation time of structured data in the first generation mode is shorter than the generation time of structured data in the second generation mode.

[0026] For example, a first generation mode may be a mode in which structured data is generated using a Web API, and a second generation mode may be a mode in which structured data is generated using SaaS. In this case, the first generation mode may be a mode in which document data is received from the user device 20, the received document data is converted into structured data using a structure model for generating structured data, and the structured data is transmitted to the user device 20. In this first generation mode, the conversion from document data to structured data is automated by the structure model, so that structured data of the quality guaranteed by the structure model can be delivered to the user in a short delivery time.

[0027] An example of a structured model is a machine learning model such as a deep learning model that converts input document data into structured data and outputs it. Such a structured model can be generated in advance by learning from training document data. Examples of training document data include, but are not limited to, data in which document data is associated with converted document data that results in a correct conversion (structured data that results in a correct structuring) as a correct answer label. Note that structured models may be prepared for each type of document data (such as a business manual or a product manual).

[0028] In addition, the second generation mode can be a mode in which document data is received from the user device 20, the received document data is converted into structured data using the structured model, the conversion result is checked by an operator, and the data is transmitted to the user device 20. In this second generation mode, not only is the conversion from document data to structured data automated by the structured model, but the conversion result is also checked by an operator, so that structured data of higher quality than that guaranteed by the structured model can be delivered to the user. Note that the second generation mode is expected to take longer to deliver than the first generation mode because the conversion result must be checked by an operator.

[0029] In an embodiment of the present disclosure, whether the structured data generation device 30 generates structured data in the first generation mode or the second generation mode is assumed to be predetermined, for example, by a contract between the user and the structuring provider, but is not limited thereto. For example, the user device 20 may be able to select whether to cause the structured data generation device 30 to generate structured data in the first generation mode or the second generation mode. For example, when transmitting document data to the structured data generation device 30, the user device 20 may select whether to generate structured data from the document data in the first generation mode or the second generation mode. This allows for flexible response, such as using the first generation mode for document data that is less likely to be converted into structured data, such as text data, and using the second generation mode for document data that is more likely to be converted into structured data, such as document data containing table data.

[0030] The knowledge database 40 (an example of a database) is a database for Retrieval-Augmented Generation (RAG) that is constructed by accumulating structured data provided by the user device 20 as reference data. In other words, the knowledge database 40 is a database that stores reference data that is referenced when the generative model 60 generates an answer during the above-described dialogue processing.

[0031] The generative model operation device 50 (an example of a second information processing device) is an information processing device managed by a generative model operator. Examples of generative model operators include, but are not limited to, companies that provide AI solutions, such as customizing and introducing the usage environment of the generative model 60 for each user. In the first embodiment of the present disclosure, the knowledge database 40 is also managed by the above-mentioned generative model operator, but is not limited to this. In the first embodiment of the present disclosure, a case will be described in which the cloud services provided by the knowledge database 40 and the generative model operation device 50 and the cloud services provided by the generative model 60 are operated by different entities, and the usage environment of the generative model 60 is provided to the user as a multi-cloud service that combines these cloud services. However, this is not limited to this, and the cloud services provided by the knowledge database 40, the generative model operation device 50, and the generative model 60 may be operated by the same entity, and the usage environment of the generative model 60 may be provided to the user as a single cloud service.

[0032] The generative model operation device 50 includes an interaction processing application 80. Using the functions of the interaction processing application 80, the generative model operation device 50 works in conjunction with the knowledge database 40 and the generative model 60 to support (achieve) interaction processing between the user device 20 and the generative model 60. Specifically, when the user device 20 performs interaction processing with the generative model 60, the generative model operation device 50 receives question data from the user device 20 and extracts reference data related to the question data from the knowledge database 40. The generative model operation device 50 inputs prompt data including the question data and the extracted reference data to the generative model 60, obtains an answer based on the prompt data from the generative model 60, and transmits the answer to the user device 20.

[0033] The generative model 60 is a natural language processing model known as generative AI (generative artificial intelligence), and provides a service using natural language processing. In the embodiment of the present disclosure, the generative model 60 is described as an example of an interactive AI service in which a large language model (LLM: Large Language Models) such as ChatGPT (registered trademark) is fine-tuned, but is not limited to this.

[0034] Here, the user device 20, the structured data generation device 30, and the generative model operation device 50 may be realized by a computing device such as a server, a personal computer (PC), a smartphone, or a tablet, and may have a hardware configuration such as that shown in Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the user device 20, the structured data generation device 30, and the generative model operation device 50 according to the first embodiment of the present disclosure.

[0035] That is, the user device 20 has a drive device 21, a storage device 22, a memory device 23, a processor 24, a user interface (UI) device 25, and a communication device 26, which are interconnected via a bus B. Similarly, the structured data generation device 30 has a drive device 31, a storage device 32, a memory device 33, a processor 34, a user interface (UI) device 35, and a communication device 36, which are interconnected via a bus B. Similarly, the generative model operation device 50 has a drive device 51, a storage device 52, a memory device 53, a processor 54, a user interface (UI) device 55, and a communication device 56, which are interconnected via a bus B.

[0036] The programs or instructions that realize the various functions and processes of the user device 20, the structured data generation device 30, and the generative model operation device 50 may be stored on a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 21, 31, or 51, the programs or instructions are installed from the storage medium via the drive device 21, 31, or 51 into the storage device 22, 32, or 52 or the memory device 23, 33, or 53. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0037] The storage devices 22, 32, 52 are realized by hard disk drives or the like, and store installed programs or instructions as well as files, data, etc. used in executing the programs or instructions.

[0038] The memory devices 23, 33, and 53 are realized by random access memory, static memory, etc., and when a program or instruction is activated, they read and store the program, instruction, data, etc. from the storage devices 22, 32, and 52. The storage devices 22, 32, and 52, the memory devices 23, 33, and 53, and the removable storage medium may be collectively referred to as non-transitory storage medium.

[0039] The processors 24, 34, 54 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and perform various functions and processes of the user device 20, the structured data generation device 30, and the generative model operation device 50 in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc. stored in the memory devices 23, 33, 53.

[0040] The user interface (UI) devices 25, 35, 55 may be composed of input devices such as a keyboard, mouse, camera, microphone, etc., output devices such as a display, speaker, headset, printer, etc., and input / output devices such as a touch panel, and realize an interface between the user and the user device 20, the structured data generation device 30, and the generative model operation device 50. For example, the user operates a GUI (Graphical User Interface) displayed on a display or touch panel using a keyboard, mouse, etc. to operate the user device 20, the structured data generation device 30, and the generative model operation device 50.

[0041] The communication devices 26, 36, 56 are realized by various communication circuits that execute wired and / or wireless communication processing with communication networks such as external devices, the Internet, a LAN (Local Area Network), and a cellular network.

[0042] However, the above-described hardware configuration is merely an example, and the user device 20, structured data generation device 30, and generative model operation device 50 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0043] 3 is a sequence diagram showing an example of the construction process of the knowledge database 40 performed in the information processing system 10 according to the first embodiment of the present disclosure, and an example of the interactive process for the generative model 60 using the knowledge database 40. Note that the processes from steps S101 to S105 in the sequence diagram shown in FIG. 3 correspond to the construction process of the knowledge database 40, and the processes from steps S106 to S111 correspond to the interactive process for the generative model 60 using the knowledge database 40.

[0044] 3, in step S101, the processor 24 of the user device 20 transmits document data to the structured data generation device 30. Typically, the document data is data of various documents such as those described above that the user desires to use in interactive processing with the generative model 60. The format of the document data is not limited to plain text data, and more generally may be data in various formats such as PDF (Portable Document Format), WORD, Excel, or images.

[0045] 4 is a diagram showing an example of a document data transmission screen 101 displayed on the user device 20 in the first embodiment of the present disclosure. The document data transmission screen 101 shown in Fig. 4 is a screen on a web service that converts document data into structured data, and is a screen for transmitting document data to the structured data generation device 30. The document data transmission screen 101 is displayed when the user device 20 accesses the structured data generation device 30 via a browser or an application displayed on the UI device 25.

[0046] The user sets document data 201 as a target for transmission to structured data generation device 30 by moving (drag-and-dropping) document data 201 to drop area 102 on document data transmission screen 101 or by pressing file selection button 103 to select document data 201. When the user sets document data 201 as a target for transmission and presses file selection button 103, processor 24 of user device 20 transmits document data 201 to structured data generation device 30. When the user presses back button 104, the screen returns to the previous screen.

[0047] Fig. 5 is a diagram showing an example of a standby screen 111 displayed after document data is transmitted by the user device 20 in the first embodiment of the present disclosure. The standby screen 111 shown in Fig. 5 is displayed on the UI device 25 until the structured data generation device 30 converts the document data 201 into structured data. In the standby screen 111 shown in Fig. 5, the process of converting the document data 201 into structured data by the structured data generation device 30 is shown, with the process being divided into performed processes and unperformed processes.

[0048] Next, in step S102, the processor 34 of the structured data generation device 30 performs document structure recognition processing on the document data received from the user device 20 to generate structured data.

[0049] FIG. 6 is an explanatory diagram illustrating an example of an outline of document structure recognition processing by the structured data generation device 30 according to the first embodiment of the present disclosure. As shown in FIG. 6 , when document data 201 to be processed is provided, the processor 34 of the structured data generation device 30 executes document structure recognition processing on the document data 201. As a result, each element in the document data 201 is converted into microcontent, and structured data 301 is generated. In the example shown in FIG. 6 , each element in the document data is extracted in association with a label such as "heading" or "section," and structured as structured data 301 indicating the relationship between each element (in the illustrated example, the "heading" element is associated with each "section" element). Note that such document structure recognition processing may be implemented using any known method, such as the structuring model described above. As an example, each element in the document data may be extracted in association with a label based on the layout of the document data.

[0050] 7 to 9, the document structure recognition process performed by the structured data generation device 30 will be described in detail. Fig. 7 is a diagram showing an example of document data 211 in the first embodiment of the present disclosure. The document data 211 shown in Fig. 7 is a financial statement, and is unstructured data.

[0051] The processor 34 of the structured data generation device 30 performs document structure recognition processing on the document data 211. The document structure recognition processing includes, but is not limited to, layout recognition processing, table recognition processing, and data integration processing. The layout recognition processing and table recognition processing can be realized, for example, by the structured model described above. In the layout recognition processing, the layout structure of the document data 211 is recognized (detected) from the document data 211. The layout structure includes each element (object) constituting the layout structure and its coordinate information. In the layout recognition processing, each recognized element is further tagged with a label.

[0052] 8 is a diagram showing an example of document data 311 (structured data in the structuring process) after layout recognition processing in the first embodiment of the present disclosure. In the document data 311 shown in Fig. 8, an element 321 indicating "beverage business," an element 331 indicating "beverage division," an element 341 indicating "(2) Overview of financial status for the current period," and an element 351 indicating "(3) Overview of cash flow for the current period" are extracted as headings and tagged with the label "section."

[0053] In addition, in the document data 311 shown in Figure 8, element 332 indicating the sentence (main text) directly below element 331, element 342 indicating the sentence (main text) directly below element 341, and elements 352 and 354 indicating the sentence (main text) directly below element 351 are extracted as text and tagged with the label "text".

[0054] In the document data 311 shown in FIG. 8, an element 353 immediately below the element 351, which indicates a table relating to cash flow indexes, is extracted as a table and tagged with the label "table."

[0055] In the table recognition process, the structure is recognized (extracted) from the table (element 353) recognized in the layout recognition process. Specifically, in the table recognition process, it is recognized how many rows and columns the table (element 353) to be recognized consists of, and what values ​​are entered in each field, and this is expressed in, for example, a CSV (Comma Separated Values) format.

[0056] In the data integration process, elements recognized in the layout recognition process and the table recognition process are classified into blocks of the same meaning or content unit to form semantic blocks.

[0057] 9 is a diagram showing an example of document data 316 (structured data in the structuring process) after semantic block formation in the first embodiment of the present disclosure. In the document data 316 shown in FIG. 9, elements 321, 331, and 332 are identified as cohesive parts of semantic content and form semantic block 411. Similarly, elements 341 and 342 are identified as cohesive parts of semantic content and form semantic block 421. Similarly, elements 351, 352, 353, and 354 are identified as cohesive parts of semantic content and form semantic block 431.

[0058] In the data integration process, the table (element 353) in the document data 316 after the semantic block formation has been subjected to table recognition processing is rewritten, for example, in Markdown format, and the entire document data 316 is converted into structured data, for example, by converting it into text data in XML format or JSON format.

[0059] In the data integration process, the formation of semantic blocks is not an essential process and may be omitted.

[0060] Here, the process of generating structured data in the first generation mode and the process of generating structured data in the second generation mode by the structured data generating device 30 will be described.

[0061] 10 is a flowchart showing an example of a process for generating structured data (document structure recognition process) in the first generation mode by the structured data generation device 30 in the first embodiment of the present disclosure. Note that the selection of a structured model to be used in the process for generating structured data in the first generation mode may be performed automatically by the structured data generation device 30, or may be performed by the user on the document data transmission screen 101 shown in FIG.

[0062] First, the processor 34 of the structured data generating device 30 performs layout recognition processing on the document data (step S201).

[0063] Next, if the processor 34 of the structured data generation device 30 detects as a result of the layout recognition process that a table is included in the document data (Yes in step S203), it further performs table structure recognition processing on the detected table (step S205).

[0064] If the table recognition process does not detect that the document data contains a table (No in step S203), the process in step S205 is not performed.

[0065] Next, the processor 34 of the structured data generating device 30 performs data integration processing on the document data that has been subjected to the layout recognition processing and table recognition processing, and generates structured data (step S207).

[0066] 11A is a diagram showing an example of a UI screen 501 used in the process of generating structured data in the second generation mode (document structure recognition process) by the structured data generation device 30 in the first embodiment of the present disclosure. The UI screen 501 shown in Fig. 11A is used for selecting a structured model to be used in the process of generating structured data in the second generation mode, and for an operator to check the result of conversion from document data to structured data.

[0067] For example, when an operator works using the structured data generation device 30, the UI screen 501 shown in Fig. 11A is displayed on the structured data generation device 30 by starting an application included in the structured data generation device 30. Also, when an operator works using a work terminal (not shown), the UI screen 501 shown in Fig. 11A can be displayed on the work terminal by accessing a web application included in the structured data generation device 30 from the work terminal.

[0068] The UI screen 501 shown in FIG. 11A includes a file selection button 511, radio buttons 521, 523, 525, and 527 for selecting a structured model, an upload button 532, a layout edit button 541, a table edit button 543, and a download button 551.

[0069] The file selection button 511 is a button for selecting document data to be converted into structured data from the document data received from the user device 20. For example, the operator selects the file selection button 511 with the cursor 505 by operating the mouse or the like to open a file selection screen and selects the document data to be converted into structured data.

[0070] Radio button 521 is a button for selecting a structured model for a business manual. The structured model for a business manual is a structured model suitable for converting document data whose document type is "business manual" into structured data. The structured model for a business manual is a structured model generated by learning using document data of a business manual as learning data.

[0071] Radio button 523 is a button for selecting a structured model for a type 1 product manual. A structured model for a type 1 product manual is a structured model suitable for converting document data whose document type is "product manual (type 1)" into structured data. A structured model for a type 1 product manual is a structured model generated by learning using document data of a type 1 product manual as learning data.

[0072] Radio button 525 is a button for selecting a structured model for a Type 2 product manual. A structured model for a Type 2 product manual is a structured model suitable for converting document data whose document type is "product manual (Type 2)" into structured data. A structured model for a Type 2 product manual is a structured model generated by learning using document data of a Type 2 product manual as learning data.

[0073] Radio button 527 is a button for selecting a general-purpose structured model. A general-purpose structured model is a structured model that can convert document data into structured data with a certain degree of accuracy regardless of the document type. The general-purpose structured model is a structured model generated by learning document data of various document types handled by structured data generation device 30, such as business manuals, type 1 product manuals, and type 2 product manuals, as learning data.

[0074] For example, the operator selects one of radio buttons 521, 523, 525, and 527 with cursor 505 by operating a mouse or the like to select a structured model to be used for converting document data selected by selecting file selection button 511 into structured data. Note that it is assumed that a general-purpose structured model will be selected when the operator cannot determine a structured model suitable for converting document data selected by selecting file selection button 511 into structured data. In the second generation mode, the operator checks the results of the conversion from document data to structured data. For this reason, a general-purpose structured model is provided based on the idea that if the operator is unsure which structured model to select, it would be more efficient to use a general-purpose structured model to convert into structured data and have the operator check any conversion errors or mistakes.

[0075] 11A has been described using an example in which the structured model types are the four types described above, the types of structured models are not limited to these. Furthermore, the method for selecting a structured model is not limited to the example using radio buttons described above, and any method may be used, such as selecting from a list display.

[0076] 11A has been described taking as an example a case where the structure model is selected by an operator, but the present invention is not limited to this and the structure model may be automatically selected based on a predetermined algorithm. The predetermined algorithm may be, for example, a method of predicting in advance the conversion accuracy when document data selected by the selection operation of the file selection button 511 is converted (structured) with each structure model, and automatically selecting the structure model with the highest conversion accuracy.

[0077] Specifically, a prediction model associated with each structured model is prepared, and the prediction model is used to predict in advance the conversion accuracy when selected document data is converted (structured) using the corresponding structured model. The conversion accuracy output by the prediction model can be defined as a prediction problem of an AP (Average Precision) value, which indicates the quality of the structuring result when the selected document data is converted using the corresponding structured model. Examples of training data for training the prediction model include, but are not limited to, data in which the AP value of the structured model is associated as a correct answer label with an intermediate output value when document data in the corresponding structured model is input.

[0078] Even when a structured model is automatically selected, it is preferable to display which structured model is selected on the UI screen 501 shown in Fig. 11A using radio buttons 521, 523, 525, and 527. In this way, the operator can notice that an unintended structured model has been selected, and can manually reselect a structured model.

[0079] The upload button 532 is a button for uploading document data selected by the selection operation of the file selection button 511 in order to convert the document data into structured data using a structured model selected by one of the radio buttons 521, 523, 525, and 527. For example, the operator uploads the document data selected by the selection operation of the file selection button 511 by selecting the upload button 532 with the cursor 505 using a mouse or the like.

[0080] The layout edit button 541 is a button for confirming and editing the layout of structured data converted from uploaded document data. For example, the operator selects the layout edit button 541 with the cursor 505 by operating the mouse or the like to open a layout confirmation and edit screen and confirm and edit the layout of the structured data.

[0081] The table edit button 543 is a button for confirming and editing the table included in the structured data into which the uploaded document data has been converted. For example, the operator selects the table edit button 543 with the cursor 505 by operating the mouse or the like to open a table confirmation and edit screen, and confirms and edits the table included in the structured data.

[0082] The download button 551 is a button for downloading the checked and edited structured data. For example, the operator downloads the checked and edited structured data by selecting the download button 551 with the cursor 505 by operating the mouse or the like.

[0083] In this way, structured data is generated from document data using at least a structured model suitable for the document data from among multiple structured models provided by the structured data generation device 30. The structured data generation device 30 also displays a UI screen 501 (an example of a selection screen) for selecting a structured model to be used for structuring. The structured data generation device 30 also displays the selected structured model suitable for the document data in an identifiable manner on the UI screen 501. The multiple structured models also include a general-purpose structured model trained using training data used to train each of the multiple structured models.

[0084] FIG. 11B is a flowchart showing an example of the process of generating structured data (document structure recognition process) in the second generation mode by the structured data generation device 30 in the first embodiment of the present disclosure.

[0085] First, the processor 34 of the structured data generation device 30 accepts access to the application and displays the UI screen (step S301).

[0086] Next, the processor 34 of the structured data generation device 30 accepts the selection of the document data to be converted into structured data on the UI screen (step S303).

[0087] Next, the processor 34 of the structured data generation device 30 accepts, on the UI screen, the selection of a structured model to be used for conversion into structured data (step S305).

[0088] Next, the processor 34 of the structured data generation device 30 accepts, on the UI screen, an operation to upload the document data to be converted into structured data, and uploads the document data for conversion into structured data (step S307).

[0089] Next, the processor 34 of the structured data generation device 30 performs layout recognition processing on the uploaded document data (step S309).

[0090] Next, if the processor 34 of the structured data generation device 30 detects as a result of the layout recognition process that a table is included in the document data (Yes in step S311), it further performs table structure recognition processing on the detected table (step S313).

[0091] Next, when the processor 34 of the structured data generation device 30 accepts the selection of the layout edit button 541 on the UI screen, it opens a layout confirmation / edit screen in a manner that allows the results of the layout recognition process to be confirmed and edited, and accepts an operation from the operator to correct any errors in the layout recognition (step S315).

[0092] Next, when the processor 34 of the structured data generation device 30 accepts the selection of the table edit button 543 on the UI screen, it opens a table confirmation / edit screen in a manner that allows the results of the table recognition processing to be confirmed and edited, and accepts an operation from the operator to correct any errors in the table recognition (step S317).

[0093] If the table recognition process does not detect that the document data contains a table (No in step S311), steps S313 and S317 are not performed.

[0094] Next, the processor 34 of the structured data generation device 30 accepts an operation to download the confirmed and edited structured data on the UI screen, and performs a data integration process on the results of the confirmed and edited layout recognition process and the results of the table recognition process to generate and download the structured data (step S319).

[0095] 3, in step S103, the processor 34 of the structured data generation device 30 transmits the generated (converted) structured data to the user device 20. This ends the display of the standby screen 111 shown in FIG. 5 on the user device 20.

[0096] Next, in step S104, the processor 24 of the user device 20 receives the structured data from the structured data generation device 30 and performs chunk division on the received structured data by dividing the data into any chunk units based on the user's operation. Note that the chunk division in step S104 does not have to be performed by the user device 20, and may be performed by the structured data generation device 30 between steps S102 and S103, for example. However, chunk division of the structured data is not a required process, and may not be performed (omitted) by either the user device 20 or the structured data generation device 30.

[0097] The chunk unit, which is the unit for dividing the chunk, may be any unit such as a unit of the number of characters, a unit of a page of document data, or a unit of a collection of semantic content of document data.

[0098] 12 is an explanatory diagram illustrating an example of an overview of chunk division by the user device 20 in the first embodiment of the present disclosure. As shown in FIG. 12, when structured data 301 to be processed is provided, the processor 24 of the user device 20 performs chunk division on the structured data 301 based on a user operation. As shown in FIG. 12, in the structured data 301, a "heading" is associated with "term 1," "term 2," and "term 3," respectively. Therefore, in the example shown in FIG. 12, the structured data 301 is divided into a chunk 306 including a "heading" and "term 1," a chunk 307 including a "heading" and "term 2," and a chunk 308 including a "heading" and "term 3."

[0099] 13 is an explanatory diagram of an example of chunk 531 in the first embodiment of the present disclosure. Note that chunk 531 corresponds to the chunk of semantic block 431 when document data 316 shown in FIG. 9 is converted into structured data and divided into chunks based on the semantic content of semantic blocks 411, 421, and 431. Specifically, element 351 of document data 316 shown in FIG. 9 corresponds to element 451 of chunk 531, element 352 corresponds to element 452 of chunk 531, element 353 of document data 316 corresponds to element 453 of chunk 531, and element 354 of document data 316 corresponds to element 454 of chunk 531. In addition to element 351, element 451 of chunk 531 also has information such as id, title, filename, and headerline added to it, but this information may be added by the data integration process or may be added when the chunk is divided.

[0100] 3, in step S105, the processor 24 of the user device 20 transmits the structured data to the knowledge database 40 and stores it as reference data. Specifically, the processor 24 of the user device 20 transmits the structured data divided into chunks (chunks) to the knowledge database 40 and stores them as reference data.

[0101] 3, the process of building the knowledge database 40 is realized. The knowledge database 40 can be accessed by the dialogue processing application 80 implemented in the generative model operation device 50.

[0102] The following continues with a description of the interactive processing for the generative model 60 using the knowledge database 40. Note that an input / output interface for interfacing with the interactive processing application 80 of the generative model operation device 50 may be installed on the user device 20, and the user may use the generative model 60 by operating the input / output interface screen. Alternatively, the interactive processing application 80 may be provided to the user device 20 through a website operated by the generative model operation device 50, and the user may use the generative model 60 by operating the input / output interface screen on the website.

[0103] In step S106, the processor 24 of the user device 20 transmits question data for the generative model 60 to the generative model operation device 50.

[0104] Next, in step S107, the processor 54 (dialogue processing application 80) of the generative model operation device 50 performs any known language processing on the question data to extract words contained in the question data. The processor 54 (dialogue processing application 80) of the generative model operation device 50 transmits the words extracted from the question data and / or words similar to the extracted words to the knowledge database 40 as search queries.

[0105] Next, in step S108, the processor 54 (dialogue processing application 80) of the generative model operation device 50 extracts and acquires reference data related to the search query from the knowledge database 40. For example, the reference data extracted from the knowledge database 40 may include words corresponding to the search query or words similar thereto.

[0106] Next, in step S109, the processor 54 (dialogue processing application 80) of the generative model operation device 50 generates prompt data to be input to the generative model 60 based on the question data and the extracted reference data related to the question data, and inputs the generated prompt data to the generative model 60. For example, the processor 54 (dialogue processing application 80) of the generative model operation device 50 may generate prompt data in which structured data is embedded.

[0107] Next, in step S110 , the processor 54 (dialogue processing application 80 ) of the generative model operation device 50 acquires response data from the generative model 60 .

[0108] Next, in step S111 , the processor 54 (dialogue processing application 80 ) of the generative model operation device 50 returns the response data to the user device 20 .

[0109] The above-described processing from steps S106 to S111 in the sequence diagram shown in FIG. 3 realizes interactive processing for the generative model 60 using the knowledge database 40.

[0110] According to the first embodiment of the present disclosure, the structured data generation device 30 can provide a service that automatically converts document data into structured data, thereby generating structured data from document data while reducing various costs. Furthermore, according to the first embodiment, the generated structured data is returned to the user, allowing the user to modify the structured data, such as by dividing it into chunks, before building the knowledge database 40. Furthermore, according to the first embodiment, multiple generation modes can be provided to the user when generating structured data from document data, allowing the user to respond flexibly. Thus, according to the first embodiment of the present disclosure, a development environment with a high degree of freedom for building the knowledge database 40 can be provided.

[0111] Furthermore, in the first embodiment, the prompt data input to the generative model 60 is created using reference data created based on document data provided by the user device 20. The reference data used is composed of chunks, which are structured data divided into units with coherent semantic content. Therefore, the response data to the prompt data can have higher response accuracy than RAG, which uses chunks created in units of pages or characters.

[0112] Second Example Next, an information processing system according to a second example of the present disclosure will be described. The information processing system according to the second example differs from the first example in that the knowledge database is managed by a structuring business operator who manages a structured data generation device, rather than by a generative model operator who manages a generative model operation device. Note that in the second example, differences from the first example will be mainly described, and descriptions of similarities to the first example will be omitted.

[0113] 14 is a block diagram showing an example of the system configuration of an information processing system 1010 according to the second embodiment of the present disclosure. As shown in Fig. 14, the information processing system 1010 differs from the first embodiment in a user device 1020, a structured data generation device 1030, a knowledge database 1040, and a generative model operation device 1050.

[0114] In the second example, the user device 1020 and the generative model operation device 1050 are connected to an external network 2. In addition, the structured data generation device 1030 and the knowledge database 1040 are connected to an internal network 1003, and are connected to the external network 2 via the internal network 1003.

[0115] In the second embodiment, the user device 1020 does not store the structured data as reference data in the knowledge database 1040, but the structured data generation device 1030 stores the structured data as reference data in the knowledge database 1040. Also, as described above, the structuring business operator who manages the structured data generation device 1030, rather than the generative model operator who manages the generative model operation device 1050, manages the knowledge database 1040. For this reason, the knowledge database 1040 is provided on the internal network 1003 under the control of the structuring business operator.

[0116] FIG. 15 is a sequence diagram showing an example of the process of building the knowledge database 1040 performed in the information processing system 1010 according to the second embodiment of the present disclosure, and an example of the interactive process for the generative model 60 using the knowledge database 1040.

[0117] First, the processes in steps S101 and S102 are the same as those in FIG.

[0118] Next, in step S404, the chunk division that was performed by the user device 20 in S104 of Fig. 3 is performed by the structured data generation device 1030 based on an operation by an operator. Note that an arbitrary chunk unit that is the unit of chunk division may be specified, for example, on the document data transmission screen 101 shown in Fig. 4.

[0119] Next, in step S405, the structured data generating device 1030 transmits the structured data to the knowledge database 40, instead of transmitting the structured data by the user device 20 in S105 of FIG.

[0120] The subsequent processes from step S106 to step S111 are the same as those in FIG.

[0121] [Third Example] Next, an information processing system according to a third example of the present disclosure will be described. The information processing system according to the third example differs from the second example in that the structuring business operator who manages the structured data generation device, rather than the generative model operator who manages the generative model operation device, manages the generative model operation device and plays the role of generative model operator. In other words, in the third example, the structuring business operator also serves as the generative model operator. Note that in the third example, the differences from the second example will be mainly described, and explanations of the same parts as the second example will be omitted.

[0122] 16 is a block diagram showing an example of a system configuration of an information processing system 2010 according to the third embodiment of the present disclosure. As shown in Fig. 16, in the information processing system 2010, a generative model operation device 2050 differs from that in the second embodiment.

[0123] In the third embodiment, the generative model operation device 2050 is further connected to the internal network 1003. As described above, the structuring business operator who manages the structured data generation device 1030, rather than the generative model operator who manages the generative model operation device, manages the generative model operation device 2050 and plays the role of the generative model operator. For this reason, the generative model operation device 2050 is provided on the internal network 1003 under the control of the structuring business operator.

[0124] The processing performed by the information processing system 2010 is the same as that in the second embodiment.

[0125] [Variant Example] In one embodiment, the structured data generation device 30 may accept, via the document data transmission screen 101 shown in Figure 4, a specification of the chunk unit (granularity) along with the document data selected as the target for creating structured data.

[0126] In one embodiment, a machine learning model trained to convert document data into structured data may be available for each data type of document data. In this case, the structured data generation device 30 may select a machine learning model corresponding to the data type of the document data to be processed, and generate structured data from the document data using the selected machine learning model.

[0127] The selection of such a machine learning model may be performed automatically by the structured data generation device 30, or may be performed by the user on the document data transmission screen 101 shown in Fig. 4. Furthermore, if no selection is made by the user, a default general-purpose machine learning model may be applied.

[0128] Although examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure as set forth in the claims.

[0129] The present disclosure also includes the following aspects.

[0130] (1) An information processing system comprising: a first information processing device managed by a user who uses a generative model; a second information processing device managed by a generative model operator; and a third information processing device managed by a structuring business operator that structurizes document data; wherein the first information processing device transmits the document data to the third information processing device; the structured data generated from the document data is stored in a database; and the first information processing device transmits a question to the second information processing device and receives an answer generated using the generative model.

[0131] (2) The information processing system according to (1), wherein the first information processing device receives the structured data from the third information processing device and stores the received structured data in the database.

[0132] (3) The information processing system according to (1), wherein the third information processing device stores the structured data in the database.

[0133] (4) The information processing system according to (1) above, wherein the structuring business operator also serves as the generative model operator.

[0134] (5) The information processing system according to (1), wherein the structured data is generated from the document data using at least one structured model suitable for the document data from among a plurality of structured models provided by the third information processing device.

[0135] (6) The information processing system according to (5) above, wherein the third information processing device displays a selection screen for selecting a structured model to be used for structuring.

[0136] (7) The information processing system according to (6), wherein the third information processing device displays the selected structured model suited to the document data in an identifiable manner on the selection screen.

[0137] (8) The information processing system according to (6), wherein the plurality of structured models includes a general-purpose structured model trained using training data used in training each of the plurality of structured models.

[0138] (9) The information processing system described in (1) above, wherein the generation mode of the structured data by the third information processing device includes at least a first generation mode in which the structured data is generated from the document data at a first quality, and a second generation mode in which the structured data is generated from the document data at a second quality that is higher than the first quality.

[0139] (10) The information processing system according to (9) above, wherein the generation time of the structured data in the first generation mode is shorter than the generation time of the structured data in the second generation mode.

[0140] (11) The information processing system according to (9), wherein whether the third information processing device generates the structured data in the first generation manner or the second generation manner is determined in advance.

[0141] (12) The information processing system according to (9), wherein the first information processing device is capable of selecting whether to cause the third information processing device to generate the structured data in the first generation manner or the second generation manner.

[0142] (13) The information processing system according to (2), wherein the first information processing device performs chunk division to divide the structured data received from the third information processing device into arbitrary chunk units, and transmits the structured data divided into chunk units to the database.

[0143] (14) The information processing system according to (1), wherein the prompt data input to the generative model is created from question data and the structured data related to the question data.

[0144] (15) The information processing system according to (1), wherein the structured data is text data having a heading element and a body element.

[0145] (16) An information processing device managed by a user who uses a generative model, the information processing device having a processor, the processor transmits the document data to an information processing device for a structuring business that is managed by a structuring business that structures the document data, the structured data generated from the document data is stored in a database, and the processor transmits a question to an information processing device for a generative model operator that is managed by the generative model operator, and receives an answer generated using the generative model.

[0146] (17) A database construction method executed on an information processing device managed by a user who uses a generative model, comprising: a step in which a processor provided in the information processing device transmits the document data to an information processing device for a structuring business that is managed by a structuring business that structures the document data; the structured data generated from the document data is stored in a database; and a step in which the processor transmits a question to an information processing device for a generative model operator that is managed by the generative model operator; and a step in which the processor receives an answer generated using the generative model.

[0147] (18) A program executed on an information processing device managed by a user who uses a generative model, causing the computer to execute the steps of: sending document data to an information processing device for a structuring business managed by a structuring business that structures the document data; the structured data generated from the document data is stored in a database; sending a question to an information processing device for a generative model operator managed by the generative model operator; and receiving an answer generated using the generative model.

[0148] This application is based on the Japanese application of Patent Application No. 2024-085104 filed on May 24, 2024, the Japanese application of Patent Application No. 2024-085105 filed on May 24, 2024, and the Japanese application of Patent Application No. 2025-020021 filed on February 10, 2025, the contents of which are all incorporated by reference into this application.

[0149] 10, 1010, 2010 Information processing system 20, 1020 User device 30, 1030 Structured data generation device 40, 1040 Knowledge database 50, 1050, 2050 Generative model operation device 60 Generative model 80 Dialogue processing application

Claims

1. An information processing system comprising: a first information processing device managed by a user who uses a generative model; a second information processing device managed by a generative model operator; and a third information processing device managed by a structuring business operator that structurizes document data; wherein the first information processing device transmits the document data to the third information processing device; the structured data generated from the document data is stored in a database; and the first information processing device transmits a question to the second information processing device and receives an answer generated using the generative model.

2. The information processing system according to claim 1, wherein the first information processing device receives the structured data from the third information processing device and stores the received structured data in the database.

3. The information processing system according to claim 1, wherein said third information processing device stores said structured data in said database.

4. The information processing system according to claim 1, wherein the structuring business operator also serves as the generative model operator.

5. An information processing system as described in claim 1, wherein the structured data is generated from the document data using at least one structured model suitable for the document data from among a plurality of structured models provided by the third information processing device.

6. The information processing system according to claim 5, wherein the third information processing device displays a selection screen for selecting a structured model to be used for structuring.

7. The information processing system according to claim 6, wherein the third information processing device displays the selected structured model suited to the document data in an identifiable manner on the selection screen.

8. The information processing system according to claim 5, wherein the plurality of structured models includes a general-purpose structured model trained using training data used to train each of the plurality of structured models.

9. An information processing system as described in claim 1, wherein the generation mode of the structured data by the third information processing device includes at least a first generation mode in which the structured data is generated from the document data at a first quality, and a second generation mode in which the structured data is generated from the document data at a second quality that is higher than the first quality.

10. An information processing system according to claim 9, wherein the time required to generate the structured data in the first generation mode is shorter than the time required to generate the structured data in the second generation mode.

11. An information processing system according to claim 9, wherein it is predetermined whether the third information processing device generates the structured data in the first generation manner or the second generation manner.

12. An information processing system according to claim 9, wherein the first information processing device is capable of selecting whether to have the third information processing device generate the structured data in the first generation manner or the second generation manner.

13. The information processing system of claim 2, wherein the first information processing device performs chunk division on the structured data received from the third information processing device, dividing the data into arbitrary chunk units, and transmits the structured data divided into chunk units to the database.

14. The information processing system of claim 1, wherein the prompt data input to the generative model is created from question data and the structured data related to the question data.

15. The information processing system according to claim 1, wherein the structured data is text data having a heading element and a body element.

16. An information processing device managed by a user who uses a generative model, the information processing device having a processor, the processor transmits the document data to an information processing device for a structuring business that is managed by a structuring business that structures the document data, the structured data generated from the document data is stored in a database, and the processor transmits a question to an information processing device for a generative model operator that is managed by the generative model operator, and receives an answer generated using the generative model.

17. A database construction method executed on an information processing device managed by a user who uses a generative model, comprising a step in which a processor provided in the information processing device transmits the document data to an information processing device for a structuring business managed by a structuring business that structures the document data, the structured data generated from the document data being stored in a database, and the database construction method comprising a step in which the processor transmits a question to an information processing device for a generative model operator managed by the generative model operator, and a step in which the processor receives an answer generated using the generative model.

18. A program executed on an information processing device managed by a user who uses a generative model, causing the computer to execute the steps of: sending document data to an information processing device for a structuring business managed by a structuring business that structures the document data; the structured data generated from the document data is stored in a database; sending a question to an information processing device for a generative model operator managed by the generative model operator; and receiving an answer generated using the generative model.

Images