Information processing device, information processing method, and program

Abstract

This information processing device comprises: a means for receiving, from a user, input data, a table structure instruction which pertains to a table structure, and a first prompt which is a natural language instruction pertaining to table elements that are to be extracted from the features of the input data; a means for determining the table structure on the basis of the table structure instruction; and a means for generating table data by inputting the input data and the first prompt to a trained AI model. The means for generating table data generates the table data by extracting the table elements from the features of the input data on the basis of the first prompt and assigning the table elements to the table structure.

Description

Information Processing Apparatus, Information Processing Method, and Program

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

[0002] Generally, in various business areas, documents including a table structure (hereinafter referred to as "table documents", for example, estimates) are used. Such table documents use tables with various table structures. There is a demand from users to convert tables with various table structures into tables with a desired table structure.

[0003] In recent years, the development of technologies for extracting information by utilizing AI (artificial intelligence) has also advanced. For example, Patent No. 7468960 discloses a technology for efficiently extracting necessary information from a large number of files with different formats.

[0004] However, the technology of Patent No. 7468960 is not a technology specialized for table structures. Therefore, it is not easy to generate a table with a table structure desired by the user.

[0005] An object of the present invention is to easily generate a table with a table structure desired by the user.

[0006] One aspect of the present invention is an information processing apparatus including: means for receiving, from a user, input data, a table structure instruction regarding a table structure, and a first prompt that is a natural language instruction regarding table elements to be extracted from features of the input data; means for determining a table structure based on the table structure instruction; means for generating table data by inputting the input data and the first prompt into a learned AI model, wherein the generating means extracts the table elements from the features of the input data based on the first prompt and generates the table data by assigning the table elements to the table structure.

[0007] This is a block diagram showing the configuration of the information processing system of this embodiment. This is a functional block diagram of the information processing system of Figure 1. This is an explanatory diagram of the overview of this embodiment. This is a sequence diagram of the generation process of this embodiment. This is a diagram showing an example of a screen displayed in the information processing of Figure 4. This is a diagram showing an example of input data and table data in the information processing of Figure 4. This is a diagram showing an example of column names, column types, and the first prompt of this embodiment. This is an explanatory diagram of the overview of Modification 1. This is a diagram showing an example of a screen displayed in the information processing of Modification 1. This is an explanatory diagram of the overview of Modification 2. This is a diagram showing an example of a screen displayed in the information processing of Modification 2. This is an explanatory diagram of the first example of table element extraction in Modification 2. This is an explanatory diagram of the second example of table element extraction in Modification 2. This is an explanatory diagram of the overview of Modification 3. This is an explanatory diagram of the first example of table element extraction in Modification 3. This is an explanatory diagram of the second example of table element extraction in Modification 3.

[0008] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. In the drawings used to illustrate the embodiment, the same reference numerals are generally used for identical components, and repeated descriptions thereof will be omitted.

[0009] (0) Definitions of Terms The definitions of terms in this embodiment are explained below.

[0010] A "pre-trained AI model" is a machine learning model that has been pre-trained to handle general-purpose tasks. A well-known example of a pre-trained AI model is the Large Language Model (LLM), but other models are also included (for example, small language models, or multimodal language models that can handle images and other formats).

[0011] A "prompt" is an instruction given to a trained AI model, and is an instruction in natural language (hereinafter referred to as a "natural language instruction").

[0012] A "table structure instruction" is an instruction concerning the structure of a table.

[0013] "Input data" refers to the data provided to a trained AI model. Input data is not limited to a specific format, but includes the following: • Document creation application files • PDF (Portable Document Format) files • Spreadsheet application files • Image files • CSV (Comma Separated Values) files • Text files • BIM (Building Information Modeling) files • 2D CAD (Computer Aided Design) files • 3D model files

[0014] An "input value" is a value entered by the user into an input field or similar. Input values ​​are not limited to a specific format, but include the following: • Symbols (letters or numbers) • Images • Charts • Graphs • BIM • 2D CAD • 3D CAD • 3D models • Various electronic objects that can be created, processed, and stored in a digital environment (for example, audio).

[0015] (1) Configuration of the Information Processing System The configuration of the information processing system will be explained. Figure 1 is a block diagram showing the configuration of the information processing system in this embodiment. Figure 2 is a functional block diagram of the information processing system in Figure 1.

[0016] As shown in Figure 1, the information processing system 1 comprises a client device 10 and a server 30. The client device 10 and the server 30 are connected via a network (for example, the Internet or an intranet) NW.

[0017] The client device 10 is a computer (an example of an "information processing device") that sends and receives information to and from the server 30. The client device 10 is, for example, a smartphone, a tablet terminal, or a personal computer.

[0018] Server 30 is a computer (an example of an "information processing device") that sends and receives information with the client device 10. Server 30 is, for example, a web server.

[0019] (1-1) Configuration of the client device The configuration of the client device 10 will be described below.

[0020] As shown in Figure 2, the client device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14.

[0021] The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of ROM (Read Only Memory), RAM (Random Access Memory), and storage (for example, flash memory or hard disk).

[0022] The programs include, for example, the following: • Operating System (OS) programs • Parameters, functions, and models used in information processing • Programs for applications that execute information processing (e.g., web browsers)

[0023] The data includes, for example, the following: • Databases referenced in information processing • Data obtained by performing information processing (i.e., the results of information processing)

[0024] The processor 12 is configured to realize the functions of the client device 10 by starting a program stored in the storage device 11. The processor 12 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0025] The input / output interface 13 is configured to receive user instructions from an input device connected to the client device 10 and to output information to an output device connected to the client device 10. The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display.

[0026] The communication interface 14 is configured to control communication between the client device 10 and the server 30.

[0027] (1-2) Server Configuration The configuration of server 30 will be explained.

[0028] As shown in Figure 2, the server 30 includes a storage device 31, a processor 32, an input / output interface 33, and a communication interface 34.

[0029] The storage device 31 is configured to store programs and data. The storage device 31 is, for example, a combination of ROM, RAM, and storage (e.g., flash memory or hard disk).

[0030] The program includes, for example, the following: • OS programs • Parameters, functions, and models used in information processing • Programs for applications that execute information processing

[0031] The data includes, for example, the following: • Databases referenced in information processing • Results of information processing execution

[0032] The processor 32 is configured to realize the functions of the server 30 by starting a program stored in the storage device 31. The processor 32 is, for example, a CPU, GPU, ASIC, FPGA, or a combination thereof.

[0033] The input / output interface 33 is configured to receive user instructions from an input device connected to the server 30 and to output information to an output device connected to the server 30. The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display.

[0034] The communication interface 34 is configured to control communication between the server 30 and the client device 10.

[0035] (2) Overview of the Embodiment The overview of this embodiment will be described below. Figure 3 is an explanatory diagram of the overview of this embodiment.

[0036] As shown in Figure 3, the server 30 has the following functions: • A function to receive input data including table T, a table structure instruction, and a first prompt from the user via the client device 10. • A function to determine the table structure of the table data based on the table structure instruction. • A function to extract table elements to be assigned to the table structure by inputting the input data and the first prompt into a trained AI model. • A function to generate table data by assigning the extracted table elements to the determined table structure. • A function to display the table data on the client device 10.

[0037] (3) Information Processing The information processing of this embodiment will be described. Figure 4 is a sequence diagram of the generation process of this embodiment. Figure 5 is a diagram showing an example of the screen displayed in the information processing of Figure 4. Figure 6 is a diagram showing an example of input data and table data in the information processing of Figure 4. Figure 7 is a diagram showing an example of column names, column types, and the first prompt of this embodiment.

[0038] [Acquisition of input data (S1110)] As shown in Figure 4, the client device 10 performs the acquisition of input data (S1110). Specifically, the processor 12 displays screen P1110 (Figure 5) on the display.

[0039] Screen P1110 includes the operation object B1110 and the field object F1110.

[0040] Operation object B1110 is an object that receives user instructions to confirm input to field object F1110.

[0041] Field object F1110 is an object that accepts input data (for example, file data).

[0042] [Acquisition of Instruction (S1111)] After step S1110, the client device 10 executes acquisition of an instruction (S1111). Specifically, when the user inputs input data to the field object F1110 and operates the operation object B1110, the processor 12 displays the screen P1111 (FIG. 5) on the display. The input data is, for example, at least one table image data as shown in FIG. 6A.

[0043] The screen P1111 includes a plurality of first sections PS1, a second section PS2, and operation objects B1111a to B1111b.

[0044] The operation object B1111a is an object that receives a user instruction for adding the first section PS1.

[0045] The operation object B1111a is an object that receives a user instruction for finalizing the input to the first section PS{1} to the second section PS2.

[0046] Each of the plurality of first sections PS1 is a section that receives a table structure instruction and a first prompt. The table structure instruction is an input value regarding the table structure (e.g., columns) of the table data. The first prompt is a natural language instruction regarding the table elements to be extracted from the characteristics of the input data. Each first section PS1 is distinguished for each column. That is, each first section PS1 is configured to receive an instruction for each column. Each first section PS1 includes field objects F1111a to F1111e. Thus, the first section PS1 is configured to receive an instruction in column units.

[0047] The field object F1111a is an input field object that receives a user instruction (table structure instruction) for specifying the column name for each column. As shown in FIG. 7, the column name includes, for example, the following: - Item - Quantity - Unit - Unit price - Amount

[0048] Field object F1111b is an input field object that accepts user instructions (table structure instructions) specifying the type of column value for each column (hereinafter referred to as "column type"). As shown in Figure 7, column types include, for example, the following: • Item column type "STRING" (string) • Quantity column type "INT" (integer value) • Unit column type "STRING" (string) • Unit price column type "FLOAT" (value including decimals) • Amount column type "INT" (integer value)

[0049] The field object F1111c is an input field object that accepts user instructions (table structure instructions) specifying the index key for each column.

[0050] Field object F1111d is an input field object that accepts user instructions specifying a first prompt for each column. The first prompt is unique to each column. As shown in Figure 7, the first prompt may include, for example: • Item column type: "Enter the estimate item." • Quantity column type: "Enter the quantity unit price." • Unit column type: "The unit can be either meters or square meters, so if there is a superscript, recognize and display it as square meters. Also, m2 should be m." ２ Please correct it to the following: • Unit price column type: "Please enter the unit price. Please include two decimal places." • Amount column type: "Please enter the amount."

[0051] Section 2 PS2 includes the field object F1111e.

[0052] Field object F1111e is an input field object that accepts user instructions specifying a second prompt. The second prompt contains instructions for generating table data (i.e., natural language instructions that apply to all columns, not just each individual column). Thus, the second section PS2 is configured to accept a second prompt, which is a natural language instruction for the entire table.

[0053] In the first example of step S1111, the processor 12 receives user instructions in response to user operations (e.g., keyboard input) on field objects F1111a to F1111e.

[0054] In the second example of step S1111, the processor 12 identifies the column name and column type by applying OCR (Optical Character Recognition) processing to the input data obtained in step S1110 (for example, image data of a table). The processor 12 displays the specific results (column name and column type) in field objects F1111a to F1111b, respectively, and accepts user instructions in response to user editing operations (for example, keyboard input). The field objects F1111c to F1111e are the same as in the first example of step S1111.

[0055] [Generation Request (S1112)] After step S1111, the client device 10 executes a generation request (S1112). Specifically, when the user inputs user instructions into field objects F1111a to F1111e and operates the operation object B1111b, the processor 12 sends generation request data to the server 30. The generation request data includes the following information: - Input data obtained in step S1110 - User instructions (table structure instructions and first prompts) entered into multiple first sections PS1 - User instructions (second prompt) entered into second section PS2

[0056] [Determination of Table Structure (S1130)] After step S1112, the server 30 performs the determination of the table structure (S1130). Specifically, the processor 32 determines the table structure (for example, column names and column types) based on the table structure instructions (information entered in field object F1111a) included in the generation request data.

[0057] [Extraction of Table Elements (S1131)] After step S1130, the server 30 performs extraction of table elements (S1131). Specifically, the storage device 31 stores the trained AI model.

[0058] The first example of step S1131 is one in which table elements are extracted based on a first prompt (i.e., user instructions to the second section PS2 are omitted). In this case, the processor 32 extracts table elements from the features of the input data by providing the input data and the first prompt (information entered into the field object F1111d) to the trained AI model.

[0059] A second example of step S1131 is an example in which table elements are extracted based on a combination of the first prompt and the second prompt (Figure 5). In this case, the processor 32 extracts table elements from the features of the input data by providing the input data, the first prompt (information entered into field object F1111d), and the second prompt (information entered into field object F1111e) to the trained AI model.

[0060] A third example of step S1131 is one in which the second prompt includes instructions for a calculation on a column of column type "numeric". For example, the second prompt includes instructions to calculate the sum of the values ​​in the column named "Amount" (for example, the prompt "Calculate the sum of the values ​​in the column "Amount" and include it in the table with the label "Total""). In this case, the processor 32 performs the following processes: - Extract table elements from the characteristics of the input data, similar to the second example of step S1131. - Calculate the sum of the values ​​assigned to the column "Amount" (i.e., the total amount). - Add a column labeled "Total Amount" to the table structure. - Extract the result of the calculation of the total amount as information to be assigned to the column labeled "Total Amount".

[0061] [Generating Table Data (S1132)] After step S1131, the server 30 performs table data generation (S1132). Specifically, the processor 32 generates table data by assigning the table elements extracted in step S1131 to the table with the table structure determined in step S1130.

[0062] After step S1132, the server 30 executes a generation response (S1133). Specifically, the processor 32 sends generation response data to the client device 10. The generation response data includes the table data obtained in step S1131.

[0063] [Presentation of Table Data (S1113)] After step S1133, the client device 10 performs the presentation of table data (S1113). Specifically, the processor 12 displays screen P1112 (Figure 6B) on the display.

[0064] Screen P1112 contains an image of the tabular data included in the generated response data.

[0065] (4) Summary of this embodiment According to this embodiment, the system includes means (S1130) for receiving input data, a table structure instruction, and a first prompt from a user, means (S1130) for determining a table structure based on the table structure instruction, and means (S1131 to S1132) for generating table data by inputting the input data and the first prompt into a trained AI model. The generating means generates table data by extracting table elements from the characteristics of the input data based on the first prompt and assigning the table elements to a table structure (S1132). This allows the user to easily generate a table having a table structure specified by the user.

[0066] The means of acceptance may also accept table structure instructions regarding the columns of the table (S1130). This allows the user to reliably generate a table with the table structure they specify.

[0067] The means of receiving input may include receiving table structure instructions and a first prompt via input fields PS1 to PS2 for each column. This allows the user to generate new values ​​and create table data even for columns that do not exist in the file. For example, the result of calculations between existing columns may be obtained, or detailed descriptions of existing columns may be added.

[0068] The means of acceptance may include a table structure instruction (F1111a) specifying column names. This allows the user to reliably generate a table with the table structure they specify.

[0069] The means of acceptance may also accept a table structure instruction that specifies a column type (F1111b) indicating the type of value for each column. This allows the user to reliably generate a table with the table structure they specify.

[0070] The method of acceptance may include a table structure instruction (F1111c) that specifies the index key of a column. This ensures that table data can be reliably generated even for information that is not represented as a table within the file. For example, in the case shown in Figure 11, the information "room name," "size," and "floor number" is not written as a table. In this case, one method is to use "room name" as the key index. By specifying "room name" as the key index, it is possible to first search for a "list of rooms," generate rows in the table structure, and then use each "room name" as the key to extract the "size" and "floor number" for each room. In this way, key indexes are effective for extracting table data from files where the information is not organized as table data.

[0071] The receiving means may receive a second prompt (F1111e), which is a natural language instruction common to each column of the table, and the generating means may generate table data by inputting the input data, the first prompt, and the second prompt into a trained AI model (S1131-S1132). This allows the user to easily obtain table data corresponding to more complex prompts.

[0072] The generation method may involve performing calculations on the numerical values ​​included in the input data in accordance with the calculation instructions, if the second prompt includes instructions for calculation, and then assigning the results of the calculations to the table structure. This allows the user to easily obtain table data that reflects the calculation results.

[0073] The input data may include at least one table. This ensures that the user can reliably generate the desired table.

[0074] (5) Modified Examples A modified example of this embodiment will be described.

[0075] (5-1) Modification Example 1 Modification Example 1 is explained below. Modification Example 1 is an example in which the input data includes multiple tables.

[0076] (5-1-1) Overview of Modification Example 1 The overview of Modification Example 1 is explained below. Figure 8 is an explanatory diagram of the overview of Modification Example 1.

[0077] As shown in Figure 8, the server 30 has the following functions: • A function to receive input data including multiple tables (e.g., Table T1 and Table T2), a table structure instruction, and a first prompt from the user via the client device 10. • A function to determine the table structure of the table data based on the table structure instruction. • A function to extract table elements to be assigned to the table structure by inputting the input data and the first prompt into a trained AI model. • A function to generate table data based on multiple tables T1 to T2 by assigning the extracted table elements to the determined table structure. • A function to display the table data on the client device 10.

[0078] (5-1-2) Information Processing of Modified Example 1 The information processing of Modified Example 1 will be explained. Figure 9 shows an example of a screen displayed in the information processing of Modified Example 1.

[0079] [Acquisition of Input Data (S1110)] As shown in Figure 4, the client device 10 performs the acquisition of input data (S1110). Specifically, the processor 12 receives input data including multiple tables (table T1 "Our Specifications" and table T2 "Other Companies' Specifications" in Figure 9A) via the field object F1110.

[0080] [Acquisition of Instructions (S1111) to Generation Request (S1112)] After step S1110, the client device 10 executes the acquisition of instructions (S1111) to the generation request (S1112) in the same manner as in this embodiment.

[0081] [Determination of table structure (S1130) to generation response (S1133)] After step S1112, the server 30 executes the determination of table structure (S1130) to generation response (S1133) in the same manner as in this embodiment.

[0082] [Presentation of Table Data (S1113)] After step S1133, the client device 10 performs the presentation of table data (S1113). Specifically, the processor 12 displays screen P1112 (Figure 9B) on the display. Screen P1112 includes an image of the table data contained in the generated response data.

[0083] (5-1-3) Summary of Modification 1 According to Modification 1, the input data may include multiple tables. This allows the user to reliably generate the desired table even if the amount of tables in the input data increases.

[0084] (5-2) Modification 2 Modification 2 will be explained. Modification 2 is an example where the input data is in a format other than a table (i.e., does not include a table).

[0085] (5-2-1) Overview of Modification Example 2 The overview of Modification Example 2 will be explained. Figure 10 is an explanatory diagram of the overview of Modification Example 2.

[0086] As shown in Figure 10, the server 30 has the following functions: • A function to receive input data including non-table information NT, a table structure instruction, and a first prompt from the user via the client device 10. • A function to determine the table structure of the table data based on the table structure instruction. • A function to extract table elements to be assigned to the table structure by inputting the input data and the first prompt into a trained AI model. • A function to generate table data based on non-table information NT by assigning the extracted table elements to the determined table structure. • A function to display the table data on the client device 10.

[0087] (5-2-2) Information Processing of Modified Example 2 The information processing of Modified Example 2 will be explained. Figure 11 is a diagram showing an example of a screen displayed in the information processing of Modified Example 2. Figure 12 is an explanatory diagram of the first example of extracting table elements in Modified Example 2. Figure 13 is an explanatory diagram of the second example of extracting table elements in Modified Example 2.

[0088] [Acquisition of input data (S1110)] As shown in Figure 4, the client device 10 performs the acquisition of input data (S1110). Specifically, the processor 12 receives input data including data other than tables NT1 to NT2 (for example, the architectural drawings in Figure 11A) via the field object F1110.

[0089] [Acquisition of Instructions (S1111) to Generation Request (S1112)] After step S1110, the client device 10 executes the acquisition of instructions (S1111) to the generation request (S1112) in the same manner as in this embodiment.

[0090] [Determination of Table Structure (S1130)] After step S1112, the server 30 performs determination of the table structure (S1130) in the same manner as in this embodiment.

[0091] [Extraction of table elements (S1131)] After step S1130, the server 30 performs extraction of table elements (S1131).

[0092] In the first example of step S1131, as shown in Figure 12, the generated request data includes the following table structure instructions TS1 to TS3 and first prompts PR1-1 to PR1-3 for each column: • Table structure instruction TS1: Column name "Room name" • First prompt PR1-1: "Extract the room name." • Table structure instruction TS2: Column name "Size" • First prompt PR1-2: "Extract the room size." • Table structure instruction TS3: Column name "Floor" • First prompt PR1-3: "Extract the room floor."

[0093] The processor 32 determines the table structure based on the table structure instructions TS1 to TS3.

[0094] The processor 32 inputs the input data and the first prompts PR1-1 to PR1-3 to a common pre-trained AI model M. The pre-trained AI model M extracts table elements E from the input data (Figure 10A) corresponding to the first prompts PR1-1 to PR1-3.

[0095] In the second example of step S1131, as shown in Figure 13, the generation request data includes table structure instructions TS1 to TS3 and first prompts PR1-1 to PR1-3, similar to those in Figure 12.

[0096] The processor 32 determines the table structure in the same manner as in Figure 12.

[0097] The processor 32 inputs the first prompts PR1-1 to PR1-3 to different pre-trained AI models M1 to M3. Each of M1 to M3 extracts table elements E from the input data (Figure 10A) corresponding to the first prompts PR1-1 to PR1-3.

[0098] [Generating table data (S1132)] After step S1131, the server 30 performs the generation of table data (S1132).

[0099] The first example of step S1132 is an example where the first example (Figure 12) is executed in step S1131. In this case, the server 30 executes the generation of table data (S1132) in the same manner as in this embodiment.

[0100] The second example of step S1132 is an example where the second example (Figure 13) is executed in step S1131. In this case, the server 30 generates table data by integrating the table elements E1 to E3, which are the outputs of each of M1 to M3.

[0101] [Generation Response (S1133)] After step S1132, the server 30 executes a generation response (S1133) in the same manner as in this embodiment. [Presentation of Table Data (S1113)] After step S1133, the client device 10 executes presentation of table data (S1113). Specifically, the processor 12 displays screen P1112 (Figure 11B) on the display. Screen P1112 includes an image of the table data contained in the generation response data.

[0102] (5-2-3) Summary of Modification 2 According to Modification 2, the input data may be at least one image file. This allows the user to reliably generate the desired table even if the input data is an image.

[0103] (5-3) Modification 3 Modification 3 is explained below. Modification 3 is an example in which the table structure is determined using a trained AI model (i.e., the table structure instruction is included in the first prompt).

[0104] (5-3-1) Overview of Modification Example 3 The overview of Modification Example 3 will be explained. Figure 14 is an explanatory diagram of the overview of Modification Example 3.

[0105] As shown in Figure 14, the server 30 has the following functions: • A function to receive input data including non-table information NT, a table structure instruction, and a first prompt from the user via the client device 10. • A function to determine the table structure and extract table elements to be assigned to the table structure by inputting the input data, table structure instruction, and first prompt into a trained AI model. • A function to generate table data based on non-table information NT by assigning the extracted table elements to the determined table structure. • A function to display the table data on the client device 10.

[0106] (5-3-2) Information Processing of Modified Example 3 The information processing of Modified Example 3 will be explained. Figure 15 is an explanatory diagram of the first example of extracting table elements in Modified Example 3. Figure 16 is an explanatory diagram of the second example of extracting table elements in Modified Example 3.

[0107] [Acquisition of input data (S1110) to generation request (S1112)] The client device 10 executes the acquisition of instructions (S1111) to generation request (S1112) in the same manner as in the modified example 2.

[0108] [Determination of table structure (S1130)] After step S1112, the server 30 performs the determination of the table structure (S1130).

[0109] In the first example of step S1130, as shown in Figure 15, the generated request data includes the following column-specific table structure instructions and first prompts: • Table structure instruction: Column name "Room name" • First prompt: "Extract the room name." • Table structure instruction: Column name "Size" • First prompt: "Extract the room size." • Table structure instruction: Column name "Floor" • First prompt: "Extract the room floor."

[0110] The processor 32 determines the table structure (column names "Room Name", "Size", and "Floor Number") by inputting each table structure instruction into a pre-trained AI model common to all columns.

[0111] In the second example of step S1130, as shown in Figure 16, the generation request data includes a table structure instruction and a first prompt similar to those in Figure 15.

[0112] The processor 32 determines the table structure (column names "room name", "size", and "floor number") by inputting each table structure instruction into individual M1 to M3 for each column.

[0113] [Extraction of table elements (S1131) to generation response (S1133)] After step S1130, the server 30 performs the extraction of table elements (S1131) to generation response (S1133) in the same manner as in the modified example 2.

[0114] [Presentation of table data (S1113)] After step S1133, the client device 10 performs presentation of table data (S1113) in the same manner as in the modified example 2.

[0115] (5-3-3) Summary of Modification 3 According to Modification 3, the means for determining the table structure may be to input the table structure instructions into a trained AI model. This allows the user to provide the table structure instructions using natural language.

[0116] (6) Other variations and other variations will be explained.

[0117] The storage device 11 may be connected to the client device 10 via a network NW. The storage device 31 may be connected to the server 30 via a network NW.

[0118] Each of the above information processing steps can be executed by either the client device 10 or the server 30. For example, if the client device 10 is capable of executing all of the above information processing steps, the client device 10 functions as a standalone information processing device without sending a request to the server 30.

[0119] User instructions may specify not only the information itself, but also the location of the information (address, path, etc.).

[0120] Input values ​​can be accepted in various ways. For example, users can input text, upload images, specify strings written in a document file, specify cells in a spreadsheet file, accept pre-registered input values ​​based on some trigger, or accept values ​​extracted from a pre-registered file based on some trigger. More specifically, a user click could be used as a trigger to extract the "building name" from a pre-registered drawing and use that as the input value.

[0121] User input can be provided not only through on-screen input, but also by accepting variables passed from an API (Application Programming Interface) or other scripts as input values.

[0122] When there are multiple processes performed by pre-trained AI models, for example, when there is a process to generate a script and a process to modify a script, the pre-trained AI models used for each process do not need to be exactly the same.

[0123] When performing some processing using a pre-trained AI model, it is not necessary to perform all of the processing using the pre-trained AI model; rather, some of the processing may be performed using the pre-trained AI model, and the remaining processing may be performed by other means.

[0124] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the embodiments described above. Furthermore, the embodiments described above can be improved or modified in various ways without departing from the spirit of the present invention. In addition, the embodiments and modifications described above can be combined.

[0125] 1: Information processing system 10: Client device 11: Storage device 12: Processor 13: Input / output interface 14: Communication interface 30: Server 31: Storage device 32: Processor 33: Input / output interface 34: Communication interface

Claims

1. An information processing device comprising: means for receiving input data, a table structure instruction relating to a table structure, and a first prompt which is a natural language instruction relating to table elements to be extracted from the characteristics of the input data from a user; means for determining a table structure based on the table structure instruction; and means for generating table data by inputting the input data and the first prompt into a trained AI model; wherein the generating means generates the table data by extracting the table elements from the characteristics of the input data based on the first prompt and assigning the table elements to the table structure.

2. The information processing apparatus according to claim 1, wherein the receiving means receives table structure instructions relating to the columns of the table.

3. The information processing apparatus according to claim 2, wherein the receiving means receives the table structure instruction and the first prompt via the input field for each column.

4. The information processing apparatus according to claim 3, wherein the receiving means receives a table structure instruction specifying a column name.

5. The information processing apparatus according to claim 3, wherein the receiving means receives a table structure instruction specifying a column type that indicates the type of value for each column.

6. The information processing apparatus according to claim 3, wherein the receiving means receives a table structure instruction specifying an index key for the column.

7. The information processing apparatus according to claim 6, wherein the means for determining the table structure determines the rows of the table based on the index key, and the means for generating generates the table data based on the rows.

8. The information processing apparatus according to any one of claims 1 to 7, wherein the receiving means receives a second prompt which is a natural language instruction common to each column of the table, and the generating means generates the table data by inputting the input data, the first prompt, and the second prompt into the trained AI model.

9. The information processing apparatus according to claim 8, wherein, when the second prompt includes an instruction for calculation, the generating means performs a calculation on the numerical values ​​included in the input data in accordance with the instruction for calculation, and assigns the result of the calculation to the table structure.

10. The information processing apparatus according to any one of claims 1 to 7, wherein the input data includes at least one table.

11. The information processing apparatus according to any one of claims 1 to 7, wherein the input data is at least one image data.

12. The information processing apparatus according to any one of claims 1 to 7, wherein the means for determining the table structure determines the table structure by inputting the table structure instruction into the trained AI model.

13. A computer-based information processing method, comprising: a step of receiving from a user input data, a table structure instruction relating to a table structure, and a first prompt which is a natural language instruction relating to table elements to be extracted from the characteristics of the input data; a step of determining a table structure based on the table structure instruction; a step of generating table data by inputting the input data and the first prompt into a trained AI model; and a step of generating table data by extracting table elements from the characteristics of the input data based on the first prompt and assigning the table elements to the table structure.

14. A program for which a computer functions as a means for receiving input data, a table structure instruction relating to a table structure, and a first prompt which is a natural language instruction relating to table elements to be extracted from the characteristics of the input data from a user; functions as a means for determining a table structure based on the table structure instruction; functions as a means for generating table data by inputting the input data and the first prompt into a trained AI model; and the generating means generates the table data by extracting the table elements from the characteristics of the input data based on the first prompt and assigning the table elements to the table structure.

Images