Information processing program, method, and device

Abstract

An information processing device according to the present invention converts table data, in which multiple elements are represented in a table format, into data of a format capable of distinguishing whether each of the plurality of elements is a header or a value, inputs the converted data into a large language model (LLM), and causes the LLM to carry out inference processing on the converted data.

Description

Information processing program, method, and device

[0001] The disclosed technology relates to an information processing program, an information processing method, and an information processing device.

[0002] Since the emergence of large language models (LLMs) in 2018, research and development in language processing has focused on LLMs, and various applications utilizing LLMs, such as ChatGPT (Chat Generative Pre-trained Transformer), have become widespread. LLMs excel at question answering and sentence generation for natural language input. Furthermore, input to LLMs can include not only natural language text but also tabular data.

[0003] For example, one study aimed to understand how well LLMs could perform table-related tasks using few-shot, in-context learning. In this study, inference tests were conducted on CSV-formatted data using the following three methods: 1) a method in which table data was provided and questions were asked; 2) a step-by-step inference method using the Chain of Thought (CoT) format; and 3) a CoT+SC (Self-Consistency: inference from multiple tries) format. This study confirmed that the CoT+SC method had the highest accuracy.

[0004] Wenhu Chen, "Large Language Models are few(1)-shot Table Reasoners," arXiv:2210.06710, 13 Oct 2022.

[0005] As mentioned above, LLM excels at question-answering and sentence generation for natural language input, but there is still room for improvement in accuracy for inference processing such as semantic analysis and response generation for tabular data.

[0006] In one aspect, the disclosed technology aims to improve the inference accuracy of large-scale language models for tabular data.

[0007] In one aspect, the disclosed technology converts tabular data in which multiple elements are represented in a table format into data in a format in which each of the multiple elements can be distinguished as a header or a value, and then inputs the converted data into a large-scale language model, causing the large-scale language model to perform inference processing on the converted data.

[0008] One aspect is that it has the effect of improving the inference accuracy of a large-scale language model for tabular data.

[0009] 1 is a diagram showing an example of table data; FIG. 1 is a diagram showing an example of CSV data converted from table data and an output of ChatGPT; FIG. 2 is a diagram for explaining inference using LLM for CSV data; FIG. 3 is a diagram for explaining a case where table data is converted into a format in which headers and values ​​can be distinguished and inference is performed using LLM; FIG. 4 is a functional block diagram of an information processing device according to a first embodiment; FIG. 5 is a diagram for explaining determination of the type of specific format into which table data is converted; FIG. 6 is a diagram showing an example of inference results; FIG. 7 is a block diagram showing an example of a schematic configuration of a computer functioning as an information processing device;

[0010] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings.

[0011] In this embodiment, data in which multiple elements are represented in a table format (hereinafter referred to as "table data") is input to the LLM to execute inference processing. In this embodiment, the table data is, for example, a software design document, a settlement report, etc., and also includes documents that include tables in some parts.

[0012] Here, we will explain the problems involved in making the LLM understand tabular data.

[0013] For example, a software design document created using spreadsheet software such as Excel contains not only a simple table representation, but also many sections in which pairs of headers (column names) and values ​​are associated with left and right or top and bottom cells, as shown in Figure 1. A simple table representation is a table in which headers are associated with the top row or left column, and values ​​corresponding to the headers are associated with each of the other rows or columns.

[0014] Converting such table data to CSV (Comma Separated Values) format often results in Ill-formatted data, i.e., poorly formatted data, as shown in the upper part of Figure 2. If an LLM is made to recognize such CSV data, its ability to understand the table data represented by the CSV data will be significantly reduced. For example, if the CSV data shown in the upper part of Figure 2 is input, ChatGPT will have difficulty understanding the contents of the table data, and the ChatGPT output will be as shown in the lower part of Figure 2.

[0015] As shown in Figure 3, when Ill-formatted CSV data is input to an LLM built using a group of texts in various formats as training data, the LLM performs inference using the CSV format as is. In other words, the CSV data used to train the LLM does not contain information to distinguish between headers and values. Therefore, if inference is performed using the CSV format as is, the LLM is likely to proceed with understanding without knowing which elements are headers and which elements are values. As a result, the inference results output by the LLM will be of low accuracy.

[0016] Therefore, in the following embodiments, as shown in Figure 4, table data is converted into a format in which the relationship between headers and values ​​is self-evident and easy to infer using LLM, i.e., a format in which headers and values ​​can be distinguished (hereinafter referred to as a "specific format"), and then inference using LLM is performed. The information processing device according to each embodiment will be described in detail below.

[0017] First Embodiment As shown in FIG. 5 , table data and inference content are input to the information processing device 10. The table data is data in which multiple elements are represented in a table format, such as a software design document or a company's financial statement as shown in FIG. 1 . The inference content is a question or the like indicating the inference to be performed by the LLM 24 regarding the content of the table data. An example of the inference content is, "What is the process after calling a WEBAPI with callApi?" The LLM 24 analyzes the content of the table data, infers an answer to the question, and outputs the inference result. In this embodiment, the LLM 24 used for inference is an existing one, such as the GPT series used in ChatGPT.

[0018] 5, the information processing device 10 according to the first embodiment functionally includes an assignment unit 12, a determination unit 14, a conversion unit 16, and an execution unit 18. An assignment model 22 and an LLM 24 are stored in a predetermined storage area of ​​the information processing device 10. The determination unit 14 and the conversion unit 16 are examples of conversion units of the disclosed technology.

[0019] The assignment unit 12 assigns information to each of a plurality of elements included in the table data, indicating whether the element is a header or a value. In this embodiment, for example, the headers or values ​​included in each cell of the table data, that is, when the table data is converted into CSV data, each of the headers or values ​​separated by a comma is referred to as one element.

[0020] Specifically, the attachment unit 12 converts the table data into CSV data and tags each element included in the CSV data with information on whether it is a heading or a value, using the attachment model 22. The attachment model 22 is a machine learning model constructed by the attachment model construction unit 30 by executing machine learning using an attachment model training data group 32. Each training data included in the attachment model training data group 32 is data in which an element is paired with a correct label indicating whether the element is a heading or a value.

[0021] For example, the assignment model construction unit 30 defines the objective variable y, which indicates headline-likeness, using the following multiple regression equation: y = ax1 + bx2 + cx3 + ... xi (i = 1, 2, 3, ...) are explanatory variables, which increase as the element becomes more headline-like. For example, x1 may be a variable that, with respect to the character size included in the table data, is expressed as x1 = (number of types + 1 - rank) / number of types, increasing as the character size of the element increases. The number of types is the number of character size types included in the table data. The rank is set in descending order of character size, with 1st, 2nd, .... Furthermore, for example, x2 may be the reciprocal of the number of characters in the element. Other explanatory variables that may be applied as xi include explanatory variables related to the arrangement of elements in the table data, explanatory variables indicating whether predetermined words that are likely to be used in headlines are used, and so on. Furthermore, a, b, c, ... are weights for each explanatory variable.

[0022] The assignment model construction unit 30 calculates weights a, b, c, ... of the multiple regression equation by multiple regression analysis, setting y = 1 when the correct label of each training data included in the assignment model training data group 32 is "heading", and y = 0 when the correct label is "value". The assignment model construction unit 30 provides the multiple regression equation to which the calculated weights are applied as the assignment model 22 to the information processing device 10.

[0023] The attached model construction unit 30 that constructs the attached model 22 and the attached model training data group 32 may be included in the information processing device 10, or may be included in an external device separate from the information processing device 10. In the latter case, the attached model 22 constructed in the external device may be read into the information processing device 10.

[0024] When the assignment unit 12 uses the above-described multiple regression equation as the assignment model 22, if the value obtained by inputting each element included in the CSV data converted from table data into the assignment model 22 is equal to or greater than a predetermined threshold TH1, the assignment unit 12 tags the element as a "heading." On the other hand, if the value obtained by inputting the element into the assignment model 22 is less than the threshold TH1, the assignment unit 12 tags the element as a "value."

[0025] In this way, by adding information indicating whether each element contained in the table data is a header or a value, LLM24's ability to recognize headers and values ​​can be improved, and the accuracy can be improved when converting table data to Markdown format or JSON format (details will be described later).

[0026] The discriminator 14 discriminates the type of specific format when converting table data into a format in which each of multiple elements can be distinguished as a header or a value, i.e., data in a specific format. By discriminating the type of specific format in this way, conversion into a specific format that makes it easy for the LLM 24 to understand the contents can be performed according to the original table data.

[0027] Examples of specific formats include Markdown format, JSON format, etc. The Markdown format is often used for descriptions in natural language, and therefore, when inference is performed based on such training data, the inference accuracy is improved for table data that contains many natural language elements. On the other hand, the JSON format is often used for descriptions in symbolic expressions such as programs, and therefore, when inference is performed based on such training data, the inference accuracy is improved for table data that contains many symbolic expression information.

[0028] Specifically, the determination unit 14 determines the type of specific format to convert the table data into based on at least one of the proportion of natural language elements and the proportion of symbolic expression elements among the multiple elements contained in the CSV data converted from the table data. More specifically, the determination unit 14 assigns part-of-speech information to each lexeme contained in the CSV data by morphologically analyzing the CSV data converted from the table data. For example, the bottom diagram of Figure 6 shows an example in which CSV data converted from table data such as that shown in the top diagram of Figure 6 is morphologically analyzed and part-of-speech information is assigned to each lexeme.

[0029] If the proportion of lexemes to which predetermined part-of-speech information indicating symbolic expressions is assigned is less than a predetermined threshold TH2, the discrimination unit 14 determines that the data is table data containing many natural language elements, and determines that the specific format to be converted is the Markdown format. On the other hand, if the proportion of lexemes to which predetermined part-of-speech information indicating symbolic expressions is assigned is equal to or greater than the threshold TH2, the discrimination unit 14 determines that the data is table data containing many symbolic expression elements, and determines that the format to be converted is the JSON format.

[0030] The part-of-speech information indicating a symbolic expression may be, for example, "symbol," "character," "noun (no dictionary information)," etc. "Noun (no dictionary information)" refers to the part-of-speech of a lexeme that is determined to be a noun but is not registered in the dictionary used in morphological analysis. In the example shown in the lower diagram of Figure 6, the discrimination unit 14 calculates the proportion of lexemes to which a part-of-speech indicating a symbolic expression is assigned, such as the lexemes indicated by the dashed line, out of the total number of vocabulary words, and compares the calculated proportion with a threshold value TH2 to discriminate the type of specific format to be converted.

[0031] The conversion unit 16 converts the table data into data in the specific format determined by the determination unit 14. Specifically, the conversion unit 16 generates a prompt (hereinafter referred to as a "conversion prompt") for converting the data format into the specific format determined by the determination unit 14. For example, the conversion unit 16 generates a question such as "Please rewrite the following CSV data in Markdown format," and generates the conversion prompt by converting this question into a format that is easy for the LLM 24 to understand using a general prompting process. The conversion unit 16 inputs the CSV data converted from the table data and the generated conversion prompt to the LLM 24, and obtains the data converted into the determined specific format as an output of the LLM 24.

[0032] The execution unit 18 inputs the data converted by the conversion unit 16 to the LLM 24 and causes the LLM 24 to execute inference processing on the converted data. Specifically, the execution unit 18 generates a prompt (hereinafter referred to as an "inference prompt") for executing inference by converting the inference content input to the information processing device 10 into a format that is easy for the LLM 24 to understand using a general prompting process. The execution unit 18 inputs the CSV data converted from the table data and the generated inference prompt to the LLM 24 and obtains, as an output from the LLM 24, an inference result such as that shown in FIG. 7.

[0033] CSV data does not contain information to distinguish between headers and values. On the other hand, for training data in Markdown or JSON format, it is clear during the training stage of LLM24 which elements are headers and which elements are values. By using such training data, LLM24 can perform inference while clearly understanding the distinction between headers and values.

[0034] Note that Few-Shot Prompting may be applied to the flow from the processing by the conversion unit 16 to the processing by the execution unit 18 described above.

[0035] The information processing device 10 may be realized by, for example, a computer 40 shown in FIG. 8 . The computer 40 includes a CPU (Central Processing Unit) 41, a GPU (Graphics Processing Unit) 42, a memory 43 as a temporary storage area, and a non-volatile storage device 44. The computer 40 also includes an input / output device 45 such as an input device and a display device, and an R / W (Read / Write) device 46 that controls reading and writing of data from and to a storage medium 49. The computer 40 also includes a communication I / F (Interface) 47 that is connected to a network such as the Internet. The CPU 41, GPU 42, memory 43, storage device 44, input / output device 45, R / W device 46, and communication I / F 47 are connected to one another via a bus 48.

[0036] The storage device 44 is, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage device 44 serving as a storage medium stores an information processing program 50 for causing the computer 40 to function as the information processing device 10. The information processing program 50 includes an assignment process control instruction 52, a discrimination process control instruction 54, a conversion process control instruction 56, and an execution process control instruction 58. The storage device 44 also includes an information storage area 60 in which information constituting each of the assignment model 22 and the LLM 24 is stored.

[0037] The CPU 41 reads the information processing program 50 from the storage device 44, loads it into the memory 43, and sequentially executes the control instructions of the information processing program 50. The CPU 41 operates as the assignment unit 12 shown in FIG. 5 by executing the assignment process control instruction 52. The CPU 41 operates as the determination unit 14 shown in FIG. 5 by executing the discrimination process control instruction 54. The CPU 41 operates as the conversion unit 16 shown in FIG. 5 by executing the conversion process control instruction 56. The CPU 41 operates as the execution unit 18 shown in FIG. 5 by executing the execution process control instruction 58. The CPU 41 also reads information from the information storage area 60 and loads the assignment model 22 and the LLM 24 into the memory 43. As a result, the computer 40 that executes the information processing program 50 functions as the information processing device 10. The CPU 41 that executes the program is hardware. A portion of the program may be executed by the GPU 42.

[0038] The functions realized by the information processing program 50 may be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0039] Next, the operation of the information processing device 10 according to the first embodiment will be described. The assignment model construction unit 30 constructs an assignment model 22 by performing machine learning using a training data group 32 for the assignment model, and provides the model to the information processing device 10. Furthermore, the LLM construction unit (not shown) constructs an LLM 24 by performing machine learning using training data, which is a group of texts in various formats, and provides the model to the information processing device 10. When table data and inference content are input to the information processing device 10 and an instruction to execute inference processing is issued, the information processing device 10 executes an information processing routine shown in FIG. 9 . Note that the information processing routine is an example of an information processing method of the disclosed technology.

[0040] In step S10, the annotation unit 12 acquires the table data and inference content input to the information processing device 10. Next, in step S12, the annotation unit 12 converts the table data into CSV data, and uses the annotation model 22 to tag each element included in the CSV data with information on headline likelihood or value likelihood.

[0041] Next, in step S14, the discrimination unit 14 performs morphological analysis on the CSV data converted from the table data, thereby assigning part-of-speech information to each lexeme included in the CSV data. Next, in step S16, the discrimination unit 14 determines whether the proportion of lexemes assigned with predetermined part-of-speech information indicating symbolic expressions, among the part-of-speech information assigned to each element of the table data, is equal to or greater than a threshold value TH2. If the proportion is equal to or greater than the threshold value TH2, the process proceeds to step S18. If the proportion is less than the threshold value TH2, the process proceeds to step S20.

[0042] In step S18, the discrimination unit 14 determines that the table data contains many symbolic expression elements and determines that the specific format to be converted is JSON format. The conversion unit 16 then generates a conversion prompt for converting the table data into JSON format data. On the other hand, in step S20, the discrimination unit 14 determines that the table data contains many natural language elements and determines that the specific format to be converted is Markdown format. The conversion unit 16 then generates a conversion prompt for converting the table data into Markdown format data.

[0043] Next, in step S22, the conversion unit 16 inputs the CSV data converted from the table data and the conversion prompt generated in step S18 or S20 to the LLM 24, and obtains the data converted into the identified specific format as the output of the LLM 24.

[0044] Next, in step S24, the execution unit 18 generates an inference prompt by using a general prompting process to convert the inference content acquired in step S10 into a format that is easy for the LLM 24 to understand. Next, in step S26, the execution unit 18 inputs the CSV data converted from the table data and the inference prompt generated in step S24 to the LLM 24, and acquires the inference result as the output of the LLM 24. The execution unit 18 then outputs the acquired inference result, and the information processing routine ends.

[0045] As described above, the information processing device according to the first embodiment converts table data in which multiple elements are represented in a table format into data in a format in which each of the multiple elements can be distinguished as a header or a value. The information processing device then inputs the converted data into a large-scale language model and causes the large-scale language model to perform inference processing on the converted data. This improves the inference accuracy of the large-scale language model for tabular data. Therefore, it is possible to develop a method for utilizing the LLM that allows the LLM to understand sentences containing tables and provides accurate information in response to user questions and requests.

[0046] Second Embodiment Next, a second embodiment will be described. In the information processing device according to the second embodiment, the same components as those in the information processing device 10 according to the first embodiment will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0047] In the second embodiment, it is assumed that the training data used to construct the LLM 24 is self-evident. In the second embodiment, a case will be described in which a specific format for converting table data is determined based on this training data.

[0048] 10 , the information processing device 210 according to the second embodiment functionally includes an assignment unit 12, a discriminant model construction unit 213, a discrimination unit 214, a conversion unit 16, and an execution unit 18. In addition, an assignment model 22, an LLM 24, and a discriminant model 226 are stored in a predetermined storage area of ​​the information processing device 210.

[0049] The discriminant model construction unit 213 constructs the word occurrence distribution in the LLM training data group 36 used to construct the LLM 24 by the LLM construction unit 34 as a discriminant model 226 for each type of specific format into which the table data is converted. For example, the discriminant model construction unit 213 creates the word occurrence distribution in CSV-format training data, the word occurrence distribution in Markdown-format training data, the word occurrence distribution in JSON-format training data, etc. Then, the discriminant model construction unit 213 compiles these word occurrence distributions for each type and constructs the discriminant model 226. The word occurrence distribution may be, for example, a histogram of the number of occurrences of each word that appears in the LLM training data group 36, and the histogram may be represented as a vector.

[0050] The discrimination unit 214 discriminates the type of specific format into which the table data is to be converted, based on a comparison between the appearance distribution of words in the table data and the discriminant model 226. Specifically, the discrimination unit 214 creates an appearance distribution of words for the table data, similar to the processing of the discriminant model construction unit 213. Then, the discrimination unit 214 discriminates, from among the appearance distributions of words for each type of specific format stored as the discriminant model 226, the type corresponding to the appearance distribution that is most similar to the appearance distribution of words in the table data, as the type of specific format into which the table data is to be converted.

[0051] The information processing device 210 may be realized by, for example, a computer 40 shown in Fig. 8. An information processing program 250 for causing the computer 40 to function as the information processing device 210 is stored in a storage device 44 of the computer 40. The information processing program 250 includes an assignment process control instruction 52, a discriminant model construction process control instruction 253, a discrimination process control instruction 254, a conversion process control instruction 56, and an execution process control instruction 58. The storage device 44 also includes an information storage area 260 in which information constituting each of the assignment model 22, the LLM 24, and the discriminant model 226 is stored.

[0052] The CPU 41 reads the information processing program 250 from the storage device 44, loads it in the memory 43, and sequentially executes the control instructions of the information processing program 250. The CPU 41 operates as the discriminant model construction unit 213 shown in FIG. 10 by executing the discriminant model construction process control instruction 253. The CPU 41 also operates as the discriminant unit 214 shown in FIG. 10 by executing the discrimination process control instruction 254. The CPU 41 also reads information from the information storage area 260 and loads each of the assignment model 22, the LLM 24, and the discriminant model 226 in the memory 43. The other control instructions are the same as those of the information processing program 50 according to the first embodiment. As a result, the computer 40 that executes the information processing program 250 functions as the information processing device 210.

[0053] The functions realized by the information processing program 250 may be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC, an FPGA, or the like.

[0054] Next, the operation of the information processing device 210 according to the second embodiment will be described. The assignment model construction unit 30 constructs an assignment model 22 and provides it to the information processing device 210. The LLM construction unit 34 constructs an LLM 24 and provides it to the information processing device 210. The discriminant model construction unit 213 constructs a word appearance distribution in the LLM training data group 36 used to construct the LLM 24 by the LLM construction unit 34 as a discriminant model 226 for each type of specific format into which table data is converted. When table data and inference content are input to the information processing device 210 and an instruction to execute inference processing is issued, the information processing device 210 executes the information processing routine shown in FIG. 9.

[0055] In the second embodiment, in steps S14-S20, the discrimination unit 214 discriminates the type of specific format to which the table data is to be converted based on a comparison between the appearance distribution of words in the table data and the discrimination model 226. Then, the conversion unit 16 generates a conversion prompt for converting the table data into data of the determined specific format.

[0056] As described above, the information processing device according to the second embodiment creates a word occurrence distribution in the training data used to construct the LLM for each type of specific format into which the table data is converted. The information processing device then determines the type of specific format into which the table data is converted based on a comparison between the word occurrence distribution in the table data and the word occurrence distribution for each type. In this way, when the training data used to construct the LLM is self-evident, it is possible to determine the type of specific format that is easy for the LLM to understand based on the word occurrence distribution and in accordance with the original table data.

[0057] Here, the results of an experiment using an existing LLM will be described.

[0058] Figure 11 shows an example of CSV data converted from table data containing a large amount of natural language information, and data converted from that CSV data into Markdown format using LLM. In this case, the table data has been properly converted into Markdown format. For example, the output of LLM for the question, "What is the process after calling a WEBAPI using callApi?" is as shown in Figure 7, and the correct answer to the question is output.

[0059] On the other hand, Figure 12 shows an example of data that was converted into JSON format using LLM from CSV data, which was created by converting table data containing a lot of natural language information, similar to the upper diagram of Figure 11. In this case, the data converted into JSON format has an inappropriate hierarchical expression, and the inference result of LLM does not provide an appropriate answer to the question.

[0060] FIG. 13 shows an example of CSV data converted from table data containing a large amount of symbolic information, and data converted from that CSV data to JSON format using LLM. In this case, the table data has been properly converted to JSON format. FIG. 14 shows the inference results obtained by LLM using data converted to JSON format. The inference results output the correct answer to the question.

[0061] On the other hand, Figure 15 shows an example of data obtained by converting CSV data, which was created by converting table data containing a large amount of symbolic information, into Markdown format, as well as the inference results, similar to the upper diagram of Figure 13. In this case, the data converted into Markdown format is simply CSV data converted directly into pipe expressions, and is not properly converted into Markdown format. Furthermore, the inference results using this Markdown format data also provide an incorrect answer to the question.

[0062] In the above embodiments, Markdown format and JSON format have been used as examples of formats that allow for distinguishing between headers and values, but formats other than these may also be applied as long as they allow for distinguishing between headers and values.

[0063] In addition, although the information processing program is pre-stored (installed) in the storage device in each of the above embodiments, the present invention is not limited to this. The program according to the disclosed technology may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, or a USB memory.

[0064] 10, 210 Information processing device 12 Assignment unit 213 Discriminant model construction unit 14, 214 Discrimination unit 16 Conversion unit 18 Execution unit 22 Assignment model 226 Discriminant model 30 Assignment model construction unit 32 Training data group for assignment model 34 LLM construction unit 36 ​​Training data group for LLM 40 Computer 41 CPU 42 GPU 43 Memory 44 Storage device 45 Input / output device 46 R / W device 47 Communication I / F 48 Bus 49 Storage medium 50, 250 Information processing program 52 Assignment process control command 253 Discriminant model construction process control command 54, 254 Discrimination process control command 56 Conversion process control command 58 Execution process control command 60, 260 Information storage area

Claims

1. Tabular data, in which multiple elements are represented in a table format, is converted into data in a format that allows for the distinction between whether each of the multiple elements is a header or a value. The converted data is input into a large-scale language model, and the large-scale language model is made to perform inference processing on the converted data. An information processing program that causes a computer to perform a process that includes the following.

2. The information processing program according to claim 1, wherein the process of converting the tabular data includes determining the type of format to which the tabular data should be converted, based on at least one of the proportion of natural language elements and the proportion of symbolic representation elements in the plurality of elements.

3. The information processing program according to claim 2, wherein the process for converting the table data includes converting the table data to Markdown format if the proportion of elements of natural language is equal to or greater than a predetermined first threshold, and converting the table data to JSON format if the proportion is less than the first threshold.

4. The information processing program according to claim 3, wherein determining the type of format to which the tabular data is converted includes morphological analysis of the tabular data to assign part-of-speech information to each lexeme contained in the tabular data, converting the tabular data to JSON format if the proportion of lexemes to which predetermined part-of-speech information indicating symbolic representation is assigned is equal to or greater than the first threshold, and converting the tabular data to Markdown format if the proportion of lexemes is less than the first threshold.

5. The word occurrence distributions in the training data used to construct the aforementioned large-scale language model are created for each type of format into which the tabular data is converted. The process for converting the table data includes determining the type of format to which the table data should be converted, based on a comparison of the word occurrence distribution in the table data with the word occurrence distribution for each category. The information processing program according to claim 1.

6. An information processing program according to any one of claims 1 to 5, which causes the computer to perform the action of assigning an element to each of the plurality of elements included in the table data, indicating whether the element is a header or a value.

7. Tabular data, in which multiple elements are represented in a table format, is converted into data in a format that allows for the distinction between whether each of the multiple elements is a header or a value. The converted data is input into a large-scale language model, and the large-scale language model is made to perform inference processing on the converted data. An information processing method in which a computer performs a process that includes the following.

8. A conversion unit converts tabular data, in which multiple elements are represented in a table format, into data in a format that allows for the distinction between whether each of the multiple elements is a header or a value. An execution unit that inputs the converted data into a large-scale language model and causes the large-scale language model to perform inference processing on the converted data, Information processing device including