Information processing device, information processing method, and information processing program

Abstract

The present invention provides an information processing device, method, and program that provide instructions to a trained AI model, including information about the form of the text such as the color and font of the characters, and perform extraction that reflects the form of the text. [Solution] The information processing device includes: a text data acquisition unit that acquires text data from file data; a form information acquisition unit that acquires form information, which is information about the shape of characters, from file data; a form information assignment unit that assigns the acquired form information to the acquired text data; a prompt receiving unit that receives prompts that instruct a trained AI model to extract information to be extracted from file data, and prompts that include instructions regarding form information; and an extraction unit that inputs the received prompts and the text data to which form information has been assigned to the trained AI model and extracts the information to be extracted as instructed from the text data.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus that extracts desired information from file data.

Background Art

[0002] For example, file data such as documents and drawings used in the construction, design, civil engineering, and plant industries contains various types of information and has various formats. For example, there are architectural drawings, specifications, construction management histories, design outlines, elevation views, detailed drawings, etc. For example, the number of data items obtained from a single architectural design drawing data may be several hundred or more. These file data are not neatly organized in a form like "columns and rows" or "JSON format", but rather are so-called unstructured file data and are not suitable for search, statistical processing, use in AI (artificial intelligence) learning data, etc. Therefore, there is a demand to structure these unstructured file data in a form like a spreadsheet.

[0003] In order to structure the unstructured file data as described above, it is necessary to extract individual information contained in the unstructured file data for each data item. As a technique for extracting such information, for example, there is the technique described in Patent Document 1. Specifically, it extracts predetermined information from image data generated by reading a paper-based voucher by OCR (optical character recognition) based on the position information on the paper surface defined for each item.

[0004] On the other hand, in recent years, the development of techniques for information extraction using pre-trained AI models represented by LLM (large language model) has also progressed. For example, there is also a technique for reading a bill, converting it into text data, and automatically extracting information ("bill amount", "bill issuer", etc.) necessary for accounting journal processing (Non-Patent Document 1).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] Figure 6 is a conceptual diagram illustrating an example of converting PDF file data into text data using OCR. Figure 6(a) shows a portion of a PDF file containing a building area table for houses. Figure 6(b) shows an example of converting the text for "shape," "calculation formula," and "area" described in the file in Figure 6(a) into JSON format, along with their location information. Similarly, Figure 6(c) shows an example of converting the text into Array format, also with location information. In this way, text data generated by OCR is represented by the content of the characters and their location information.

[0007] By providing the text data generated by OCR in this way to the trained AI model disclosed in Non-Patent Document 1 along with prompts, it is possible to obtain the information indicated by the prompts.

[0008] However, because the provided text data is limited to the content and positional information of the characters, it is not possible to provide instructions that include information about the form of the text, such as the color and font of the characters, and therefore it is not possible to perform extraction that reflects the form of the text. [Means for solving the problem]

[0009] Therefore, in order to solve the above problems, the present invention provides the following information processing device, etc. Specifically, an information processing device is provided which includes: a text data acquisition unit that acquires text data from file data; a form information acquisition unit that acquires form information, which is information about the form of characters, from the file data; a form information assignment unit that assigns the acquired form information to the acquired text data; a prompt receiving unit that receives a prompt that instructs a trained AI model to extract information to be extracted from file data, the prompt including an instruction regarding the form information; and an extraction unit that inputs the received prompt and the text data to which the form information has been assigned to the trained AI model and extracts the instructed information to be extracted from the text data.

[0010] Furthermore, in addition to the above features, the present invention provides an information processing device that further includes a text data acquisition unit which acquires text data from each of a plurality of file data, a form information assignment unit which assigns the form information to each of the acquired text data, and a storage unit which stores each of the text data to which the form information has been assigned.

[0011] In addition to the features described above, the extraction unit provides an information processing device that extracts the specified extraction target information from each text data to which the morphological information is attached, which is stored in the storage unit.

[0012] In addition to the features described above, the extraction unit provides an information processing device that extracts information to be extracted by a plurality of extraction processes based on the instructions of the received prompt.

[0013] Furthermore, the present invention provides an information processing method executed by an information processing device, comprising: a text data acquisition step of acquiring text data from file data; a form information acquisition step of acquiring form information, which is information about the form of characters, from the file data; a form information assignment step of assigning the acquired form information to the acquired text data; a prompt reception step of receiving a prompt that instructs a trained AI model to extract information to be extracted from file data, the prompt including an instruction regarding the form information; and an extraction step of inputting the received prompt and the text data to which the form information has been assigned to the trained AI model, and extracting the instructed information to be extracted from the text data.

[0014] The present invention also provides an information processing program that causes an information processing device to execute the following steps: a text data acquisition step of acquiring text data from file data; a form information acquisition step of acquiring form information, which is information about the form of characters, from the file data; a form information assignment step of assigning the acquired form information to the acquired text data; a prompt reception step of receiving a prompt that instructs a trained AI model to extract information to be extracted from file data, the prompt including an instruction regarding the form information; and an extraction step of inputting the received prompt and the text data to which the form information has been assigned to the trained AI model, and extracting the instructed information to be extracted from the text data. [Effects of the Invention]

[0015] The present invention provides an information processing device that can provide instructions to a trained AI model that include information about the form of text, such as the color and font of the characters, and perform extraction that reflects the form of the text. [Brief explanation of the drawing]

[0016] [Figure 1] Block diagram showing an example of the functional configuration of the information processing device in the embodiment. [Figure 2]Conceptual diagram showing an example of a mode of attaching acquired morphological information to text data [Figure 3] Conceptual diagram showing another mode of morphological information attachment [Figure 4] Conceptual diagram showing a configuration example of hardware for realizing the information processing apparatus of the embodiment [Figure 5] Flowchart briefly showing an example of the processing flow of the information processing apparatus of the embodiment [Figure 6] Conceptual diagram showing an example of converting PDF format file data into text data by OCR

Mode for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that the present invention should not be limited to these embodiments, and can be implemented in various modes without departing from the gist thereof.

[0018] <Example> <Overview> The present invention can give a prompt including an instruction regarding the form of characters to a learned AI model by attaching information regarding the form of the characters to the text data acquired from the file data, and perform a more detailed extraction in which the form of the characters is reflected.

[0019] Hereinafter, the functions and processing flow of the information processing apparatus, as well as the details of the hardware, will be described. Note that the functional blocks of the present information processing apparatus described below can be realized as a combination of hardware and software. Specifically, if a computer is used, it includes hardware components such as a CPU (Central Processing Unit), main memory, bus, or storage (hard disk drive, non-volatile memory, storage media such as CDs and DVDs, and reading drives for those media), input devices used for information input, other external peripheral devices, interfaces for those external peripheral devices, communication interfaces, driver programs for controlling those hardware, other application programs, and application programs for the user interface. Then, through the arithmetic processing of the CPU according to the program developed on the main memory, data input from input devices and other interfaces and held in the memory or hard disk is processed and stored, or commands for controlling the above-mentioned various hardware and software are generated. Alternatively, the functional blocks of the present information processing apparatus may be realized by dedicated hardware.

[0020] Furthermore, this invention can be realized not only as an apparatus or system but also as a method. Also, a part of such an invention can be configured as software. Additionally, a program used to cause a computer to execute such software, and a recording medium on which the program is fixed are naturally included in the technical scope of this invention.

[0021] <Functional Configuration> FIG. 1 is a block diagram showing an example of the functional configuration of the information processing apparatus of this embodiment. As shown in FIG. 1, the information processing apparatus 100 includes a text data acquisition unit 101, a morphological information acquisition unit 102, a morphological information addition unit 103, a storage unit 104, a prompt reception unit 105, and an extraction unit 106.

[0022] In this embodiment, the trained AI model is a machine learning model that has been pre-trained and is capable of handling general-purpose tasks. A well-known example is the Large-Scale Language Model (LLM), but it may also be a small-scale language model or a multimodal language model that can handle images or other formats.

[0023] <Text data acquisition section> The text data acquisition unit 101 has the function of acquiring text data from file data. There are many types and formats of file data and it is not limited to these, but examples include Word files, PDF (Portable Document Format) files, Excel files, image files, CSV (Comma Separated Values) files, text files, BIM (Building Information Modeling) files, 2DCAD (Computer Aided Design) files, and 3D model files. Text data is acquired from these file data using character recognition means such as OCR.

[0024] <Form information acquisition section> The morphological information acquisition unit 102 has the function of acquiring morphological information, which is information about the shape of characters, from the file data. Morphological information is information about the external shape and appearance of characters, such as the color, shape, and style of the characters, and includes information such as character color, character font, font size, line spacing size, number of lines, whether or not it is handwritten, whether or not it is underlined, and additional symbols. Additional symbols are symbols attached to characters, such as symbols attached to the beginning or end of characters, or characters enclosed in symbols such as triangles or squares.

[0025] Morphological information can be acquired in some cases, such as by OCR, including for character color. It can also be acquired using a pre-trained language model that has learned about character shapes. Furthermore, morphological information can be acquired by accepting input from the user.

[0026] <Form Information Assignment Unit> The morphological information assignment unit 103 has the function of assigning the acquired morphological information to the acquired text data. Figure 2 is a conceptual diagram showing an example of how the acquired morphological information is assigned to the text data.

[0027] Figure 2(a) is a file data file from which text data and morphological information are acquired, and is a PDF file data showing a kitchen floor plan. As shown in Figure 2(a), the kitchen floor plan has handwritten text 301 that says "Install 3 electrical outlets!" and text 302 that says "Do we need 2 uplights?", and the text data and morphological information for each are acquired.

[0028] Figure 2(b) shows an example of text data and form information obtained from the kitchen floor plan in Figure 2(a). As shown in the figure, the text data "Install 3 electrical outlets!" and its location information are obtained, and the form information "tegaki:True" is obtained to indicate whether it is handwritten or not. Form information is also obtained for "Do you need 2 uplights?". Furthermore, if these handwritten texts are in red, form information related to the text color, such as "color:"red"", is also obtained.

[0029] Figure 3 is a conceptual diagram showing another method of assigning morphological information. Figure 3(a) is a table showing the part number, length, and classification of a part. Here, the items "Inventory Classification," "Part Number," and "Length" are set. One part has an inventory classification of "2" enclosed in a diamond, a part number of "PG5125K," and a length of "10.02." The other part has an inventory classification of "4" enclosed in a diamond, a part number of "PH2040N," and a length of "14.58."

[0030] Figure 3(b) shows an example of adding morphological information obtained from the table in Figure 3(a) to text data. As shown in Figure 3(b), the inventory category value obtained along with the location information is indicated as "val:“2”," and its morphological information is indicated as "symbol:“Diamond”,". The other part is also indicated as "val:“4”,symbol:“Diamond”,".

[0031] The above-mentioned morphological information can be obtained using OCR or AI-OCR (optical character recognition utilizing AI technology) equipped with the function to acquire morphological information as described above. It can also be obtained by inputting it into a pre-trained AI model, such as a large-scale language model. Furthermore, it can be configured to acquire morphological information by accepting input from the user.

[0032] <Storage Section> The storage unit 104 has the function of storing each text data to which the morphological information has been attached. The text data acquisition unit acquires text data from multiple file data, the morphological information acquisition unit acquires morphological information from the acquired text data, and the morphological information attachment unit attaches the acquired morphological information to the text data. By acquiring and storing text data to which morphological information has been attached from multiple file data in this way, it is possible to perform processing such as information extraction and searching on the stored text data. It is also preferable to store the text data and morphological information in association with the original file data from which they were acquired.

[0033] <Prompt Reception Department> The prompt receiving unit 105 has the function of receiving prompts that instruct the trained AI model to extract information from file data, and which include instructions regarding morphological information.

[0034] A prompt might be something like, "Extract all comments about kitchen equipment from this drawing, written in red and handwritten text." The example prompt includes instructions about form information, such as "in red text" and "in handwritten text." Similarly, a prompt like, "Extract text with diamond-shaped symbols from this drawing," includes an instruction about form information, "text with diamond-shaped symbols."

[0035] <Extraction part> The extraction unit 106 has the function of inputting the received prompt and the text data to which the morphological information is attached into a trained AI model, and extracting the information to be extracted as instructed from the text data.

[0036] The text data may be stored in the storage unit described above, or it may be stored in external storage connected to this information processing device via a communication line or the like. When extracting text data stored in external storage, it is preferable that text information and morphological information are obtained from the file data stored in this storage.

[0037] Furthermore, the extraction unit can be configured to extract the specified information through multiple extraction processes based on the instructions of the received prompt. For example, if the prompt "Extract all handwritten comments in red from this drawing regarding kitchen equipment" is received, the unit first performs a process to extract the red text. Then, from the extracted results, it further extracts the handwritten text and from that, extracts the "comments regarding kitchen equipment." Also, if the instructions of the prompt are divided into multiple processes, the unit may be configured to output the results of each process, such as by displaying them.

[0038] <Hardware Configuration> Figure 4 is a conceptual diagram showing an example of the hardware configuration for realizing the information processing device according to the embodiment. As shown in the figure, the information processing device 400 includes a CPU 401 that performs various calculations, a RAM 402 which is a volatile recording medium, a storage device 403 such as a flash memory or HDD which is a non-volatile storage medium, a communication interface 404, and an input / output interface 405. The RAM 402 reads programs that perform various calculations for the CPU 401 to execute and provides a work area (work area) for those programs. In addition, multiple addresses are assigned to the RAM 402, and programs executed by the CPU 401 can exchange data with each other and perform processing by identifying and accessing these addresses (this is the same throughout this specification).

[0039] Here, the functions of the information processing device 100 in Figure 1, name data acquisition unit 101, name data acquisition unit 102, name data assignment unit 103, prompt reception unit 105, and extraction unit 106, are mainly realized by the CPU 401 and RAM 402 in Figure 4. The functions of the storage unit 104 are mainly realized by the storage 403 in Figure 4. Furthermore, when the name data acquisition unit 102, extraction unit 106, etc., use an externally existing trained AI model, each function is realized by exchanging signals and information with each other via the communication interface 404 and input / output interface 405.

[0040] <Processing flow> Figure 5 is a simplified flowchart showing an example of the processing flow of the information processing device of Embodiment 1. First, text data is acquired from file data (S501: Text data acquisition step). Then, morphological information, which is information about the shape of characters, is acquired from the file data (S502: Morphological information acquisition step). Then, the acquired morphological information is added to the acquired text data (S503: Morphological information addition step). Then, a prompt is received that instructs the trained AI model to extract information to be extracted from the file data, and the prompt includes an instruction regarding the morphological information (S504: Prompt reception step). Then, the received prompt and the text data to which the morphological information has been added are input to the trained AI model, and the information to be extracted as instructed is extracted from the text data (S505: Extraction step).

[0041] <Effects> According to this embodiment, the information processing device can provide a trained AI model with instructions that include information about the form of the text, such as the color and font of the characters, and perform extraction that reflects the form of the text. [Explanation of Symbols]

[0042] 100, 400: Information Processing Devices 101: Text data acquisition unit 102: Shape information acquisition unit 103: Morphological information assignment unit 104: Storage Unit 105: Prompt Reception Department 106:Extraction part 401:CPU 402:RAM 403: Storage 404: Communication Interface 405: Input / Output Interface

Claims

1. A text data acquisition unit that acquires text data from file data, A form information acquisition unit that acquires form information, which is information about the shape of characters, from the aforementioned file data, A form information assignment unit that assigns the acquired form information to the acquired text data, A prompt that instructs a trained AI model to extract information from file data, the prompt receiving unit receiving the prompt which includes instructions regarding the morphological information, An extraction unit inputs the received prompt and the text data to which the morphological information is attached to a trained AI model, and extracts the information to be extracted as instructed from the text data. An information processing device having

2. The text data acquisition unit acquires text data from each of the multiple file data, The morphological information assignment unit assigns the morphological information to each of the acquired text data, The information processing apparatus according to claim 1, further comprising a storage unit for storing each of the text data to which the aforementioned morphological information has been assigned.

3. The information processing apparatus according to claim 2, wherein the extraction unit extracts the information to be extracted as instructed from each text data to which the morphological information is attached and stored in the storage unit.

4. The information processing apparatus according to claim 2, wherein the extraction unit extracts the information to be extracted by a plurality of extraction processes based on the instructions of the received prompt.

5. An information processing method performed by an information processing device, A text data acquisition step that obtains text data from file data, A step of acquiring form information, which is information about the shape of characters, from the aforementioned file data, A step of adding shape information to the acquired text data, A prompt receiving step that receives a prompt that instructs a trained AI model to extract information from file data, the prompt including an instruction regarding morphological information, An extraction step in which the received prompt and the text data to which the morphological information is attached are input to a trained AI model to extract the information to be extracted as instructed from the text data, An information processing method having

6. A text data acquisition step that obtains text data from file data, A step of acquiring form information, which is information about the shape of characters, from the aforementioned file data, A step of adding shape information to the acquired text data, A prompt receiving step that receives a prompt that instructs a trained AI model to extract information from file data, the prompt including an instruction regarding morphological information, An information processing program that causes an information processing device to perform an extraction step, which involves inputting the received prompt and the text data to which the morphological information is attached into a trained AI model and extracting the specified information to be extracted from the text data.

Images