Image recognition device, and method thereof

Abstract

To accurately perform image recognition.SOLUTION: An image recognition device comprises a plurality of AI image reading devices which read image files, and a large-scale language model having an image reading function for the image files. The image recognition device inputs a plurality of outputs obtained by reading the image files with the plurality of reading devices, the image files, and an output format instruction into the large-scale language model to achieve consistency within the large-scale language model, and outputs an image recognition result of the image files in accordance with the instruction from the large-scale language model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image recognition device and method. [Background technology]

[0002] For example, Non-Patent Document 1 states, "We have described a method for automating OCR of invoices using Google Cloud's DocumentAI, and a method for normalizing OCR data and creating a JSON format by combining ChatGPT." [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Automated Invoice OCR: Outputting unstructured data in JSON using Document AI + ChatGPT API" https: / / recruit.gmo.jp / engineer / jisedai / blog / automated-invoice-ocr-with-chatgpt-and-documentai / Summary of the Invention [Problem to be solved by the invention]

[0004] However, while the system in Non-Patent Document 1 is capable of OCR of simple formats, it is not suitable for OCR of documents with complex formats in which the item names in the document that have the same meaning as the item names to be output are written differently. [Means for solving the problem]

[0005] The system is characterized by comprising a plurality of AI image reading devices that read image files, and a large-scale language model that has the image reading function of the image files, and by inputting a plurality of outputs resulting from reading the image files with the plurality of reading devices, the image files, and output format instructions in the large-scale language model, ensuring consistency within the large-scale language model, and outputting the image recognition results of the image files from the large-scale language model according to the instructions. [Effects of the Invention]

[0006] According to the present invention, image recognition can be performed with high accuracy. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a schematic diagram showing the configuration of an image recognition device 10 according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] (1) Configuration of image recognition device

[0009] 1 is a schematic diagram showing the configuration of an image recognition device 10 according to this embodiment. For convenience, in the following description, it is assumed that the target of image recognition is an image file.

[0010] OCR112 is, for example, an OCR engine using Google Drive's AI. OCR213 is, for example, an OCR engine using Microsoft's (registered trademark) AI. LLM11 with OCR functionality is, for example, GhatGPT4.

[0011] LLM11 is a large language model (LLM) built using large amounts of data and deep learning techniques. Furthermore, LLM11 is equipped with OCR functionality.

[0012] The output format of LLM11 is specified in natural language, for example. The output format is JSON (JavaScript Object Notation), but this JSON format is created within LLN11, not outside LLN11. Needless to say, instructions can be given in JSON format within LLN11 instead of natural language, or in JSON format from outside LLN11.

[0013] (2) Operation of the image recognition device The operation of the image recognition device 10 according to this embodiment will be described with reference to FIG.

[0014] First, there is an image file that has not undergone OCR. The image is read and output using multiple AI-based OCRs 12 and 13. These multiple outputs and the same image file are input into LLM 11, which has OCR, and consistency is achieved using LLM 11.

[0015] At this time, to specify the output format of LLM11, you can instruct LLM11 to output in JSON format by writing in natural language "Please output in JSON format." When giving this instruction, for example, if the image file contains a mixture of Japanese and Western calendar years, you can instruct it to display in the Western calendar year. This instruction can be written in natural language. This allows LLM11 to output the image recognition results in the desired format.

[0016] (3) Effects According to this embodiment, not only simple formats but also complex formats of forms can be accurately recognized. Furthermore, even if the item names in an image file have the same meaning as the item names in the output format but are different, the LLM can estimate the meaning and output the data as instructed in the output format. Furthermore, even when using multiple OCR reading outputs, the LLM was not confused, and the image recognition rate improved to almost 100%.

[0017] <Other embodiments> The present disclosure is not limited to the above-described embodiments as they are. The present disclosure can be embodied by modifying the components within the scope of the gist of the disclosure in the implementation stage. Furthermore, the present disclosure can be formed into various disclosures by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, the components may be appropriately combined in different embodiments. [Explanation of symbols]

[0018] 10 Image recognition device 11 LLM 12 OCR1 13. OCR2

Claims

1. A plurality of AI image reading devices that read image files; a large-scale language model having an image reading function for the image file; Equipped with An image recognition device that inputs multiple outputs resulting from reading the image file using the multiple reading devices, the image file, and output format instructions in the large-scale language model, ensures consistency within the large-scale language model, and outputs the image recognition results of the image file from the large-scale language model according to the instructions.

2. A step of reading the same image file with a plurality of AI image reading devices and outputting the reading results; a step of inputting the plurality of outputs, the image file, and an output format instruction in the large-scale language model into a large-scale language model having an image reading function, and achieving consistency within the large-scale language model; outputting the image recognition results of the image file from the large-scale language model according to the instructions; An image recognition method comprising:

Images