Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program that provide an OCR with a simple configuration.SOLUTION: An information processing device includes: a storage unit 12 that stores, on the basis of a plurality of input vectors, a trained model that outputs a vector indicating a character string predicted on the basis of the plurality of vectors; an acquisition unit 131 that acquires image data including the character string; a division unit 132 that divides the image data into a plurality of divided image data of a predetermined size; a conversion unit 133 that converts each of the plurality of divided image data into a vector to generate a plurality of vectors; and an inference unit 134 that, after each of the plurality of vectors generated by the conversion unit 133 is input into the trained model, outputs a character string corresponding to the vector output by the trained model.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A method for providing OCR (Optical Character Recognition) using a language model is known (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei. Trocr: Transformer-baCsed optical character recognition with pre-trained models. arXiv preprint arXiv:2109.10282, 2021. Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, characters contained in an image are recognized using features related to the meaning of the subjects contained in the image extracted by the encoder, which results in a complex system configuration and a heavy burden on the system in providing OCR.

[0005] The present invention has been made in consideration of these points, and has an object to provide an OCR with a simple configuration. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present invention has a memory unit that stores a trained model that outputs a vector indicating a character string predicted based on a plurality of input vectors, an acquisition unit that acquires image data including a character string, a division unit that generates a plurality of divided image data by dividing the image data into a predetermined size, a conversion unit that generates a plurality of vectors by converting each of the plurality of divided image data into a vector, and an inference unit that inputs each of the plurality of vectors generated by the conversion unit to the trained model and outputs a character string corresponding to the vector output by the trained model.

[0007] Each of the multiple vectors converted from the split image data indicates characteristics of the range in the image data to which the split image data corresponds, and the trained model does not need to accept input of features indicating characteristics related to a subject included in a range spanning multiple split image data.

[0008] Each of the multiple vectors generated by the conversion unit indicates low-level features in each of the divided image data, and the trained model does not need to accept input of high-level features.

[0009] The trained model may output a vector indicating a character string using as input a plurality of vectors generated by the conversion unit and a vector output by the trained model.

[0010] An information processing method of the second aspect of the present invention includes, executed by a computer, an acquisition step of acquiring image data including a character string, a division step of generating a plurality of divided image data by dividing the image data into predetermined sizes, a conversion step of generating a plurality of vectors by converting each of the plurality of divided image data into a vector, and an inference step of inputting each of the generated plurality of vectors into a trained model that outputs a vector indicating a character string predicted based on the plurality of vectors based on the input plurality of vectors, and outputting a character string corresponding to the vector output by the trained model.

[0011] In a program of the third aspect of the present invention, a computer is caused to execute an acquisition step of acquiring image data including a character string, a division step of generating a plurality of divided image data by dividing the image data into predetermined sizes, a conversion step of generating a plurality of vectors by converting each of the plurality of divided image data into a vector, and an inference step of inputting each of the generated plurality of vectors into a trained model that outputs a vector indicating a character string predicted based on the plurality of vectors based on the input plurality of vectors, and outputting a character string corresponding to the vector output by the trained model. Effect of the Invention

[0012] According to the present invention, it is possible to provide an OCR function with a simple configuration. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram for explaining an overview of an information processing system S according to an embodiment. [Diagram 2] 1 is a diagram for explaining an overview of an information processing system S according to an embodiment. [Diagram 3] 1 is a block diagram showing a configuration of an information processing device 1. FIG. [Figure 4] 3 is a flowchart showing a process flow in the information processing device 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] [Outline of Information Processing System S] 1 and 2 are diagrams for explaining an overview of an information processing system S according to an embodiment. The information processing system S provides OCR that outputs character strings included in image data. The information processing system S includes an information processing device 1 and an information terminal 2.

[0015] The information processing device 1 is a device that provides OCR. The information processing device 1 stores a trained model M. The trained model M is a machine learning model for predicting a character string included in image data. Based on a plurality of input vectors, the trained model M outputs a vector indicating a character string predicted based on the plurality of vectors. The trained model M is based on a Transformer and is composed of only a decoder.

[0016] The information terminal 2 is a terminal used by a user of the information processing system S. The information terminal 2 is, for example, a tablet, a smartphone, or a personal computer. As an example, the information terminal 2 transmits an instruction to recognize a character string to the information processing device 1, acquires the character string recognized by the information processing device 1, and displays it on a display unit.

[0017] The process in the information processing system S will be described with reference to Fig. 2. The information processing device 1 acquires image data D1. A character string to be recognized by the information processing system S is captured in the image data D1.

[0018] The information processing device 1 generates divided image data D2, which is an image obtained by dividing the acquired image data D1 into images of a predetermined size. The information processing device 1 converts each of the divided image data V2 into a plurality of vectors V. As an example, the predetermined size is determined based on the number of dimensions of the input layer of the trained model M.

[0019] Each of the multiple vectors V indicates features in the range of the divided image data D2 corresponding to the target image data D2 from which the vector is generated in the image data D1. Also, each of the multiple vectors V indicates low-level features in each of the divided image data D2. Low-level features indicate local features in the partial image data D2. The vector V, which is a low-level feature, indicates features such as the direction of lines and color in the divided image data from which the vector V is generated. In contrast, high-level features indicate features such as, for example, the subject reflected in the image or the meaning of a character string. High-level features are typically features related to a subject reflected in a range spanning multiple divided image data D2.

[0020] The information processing device 1 adds data indicating a data separator ([SEP] in FIG. 2) to the end of the input vector V. The information processing device 1 inputs a plurality of vectors V to the trained model M, and causes the trained model M to output a character string included in the image data. The trained model M outputs a vector indicating a character string using the input plurality of vectors V and the vector output by the trained model M as input. More specifically, the trained model M outputs a vector indicating a character string by making an autoregressive prediction using the input plurality of vectors V and the vector output by the trained model M as input.

[0021] Specifically, (a) the information processing device 1 adds vector V2 output by the trained model M to vector V1 to generate input data. (b) The information processing device 1 inputs the generated input data again to the trained model M. The information processing device 1 repeats the above processes (a) and (b) until the trained model M outputs a vector indicating the end of inference ([EOS] in FIG. 2).

[0022] In the example of FIG. 2, in the first input, the trained model M receives vector V1 as input and outputs vector V2 corresponding to the character "B." In the second input, the trained model M receives a vector group in which vector V2 corresponding to the character "B" is added to vector V1, and outputs a vector corresponding to the character "Tai." The trained model M repeats this process until a character string corresponding to [EOS] is output.

[0023] When the trained model M outputs a vector indicating the end of inference, the information processing device 1 outputs the character string output by the trained model M to the information terminal 2.

[0024] It should be noted that the trained model M does not accept input of features (high-level features) indicating features in an area spanning multiple divided image data. In other words, the trained model M does not have an input layer for inputting high-level features. As described above, the trained model M outputs based on the vector V indicating low-level features, and does not accept input of high-level features. Therefore, the information processing device 1 does not require an encoder for generating high-level features, and it becomes possible to output character strings included in an image using only a decoder, making it possible to provide OCR with a simple configuration.

[0025] [Configuration of information processing device 1] 3 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has an acquisition unit 131, a division unit 132, a conversion unit 133, and an inference unit .

[0026] The communication unit 11 is a communication interface for transmitting and receiving data to and from other devices via a network. The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), a hard disk drive, etc. The storage unit 12 stores in advance a program to be executed by the control unit 13.

[0027] The storage unit 12 stores the trained model. The trained model M is generated by training the pre-trained model based on teacher data that associates a plurality of vectors generated from the divided image data with the correct answers of character strings corresponding to the plurality of vectors. The pre-trained model is a machine learning model that acquires language knowledge by pre-training a natural language processing task. By configuring the trained model M in this way, it becomes possible for the information processing device 1 to output character strings captured in image data based on low-level feature amounts extracted from the divided image data without using high-level feature amounts.

[0028] The image data on which the trained model M learns may include various types of character strings, such as handwritten characters, printed character strings, character strings included in a scenic image, etc. The trained model M may learn vectors generated based on image data including horizontally written character strings, or may learn vectors generated based on image data including vertically written character strings.

[0029] The control unit 13 is a processor such as a CPU (Central Processing Unit), etc. The control unit 13 executes a program stored in the storage unit 12 to function as an acquisition unit 131, a division unit 132, a conversion unit 133, and an inference unit 134.

[0030] The acquisition unit 131 acquires image data including a character string. The acquisition unit 131 may acquire the image data D1 from the information terminal 2, or may acquire the image data D1 from an external device.

[0031] The division unit 132 divides the image data D1 into a plurality of divided image data D2 of a predetermined size. The predetermined size is equal to the number of dimensions of the input layer of the trained model M, for example.

[0032] The conversion unit 133 converts each of the multiple divided image data D2 into a vector to generate multiple vectors V. The conversion unit 133 converts each of the multiple divided image data D2 into one-dimensional sequence data and linearly projects the data to convert each of the multiple divided image data D2 into a vector.

[0033] The number of dimensions of the vector V is equal to the number of dimensions of the input layer of the trained model M. The conversion unit 133 may add to the vector V position information indicating in which part of the entire image the generated vector V is located.

[0034] The inference unit 134 inputs each of the multiple vectors generated by the conversion unit 133 to the trained model M, and outputs a character string corresponding to the vector output by the trained model M. The inference unit 134 may display the character string output by the trained model M on the information terminal 2, or may output it to an external device (not shown).

[0035] [Processing flow in information processing device 1] 4 is a flowchart showing the flow of processing in the information processing device 1. The flowchart shown in FIG.

[0036] The acquiring unit 131 acquires image data D1 (S01). The dividing unit 132 divides the image data D1 into pieces of a predetermined size to generate divided image data D2 (S02).

[0037] The conversion unit 133 generates a vector V based on each of the multiple divided image data D2 (S03). The inference unit 134 inputs the vector V to the trained model M, and causes it to output a character string (S04). Then, the information processing device 1 ends the process.

[0038] [Effects of the information processing device 1] By configuring the information processing device 1 as described above, it is possible to provide an effect of providing OCR with a simple configuration without using high-level features.

[0039] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0040] 1. Information processing device 2. Information terminal 11 Communications Department 12 Storage section 13 Control section 131 Acquisition Department 132 Division 133 Conversion Unit 134 Reasoning part

Claims

1. A storage unit that stores a trained model that outputs a vector indicating a character string predicted based on a plurality of input vectors; An acquisition unit that acquires image data including a character string; A division unit that divides the image data into a plurality of divided image data of a predetermined size; A conversion unit that converts each of the plurality of divided image data into a vector to generate a plurality of vectors; an inference unit that inputs each of the multiple vectors generated by the conversion unit into the trained model and outputs a character string corresponding to the vector output by the trained model; An information processing device having the above configuration.

2. Each of the plurality of vectors converted from the divided image data indicates a characteristic of a range in the image data to which the divided image data corresponds, The trained model does not accept input of a feature amount indicating a feature related to a subject included in a range spanning multiple divided image data, The information processing device according to claim 1 .

3. each of the plurality of vectors generated by the conversion unit indicates a low-level feature amount in each of the divided image data; The trained model does not accept input of high-level features.

3. The information processing device according to claim 1 or 2.

4. The trained model receives as input a plurality of vectors generated by the conversion unit and a vector output by the trained model, and outputs a vector indicating a character string. The information processing device according to claim 1 .

5. The computer executes An acquisition step of acquiring image data including a character string; A division step of dividing the image data into a plurality of divided image data of a predetermined size; A conversion step of generating a plurality of vectors by converting each of the plurality of divided image data into a vector; an inference step of inputting each of the generated vectors to a trained model that outputs a vector indicating a character string predicted based on the input vectors, and outputting a character string corresponding to the vector output by the trained model; An information processing method comprising the steps of:

6. On the computer, An acquisition step of acquiring image data including a character string; A division step of dividing the image data into a plurality of divided image data of a predetermined size; A conversion step of generating a plurality of vectors by converting each of the plurality of divided image data into a vector; an inference step of inputting each of the generated vectors to a trained model that outputs a vector indicating a character string predicted based on the input vectors, and outputting a character string corresponding to the vector output by the trained model; A program that executes the following.

Images