Learning device, method, program, and inference device

Abstract

To improve the inference accuracy of a model.SOLUTION: A learning device according to an embodiment includes an acquisition unit, a conversion unit, and a training unit. The acquisition unit acquires a document to which a tag is added. The conversion unit converts the document into an image to generate a document image, and converts the tag into an image according to composition of the document image to generate a tag image. The training unit trains a network model using the document image as input data and the tag image as correct answer data to generate a learned model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a learning device, a method, a program, and an inference device. [Background technology]

[0002] With the recent development of deep learning, a certain level of inference accuracy can be achieved even when natural language processing using machine learning is performed on a character string basis rather than on a word basis. Much research has been conducted on image recognition using machine learning. When characters are visualized, the sequence of 0s or 1s in the pixels that represent the characters can be considered the ID of the character, and since characters can be uniquely identified, it is anticipated that machine learning can be applied to imaged characters. In machine learning, preparing a large amount of training data leads to improved accuracy, so it is important to increase the amount of training data through data augmentation.However, unlike general images, text requires handling of discrete values ​​such as characters or words.As a result, data augmentation cannot be performed by simple image rotation or flipping, as is the case with general images, making it difficult to prepare a large amount of training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6838209 Specification Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a learning device, a method, a program, and an inference device that can improve the inference accuracy of a model. [Means for solving the problem]

[0005] The learning device according to this embodiment includes an acquisition unit, a conversion unit, and a training unit. The acquisition unit acquires a document to which tags have been added. The conversion unit converts the document into an image to generate a document image, and converts the tags into images according to the typesetting of the document image to generate a tag image. The training unit trains a network model using the document image as input data and the tag image as correct answer data to generate a trained model. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing a learning device according to a first embodiment. [Figure 2] 4 is a flowchart showing an example of the operation of the learning device according to the first embodiment. [Figure 3] FIG. 4 is a diagram showing an example of tag data according to the embodiment. [Figure 4] FIG. 2 is a diagram showing a first example of a document image and a tag image. [Figure 5] FIG. 10 is a diagram showing a second example of a document image and a tag image. [Figure 6] FIG. 10 is a diagram showing a third example of a document image and a tag image. [Figure 7] FIG. 10 is a diagram showing a fourth example of a document image and a tag image. [Figure 8] FIG. 1 illustrates an example of data augmentation of a document image. [Figure 9] FIG. 10 is a diagram showing an example of padding in convolution processing. [Figure 10] FIG. 10 is a block diagram showing an inference device according to a second embodiment. [Figure 11] 10 is a flowchart showing an example of the operation of the inference device according to the second embodiment. [Figure 12] FIG. 2 is a diagram showing an example of the hardware configuration of a learning device and an inference device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] The learning device, method, program, and inference device according to the present embodiment will be described in detail below with reference to the drawings. Note that in the following embodiments, parts with the same reference numerals perform similar operations, and redundant explanations will be omitted as appropriate.

[0008] (First embodiment) A learning device according to a first embodiment will be described with reference to the block diagram of FIG. The learning device 10 according to the first embodiment includes an acquisition unit 101, a calculation unit 102, a selection unit 103, a determination unit 104, a conversion unit 105, a training unit 106, and a storage unit 107.

[0009] The acquisition unit 101 acquires multiple tagged documents, which are documents to which tags have been assigned. The documents are assumed to be text data consisting of one or more character lines. Tags are metadata that indicate, for a document, for example, the range of words that make up named entities, the types of parts of speech, causal relationships, etc. Labels determined in other sequence labeling problems can also be applied as tags. Hereinafter, multiple tagged documents will also be referred to as a tagged document group. The calculation unit 102 calculates the maximum number of characters and the maximum number of lines in a tagged document included in a tagged document group. The selection unit 103 selects a font for the document. The determination unit 104 determines the typesetting within the range of the maximum number of characters and the maximum number of lines for each document.

[0010] The conversion unit 105 converts the document into an image to generate a document image, and converts the tags into images in accordance with the typesetting of the document image to generate a tag image. The training unit 106 trains the network model using the document image as input data and the tag image as correct answer data, and generates a trained model. The storage unit 107 stores tagged documents, trained models, and the like.

[0011] Next, an example of the operation of the learning device 10 according to the first embodiment will be described with reference to the flowchart of FIG.

[0012] In step S201, the acquiring unit 101 acquires a tagged document group including a plurality of tagged documents. In step S202, the calculation unit 102 calculates the maximum number of characters NX and the maximum number of lines NY in one document from the tagged document group. The calculation unit 102 may determine the maximum number of characters NX and the maximum number of lines NY in advance. In step S203, the selection unit 103 selects a font for the document. In this embodiment, it is assumed that the font is a monospaced font with a fixed character size, but a proportional font may also be used.

[0013] In step S204, the determination unit 104 determines the typesetting of the document. For example, if the width of one character of the font selected in step S203 is FX [dots] and the height is FY [dots], the width of the document image can be specified as "NX x FX" and the height of the document image as "NY x FY". In step S205, the conversion unit 105 converts the typeset document into an image to generate a document image. Simply, the image may be the brightness distribution displayed when a text file is opened in word processing software such as WORD, or, if processing is performed in-house, font images may be arranged character by character on a plain image according to which character and line each character in the text corresponds to.

[0014] In step S206, the conversion unit 105 converts the tag data assigned to the tagged document into an image to generate a tag image. Specifically, the conversion unit 105 generates a tag image with a number of channels corresponding to the number of tag types. The tag image corresponds to the same typesetting as the document image, and has the same size as the document image. The conversion unit 105 generates image data in which an area on the tag image that corresponds to the character-size area of ​​the character string to which the tag is assigned in the document image is set to "on." "On" indicates, for example, that brightness is set to "0" or "255," i.e., black or white, and that the probability is set to "1.0" for a neural network. In other words, a tag image is mask data in which the corresponding areas are filled in according to the type of tag, such as a proper noun.

[0015] In step S207, the training unit 106 trains the network model using the document image as input data and the tag image as ground truth data. In this embodiment, since semantic segmentation processing is assumed, a neural network structure such as SegNet, U-Net, PSPNet, or RefineNet, which includes a convolutional neural network (CNN) including a convolutional layer, may be used as the network model. Note that the network model is not limited to the above-mentioned neural network structure, and any network model may be used as long as it is used in semantic segmentation processing.

[0016] In step S208, the training unit 106 determines whether training of the network model has ended. Whether training has ended may be determined, for example, by determining that training has ended when the loss value of the loss function of the network model is equal to or less than a threshold. Alternatively, training may be determined to have ended when the decrease in the loss value has converged. Furthermore, training may be determined to have ended when training for a predetermined number of epochs has ended. If training has ended, a trained model is generated, and the process proceeds to step S209. On the other hand, if training has not ended, the process proceeds to step S210.

[0017] In step S209, the storage unit 107 stores the generated trained model. In step S210, the training unit 106 updates the parameters (weighting coefficients, biases, etc.) of the network model so that the loss value of the loss function is minimized, and the process returns to step S207 to repeat the same processing.

[0018] Next, an example of tag data according to this embodiment will be described with reference to FIG. FIG. 3 shows an example in which tags are assigned to a range of words that make up a named entity in a document. The top row of Figure 3 shows the text string of the document, and the bottom row shows the tags assigned to each word in the text string in the top row. Here, the named entity tags include a person's name (PSN), a date (DAT), and a time (TIM), and the first word that makes up the named entity is assigned a "B-", subsequent words are assigned an "I-", and words that are not named entities are assigned an "O".

[0019] Specifically, "Taro" is a person's name, so it is tagged with "B-PSN." Also, since "May 18th" is a date, "5" is tagged with "B-DAT," the following "month" with "I-DAT," the following "18" with "I-DAT," and the following "day" with "I-DAT." On the other hand, the particle "no" is not a named entity, so it is tagged with "O."

[0020] Next, a first example of a document image and a tag image will be described with reference to FIG. In Figure 4, we consider a tagged document in which a causal relationship tag has been added to the document "Water leaked from the pipe, causing rust to form on downstream parts." Here, we assume that the causal relationship tags associated with the document are "Water leaked from the pipe" and "Rust occurred."

[0021] The input data for the network model 40 is a document image 41 obtained by converting the document "Water leaked from the pipes, causing rust to form on downstream components." into an image. The correct answer data for the network model 40 is a tag image 42 in which the brightness value of character-sized regions in the character strings "Water leaked from the pipes" and "Rust occurred" corresponding to the causal relationship tag is set to zero (black in this case), and other regions are expressed with the maximum brightness value (white in this case). In the following explanation, setting a certain region in the tag image to a brightness value of zero is also referred to as "masking."

[0022] By training using the document image 41 and the tag image 42, the network model 40 performs semantic segmentation processing on the document image 41, and learns whether or not each pixel in the document image 41 is a tagged area. In other words, when the semantic segmentation processing is performed on a character-sized area of ​​one character, the area is segmented according to the shape of the character (or character string). Therefore, the network model 40 can learn what tag should be assigned to the shape of the character string without interpreting the meaning of the character.

[0023] Next, a second example of a document image and a tag image will be described with reference to FIG. In Figure 5, as correct answer data for document image 41, we provide tag image 51, which masks areas that have been tagged with causal relationships; tag image 52, which masks areas of the document image where text strings exist that have not been tagged with causal relationships; and tag image 53, which masks areas of document image 41 where no text exists. In this way, by training the network model 40 using the three types of tag images 51 to 53 as correct answer data, more robust inference performance can be obtained.

[0024] Next, a third example of a document image and a tag image will be described with reference to FIG. FIG. 6 shows an example of generating tag images according to tag types as correct answer data for a document image 41. Here, tag images according to parts of speech are assumed. Specifically, a tag image 61 in which nouns present in the document included in the document image 41 are masked, a tag image 62 in which verbs are masked, and a tag image 63 in which adjectives are masked are prepared. Since the document image 41 does not contain any adjectives, the tag images do not contain any masked areas. For example, in recurrent neural networks (RNNs), padding and masking are often used to adjust the number of characters per sentence. Therefore, in this embodiment, in order to explicitly handle padded areas to adjust the number of characters in a document, the conversion unit 105 may generate a padding channel, i.e., a tag image indicating the padded area, and the training unit 106 may use the tag image as correct answer data. In this way, by generating and learning correct answer data for multiple different tags, inference results can be obtained for each type of tag during inference.

[0025] Next, a fourth example of a document image and a tag image will be described with reference to FIG. 7 shows an example of training data assuming a causal relationship extraction task, in which in addition to a document image 41, a tag image 71 indicating the range of cause events is input as teaching data to the network model 40. Meanwhile, a tag image 72 indicating the range of result events is input as correct answer data to the network model 40. This makes it possible to generate a trained model 45 capable of inferring causal relationships.

[0026] Next, an example of data augmentation of a document image will be described with reference to FIG. As an example of data augmentation for a document image, the determination unit 104 may determine multiple typesettings with different character spacing and / or different line spacing. FIG. 8 shows document image 81 in which the spacing between characters is increased compared to document image 41, which has a predetermined character spacing. In other words, the center-to-center distance between characters does not need to be the same as the font size, and characters are arranged with a certain margin M. That is, the determination unit 104 defines the width of the document image as "NX × (FX + M)" and the height of the document image as "NY × (FY + M)," and the conversion unit 105 converts the document into an image, thereby generating document image 81. On the other hand, the determination unit 104 may allow characters to overlap each other, and generates document image 82 by defining the width of the document image as "NX x (FX-M)" and the height of the document image as "NY x (FY-M)".

[0027] The determining unit 104 may also determine multiple typesettings with different margins. The number of characters per line in the document image is not limited to NX [characters], but the converting unit 105 may determine typesetting so that character strings are folded at NX' [characters], which is a number of characters less than NX, and generate a document image 83. The document image 83 has a large margin 831 on the right side. However, if the number of lines exceeds the maximum number of lines NY when the character string is folded at NX' [characters], the character string exceeding the maximum number of lines NY may be deleted.

[0028] Furthermore, the determining unit 104 is not limited to positioning the line head of the character string at the left end, and may position the line head at any position, such as from the center of the area to be the document image, during typesetting. By positioning the line head near the center of the line, a document image 84 is generated. The document image 84 has a margin 841 at the start position.

[0029] Furthermore, the determination unit 104 may determine a plurality of typesetting patterns within the area of the document image, while maintaining the reading direction of the character string, and varying the character arrangement according to a predetermined rule or randomly within a line. For example, in the case of a document with the text "Tomorrow will be sunny", the characters may be randomly arranged within the allowable range where the character string can be read forward, such as placing "明" at the bottom within the line and "日" at the upper right within the line. In addition, the determination unit 104 may determine a plurality of typesetting patterns displayed in different fonts or different character sizes. For example, a typesetting pattern with the character string displayed in Mincho font and a typesetting pattern with the character string displayed in Gothic font may be determined, and respective document images may be generated. Of course, a plurality of different typesetting patterns may be determined by appropriately combining the above-described typesetting variation methods. In this way, by the determination unit 104 preparing variations of the typesetting of the character string from one document and generating a plurality of different document images, document data augmentation can be easily executed.

[0030] When a document image is generated by typesetting variation as shown in FIG. 8 etc., as the corresponding tag image, a tag image (first tag image) corresponding to the positions of the character strings corresponding to the plurality of different typesetting patterns may be generated, or a tag image (second tag image) corresponding to the positions of the character strings in the case of arranging the characters evenly as in the document image 41 of FIG. 4, for example, with default settings, may be generated. The conversion unit 105 may select the correct answer data according to a user instruction, for example, whether to use the first tag image as the correct answer data or the second tag image as the correct answer data.

[0031] Next, a padding example in the convolutional processing in the network model will be described with reference to FIG. 9. FIG. 9 is an example in which a convolutional filter 91 (kernel) of the convolutional layer is set for the end portion of the document image 41 input to the network model. Here, it is assumed that the filter 91 has a size corresponding to 2×2 characters.

[0032] In the convolution of a general image, for the pixel positions at the ends, the peripheral area outside the image is zero-padded. However, at the ends of the document image 41 according to this embodiment, considering the connection of the document, it is padded with the strings before and after the wrapping of the character string to be processed. In the example of FIG. 9, when the filter 91 is set in the areas related to "no" and "sei" in the character string "rust has occurred in the downstream parts", the character "ryu" which is the character immediately before the wrapping of "no", and the character "hatsu" which is the character string immediately before the wrapping of "sei" are padded, and the convolution process is executed by the filter 91.

[0033] As a result, instead of linguistically recognizing the meaning of the character string, the convolution process can be performed to extract the connection of the character string based on the image.

[0034] According to the first embodiment shown above, a document image obtained by converting a document into an image and a tag image related to the tag information given to the document are generated, and a network model is trained using the document image and the tag image to generate a learned model. As a result, natural language processing tasks such as causal relationship extraction related to a document can be inferred based on an image. Also, by recognizing a document as an image, since Chinese characters having the same radical often have similar meanings, the meaning of Chinese characters can be learned from such commonality of radicals. Furthermore, when performing natural language processing on a document as it is, it is difficult to recognize indentation information and layout information such as bullet points. However, by converting the document into an image and training the network as a two-dimensional layout, layout information such as bullet points can also be learned. Furthermore, by changing the typesetting of a document and converting it into an image, data augmentation of the document can be easily executed. For example, by changing the arrangement such as changing the typeface or size, or shifting the character position, since the line feed changes when the width of the typesetting changes, different variations can be generated as document images even for the same document. Therefore, a large number of training data can be prepared, and as a result, the inference accuracy of the model can be improved.

[0035] (Second Embodiment) In the second embodiment, an inference device that executes inference processing using the learned model trained in the first embodiment will be described.

[0036] An inference device according to the second embodiment will be described with reference to the block diagram of FIG. The inference device 20 includes an acquisition unit 201 , a conversion unit 202 , an estimation unit 203 , an assignment unit 204 , and a storage unit 205 .

[0037] The acquisition unit 201 acquires a target document as a processing target. The conversion unit 202 converts the target document into an image, and generates a target document image. The estimation unit 203 inputs a target document image into the trained model trained in the first embodiment, obtains an inference result, and estimates a tag image related to the target document image from the inference result. The tagging unit 204 assigns tag information to the target document based on the corresponding positional relationship between the target document image and the tag image. The storage unit 205 stores the target document to which tag information has been added.

[0038] Next, an example of the operation of the inference device 20 according to the second embodiment will be described with reference to the flowchart of FIG.

[0039] In step S1101, the acquisition unit 201 acquires a target document. In step S1102, the conversion unit 202 converts the target document into an image to generate a target document image. The conversion unit 202 generates the target document image using the document image settings made in the learning phase, based on the maximum number of characters per line (NX), the maximum number of lines (NY), and the font size (FX×FY) ​​of the document set in the learning phase according to the first embodiment. For example, if settings such as margin settings and the number of characters for wrapping text were made in the learning phase, the target document image may be generated using the same settings.

[0040] In step S1103, the estimation unit 203 inputs the target document image into the trained model and generates an inference result. The inference result is obtained as a probability indicating which tag the value for each channel of each dot in the target document image corresponds to. Generally, even in pixel areas corresponding to character portions of the target document image, there are variations in brightness value. Therefore, to determine which tag a character represented in the target document image belongs to, it is sufficient to determine that the character in that pixel area belongs to the tag with the largest average probability value of tags assigned to all dots in the pixel area of ​​the character. Alternatively, it is also possible to count the tags estimated for each dot in the pixel area of ​​the character and determine that the character in that pixel area belongs to the tag with the most frequent value. Depending on the character shape, etc., the tag may change for each character, so the tag to which the target character belongs may be determined to be the average or most frequent value within a range of several characters above, below, left, and right of the target character.

[0041] Based on the above-described inference result, the estimation unit 203 generates a tag image. For example, if tagging is related to causal relationship extraction, the estimation unit 203 may perform image processing so that the font size area of ​​the character string inferred to have a causal relationship is masked. Also, if tagging is related to classifying parts of speech, the estimation unit 203 may perform image processing so that the font size area of ​​the character string has a different brightness value or color for each type of estimated part of speech. Note that if a network model is trained to output the above-described tag image, the above-described tag image may be output as the inference result of the trained model.

[0042] In step S1104, the tagging unit 204 determines the range of tags in the document based on the corresponding positional relationship between the target document image and the tag image, and tags the target document. For example, since the target document image and the tag image have the same image size, the area where the target document image and the tag image overlap can be determined as the range of tags for the target document. This allows the target document and tags acquired in step S1101 to be associated with each other. In step S1105, the storage unit 205 stores the tagged target document.

[0043] According to the second embodiment described above, a document is input, and a learned model that infers a tag image for the document is used to infer a tag image from the target document with high accuracy.

[0044] Next, an example of the hardware configuration of the learning device 10 according to the above embodiment is shown in the block diagram of Figure 12. Note that although an example of the learning device 10 is shown below, the inference device 20 may also have a similar hardware configuration.

[0045] The learning device 10 includes a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, a ROM (Read Only Memory) 33, storage 34, a display device 35, an input device 36, and a communication device 37, each of which is connected by a bus.

[0046] The CPU 31 is a processor that executes arithmetic processing, control processing, etc. according to a program. The CPU 31 uses a predetermined area of ​​the RAM 32 as a working area and executes the processing of each part of the learning device 10 described above in cooperation with the programs stored in the ROM 33 and the storage 34.

[0047] The RAM 32 is a memory such as an SDRAM (Synchronous Dynamic Random Access Memory), and functions as a work area for the CPU 31. The ROM 33 is a memory that stores programs and various types of information in a non-rewritable manner.

[0048] The storage 34 is a device that writes and reads data to a magnetic recording medium such as a hard disk drive (HDD), a semiconductor storage medium such as a flash memory, a magnetically recordable storage medium such as a HDD, an optically recordable storage medium, etc. The storage 34 writes and reads data to the storage medium in accordance with control from the CPU 31.

[0049] The display device 35 is a display device such as an LCD (Liquid Crystal Display), etc. The display device 35 displays various information based on a display signal from the CPU 31.

[0050] The input device 36 is an input device such as a mouse, a keyboard, etc. The input device 36 receives information input by a user as an instruction signal, and outputs the instruction signal to the CPU 31.

[0051] The communication device 37 communicates with external devices via a network under the control of the CPU 31 .

[0052] The instructions shown in the processing procedures described in the above-described embodiments can be executed based on a software program. A general-purpose computer system can store this program in advance and, by loading this program, achieve effects similar to those achieved by the control operations of the learning device and inference device described above. The instructions described in the above-described embodiments are recorded as a computer-executable program on a magnetic disk (such as a flexible disk or hard disk), an optical disk (such as a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, or Blu-ray (registered trademark) Disc), a semiconductor memory, or a similar recording medium. The recording medium may take any storage format as long as it is readable by a computer or embedded system. A computer can load the program from the recording medium and execute the instructions described in the program on a CPU based on the program, thereby achieving operations similar to those of the learning device and inference device described in the above-described embodiments. Of course, the computer may acquire or load the program via a network. In addition, an OS (operating system), database management software, network middleware, etc. running on a computer may execute some of the processes required to realize this embodiment based on instructions from a program installed on the computer or embedded system from a recording medium. Furthermore, the recording medium in this embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like. Furthermore, the number of recording media is not limited to one, and cases where the processing in this embodiment is executed from multiple media are also included in the recording media in this embodiment, and the media may have any configuration.

[0053] The computer or embedded system in this embodiment is for executing each process in this embodiment based on a program stored on a recording medium, and may be configured as either a device consisting of a single device such as a personal computer or a microcomputer, or a system in which multiple devices are connected to a network. Furthermore, the computer in this embodiment is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions in this embodiment by a program.

[0054] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0055] 10...Learning device, 20...Inference device, 31...CPU, 32...RAM, 33...ROM, 34...Storage, 35...Display device, 36...Input device, 37...Communication device, 40...Network model, 41, 81-84...Document image, 42, 51-53, 61-63, 71, 72...Tag image, 45...Trained model, 51-53...Tag image, 91...Filter, 101, 201...Acquisition unit, 102...Calculation unit, 103...Selection unit, 104...Decision unit, 105, 202...Conversion unit, 106...Training unit, 107, 205...Storage unit, 203...Estimation unit, 204...Assignment unit, 831, 841...Margin

Claims

1. An acquisition unit that acquires a document to which tags relating to at least one of the contents of the document and the contents of character strings contained in the document have been added; a conversion unit that converts the document into an image to generate a document image, and converts an area where a character string included in the document and associated with the tag exists into an image according to typesetting of the document image to generate a tag image; a training unit that trains a network model using the document image as input data and the tag image as correct answer data to generate a trained model; A learning device comprising:

2. a determination unit for determining a plurality of typesettings in which the document is displayed in different fonts; The learning device according to claim 1 , wherein the conversion unit generates a plurality of document images corresponding to the plurality of typesettings.

3. When arranging character strings of the document, a determination unit is further provided to determine a plurality of typesettings with different margins; The learning device according to claim 1 , wherein the conversion unit generates a plurality of document images corresponding to the plurality of typesettings.

4. When arranging character strings of the document, a determination unit determines a plurality of typesettings in which at least one of different character spacing and different line spacing is set, The learning device according to claim 1 , wherein the conversion unit generates a plurality of document images corresponding to the plurality of typesettings.

5. When arranging character strings of the document, a determination unit is further provided for determining a plurality of typesettings in which character arrangements are varied within a line according to a predetermined rule or randomly; The learning device according to claim 1 , wherein the conversion unit generates a plurality of document images corresponding to the plurality of typesettings.

6. The learning device according to claim 2, wherein the conversion unit generates, in response to a user instruction, the tag image corresponding to the position of a character string corresponding to the plurality of typesettings, or the tag image corresponding to the position of a character string when characters are evenly arranged.

7. the network model includes a convolution layer that performs convolution processing; 7. The learning device according to claim 1, wherein when the training unit performs the convolution process on an end of the document image, the training unit pads the end with one or more characters before or after folding of a character string located at the end.

8. A learning device described in any one of claims 1 to 7, wherein the tag indicates at least one of the range of words that make up a named entity, the type of part of speech, and a causal relationship.

9. Acquire a document to which tags relating to at least one of the contents of the document and the contents of character strings contained in the document have been added; converting the document into an image to generate a document image; converting an area where a character string included in the document and associated with the tag exists into an image according to typesetting of the document image to generate a tag image; training a network model using the document image as input data and the tag image as correct answer data to generate a trained model; How to learn.

10. Computer, an acquisition means for acquiring a document to which a tag relating to at least one of the contents of the document and the contents of a character string included in the document has been added; a conversion means for converting the document into an image to generate a document image, and converting an area where a character string included in the document and associated with the tag exists into an image according to typesetting of the document image to generate a tag image; a training means for training a network model using the document image as input data and the tag image as correct answer data to generate a trained model; A learning program to function as a

11. an acquisition unit that acquires a target document; a conversion unit that converts the target document into an image to generate a document image; an estimation unit that estimates a tag image corresponding to an area where a character string included in the target document exists from the document image; an assigning unit that assigns a tag to the target document related to at least one of the content of the target document and the content of a character string included in the target document based on a positional relationship between the document image and the tag image; An inference device comprising:

12. The inference device according to claim 11, wherein the estimation unit inputs the document image into a learned model trained by a learning device according to any one of claims 1 to 8, and outputs the tag image.

13. An acquisition unit that acquires a document to which a tag has been added; a conversion unit that converts the document into an image to generate a document image, and converts the tags into images according to typesetting of the document image to generate a tag image; a training unit that trains a network model using the document image as input data and the tag image as correct answer data to generate a trained model; Equipped with the network model includes a convolution layer that performs convolution processing; When performing the convolution process on an end of the document image, the training unit pads the end with one or more characters before or after folding of the character string located at the end.

Images