Voucher structure analysis system, voucher structure analysis method, and voucher structure analysis program

Abstract

To achieve processing work such as confirmation work and correction work of voucher information by a user with a lower load.SOLUTION: A voucher display unit that displays a digitized voucher on a voucher screen, a text display unit that displays a part or all of text related information indicating information corresponding to a part or all of text included in the digitized voucher on a text screen in parallel with the voucher screen, an operation monitoring unit that acquires operation information indicating an operation of a user on at least one screen of the voucher screen and the text screen, A display control unit that controls display of the text corresponding to the operation on at least a screen on which the operation is not performed among the voucher screen and the text screen.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a document structure analysis system, a document structure analysis method, and a document structure analysis program. do. [Background technology]

[0002] When processing supporting documents, the person in charge must provide supporting information such as the payment amount and payee. The information is then entered into the accounting system, such as accounting software. It has been pointed out that inputting all the information required for the management is a very heavy workload. Therefore, in order to reduce the workload of the person in charge, the accounting system is equipped with image recognition technology. In some cases, applied technology is used to electronically extract supporting information.

[0003] When extracting supporting evidence using image recognition technology, the characters written on the supporting evidence must be accurate. In some cases, the characters are not recognized properly, or the characters are recognized properly but the sentences made up of the recognized characters are If the attributes attached to the string (items such as "issue date" and "invoice amount") are inappropriate, Errors occur where information is not extracted properly. If an error occurs, the person in charge must The error is corrected and the voucher is processed. For example, in Patent Document 1, the user Compare the extracted results of the supporting documents and supporting information and correct any errors. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6732325 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, the user's input using a keyboard is not required to correct the error. Since manual input is required, the burden of correction work on users is high, which is a problem. do.

[0006] Therefore, the present invention is to facilitate the processing work such as confirmation and correction of evidence information by the user. The objective is to provide a document structure analysis system that requires little load. [Means for solving the problem]

[0007] The document structure analysis system according to one aspect of the present invention displays electronic documents on a document screen. The document display section and the document screen are arranged in parallel, and some or all of the text contained in the electronic document can be displayed. Display part or all of the text-related information indicating information corresponding to the text on the text screen. A text display section and a user's screen on at least one of the evidence screen and the text screen. An operation monitoring unit that acquires operation information indicating an operation, and based on the operation information, The text corresponding to the operation on the screen where at least no operation is being performed. and a display control unit that controls the display of the

[0008] Furthermore, a document structure analysis method according to one aspect of the present invention is a method for analyzing a document structure by a computer. is displayed on the evidence screen, and is displayed in parallel with the evidence screen, and part or all of the information contained in the electronic evidence is displayed. Displaying part or all of the text-related information indicating information corresponding to the text on the text screen and an operation indicating a user's operation on at least one of the evidence screen and the text screen. The information is acquired, and based on the operation information, at least one of the evidence screen and the text screen is operated. Controls the display of text corresponding to an operation on screens where no operation is being performed.

[0009] Furthermore, the document structure analysis program according to one aspect of the present invention is a computer program. The document display section displays the document on the document screen, and the document display section is parallel to the document screen, allowing you to view the electronic document. Part or all of the text-related information indicating information corresponding to part or all of the included text a text display unit that displays the document on a text screen, and at least one of the document screen and the text screen. an operation monitoring unit that acquires operation information indicating user operations on the screen; In this case, among the evidence screen and text screen, at least the screen on which no operation is being performed and a display control unit that controls the display of text corresponding to the operation.

[0010] In the present invention, the term "part" does not simply mean a physical means, but This also includes cases where the functions of a "part" or "device" are realized by software. Even if the functions of a device are realized by two or more physical means or devices, it is not considered to be two or more "parts" or The functionality of a device may be realized by one physical means or device. [Effects of the Invention]

[0011] According to the present invention, the processing work such as confirmation and correction of evidence information by the user can be reduced. It is possible to provide a document structure analysis system that can be implemented without burden. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of a document structure analysis system 100 according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an outline of the processing in the document structure analysis system 100. [Figure 3] FIG. 3 is a diagram showing an example of a voucher accepted by the voucher data acquisition unit 120. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of information stored in the evidence data storage unit 121. As shown in FIG. [Figure 5] FIG. 5 is a diagram showing an example of acquiring coordinate information stored in the evidence data storage unit 121. As shown in FIG. [Figure 6] FIG. 6 is a diagram showing an example of how the learning data 203 is generated. [Figure 7] FIG. 7 is a diagram showing an example of information stored in the learning data storage unit 131. As shown in FIG. [Figure 8] FIG. 8 is a diagram showing an example of information stored in the text-related information storage unit 141. As shown in FIG. [Figure 9] FIG. 9 is a diagram showing an example of the display of the text screen and the evidence screen. [Figure 10] FIG. 10 is a diagram showing an example of highlighting on the text screen and the evidence screen. [Figure 11] FIG. 11 is a diagram showing an example of a display based on the control by the display control unit 153. As shown in FIG. [Figure 12] FIG. 12 is a diagram showing an example of a display based on the control by the display control unit 153. [Figure 13] FIG. 13 is a diagram showing an example of a display based on the control by the display control unit 153. As shown in FIG. [Figure 14] FIG. 14 is a diagram showing an example of a display based on the control by the display control unit 153. [Figure 15] FIG. 15 is a diagram showing an example of a display based on the control by the display control unit 153. [Figure 16] FIG. 16 is a flowchart showing an example of a process for generating model data in the document structure analysis system 100. [Figure 17] FIG. 17 is a flowchart showing an example of a display control process in the document structure analysis system 100. DETAILED DESCRIPTION OF THE INVENTION

[0013] A preferred embodiment of the present invention will be described with reference to the accompanying drawings. 1 is a diagram showing the configuration of a document structure analysis system 100 according to an embodiment.

[0014] The document structure analysis system 100 can communicate with the user terminal 110 via the Internet. The document structure analysis system 100 is a system connected to the .

[0015] The user terminal 110 is a computer used by a user, and may be a smartphone, a tablet, or the like. The user terminal 110 is a personal computer or the like. In this case, the vouchers to be processed by the voucher structure analysis system 100 are The user terminal 110 provides the document in the form of a paper document with an image or a scanned image. The data may be provided to the document structure analysis system 100, or the electronically created document may be provided to the document structure analysis system 100. The user terminal 110 may be provided to the structural analysis system 100. Although three user terminals 110a, 110b, and 110c are shown, the user terminal 110 The number is not limited to this.

[0016] FIG. 2 is a diagram showing an outline of the process in the document structure analysis system 100. In the structural analysis system 100, the model data (learning A large number of learning documents 201 (e.g., invoices, etc.) that are the basis for generating the learning model are electronically collected. The read learning voucher data 202 is generated. Based on 202, some or all of the texts listed in the study voucher 201 (e.g. If the document is an invoice, the document will be a string such as "August 31, 2021" or "74,220" or Characters such as "2", "0", etc.) and information corresponding to the text (e.g., text attributes, etc.) More specifically, if the document is an invoice, the "payment due date" and "invoice amount" are required. Then, the learning data 203 showing the above-mentioned items is generated. Machine learning is performed using the generated learning data 203, and the learning voucher data A model that predicts the correspondence between the text included in the data 202 and the information corresponding to the text. Data 204 is generated.

[0017] Next, in the document structure analysis system 100, the display object in the document structure analysis system 100 Display target evidence data 212 is generated by electronically reading the display target evidence 211. Next, based on the generated display target evidence data 212 and model data 204, Part or all of the text described in the display subject evidence 211 and information corresponding to the text Text-related information 213 including the information is generated.

[0018] In the document structure analysis system 100, the document screen for displaying the display target document 211 is The extracted text related information is provided to the user terminal 110 and displayed in parallel with the evidence screen. A text screen displaying some or all of the information 213 is provided to the user terminal 110 (S 221, S222). In addition, in the document structure analysis system 100, the text displayed on the text screen The text-related information 213 and the document corresponding to the text-related information 213 are displayed on the document screen. The text to be displayed (the displayed text itself or the area surrounding the displayed text) It is displayed in an emphasized manner on each screen.

[0019] Furthermore, in the document structure analysis system 100, at least one of the document screen and the text screen On the other hand, based on an operation by a user through the user terminal 110, at least the operation Text or text-related information corresponding to the operation on a screen where the operation is not performed The display of 213 is controlled (S223, S224). Then, the user terminal 11 is operated by the user on at least one of the evidence screen and the text screen. Control of display based on manipulation through 0 (e.g., inferred based on model data 204) Based on the operation to modify the information, the academic record corresponding to the evidence that has been manipulated or the display control has been performed Training data is generated, and re-training is performed using this training data at any time. (S225).

[0020] Next, details of the document structure analysis system 100 will be described. The structural analysis system 100 includes a voucher data acquisition unit 120, a voucher data storage unit 121, and a learning unit. A data generating unit 130, a learning data storage unit 131, a model data generating unit 132, and a model data data storage unit 133, text-related information generation unit 140, text-related information storage unit 141, A display unit 150, a text display unit 151, an operation monitoring unit 152, a display control unit 153, and a label The computer that constitutes the document structure analysis system 100 is a processor. Each unit shown in FIG. 1 uses a memory area, This can be achieved by the processor executing a program stored in the memory. do.

[0021] The document data acquisition unit 120 acquires the document structure analysis system by operation through the user terminal 110. Accept the vouchers that are the subject of processing by M100, and All text is electronically read and the supporting data is acquired, and the acquired supporting data is stored as supporting data. The evidence data is stored in the data storage unit 121. Here, the evidence data is data acquired for each evidence. This is the basic data used when processing supporting documents.

[0022] The evidence data acquisition unit 120 can receive evidence from the user terminal 110. In addition, it is possible to accept evidence stored in cloud storage. The voucher attached to the email sent through the user terminal 110 can be accepted. In addition, the evidence data acquisition unit 120 acquires the model data generated by the model data generation unit 132, which will be described later. Accepts learning voucher 201, which is the source of data generation, and acquires learning voucher data 202 In addition, the text-related information 21 generated by the text-related information generating unit 140 (described later) can be generated. Accept the display target evidence 211 that is the basis of 3 and acquire the display target evidence data 212 can be done.

[0023] The timing at which the evidence data acquisition unit 120 reads the evidence can be set arbitrarily. For example, the voucher data acquisition unit 120 receives the voucher from the user terminal 110. The vouchers may be read in by the log, or a certain number of vouchers may be read in by the log when a certain number of vouchers are received. The evidence data acquisition unit 120 may read the evidence data at once. The text-related information generating unit 14 may read the text immediately before the processing by the text-related information generating unit 132, or may read the text immediately before the processing by the text-related information generating unit 14. The voucher to be processed may be read immediately before processing by 0.

[0024] FIG. 3 is a diagram showing an example of a voucher accepted by the voucher data acquisition unit 120. The area 301 indicates the title of the voucher, the area 302 indicates the date of issue of the voucher, the amount invoiced and the payment due date. The document data acquisition unit 120 includes an area 303 showing the document, and an area 304 showing the details of the claim. The layout of the accepted voucher is not limited to that shown in Figure 3. Also, Figure 3 shows the layout of the invoice voucher. However, the evidence data acquisition unit 120 accepts evidence not limited to invoices, but also includes receipts and sales slips. Any supporting document that is subject to accounting processing, such as a receipt, is acceptable.

[0025] 4 is a diagram showing an example of information stored in the evidence data storage unit 121. The information stored in the storage unit 121 includes, for example, a document type ID, character information, and coordinate information.

[0026] The document type ID is information indicating, for example, the format, issuer, or layout of the document. The character information is the document type identification information for identifying the type of document. The characters include numbers, symbols, spaces, etc., and are information that indicates the characters recorded on the document. The coordinate information indicates the location on the document where the characters are written. The coordinate information is, for example, the left and right coordinates of a rectangle that indicates the position where a character or a character string is written. The bottom (starting point) and top right (ending point) XY coordinates (X coordinate (starting point), Y coordinate (starting point), X coordinate (ending point) The data may be in the form of a line (point), Y coordinate (end point), or other formats. The evidence data stored in the storage unit 121 may be partially missing.

[0027] FIG. 5 is a diagram showing an example of coordinate information stored in the evidence data storage unit 121. 3 is an enlarged view of the upper part of FIG. 3, and further includes the area 301 showing the title of the document. For the string "invoice" that is displayed, a rectangular area is created for each character that indicates the location of the character. (For example, area 500 is set for the character "").

[0028] Here, the start point of the area 500 is point 501, the end point of the area 500 is point 502, and the point 501 The X and Y coordinates of point 501 are X1 and Y1, respectively, and the X and Y coordinates of point 502 are X2 and Y2, respectively. As marking information, the bottom left (starting point) and top right (ending point) of the rectangle indicating the position where the characters are written are XY coordinates (X coordinate (starting point), Y coordinate (starting point), X coordinate (ending point), Y coordinate (ending point)) are evidence When stored in the data storage unit 121, the coordinate information of the character "" is X1, Y1, X 2 and Y2 are stored in the evidence data storage unit 121. In FIG. 5, a rectangular area is created for each character. However, a rectangular area may be set for each character string.

[0029] The learning data generating unit 130 stores the data in the voucher data storage unit 110 based on the operation of the user terminal 110. The learning data 202 of the learning voucher 201 stored in 121 is structured. and stores at least one learning data 203 in the learning data storage unit 131. Here, the learning data 203 is stored in the learning document data 202. The information is integrated into character string information, and further, for each learning voucher 201, the information described in the learning voucher 201 is The character string information and the attribute corresponding to the character string information are associated with each other to generate the learning evidence data 20. 2. The learning data 203 is structured information for each learning voucher 201. Character information included in the learning evidence data 202 and information associating attributes corresponding to the character information It may also be information.

[0030] The learning data generation unit 130 generates the learning data 203 by using the learning voucher data For each 202, the character information contained in the learning evidence data 202 is integrated into certain character string information. The character string information is associated with an appropriate attribute to structure the learning voucher data 202. The learning data generating unit 130 may, for example, integrate adjacent pieces of character information into certain character string information. It is possible.

[0031] FIG. 6 is a diagram showing an example of generating the learning data 203. FIG. 6 is an enlarged view of the upper part of FIG. In the figure, the character string "July 15, 2021" contained in the field 302 indicating the date of issuance of the voucher is also included. For each character, a rectangular area indicating the position of the character (for example, the first An area 601 is set for the character "2" and an area 602 is set for the character "0".

[0032] The learning data generation unit 130 generates learning data based on the operation of the user terminal 110. are integrated as the string "July 15, 2021" to generate the string information "July 15, 2021". Furthermore, the learning data generation unit 130 determines the position where the character string "July 15, 2021" is written. A rectangular area 610 is set and coordinate information of the area 610 is acquired. The coordinate information is, for example, the XY coordinates (X coordinate ( It can be a starting point, a Y coordinate (starting point), a X coordinate (ending point), a Y coordinate (ending point), or any other shape. In addition, the learning data 203 may be a learning voucher for each learning voucher 201. In the case where the data 202 is information in which character information and attributes corresponding to the character information are associated with each other, In this case, the learning data generating unit 130 selects a position where a character corresponding to the character information is written. Alternatively, a rectangular area indicating the location may be set and the coordinate information of the area may be acquired.

[0033] Furthermore, the learning data generation unit 130 receives the character string “July 1, 2021” via the user terminal 110. For example, the attribute "issue date" is set as the attribute corresponding to "5th". The data generation unit 130 generates information in which the character string "July 15, 2021" and the attribute "issue date" are associated with each other. , and stored in the learning data storage unit 131. For each document 201, character information included in the learning document data 202 and attributes corresponding to the character information are When associating gender, the learning data generating unit 130 assigns genders to each character such as "2" and "0", for example. For example, the attribute "issue date" can be associated.

[0034] FIG. 7 is a diagram showing an example of information stored in the learning data storage unit 131. The information stored in the data storage unit 131 includes, for example, a voucher ID, a voucher type ID, character information, and 7, the information stored in the learning data storage unit 131 is Although character information is shown as information, the character string information is stored in the learning data storage unit 131. This may also be done.

[0035] The document ID is an ID assigned to each document and is document identification information used to identify the document. The label information is information indicating the attributes of the character information or character string information, such as the publication date. The coordinate information includes the location on the document where the character or character string is written. The coordinate information is information indicating the position where a character or a character string is written. The coordinates of the four points of the rectangle (X coordinate (start point), Y coordinate (start point), X coordinate (end point), Y coordinate (end point) ) or other formats.

[0036] Note that the label information and character string information stored in the learning data storage unit 131 are not necessarily It does not have to be the text written on the document. Regarding label information, for example, If similar terms such as "issue date" or "creation date" are used in both documents, "issue date" and "creation date" shall be used interchangeably. The character string information may be stored in the learning data storage unit 131 as a string, for example. Convert the string "July 15, 2021" on the document to a string such as "20210715" The data may be stored in the learning data storage unit 131.

[0037] The model data generation unit 132 generates the learning data 2 stored in the learning data storage unit 131. 03 to perform machine learning to generate model data 204, and The model data 204 is stored in the memory 133. Here, the model data 204 is the attribute of the text and the data written in the document. It is a trained model that predicts the correspondence between a given sentence and the text.

[0038] The model data generation unit 132, for example, identifies label information included in the learning data 203. By using machine learning to predict the text corresponding to the label information, the model data is The model data generating unit 132 can generate the learning data 204. By machine learning using the coordinate information contained in 203 as a feature, text corresponding to the label information is generated. By calculating the position on the document where the document is presumed to be written, the model data 204 is The model data generating unit 132 may generate the model data for each of the learning data 203. The learning data 203 is weighted according to the update date and time, and then the model data is It is also possible to generate data 204.

[0039] The model data generation unit 132 generates model data in response to, for example, an instruction from a system administrator. It is also executed automatically at predetermined times such as during system maintenance. Machine learning may be performed, or may be performed immediately after the processing of the relabeling unit 154 described later. The machine learning algorithm is not particularly limited, but for example, a recurrent neural network Recurrent Neural Network, Attention-Based Neural Network An Attention-Based Neural Network (ATN) can be used.

[0040] The text-related information generating unit 140 generates the display target evidence stored in the evidence data storage unit 121. The display target voucher data 212 of 211 and the model data stored in the model data storage unit 133 The display target evidence data 212 stored in the evidence data storage unit 121 based on the data 204 is and stores the text-related information 213 corresponding to the text in the text-related information storage unit 141. Here, the text-related information 213 includes a part or all of the text and and information corresponding to the text (for example, text attributes, etc.).

[0041] When generating the text-related information 213, the text-related information generating unit 140 uses a model data Based on the data 204, the information indicating the text itself included in the display target evidence data 212 is In response to this information, attributes corresponding to the information are associated with the information, and the display target evidence data 212 is structured. The text-related information generating unit 140 generates the display target document as information indicating the text itself. The character information included in the data 212 may be used, or the character information included in the display target evidence data 212 may be used. The character string information included in the display target evidence data 212 may be used. In this case, the text-related information generating unit 140 generates the text related to the characters included in the display target evidence data 212. The information is integrated into certain character string information, and the attribute corresponding to the character string information is assigned to the character string information. The text-related information generating unit 14 structures the display target evidence data 212 by associating the text with the evidence data. 0 is, for example, the value included in the display target evidence data 212 stored in the evidence data storage unit 121. Based on the coordinate information contained in the image, adjacent character information can be integrated to generate character string information. can.

[0042] Furthermore, the text-related information generating unit 140 may generate text-related information by using, for example, the labels included in the learning data 203. Based on the model data 204 generated by machine learning using the feature information of the Therefore, appropriate label information can be associated with information indicating the text itself.

[0043] Furthermore, the text-related information generating unit 140 automatically generates text-related information based on the model data 204. When multiple label information is predicted for the same information indicating the body, the multiple label information Alternatively, the information may be generated as label information of the same information indicating the text itself, or the information may be generated as label information of the same information indicating the text itself. The label information of the same information that indicates the text itself is selected. It may also be generated as information.

[0044] FIG. 8 is a diagram showing an example of information stored in the text-related information storage unit 141. The information stored in the store-related information storage unit 141 includes, for example, character string information, label information, accuracy information, and the like. The accuracy information is the degree of accuracy of the correspondence between the character string information and the label information. 8, the text-related information storage unit 141 stores character string information. However, character information may be stored instead of character string information.

[0045] The voucher display unit 150 displays the voucher provided to the voucher structure analysis system 100. The voucher display unit 150 generates a screen and displays the voucher screen on the user terminal 110. Depending on the method of providing the evidence according to 110, the evidence may be displayed in image format or in document format. The voucher display unit 150 may display the entire voucher provided, or Only a part of the evidence may be displayed. Also, the evidence display unit 150 may display the label information and the character string information. Depending on the accuracy of the correspondence, the corresponding text itself or the surrounding area of ​​the text is highlighted. For example, the text itself or texts with a matching probability of less than a certain value may be displayed. The area surrounding the target may be highlighted.

[0046] The text display unit 151 displays the text-related information stored in the text-related information storage unit 141. Based on the information 213, a text screen for displaying the text-related information 213 is generated, and a document image is generated. A text screen is displayed on the user terminal 110 in parallel with the screen.

[0047] The text display unit 151 may display all of the text-related information 213, or only a part of the text. Alternatively, a part of the text-related information 213 may be displayed. The information is not limited to the information itself contained in the text-related information, for example, text-related information 2 13 and the corresponding character strings. Based on this, the text display unit 151 may display a character string corresponding to the label information. The text display unit 151 also displays the displayed text-related information 213 in a user-defined manner. Direct deletion and input by operating the text screen through the user terminal 110 The text-related information 213 may be displayed in a format that allows the user to It may be displayed in a format that cannot be directly deleted or entered by operations through 10. stomach.

[0048] In addition, if there are multiple candidates for the string information corresponding to the label information, the user can select In the initial display before any operation is performed through the user terminal 110, the text display unit 151 displays The character string information with the highest probability of association is displayed on the text screen.

[0049] The text display unit 151 displays character string information whose association probability is less than a certain value. may be highlighted and displayed.

[0050] FIG. 9 is a diagram showing an example of the display of the text screen and the evidence screen. The area 910 displays a text screen and the area 920 displays a voucher screen. The screen shown in FIG. 9 displays label information stored in the text-related information storage unit 141 in an area 910. The user can directly input the text by operating the text screen on the user terminal 110. An area 911 that displays information in a format that cannot be deleted or entered, and an area 912 that displays information corresponding to label information The character string information stored in the text-related information storage unit 141 is input to the user terminal 1 by the user. A format that can be deleted and entered directly through the operation on the text screen through 10 912.

[0051] In addition, although the screen shown in FIG. 9 has areas 911 and 912 provided as separate areas, The areas 911 and 912 may be provided in the same area. Direct deletion by the user through an operation on the text screen via the user terminal 110 In addition, on the screen shown in FIG. 9, the text-related information 21 3, but the label information itself included in the text-related information 213 is displayed. based on a table that associates the label information to be stored with the character string corresponding to the label information, A character string corresponding to the label information may be displayed.

[0052] The area 920 is a screen for displaying the evidence shown in FIG. The entire document shown in FIG. 3 may be displayed in the area 920, or only a part of the document shown in FIG. 3 may be displayed. In addition, the screen shown in Figure 9 displays a text screen on the left and a document image on the right. Although the left and right sides are displayed, the left and right sides may be displayed in reverse, or other parallel arrangements such as up and down may be used. It may be displayed as follows.

[0053] The operation monitoring unit 152 monitors whether the user is Operation information indicating an operation performed by the user through the user terminal 110 is acquired.

[0054] The user's operation on the text screen through the user terminal 110 is, for example, Directly delete and input the text displayed on the text screen. This includes operations to select the currently displayed text and operations to display text candidates, which will be described later. The user's operation on the evidence screen through the user terminal 110 is, for example, The text is highlighted and the operation to select the area around the text is performed. This includes operations to select a range containing the part.

[0055] The display control unit 153 displays the evidence screen and the Among the text screens, at least the screens on which the operation related to the operation information has not been performed , the display of the text corresponding to the operation is controlled. When an operation through 0 is performed on the evidence screen, the display control unit 153 performs Based on the above, the display of at least a text screen is controlled, and the user can input the text through the user terminal 110. If the operation is performed on the text screen, the display control unit 153 displays the This controls at least the display of the evidence screen.

[0056] Below, the display of the text display section and the evidence display section will be explained using examples of the screens shown in Figs. 10 to 13. An example of the control of the display and the display control unit will be described.

[0057] FIG. 10 shows an example of highlighting on the text screen and the evidence screen. The screen shown corresponds to the screen shown in FIG. 9, but in addition, the text displayed on the text screen The three sets of character string information and the text in the evidence screen corresponding to the character string information (area 1001 a and b, areas 1002a and b, and areas 1003a and b) are each formed in the same frame rectangle. It is highlighted in the shape.

[0058] In the area 1001a, the character string information corresponding to the label information "issue date" is "July 1, 2021." In the area 1001b, the character string information "July 15, 2021" is displayed. The text "July 15, 2021" is displayed. The display unit 150 displays "July 15, 2021" in the area 1001a and the area 1001b. The text "July 15, 2021" displayed in b is enclosed in a rectangle with the same border. The text display section 151 and the evidence display section 150 are The character string information "August 31, 2021" is displayed in the area 1002a, and the character string information "August 31, 2021" is displayed in the area 1002b. The combination of the displayed text "August 31, 2021" and the displayed text in area 1003a The character string information "74,220" displayed in the area 1003b and the text "74,220" displayed in the area 1003c are Each combination of " is highlighted by surrounding the character string with a rectangle of the same border. It shows.

[0059] FIG. 11 is a diagram showing an example of a display based on control by the display control unit 153. The screen shown corresponds to the screen shown in Figure 9, but in addition, labels are added to the text screen. A mark indicating that there are multiple candidates for the character string information that corresponds to the email information "Billing amount" is displayed. By selecting the area 1101 to be displayed and the mark displayed in the area 1101, character string information can be displayed. The area 1102 includes a display area for the candidates.

[0060] When candidates for character string information are displayed by a user's operation through the user terminal 110, In this case, for example, the text-related information generating unit 140 associates the character string information with the label information. This occurs when the mapping is not performed properly and the user corrects the mapping accordingly.

[0061] Here, on the screen shown in Figure 11, the voucher displayed on the voucher screen shows " Although the billing amount is listed as "74,220," the text screen shows "20,000" as the billing amount. Therefore, the user tries to correct the mapping appropriately.

[0062] The order of the candidates for character string information displayed in the area 1102 is determined by the order of the candidates for character string information stored in the text-related information storage unit. Even if the order is based on the coordinate information included in the text-related information 213 stored in 141 Alternatively, the order may be based on the accuracy information included in the text-related information 213.

[0063] The mark displayed in the area 1101 is changed by the user's operation through the user terminal 110. When a check mark is selected, the operation monitor unit 152 acquires operation information related to the selection. The display control unit 153 controls the display of the evidence screen based on the operation information acquired by the operation monitoring unit 152. Controls the display.

[0064] Then, the voucher display unit 150 displays, for example, the area The text corresponding to the candidate string information displayed in 1102 (area 1103 and area 1104) 04) is highlighted. In the screen shown in Figure 11, the same frame is used to display the rectangle. By enclosing the character string, the corresponding text on the evidence screen is highlighted. do.

[0065] FIG. 12 is a diagram showing an example of a display based on control by the display control unit 153. The screen shown in FIG. 11 follows the screen shown in FIG. 11, but furthermore, the user By operating through 10, the text (e.g., The screen shows the screen in which "74,220" is selected (highlighted in area 1103). .

[0066] The user operates the user terminal 110 to highlight and display the document on the document screen. The text (for example, "74,220" highlighted in area 1103) is selected. Then, the operation monitoring unit 152 acquires operation information related to the selection. The control unit 153 controls the display of the text screen based on the operation information acquired by the operation monitoring unit 152. Control.

[0067] The text display unit 151 is controlled by the display control unit 153 and displays text-related information. Based on the text-related information 213 stored in the storage unit 141, The corresponding character string information (for example, "74,220") is displayed in the corresponding display field (for example, area 1003 a) is displayed.

[0068] FIG. 13 is a diagram showing an example of a display based on control by the display control unit 153. The screen shown in FIG. 11 corresponds to the screen shown in FIG. 11, but differs from the screen shown in FIG. The characters displayed in the area 1102 are changed by the user's operation through the user terminal 110. One of the column information candidates (for example, "74,220" displayed in area 1301) is selected. Shows the screen of the scene.

[0069] One of the character string information candidates is selected by the user through an operation of the user terminal 110. In this case, for example, the text-related information generating unit 140 generates character string information and label information. This occurs when the matching is not done properly and the user corrects the matching accordingly.

[0070] Here, on the screen shown in Figure 13, the voucher displayed on the voucher screen shows " Although the billing amount is listed as "74,220," the text screen shows "20,000" as the billing amount. Therefore, the user tries to correct the mapping appropriately.

[0071] The text displayed in the area 1102 is changed by the user's operation through the user terminal 110. One of the candidate strings (for example, "74,220" displayed in area 1301) is selected. Then, the operation monitoring unit 152 acquires operation information related to the selection. 153 is a block diagram showing the operation information acquired by the operation monitoring unit 152 and stored in the text-related information storage unit 141. The display of the evidence screen is controlled based on the stored text related information 213.

[0072] Then, the voucher display unit 150 displays, for example, the voucher under the control of the display control unit 153. The corresponding text displayed on the screen (e.g., "74,2" displayed in area 1302) 20") is highlighted. In the screen shown in Figure 13, the character string can be displayed by enclosing it in a rectangle. As a result, the corresponding text on the evidence screen is displayed in a highlighted manner.

[0073] FIG. 14 is a diagram showing an example of a display based on control by the display control unit 153. The screen shown in FIG. 11 corresponds to the screen shown in FIG. 11, but differs from the screen shown in FIG. The part where the text is displayed (for example, For example, area 1401) shows the screen of the scene selected in the range.

[0074] A portion where text is displayed by a user's operation through the user terminal 110 When the range is selected, for example, the voucher data is read by the voucher data acquisition unit 120. When the text-related information generating unit 140 does not generate character string information properly, When the display of the text-related information 213 is not performed properly, the user can correct the display of the text-related information 213. When the user selects the document, the character string information that the user wants to display on the document screen is included in the candidates for the character string information. This occurs when the

[0075] Here, on the screen shown in Figure 14, the voucher displayed on the voucher screen shows " Although "74,220" is listed, the text screen shows "74,220" as a possible billing amount. It is not shown, so the user is trying to correct the mapping appropriately.

[0076] A portion where text is displayed by a user's operation through the user terminal 110 When the range (for example, the area 1401 including "74,220") is selected, the operation monitor unit 152 The display control unit 153 then acquires operation information related to the range selection. 52, based on the acquired operation information, text-related information in the selected range The generation unit 140 generates the text-related information 213 and stores it in the text-related information storage unit 141. The display of the text screen is controlled while the storage process is performed.

[0077] The text display unit 151 is controlled by the display control unit 153 and displays text-related information. Based on the text-related information 213 stored in the storage unit 141, for example, The string information (e.g., "74,220") contained in the specified position is displayed on the text screen (e.g., The text display unit 151 displays the text in the area 1003a. Based on the text-related information 213 stored in the control and text-related information storage unit 141 For example, the string information (e.g., "74,220") contained in the range selected position is The candidate character string information may be displayed in an area 1102 where the candidate character string information is displayed.

[0078] FIG. 15 is a diagram showing an example of a display based on control by the display control unit 153. The screen shown in FIG. 14 corresponds to the screen shown in FIG. 14, but differs from the screen shown in FIG. The user enters the character string information in a display field (for example, area 1003a) through the user terminal 110. By this operation, a character or a character string (for example, the character "7" displayed in the area 1501) The screen shows a scene where the character is being input.

[0079] By the user's operation through the user terminal 110, characters or When a character string is input, for example, the document data acquisition unit 120 reads the document data. When the processing is not performed properly or when the text-related information generating unit 140 generates character string information, When the composition is not performed properly, the user can correct the display of the text-related information 213. This occurs when

[0080] Here, on the screen shown in Figure 15, the voucher displayed on the voucher screen shows " Although "74,220" is listed, the text screen shows "74,220" as a possible billing amount. It is not shown, so the user is trying to correct the mapping appropriately.

[0081] By the user's operation through the user terminal 110, the display field of the character string information (for example, A character or a string of characters (for example, the character "7" displayed in the area 1501) is displayed in the area 1003a. ) is input, the operation monitor unit 152 acquires operation information related to the input. The display control unit 153 displays the operation information acquired by the operation monitoring unit 152 and the input characters or controls the display of the evidence screen based on the text-related information 213 including the character string.

[0082] The text display unit 151 is controlled by the display control unit 153 and displays text-related information. Based on the text-related information 213 stored in the storage unit 141, for example, In the entered part (for example, area 1003a), a text including the entered character or character string is displayed. The document related information 213 (for example, character string information "74,220") is displayed. 50 is controlled by the display control unit 153 and stored in the text-related information storage unit 141. Based on the text-related information 213, the character string information displayed on the text display screen is The text (e.g., "74,220" displayed in area 1502) may be highlighted. In addition, on the screen shown in Figure 15, by enclosing the character string in a rectangle, it is possible to The corresponding text is highlighted.

[0083] In the screen shown in FIG. 15, the character string information that the user wants to display on the evidence screen is This shows a case where the character string is not included in the candidates for character string information, but the user If the string information you want to display is included in the string information candidates, the same display control is performed. It can be broken.

[0084] The highlighting methods shown in FIGS. 10 to 13 and 15 are rectangular tables using the same frame lines. For example, a method of displaying a rectangle of the same color or a method of filling the background with the same color may be used. Alternatively, the method of displaying the relevant characters in the same color may be used. The selection operation on the screen and text screen by the user through the user terminal 110 is For example, mouse clicks and mouseovers (moving the mouse over text) This may be done by the work.

[0085] 9 to 15 show the display of the text display section and the evidence display section, and These are examples of control by the control unit, and the display and control interfaces are not limited to these.

[0086] The relabeling unit 154 receives operation information acquired by the operation monitoring unit 152 and the display control unit 153. and a control information indicating information on a control result by the operation information. The document screen or text screen or the document that has been changed by the operation or the control information The re-learning data corresponding to both of these displays is generated, and the re-learning data is stored in the training data storage. The data is stored in the memory unit 131.

[0087] For example, the user may operate the user terminal 110 in response to the screen shown in FIG. When the display of the text screen is controlled by the relabeling unit 154, the label information and Re-learning data is generated in association with the corrected character string information.

[0088] The timing at which the re-labeling unit 154 generates the re-learning data is determined by the user. It may be for each operation through the user terminal 110, or for each display target certificate to be processed. This may be after the user has completed processing for the request 211. 4, even if the user does not perform any operation through the user terminal 110, the re-learning data The data may be generated.

[0089] FIG. 16 shows an example of a process for generating model data in the document structure analysis system 100. First, the document structure analysis system 100 is The learning voucher 201 for generating the model data of 00 is received from the user terminal 110. , and electronically read some or all of the texts listed on the accepted study voucher 201. The learning voucher data 202 is acquired and stored in the voucher data storage unit 1 21 (S1601).

[0090] Next, the document structure analysis system 100 analyzes the document data based on the operation of the user terminal 110. The learning data 203 is structured from the learning voucher data 202 stored in the data storage unit 121. and stores at least one learning data set 203 in the learning data storage unit 131. (S1602). Then, the document structure analysis system 100 stores the data in the learning data storage unit 131. Model data 204 is generated based on the stored learning data 203, and the model data The result is stored in the storage unit 133 (S1603).

[0091] FIG. 17 is a flowchart showing an example of a display control process in the document structure analysis system 100. First, the document structure analysis system 100 is operated through the user terminal 110. The display target voucher 211 to be processed by the voucher structure analysis system 100 is received from the Electronically read some or all of the text contained in the attached display voucher 211. The display target voucher data 212 is acquired, and the display target voucher data 212 is stored in the voucher data storage unit Store it in 121 (S1701).

[0092] Next, the document structure analysis system 100 analyzes the display data stored in the document data storage unit 121. The target evidence data 212 and the model data 20 stored in the model data storage unit 133 4, and generates text-related information 213 about the accepted evidence. The document structure analysis system 100 then stores the document structure in the information storage unit 141 (S1702). Based on the text-related information 213 stored in the text-related information storage unit 141, A voucher screen that displays the received voucher and a text screen that displays text-related information 213 are provided. The images are displayed in parallel (S1703).

[0093] The document structure analysis system 100 then executes the following operations by the user through the user terminal 110: Depending on the document, the text displayed on the document screen or the text displayed on the text screen Control the display of text-related information or both, such as highlighting and changing the displayed content. (S1704). Furthermore, the document structure analysis system 100 receives the document structure analysis result from the user terminal 11. Generate re-learning data based on the operation through 0 and the display control corresponding to the operation. (S1705).

[0094] The present invention has been described above. The document structure analysis system 100 is a display object. A document screen for displaying the document 211 and a text screen for displaying text-related information 213 are provided. The user terminal 11 displays the voucher screen and the text screen in parallel. Controlling the display on a screen where no operation is being performed based on the operation through 0 This allows users to more intuitively understand the correspondence between the evidence screen and the text screen. and, as a result, more easily review and correct information entered on the text screen. can be done.

[0095] In addition, the document structure analysis system 100 is operated through the user terminal 110 and document structure analysis is performed. and / or the analysis system 100 controls the display based on the text-related information 213. Based on this, re-learning data can be generated. The correction work can be reflected in the learning data immediately, and the model data 204 can be immediately updated. This allows for efficient updates.

[0096] In addition, the document structure analysis system 100 analyzes the text displayed on the document screen and the text The text-related information 213 displayed on the main screen can be displayed in a highlighted manner. This allows the user to link the display of the evidence and the display of the text-related information 213. As a result, the text displayed on the evidence screen and The correspondence between the text and the text-related information 213 displayed on the text screen can be grasped more intuitively. This allows you to more easily check and correct information entered on the text screen. Cut.

[0097] In addition, the document structure analysis system 100 analyzes the text and text displayed on the document screen. A user operation on at least one of the text-related information 213 displayed on the screen Based on this, the display of the corresponding text screen and evidence screen can be controlled. Therefore, the user can view the display of the document and the text-related information 213 in a linked manner. As a result, the text-related information 213 displayed on the text screen This allows for more intuitive editing and makes it easier to edit information entered on the text screen. It can be checked and corrected.

[0098] It should be noted that the present embodiment is provided to facilitate understanding of the present invention and does not limit the present invention. The present invention may be modified or improved without departing from the spirit and scope of the present invention. and the present invention includes equivalents thereof. [Explanation of symbols]

[0099] 100 Evidence structure analysis system, 110 User terminal, 120 Evidence data acquisition unit, 12 1 Voucher data storage unit, 130 Learning data generation unit, 131 Learning data storage unit, 1 32 model data generation unit, 133 model data storage unit, 140 text-related information generation unit 141 text-related information storage unit, 150 evidence display unit, 151 text display unit , 152 operation monitoring unit, 153 display control unit, 154 relabeling unit, 201 learning Evidence, 202 Learning evidence data, 203 Learning data, 204 Model data, 21 1 Displayed evidence, 212 Displayed evidence data, 213 Text-related information

Claims

1. a voucher display unit that displays the electronic voucher on a voucher screen; A part or all of the text contained in the electronic voucher is displayed in parallel with the voucher screen. A text displaying part or all of the text-related information showing the information corresponding to the text on the text screen. a display unit, A user operation on at least one of the evidence screen and the text screen is indicated. an operation monitoring unit that acquires operation information; Based on the operation information, at least one of the evidence screen and the text screen is A display that controls the display of text corresponding to an operation on a screen where no operation is being performed. A control unit; A document structure analysis system that includes:

2. The text-related information is a label indicating the attribute of the text included in the electronic evidence.

2. The document structure analysis system according to claim 1, wherein the document structure analysis system includes at least information indicating the document structure and the text itself. Tem.

3. The operation information and at least one piece of control information indicating information related to a control result by the display control unit and changing at least the label information included in the text-related information based on at least one of the The evidence structure according to claim 2, further comprising a relabeling unit that generates updated re-learning data. Structural analysis system.

4. The text display unit displays at least one of the texts included in the electronic document. displaying at least one piece of said text-related information for each attribute; The voucher display unit displays at least one of the text-related information corresponding to the at least one of the text-related information. Highlight a single piece of text, The document structure analysis system according to claim 1 .

5. The operation monitoring unit is configured to monitor the at least one of the texts displayed on the text screen. Acquire operation information on the text screen for the related information; The voucher display unit displays the voucher based on the control by the display control unit that is based on the operation information. highlighting the at least one piece of text corresponding to the at least one piece of text-related information; and display, The document structure analysis system according to claim 4.

6. The operation monitoring unit is configured to: Acquire operation information on the evidence screen for The text display unit, based on the control by the display control unit based on the operation information, The at least one piece of text-related information corresponding to the at least one piece of text is Display, The document structure analysis system according to claim 4.

7. The operation monitoring unit receives coordinate information indicating a position selected by a user on the evidence screen. Get information and The text display unit, based on the control by the display control unit based on the coordinate information, Displaying the text-related information corresponding to the text included in the position indicated by the coordinate information 、 The document structure analysis system according to claim 1 .

8. The text-related information includes coordinate information indicating the position of the text and a plurality of texts.

3. The document structure according to claim 2, further comprising at least one piece of coordinate information indicating the position of a character string to be displayed. Structural analysis system.

9. The computer The electronic voucher is displayed on the voucher screen. A part or all of the text contained in the electronic voucher is displayed in parallel with the voucher screen. a part or all of the text-related information indicating the information corresponding to the A user operation on at least one of the evidence screen and the text screen is indicated. Obtain operation information, Based on the operation information, at least one of the evidence screen and the text screen is Controlling the display of text corresponding to the operation on a screen where no operation is being performed; Evidence structure analysis method.

10. On the computer, a voucher display unit that displays the electronic voucher on a voucher screen; A part or all of the text contained in the electronic voucher is displayed in parallel with the voucher screen. A text displaying part or all of the text-related information showing the information corresponding to the text on the text screen. a display unit, A user operation on at least one of the evidence screen and the text screen is indicated. an operation monitoring unit that acquires operation information; Based on the operation information, at least one of the evidence screen and the text screen is A display that controls the display of text corresponding to an operation on a screen where no operation is being performed. A control unit; A document structure analysis program to achieve this.

Images