Information processing device and information processing program

The information processing device enhances document filling by identifying and superimposing unfilled items, improving accuracy and preventing incomplete submissions through user-friendly format updates and obstruction handling.

JP7947826B2Active Publication Date: 2026-11-11FUJIFILM BUSINESS INNOVATION CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2022136831
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-08-30
Publication Date
2026-11-11
Estimated Expiration
2042-08-30

AI Technical Summary

Technical Problem

Users often fail to fill in all required fields on documents due to the large number of entry items, leading to incomplete submissions.

Method used

An information processing device that acquires video footage of document filling, identifies entry items, applies formats, and superimposes unfilled items on the image for easy recognition, with the ability to update formats based on user interactions and prevent scanning with missing entries.

Benefits of technology

Assists users in checking for missing entries, improves entry item identification accuracy, facilitates format updates, and prevents incomplete document scanning by highlighting unfilled items and obstructions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007947826000001
    Figure 0007947826000001
  • Figure 0007947826000002
    Figure 0007947826000002
  • Figure 0007947826000003
    Figure 0007947826000003
Patent Text Reader

Abstract

To provide an information processing device and an information processing program that are capable of supporting a user in checking whether there is a missing entry.SOLUTION: An information processing device includes a processor. The processor obtains an image captured on a side where a document including entry items is filled out, and outputs an incomplete item among the entry items of the document in the image.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The technology of the present invention relates to an information processing apparatus and an information processing program.

Background Art

[0002] There is a technology that assists a user in using an image processing apparatus. Patent Document 1 discloses an information input apparatus, comprising: a registration unit that registers form information by associating feature points of a form in which a plurality of entry fields are set with the type of object that a user should use for each entry field; a projection unit that projects a specific image; a reading unit that reads the placed form; an acquisition unit that acquires a depth image corresponding to a user's operation; a first recognition unit that recognizes the placed form and the plurality of entry fields printed on the form by collating feature points extracted from analysis of the read form image with the registered feature points of the form; a second recognition unit that analyzes the depth image acquired by the acquisition unit to recognize an object operated by the user and the type of the object; and a control unit that controls the projection unit to project the specific image onto the entry field of the form associated with the type of object that the user should use, among the plurality of entry fields recognized by the first recognition unit, in response to a change in the type of the object operated by the user recognized by the second recognition unit.

[0003] Patent Document 2 discloses an image generation method, characterized by monitoring the motion of an area to be imaged, and triggering the capture of an image of the area in response to monitoring of the stop of motion in the area.

[0004] Patent Document 3 discloses an image processing apparatus, comprising: a confirmation unit that confirms the content entered in a predetermined entry area on a document read by an image reading apparatus; and a determination unit that determines whether or not the read document may be transmitted based on the content entered in the entry area as a result of the confirmation.

Prior Art Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2017-033454 [Patent Document 2] Japanese Patent Publication No. 2007-013972 [Patent Document 3] Japanese Patent Publication No. 2016-012751 [Overview of the project] [Problems that the invention aims to solve]

[0006] In various procedures, users fill in the required information on the documents and submit their applications.

[0007] Due to the large number of fields to fill out and other factors, there is a possibility that some fields may be left blank.

[0008] The technology disclosed herein aims to provide an information processing device and an information processing program that can assist users in checking for any missing information. [Means for solving the problem]

[0009] The information processing device according to the first embodiment includes a processor, the processor acquires video footage taken from the side of a document containing entry items, and outputs the unentered entry items in the video footage.

[0010] The information processing device according to the second embodiment, in the information processing device according to the first embodiment, receives a format that identifies the entry items of the document, and outputs the entry items identified by the format of the document in the video.

[0011] The information processing device according to the third embodiment is an information processing device according to the second embodiment in which the processor receives video footage of an identification notation representing information that identifies the format, and applies the format identified from the captured identification notation to the document.

[0012] The information processing device according to the fourth embodiment is an information processing device according to the second embodiment in which the processor receives a user's specification of the entry items and associates information for identifying the specified entry items with the format.

[0013] The information processing device according to the fifth embodiment is an information processing device according to the second embodiment in which the processor associates information for identifying the entry items entered by the user from the video with the format.

[0014] The information processing device according to the sixth embodiment is an information processing device according to the first embodiment, wherein the processor displays the area of ​​the unentered items superimposed on the image.

[0015] The information processing device according to the seventh embodiment is an information processing device according to the sixth embodiment, wherein the processor changes the display manner of the area of ​​the blank item according to the type of blank item.

[0016] The information processing device according to the eighth embodiment, in the information processing device according to the sixth embodiment, the processor displays the area of ​​the unrecorded item superimposed on the obstruction when the obstruction overlaps with the area of ​​the unrecorded item in the photograph.

[0017] In the information processing device according to the ninth embodiment, the processor instructs the execution of a scan when all of the entry items have been filled in, as in the information processing device according to the first embodiment.

[0018] The information processing program according to the tenth embodiment causes the processor to acquire video footage taken from the side of the document containing the entry items, and to output the unentered entry items in the video footage.

Effects of the Invention

[0019] According to the first aspect and the tenth aspect, it is possible to assist the user in checking for any missing entries.

[0020] According to the second aspect, the accuracy of identifying entry items can be improved.

[0021] According to the third aspect, the format can be applied based on the captured identification notation.

[0022] According to the fourth aspect, the format can be updated based on user operations.

[0023] According to the fifth aspect, the format can be updated based on user entries.

[0024] According to the sixth aspect, the position of unfilled items can be easily visually recognized.

[0025] According to the seventh aspect, the type of items that require entry can be easily distinguished.

[0026] According to the eighth aspect, it is possible to prevent overlooking entry items due to obstructing objects.

[0027] According to the ninth aspect, it is possible to prevent execution of scanning for a document with missing entries.

Brief Description of the Drawings

[0028] [Figure 1] It is a perspective view showing an example of the overall configuration of the image processing apparatus according to the present embodiment. [Figure 2] It is a block diagram showing the hardware configuration of the image processing apparatus according to the present embodiment. [Figure 3] It is a flowchart showing the flow of assistance processing of the image processing apparatus according to the present embodiment. [Figure 4] This is an example showing a document placed on a document platen and an identification notation. [Figure 5]This is an example of the document display area shown on the display unit. [Figure 6] This is an example of a display unit that accepts the specification of fields to be entered. [Figure 7] This is an example showing documents placed on a document tray. [Figure 8] This is an example of the document display area shown on the display unit. [Figure 9] This is an example showing documents placed on a document tray. [Figure 10] This is an example of a display unit during a scan. [Figure 11] This is an example showing documents placed on a document tray. [Figure 12] This is an example of a display section for cases where an obstruction overlaps with an unfilled item. [Modes for carrying out the invention]

[0029] Hereinafter, an example of an embodiment of the technology of the present invention will be described with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from the actual ratios.

[0030] Figure 1 is a perspective view showing an example of the overall configuration of the image processing device 10 of the present disclosure. The present disclosure describes an example of an application of the information processing device technology to an image processing device 10 that is installed in a government office, office, school, or home, etc., and has a document camera.

[0031] The image processing device 10 includes a document tray 30 on which the original document is placed, a user interface unit 40 for displaying and inputting various information, and a document camera 70.

[0032] The document stand 30 is a workbench where the user places documents and fills in the necessary information.

[0033] The user interface unit 40 includes an input unit 15 for receiving input and a display unit 16 consisting of a liquid crystal display or the like. The user interface unit 40 may also have an integrated configuration of the input unit 15 and the display unit 16 by using a touch panel. The image processing device 10 displays the image captured by the document camera 70 on the display unit 16.

[0034] The document camera 70 is configured to capture images of the upper surface of the document table 30. For example, the document camera 70 is mounted on the other end of an arm, one end of which is fixed to the rear side of the document table 30, and is positioned so that its field of view substantially coincides with the document placement area 32 on the document table 30. Alternatively, a smartphone or tablet device that includes a camera may be used instead of the document camera 70. In this case, the image processing device 10 receives images captured from the smartphone or tablet device.

[0035] Figure 2 is a block diagram showing the hardware configuration of the image processing device 10. The image processing device 10 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, storage 14, input unit 15, display unit 16, communication interface (communication I / F) 17, image forming unit 18, and document camera 70. Each component is connected to the others via a bus 19 so that they can communicate with each other.

[0036] The CPU 11 is a central processing unit that executes various programs and controls various components. Specifically, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculations according to the program recorded in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores an information processing program that helps the user check for any missing entries.

[0037] ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. Storage14 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs, including the operating system, and various data.

[0038] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used for various types of input.

[0039] The display unit 16 is, for example, a liquid crystal display, which displays various types of information.

[0040] The user interface unit 40 includes an input unit 15 and a display unit 16. The user interface unit 40 may also have a configuration in which the input unit 15 and the display unit 16 are integrated, using a touch panel method for the liquid crystal display.

[0041] The communication interface 17 is an interface for communicating with other devices such as databases, and standards such as Ethernet®, FDDI, and Wi-Fi® are used.

[0042] The image forming unit 18 forms an image on a recording medium such as paper based on the received image. The image forming method may include a toner-based method or an inkjet method. The image forming unit 18 may also have functions such as copying, scanning, and printing.

[0043] Next, the operation of the image processing device 10 will be explained. Figure 3 is a flowchart showing the flow of support processing by the image processing device 10. Support processing is performed when the CPU 11 reads the support program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it.

[0044] In step S101, the CPU 11 determines whether or not a document has been detected. That is, the CPU 11 detects whether or not a document is placed within the document placement area 32 of the document glass 30. If it determines that a document has been detected (step S101: YES), the CPU 11 proceeds to step S102. If it determines that no document has been detected (step S101: NO), the CPU 11 waits until a document is detected.

[0045] In step S102, the CPU 11 acquires video footage taken from the side of document 1 being filled out. Video footage taken from the side of document filling out refers to video footage taken using a document camera 70 installed above the writing surface of document 1, or video footage of the writing items taken using a smartphone or tablet device, etc. The CPU 11 then proceeds to step S103.

[0046] In step S103, the CPU 11 determines whether or not it has detected a format. If it determines that it has detected a format (step S103: YES), the CPU 11 proceeds to step S104. If it determines that it has not detected a format (step S103: NO), the CPU 11 proceeds to step S107.

[0047] Here, Figure 4 shows an example of a document 1 and identification mark 2 placed on the document table 30. The user, for example, writes their address, signs, and stamps document 1, which includes fields to fill in. Document 1 is, for example, a contract or various application forms. Identification mark 2 is an identifier that represents information that identifies the format. Identification mark 2 may be, for example, a string of characters such as a URL (Uniform Resource Locator) and an identification number, or a piece of paper or marker with a two-dimensional code written on it. Identification mark 2 may also be written on document 1. The format is a pre-created template that identifies the fields to fill in. The image processing device 10 detects the document if document 1 is in the document placement area 32. The image processing device 10 detects identification mark 2 if identification mark 2 is in the document placement area 32. That is, by detecting identification mark 2, the image processing device 10 detects the format identified by identification mark 2.

[0048] In step S104, the CPU 11 accepts the format. For example, the CPU 11 accepts the video footage of the identification mark 2 and obtains the format identified from the identification mark 2. Alternatively, the CPU 11 may accept a format specification from the input unit 15. The CPU 11 then proceeds to step S105.

[0049] In step S105, the CPU 11 applies the format to document 1. Note that if multiple documents 1 are placed in the document placement area 32, the CPU 11 may apply the format to each document 1 individually, or it may apply one format to multiple documents 1. The CPU 11 then proceeds to step S106.

[0050] In step S106, CPU 11 outputs the entry items specified by the format. CPU 11 then proceeds to step S110.

[0051] Here, Figure 5 is an example of the document display area 3 displayed on the display unit 16. The document display area 3 is the area that displays document 1 in the captured video. Figure 5 shows the display of the document display area 3 corresponding to document 1 in Figure 4. In other words, the image processing device 10 displays document 1 captured by the document camera 70 in the document display area 3. The document display area 3 includes an item display area 4. The item display area 4 is, for example, an area that indicates that it is an item. The image processing device 10 displays the item display area 4 superimposed on the entry items specified by the format. The image processing device 10 may also accept user input when the signature start button 50 is selected. Furthermore, the image processing device 10 may change the display mode of the item display area 4 to highlight or blink. The image processing device 10 may also change the display mode of the item display area 4 according to the type of entry item. The type of entry item is classified, for example, by whether the entry target is text or an image. For example, as a text type, the entry item may be a written or multiple-choice format, an address or telephone number format, or a format consisting only of text or only of numbers. For example, as an image type, the entry item may be a seal, a passport photograph, a stamp or various revenue stamps. In Figure 5, the image processing device 10 represents the fields for entering the address and company name / name as a text-type item display area 4A, and the field for affixing a seal as a graphic-type item display area 4B.

[0052] In step S107, the CPU 11 transmits video to an external device. For example, the CPU 11 transmits video to a terminal device operated by an operator familiar with document 1. The CPU 11 then proceeds to step S108.

[0053] In step S108, the CPU 11 accepts the specification of the entry items. The CPU 11 accepts specifications, for example, from the user filling out the form or from an operator operating an external device. The CPU 11 then proceeds to step S109.

[0054] Here, Figure 6 shows an example of a display unit 16 that accepts the specification of input items. The image processing device 10 displays a pointer 5, a free text button 51, an image button 52, a destination registration button 53, and a format registration button 54 on the display unit 16. The pointer 5 is the coordinate of the display unit 16 specified by the user using the input unit 15. When the free text button 51 is selected, the image processing device 10 sets the type of input field in the range specified by the pointer 5 to free text, which is a descriptive format. When the image button 52 is selected, the image processing device 10 sets the type of input field in the range specified by the pointer 5 to image format. When the destination registration button 53 is selected, the image processing device 10 displays a screen for registering the destination on the display unit 16. The image processing device 10 may also associate the destination with the format. The image processing device 10 registers a format when the format registration button 54 is selected. That is, the image processing device 10 sets the type of entry item or associated contact information set in the format.

[0055] In step S109, the CPU 11 creates a format. Specifically, the CPU 11 creates a format by selecting the format registration button 54 in Figure 6 described above. The CPU 11 associates the information to identify the specified entry items with the created format. The CPU 11 then proceeds to step S106.

[0056] In step S110, the CPU 11 determines whether or not the user has made an entry. If it determines that the user has made an entry (step S110: YES), the CPU 11 proceeds to step S111. If it determines that the user has not made an entry (step S110: NO), the CPU 11 waits until the user makes an entry.

[0057] In step S111, the CPU 11 associates information with the format to identify the fields entered by the user. That is, the CPU 11 updates the format to recognize the fields entered by the user. The CPU 11 then proceeds to step S112.

[0058] In step S112, the CPU 11 detects the unfilled items. That is, the CPU 11 detects the unfilled items in the document in the video. For example, the CPU 11 compares the video at the time of step S102 with the video at the current time and detects items that do not differ from each other as unfilled items. For example, the CPU 11 detects items as unfilled items if the text cannot be detected by OCR (Optical Character Recognition). The CPU 11 then proceeds to step S113.

[0059] In step S113, the CPU 11 outputs the unfilled items among the entry items specified by the format. For example, the CPU 11 displays the area of ​​the unfilled items superimposed on the video. The CPU 11 may also output the unfilled items by, for example, displaying the name of the unfilled item or reading it aloud. The CPU 11 then proceeds to step S114.

[0060] Here, Figures 7 and 8 show an example of the processing when some fields are filled in by the user. Figure 7 shows an example of Document 1 placed on the document table 30. Document 1 in Figure 7 has the address and signature added, but no seal has been affixed, compared to Document 1 in Figure 4.

[0061] Figure 8 shows the display of document display area 3 corresponding to document 1 in Figure 7, and is an example of document display area 3 displayed on the display unit 16. In Figure 8, the item display area 4A is removed from the display on the display unit 16 in Figure 5, and item display area 4B is displayed. The image processing device 10 changes the display mode by deleting item display area 4, etc., when an unfilled item is filled in and is no longer an unfilled item.

[0062] In step S114, CPU 11 determines whether all fields have been filled in. If it determines that all fields have been filled in (step S114: YES), CPU 11 proceeds to step S115. If it determines that there are any fields that have not been filled in (step S114: NO), CPU 11 proceeds to step S110.

[0063] In step S115, the CPU 11 instructs the system to perform a scan. That is, the CPU 11 performs a scan using the image forming unit 18. The CPU 11 then proceeds to step S116.

[0064] Here, Figures 9 and 10 show an example of the process when all fields have been filled in by the user. Figure 9 shows an example of document 1 placed on document table 30. Document 1 in Figure 9 has been stamped and all fields have been filled in, as in document 1 in Figure 7. Figure 10 shows the display of document display area 3 corresponding to document 1 in Figure 9, and is an example of document display area 3 displayed on the display unit 16. In the document display area 3 in Figure 10, the item display area 4B has been removed from the display on the display unit 16 in Figure 8, and no item display areas 4 are displayed. The image processing device 10 instructs to perform a scan of document 1 if all fields in document 1 have been filled in. The image processing device 10 cancels the scan if the cancel button 55 is selected.

[0065] In step S116, the CPU 11 sends the scanned data to the destination. For example, the CPU 11 sends the scanned data to a destination associated with the format. The CPU 11 then terminates the support process.

[0066] As described above, through the processing in steps S101 to S113, the CPU 11 acquires video footage taken from the side of the person filling out the document, and outputs the unfilled items in the document in the video.

[0067] [Differentiation] The image processing apparatus 10 of this embodiment has been described above. However, this disclosure is not limited to the above-described embodiment. Various improvements or modifications are possible.

[0068] While this embodiment describes a case where the information processing device is an image processing device 10, it is not limited to this. That is, the information processing device may be integrated with the image processing device or may be separate from the image processing device. For example, the information processing device may be a smartphone or tablet terminal equipped with a camera, and the unfilled items may be output based on the image captured by the camera.

[0069] In this embodiment, the image processing device 10 may accept user input fields even after detecting a format. That is, in step S103 of Figure 3, if the CPU 11 detects a format, it may execute steps S107 to S109 in addition to steps S104 and S105. In this case, as step S109, the CPU 11 executes a process to update the detected format.

[0070] The image processing device 10 according to this embodiment may detect entry items using prior art. For example, the image processing device 10 may determine that a rectangle is an entry item by extracting a rectangle and an item name from document 1 in the image.

[0071] In this embodiment, the image processing device 10 may display the area of ​​an unfilled item by superimposing it on the obstruction 6 when the obstruction 6 overlaps with the area of ​​an unfilled item during imaging. Figures 11 and 12 show an example of processing when an obstruction overlaps with the area of ​​an unfilled item. Figure 11 shows an example of a document placed on a document glass. Figure 11 shows that in the image captured by the document camera 70, a pencil, which is the obstruction 6, is in an unfilled item on document 1, and therefore part of the unfilled item is not displayed. Figure 12 shows the display of the document display area 3 corresponding to document 1 in Figure 11, and is an example of the document display area 3 displayed on the display unit 16. The image processing device 10 displays the item display area 4A so as to superimpose it on the pencil, which is the occluding object 6.

[0072] The image processing apparatus 10 according to this embodiment may output supplementary information regarding the items to be filled in. In step S113 of Figure 3, the CPU 11 displays help information regarding the items to be filled in, the content to be filled in, or the method of filling in on the display unit 16 for the items that have not been filled in.

[0073] The above process can also be implemented using dedicated hardware circuits. In this case, it may be executed on a single piece of hardware or on multiple pieces of hardware.

[0074] In the above embodiment, the term "processor" refers to a broad type of processor, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0075] Furthermore, the operation of the processor in the above embodiment may not be performed by a single processor, but may be performed by multiple processors located in physically separate locations working together. Also, the order of each processor operation is not limited to the order described in the above embodiment, but may be changed as appropriate.

[0076] Furthermore, the program for operating the image processing device 10 may be provided on a computer-readable recording medium such as a USB (Universal Serial Bus) memory, a flexible disk, or a CD-ROM (Compact Disc Read Only Memory), or it may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in memory or storage. This program may also be provided, for example, as a standalone application software, or it may be incorporated into the software of each device as a function of the image processing device 10.

[0077] [Note] The following are preferred forms of this disclosure.

[0078] (((1))) Including the processor, The aforementioned processor, We obtained video footage taken from the perspective of the person filling out the document, including the fields to be filled in. Output the unfilled items in the document shown in the video. Information processing device.

[0079] (((2))) The aforementioned processor, We accept a format that identifies the aforementioned fields to be filled in on the aforementioned document. The video outputs the entry items specified by the format of the document. The information processing device described in (((1))).

[0080] (((3))) The aforementioned processor, The system receives video footage of an identification mark representing information that identifies the aforementioned format. Apply the format identified from the photographed identification mark to the document. The information processing device described in (((2))).

[0081] (((4))) The aforementioned processor, The user specifies the fields to be filled in, Associate the information used to identify the specified entry items with the format. The information processing device described in (((2))) or (((3))).

[0082] (((5))) The aforementioned processor, Information for identifying the fields entered by the user from the aforementioned video is associated with the aforementioned format. An information processing device described in any one of (((2))) to (((4))).

[0083] (((6))) The aforementioned processor, The area of ​​the unfilled item is superimposed on the image. An information processing device described in any one of (((1))) through (((5))).

[0084] (((7))) The aforementioned processor, Depending on the type of the blank item, the display method of the area for the blank item will be changed. The information processing device described in (((6))).

[0085] (((8))) The aforementioned processor, If an obstruction overlaps with the area of ​​the unfilled item in the photograph, the area of ​​the unfilled item will be displayed superimposed on the obstruction. The information processing device described in (((6))) or (((7))).

[0086] (((9))) The aforementioned processor, If all the above fields have been filled in, proceed with the scan. An information processing device described in any one of (((1))) through (((8))).

[0087] (((10))) We obtained video footage taken from the perspective of the person filling out the document, including the fields to be filled in. Output the unfilled items in the document shown in the video. An information processing program that causes a processor to perform a task.

[0088] The effects of the configuration described below are explained below.

[0089] According to the information processing device (((1))) or (((10))), it is possible to assist the user in checking whether or not there are any missing entries.

[0090] The information processing device (((2))) can improve the accuracy of identifying the fields to be filled in.

[0091] According to the information processing device (((3))), a format can be applied based on the captured identification notation.

[0092] The information processing device (((4))) can update the format based on user operations.

[0093] The information processing device (((5))) can update the format based on user input.

[0094] The information processing device (((6))) makes it easy to see the location of unfilled items.

[0095] The information processing device (((7))) makes it easy to determine the type of item that needs to be filled in.

[0096] The information processing device (((8))) can prevent overlooking entry items due to obstructions.

[0097] The information processing device (((9))) can prevent the scanning of documents that have missing information. [Explanation of Symbols]

[0098] 1. Document 2. Identification mark 3. Document display area 4, 4A, 4B item display area 5 pointers 6 Shield 10 Image Processing Device 30 Manuscript Stand 32 Original placement area 50 Start Signing Button 51 Free text button 52 Image Buttons 53. Register recipient button 54. Format Registration Button 55 Cancel button 70 Document Camera

Claims

1. Including the processor, The aforementioned processor, We obtained video footage taken from the perspective of the person filling out the document, including the fields to be filled in. When outputting the unfilled items in the document shown in the video, the area of ​​the unfilled items is superimposed on the video and displayed. An information processing device that, when an obstruction overlaps with the area of ​​the unfilled item in a photograph, displays the area of ​​the unfilled item by superimposing it on the obstruction.

2. The aforementioned processor, We accept a format that identifies the aforementioned fields to be filled in on the aforementioned document. The video outputs the entry items specified by the format of the document. The information processing apparatus according to claim 1.

3. The aforementioned processor, The system receives video footage of an identification mark representing information that identifies the aforementioned format. Apply the format identified from the photographed identification mark to the document. The information processing apparatus according to claim 2.

4. The aforementioned processor, The user specifies the fields to be filled in, Associate the information used to identify the specified entry items with the format. The information processing apparatus according to claim 2.

5. The aforementioned processor, Information for identifying the fields entered by the user from the aforementioned video is associated with the aforementioned format. The information processing apparatus according to claim 2.

6. The aforementioned processor, Depending on the type of the blank item, the display method of the area for the blank item will be changed. The information processing apparatus according to claim 1.

7. The aforementioned processor, If all the above fields have been filled in, proceed with the scan. The information processing apparatus according to claim 1.

8. We obtained video footage taken from the perspective of the person filling out the document, including the fields to be filled in. When outputting the unfilled items in the document shown in the video, the area of ​​the unfilled items is superimposed on the video and displayed. If an obstruction overlaps with the area of ​​the unfilled item in the photograph, the area of ​​the unfilled item will be displayed superimposed on the obstruction. An information processing program that causes a processor to perform a task.

Citation Information

Patent Citations

  • Document processing system, document output apparatus, document processor, and document

    JP2005328361A

  • Controlling scanner and copying machine through implicit gesture

    JP2007013972A

  • Business form discrimination device

    JP2007109051A

  • Electronic application processing system

    JP2009048321A

  • Image forming apparatus and transmission method

    JP2016012751A