Form processing system, form processing method and program

Abstract

This business forms processing system comprises: an acquisition means for acquiring an image capturing (a) business form(s) picked up by a robot in a direction intersecting the pick-up direction; and a determination means for referring to the image to ascertain if the business form(s) having been picked up is (are) a bundle.

Description

[Technical Field]

[0001] The present disclosure relates to a form processing system and the like. [Background technology]

[0002] It is common to process forms using machines. For example, rollers are used to transport forms. In addition, a robot may be used to lift up a stack of forms and move them. For example, Patent Document 1 discloses an articulated robot that grasps the topmost sheet of paper stacked in one location and transports it to another location. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-037157 Summary of the Invention [Problem to be solved by the invention]

[0004] There are cases where multiple documents are stapled together to form a stack. There are also cases where multiple documents are stuck together by static electricity, for example. A stack of documents can hinder subsequent processing.

[0005] The present disclosure aims to provide a form processing system and the like that can appropriately process bundles of forms. [Means for solving the problem]

[0006] The document processing system according to the present disclosure includes an acquisition means for acquiring an image of a document being lifted by a robot taken from a direction intersecting the direction in which the document is being lifted, and a determination means for determining, based on the image, whether the documents being lifted are in a stack.

[0007] The document processing method disclosed herein acquires an image of a document being lifted by a robot taken from a direction intersecting the direction in which the document is being lifted, and determines whether the documents being lifted are in a stack based on the image.

[0008] The program according to the present disclosure causes a computer to execute a process of acquiring an image of a document being lifted by a robot from a direction intersecting the direction of lifting, and determining whether the document being lifted is in a stack based on the image. The program may be stored in a computer-readable non-transitory recording medium. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to appropriately process stacks of documents. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of a form processing system 100 according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of bound forms. [Figure 3] FIG. 10 is a diagram showing a scene of a document being lifted as seen from the side. [Figure 4] 4 is a flowchart showing an example of the operation of the form processing system 100 according to the first embodiment. [Figure 5] 1 is a block diagram showing an example of the configuration of a document sorting system 1 according to a second embodiment. [Figure 6] 1 is a diagram showing an example of the arrangement of a robot 10, a transport device 20, a camera 30, and a camera 31. FIG. [Figure 7] 1 is a diagram showing an example of the arrangement of a conveying device 20 and a sorting mechanism 40. FIG. [Figure 8] 10 is a flowchart showing an example of the operation of the form processing system 100 according to the second embodiment. [Figure 9] FIG. 10 is a diagram showing a scene in which the lifted document is viewed from one direction. [Figure 10] FIG. 10 is a diagram showing the lifted document as seen from another direction. [Figure 11] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer 500. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, with reference to the drawings, embodiments of a form processing system, a form processing method, a program, and a recording medium for non-temporarily recording the program according to the present disclosure will be described. Forms include, for example, but are not limited to, forms for collection, slips, gift certificates, or receipts, and may include other documents. The present disclosure is also applicable to the processing of sheets other than forms. Processable sheets include, for example, thin metal sheets, plastic clear files, flyers, and magazines.

[0012] [First embodiment] A form processing system 100 according to the first embodiment is a system that determines whether forms picked up by a robot are in a stack. Fig. 1 is a block diagram showing an example of the configuration of the form processing system 100 according to the first embodiment. The form processing system 100 includes an acquisition unit 101 and a determination unit 102.

[0013] The acquisition unit 101 acquires an image of the form being lifted by the robot taken from a direction intersecting the direction in which the form is being lifted. Note that the acquisition unit 101 may also acquire information from other sensors that measure the state of the form being lifted by the robot.

[0014] The robot holds and lifts the document. The robot may lift the topmost document from a location where multiple documents are stacked. The robot may hold the document by using static electricity or suction or ejection of air to attract the document to a flat plate or multiple contacts provided on the robot arm. Alternatively, the robot may hold the document by grasping the edges of the document.

[0015] The determination unit 102 determines whether the lifted forms are in a stack based on the image acquired by the acquisition unit 101. "In a stack" refers to a state in which some of the layers of multiple forms are not separated and are connected together.

[0016] When documents are bundled, this includes, for example, when the documents are fastened together with a fastener. Fasteners include, for example, staples, clips, tape, and files. When documents are bundled, this also includes when the documents are glued together or when static electricity prevents them from separating. Furthermore, when documents are bundled, this may also include when one or more documents are folded. In other words, when documents are not bundled, this means that a sheet, such as a single document, is flat.

[0017] The determination unit 102 determines whether the forms are in a stack using any image recognition method. The determination unit 102 may determine whether the forms are in a stack by recognizing hanging forms based on an image. For example, when a robot holds forms by suction, the robot only picks up the topmost form of the stack of forms, causing the other forms to hang down. Note that if the acquisition unit 101 acquires information from other sensors that measure the state of forms being lifted by the robot, the determination unit 102 may determine whether the forms are in a stack based on this information.

[0018] 2 is a diagram showing an example of bound forms, in which form A and form B are bound together with a staple.

[0019] Here, we will explain an example in which documents are picked up by being attracted to the robot's suction plate. As shown in Figure 2, documents are stacked together, with document A on top and document B on the bottom, and then lifted up by the robot.

[0020] Figure 3 is a diagram showing a scene in which a document is being lifted up, viewed from the side. In Figure 3, for example, the x-axis direction is the horizontal direction along the ground surface, and the y-axis direction is the direction opposite to the direction of gravity. In other words, Figure 3 shows a scene in which a document placed on a horizontal surface is being lifted up vertically.

[0021] As an example, the acquisition unit 101 may acquire an image taken from the z-axis direction shown in Fig. 3. When the determination unit 102 recognizes hanging forms B based on the image, it determines that the forms are bundled. The acquisition unit 101 may acquire an image taken from a direction intersecting with the y-axis direction shown in Fig. 3.

[0022] 3, the scene in which the placed document is lifted vertically has been described, but the lifted document may be set up perpendicular to the ground surface. The acquisition unit 101 may acquire an image of the vertically set document taken from a direction intersecting the direction in which it is being lifted. For example, the acquisition unit 101 may acquire an image in which the y-axis direction in FIG. 3 is the horizontal direction along the ground surface and the x-axis direction is the direction opposite to the direction of gravity.

[0023] FIG. 4 is a flowchart showing an example of the operation of the form processing system 100 according to the first embodiment.

[0024] First, the acquisition unit 101 acquires an image of the documents being lifted by the robot, taken from a direction intersecting the direction of lifting (step S1). The acquisition unit 101 transmits the acquired image to the determination unit 102. Then, the determination unit 102 determines whether the images being lifted are in a stack based on the image (step S2).

[0025] According to the first embodiment, it is determined whether the documents being picked up by the robot are in a stack based on an image taken from a direction intersecting the direction in which the documents are being picked up. Therefore, it is possible to appropriately process the stacked documents.

[0026] It may be difficult to determine whether the documents are stacked from an image of the document surface when the documents are placed on the table. According to the first embodiment, whether the documents are stacked is determined based on an image of the documents being picked up taken from a direction intersecting the direction in which they are being picked up, making it possible to appropriately process the stacked documents.

[0027] Furthermore, when using image recognition to recognize whether documents are stapled, it may be difficult to detect the presence or absence of staples depending on the color or shape of the staples. For example, when non-metallic staples are used, when the staples and the document are similar in color, or when the staples overlap with text written on the document. In these cases, it is difficult to recognize staples from an image of the document. According to the first embodiment, it is possible to determine whether documents are in a stack, even when it is difficult to recognize staples.

[0028] The first embodiment can also be applied to systems other than a form processing system using a robot. For example, the form processing system 100 may be a determination system that determines whether forms picked up by a person are in a stack.

[0029] [Second embodiment] Next, as a second embodiment, another example of the form processing system 100 will be described. In the second embodiment, the form processing system 100 is included in the form sorting system 1. FIG. 5 is a block diagram showing an example of the configuration of the form sorting system 1 according to the second embodiment. Of the configuration of the form processing system 100 according to the second embodiment, the description of the same configuration as the configuration according to the first embodiment will be omitted.

[0030] The document sorting system 1 according to the second embodiment includes a robot 10, a conveying device 20, a camera 30, a camera 31, and a sorting mechanism 40. The document processing system 100 according to the second embodiment includes an acquisition unit 101, a determination unit 102, a recognition unit 103, and a control unit 104.

[0031] The robot 10 operates under the control of the control unit 104. For example, the robot 10 lifts up the placed forms and proceeds to sorting processing. For example, the robot 10 places the forms on the conveying device 20 to proceed to sorting processing. Furthermore, if it is determined that the forms are bundled, the robot 10 stops the operation of proceeding to sorting processing.

[0032] The conveying device 20 conveys the forms to the sorting mechanism 40. The conveying device 20 is, for example, a roller conveyor that conveys the forms by rotating each of rollers arranged in parallel. The conveying device 20 may also be equipped with a conveyor belt.

[0033] The camera 30 photographs the document being lifted by the robot 10 from a direction intersecting the direction in which the document is being lifted.

[0034] The camera 31 photographs the placed form. The camera 31 may photograph a location where a plurality of forms are placed one on top of the other.

[0035] In this embodiment, an example is shown in which the camera 30 and the camera 31 are realized by separate cameras, but the camera 30 and the camera 31 may also be realized by a single camera.

[0036] The sorting mechanism 40 sorts the documents into a plurality of destinations according to their type using existing sorting technology. The types of documents to be sorted into each destination are set in advance by an administrator or the like.

[0037] The sorting mechanism 40 sorts documents into destinations based on their type (contents and size). For example, the sorting mechanism 40 may sort documents by paper size. Documents may also be sorted by their content, such as agency collection forms, slips, gift certificates, receipts, and other documents. Documents may also be sorted based on subsequent processing steps, such as whether or not they are subject to data input, whether or not they are subject to storage, and whether or not they are subject to return to a store. Documents may also be sorted by document name. If the document is a gift certificate, the gift certificate may be further sorted based on whether or not they are subject to cashing. Note that multiple different types of documents may be sorted to the same destination.

[0038] The recognition unit 103 recognizes the placed form. The recognition unit 103 may acquire an image captured by the camera 31 and recognize the placed form based on the image. Alternatively, the recognition unit 103 may receive the result of the camera 31 recognizing the form.

[0039] The recognition unit 103 may recognize the presence or absence of a form. The recognition unit 103 may also recognize the position and size of the form by recognizing, for example, the sides or corners of the form. The recognition unit 103 may also recognize the position and size of the form by recognizing the positions of characters or codes printed on the form.

[0040] The recognition unit 103 recognizes the topmost form among multiple forms stacked one on top of the other. For example, the recognition unit 103 recognizes forms whose edges or printed characters or codes are not hidden by other forms.

[0041] The recognition unit 103 may determine whether the documents are bundled by recognizing whether a staple is attached to the documents based on an image of the documents. For example, the recognition unit 103 may recognize a staple based on an image captured by the camera 31.

[0042] The acquisition unit 101 acquires an image captured by the camera 30. Then, the determination unit 102 determines, based on the image acquired by the acquisition unit 101, whether the forms are bundled.

[0043] The control unit 104 controls the operation of the robot 10 to lift the form based on the recognition result by the recognition unit 103. The control unit 104 controls the position of the robot arm according to the position and size of the form. The control unit 104 also controls the position on a flat plate provided on the robot arm to adsorb the form. The control unit 104 controls the operation of the robot 10 to proceed with the form sorting process. In order to proceed with the form sorting process, the control unit 104 controls the robot 10 to lift the form and place it on the conveying device 20.

[0044] Furthermore, if it is determined based on the determination result by the determination unit 102 that the forms are bundled, the control unit 104 controls the robot 10 to stop the operation of proceeding with the sorting process for the forms.

[0045] When it is determined that the forms are in a stack, the control unit 104 may control the robot 10 to place the stack of forms in an error form storage area. Placing the form in the error form storage area enables the robot 10 to pick up the next form that was placed below the form in question.

[0046] The camera 31 and the recognition unit 103 may be provided as needed. For example, if a form of a specified size is placed at a specified position and the form can be lifted without the camera 31 and the recognition unit 103, the camera 31 and the recognition unit 103 may not be provided.

[0047] The document sorting system 1 according to the second embodiment is a system for sorting documents collected from convenience stores, for example, but the documents to be sorted may be other documents as well, and are not particularly limited.

[0048] FIG. 6 is a diagram showing an example of the arrangement of the robot 10, the conveying device 20, the camera 30, and the camera 31. In the example of FIG. 6, the forms are stacked in baskets. The forms collected from convenience stores are transported together by store. The transported forms are placed in, for example, baskets for sorting. The forms may be placed in different baskets for each store.

[0049] When the forms are placed in the basket and the basket is placed in a predetermined position for sorting, the camera 31 photographs the forms in the basket. The robot 10 removes the forms from the basket and places them on the conveying device 20. The camera 30 photographs the forms being picked up by the robot 10 from a direction intersecting the direction in which they are being picked up. The conveying device 20 determines that the forms are not in a stack based on the image taken by the camera 30, and conveys the forms placed by the robot 10 to the sorting mechanism 40.

[0050] 7 is a diagram showing an example of the arrangement of the conveying device 20 and the sorting mechanism 40. The sorting mechanism 40 sorts the forms into each sorting destination (box X, box Y, box Z) according to the type of the form.

[0051] 8 is a flowchart showing an example of the operation of the form processing system 100 according to the second embodiment. First, the recognition unit 103 recognizes the placed form (step S11). The recognition unit 103 transmits the recognition result to the control unit 104.

[0052] The control unit 104 controls the robot 10 based on the recognition result so that the robot 10 lifts the placed form (step S12).

[0053] The acquisition unit 101 acquires an image of the document being lifted by the robot 10, taken from a direction intersecting the direction in which the document is being lifted (step S13). The acquisition unit 101 transmits the acquired image to the determination unit 102.

[0054] The determination unit 102 determines whether the picked-up documents are a stack based on the image (step S14). The determination unit 102 transmits the determination result to the control unit 104.

[0055] If it is determined that the forms are not bundled (step S14: No), the control unit 104 controls the robot 10 to proceed with the sorting process for the forms (step S15). If it is determined that the forms are bundled (step S14: Yes), the control unit 104 controls the robot 10 to stop the operation of proceeding with the sorting process for the forms (step S16).

[0056] Depending on the machine, stacked documents may not be sorted properly. Also, stacked documents may be separated before being sorted. According to the second embodiment, when the robot 10 proceeds with the sorting process for documents, the determination unit 102 determines whether the documents are stacked. Furthermore, if it is determined that the documents are stacked, the control unit 104 controls the robot 10 so that the robot 10 stops the operation of proceeding with the sorting process for documents. Therefore, stacked documents can be processed properly.

[0057] In the second embodiment, the case where the document sorting system 1 includes the conveying device 20 and the sorting mechanism 40 has been described. However, the document sorting system 1 does not have to include the conveying device 20. For example, the robot 10 may place documents on the sorting mechanism 40. Furthermore, the document sorting system 1 does not have to include the sorting mechanism 40. The robot 10 may sort documents by type according to control by the control unit 104. For example, the robot 10 may sort the documents it picks up into multiple boxes by type.

[0058] [Variations] The above-described embodiments can be modified in various ways, and modifications will now be described.

[0059] The acquisition unit 101 may acquire images of the lifted document taken from multiple directions intersecting the direction in which the document is being lifted. The images from multiple directions may be taken using multiple cameras 30 installed at positions where the document is being lifted. Furthermore, the cameras 30 may take multiple images while the robot 10 rotates the lifted document. Alternatively, images from multiple directions may be taken by moving a single camera 30 around the document.

[0060] The determination unit 102 may determine whether the forms are bundled based on a plurality of images taken from a plurality of directions intersecting the direction in which the forms are lifted.

[0061] The determination unit 102 determines whether the forms are bundled based on a plurality of images photographed from a plurality of directions, so that it becomes possible to determine whether the forms are bundled even when it is difficult to determine from one direction.

[0062] For example, when a lifted document is photographed from a direction intersecting the direction of lifting, only the edge of the hanging document may be photographed, and the page may not be photographed. FIG. 9 is a diagram showing a scene in which a lifted document is viewed from one direction. When an image is photographed from the direction of FIG. 9, if the document is thin, it is difficult to determine from the image whether the document is hanging. However, by using an image photographed from a direction different from the direction of the above-mentioned position, it becomes possible to determine the hanging portion of the document. FIG. 10 is a diagram showing a scene in which the document of FIG. 9 is viewed from another direction. When the determination unit 102 recognizes hanging document B based on the image photographed from the direction of FIG. 10, it determines that the documents are in a stack.

[0063] If the image resolution is sufficient, the determination unit 102 may determine whether the forms are in a stack even if the forms are not hanging down. For example, even if two or more forms are overlapping, the determination unit 102 may determine whether the forms are in a stack by recognizing, based on the image, whether the thickness of the lifted form is equal to or greater than the thickness of two forms. The determination unit 102 may also determine that the forms are in a stack if there is a gap between the robot and the suction surface of the form.

[0064] [Hardware configuration] In each of the above-described embodiments, each component of the form processing system 100 represents a functional block. Some or all of the components of the form processing system 100 may be realized by any combination of a computer 500 and a program.

[0065] Fig. 11 is a block diagram showing an example of the hardware configuration of a computer 500. Referring to Fig. 11, the computer 500 includes, for example, a processor 501, a read only memory (ROM) 502, a random access memory (RAM) 503, a program 504, a storage device 505, a drive device 507, a communication interface 508, an input device 509, an input / output interface 511, and a bus 512.

[0066] The processor 501 controls the entire computer 500. The processor 501 may be, for example, a CPU (Central Processing Unit). The number of processors 501 is not particularly limited, and there may be one or more processors 501.

[0067] The program 504 includes instructions for implementing each function of the form processing system 100. The program 504 is stored in advance in the ROM 502, RAM 503, or storage device 505. For example, the processor 501 of the form processing system 100 executes the instructions included in the program 504 to implement the functions of the form processing system 100. The RAM 503 may also store data to be processed in each function of the form processing system 100.

[0068] The drive device 507 reads and writes data from and to the recording medium 506. The communication interface 508 provides an interface with a communication network. The input device 509 is, for example, a mouse or a keyboard, and receives information input from an administrator or the like. The output device 510 is, for example, a display, and outputs (displays) information to an administrator or the like. The input / output interface 511 provides an interface with peripheral devices. The bus 512 connects these hardware components. The program 504 may be supplied to the processor 501 via a communication network, or may be stored in advance on the recording medium 506, read by the drive device 507, and supplied to the processor 501.

[0069] It should be noted that the hardware configuration shown in FIG. 11 is an example, and other components may be added, or some components may not be included.

[0070] There are various variations in the method of realizing the form processing system 100. For example, the form processing system 100 may be realized by any combination of different computers and programs for each component. Furthermore, multiple components included in the form processing system 100 may be realized by any combination of a single computer and program.

[0071] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, the configurations in the respective embodiments can be combined with each other without departing from the scope of the present disclosure. [Explanation of symbols]

[0072] 100 Report Processing System 101 Acquisition Department 102 Judgment section 103 Recognition part 104 Control Unit 1. Document sorting system 20. Conveyor 30, 31 Camera 40 Sorting mechanism

Claims

1. An acquisition means for acquiring an image of a document being picked up by a robot from a location where multiple documents are placed in order to sort them one by one, taken from a direction intersecting the direction in which the document is being picked up; a determining means for determining whether the lifted documents are in a stack based on the image; A document processing system comprising:

2. The determining means determines whether the documents are bundled by recognizing the hanging documents based on the image. The document processing system according to claim 1 .

3. The determining means determines whether the documents are bundled based on a plurality of images taken from a plurality of directions intersecting the direction in which the documents are lifted.

3. A form processing system according to claim 1.

4. The robot lifts the form by suction. The form processing system according to any one of claims 1 to 3.

5. A recognition means for recognizing a placed document; a control means for controlling the operation of the robot to lift the placed document and proceed to a sorting process based on the recognition result; Equipped with When it is determined that the documents are bundled, the control means controls the robot to stop the operation of proceeding the documents to the sorting process. The form processing system according to any one of claims 1 to 4.

6. the operation is an operation of placing the form on a conveying device that conveys the form to a sorting mechanism; The document processing system according to claim 5.

7. The recognition means determines whether the documents are bundled by recognizing whether a fastener is fastened to the documents based on an image of the documents.

7. A form processing system according to claim 5 or 6.

8. The control means controls the robot so as to place the bundle of forms in an error form storage area. The form processing system according to any one of claims 5 to 7.

9. A computer comprising: The robot picks up a document from a location where multiple documents are placed, in order to sort them one by one. The robot captures an image of the document from a direction intersecting the direction in which the document is being picked up. Based on the image, it is determined whether the lifted documents are in a stack. How to process documents.

10. A method for obtaining an image of a document being picked up by a robot from a location where multiple documents are placed in order to sort them one by one, the image being taken from a direction intersecting the direction in which the document is being picked up; Based on the image, it is determined whether the lifted documents are in a stack. A program that causes a computer to perform a process.

Images