Image processing system and program

Abstract

To enable a specification of a size of each script from a photographing image generated by photographing a plurality of scripts mounted on a script stage to be arranged at once without designation by a user.SOLUTION: A compound machine 10 includes: a photographing part 12 that performs a previous scanning by photographing a script on a script stage by using a painting and calligraphic work camera before a scanning instruction by a user is made in the case where the user wants to scan a plurality of scripts while being mounted onto the script stage; a full-script mounting region extraction part 131 that extracts a full-script mounting region where all of the scripts on the script stage is mounted by the previous scanning; an unfixed region extraction part 132 that extracts a mounting region after the mounting region of the script stage to which the mounting region is fixed is removed as an unfixed script region from the full-script mounting region; and a script region fixing part 133 that fixes the unfixed script region to be extracted as a mounting region of the script that is newly mounted onto the script stage.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing system and a program. [Background technology]

[0002] Nowadays, instead of a document reading device generally called a scanner, it is possible to use, for example, a camera of a mobile terminal to capture an image. For example, it is possible to print an image captured by the camera of a mobile terminal by sending the image to a printer.

[0003] The same is true for multifunction copiers, where a method is being considered of equipping a photographing means such as a document camera and using the photographed image as the scanned image. When a document camera is installed in a multifunction copier, the user places the document on the document table with the printed side facing up. The document camera scans the document on the document table by photographing it from above. When a document camera is used, there is no need to open and close the platen cover when setting the document on the document table, as with conventional multifunction copiers. It is also convenient for scanning the pages of three-dimensional objects such as books.

[0004] Incidentally, there have been multifunction machines that can provide a multi-crop function for some time now. The multi-crop function is a function that allows multiple documents, such as business cards and receipts, to be placed on the document tray of the multifunction machine, scanned all at once, and the scanned image is divided and cut out for each document, and the data is saved individually.

[0005] Conventionally, a technique has been proposed in which multiple documents are placed on a document tray closely spaced from one another, then scanned together, the scanned images are divided into unit sizes designated in advance, and the divided images are saved as individual files (for example, Patent Document 1). In order to detect each document placed on the document tray individually, it was necessary to place the documents with gaps between them, but Patent Document 1 allows the user to specify the unit size in advance, making it possible to place the documents without gaps between them. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2008-035081 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the conventional technology, it is necessary for the user to specify the unit size, that is, the size of the document, in advance.

[0008] To provide a method for identifying the size of each document from a captured image generated by simultaneously capturing a plurality of documents arranged and placed on a document table, without requiring a user to specify the size of each document. [Means for solving the problem]

[0009] The image processing system of the present invention is equipped with an imaging means and a processor, and when the imaging means photographs the top of a document table and generates the photographed image as a current photographed image, if a change is detected in the current photographed image from the photographed image as the immediately previous photographed image generated by the immediately previous photographing, the processor analyzes the currently photographed image to identify the placement area of ​​all documents placed on the document table as the total document placement area, and determines the placement area remaining after removing from the total document placement area the placement area of ​​documents whose placement areas have already been determined as the placement area of ​​one document placed on the document table between the generation of the immediately previous photographed image and the generation of the current photographed image.

[0010] The processor is further characterized in that, when the total document placing area identified from the current captured image is set as the current total document placing area and the total document placing area identified from the immediately previous captured image is set as the immediately previous total document placing area, if the relationship between the immediately previous total document placing area and the current total document placing area satisfies a predetermined condition that allows it to be determined that the one document has been placed overlapping another document placed on the document table, the processor corrects the placing area of ​​the one document whose placing area has been determined to include a difference area between the immediately previous total document placing area and the current total document placing area.

[0011] The predetermined condition is characterized in that the difference area is adjacent to the immediately preceding entire document placement area, and the ratio of the difference area to the immediately preceding entire document placement area is equal to or smaller than a predetermined threshold value.

[0012] In addition, when the processor determines the total document placing area identified from the currently captured image as the current total document placing area and the total document placing area identified from the immediately previous captured image as the immediately previous total document placing area, by comparing the immediately previous total document placing area with the current total document placing area, if it detects that the placing area of ​​a document whose placing area has already been determined has moved, it performs processing according to the state of the document after the movement.

[0013] In addition, when the size of the placement area for the moved documents has changed, the processor notifies the user of the overlap of the documents.

[0014] The processor is also characterized in that, if the size of the placement area for the moved document has not changed, the placement area for the document is updated to the placement area after the movement.

[0015] The program of the present invention enables a computer to realize the following functions: when an imaging means captures the top of a document table and generates a captured image as a current captured image, if a change is detected in the current captured image from the captured image generated by the previous capture, the computer analyzes the current captured image to identify the placement area of ​​all documents placed on the document table as the total document placement area; and the computer determines the placement area remaining after removing the placement areas of documents whose placement areas have already been determined from the total document placement area as the placement area of ​​one document placed on the document table between the previous captured image and the current captured image. Effect of the Invention

[0016] According to the invention described in claim 1, the size of each document can be identified from a captured image generated by simultaneously photographing multiple documents placed side by side on a document table without the user having to specify the size.

[0017] According to the invention described in claim 2, even when a user shifts one document placed overlapping another document to eliminate the overlap, the placement area of ​​the one document can be correctly identified.

[0018] According to the invention as recited in claim 3, it is possible to appropriately determine whether the overlapping of documents has been eliminated by the user.

[0019] According to the invention as recited in claim 4, it is possible to take some kind of process when a document with a fixed placement area is moved.

[0020] According to the invention as recited in claim 5, it is possible to allow the user to take measures to eliminate overlapping of documents.

[0021] According to the invention as set forth in claim 6, it is possible to determine that the document has simply been moved on the document platen, and to permit the document movement, thereby updating the document placement area.

[0022] According to the invention described in claim 7, the size of each document can be identified from a captured image generated by simultaneously photographing multiple documents placed side by side on a document table without requiring the user to specify the size. [Brief description of the drawings]

[0023] [Figure 1] 1 is a block configuration diagram of a multifunction peripheral forming an image processing system in accordance with a first embodiment. [Diagram 2] 1 is a schematic perspective view of a multifunction peripheral according to a first embodiment. [Diagram 3] 2 is a hardware configuration diagram of the multifunction peripheral according to the first embodiment. FIG. [Figure 4] 5 is a flowchart showing a document image storage process according to the first embodiment. [Diagram 5] 5A to 5C are diagrams showing a state of a document placed on a document platen and a transition of a detected document area during prescanning in the first embodiment. [Figure 6] 5A to 5C are diagrams showing the state of a document placed on a document platen and the transition of a detected document area during main scanning in the first embodiment. [Figure 7] FIG. 11 is a block diagram showing the configuration of a multifunction peripheral forming an image processing system according to a second embodiment. [Figure 8] 13 is a flowchart showing a document image storage process according to the second embodiment. [Figure 9] 13A to 13C are diagrams showing the state of a document placed on a document platen and the transition of a detected document area during prescanning in the second embodiment. [Figure 10] 13A to 13C are diagrams showing the state of a document placed on another document platen during prescanning in the second embodiment and the transition of a detected document area. [Figure 11] 13 is a flowchart showing a modified example of the document image storage process in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0025] Embodiment 1 Fig. 1 is a block diagram of a multifunction device 10 forming an embodiment of an image processing system according to the present invention. Fig. 2 is a schematic perspective view of the multifunction device 10 in this embodiment, and Fig. 3 is a hardware configuration diagram of the multifunction device 10 in this embodiment.

[0026] The multifunction device 10 in this embodiment is characterized in that, as part of its device configuration, it is equipped with a document camera 21 as shown in Fig. 2. The multifunction device 10 has a document table 23 provided on the top surface of a device main body 22, and is provided with a document camera 21 that can capture the entire document table 23. The document camera 21 is supported by a support rod 24. Also, on the top surface of the device main body 22, there is provided an operation panel 25 that functions as an input means operated by the user and as a display means for displaying information.

[0027] The multifunction device 10 in this embodiment is a form of an image forming device equipped with various functions such as a printing function, a copying function, a scanning function, etc., and is a device with a built-in computer. That is, as shown in Fig. 3, the multifunction device 10 includes a CPU 31, a ROM 32, a RAM 33, an HDD (Hard Disk Drive) 34 as a storage means, the operation panel 25 as a user interface, the document camera 21 as an image capturing means, a printer 35 that provides a printing function, and a network interface (IF) 36 as a communication means. An address data bus 37 connects the various mechanisms 31 to 36, 21, and 25 that are controlled by the CPU 31, and communicates data.

[0028] Incidentally, conventional multifunction machines are generally equipped with a scanner that optically reads the document 2 on the document table 23 to realize a scanning function. The multifunction machine 10 in this embodiment is equipped with a document camera 21 instead of a scanner. Therefore, the read image of the document 2 is generated from a captured image obtained by capturing an image with the document camera 21, or by analyzing the captured image. The document camera 21 generates a captured image of the document 2 by capturing an image of the document 2 on the document table 23, but since it can also be said that the document 2 is read by capturing an image, in this embodiment, the terms "capturing" and "reading" are used synonymously without being strictly distinguished from each other. Therefore, by replacing "capturing" by the document camera 21 with "reading", "capturing" by the document camera 21 may also be called "reading" or "scanning".

[0029] 1, a multifunction device 10 in this embodiment includes a user interface (UI) unit 11, a photographing unit 12, an image processing unit 13, an original image output unit 14, a control unit 15, and an original area information storage unit 16. Note that components that are not used in the description of this embodiment are omitted from the figure.

[0030] The user interface unit 11 is realized by an operation panel 25, and displays various menu screens and the like in the multifunction device 10. The user interface unit 11 also accepts user operations on the displayed screen. The photographing unit 12 is realized by a document camera 21, and photographs the entire top surface of the manuscript table 23 as its photographing range. For the sake of convenience in this embodiment, it is assumed that the photographing range of the photographing unit 12 and the area of ​​the top surface of the manuscript table 23 match. Even if they do not match, it is possible for the image processing unit 13 to perform image processing to extract an image of only the top surface of the manuscript table 23 from the photographed image.

[0031] The image processing unit 13 performs image processing on the image captured by the document camera 21. The image processing unit 13 has a total document placement area extraction unit 131, an undetermined area extraction unit 132, a document area determination unit 133, an area division unit 134, and a document image generation unit 135.

[0032] The whole document placement area extraction unit 131 extracts the area where the document 2 is placed from the captured image obtained from the photographing unit 12. The document 2 has a size called the paper size, as a matter of course. When the document 2 is placed on the document table 23, the entire surface of the document 2 abuts against the upper surface of the document table 23, and occupies a placement area of ​​the upper surface of the document table 23 that is equal to the paper size. Of course, the document 2 may be placed on top of another document 2, or may be placed so as to protrude from the document table 23. In this embodiment, the document 2 is placed on the document table 23, and the placed document 2 comes into contact with the upper surface of the document table 23, and the range (or area) that occupies the upper surface of the document table 23 is referred to as the "document placement area" or, for short, the "document area". When the entire document is in contact with the upper surface of the document table 23, the paper size of the document 2 and the document area are the same size.

[0033] When multiple documents 2 are placed, the total document placement area extraction unit 131 extracts the placement area of ​​the multiple documents 2, i.e., all documents 2 placed on the document table 23. Therefore, in this embodiment, the placement area of ​​the documents 2 on the upper surface of the document table 23 extracted by the total document placement area extraction unit 131 is referred to as the "total document placement area." Note that even when only one document 2 is placed, the range in which that one document 2 is placed corresponds to the total document placement area.

[0034] The unconfirmed area extraction unit 132 extracts an unconfirmed original area by removing the confirmed original area from the entire original placement area extracted by the entire original placement area extraction unit 131. The "confirmed original area" refers to the original area of ​​the original 2 in which the range of the position where the original 2 is placed on the original table 23 (i.e., the original area) has been confirmed by a prescan performed prior to the actual scan, as will be described in detail later. Strictly speaking, even if the original area of ​​the original 2 has been confirmed by the immediately preceding prescan, there is an exceptional possibility that the confirmed original area may be corrected. This correction will be described later.

[0035] The manuscript area determination unit 133 determines the undetermined manuscript area extracted by removing the confirmed manuscript area from the entire manuscript placement area as the manuscript area of ​​the newly detected manuscript 2. The undetermined manuscript area becomes a confirmed manuscript area as a result of the determination by the manuscript area determination unit 133. By determining the manuscript area, area information that can identify the manuscript area is stored and managed in the manuscript area information storage unit 16. The area information that can identify the manuscript area can be represented, for example, by coordinate values ​​in two-dimensional coordinates in which the top surface of the manuscript table 23 corresponds to XY coordinates and one corner of the manuscript table 23 is the origin of the XY coordinates.

[0036] The area division unit 134 divides the entire document placement area extracted by the unconfirmed area extraction unit 132 by the confirmed document areas when the main scan is performed. The document image generation unit 135 generates a read image of each document 2 (hereinafter also referred to as an "original image") by cutting out a captured image of each document 2 from the captured image of the entire document placement area according to the areas formed by division by the area division unit 134.

[0037] The document image output unit 14 outputs the scanned image of each document 2 generated by the document image generation unit 135 to a predetermined output destination. The output destination may be, for example, a predetermined storage means, an operation panel 25, or a display means of another computer. The control unit 15 controls the operations of the above components 11 to 14.

[0038] The document area information storage unit 16 stores area information corresponding to the document area of ​​the document 2 determined by the document area determination unit 133 during prescanning.

[0039] Each of the components 11 to 15 in the multifunction device 10 is realized by the cooperative operation of a computer mounted on the multifunction device 10 and a program running on a CPU 31 mounted on the computer. Also, the document area information storage unit 16 is realized by an HDD 34 mounted on the multifunction device 10. Alternatively, the RAM 33 or an external storage means may be used via a network.

[0040] Furthermore, the programs used in the present embodiment can be provided by communication means, and can also be provided by storing them in a computer-readable recording medium such as a USB memory. The programs provided from the communication means or recording medium are installed in a computer, and various processes are realized by the CPU 31 of the computer sequentially executing the programs.

[0041] Furthermore, although the multifunction device 10 constituting the image processing system is illustrated as a single device, it may be realized by combining a plurality of computers, information processing devices, or image forming devices.

[0042] Next, the operation of this embodiment will be described.

[0043] In the present embodiment, even when a plurality of documents 2 are placed on the document table 23 and then scanned collectively, the scanned images of all documents 2 can be individually generated and saved without requiring the user to specify the paper size of each of the documents 2 or a paper size common to all of the documents 2, or setting conditions or restrictions such as providing gaps between the documents 2 placed. The document image saving process in this embodiment will be described below with reference to the flowchart shown in FIG. 4. At the time when the document image saving process shown in FIG. 4 is started, the document area information storage unit 16 does not store area information. The document area information storage unit 16 may be initialized at an appropriate timing, such as at the start of the document image saving process or at the end of the main scan. In addition, the case where two documents A and B are scanned collectively as the plurality of documents 2 will be described as an example.

[0044] When a user uses the scan function of the multifunction device 10, the user logs in to the multifunction device 10 or operates the home screen or the like displayed on the operation panel 25. The document camera 21 starts capturing images at regular time intervals in response to this user operation. Note that the capturing of images by the document camera 21 that starts upon login or the like is called "pre-scanning" because it starts before a scan is instructed by the user. On the other hand, the scan performed to obtain a scanned image of the document 2 in response to a scan instruction from the user is called "main scanning."

[0045] The user places the originals 2 on the platen 23 before or after the start of prescanning. At this time, the user places the originals 2 one by one so that the printed surface to be read by the document camera 21 faces upward.

[0046] If the user does not issue an instruction for main scanning (N in step S101), the image capturing unit 12 performs a pre-scan using the document camera 21 (step S102). Since the user issues an instruction for main scanning after placing the documents 2 to be scanned all at once on the document table 23, the image capturing unit 12 repeatedly performs pre-scanning using the document camera 21 until the user has placed all of the documents 2, as shown in FIG.

[0047] FIG. 5 is a diagram used to explain the processing during prescanning. Each of FIG. 5(a)-(f) is illustrated in a set of an upper row and a lower row. The upper row is an image captured by prescanning, and shows the state in which the original 2 is placed on the platen 23 when prescanned. The lower row shows the original area obtained from the captured image shown in the upper row. Specifically, FIG. 5(a)-(c) shows the processing for obtaining a confirmed original area 53A from an image captured in one prescan. Similarly, FIG. 5(d)-(f) also shows the processing for obtaining a confirmed original area 53B from an image captured in one prescan.

[0048] Incidentally, when a user places the document 2 on the document table 23 with a time interval equal to or greater than the time interval of the prescan performed by the document camera 21, there is basically no change between an image captured at a certain time and an image captured immediately afterward. Therefore, even if prescanning is repeatedly performed at regular intervals, if no change is detected in the image captured by the prescan performed at the present time (hereinafter, the "currently captured image") from the image captured by the immediately preceding prescan (hereinafter, the "previously captured image"), no special processing is required in the document image storage processing of this embodiment. Therefore, the processing during prescanning shown in FIG. 4 (steps S102 to S105) is performed when a change is detected between the currently captured image and the immediately preceding captured image.

[0049] First, when the total document placement area extraction section 131 in the image processing section 13 acquires a captured image from the imaging section 12, it analyzes the captured image (the "current captured image" at this point in time) to identify the total document placement area, which is the combined document areas of all documents 2 placed on the document table 23, from the current captured image (step S103). This makes it possible to easily extract the total document placement area from the captured image by obtaining the difference between, for example, the captured image of the top surface of the document table 23 when no documents 2 are placed and the captured image in the current prescan.

[0050] The upper part of Fig. 5(a) shows a state in which only one original 2a ("original A" in Fig. 5) is placed on the original table 23. The corresponding lower part of Fig. 5(a) shows a total original placement area 51 in this placement state. At this point, since only one original 2a is placed on the original table 23, the total original placement area 51 is equal to the area in which the original 2a is placed. Note that in Fig. 5, the total original placement area 51 is shown with a dot pattern. The same applies to Figs. 6 and 9 shown below.

[0051] Next, the undetermined area extraction unit 132 retrieves area information of the confirmed manuscript area from the manuscript area information storage unit 16, and extracts the undetermined manuscript area 52 by removing the confirmed manuscript area, i.e., the manuscript area of ​​manuscript 2 whose manuscript area has been confirmed, from the entire manuscript placement area 51 by referring to the area information (step S104).

[0052] In FIG. 5(a), the original 2a is the original 2 placed on the original table 23 first. If the captured image shown in the upper part of FIG. 5(a) is the current captured image, the immediately preceding captured image at this point is the image captured when the original 2 is not placed on the original table 23. No confirmed original area is additionally registered from the immediately preceding captured image. Therefore, there is no confirmed original area corresponding to the area information that the unconfirmed area extraction unit 132 extracts from the original area information storage unit 16. Therefore, as shown in FIG. 5(b), the entire original placement area 51 corresponds to the unconfirmed original area 52. In FIG. 5, the unconfirmed original area 52 is illustrated with a lattice pattern. The same applies to FIG. 9 shown below.

[0053] Next, the document area determination unit 133 determines the undetermined document area 52 extracted by the undetermined area extraction unit 132 as the document area of ​​one document 2 placed on the document table 23 between the generation of a photographed image when the document 2 was not placed (i.e., the immediately preceding photographed image at this time) and the generation of a photographed image shown in the upper part of FIG. 5(a) (i.e., the current photographed image at this time). That is, the undetermined document area 52 extracted by the undetermined area extraction unit 132 becomes a confirmed document area 53A for the document 2a. In FIG. 5, the confirmed document area 53 is illustrated as being filled in grayscale. The same applies to FIGS. 6, 9, and 10 shown below.

[0054] The image processing unit 13 additionally registers area information that identifies the confirmed document area 53A in the document 2a in the document area information storage unit 16 (step S105). After that, the process proceeds to step S101.

[0055] Since the user wishes to scan two originals 2 together, he or she subsequently places the second original 2b on the platen 23. At this point, no instruction for actual scanning is given. In this case (N in step S101), the photographing unit 12 performs a prescan using the document camera 21 (step S102). The captured image obtained by this prescan is shown in the upper part of FIG. 5(d). As a result of this prescan, the currently captured image is the captured image shown in the upper part of FIG. 5(d), and the immediately preceding captured image is the captured image shown in the upper part of FIG. 5(a).

[0056] As described above, when the total document placement area extraction unit 131 acquires an image captured by the document camera 21, it extracts from the captured image a total document placement area 51 that is the combined document area of ​​all documents 2 placed on the document table 23 (step S103). As shown in the lower part of Fig. 5(d), the total document placement area 51 at this time is the document area that combines documents 2a and 2b ("document B" in Fig. 5). Here, documents 2a and 2b are placed with no gaps between them, so the total document placement area 51 is recognized as a single document area having the size of two documents 2.

[0057] Next, the undetermined area extraction unit 132 extracts the undetermined original area 52 by removing the original area of ​​the confirmed original 2 from the entire original placement area 51 in the same manner as described above (step S104). Here, as shown in the lower part of FIG. 5(e), since the confirmed original area 53A corresponding to the original 2a exists, the undetermined area extraction unit 132 determines that the original area of ​​the original 2b newly placed on the platen 23 corresponds to the undetermined original area 52.

[0058] Next, the document area determination unit 133 determines the unconfirmed document area 52 extracted by the unconfirmed area extraction unit 132 as the document area of ​​one document 2 placed on the document table 23 between the generation of the photographed image shown in FIG. 5(a) (i.e., the immediately preceding photographed image at this time) and the generation of the photographed image shown in the upper part of FIG. 5(d) (i.e., the current photographed image at this time). That is, the unconfirmed document area 52 extracted by the unconfirmed area extraction unit 132 becomes the confirmed document area 53B for the document 2b. The image processing unit 13 additionally registers area information specifying the confirmed document area 53B for the document 2b in the document area information storage unit 16 (step S105). As a result, the document area information storage unit 16 registers area information of the confirmed document area 53A corresponding to the document 2a at this time and area information of the confirmed document area 53B corresponding to the document 2b. Then, the process proceeds to step S101.

[0059] The user places two originals 2a and 2b that the user wants to scan together, and instructs a main scan by performing a predetermined operation on the operation panel 25. In this case (Y in step S101), the photographing unit 12 performs a main scan using the document camera 21 (step S111). Note that the scanning process itself by the document camera 21 is the same for pre-scanning and main scanning.

[0060] Next, when the total document placement area extraction unit 131 acquires the image captured by the document camera 21, it identifies the total document placement area, which is the combined document areas of all documents 2 placed on the document table 23, from the captured image, in the same manner as in the pre-scan (step S112).

[0061] Fig. 6 is a diagram used to explain the processing during the main scan. As in Fig. 5, Fig. 6(a) and (b) also show a set of an upper row and a lower row. Fig. 6(a) is the same as Fig. 5(d) for both the upper and lower rows. As shown in the lower row of Fig. 6(a), the total document placement area 51 is the size of the document area including both the document areas of the documents 2a and 2b.

[0062] Next, the area dividing unit 134 extracts area information of the confirmed original area from the original area information storage unit 16, and divides the entire original placement area 51 by the confirmed original area corresponding to the area information. This identifies the original area of ​​each original 2a, 2b (step S113). The lower part of FIG. 6(b) shows the divided and confirmed original area of ​​each original 2a, 2b. Both the upper and lower parts of FIG. 6(b) are the same as FIG. 5(f).

[0063] Once the document areas of all documents 2a, 2b placed on the document table 23 have been determined in the above manner, the document image generating unit 135 extracts the read images of each document 2a, 2b by cutting out the captured images of each document 2a, 2b from the captured image by the document camera 21 or from the captured image of the entire document placement area 51 (step S114).

[0064] Then, the original image output unit 14 stores the scanned images 61a, 61b of each original 2a, 2b extracted and generated by the original image generation unit 135 in a predetermined output destination, for example, a storage means designated as the output destination, as shown in FIG. 6(c) (step S115).

[0065] As described above, in this embodiment, even if a plurality of documents 2a and 2b are read by one scan, the read image of the document 2 can be saved for each document 2. In particular, even if the entire document placement area 51 is detected as a single area because the plurality of documents 2a and 2b are placed side by side with no gaps as shown in Fig. 6(a), the read images of the plurality of documents 2 placed on the document table 23 can be saved for each document 2. For example, even if the A4 size documents 2a and 2b are placed side by side with no gaps, in this embodiment, they can be correctly determined as two A4 size documents 2, not as one A3 size document.

[0066] Embodiment 2 As described above, the user places multiple documents 2 one by one on the document table 23 in order, and then issues an instruction to scan them all at once (the above-mentioned "main scan"). However, the user is not necessarily able to place the documents 2 at the desired positions on the document table 23 without shifting them. For example, a case may occur in which the next document 2 to be placed overlaps a document 2 already placed. In this case, the user is likely to shift the document 2 once placed on the document table 23 to eliminate the overlap. Also, a document 2 already placed, that is, a document 2a or document 2b whose document area has already been determined as described in the above-mentioned first embodiment, may be shifted. In this embodiment, it is possible to handle such a case.

[0067] 7 is a block diagram of a multifunction device 10 forming an image processing system in this embodiment. The multifunction device 10 in this embodiment has a configuration in which a notification unit 17 is added to the multifunction device 10 shown in the first embodiment. In addition, a document area correction unit 136 and a document overlap detection unit 137 are newly added to the image processing unit 13.

[0068] When overlapping of documents 2 is detected, the notification unit 17 notifies the user of that fact. The notification unit 17 is realized by cooperative operation between a computer mounted on the multifunction device 10 and a program running on a CPU 31 mounted on the computer. When a document area correction unit 136 determines that it is necessary to correct the confirmed document area due to documents 2 being placed overlapping, it corrects the confirmed document area. A document overlap detection unit 137 detects overlapping of documents 2 whose document areas have been confirmed.

[0069] The hardware configuration of the multifunction device 10 in this embodiment may be the same as that in the first embodiment shown in FIGS.

[0070] The document image saving process in this embodiment will be described below with reference to the flowchart shown in Fig. 8. The same process steps as those in the document image saving process in the first embodiment shown in Fig. 4 are given the same reference numerals, and the description will be omitted as appropriate. In this embodiment, too, as in the first embodiment, an example will be described in which two documents 2a and 2b are scanned together.

[0071] Fig. 9 is a diagram used to explain the processing during prescanning, and corresponds to Fig. 5 used in the first embodiment. In the present embodiment, the processing for the first original 2a may be the same as the processing explained in the first embodiment using Figs. 5(a) to (c), and is therefore omitted in Fig. 9. However, when explained in conjunction with the flowchart shown in Fig. 8, as shown in Fig. 5(a), when the first original 2a is placed on the original table 23, there is no confirmed original area (N in step S101, S102 to S104). Therefore, the undetermined document area 52 (denoted as "X" in FIG. 8 for convenience) corresponding to the document 2a shown in FIG. 5(b) is not adjacent to the confirmed document area (denoted as "Y" in FIG. 8 for convenience), in other words, there is no adjacent confirmed document area (N in step S201), so the image processing unit 13 additionally registers area information specifying the confirmed document area 53A in the document 2a in the document area information storage unit 16 as in the first embodiment (step S105). Then, the process proceeds to step S101.

[0072] Next, suppose that the user attempts to place document 2b next to document 2a with no gap between them as shown in Fig. 5(d) (or Fig. 9(d)), but ends up placing them so that they slightly overlap as shown in the upper part of Fig. 9(a). When document 2b is placed overlapping document 2a, the overlapping portion of document 2a with document 2b is not photographed by document camera 21. This embodiment is an embodiment for solving the problem of documents 2 overlapping each other.

[0073] If the entire document placing area 51 identified from the currently captured image shown in the upper part of Fig. 9(a) is referred to as the "current entire document placing area" and the entire document placing area 51 identified from the immediately preceding captured image shown in the upper part of Fig. 5(a) is referred to as the "previous entire document placing area", in this embodiment, when the relationship between the immediately preceding entire document placing area and the current entire document placing area satisfies a predetermined condition for determining that one document (at this point, document 2b) has been placed overlapping another document (i.e., document 2a) placed on document table 23, document 2b is determined to have been placed overlapping document 2a. In this embodiment, the predetermined condition is exemplified in steps S201 to S202.

[0074] 9(b), the undetermined area extraction unit 132 extracts the document area of ​​the document 2b as an undetermined document area 52 (step S104). In this embodiment, the undetermined document area 52 is positioned as a difference area between the previous entire document placement area and the current entire document placement area. Since the document 2b overlaps with the document 2a, the undetermined document area 52 is narrower than the actual paper size of the document 2b.

[0075] Incidentally, the undetermined document area 52 corresponding to the document 2b is adjacent to the document area 53A of the confirmed document 2a without forming a gap. In this case (Y in step S201), the image processing unit 13 then calculates the ratio of the undetermined document area 52 (i.e., the difference area) to the size of the immediately preceding all document placement area (i.e., the confirmed document area 53A). Here, if the size of the undetermined document area 52 is not equal to or smaller than a predetermined threshold with respect to the size of the immediately preceding all document placement area (N in step S202), the document area determination unit 133 determines the undetermined document area 52 as the document area of ​​the newly placed document 2b. Details of the threshold will be described later. Then, the image processing unit 13 additionally registers area information specifying the confirmed document area 53B in the document 2b in the document area information storage unit 16 (step S105). Then, the process proceeds to step S101. From the above description, it can be easily assumed that the confirmed document area 53B needs to be corrected.

[0076] Here, the user realizes that he has placed the document 2b on top of the document 2a, and moves the document 2b slightly on the document table 23 to eliminate the overlap between the documents 2a and 2b, as shown in the upper part of Fig. 9(d). Fig. 9(d), which shows the state after the user has eliminated the overlap, is the same as Fig. 5(d). Note that, as is clear from this user operation, the user has not placed a new document 2 on the document table 23.

[0077] By shifting this original 2b and then performing a prescan, the total original placement area 51 shown in the lower part of Figure 9(a) becomes the previous total original placement area, and the total original placement area 51 shown in the lower part of Figure 9(d) becomes the current total original placement area.

[0078] Next, the undetermined area extraction unit 132 extracts area information of the document areas 53A and 53B of the confirmed documents 2a and 2b from the document area information storage unit 16, and removes the document areas 53A and 53B corresponding to the area information from the entire document placement area 51 shown in the lower part of FIG. 9(d) to extract the undetermined document area 52 as shown in the lower part of FIG. 9(e) (step S104). The undetermined document area 52, in other words the difference area between the previous entire document placement area and the current entire document placement area, is actually a part of the document 2b that appears due to a user operation to eliminate overlap, but with only the basic processing described in the first embodiment, it is processed as if a new document has been placed. In other words, the confirmed document area 53B and the undetermined document area 52 shown in FIG. 9(e) are processed as different documents 2.

[0079] Therefore, in this embodiment, the judgment is made based on the above-mentioned predetermined condition. That is, the undetermined document area 52 shown in FIG. 9(e) is adjacent to the determined document area 53B without forming a gap. In this case (Y in step S201), the image processing unit 13 then calculates the ratio of the undetermined document area 52 (i.e., the difference area) to the size of the immediately preceding total document placement area (i.e., the determined document areas 53A and 53B). Here, if the size of the undetermined document area 52 is not equal to or smaller than a predetermined threshold value with respect to the size of the immediately preceding total document placement area (N in step S202), the image processing unit 13 regards the undetermined document area 52 as the determined document area, and additionally registers area information specifying the determined document area in the document area information storage unit 16 (step S105).

[0080] On the other hand, if the ratio of the undetermined document area 52 to the size of the previous entire document placement area is equal to or smaller than a predetermined threshold (Y in step S202), the document area correction unit 136 corrects the document area 53B of the document 2b by including the difference area between the previous entire document placement area and the current entire document placement area, that is, the undetermined document area 52 obtained from the currently captured image as shown in the lower part of FIG. 9(e), in the document area 53B of the document 2b that was determined in the process during the previous prescan (step S203). In this way, the image processing unit 13 retrieves the document area 53B of the document 2b registered in the document area information storage unit 16, combines it with the undetermined document area 52 to form an undetermined document area obtained from the currently captured image, and additionally registers area information specifying this undetermined document area in the document area information storage unit 16 (step S105). Then, the process proceeds to step S101.

[0081] As described above, in this embodiment, the predetermined condition for determining that one document (document 2b in the above example) has been placed overlapping another document (document 2a in the above example) placed on the document table 23 is the positional relationship between the difference area (undetermined document area 52) between the previous document placing area and the current document placing area and the previous document placing area, specifically, that they are adjacent to each other. This means that if they are not adjacent, it is clear that the document 2 corresponding to the difference area is a separate document 2 from the one or more documents 2 corresponding to the previous document placing area. In other words, the undetermined document area 52 has not been formed by a user operation to eliminate the overlap.

[0082] Also, the ratio of the difference area to the size of the current entire document placement area is compared with a predetermined threshold because if the undetermined document area 52 is formed by a user operation to eliminate overlap, the size of the undetermined document area 52 is considered to be relatively small. In this embodiment, the threshold used as the criterion for determining whether there is an overlap is assumed to be, for example, 50%, but it is not limited to this. For example, it may be set appropriately depending on the size of the document table 23, the size and number of documents 2 to be scanned collectively, etc. Alternatively, it may be set by the user.

[0083] However, when placing the originals 2 on the platen 23, the user may mistakenly grab two originals 2 and place the two originals 2 on top of each other with a slight misalignment. In this case, it is conceivable that the size of the undetermined original area 52 formed by shifting one original 2 may exceed a threshold value of about 50% of the entire previous original placement area. In this embodiment, the above threshold value is set to automatically detect a user operation to eliminate the overlap and automatically correct the determined original area 53, but the user may be notified that the originals 2 may be overlapped, allowing the user to make a decision. Also, a message may be displayed to prompt the user to place the originals 2 again.

[0084] According to this embodiment, even if the user places a new document 2b so that it slightly overlaps the document 2a already placed, the document area 53B of the document 2b can be corrected as described above. This allows the scanned images of the documents 2a and 2b to be correctly stored, and each document 2 can be saved individually.

[0085] Incidentally, in the above-mentioned first embodiment, a basic process for identifying the position of the original 2 placed on the original table 23 by identifying the area on the upper surface of the original table 23 where the original 2 is placed has been described. In the present embodiment, in addition to this basic process, a process for dealing with a case where the user places a new original 2b so that it overlaps another original 2a whose position has already been confirmed, as described above, and the original 2b is shifted to eliminate the overlap has been described. In addition to this, for example, there is a possibility that the original 2 whose original area has already been confirmed may be shifted on the original table 23 due to a user operation or the like. In the present embodiment, this case can also be dealt with.

[0086] FIG. 10 is a diagram used to explain the process during prescanning, and corresponds to FIG. 5 used in the first embodiment. Among these, FIG. 10(a)-(c) corresponds to FIG. 5(d)-(f), and shows that the document areas 53A, 53B of the two documents 2a, 2b have been confirmed. The upper part of FIG. 10(d) shows a case where the document 2b has moved in the direction of the arrow C, that is, in the direction of the document 2a, and overlaps with the document 2a. In this way, the document image saving process that can handle the case where the document 2 is shifted after the document area is confirmed will be described using the flowchart shown in FIG. 11. Note that the same process steps as those included in the document image saving process in the first embodiment shown in FIG. 4 are given the same reference numerals, and the description will be omitted as appropriate.

[0087] 10(a) to (c) are the same as Fig. 5(d) to (f), and therefore the explanation will be omitted. As shown in the upper part of Fig. 10(d), when the document 2b is prescanned in a state where it is shifted and overlaps with the document 2a after the document area is determined (N in step S101, S102), the entire document placement area extraction unit 131 extracts the entire document placement area 51 as shown in the lower part of Fig. 10(d) (step S103).

[0088] Here, the document overlap detection unit 137 compares the previous document placement area 51 shown in the lower part of FIG. 10(a) with the current document placement area 51 shown in the lower part of FIG. 10(d). Since the size of the document 2 does not change physically, the current document placement area 51 being smaller than the previous document placement area 51 means that the document area 53A and the document area 53B overlap as shown in the lower part of FIG. 10(e), resulting in an overlapping area 54 surrounded by a dashed line. In this way, the overlapping area 54 makes the current document placement area 51 smaller than the previous document placement area 51. This allows the document overlap detection unit 137 to detect that one of the documents 2 (document 2b in FIG. 10(d)) has moved and overlapped with another document 2 (document 2a in FIG. 10(d)). Note that even if a change is detected in the captured image on the document table 23, the undetermined document area cannot be extracted in step S104 even if the undetermined area extraction unit 132 removes the document areas 53A and 53B of the confirmed documents 2a and 2b from the current document table 23, so it can be detected that the current document table 51 includes an overlapping area 54. Incidentally, if the size of the current document table 51 becomes the same as that of one of the documents 2, for example, document 2a, it can be determined that document 2b has been removed from the document table 23. Also, if the current document table 51 is larger than the size of the previous document table 51, there is a possibility that document 2b has been moved to eliminate the overlap, as described with reference to FIGS. 8 and 9.

[0089] In this manner, when the current all-original placement area 51 is smaller than the previous all-original placement area 51 (Y in step S212), the original overlap detection unit 137 recognizes that the confirmed original areas 53 overlap. When the original overlap detection unit 137 detects an overlap of the originals 2 in this manner, the notification unit 17 warns the user by displaying a notice to that effect on the operation panel 25 (step S213). The multifunction device 10 maintains the warning state until the overlap of the originals 2 is eliminated (N in step S214). Then, when the overlap of the originals 2 is eliminated by the user adjusting the position of the originals 2b in response to the warning (Y in step S214), the process proceeds to step S101.

[0090] Meanwhile, whether the overlapping of the documents 2 has been eliminated may be confirmed by having the user input, via the operation panel 25, that the documents 2 have been moved to eliminate the overlapping. Alternatively, whether the overlapping of the documents 2 has been eliminated may be confirmed by executing the processes (steps S102 to S212) during prescanning.

[0091] As described above, according to this embodiment, it is possible to detect when the position of an original 2b, whose original area has already been determined, shifts and overlaps with another original 2a, and to issue a warning to the user.

[0092] However, even if the position of the original 2b is shifted, as long as it does not overlap with another original 2a, the original 2b can be scanned correctly and the read image of the original 2b can be obtained normally. Therefore, it is considered that it is not always necessary to issue a warning. Therefore, in this embodiment, when it is detected that the placement area of ​​the original 2b, whose placement area has been confirmed, has moved by comparing the previous entire original placement area specified from the previous captured image with the current entire original placement area specified from the current captured image, the image processing unit 13 performs processing according to the state of the original 2b after the movement.

[0093] In the case of the movement of the document 2b, as described above, there are two possible cases: a case where the document 2b is moved so as to overlap with the document 2a, and a change in the current document placement area becomes smaller than the previous document placement area, and a case where the document 2b is moved so as not to overlap with the document 2a, and the current document placement area and the previous document placement area simply do not match, and no change in size is detected. In the former case, as described above, the notification unit 17 notifies the user of the overlap of the document 2 under the control of the control unit 15. In the latter case, the position of the document 2b on the document table 23 simply moves. Therefore, in this case, the image processing unit 13, under the control of the control unit, may update the area information of the confirmed document area of ​​the document 2b registered in the document area information storage unit 16 with the area information of the document area of ​​the document 2b after the movement. In this case, the notification unit 17 may or may not notify the user of the movement of the document 2b.

[0094] As described above, in this embodiment, for convenience of explanation, the process for eliminating overlapping of documents 2 (FIG. 8) and the process for detecting overlapping of documents 2 (FIG. 11) are illustrated as separate flows, but it is possible to integrate them. Specifically, if the current total document placement area 51 is not smaller than the previous total document placement area 51 (N in step S212), the process may proceed to step S104 shown in FIG. 8 instead of proceeding to step S101.

[0095] In this embodiment, the document camera 21 is mounted on the multifunction device 10 instead of a conventional scanner as a means for reading the original 2. However, by mounting a conventional scanner on the multifunction device 10, it is possible to use the document camera 21 and the scanner together as a means for reading the original 2. As described above, the document camera 21 in this embodiment performs two types of scans, a "pre-scan" and a "main scan", but if the multifunction device 10 is equipped with a conventional scanner, the "main scan" may be performed by the scanner.

[0096] In the above embodiments, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0097] In addition, the operations of the processor in the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. The order of the operations of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate.

[0098] (Additional Note) (((1))) A photographing means and a processor are provided, The processor, When the photographing means photographs the top of the document table and generates a photographed image as a current photographed image, if a change is detected from the current photographed image as a previous photographed image generated by the previous photographing, the current photographed image is analyzed to identify a placement area of ​​all documents placed on the document table as a total document placement area, determining a placement area remaining after removing the placement areas of the documents whose placement areas have already been determined from the entire document placement area as a placement area of ​​one document placed on the document table during a period from when the immediately preceding photographed image is generated to when the current photographed image is generated; 1. An image processing system comprising: (((2))) The processor is characterized in that, when the total document placing area identified from the current captured image is the current total document placing area and the total document placing area identified from the immediately previous captured image is the immediately previous total document placing area, if the relationship between the immediately previous total document placing area and the current total document placing area satisfies a predetermined condition that allows it to be determined that the one document has been placed overlapping another document placed on the document table, the processor corrects the placing area of ​​the one document whose placing area has been determined to include a difference area between the immediately previous total document placing area and the current total document placing area. (((3))) The image processing system described in (((2))) is characterized in that the specified condition is that the difference area is adjacent to the immediately preceding entire document placement area, and the ratio of the difference area to the size of the immediately preceding entire document placement area is equal to or less than a specified threshold value. (((4))) The image processing system described in (((1))) or (((2))) is characterized in that, when the processor designates the entire document placing area identified from the current captured image as the current entire document placing area and the entire document placing area identified from the immediately previous captured image as the immediately previous entire document placing area, by comparing the immediately previous entire document placing area with the current entire document placing area, if it detects that the placing area of ​​a document whose placing area has already been determined has moved, it performs processing according to the state of the document after the movement. (((5))) The image processing system according to (((4))) is characterized in that, when the size of the placement area for the moved documents has changed, the processor notifies the user of the overlap of the documents. (((6))) The image processing system described in (((4))) is characterized in that, if the size of the placement area of ​​the moved document has not changed, the processor updates the placement area of ​​the document to the placement area after the movement. (((7))) On the computer, a function of, when an image capturing means captures an image of the top of the document table and generates a captured image as a current captured image, if a change is detected in the current captured image from the image captured immediately before as a previous captured image, identifying the placement area of ​​all documents placed on the document table as a total document placement area by analyzing the current captured image; a function of determining, from the entire document placement area, a placement area of ​​a document whose placement area has already been determined, as a placement area of ​​one document placed on the document table between the immediately preceding captured image and the currently captured image; A program to achieve this.

[0099] According to the invention described in (((1))), it is possible to identify the size of each document from the captured image generated by simultaneously photographing multiple documents placed side by side on a document table, without the user having to specify the size. According to the invention described in (((2))), even when a user shifts a document that is placed overlapping another document to eliminate the overlap, the placement area of ​​the document can be correctly identified. According to the invention described in (((3))), it is possible to appropriately determine whether the overlapping of documents has been resolved by the user. According to the invention described in (((4))), it is possible to take some kind of action when a document whose placement area has been determined is moved. According to the invention described in (((5))), it is possible to allow the user to take measures to eliminate overlapping documents. According to the invention described in (((6))), it is possible to determine that the document has simply been moved on the document platen, and to permit the document to be moved, thereby updating the document placement area. According to the invention described in (((7))), it is possible to identify the size of each document from the captured image generated by simultaneously photographing multiple documents placed side by side on a document table, without the user having to specify the size. [Explanation of symbols]

[0100] 10 multifunction device, 11 user interface (UI) unit, 12 imaging unit, 13 image processing unit, 14 original image output unit, 15 control unit, 16 original area information storage unit, 17 notification unit, 21 document camera, 22 device body, 23 original table, 24 support rod, 25 operation panel, 31 CPU, 32 ROM, 33 RAM, 34 hard disk drive (HDD), 35 printer, 36 network interface (IF), 37 address data bus, 131 total original placement area extraction unit, 132 undetermined area extraction unit, 133 original area determination unit, 134 area division unit, 135 original image generation unit, 136 original area correction unit, 137 original overlap detection unit.

Claims

1. A photographing means and a processor are provided, The processor, When the photographing means photographs the top of the document table and generates a photographed image as a current photographed image, if a change is detected from the current photographed image as a previous photographed image generated by the previous photographing, the current photographed image is analyzed to identify a placement area of ​​all documents placed on the document table as a total document placement area, determining a placement area remaining after removing the placement areas of the documents whose placement areas have already been determined from the entire document placement area as a placement area of ​​one document placed on the document table during a period from when the immediately preceding photographed image is generated to when the current photographed image is generated; 1. An image processing system comprising:

2. The image processing system of claim 1, wherein when the processor determines the entire document placing area identified from the current captured image as the current entire document placing area and the entire document placing area identified from the immediately previous captured image as the immediately previous entire document placing area, if the relationship between the immediately previous entire document placing area and the current entire document placing area satisfies a predetermined condition that allows it to be determined that the one document has been placed overlapping another document placed on the document table, the processor corrects the placing area of ​​the one document whose placing area has been determined to include a difference area between the immediately previous entire document placing area and the current entire document placing area.

3. 3. The image processing system according to claim 2, wherein the predetermined condition is that the difference area is adjacent to the immediately preceding entire document placement area, and the ratio of the difference area to the immediately preceding entire document placement area is equal to or smaller than a predetermined threshold value.

4. The image processing system of claim 1, characterized in that when the processor determines the total document placing area identified from the currently captured image as the current total document placing area and the total document placing area identified from the immediately previous captured image as the immediately previous total document placing area, when it detects that the placing area of ​​a document whose placing area has already been confirmed has moved by comparing the immediately previous total document placing area with the current total document placing area, it performs processing according to the state of the document after the movement.

5. 5. The image processing system according to claim 4, wherein, when a size of a placement area for the moved documents has changed, the processor notifies a user of an overlap of the documents.

6. 5. The image processing system according to claim 4, wherein, when the size of the placement area of ​​the moved document remains unchanged, the processor updates the placement area of ​​the document to the placement area after the movement.

7. On the computer, a function of, when an image capturing means captures an image of the top of the document table and generates a captured image as a current captured image, if a change is detected in the current captured image from the image captured immediately before as a previous captured image, identifying the placement area of ​​all documents placed on the document table as a total document placement area by analyzing the current captured image; a function of determining, from the entire document placement area, a placement area of ​​a document whose placement area has already been determined, as a placement area of ​​one document placed on the document table between the immediately preceding captured image and the currently captured image; A program to achieve this.

Images