Image processing device, image processing method, and image processing program

Abstract

This invention provides an image processing device, method, and program that can detect multiple documents with high accuracy and acquire image data, even when the density difference between the background and the document is small. [Solution] The method involves detecting multiple content regions from image data obtained by scanning multiple documents at once, generating multiple content enlargement region images by extracting pixels below a first threshold from a content distance image showing distance information between pixels corresponding to and not corresponding to a content region, generating edge distance images showing distance information between pixels corresponding to and not corresponding to an edge region from edge images obtained by detecting edge regions from image data, generating multiple edge enlargement region images by extracting pixels below a second threshold from the edge distance images, extracting the enlargement region of each document from the enlargement region images of multiple documents generated from the content enlargement region images and edge enlargement region images, and determining the region and tilt of each document based on the edge regions and content regions within the enlargement region of each document.

Description

Technical Field

[0001] This technology relates to an image processing apparatus, an image processing method, and an image processing program, and more particularly to an image processing apparatus, an image processing method, and an image processing program that perform image processing to detect a plurality of originals and acquire image data.

Background Art

[0002] Conventionally, there is known an image processing apparatus that acquires a plurality of image data by detecting images of a plurality of originals such as receipts and business cards placed on a document placement table and performing image processing.

[0003] In such an image processing apparatus that reads a plurality of originals, if the density difference between the background and the original is small, it may be difficult to identify the original area.

[0004] For example, when the luminance value of the background is RGB = (255, 255, 255) and the luminance value of the edge of the original is RGB = (248, 248, 248), the color of the edge of the original is close to the white of the background. In this case, since it becomes difficult to distinguish between the edge of the original and the background, if there is a blank area (for example, a vertical width of about 1 cm, a horizontal width of about 5 cm, etc.) between the contents of the original such as characters, logos, and figures, separation of the contents may occur. As a result, there is a possibility that one original may be erroneously determined to be a plurality of originals.

[0005] In order to solve such a problem, conventionally, there is an image processing apparatus that performs a crop process on the area of the original on the image data obtained by scanning the area including the original, and includes a detection unit that detects an edge component in a first image obtained by scanning the original to acquire position information of the original, and a cropping unit that cuts out an image area corresponding to the original from a second image obtained by scanning the original based on the acquired position information, and the first image is darker than the second image. A technique of an image processing apparatus characterized by this is disclosed (for example, see Patent Document 1).

[0006] Furthermore, an image processing apparatus and image processing program are disclosed that can integrate multiple regions of a scanned image according to the type of document, even when edges cannot be extracted from the scanned image of a document. This is achieved by detecting multiple regions having foreground characteristics from a scanned image obtained by scanning a document, deriving the straight-line distance between the outer edge of one region and the outer edge of the other region for each combination of two regions obtained from the multiple regions, deriving the area of ​​the gap portion (the part where one region and the other region do not exist) from the area of ​​the inclusion region (a predetermined shape region encompassing one region and the other region), and selecting and integrating the combination for each of the multiple regions where the straight-line distance is within a predetermined range and the area of ​​the gap portion is minimized, thereby enabling the integration of multiple regions of a scanned image according to the type of document, even when edges cannot be extracted from the scanned image of a document (see, for example, Patent Document 2). [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2019-205110 [Patent Document 2] Japanese Patent Publication No. 2021-096800 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, conventional techniques that extract the image region corresponding to a document based on the positional information of the document obtained by detecting only the edge components may not be able to properly crop multiple documents such as receipts.

[0009] Furthermore, the technique of merging multiple regions based on the straight-line distance and the area of ​​the gap between them has the problem of requiring extra memory because the image size does not change when detecting multiple regions, and the accuracy can change depending on whether or not the arrangement and number of documents are set.

[0010] Furthermore, there is a technology that uses OCR (Optical Character Recognition) processing on each cropped image to obtain character recognition results for content separation and integration. However, some content consists only of graphic information such as QR codes (registered trademarks) and barcodes, and does not contain text information, which presents the problem of time-consuming OCR processing.

[0011] This disclosure is made in view of these challenges and provides an image processing device that can detect multiple documents and acquire image data with lower processing burden and higher accuracy than conventional methods, even when the density difference between the background and the document is small. [Means for solving the problem]

[0012] This disclosure provides an image processing apparatus comprising: an image data acquisition unit that acquires image data obtained by reading multiple documents placed on a document placement table in a single operation; a content enlargement area image generation unit that generates a content image by detecting multiple content areas from the image data, generates a content distance image showing distance information between pixels corresponding to the content areas and pixels not corresponding to the content areas in the content image, and generates multiple content enlargement area images obtained by extracting pixels below a predetermined first threshold from the content distance image; an edge enlargement area image generation unit that generates an edge image by detecting multiple edge areas from the image data, generates an edge distance image showing distance information between pixels corresponding to the edge areas and pixels not corresponding to the edge areas from the edge image, and generates multiple edge enlargement area images obtained by extracting pixels below a predetermined second threshold from the edge distance image; and a document area determination unit that generates enlargement area images of the multiple documents based on the content enlargement area image and the edge enlargement area image, extracts the enlargement area of ​​each document from the enlargement area images of the multiple documents, and determines the area and inclination of each document based on the edge areas and content areas within the enlargement area of ​​each document. Furthermore, this disclosure provides an image processing method characterized by acquiring image data obtained by reading multiple documents placed on a document placement table in a single operation; generating a content image by detecting multiple content regions from the image data; generating a content distance image in the content image that shows distance information between pixels corresponding to the content regions and pixels that do not correspond to the content regions; generating multiple content enlargement region images obtained by extracting pixels below a predetermined first threshold from the content distance image; generating an edge image by detecting multiple edge regions from the image data; generating an edge distance image from the edge image that shows distance information between pixels corresponding to the edge regions and pixels that do not correspond to the edge regions; generating multiple edge enlargement region images obtained by extracting pixels below a predetermined second threshold from the edge distance image; generating enlargement region images of the multiple documents based on the content enlargement region images and the edge enlargement region images; extracting the enlargement region of each document from the enlargement region images of the multiple documents; and determining the region and inclination of each document based on the edge regions and content regions within the enlargement region of each document. Furthermore, this disclosure provides an image processing program characterized by causing the processor of an image reader to acquire image data obtained by reading multiple documents placed on a document placement table in a batch; generate a content image in which multiple content regions are detected from the image data; generate a content distance image in which distance information is shown between pixels corresponding to the content regions and pixels not corresponding to the content regions in the content image; generate multiple content enlargement region images obtained by extracting pixels below a predetermined first threshold from the content distance image; generate an edge image in which multiple edge regions are detected from the image data; generate an edge distance image in which distance information is shown between pixels corresponding to the edge regions and pixels not corresponding to the edge regions from the edge image; generate multiple edge enlargement region images obtained by extracting pixels below a predetermined second threshold from the edge distance image; generate enlargement region images of the multiple documents based on the content enlargement region image and the edge enlargement region image; extract the enlargement region of each document from the enlargement region images of the multiple documents; and execute a process to determine the region and inclination of each document based on the edge regions and the content regions within the enlargement region of each document. [Effects of the Invention]

[0013] According to this disclosure, by utilizing not only content information but also edge information, and further generating boundary lines between documents based on the edge information, it is possible to realize an image processing device that can detect multiple documents and acquire image data with lower processing load and higher accuracy than conventional methods. [Brief explanation of the drawing]

[0014] [Figure 1] This is a perspective view showing the appearance of the digital multifunction printer in this disclosure. [Figure 2] Figure 1 is an explanatory diagram showing the document tray when the document cover of the digital multifunction printer is opened. [Figure 3] Figure 2 is an explanatory diagram showing an example of the arrangement of multiple documents placed on the document placement table and an example of the content area obtained by scanning the documents. [Figure 4] It is a block diagram showing a schematic configuration of the digital multifunction machine of FIG. 1. [Figure 5] It is an explanatory diagram showing an example of problems in extracting a content area of a document obtained by an image data acquisition unit of a conventional digital multifunction machine. [Figure 6] It is an explanatory diagram showing another example of problems in extracting a content area of a document obtained by an image data acquisition unit of a conventional digital multifunction machine. [Figure 7] It is a flowchart showing an example of extraction and integration processing of content areas of a plurality of documents obtained by an image data acquisition unit of the digital multifunction machine of FIG. 1. [Figure 8] It is a flowchart showing an example of detection processing of a plurality of documents by an image processing unit of the digital multifunction machine of FIG. 1. [Figure 9] It is an explanatory diagram showing an example of generation processing of a content enlarged area image from content areas of a plurality of document images obtained by an image data acquisition unit of the digital multifunction machine of FIG. 1. [Figure 10] It is an explanatory diagram showing an example of generation processing of a content distance image from a content area. [Figure 11] It is an explanatory diagram showing an example of generation processing of an edge enlarged area image from edge areas of a plurality of document images obtained by an image data acquisition unit of the digital multifunction machine of FIG. 1. [Figure 12] It is an explanatory diagram showing an example of integration of the content enlarged area image of FIG. 9 and the edge enlarged area image of FIG. 11.

Mode for Carrying Out the Invention

[0015] In this disclosure, an "image processing apparatus" is an apparatus that acquires and outputs a plurality of image data by detecting images of a plurality of documents placed on a document placement table and performing image processing. A "content area" is an area of image data corresponding to the content of a document such as characters, logos, figures, etc. An "edge area" is a boundary (edge) area where the image density changes from a dark area to a light area. Furthermore, preferred embodiments of this disclosure will be described.

[0016] The image processing apparatus disclosed herein may further include an image data processing unit that generates reduced image data by reducing the image data at a predetermined ratio, wherein the image data processing unit generates first reduced image data by reducing the image data at a predetermined first ratio, generates a first edge image by detecting a plurality of the edge regions and a first content image by detecting a plurality of the content regions from the first reduced image data, generates second reduced image data by reducing it at a predetermined second ratio, generates a second content image from the second reduced image data, and generates the content image by removing noise from the first content image using the second content image.

[0017] In this way, an image processing device can be realized that detects multiple originals and acquires image data with lower processing load and higher accuracy than conventional methods. This is achieved by generating a first content image based on first reduced image data obtained by reducing image data at a predetermined first ratio, generating a second content image based on second reduced image data obtained by reducing image data at a predetermined second ratio, and generating a content image by removing noise from the first content image using the second content image.

[0018] In the image processing apparatus disclosed herein, the edge enlargement region image generation unit may generate an edge image by stretching a plurality of edge regions detected from the image data in one or more predetermined directions by a predetermined length.

[0019] In this way, by extending multiple edge regions detected from image data by a predetermined length in one or more predetermined directions to generate edge images, it is possible to realize an image processing device that can detect multiple documents and acquire image data with lower processing load and higher accuracy than conventional methods.

[0020] In the image processing apparatus disclosed herein, the document area determination unit may generate enlarged area images of the plurality of documents by performing a logical OR operation between the value of the edge enlarged area image and the value of the content enlarged area image.

[0021] In this way, by generating enlarged region images of multiple documents through a logical OR operation between the values ​​of the edge enlargement region image and the values ​​of the content enlargement region image, it is possible to realize an image processing device that can detect multiple documents with higher accuracy and less processing burden than conventional methods, and acquire image data.

[0022] The image processing apparatus disclosed herein may further include a crop image data generation unit that generates a plurality of crop image data corresponding to the regions of the plurality of originals, wherein the crop image data generation unit uses a bounding rectangle that circumscribes the region of each determined original as a cropping region and generates the crop image data corresponding to the region included in the cropping region.

[0023] In this way, by using a bounding rectangle that circumscribes the area of ​​each determined document as the cropping area, and generating cropped image data corresponding to the area included in the cropping area, it is possible to realize an image processing device that can detect multiple documents and acquire image data with lower processing load and higher accuracy than conventional methods.

[0024] The following details of this disclosure will be illustrated with drawings. The following description is illustrative in all respects and should not be construed as limiting the scope of this disclosure.

[0025] [Embodiment 1] <Outline configuration of digital multifunction printer 1> Based on Figures 1 to 5, the configuration of a digital multifunction device 1, as an example of an image forming apparatus equipped with the image processing apparatus disclosed herein, will be described.

[0026] Figure 1 is a perspective view showing the external appearance of the digital multifunction printer 1 of this disclosure.

[0027] Digital multifunction printer 1 is a device that has copying, scanning, and facsimile functions, and processes image data read from a document before outputting it.

[0028] Furthermore, the technology disclosed herein is not limited to the digital multifunction printer 1, but is applicable to all devices that have an image processing function for extracting image data read by a scanner or camera. For example, this technology may be applied to the extraction of image data obtained when multiple receipts placed on paper are photographed with the camera of a mobile device such as a smartphone.

[0029] Figure 2 is an explanatory diagram showing the document tray 181 of the digital multifunction printer 1 shown in Figure 1 when the document cover 18 is opened. Figure 2 shows an example of the arrangement of multiple documents M1 and M2 placed on the document tray 181. Figure 3 is an explanatory diagram showing an example of the arrangement of multiple documents M1 and M2 placed on the document placement table 181 in Figure 2, and an example of content areas C1 and C2 obtained by reading documents M1 and M2.

[0030] As shown in Figures 2 and 3, the user opens the document cover 18 upwards and places multiple documents M1 and M2 on the document tray 181. The digital multifunction printer 1 performs scanning of documents M1 and M2 based on the scanning command received from the user through the operation unit 172.

[0031] Figure 4 is a block diagram showing the schematic configuration of the digital multifunction printer 1 shown in Figure 1. As shown in Figure 4, the digital multifunction printer 1 comprises a control unit 10, an image data acquisition unit 11, an image forming unit 12, a storage unit 13, an image processing unit 14, a communication unit 15, a paper feeding unit 16, and an operation panel 17. The following describes each component of the digital multifunction printer 1.

[0032] The control unit 10 comprehensively controls the digital multifunction printer 1 and consists of at least one CPU (Central Processing Unit), at least one RAM (Random Access Memory), at least one ROM (Read-only memory), various interface circuits, and the like.

[0033] The control unit 10 monitors and controls all loads, including detection by each sensor, motor, clutch, and control panel 17, in order to control the operation of the entire digital multifunction printer 1.

[0034] The image data acquisition unit 11 is the part that reads the document placed on the document placement table and acquires image data.

[0035] The image forming unit 12 is the part that prints the image data acquired by the image data acquisition unit 11 and processed by the image processing unit 14 onto paper.

[0036] The memory unit 13 is an element or storage medium that stores information and control programs necessary to realize the various functions of the digital multifunction device 1. For example, semiconductor elements such as RAM and ROM, storage media such as hard disks, flash memory units, and SSDs (Solid State Drives) are used. Furthermore, the data may be stored in different devices, such as a hard disk drive for the data storage area and a flash memory unit for the program storage area.

[0037] The image processing unit 14 is the part that converts the image data input from the image data acquisition unit 11 into appropriate electrical signals and processes them to make them suitable for output such as enlargement and reduction, based on the results of analyzing the print job commands acquired from the user terminal 2 or the operation unit 172. Furthermore, the image processing unit 14 detects multiple documents placed on the document placement table and performs image processing such as content extraction and integration, as described later, to acquire multiple image data.

[0038] The communication unit 15 is the part that communicates with external devices such as user terminals via a network and sends and receives data with these external devices.

[0039] The paper feeding unit 16 is equipped with a paper feed cassette and a manual feed tray, and is responsible for transporting the paper stored in these to the image forming unit 12.

[0040] The operation panel 17 consists of a display panel made of a liquid crystal panel or the like, and a touch panel, such as a capacitive type, which is placed on top of the display panel and detects the position where a finger is touched, and includes a display unit 171 and an operation unit 172.

[0041] The display unit 171 is the part that displays various types of information. The display unit 171 is composed of, for example, a CRT display, a liquid crystal display, or an EL display, and is a display device such as a monitor or line display for displaying electronic data such as processing status by the operating system or application software. The control unit 10 displays the operation and status of the digital multifunction printer 1 through the display unit 171.

[0042] The control unit 172 is an interface for operating the digital multifunction printer 1 and is the part that receives commands from the user. The operation unit 172 does not necessarily have to support touch operation in its entirety; some or all of it may consist of physical keys independent of the display unit 171.

[0043] <Problems with the extraction of content areas from documents using conventional digital multifunction printers (1C)> Next, based on Figures 5 and 6, we will explain the problems with extracting the content area of ​​a document using the conventional digital multifunction printer 1C.

[0044] Figure 5 is an explanatory diagram illustrating an example of a problem in extracting the content area of ​​a document acquired by the image data acquisition unit of a conventional digital multifunction printer 1C. When the image data acquisition unit of the conventional digital multifunction printer 1C extracts the content area of ​​the receipt shown in Figure 5(A), there is a problem that when the color of the receipt's border is almost the same as the background color, an area narrower than the receipt's border is cropped, as shown by the dotted line in Figure 5(B).

[0045] Figure 6 is an explanatory diagram illustrating another example of the problems encountered when extracting the content area of ​​a document acquired by the image data acquisition unit of a conventional digital multifunction printer 1C. As shown in Figure 6, when extracting the content areas of multiple receipt documents M1-M4 arranged at various angles on the document tray, if the color of the receipt border is almost the same as the background color, a region enclosed by a dotted line that is significantly different from the original documents M1-M4 may be cropped.

[0046] Thus, when the color of the receipt's border is difficult to distinguish from the background color, problems can arise such as the inability to extract the content area according to the receipt's size, or mis-separation and mis-merging of content areas, where multiple content areas are extracted from a single receipt, or multiple receipts are merged into a single content area.

[0047] Furthermore, if the content area contains only shapes and no text, it becomes difficult to determine the top and bottom orientation of the receipt, which can lead to problems in properly extracting the content area.

[0048] To address these issues, this disclosure provides a digital multifunction printer 1 that utilizes not only content information but also edge information, and further generates boundaries between documents based on the edge information, thereby enabling detection of multiple documents and acquisition of image data with lower processing load and higher accuracy than conventional methods.

[0049] <Outline of the extraction and integration process of content areas of multiple documents acquired by the image data acquisition unit 11 of the digital multifunction printer 1> Next, with reference to Figures 7 and 8, an outline of the determination process for multiple document areas acquired by the image data acquisition unit 11 of the digital multifunction printer 1 according to Embodiment 1 of this disclosure will be described.

[0050] Figure 7 is a flowchart showing an example of the extraction and integration process of content areas of multiple documents acquired by the image data acquisition unit of the digital multifunction printer 1 shown in Figure 1.

[0051] Here, the content area extraction process is the process of simultaneously detecting multiple documents placed on the document tray of the digital multifunction printer 1, detecting the content of each document such as text, logos, and diagrams, and acquiring multiple image data corresponding to the multiple documents.

[0052] In step S1 of Figure 7, the control unit 10 of the digital multifunction printer 1 causes the image data acquisition unit 11 to detect multiple document images, and causes the image processing unit 14 to detect the content area of ​​each document and remove noise (step S1). Details regarding the detection and noise reduction processes for multiple source images will be described later in the explanation of Figure 8.

[0053] Next, in step S2, the control unit 10 causes the image processing unit 14 to detect edge regions and remove noise from each original image (step S2). Here, the edge region is the boundary (edge) region where the image density changes from a high-density area to a low-density area, and it is used to determine the cropping range. The edge region can be obtained, for example, by calculating the difference between two adjacent pixels.

[0054] Next, in step S3, the control unit 10 extends the edge region (step S3). Specifically, the edge region is stretched by applying a correction that extends the area with high image density in the edge region by a predetermined length in a predetermined direction.

[0055] Next, in step S4, the control unit 10 generates distance images in the content area and the edge area, respectively (step S4). A content distance image is an image that shows distance information between pixels corresponding to a content area and pixels that do not correspond to a content area. An edge distance image is an image that shows distance information between pixels that correspond to an edge region and pixels that do not correspond to an edge region. The specific method for generating the depth image will be described later in the explanation of Figure 10.

[0056] Next, in step S5, the control unit 10 extracts pixels below a predetermined threshold from the edge region image to generate an edge expansion region (step S5).

[0057] Next, in step S6, boundaries are generated and regions are divided based on the edge region image (step S6).

[0058] Next, in step S7, the control unit 10 extracts pixels below a predetermined threshold from the content area image to generate a content expansion area (step S7). Note that the threshold for generating the edge expansion area does not necessarily have to be the same as the threshold for generating the content expansion area; they may be different.

[0059] Next, in step S8, the control unit 10 performs a process to integrate the edge enlargement region image and the content enlargement region image (step S8). Specifically, the region obtained by performing an OR operation on the edge expansion region and the content expansion region is defined as the integrated region.

[0060] Next, in step S9, a process is performed to check for and remove noise (step S9). This is to prevent only the elongated lines of a portion of the document's outline from being cropped.

[0061] Finally, in step S10, the regions and orientations of the multiple originals are determined by cropping each of the finally obtained regions by enclosing them in rectangles (step S10). For example, a rectangle circumscribing a given region is calculated from a set of points in that region, and the area enclosed by that rectangle is then cropped.

[0062] Next, the detection process of multiple document images in the digital multifunction printer 1 will be explained based on Figure 8. Figure 8 is a flowchart showing an example of the detection process for multiple document images in the digital multifunction printer 1 shown in Figure 1.

[0063] In step S21 of Figure 8, the control unit 10 causes the image data acquisition unit 11 to acquire the original image data (RGB values) (step S21).

[0064] Next, in step S22, the control unit 10 instructs the image processing unit 14 to convert the RGB value document image data into grayscale image data (step S22).

[0065] Next, in step S23, the control unit 10 instructs the image processing unit 14 to reduce the grayscale image data by a predetermined magnification to create reduced image data (step S23). Furthermore, the control unit 10 stores the reduced image data along with the number of reductions in the storage unit 13 for use in noise reduction, which will be described later in the explanation of step S27.

[0066] Next, in step S24, the control unit 10 determines whether or not the image data has been reduced N times (step S24). Here, N is a predetermined positive integer value (for example, N=5).

[0067] If the image data has not been reduced N times (i.e., the determination in step S24 is No), the control unit 10 returns to step S23. On the other hand, if the image data is reduced N times (if the determination in step S24 is Yes), in step S25, the control unit 10 instructs the image processing unit 14 to perform O-bit binarization on the L-th reduced image (step S25). Here, L is a predetermined positive integer value smaller than N (for example, L=1).

[0068] Next, in step S26, the control unit 10 instructs the image processing unit 14 to perform Otsu binarization on the Mth reduced image (step S26). Here, M is a predetermined positive integer value that is greater than L and less than N (for example, M=3). Furthermore, in steps S26 and S27, the process is not limited to Otsu binarization; binarization may also be performed based on a predetermined threshold.

[0069] Next, in step S27, the control unit 10 instructs the image processing unit 14 to perform noise reduction on the image data based on the L and M reduction images (step S27). In this way, the content area and the edge area are detected.

[0070] <Example of generating a content enlargement area image from the content areas of multiple originals M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1> Next, an example of generating a content enlargement area image from the content areas of multiple originals M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1 according to Embodiment 1 of this disclosure will be described based on Figures 9 and 10.

[0071] Figure 9 is an explanatory diagram showing an example of the process of generating a content enlargement area image from the content areas of multiple originals M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1 shown in Figure 1. Figure 9(A) shows an example of multiple receipt documents M1 to M5 placed on the document tray 181. Figure 9(B) shows an example of content areas detected from receipt originals M1-M5 in Figure 9(A). In Figure 9(B), the white area represents the content area, such as text and graphics, and the black area represents the background area.

[0072] Figure 9(C) is a content distance image generated from the content area in Figure 9(B) by the above process.

[0073] Figure 10 is an explanatory diagram illustrating an example of the process for generating a content distance image from a content area. The gray areas in Figure 10(A) correspond to the pixels (background) in the black areas of Figure 9(B), and the white areas in Figure 10(A) correspond to the pixels (text, etc.) in the white areas of Figure 9(B).

[0074] Figure 10(B) shows the calculated distance from each pixel in Figure 10(A) to the pixels in the white areas. Specifically, the pixels in the white areas (the parts of the text, etc.) have a distance of 0, the adjacent pixel has a distance of 1, and the next adjacent pixel has a distance of 2. The generation of edge distance images, which will be described later, is processed in the same manner.

[0075] Figure 9(D) is an example of a content enlargement region image created by extracting pixels below a predetermined threshold from the content distance image in Figure 9(C).

[0076] <Example of generating an edge-enlarged region image from the edge regions of multiple original documents M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1> Next, an example of generating an edge-enlarged region image from the edge regions of multiple originals M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1 according to Embodiment 1 of this disclosure will be described based on Figure 11.

[0077] Figure 11 is an explanatory diagram showing an example of the process for generating an edge-enlarged region image from the edge regions of multiple document images acquired by the image data acquisition unit 11 of the digital multifunction printer 1 shown in Figure 1. Figure 11(A) shows an example of multiple receipts placed on the document tray 181. Figure 11(B) shows an example of an edge region detected from the receipt in Figure 11(A).

[0078] Figure 11(C) is an example of the result of extending the edge region of Figure 11(B). In the example in Figure 11(C), the vertical edges of Figure 11(B) are extended upwards by 20 pixels and downwards by 20 pixels. Additionally, the horizontal edges are extended to the left by 20 pixels and to the right by 20 pixels.

[0079] Figure 11(D) is an example of an edge-enlarged region image created by generating an edge distance image from the edge region of Figure 11(B) and then extracting pixels below a predetermined threshold. In Figure 11(D), it can be seen that the area indicated by the dotted circle has been incorrectly merged. Therefore, in order to prevent these misintegrations, the following process is performed.

[0080] Figure 11(E) is an example of a boundary line generated based on the edge region image in Figure 11(B). For example, by applying Voronoi diagrams, it is possible to generate boundary lines based on edge region images.

[0081] By using the boundary line in Figure 11(E) to divide the edge-enlarged region image in Figure 11(D), it becomes possible to prevent misintegration. Figure 11(F) is the final edge-enlarged region image obtained in this manner.

[0082] <An example of integrating content-enlarged area images and edge-enlarged area images of multiple original documents M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1> Next, an example of integrating content-enlarged area images and edge-enlarged area images of multiple originals M1 to M5 acquired by the image data acquisition unit 11 of the digital multifunction printer 1 according to Embodiment 1 of this disclosure will be described based on Figure 12.

[0083] Figure 12 is an explanatory diagram showing an example of integrating the content-enlarged area image from Figure 9 with the edge-enlarged area image from Figure 11. Figure 12(A) is a magnified content area image of Figure 9(D), and Figure 12(B) is a magnified edge area image of Figure 11(F). Figure 12(C) is the region obtained by integrating the content-enlarged region image of Figure 12(A) and the edge-enlarged region image of Figure 12(B) using an OR operation.

[0084] Figure 12(D) shows the areas where cropping is performed on original documents M1 to M5 based on each region in Figure 12(C). In Figure 12(D), noise can be seen in the area enclosed by the dashed white circle, so it is necessary to perform a process to remove the noise. Figure 12(E) shows the cropped results of the final manuscripts M1 to M5. Based on the cropping results, the regions and orientations of multiple original documents M1 to M5 can be determined.

[0085] In this way, by utilizing not only content information but also edge information, and further generating boundary lines between documents based on the edge information, a digital multifunction printer 1 can be realized that can detect multiple documents with higher accuracy and less processing burden than conventional methods, and acquire image data.

[0086] Preferred embodiments of this disclosure include combinations of any of the embodiments described above. In addition to the embodiments described above, various modifications of this disclosure are possible. These modifications should not be construed as being outside the scope of this disclosure. This disclosure should include the meaning of equivalents to the claims and all variations within that scope. [Explanation of symbols]

[0087] 1: Digital multifunction printer, 1C: Conventional digital multifunction printer, 10: Control unit, 11: Image data acquisition unit, 12: Image forming unit, 13: Storage unit, 14: Image processing unit, 15: Communication unit, 16: Paper feed unit, 17: Operation panel, 18: Document cover, 171: Display unit, 172: Operation unit, 181: Document tray, C1, C2: Content area, M1~M5: Document

Claims

1. An image data acquisition unit that acquires image data obtained by simultaneously scanning multiple documents placed on a document placement table, A content image is generated from the aforementioned image data, in which multiple content regions are detected. A content distance image is generated in the content image, showing distance information between pixels corresponding to the content area and pixels not corresponding to the content area. A content enlargement region image generation unit generates a plurality of content enlargement region images obtained by extracting pixels below a predetermined first threshold from the content distance image, An edge image is generated from the aforementioned image data by detecting multiple edge regions. An edge distance image is generated from the edge image, showing distance information between pixels corresponding to the edge region and pixels not corresponding to the edge region. An edge-enlarged region image generation unit generates a plurality of edge-enlarged region images obtained by extracting pixels below a predetermined second threshold from the aforementioned edge distance image, Based on the content enlargement region image and the edge enlargement region image, enlargement region images of the multiple documents are generated. An image processing apparatus comprising: a document area determination unit that extracts the enlarged area of ​​each document from the enlarged area images of a plurality of documents, and determines the area and inclination of each document based on the edge area and the content area within the enlarged area of ​​each document.

2. The image processing apparatus according to claim 1, wherein the edge enlargement region image generation unit extracts pixels below the second threshold and generates boundary lines by dividing the region among the plurality of edge regions obtained and adds them to the edge enlargement region, and groups the edge enlargement region image based on the edge enlargement region and the boundary lines.

3. The system further includes an image data processing unit that generates reduced image data by reducing the aforementioned image data by a predetermined ratio. The image data processing unit generates first reduced image data by reducing the image data by a predetermined first ratio, and generates a first edge image by detecting a plurality of the edge regions and a first content image by detecting a plurality of the content regions from the first reduced image data. A second reduced image data is generated by reducing it to a predetermined second ratio, and a second content image is generated from the second reduced image data. The image processing apparatus according to claim 1, which generates a content image obtained by removing noise from the first content image using the second content image.

4. The image processing apparatus according to claim 1, wherein the edge enlargement region image generation unit generates an edge image by stretching a plurality of edge regions detected from the image data in one or more predetermined directions by a predetermined length.

5. The image processing apparatus according to claim 1, wherein the document area determination unit generates enlarged area images of the plurality of documents by performing a logical OR operation between the value of the edge enlarged area image and the value of the content enlarged area image.

6. The system further includes a crop image data generation unit that generates multiple crop image data corresponding to the regions of the multiple original documents, The image processing apparatus according to claim 1, wherein the crop image data generation unit generates crop image data corresponding to the areas included in the crop processing area, with the circumscribing rectangle that circumscribes the determined area of ​​each original document as the crop processing area.

7. Image data obtained by scanning multiple documents placed on the document tray simultaneously is acquired. A content image is generated from the aforementioned image data, in which multiple content regions are detected. A content distance image is generated in the content image, showing distance information between pixels corresponding to the content area and pixels not corresponding to the content area. Multiple content enlargement region images are generated by extracting pixels below a predetermined first threshold from the aforementioned content distance image. An edge image is generated from the aforementioned image data by detecting multiple edge regions. An edge distance image is generated from the edge image, showing distance information between pixels corresponding to the edge region and pixels not corresponding to the edge region. Multiple edge-extended region images are generated by extracting pixels below a predetermined second threshold from the aforementioned edge distance image. Based on the content enlargement region image and the edge enlargement region image, enlargement region images of the multiple documents are generated. An image processing method characterized by extracting the enlarged region of each document from the enlarged region images of a plurality of documents, and determining the region and tilt of each document based on the edge region and the content region within the enlarged region of each document.

8. The image reading device's processor acquires image data obtained by simultaneously scanning multiple documents placed on the document tray. A content image is generated from the aforementioned image data, in which multiple content regions are detected. A content distance image is generated in the content image, showing distance information between pixels corresponding to the content area and pixels not corresponding to the content area. Multiple content enlargement region images are generated by extracting pixels below a predetermined first threshold from the aforementioned content distance image. An edge image is generated from the aforementioned image data by detecting multiple edge regions. An edge distance image is generated from the edge image, showing distance information between pixels corresponding to the edge region and pixels not corresponding to the edge region. Multiple edge-extended region images are generated by extracting pixels below a predetermined second threshold from the aforementioned edge distance image. Based on the content enlargement region image and the edge enlargement region image, enlargement region images of the multiple documents are generated. An image processing program characterized by extracting the enlarged region of each document from the enlarged region images of multiple documents, and performing a process to determine the region and tilt of each document based on the edge region and the content region within the enlarged region of each document.

Images