Information processing device and program

Abstract

To associate the front and back surfaces of a plurality of sheets of paper even when neither marks nor characters for the association are printed on each sheet of paper in advance on the occasion where the front and back surfaces of a plurality of sheets are to be correlated.SOLUTION: A processor accepts first image data representing front faces of a plurality of sheets generated by one reading, and second image data representing back faces of the plurality of sheets generated by one reading. The processor associates the front and back faces of each sheet of paper on the basis of a positional relationship between each sheet of paper represented by the first image data and each sheet of paper represented by the image data of the back face.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] Techniques for associating the front and back surfaces of a sheet are known.

[0003] Patent Document 1 describes a method for associating the front and back surfaces of a sheet based on the reverse image of the sheet.

[0004] Patent Document 2 describes a system that acquires first identification information for identifying a first medium surface and second identification information for identifying a second medium surface and associated with the first identification information, generates a first code image to be printed on the first medium surface from the first identification information, and generates a second code image to be printed on the second medium surface from the second identification information.

[0005] Patent Document 3 describes an apparatus that determines whether a specific object is included in both a first image and a second image of a printed matter, and determines a combination of a front surface image and a back surface image of the printed matter based on the determination result.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0007] The object of the present invention is to associate the front and back sides of multiple sheets of paper, even when no mark or characters for matching are pre-printed on each sheet. [Means for solving the problem]

[0008] The invention according to claim 1 comprises a processor, the processor receiving first image data representing the front surfaces of a plurality of sheets of paper generated by a single read, and receiving second image data representing the back surfaces of the plurality of sheets of paper generated by a single read, If the number of sheets of paper shown in the first image data is different from the number of sheets of paper shown in the second image data, The first image data and the second image data Includes the distance between them This is an information processing device that associates the front and back sides of each sheet of paper based on their relative positions.

[0009] The invention according to claim 2 is an information processing apparatus according to claim 1, wherein the positional relationship includes the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data.

[0010] Apparatus related to Reference Example 1 The information processing apparatus according to claim 1 or claim 2, wherein the positional relationship includes the relationship of the distance between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data.

[0011] Apparatus related to Reference Example 2 The processor, when the number of sheets of paper represented in the first image data is different from the number of sheets of paper represented in the second image data, associates the front and back sides of each sheet of paper based on the distance between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data. Reference example 1 This is the information processing device described above.

[0012] Claim 3The invention relating to the present invention is that, when the number of sheets of paper represented in the first image data is the same as the number of sheets of paper represented in the second image data, the processor associates the front and back surfaces of each sheet of paper with the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data. 1 This is the information processing device described above.

[0013] Claim 4 The invention relating to the above is that the processor determines the positional relationship and the similarity between the attributes of each sheet of paper represented in the first image data and the attributes of each sheet of paper represented in the second image data. Obtain , If there are multiple combinations of the front and back surfaces with the highest degree of similarity, the positional relationship can be used to further determine the positional relationship. The front and back sides of each sheet of paper are associated, as per claims 1 to 1. 3 It is an information processing device described in any one of the items.

[0014] Claim 5 The invention relating to the claim is that the attribute includes at least one of the paper size, material, shape, thickness and color. 4 This is the information processing device described above.

[0015] Claim 6 The invention relating to claims 1 to 1 further comprises a processor that determines whether the front or back of the paper is blank based on the first image data or the second image data. 5 It is an information processing device described in any one of the items.

[0016] Claim 7 The invention relates to a computer that receives first image data representing the front surfaces of multiple sheets of paper generated by a single reading, and receives second image data representing the back surfaces of the multiple sheets of paper generated by a single reading, If the number of sheets of paper shown in the first image data is different from the number of sheets of paper shown in the second image data, The first image data and the reverse side image data Includes the distance between them This program is designed to match the front and back sides of each sheet of paper based on their relative positions. [Effects of the Invention]

[0017] According to the invention according to claim 1, even when marks or characters for such association are not printed on each sheet in advance when associating the front and back surfaces of a plurality of sheets, the front and back surfaces of each sheet can be associated with each other. 7 Furthermore, even if the number of sheets of paper differs between the front and back sides, the front and back sides can still be associated.

[0018] 3 According to the invention according to claim 2, even when the position of the sheet changes between the time of reading the front surface and the time of reading the back surface, if the relative positional relationship is maintained, the front and back surfaces can be associated with each other.

[0020] 4 5 According to the invention according to claim ,

[0021] the front and back surfaces can be associated with each other using the attributes of the sheet. 6 According to the invention according to claim the front and back surfaces can be associated with each other without an operator determining whether the sheet is blank.

Brief Description of the Drawings

[0022] [Figure 1] [Figure 2] It is a block diagram showing the configuration of an image processing apparatus. [Figure 3] It is a block diagram showing the hardware configuration of an image processing apparatus. [Figure 4] It is a diagram showing the result of scanning the front surface. [Figure 5] It is a diagram showing the area of the front surface. [Figure 5] It is a diagram showing the result of scanning the back surface. [Figure 6] It is a diagram showing the area of the back surface. [Figure 7] It is a diagram showing the area of the front surface. [Figure 8] It is a diagram showing the area of the back surface. [Figure 9] It is a diagram showing the distance between the front and back surfaces [Figure 10]This diagram shows the relative positional relationship between the front and back surfaces. [Figure 11] This diagram shows the distance and relative positional relationship between the front and back surfaces. [Figure 12] This is a diagram showing the area on the reverse side. [Figure 13] This diagram shows the distance between the front and back surfaces. [Figure 14] This is a diagram showing the surface area. [Figure 15] This is a diagram showing the area on the reverse side. [Figure 16] This figure shows the similarity between the front and back surfaces. [Figure 17] This is a diagram showing the area on the reverse side. [Figure 18] This figure shows the distance and similarity between the front and back surfaces. [Figure 19] This diagram shows the results of the blank paper judgment. [Modes for carrying out the invention]

[0023] An image processing apparatus according to an embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing an example of the configuration of the image processing apparatus 10 according to an embodiment.

[0024] The image processing device 10 receives a first image data representing one side of the paper and a second image data representing the other side of the paper, and associates the front and back sides of the paper. For example, one side is the front of the paper and the other side is the back of the paper.

[0025] A single scan generates first image data representing the front surfaces of multiple sheets of paper. Similarly, a single scan generates second image data representing the back surfaces of the same multiple sheets of paper. The image processing device 10 associates the front and back surfaces of each sheet of paper based on the first and second image data.

[0026] Paper scanning can be done by scanning with a scanner or by taking a picture with a camera. A single scan can involve scanning multiple documents at once (i.e., scanning multiple documents together) or taking a picture of multiple documents together.

[0027] For example, when the front surfaces of multiple sheets of paper are scanned simultaneously by a scanner, a first image data representing the front surfaces of those sheets is generated. Similarly, when the back surfaces of multiple sheets of paper are scanned simultaneously by a scanner, a second image data representing the back surfaces of those sheets is generated. The same process occurs when a camera is used, generating both the first and second image data.

[0028] The types of paper that can be scanned are not particularly limited. For example, these may include forms (such as ledgers and slips), business cards, receipts, and other documents and papers.

[0029] The reading may be performed by the image processing device 10, or by an external device other than the image processing device 10. If the reading is performed by an external device, the first image data and the second image data are transmitted from the external device to the image processing device 10.

[0030] As shown in Figure 1, the image processing device 10 includes, for example, an image acquisition unit 12, a identification unit 14, a positional relationship calculation unit 16, a correspondence unit 18, an attribute acquisition unit 20, a similarity calculation unit 22, a candidate determination unit 24, and a blank page determination unit 26.

[0031] The image acquisition unit 12 acquires the first image data and the second image data. If the reading is performed by the image processing device 10, the image acquisition unit 12 acquires the first image data and the second image data generated by that reading. If the reading is performed by an external device other than the image processing device 10, the image acquisition unit 12 acquires the first image data and the second image data from the external device. The image acquisition unit 12 may receive the first image data and the second image data from the external device via a communication path such as the Internet or a LAN (Local Area Network), or it may acquire the first image data and the second image data via a portable recording medium (e.g., a USB memory stick).

[0032] The identification unit 14 identifies the area of ​​each sheet of paper represented in the first image data and the second image data, respectively. Specifically, the identification unit 14 identifies the front surface area of ​​each sheet of paper represented in the first image data and identifies the back surface area of ​​each sheet of paper represented in the second image data. Known techniques are used to identify the area of ​​the paper from the image data. For example, the identification unit 14 extracts the outline of the front surface of the paper represented in the first image data and identifies the area enclosed by that outline as the front surface area. Similarly, the identification unit 14 extracts the outline of the back surface of the paper represented in the second image data and identifies the area enclosed by that outline as the back surface area. Hereinafter, the front surface area of ​​the paper will be referred to as the "front surface area" and the back surface area of ​​the paper will be referred to as the "back surface area".

[0033] The positional relationship calculation unit 16 calculates the positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data. For example, the positional relationship calculation unit 16 calculates the positional relationship between each front area and each back area. The concept of positional relationship includes relative positional relationships and distance relationships.

[0034] The relative positional relationship is the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data. For example, the positional relationship calculation unit 16 calculates the relative positional relationship between the sheets of paper represented in the first image data (hereinafter referred to as the "first relative positional relationship") and the relative positional relationship between the sheets of paper represented in the second image data (hereinafter referred to as the "second relative positional relationship"). The positional relationship calculation unit 16 calculates the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data by comparing the first relative positional relationship and the second relative positional relationship. Specifically, the positional relationship calculation unit 16 calculates the relative positional relationship between the front areas as the first relative positional relationship, calculates the relative positional relationship between the back areas as the second relative positional relationship, and calculates the relative positional relationship between each front area and each back area.

[0035] The distance relationship is the relationship between the distance between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data. The position relationship calculation unit 16 calculates the distance from a reference position (hereinafter referred to as the "reference position") to each sheet of paper in the first image data, and calculates the distance from the reference position to each sheet of paper in the second image data. For example, the position relationship calculation unit 16 calculates the distance from the reference coordinate (i.e., the reference position) to the coordinate of each sheet of paper (e.g., the center coordinate) in the first image data, and calculates the distance from the reference coordinate to the coordinate of each sheet of paper (e.g., the center coordinate) in the second image data. The position relationship calculation unit 16 calculates the distance relationship between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data by comparing the distance of each sheet of paper calculated based on the first image data with the distance of each sheet of paper calculated based on the second image data. The distance relationship is, for example, the difference in distance (i.e., the positional shift). For example, the positional relationship calculation unit 16 calculates the difference between the distance of each sheet represented in the first image data and the distance of each sheet represented in the second image data for each sheet, as a distance relationship. Specifically, the positional relationship calculation unit 16 calculates the distance from the reference position to each front area, calculates the distance from the reference position to each back area, and calculates the difference between these distances as a distance relationship.

[0036] The correspondence unit 18 associates the front surface of each sheet of paper shown in the first image data with the back surface of each sheet of paper shown in the second image data, based on the positional relationship calculated by the positional relationship calculation unit 16. The data indicating this correspondence may be stored in the image processing device 10, transmitted to a device other than the image processing device 10, or displayed on a display.

[0037] When a relative positional relationship is used, the correspondence unit 18 associates the front and back surfaces, which are represented at the same relative position in the first image data and the second image data, as surfaces of the same paper.

[0038] When distance is used as the positional relationship, the correspondence unit 18 associates the front and back sides as the same side of the paper based on the distance difference between each sheet of paper. For example, the correspondence unit 18 associates the front and back sides with the smallest distance difference (i.e., the smallest positional misalignment) as the same side of the paper.

[0039] The attribute acquisition unit 20 acquires data indicating the attributes of each sheet of paper represented in the first image data and data indicating the attributes of each sheet of paper represented in the second image data. Paper attributes include, for example, the size, shape, color, fingerprints on the paper, paper material, and thickness. For example, the attribute acquisition unit 20 identifies the attributes of each sheet of paper represented in the first image data by analyzing the first image data, and identifies the attributes of each sheet of paper represented in the second image data by analyzing the second image data.

[0040] The similarity calculation unit 22 calculates the similarity between the attributes of each sheet obtained from the first image data and the attributes of each sheet obtained from the second image data. The similarity is a value that indicates the degree to which the attributes of the sheets obtained from the first image data and the attributes of the sheets obtained from the second image data are similar.

[0041] The candidate determination unit 24 determines candidate paper sheets to be matched by the matching unit 18 based on at least one of the positional relationship of the paper sheets and the similarity of their attributes.

[0042] The blank paper determination unit 26 determines whether the front surface of the paper is blank based on the first image data, and whether the back surface of the paper is blank based on the second image data. The blank paper determination unit 26 may determine whether both the front and back surfaces are blank, or it may determine whether either the front or back surface is blank. For example, the blank paper determination unit 26 may determine whether the paper is blank based on the pixel values.

[0043] Note that the attribute acquisition unit 20, similarity calculation unit 22, candidate determination unit 24, and blank determination unit 26 do not necessarily have to be included in the image processing device 10. In other words, the image processing device 10 does not need to have these functions.

[0044] The hardware configuration of the image processing device 10 will be described below with reference to Figure 2. Figure 2 is a block diagram showing an example of the hardware configuration of the image processing device 10.

[0045] The image processing device 10 includes, for example, an image reading device 28, a communication device 30, a user interface (UI) 32, a memory 34, and a processor 36.

[0046] The image reading device 28 is a scanner or camera, and generates image data representing a sheet of paper by reading it. The image reading device 28 generates first image data representing the front surfaces of multiple sheets of paper by reading the front surfaces of multiple sheets of paper at once, and generates second image data representing the back surfaces of multiple sheets of paper by reading the back surfaces of multiple sheets of paper at once. If the image reading device 28 is a scanner, the type of scanner is not particularly limited. For example, a flatbed scanner can be used as the scanner.

[0047] In the example shown in Figure 2, the image reading device 28 is included in the image processing device 10, but the image reading device 28 does not necessarily have to be included in the image processing device 10. In this case, the image processing device 10 acquires the first image data and the second image data from an external device.

[0048] The communication device 30 includes one or more communication interfaces having a communication chip, communication circuits, etc., and has the function of transmitting information to other devices and the function of receiving information from other devices. The communication device 30 may have wireless communication functions such as short-range wireless communication or Wi-Fi, or it may have wired communication functions.

[0049] UI32 is a user interface and includes a display and an input device. The display is a liquid crystal display or an EL display, etc. The input device is a keyboard, mouse, input keys, or control panel, etc. UI32 may also be a UI such as a touch panel that combines a display and an input device.

[0050] Memory 34 is a device that constitutes one or more storage areas for storing data. Memory 34 is, for example, a hard disk drive (HDD), a solid state drive (SSD), various types of memory (e.g., RAM, DRAM, NVRAM, ROM, etc.), other storage devices (e.g., optical discs, etc.), or a combination thereof.

[0051] The processor 36 controls the operation of each part of the image processing device 10.

[0052] The image acquisition unit 12, identification unit 14, positional relationship calculation unit 16, correspondence unit 18, attribute acquisition unit 20, similarity calculation unit 22, candidate determination unit 24, and blank determination unit 26 are implemented by the processor 36.

[0053] The embodiments will be described in detail below with specific examples. In the following example, the image reading device 28 is a flatbed scanner having a platen glass, and multiple sheets of paper are arranged side by side on the platen glass. The image reading device 28 generates image data representing the multiple sheets of paper by scanning the multiple sheets of paper arranged side by side on the platen glass at once.

[0054] Multiple sheets of paper are placed on a platen glass by the user so that one side (e.g., the front) of each sheet is scanned, and that one side is scanned by the image reading device 28. This generates first image data.

[0055] Furthermore, the user flips each of the multiple sheets of paper over and places them on the platen glass so that the other side (for example, the back) of each sheet is scanned, and the other side is scanned by the image reading device 28. This generates a second image data.

[0056] The first and second image data, which are the targets for mapping the front and back surfaces, may be specified by the user. For example, a list of images may be displayed on the UI32 display, and the user may select the first and second image data from that list. For example, an image such as an icon representing the mapping function may be displayed on the UI32 display, and when the user presses this image, a list of candidate images for mapping may be displayed on the display.

[0057] Furthermore, if two scans are performed consecutively, the processor 36 may define the image data generated by the first scan as the first image data and the image data generated by the second scan as the second image data. Two consecutive scans refer to, for example, a series of scans in which the length of time between the end of the first scan and the start of the second scan is less than or equal to a threshold.

[0058] If an image such as an icon representing a mapping function is pressed by the user, and then two scans are performed consecutively, the processor 36 may define the image data generated by the first scan as the first image data and the image data generated by the second scan as the second image data. In this case, the processor 36 may define the image data generated by the first scan as the first image data and the image data generated by the second scan as the second image data, regardless of the length of time between the end of the first scan and the start of the second scan, as long as the execution of the function is not canceled.

[0059] Figure 3 shows the results of the surface scan. Specifically, it shows the first image 38 based on the first image data. The first image 38 shows surfaces 40, 42, 44, 46, and 48. Surface 40 is the surface of paper A, surface 42 is the surface of paper B, surface 44 is the surface of paper C, surface 46 is the surface of paper D, and surface 48 is the surface of paper E. Papers A, B, C, D, and E are placed on a platen glass so that each surface of paper A, B, C, D, and E is scanned by the image reading device 28.

[0060] The processor 36 extracts the contours of each of the surfaces 40, 42, 44, 46, and 48 represented in the first image 38, and identifies the respective surface areas of surfaces 40, 42, 44, 46, and 48.

[0061] In Figure 4, the surface areas of each surface are shown by dashed lines. Surface area 40a is the area enclosed by the contour of surface 40. Surface area 42a is the area enclosed by the contour of surface 42. Surface area 44a is the area enclosed by the contour of surface 44. Surface area 46a is the area enclosed by the contour of surface 46. Surface area 48a is the area enclosed by the contour of surface 48.

[0062] Figure 5 shows the results of scanning the reverse side. Specifically, it shows the second image 50 based on the second image data. The second image 50 shows the reverse sides 52, 54, 56, 58, and 60. Reverse side 52 is the reverse side of paper A, reverse side 54 is the reverse side of paper B, reverse side 56 is the reverse side of paper C, reverse side 58 is the reverse side of paper D, and reverse side 60 is the reverse side of paper E. Papers A, B, C, D, and E are placed on a platen glass so that the reverse side of each paper is scanned, and are scanned by the image reading device 28.

[0063] The processor 36 extracts the contours of the back surfaces 52, 54, 56, 58, and 60 represented in the second image 50, and identifies the respective back surface areas of the back surfaces 52, 54, 56, 58, and 60.

[0064] In Figure 6, the back surface areas of each back surface are shown by dashed lines. Back surface area 52a is the area enclosed by the outline of back surface 52. Back surface area 54a is the area enclosed by the outline of back surface 54. Back surface area 56a is the area enclosed by the outline of back surface 56. Back surface area 58a is the area enclosed by the outline of back surface 58. Back surface area 60a is the area enclosed by the outline of back surface 60.

[0065] When scanning the back side after scanning the front side, it is possible to flip the paper over. In this case, the tilt and position of the paper may change between the front and back scans. Referring to Figures 3 and 5, the tilt and position of each paper change between the front and back scans.

[0066] Alternatively, the user may manually identify the front and back areas of each sheet of paper.

[0067] The following describes the process of associating the front and back sides of each sheet of paper based on the positional relationship between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data.

[0068] Referring to Figures 7 to 9, the process of associating the front and back sides of each sheet of paper based on distance relationships will be explained. Figure 7 is a diagram showing the area of ​​the front side. Figure 8 is a diagram showing the area of ​​the back side. Figure 9 is a diagram showing the distance between the front and back sides.

[0069] As shown in Figure 7, a reference position 38a is defined on the first image 38. The reference position 38a may be a predetermined position or a position specified by the user. For example, a two-dimensional coordinate system may be defined, and the origin of that coordinate system may be defined as the reference position 38a. In the example shown in Figure 7, one vertex of the first image 38, which forms a rectangular region overall, is defined as the reference position 38a.

[0070] The processor 36 calculates the distance between a reference position 38a on the first image 38 and the surface area for each surface area. For example, the processor 36 calculates the distance between a coordinate on the surface area (e.g., center coordinate, centroid coordinate, or specific coordinate on the surface (e.g., vertex)) and the reference position 38a for each surface area.

[0071] In the example shown in Figure 7, the processor 36 calculates the distance between the center coordinates 40b of the surface area 40a of the surface 40 and the reference position 38a as the distance between the surface area 40a and the reference position 38a. Distances are calculated similarly for other surface areas. Hereinafter, the distance between the surface area 40a and the reference position 38a will be referred to as distance Ls1, the distance between the surface area 42a and the reference position 38a as distance Ls2, the distance between the surface area 44a and the reference position 38a as distance Ls3, the distance between the surface area 46a and the reference position 38a as distance Ls4, and the distance between the surface area 48a and the reference position 38a as distance Ls5.

[0072] Similarly, for the back surface, the processor 36 calculates the distance between the reference position and the back surface area. The reference position for the back surface has the same coordinates as the reference position 38a for the front surface.

[0073] As shown in Figure 8, the processor 36 calculates the distance between the center coordinates 52b of the back surface area 52a of the back surface 52 and the reference position 50a as the distance between the back surface area 52a and the reference position 50a. The reference position 50a has the same coordinates as the reference position 38a. Distances are calculated similarly for the other back surface areas. Hereinafter, the distance between the back surface area 52a and the reference position 50a will be referred to as distance Lb1, the distance between the back surface area 54a and the reference position 50a will be referred to as distance Lb2, the distance between the back surface area 56a and the reference position 50a will be referred to as distance Lb3, the distance between the back surface area 58a and the reference position 50a will be referred to as distance Lb4, and the distance between the back surface area 60a and the reference position 50a will be referred to as distance Lb5.

[0074] The processor 36 calculates the difference between the distance of each surface area and the distance of each back surface area as a distance relationship.

[0075] Figure 9 shows a specific example of the relationship between distances. The processor 36 calculates the difference ΔL between the distance Ls1 of the surface area 40a of the surface 40 and the distance of the back surface area of ​​each back surface. The difference ΔL between the distance Ls1 of the surface area 40a and the distance Lb1 of the back surface area 52a is 10px (pixels). The difference ΔL between the distance Ls1 and the distance Lb2 of the back surface area 54a is 300px. The difference ΔL between the distance Ls1 and the distance Lb3 of the back surface area 56a is 400px. The difference ΔL between the distance Ls1 and the distance Lb4 of the back surface area 58a is 150px. The difference ΔL between the distance Ls1 and the distance Lb5 of the back surface area 60a is 600px. The difference ΔL corresponds to the positional displacement.

[0076] The processor 36 identifies the back surface area with the smallest difference ΔL from the distance Ls1 of the front surface area 40a, and associates the front surface 40 from which the front surface area 40a was extracted with the back surface from which its back surface area was extracted, as the same side of the paper. In the example shown in Figure 9, the difference ΔL for the back surface area 52a of the back surface 52 is 10px, which is the smallest value. Therefore, the processor 36 associates the front surface 40 and the back surface 52 as the same side of the paper.

[0077] If each sheet of paper is flipped over after scanning the front side to scan the back side, it is assumed that, if it is the same sheet of paper, the paper will be positioned in approximately the same location when scanning the front side and the back side. Based on this assumption, the processor 36 associates the front side 40 and the back side 52 with the smallest difference ΔL as the same sheet of paper.

[0078] For example, the processor 36 extracts image data representing the front surface 40 from the first image data and image data representing the back surface 52 from the second image data, associates the image data representing the front surface 40 with the image data representing the back surface 52, and stores it in the memory 34. The processor 36 may also transmit the associated image data representing the front surface 40 with the image data representing the back surface 52 to an external device or display it on a display.

[0079] The processor 36 similarly identifies the combination of front and back surfaces that forms the smallest distance difference ΔL for other paper types, and associates those front and back surfaces.

[0080] By using distance relationships, the front and back sides can be associated even if the number of sheets of paper differs between the front and back sides.

[0081] The following describes the process of associating the front and back sides of each sheet of paper based on the relative positional relationship between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data, with reference to Figures 4, 6, and 10. Figure 10 is a diagram showing the relative positional relationship between the front and back sides of a sheet of paper.

[0082] The processor 36 calculates the relative positional relationship between surface areas identified from the first image 38 as the first relative positional relationship, and calculates the relative positional relationship between back surface areas identified from the second image 50 as the second relative positional relationship.

[0083] For example, using the center of the first image 38 shown in Figure 4 as a reference, the processor 36 determines the relative position of each surface area. Surface area 40a is located in the upper left, surface area 42a is located in the upper center, surface area 44a is located in the upper right, surface area 46a is located in the lower left, and surface area 48a is located in the lower right. The processor 36 calculates these positional relationships as the first relative positional relationships.

[0084] Similarly, for the back surface, the processor 36 determines the relative position of each back surface area with respect to the center of the second image 50 shown in Figure 6. Back surface area 52a is located in the upper left, back surface area 54a is located in the upper center, back surface area 56a is located in the upper right, back surface area 58a is located in the lower left, and back surface area 60a is located in the lower right. The processor 36 calculates these positional relationships as the second relative positional relationships.

[0085] The processor 36 calculates the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data by comparing the first relative positional relationship and the second relative positional relationship. Specifically, the processor 36 associates the front and back surfaces that are represented at the same relative position in the first image data and the second image data as surfaces of the same sheet of paper.

[0086] Referring to Figure 10, the front area 40a of the front surface 40 is located in the upper left, and the back area 52a of the back surface 52 is located in the upper left. Thus, the relative position of the front area 40a on the first image 38 and the relative position of the back area 52a on the second image 50 are the same, so the processor 36 associates the front surface 40 and the back surface 52 as the same side of the paper.

[0087] If each sheet of paper is flipped over after scanning the front side to scan the back side, it is assumed that, for the same sheet of paper, the paper will be positioned in approximately the same location when scanning the front side and the back side. Based on this assumption, the processor 36 associates the front side 40 and the back side 52, which are in the same relative position, as the same side of the sheet of paper.

[0088] In the example shown in Figure 10, the front surface 42 is associated with the back surface 54, the front surface 44 is associated with the back surface 56, the front surface 46 is associated with the back surface 58, and the front surface 48 is associated with the back surface 60.

[0089] By using relative positional relationships, even if the paper's position changes between scanning the front and back sides, the front and back sides can be associated as long as the relative positional relationships are maintained.

[0090] The processor 36 may associate the front and back surfaces based on either the relative positional relationship or the distance positional relationship, or it may associate the front and back surfaces based on both the relative positional relationship and the distance positional relationship.

[0091] Referring to Figure 11, the process of associating the front and back surfaces based on both their relative positional relationship and their distance-based positional relationship will be explained. Figure 11 is a diagram showing the distance and relative positional relationship between the front and back surfaces.

[0092] The processor 36 may associate the front and back surfaces, which are represented at the same relative position in the first image data and the second image data, and which have the smallest distance difference ΔL, with the same paper surface.

[0093] In the example shown in Figure 11, the relative positions of the surface area 40a of the surface 40 and the relative positions of the back area 52a of the back surface 52 are the same, and the difference ΔL is smallest. Therefore, the processor 36 associates the surface 40 and the back surface 52 as the same side of the paper. The same applies to the other sides.

[0094] If the number of sheets of paper shown in the first image data is different from the number of sheets of paper shown in the second image data, the processor 36 may associate the front and back sides of each sheet of paper based on the distance between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data. In other words, if the number of sheets of paper is different when reading the front side and when reading the back side, the processor 36 associates the front and back sides based on the distance relationship.

[0095] The following explanation will describe the process when the number of sheets of paper differs between front-side and back-side reading, referring to Figures 4, 12, and 13. Figure 12 shows the area of ​​the back side. Figure 13 shows the distance between the front and back sides.

[0096] When scanning the surface, five sheets of paper are scanned, and as a result, first image data representing surfaces 40, 42, 44, 46, and 48 is generated, as shown in Figure 4.

[0097] On the other hand, when scanning the reverse side, two sheets of paper are scanned, and as a result, a second image data representing the reverse side (56, 58) is generated, as shown in Figure 12. For example, it is conceivable that scanning of the reverse side is performed only on the sheets of paper that have information printed on the reverse side.

[0098] Thus, when the number of front and back surfaces differs, the processor 36 calculates the distance to each front surface and the distance to each back surface, as explained with reference to Figures 7 and 8, and calculates the difference ΔL between the front surface distance and the back surface distance. The processor 36 associates the front surface and back surface with the smallest difference ΔL as the same side of the paper.

[0099] In the example shown in Figure 13, the processor 36 associates the front surface 44 and the back surface 56 as the same side of the paper, and associates the front surface 46 and the back surface 58 as the same side of the paper.

[0100] Thus, even when the number of front faces and back faces are different, the relationship between the front and back faces can be appropriately matched by using the relationship between distances.

[0101] Furthermore, if the relative positional relationship of each sheet is determined based on the first image data and the relative positional relationship of each sheet is determined based on the second image data, and the number of front sides and the number of back sides are different, the processor 36 may use the distance relationship to associate the front sides with the back sides.

[0102] If the number of sheets of paper represented in the first image data is the same as the number of sheets of paper represented in the second image data, the processor 36 may associate the front and back sides of each sheet of paper based on the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data. For example, as explained with reference to Figures 4, 6, and 10, the processor 36 associates the front and back sides. Even if the position of the paper changes between reading the front side and reading the back side, the front and back sides will be appropriately associated as long as the relative positional relationship is maintained.

[0103] The processor 36 may associate the front and back sides of a paper based on the relative positions of the paper and the similarity of the paper's attributes. The similarity of the attributes is the similarity between the attributes of each paper represented in the first image data and the attributes of each paper represented in the second image data.

[0104] For example, the processor 36 identifies one or more combinations of front and back surfaces whose similarity is above a threshold, and associates the front and back surfaces of those combinations based on the relative positions of the paper. The threshold may be predetermined or changed by the user.

[0105] As attributes of paper, at least one of the following is used: paper size, shape, color, material, thickness, and the shape of fingerprints attached to the paper. The smaller the difference between the size of the front and back surfaces, the higher the similarity between the front and back surfaces. The smaller the difference between the shape of the front and back surfaces, the higher the similarity between the front and back surfaces. The smaller the difference between the color of the front and back surfaces, the higher the similarity between the front and back surfaces. The more the material of the paper on the front and back surfaces are the same or similar, the higher the similarity between the front and back surfaces. The smaller the difference between the thickness of the paper on the front and back surfaces, the higher the similarity between the front and back surfaces. Paper material can be distinguished based on light reflection, such as unprocessed paper (i.e., paper without surface coating), glossy paper (paper with a glossy surface coating), and matte paper (i.e., paper with a matte surface coating). Furthermore, by comparing the state of the paper fibers obtained from high-resolution images, it is possible to distinguish between unprocessed papers such as fine paper, plain paper, recycled paper, and Japanese paper. The thickness of the paper can be determined by the degree of bleed-through (for example, the degree to which it is translucent).

[0106] Refer to Figures 14 and 15 to explain the paper attributes. Figure 14 shows the front area. Figure 15 shows the back area. Here, as an example, paper size and color are used as paper attributes.

[0107] The processor 36 calculates the size of the surface area of ​​each surface and the pixel values ​​of the white areas (i.e., areas without text or graphics) on each surface. Similarly, the processor 36 calculates the size of the back surface area of ​​each back surface and the pixel values ​​of the white areas on each back surface.

[0108] For example, as shown in Figure 14, the processor 36 calculates the vertical length Hs1 and horizontal length Ws1 of a rectangular surface area 40a as the size of the surface area 40a. The processor 36 similarly calculates the vertical and horizontal lengths for other surface areas.

[0109] Furthermore, as shown in Figure 14, the processor 36 calculates the pixel values ​​of the white areas 44c on the surface 44 as the color of the surface 44. The processor 36 similarly calculates the pixel values ​​of the white areas on the other surfaces.

[0110] As shown in Figure 15, the processor 36 calculates the vertical length Hb1 and horizontal length Wb1 of the back surface area 52a as the size of the back surface area 52a. The processor 36 similarly calculates the vertical and horizontal lengths for the other back surface areas.

[0111] Furthermore, as shown in Figure 15, the processor 36 calculates the pixel values ​​of the white portion 56c of the back surface 56 as the color of the back surface 56. The processor 36 similarly calculates the pixel values ​​of the white portion of the other back surfaces.

[0112] The processor 36 may calculate the size of the front area as the area of ​​the front area and the size of the back area as the area of ​​the back area. For example, if the paper is not rectangular, the processor 36 may calculate the size as the area of ​​the front area and the back area, respectively. Of course, the processor 36 may calculate the size as the area regardless of the shape of the paper.

[0113] The processor 36 compares the size of each surface area (e.g., length and width) with the size of each back surface area (e.g., length and width). The processor 36 may also compare the area of ​​each surface area with the area of ​​each back surface area. The processor 36 sets the similarity between surfaces and back surfaces with small size differences to a higher value than the similarity between surfaces and back surfaces with large size differences.

[0114] Furthermore, the processor 36 compares the color of each surface (for example, the pixel value of the white area) with the color of each back surface (for example, the pixel value of the white area). The processor 36 sets the similarity between surfaces with small color differences to a higher value than the similarity between surfaces with large color differences.

[0115] Figure 16 shows an example of the similarity between the front and back surfaces.

[0116] The similarity between front 40 and back 52 is 95%. The similarity between front 40 and back 54 is 90%. The similarity between front 40 and back 56 is 95%. The similarity between front 40 and back 58 is 5%. The similarity between front 40 and back 60 is 30%.

[0117] For example, a threshold of 75% is set, and the processor 36 identifies combinations of front and back surfaces that have a similarity of 75% or higher. In the example shown in Figure 16, the back surfaces that are paired with front surface 40 are back surfaces 52, 54, and 56.

[0118] The processor 36 targets the back surfaces 52, 54, and 56 and identifies the back surface that corresponds to the front surface 40 based on the positional relationship between the front surface area and the back surface area, and associates the identified back surface with the front surface 40. At least one of the positional relationships, between relative positional relationships and distance positional relationships, is used as the positional relationship.

[0119] The processor 36 does not calculate the positional relationship between the front area and the back area for back surfaces 58 and 60. This reduces the amount of calculation required compared to calculating the positional relationship between the front area and the back area for all back surfaces, including back surfaces 58 and 60, without using similarity.

[0120] The processor 36 may associate the front and back surfaces based on similarity, without using positional relationships. For example, the processor 36 may associate the front and back surfaces with the highest similarity. If there are multiple combinations of front and back surfaces with the highest similarity, the processor 36 may select one of these combinations and associate the front and back surfaces with it.

[0121] In the example shown in Figure 16, the similarity between surface 40 and back surfaces 52 and 56 is 95%, which is the highest possible similarity value. In this case, the processor 36 associates either back surface 52 or back surface 56 with surface 40. The processor 36 may associate one or more surfaces with one or more back surfaces. In the example shown in Figure 16, the processor 36 may associate both back surfaces 52 and 56 with surface 40.

[0122] If there are multiple combinations of front and back surfaces with the highest similarity, the processor 36 may further use the positional relationship to associate the front and back surfaces. If there is only one such combination, the processor 36 may associate the front and back surfaces based on similarity without using the positional relationship.

[0123] The processing performed by the candidate determination unit 24 will now be described. As mentioned above, the processing performed by the candidate determination unit 24 is carried out by the processor 36.

[0124] The processor 36 determines candidate paper sheets to be matched between the front and back sides based on at least one of the positional relationship of the paper sheets and the similarity of their attributes. The process of determining candidate paper sheets will be described below with reference to Figures 4, 17, and 18. Figure 17 shows the area of ​​the back side. Figure 18 shows the distance and similarity between the front and back sides.

[0125] When scanning the surface, five sheets of paper are scanned, and as a result, first image data representing surfaces 40, 42, 44, 46, and 48 is generated, as shown in Figure 4.

[0126] On the other hand, when scanning the reverse side, three sheets of paper are scanned, and as a result, a second image data representing the reverse side (56, 58, and 62) is generated, as shown in Figure 17.

[0127] Here, as an example, let's assume that the reverse side 62 is a different side of paper F from paper A, B, C, D, and E.

[0128] The back side 62 is positioned closest to the front side 48 compared to the other back sides (see Figures 4 and 17), and the difference ΔL between the back side 62 and the front side 48 is minimized. Also, the relative position of the back side 62 on the second image 50 is the same as the relative position of the front side 48 on the first image 38. Therefore, using the positional relationship, the front side 48 and the back side 62 would be associated as sides of the same paper (e.g., paper E). However, in reality, the back side 62 is not the back side of paper E, but the back side of paper F, which is different from paper E, so this association is not correct. In this case, the processor 36 determines candidate papers that correspond to the front and back sides based on the similarity of attributes.

[0129] Figure 18 shows the similarity and distance difference ΔL between the reverse side 62 and the front sides 40, 42, 44, 46, and 48. For example, paper size, shape, and color are used as paper attributes to calculate each similarity. All similarities are below the threshold (e.g., 75%).

[0130] For example, if the entire surface of the reverse side (62) is yellow, and the areas of the front side (40, 42, 44, 46, 48) that do not contain text or graphics are white, then the similarity calculated based on color will be low.

[0131] The difference between the size of the back surface area 62a of back surface 62 and the size of the front surface area 48a of front surface 48 is small, and the shapes of the back surface area 62a and front surface area 48a are similar, so the similarity is higher than the similarity of the other surfaces, at 50%. However, even so, the color of back surface 62 is completely different from the color of front surface 48, so its similarity is below the threshold.

[0132] Since all similarity scores are below the threshold, the processor 36 excludes the back side 62 from the candidates for matching the front side with the back side and processes it as a different side of the paper. This prevents the back side 62 of a different paper from the paper E that has the front side 48 from being associated with the front side 48.

[0133] The processor 36 may determine the candidates for correspondence based on the positional relationship, or it may determine the candidates for correspondence based on both the positional relationship and the similarity of the attributes.

[0134] The following describes the processing performed by the blank page detection unit 26. As mentioned above, the processing performed by the blank page detection unit 26 is implemented by the processor 36.

[0135] The processor 36 determines whether the front surface of the paper is blank based on the first image data, and whether the back surface of the paper is blank based on the second image data. The processor 36 may determine whether both the front and back surfaces are blank, or it may determine whether either the front or back surface is blank.

[0136] The processor 36 determines that the surface is blank if no characters or figures are written on it (for example, if no characters or figures are detected on the surface), and determines that the surface is not blank if characters or figures are written on it (for example, if characters or figures are detected on the surface). The processor 36 similarly determines whether the back side is blank or not.

[0137] Refer to Figure 19 to explain the blank page detection process. Figure 19 is a diagram showing the result of the blank page detection.

[0138] The front side 40 of paper A is determined not to be blank, and the back side 52 of paper A is determined to be blank (blank = True, single-sided). The front side 42 of paper B is determined not to be blank, and the back side 54 of paper B is determined to be blank (blank = True, single-sided). The front side 44 and back side 56 of paper C are determined not to be blank (blank = False, double-sided). The front side 46 and back side 58 of paper D are determined not to be blank (blank = False, double-sided). The front side 48 of paper E is determined not to be blank, and the back side 60 of paper E is determined to be blank (blank = True, single-sided).

[0139] In this way, it is determined whether the paper is blank or not. The user simply needs to flip the paper over without having to determine whether it is blank or not.

[0140] For example, if one side of a sheet of paper is blank, the user might determine whether the paper is blank or not, under the condition that the front and back sides of the paper are not associated. In this case, the user needs to confirm whether the paper is blank or not.

[0141] In contrast, when the processor 36 determines whether the paper is blank or not, the user does not need to determine this themselves. By simply flipping the paper over and performing the scan, the processor automatically determines whether the paper is blank or not. This reduces the effort required from the user.

[0142] The functions of the image processing device 10 described above are realized, for example, through the cooperation of hardware and software. For instance, the processor reads and executes programs stored in the memory of each device, thereby realizing the functions of each device. The programs are stored in memory via a recording medium such as a CD or DVD, or via a communication path such as a network.

[0143] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operation of the processor in each of the above embodiments may not be performed by a single processor, but may be performed by multiple processors located in physically separate locations working together. In addition, the order of the processor's operations is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0144] 10 Image processing unit, 36 Processor, 38 First image, 50 Second image.

Claims

1. It has a processor, The aforementioned processor, It receives first image data representing the surfaces of multiple sheets of paper, generated by a single scan. The system receives a second image data representing the back sides of the multiple sheets of paper, which is generated by a single reading. If the number of sheets of paper shown in the first image data differs from the number of sheets of paper shown in the second image data, the front and back sides of each sheet of paper are associated based on the positional relationship, including the distance between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data. Information processing device.

2. The aforementioned positional relationship includes the relative positional relationship between each sheet of paper represented in the first image data and each sheet of paper represented in the second image data. The information processing apparatus according to claim 1.

3. The aforementioned processor, If the number of sheets of paper shown in the first image data is the same as the number of sheets of paper shown in the second image data, the front and back sides of each sheet of paper are associated based on the relative positional relationship between each sheet of paper shown in the first image data and each sheet of paper shown in the second image data. The information processing apparatus according to claim 1.

4. The aforementioned processor, The positional relationship and the similarity between the attributes of each sheet of paper shown in the first image data and the attributes of each sheet of paper shown in the second image data are obtained, and if there are multiple combinations of front and back surfaces with the highest similarity, the positional relationship is further used to associate the front and back surfaces of each sheet of paper. The information processing apparatus according to any one of claims 1 to 3.

5. The aforementioned attributes include at least one of the following: paper size, material, shape, thickness, and color. The information processing apparatus according to claim 4.

6. The aforementioned processor further, Based on the first image data or the second image data, it is determined whether the front or back of the paper is blank. The information processing apparatus according to any one of claims 1 to 5.

7. Computers It receives first image data representing the surfaces of multiple sheets of paper, generated by a single scan. The system receives a second image data representing the back sides of the multiple sheets of paper, which is generated by a single reading. If the number of sheets of paper shown in the first image data is different from the number of sheets of paper shown in the second image data, the front and back sides of each sheet of paper are associated based on the positional relationship including the distance between each sheet of paper shown in the first image data and each sheet of paper shown in the image data of the back side. A program designed to make it work in that way.

Images