Image processing device, control method, and control program

Abstract

To provide an image processing apparatus capable of properly connecting images formed on both sides of a medium folded in half, a control method, and a control program.SOLUTION: An image processing apparatus includes: an acquisition unit which acquires a first image obtained by imaging a first surface of a medium and a second image obtained by imaging a second surface of the medium; a straight line detection unit which detects multiple straight lines from the first image or the second image; a determination unit which determines whether the medium is an obliquely folded medium or not, on the basis of the detected straight lines; a folding side detection unit which detects, when the medium is determines as an obliquely folded medium, a folding side from among the detected straight lines on the basis of characteristics of the detected straight lines; and a generation unit which generates a connected image by connecting the first surface included in the first image and the second surface included in the second image with the detected folding side.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an image processing apparatus, a control method, and a control program. [Background technology]

[0002] In image processing devices such as scanners that read media such as manuscripts, a function for reading media larger than the imageable size has been developed. For example, the image processing device captures an image of a medium larger than the imageable size while it is folded in half, and then combines an image of one side and an image of the other side along the fold edge. The image processing device needs to properly detect the fold edge of the medium within the image so that the two images can be correctly combined.

[0003] An image processing device has been disclosed that generates a single composite image by combining a rotated image generated from a scanned image of one side of a document folded in half with a rotated image generated from a scanned image of the other side of the same document (see Patent Document 1). This image processing device acquires each scanned image as a document image, determines the outline of the document in the document image, and sets feature points at concave or convex portions of the outline. If the number of feature points located to the left of the pixel position of the center of gravity of the document outline in the document image is greater than the number of feature points located to the right of the pixel position of the center of gravity, the image processing device determines the order of the rotated image corresponding to the document image to be left. Furthermore, if the number of feature points located to the right of the pixel position of the center of gravity of the document outline in the document image is greater than the number of feature points located to the left of the pixel position of the center of gravity, the image processing device determines the order of the rotated image corresponding to the document image to be right. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-28447 Summary of the Invention [Problem to be solved by the invention]

[0005] An image processing device is required to be able to appropriately combine images captured on each side of a medium folded in half.

[0006] An object of the present invention is to provide an image processing device, a control method, and a control program that are capable of appropriately combining images captured on each side of a medium folded in half. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present invention includes: Diagonal fold a first image captured of a first surface of the medium; Diagonal fold A second image of the second side of the medium; , information indicating an overlapping area or a non-overlapping area of ​​the diagonally folded medium; and an acquisition unit for acquiring the first image or the second image. Corresponds to the edge of diagonally folded media a straight line detection unit that detects a plurality of straight lines; a determination unit that determines whether the medium is a diagonally folded medium based on the detected plurality of straight lines; and a determination unit that determines whether the medium is a diagonally folded medium based on the detected plurality of straight lines. and its relationship to overlapping or non-overlapping areas a folding edge detection unit that detects a folding edge from among the detected straight lines based on the first image and the second image, and so that the edges corresponding to and a generating unit that generates a combined image.

[0008] A control method according to one aspect of the present invention is a control method for an image processing device, comprising: Diagonal fold a first image captured of a first surface of the medium; Diagonal fold A second image of the second side of the medium; , information indicating an overlapping area or a non-overlapping area of ​​the diagonally folded medium; and obtain the first or second image. Corresponds to the edge of diagonally folded media A plurality of straight lines is detected, and whether or not the medium is a diagonally folded medium is determined based on the detected plurality of straight lines. If the medium is determined to be a diagonally folded medium, and its relationship to overlapping or non-overlapping areas Based on the above, a folding edge is detected from the detected multiple straight lines, and the first face included in the first image and the second face included in the second image are identified using the detected folding edge. so that the edges corresponding to A combined image is generated and output.

[0009] A control program according to one aspect of the present invention is a control program for an image processing apparatus, comprising: Diagonal fold a first image captured of a first surface of the medium; Diagonal fold A second image of the second side of the medium; , information indicating an overlapping area or a non-overlapping area of ​​the diagonally folded medium; and obtain the first or second image. Corresponds to the edge of diagonally folded media A plurality of straight lines is detected, and whether or not the medium is a diagonally folded medium is determined based on the detected plurality of straight lines. If the medium is determined to be a diagonally folded medium, and its relationship to overlapping or non-overlapping areas Based on the above, a folding edge is detected from the detected multiple straight lines, and the first face included in the first image and the second face included in the second image are identified using the detected folding edge. so that the edges corresponding to The image processing device is caused to generate a combined image by combining the images and output the combined image. [Effects of the Invention]

[0010] According to the present invention, the image processing device, control method, and control program are capable of appropriately combining images captured on each side of a medium folded in half. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of a configuration of an image processing system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram for explaining a transport path inside the image reading device 100. [Figure 3] 1 is a block diagram showing a schematic configuration of an image reading device 100 and the like. [Figure 4] 2 is a diagram showing a schematic configuration of a first storage device 140 and a first processing circuit 150. FIG. [Figure 5] 10 is an operation sequence showing an example of an image processing operation. [Figure 6] 10A and 10B are schematic diagrams showing an example of a surface image. [Figure 7] FIG. 10 is a schematic diagram illustrating an example of a surface image. [Figure 8] 10 is an operation sequence showing an example of another image processing operation. [Figure 9] 10A and 10B are schematic diagrams showing an example of region information. [Figure 10] FIG. 10 is a schematic diagram illustrating an example of region information. [Figure 11] 10 is a diagram for explaining a transport path inside another image reading device 300. FIG. [Figure 12] 4 is a diagram showing a schematic configuration of a second storage device 410 and a second processing circuit 420. FIG. [Figure 13] 10 is an operation sequence showing an example of another image processing operation. [Figure 14] FIG. 10 is a diagram showing a schematic configuration of another first processing circuit 550. [Figure 15] FIG. 10 is a diagram showing a schematic configuration of another second processing circuit 620. DETAILED DESCRIPTION OF THE INVENTION

[0012] An image processing device, a control method, and a control program according to one aspect of the present invention will be described below with reference to the drawings. However, please note that the technical scope of the present invention is not limited to the embodiments, but extends to the inventions set forth in the claims and their equivalents.

[0013] FIG. 1 is a diagram illustrating an example of a configuration of an image processing system 1 according to an embodiment.

[0014] The image processing system 1 includes an image reading device 100 and an information processing device 200. The image reading device 100 is an example of an image processing device, such as an image scanner. The image reading device 100 transports and captures an image of a medium that is an original. The original is a rectangular medium such as paper, cardboard, card, booklet, or passport. The original may have any shape other than a rectangle. The medium may be sandwiched between a carrier sheet and transported. The carrier sheet is used to fold a medium larger than the maximum size supported by the image reading device 100 in half and transport it. The image reading device 100 may transport a folded medium without using a carrier sheet. The image reading device 100 may be a facsimile machine, a copier, a multifunction printer (MFP), or the like. The image reading device 100 may also be a flatbed scanner device that captures an image of an original without transporting it. The information processing device 200 is a personal computer or the like. The image reading device 100 and the information processing device 200 are connected to each other for communication.

[0015] The image reading device 100 includes a lower housing 101, an upper housing 102, a placement table 103, a discharge table 104, a first operation device 105, a first display device 106, and the like.

[0016] The upper housing 102 is disposed in a position that covers the top surface of the image reading device 100, and is engaged with the lower housing 101 by a hinge so that it can be opened and closed when a medium is jammed or when the inside of the image reading device 100 is cleaned.

[0017] The loading stage 103 is engaged with the lower housing 101 so that the transported medium can be loaded thereon. The ejection stage 104 is engaged with the lower housing 101 so that the ejected medium can be held thereon.

[0018] The first operation device 105 has an input device such as a button and an interface circuit that acquires signals from the input device, accepts input operations by a user, and outputs operation signals according to the user's input operations. The first display device 106 has a display such as a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like, and an interface circuit that outputs image data to the display, and displays the image data on the display.

[0019] 1, arrow A1 indicates the medium transport direction, and arrow A2 indicates the width direction perpendicular to the medium transport direction. In the following, "upstream" refers to the upstream side of the medium transport direction A1, and "downstream" refers to the downstream side of the medium transport direction A1.

[0020] FIG. 2 is a diagram for explaining the transport path inside the image reading device 100. As shown in FIG.

[0021] The transport path inside the image reading device 100 includes a first medium sensor 111, a feed roller 112, a separation roller 113, a second medium sensor 114, an ultrasonic sensor 115, a first transport roller 116, a second transport roller 117, an imaging device 118, a third transport roller 119, and a fourth transport roller 120. The feed roller 112, the separation roller 113, and the first to fourth transport rollers 116, 117, 119, and 120 are an example of a transport unit that transports the medium. Note that the number of each roller is not limited to one, and there may be multiple of each. In this case, the rollers are arranged side by side at intervals in the width direction A2.

[0022] The top surface of the lower housing 101 forms a lower guide 107a of the medium transport path, and the bottom surface of the upper housing 102 forms an upper guide 107b of the medium transport path.

[0023] The first medium sensor 111 is disposed upstream of the feed roller 112 and the separation roller 113. The first medium sensor 111 has a contact detection sensor and detects whether or not a medium is placed on the placement table 103. The first medium sensor 111 generates and outputs a first medium signal whose signal value changes depending on whether or not a medium is placed on the placement table 103. Note that the first medium sensor 111 is not limited to a contact detection sensor, and any other sensor capable of detecting the presence or absence of a medium, such as an optical detection sensor, may be used as the first medium sensor 111.

[0024] The feed roller 112 is provided in the lower housing 101 and feeds media placed on the mounting table 103. The separation roller 113 is a so-called brake roller or retard roller, and is disposed in the upper housing 102 facing the feed roller 112. The image reading device 100 has two operating modes: a separation mode in which, when multiple media are placed on the mounting table 103, the media are separated and transported, and a non-separation mode in which media such as passports are transported without separation. In the separation mode, the separation roller 113 rotates or stops in the direction opposite to the media feeding direction, and in the non-separation mode, it rotates in the media feeding direction (it is rotated following the feed roller 112 or is rotated by the driving force from the motor).

[0025] When the non-separation mode is set as the operating mode, the image reading device 100 can transport, without separating, a medium that is larger than the maximum size supported by the image reading device 100 and that is folded in half. Also, when the downstream end of the carrier sheet is closed and the folded medium is sandwiched between the carrier sheet, the image reading device 100 can transport the folded medium without separating it, even when the separation mode is set as the operating mode.

[0026] The second medium sensor 114 is located downstream of the feed roller 112 and the separation roller 113 and upstream of the ultrasonic sensor 115, and detects whether a medium is present at that position. The second medium sensor 114 includes a light emitter and a light receiver located on one side of the medium transport path, and a light guide member located opposite the light emitter and the light receiver across the transport path. The light guide member is a light guide tube such as a U-shaped prism. The light emitter is an LED (Light Emitting Diode) or the like, and emits light toward the transport path. The light receiver is a photodiode or the like, and receives light emitted by the light emitter and guided by the light guide member, and generates and outputs a second medium signal, which is an electrical signal corresponding to the intensity of the received light. When a medium is present at the position of the second medium sensor 114, the light emitted by the light emitter is blocked by the medium, so the signal value of the second medium signal changes depending on whether a medium is present or not at the position of the second medium sensor 114.

[0027] A reflective member such as a mirror may be used instead of the light-guiding member. The light emitter and light receiver may be positioned opposite each other across the transport path, and the light-guiding member may be omitted. Second medium sensor 114 may detect the presence of a medium using a contact detection sensor or the like that passes a predetermined current when the medium is in contact or when the medium is not in contact.

[0028] The ultrasonic sensor 115 is disposed downstream of the feed roller 112 and the separation roller 113 and upstream of the first and second conveyor rollers 116 and 117. The ultrasonic sensor 115 includes an ultrasonic transmitter 115a and an ultrasonic receiver 115b. The ultrasonic transmitter 115a and the ultrasonic receiver 115b are disposed near the medium conveyance path, facing each other across the conveyance path. The ultrasonic transmitter 115a emits ultrasonic waves. Meanwhile, the ultrasonic receiver 115b receives the ultrasonic waves emitted by the ultrasonic transmitter 115a that have passed through the medium, and generates and outputs an ultrasonic signal, which is an electrical signal corresponding to the received ultrasonic waves. The ultrasonic signal indicates transmission information of the ultrasonic waves that pass through the medium at multiple positions within the medium being conveyed by the conveyance unit. The transmission information indicates the magnitude of the ultrasonic waves received by the ultrasonic receiver 115b. Note that the number of ultrasonic sensors 115 is not limited to one and may be multiple. In this case, the multiple ultrasonic sensors 115 are arranged side by side at intervals in the width direction A2.

[0029] The first conveying roller 116 and the second conveying roller 117 are disposed downstream of the feeding roller 112 and the separation roller 113, facing each other across the medium conveying path. The first conveying roller 116 and the second conveying roller 117 convey the medium fed by the feeding roller 112 and the separation roller 113 downstream.

[0030] The imaging device 118 is disposed downstream of the first conveying roller 116 and the second conveying roller 117, and captures images of the medium conveyed by the first conveying roller 116 and the second conveying roller 117. The imaging device 118 includes a first imaging device 118a and a second imaging device 118b disposed opposite each other across the medium conveying path.

[0031] The first imaging device 118a has a line sensor based on a CIS (Contact Image Sensor) of a 1:1 optical system type having CMOS (Complementary Metal Oxide Semiconductor) imaging elements arranged linearly in the main scanning direction. The first imaging device 118a also has a lens that forms an image on the imaging element and an A / D converter that amplifies and analog-to-digital (A / D) converts the electrical signal output from the imaging element. The first imaging device 118a generates and outputs a surface image by capturing an image of the surface of the medium being transported. The surface of the medium is an example of a first side of the medium, and the surface image is an example of a first image.

[0032] Similarly, the second imaging device 118b has a CIS line sensor with a 1x magnification optical system having CMOS imaging elements arranged linearly in the main scanning direction. The second imaging device 118b also has a lens that forms an image on the imaging element and an A / D converter that amplifies and analog-to-digital (A / D) converts the electrical signal output from the imaging element. The second imaging device 118b captures an image of the back side of the medium being conveyed, generates a back side image, and outputs the image. The back side of the medium is an example of the second side of the medium, and the back side image is an example of the second image.

[0033] Instead of a CIS line sensor of an equal magnification optical system type equipped with a CMOS imaging element, a CIS line sensor of an equal magnification optical system type equipped with a CCD (Charge Coupled Device) imaging element may be used, or a reduction optical system type line sensor equipped with a CMOS or CCD imaging element may be used.

[0034] The third conveying roller 119 and the fourth conveying roller 120 are disposed downstream of the imaging device 118, facing each other across the medium conveying path. The third conveying roller 119 and the fourth conveying roller 120 discharge the medium that has been conveyed by the first conveying roller 116 and the second conveying roller 117 and that has been imaged by the imaging device 118 onto the discharge tray 104.

[0035] The media placed on the mounting table 103 are transported between the lower guide 107a and the upper guide 107b in the media transport direction A1 by the rotation of the feed roller 112 in the direction of arrow A3 in FIG. 2, i.e., the media feed direction. When operating in separation mode, the separation roller 113 rotates in the direction of arrow A4, i.e., the opposite direction to the media feed direction, during media transport. When multiple media are placed on the mounting table 103, the feed roller 112 and the separation roller 113 act to separate only the media placed on the mounting table 103 that are in contact with the feed roller 112. This operates to restrict the transport of media other than the separated media (preventing double feeding). On the other hand, when operating in non-separation mode, the separation roller 113 rotates in the opposite direction of arrow A4, i.e., the media transport direction, driven by the feed roller 112.

[0036] The medium is fed between first conveyor roller 116 and second conveyor roller 117 while being guided by lower guide 107a and upper guide 107b. The medium is fed between first image capture device 118a and second image capture device 118b as first conveyor roller 116 and second conveyor roller 117 rotate in the directions of arrows A5 and A6, respectively. After being scanned by image capture device 118, the medium is ejected onto ejection tray 104 as third conveyor roller 119 and fourth conveyor roller 120 rotate in the directions of arrows A7 and A8, respectively.

[0037] FIG. 3 is a block diagram showing a schematic configuration of the image reading device 100 and the information processing device 200. As shown in FIG.

[0038] In addition to the above-described configuration, the image reading device 100 further includes a motor 131, a first interface device 132, a first storage device 140, and a first processing circuit 150. The first operation device 105, the first display device 106, the first medium sensor 111, the second medium sensor 114, the ultrasonic sensor 115, the imaging device 118, the motor 131, the first interface device 132, the first storage device 140, and the first processing circuit 150 are connected to each other via a bus.

[0039] The motor 131 includes one or more motors. The motor 131 rotates the feed roller 112, the separation roller 113, and the first to fourth transport rollers 116, 117, 119, and 120 in response to a control signal from the first processing circuit 150 to transport the medium. Note that one of the first transport roller 116 and the second transport roller 117 may be a driven roller that rotates following the rotation of the other roller. Also, one of the third transport roller 119 and the fourth transport roller 120 may be a driven roller that rotates following the rotation of the other roller.

[0040] The first interface device 132 has an interface circuit conforming to a serial bus such as a USB (Universal Serial Bus), and is electrically connected to the information processing device 200 to transmit and receive various information. Note that instead of the first interface device 132, a communication device having a communication interface circuit for transmitting and receiving signals to and from the information processing device 200 via a predetermined communication network in accordance with a predetermined communication protocol may be used. The predetermined communication protocol is TCP / IP (Transmission Control Protocol / Internet Protocol), wireless LAN (Local Area Network), etc.

[0041] The first storage device 140 includes a memory device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or optical disk. The first storage device 140 also stores computer programs, databases, tables, and the like used for various processes of the image reading device 100. The computer programs may be installed into the first storage device 140 from a computer-readable portable recording medium using a known setup program or the like. The portable recording medium is, for example, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or the like.

[0042] The first processing circuit 150 operates based on a program stored in advance in the first storage device 140. The first processing circuit 150 is, for example, a CPU (Central Processing Unit). The first processing circuit 150 may be a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0043] The first processing circuit 150 is connected to the first operation device 105, the first display device 106, the first medium sensor 111, the second medium sensor 114, the ultrasonic sensor 115, the imaging device 118, the motor 131, the first interface device 132, the first storage device 140, and the like, and controls each of these components. The first processing circuit 150 controls the driving of the motor 131 and the imaging of the imaging device 118, and acquires a front image and a back image from the imaging device 118. Based on the acquired front image or back image, the first processing circuit 150 determines whether the medium is a diagonally folded medium, and if the medium is a diagonally folded medium, detects the fold edge. The first processing circuit 150 generates a combined image by combining the front side of the medium included in the front image and the back side of the medium included in the back image at the detected fold edge, and transmits the combined image to the information processing device 200 via the first interface device 132.

[0044] The information processing device 200 includes a second operation device 201, a second display device 202, a second interface device 203, a second storage device 210, a second processing circuit 220, and the like.

[0045] The second operation device 201 has input devices such as a touch panel input device, a keyboard, and a mouse, and an interface circuit for acquiring signals from the input devices, accepts input operations by a user, and outputs operation signals according to the user's input operations. The second display device 202 has a display such as a liquid crystal display or organic electroluminescence display, and an interface circuit for outputting image data to the display, and displays the image data on the display.

[0046] The second interface device 203 has an interface circuit similar to that of the first interface device 132, and is electrically connected to the image reading device 100 to send and receive various information. Note that instead of the second interface device 203, a communication device having a communication interface circuit for sending and receiving signals to and from the image reading device 100 via a predetermined communication network in accordance with a predetermined communication protocol may be used.

[0047] The second storage device 210 includes a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or an optical disk. The second storage device 210 also stores computer programs, databases, tables, and the like used for various processes of the image reading device 100. Computer programs may be installed into the second storage device 210 from a computer-readable portable recording medium such as a CD-ROM or a DVD-ROM using a known setup program or the like.

[0048] The second processing circuit 220 operates based on a program stored in advance in the second storage device 210. The second processing circuit 220 is, for example, a CPU. The second processing circuit 220 may also be a DSP, an LSI, an ASIC, an FPGA, or the like.

[0049] The second processing circuit 220 is connected to the second operation device 201, the second display device 202, the second interface device 203, the second storage device 210, etc., and controls each of these components. The second processing circuit 220 receives an image from the image reading device 100 via the second interface device 203 and displays it on the second display device 202.

[0050] FIG. 4 is a diagram showing a schematic configuration of the first storage device 140 and the first processing circuit 150. As shown in FIG.

[0051] 4, the first storage device 140 stores a control program 141, an acquisition program 142, a line detection program 143, a determination program 144, a fold edge detection program 145, a generation program 146, an output control program 147, and the like. Each of these programs is a functional module implemented by software running on a processor. The first processing circuit 150 reads each program stored in the first storage device 140 and operates in accordance with the read program. As a result, the first processing circuit 150 functions as a control unit 151, an acquisition unit 152, a line detection unit 153, a determination unit 154, a fold edge detection unit 155, a generation unit 156, and an output control unit 157.

[0052] FIG. 5 is a flowchart showing an example of the image processing operation of the image reading device 100.

[0053] An example of the image processing operation of the image reading device 100 will be described below with reference to the flowchart shown in Fig. 5. The flow of the operation described below is executed mainly by the first processing circuit 150 in cooperation with each element of the image reading device 100 based on a program stored in advance in the first storage device 140. The flow of the operation shown in Fig. 5 is executed periodically.

[0054] First, the control unit 151 waits until the user inputs an instruction to read a medium using the first operating device 105 or the information processing device 200, and an operation signal instructing the reading of a medium is received from the first operating device 105 or the first interface device 132 (step S101).

[0055] Next, control unit 151 acquires a medium signal from first medium sensor 111 and determines whether or not a medium is placed on mounting table 103 based on the acquired medium signal (step S102). If no medium is placed on mounting table 103, control unit 151 returns the process to step S101 and waits until a new operation signal is received from first operating device 105 or first interface device 132.

[0056] On the other hand, if a medium is placed on the placement table 103, the control unit 151 drives the motor 131 to rotate the feed roller 112, the separation roller 113, and / or the first to fourth transport rollers 116, 117, 119, and 120 to transport the medium (step S103).

[0057] Next, the control unit 151 waits until the entire medium is imaged (step S104). The control unit 151 determines whether the trailing edge of the medium has passed the position of the second medium sensor 114, for example, based on the second medium signal received from the second medium sensor 114. The control unit 151 periodically acquires the second medium signal from the second medium sensor 114, and determines that the trailing edge of the medium has passed the position of the second medium sensor 114 when the signal value of the second medium signal changes from a value indicating the presence of the medium to a value indicating the absence of the medium. The control unit 151 determines that the trailing edge of the medium has passed the imaging position of the imaging device 118 and that the entire medium has been imaged when a first predetermined time has elapsed since the trailing edge of the medium passed the position of the second medium sensor 114. The first predetermined time is set to the time required for the medium to move from the position of the second medium sensor 114 to the imaging position of the imaging device 118. Note that the control unit 151 may also determine that the entire transported medium has been imaged when a predetermined time has elapsed since the start of medium feeding.

[0058] When the entire transported medium is imaged, the acquisition unit 152 acquires a front image of the medium and a back image of the medium, and stores them in the first storage device 140 (step S105).

[0059] Next, the line detection unit 153 detects the edges of the medium, that is, a plurality of lines (line segments) corresponding to the sides that make up the outer shape of the medium, from the front surface image or the back surface image (step S106).

[0060] The line detector 153 calculates, for each vertical line extending in the vertical direction (sub-scanning direction) within the front or back image, the absolute value of the difference in gradation values ​​between each pixel on each vertical line and its adjacent pixels in the vertical direction (hereinafter referred to as the adjacent difference value), starting from the top. The line detector 153 detects pixels within each vertical line whose adjacent difference value exceeds a gradation threshold as edge pixels. The gradation value may be a luminance value or a color value (R, G, or B). The gradation threshold is set to a luminance value difference (e.g., 20) that allows a person to visually distinguish differences in luminance within an image. The line detector 153 detects the first edge pixel detected within each vertical line, i.e., the uppermost pixel, as the upper edge pixel, and the last edge pixel detected within each vertical line, i.e., the lowermost pixel, as the lower edge pixel.

[0061] Similarly, the line detector 153 calculates adjacent difference values ​​for each horizontal line extending horizontally (in the main scanning direction) in the front or back image, starting from the left, and detects pixels in each horizontal line whose adjacent difference value exceeds the gradation threshold as edge pixels. The line detector 153 detects the first edge pixel detected in each horizontal line, i.e., the pixel located on the leftmost side, as the left-edge edge pixel, and detects the last edge pixel detected in each horizontal line, i.e., the pixel located on the rightmost side, as the right-edge pixel.

[0062] The line detector 153 may calculate, as the adjacent difference value, the absolute value of the difference in gradation values ​​between two pixels that are a predetermined distance away from each pixel in the input image in the horizontal or vertical direction. The line detector 153 may also detect edge pixels by comparing the gradation value of each pixel in the input image with a threshold. For example, if the gradation value of a specific pixel is less than the threshold and the gradation value of a pixel that is horizontally adjacent to the specific pixel or a pixel that is a predetermined distance away from the specific pixel is equal to or greater than the threshold, the line detector 153 detects the specific pixel as an edge pixel.

[0063] Next, the line detector 153 groups the detected top edge pixels, bottom edge pixels, left edge pixels, and right edge pixels. The line detector 153 groups the top edge pixels, bottom edge pixels, left edge pixels, and right edge pixels that are arranged so as to extend along a specific direction into the same group.

[0064] For example, the line detection unit 153 sets the top edge pixel located on the left side of the detected top edge pixels as the pixel of interest. The line detection unit 153 scans the top edge pixels toward the right, and if the slope of the line passing through the pixel of interest and a specific top edge pixel is less than a predetermined angle (e.g., 45°), it extracts the specific top edge pixel as a candidate pixel and sets the line as a candidate line. If there is a top edge pixel further to the right of the candidate pixel that is less than a predetermined distance from the candidate line, the line detection unit 153 groups the pixel of interest, the candidate pixel, and the top edge pixels that are less than the predetermined distance from the candidate line into one group. If no candidate pixel is extracted, or if there is no top edge pixel to the right of the candidate pixel that is less than the predetermined distance from the candidate line, the line detection unit 153 does not group the pixel of interest. The line detection unit 153 repeats the same process for the top edge pixels that are not grouped. The line detector 153 also performs similar processing on the bottom edge pixels, left edge pixels, and right edge pixels.

[0065] Next, the line detection unit 153 uses the least squares method to detect lines from each of the grouped top-edge pixels, bottom-edge pixels, left-edge pixels, and right-edge pixels. The line detection unit 153 may also use a Hough transform to detect lines. When overlapping lines are detected from different edge pixel groups (for example, lines whose edge pixels at both ends are within a predetermined range), the line detection unit 153 deletes one of the lines. For example, when substantially the same line is detected from the group of top-edge pixels and the group of left-edge pixels, the line detection unit 153 deletes one of the lines.

[0066] When a medium larger than the maximum size supported by the image reading device 100 is conveyed, the medium is folded in half and conveyed so that the image reading device 100 can convey it. When a folded medium is conveyed, the image reading device 100 generates a combined image by joining the front side of the medium included in the front image and the back side of the medium included in the back image at the fold edge. This allows the image reading device 100 to generate a combined image capturing an image of a medium larger than the maximum size supported by the image reading device 100. However, if the medium is folded in half so that the upper left and upper right corners and the lower left and lower right corners of the medium exactly match, the image reading device 100 cannot determine whether the conveyed medium is a folded medium or two media conveyed in a double feed. In addition, in this case, the image reading device 100 cannot determine which of the edges of the folded medium included in the front and back images is the fold edge.

[0067] Therefore, when a medium is folded in half so that the upper left and right corners and the lower left and right corners of the medium do not match (are misaligned), the image reading device 100 considers the conveyed medium to be a folded medium that has been folded in half.The image reading device 100 then generates a combined image by combining the front side of the medium included in the front side image and the back side of the medium included in the back side image at the fold edge.Hereinafter, a medium that has been folded in half so that the upper left and right corners and the lower left and right corners do not match (are misaligned) may be referred to as a diagonally folded medium.

[0068] 6A, 6B, and 7 are schematic diagrams showing examples of surface images of a diagonally folded medium.

[0069] 6A shows a front surface image P1 captured of a diagonally folded medium M1. The medium M1 has a rectangular shape surrounded by top edges T1 and T2, bottom edges T3 and T4, left edge T5, and right edge T6, and is folded in half along folding edge F so that the left edge T5 is positioned on the right edge T6 side. Hereinafter, the portion of the diagonally folded medium that is positioned on the side of the imaging device 118 that captured the front surface image P1 may be referred to as the upper portion N1, and the portion that is positioned on the opposite side of the imaging device 118 that captured the front surface image P1 may be referred to as the lower portion N2.

[0070] In the surface image P1, pixels corresponding to the upper end T1 and left end T5 of the upper portion N1 and the upper end T2 of the lower portion N2 are detected as top edge pixels. Pixels corresponding to the fold edge F, the lower end T4 of the lower portion N2, and the lower end T3 of the upper portion N1 are detected as bottom edge pixels. Pixels corresponding to the top end T1 and fold edge F of the upper portion N1 are detected as left edge pixels. Pixels corresponding to the left end T5, right end T6, and the lower end T3 of the upper portion N1 are detected as right edge pixels.

[0071] The top edge pixels are classified into a group corresponding to the top edge T1 of the top portion N1, a group corresponding to the left edge T5, and a group corresponding to the top edge T2 of the bottom portion N2, and lines L1, L5, and L2 are detected from each group. The bottom edge pixels are classified into a group corresponding to the folding edge F, a group corresponding to the bottom edge T4 of the bottom portion N2, and a group corresponding to the bottom edge T3 of the top portion N1, and lines L7, L4, and L3 are detected from each group. The left edge pixels are classified into a group corresponding to the top edge T1 of the top portion N1 and a group corresponding to the folding edge F, and lines L1 and L7 are detected from each group. The right edge pixels are classified into a group corresponding to the left edge T5, a group corresponding to the right edge T6, and a group corresponding to the bottom edge T3 of the top portion N1, and lines L5, L6, and L3 are detected from each group.

[0072] Since the diagonally extending straight lines L1, L3, L5, and L7 are detected in duplicate, one of the duplicate detected straight lines is deleted. As a result, seven straight lines, straight lines L1 to L7, are detected.

[0073] The line detection unit 153 may classify edge pixels sandwiching edge pixels belonging to other groups into different groups, detecting discontinuous lines as separate lines. In this case, among the top edge pixels, the group corresponding to the left edge T5 is classified into a group to the left of the group corresponding to the top edge T2 and a group to the right, and two lines L5 are detected separately from each group. Among the right edge pixels, the group corresponding to the left edge T5 is classified into a group above the group corresponding to the right edge T6 and a group other than the group, and two lines L4 are detected separately from each group. Among the right edge pixels, the group corresponding to the right edge T6 is classified into a group below the group corresponding to the left edge T5 and a group other than the group, and two lines L5 are detected separately from each group. Therefore, in this case, two lines L5 and two lines L6 are detected, for a total of nine lines.

[0074] 6B shows a front image P2 of a diagonally folded medium M2. Medium M2 is folded in half like medium M1. However, medium M2 is folded in half so that the bottom left corner is positioned on the right edge T6.

[0075] In the surface image P2, pixels corresponding to the upper end T1 and left end T5 of the upper portion N1 and the upper end T2 of the lower portion N2 are detected as top edge pixels. Pixels corresponding to the fold side F and the lower end T4 of the lower portion N2 are detected as bottom edge pixels. Pixels corresponding to the top end T1 and fold side F of the upper portion N1 are detected as left edge pixels. Pixels corresponding to the left end T5 and right end T6 are detected as right edge pixels.

[0076] The top edge pixels are classified into a group corresponding to the top edge T1 of the top portion N1, a group corresponding to the left edge T5, and a group corresponding to the top edge T2 of the bottom portion N2, and lines L1, L5, and L2 are detected from each group. The bottom edge pixels are classified into a group corresponding to the fold edge F and a group corresponding to the bottom edge T4 of the bottom portion N2, and lines L7 and L4 are detected from each group. The left edge pixels are classified into a group corresponding to the top edge T1 of the top portion N1 and a group corresponding to the fold edge F, and lines L1 and L7 are detected from each group. The right edge pixels are classified into a group corresponding to the left edge T5 and a group corresponding to the right edge T6, and lines L5 and L6 are detected from each group.

[0077] Since the diagonally extending straight lines L1, L5, and L7 are detected in duplicate, one of the duplicate detected straight lines is deleted. As a result, six straight lines, L1, L2, L4, L5, L6, and L7, are detected. Note that six straight lines are also detected if the medium is folded in half so that the bottom left corner is positioned inside the right edge of the medium.

[0078] 7 shows a surface image P3 of a diagonally folded medium M3. Medium M3 is folded in half like medium M1. However, medium M3 is folded in half so that the bottom left corner is aligned with the top right corner.

[0079] In the surface image P3, pixels corresponding to the top edge T1 and left edge T5 of the upper portion N1 are detected as top edge pixels. Pixels corresponding to the fold side F and the bottom edge T4 of the lower portion N2 are detected as bottom edge pixels. Pixels corresponding to the top edge T1 and fold side F of the upper portion N1 are detected as left edge pixels. Pixels corresponding to the left edge T5 and right edge T6 are detected as right edge pixels.

[0080] The top edge pixels are classified into a group corresponding to the top edge T1 of the upper portion N1 and a group corresponding to the left edge T5, and lines L1 and L5 are detected from each group. The bottom edge pixels are classified into a group corresponding to the fold edge F and a group corresponding to the bottom edge T4 of the lower portion N2, and lines L7 and L4 are detected from each group. The left edge pixels are classified into a group corresponding to the top edge T1 of the upper portion N1 and a group corresponding to the fold edge F, and lines L1 and L7 are detected from each group. The right edge pixels are classified into a group corresponding to the left edge T5 and a group corresponding to the right edge T6, and lines L5 and L6 are detected from each group.

[0081] Since the diagonally extending straight lines L1, L5, and L7 are detected in duplicate, one of the detected duplicate straight lines is deleted. As a result, five straight lines, L1, L4, L5, L6, and L7, are detected.

[0082] Next, the determination unit 154 determines whether the medium is a diagonally folded medium based on the multiple straight lines detected by the line detection unit 153 (step S107). As shown in FIGS. 6A, 6B, and 7, the number of straight lines corresponding to the edges of the medium detected from the captured image of the diagonally folded medium is five, six, or seven. Note that if discontinuous straight lines are detected as separate straight lines, nine straight lines may be detected. Furthermore, due to noise and other factors contained in the image, seven or more straight lines may be detected. Therefore, the determination unit 154 determines that the medium is a diagonally folded medium if the number of straight lines detected by the line detection unit 153 as lines corresponding to the edges of the medium is five or more, and determines that the medium is not a diagonally folded medium if the number of detected straight lines is less than five. This allows the determination unit 154 to accurately determine whether the medium is a diagonally folded medium.

[0083] In addition, the judgment unit 154 may determine that the medium is a diagonally folded medium if the number of detected multiple straight lines is five or more and seven or less, and may determine that the medium is not a diagonally folded medium if the number of detected multiple straight lines is less than five or more than seven.

[0084] If the determination unit 154 determines that the medium is not a diagonally folded medium, the output control unit 157 outputs the front and back images by transmitting them to the information processing device 200 via the first interface device 132 (step S108). Then, the output control unit 157 proceeds to step S113. The second processing circuit 220 of the information processing device 200 receives the front and back images from the image reading device 100 via the second interface device 203 and displays them on the second display device 202.

[0085] On the other hand, if the determination unit 154 determines that the medium is a diagonally folded medium, the fold edge detection unit 155 detects the fold edge from among the multiple straight lines detected by the line detection unit 153 based on the characteristics of the multiple straight lines detected by the line detection unit 153 (step S109).

[0086] For example, the folding side detection unit 155 detects the lengths of the multiple straight lines detected by the line detection unit 153 as the characteristics of each line, and detects the longest straight line among the multiple detected straight lines as the folding side.

[0087] Most media scanned by the image reading device 100 are standard-sized media, such as A-size or B-size. The aspect ratio of standard sizes, such as A-size or B-size, is 1:√2 (≈1.4). When a standard-sized medium, such as A-size or B-size, is folded in half near the center of the longitudinal edge so that the folded edge is slightly inclined relative to the lateral edge, the ratio of the length of the folded edge to the length of each longitudinal edge of the folded medium is 1:approximately 0.7. Therefore, the length of the folded edge is greater than the length of each longitudinal edge of the folded medium. Furthermore, when the folded edge is inclined relative to the lateral edge, the length of the folded edge is greater than the length of the lateral edge. Therefore, as shown in FIGS. 6(A), 6(B), and 7, the longest straight line constituting the edge of the folded medium is the folded edge. The folding side detection unit 155 can detect the folding side with high accuracy by detecting the longest straight line among the detected straight lines as the folding side.

[0088] In addition, when discontinuous lines are detected as separate lines, the fold edge detection unit 155 may detect the orientation of the multiple lines detected by the line detection unit 153 as a feature of each line, and detect a line that is not parallel to other lines among the multiple detected lines as a fold edge.

[0089] When a rectangular medium is folded in half, the two opposing sides of each of the upper and lower portions that intersect with the folding edge remain parallel to one another. As shown in Figure 6(A), when the medium is folded in half so that the corners of the upper portion do not overlap with the corners of the lower portion, a line L1 corresponding to the upper end T1 and a line L3 corresponding to the lower end T3 are detected in the upper portion N1. Furthermore, a line L2 corresponding to the upper end T2 and a line L4 corresponding to the lower end T4 are detected in the lower portion N2. Lines L1 and L3 are parallel to one another, and lines L2 and L4 are parallel to one another.

[0090] Furthermore, when a medium is folded in half so that the corners of the upper portion do not overlap the corners of the lower portion and vice versa, the two opposing sides of the fold edge partially overlap and are separated by the overlapping portions. The separated portions of each side lie on the same straight line and are parallel to each other. That is, the upper and lower portions of the straight line L5 corresponding to the left edge T5 are parallel to each other. Furthermore, the upper and lower portions of the straight line L6 corresponding to the right edge T6 are parallel to each other. However, among the straight lines detected by the straight line detection unit 153, there is no straight line parallel to the straight line L7 corresponding to the fold edge F.

[0091] Therefore, the fold edge detection unit 155 can properly detect the fold edge by detecting, among the detected multiple straight lines, a straight line that is not parallel to the other straight lines as the fold edge. Note that, as shown in FIG. 6B or FIG. 7, when a medium is folded in half so that a corner of the upper portion overlaps with a corner of the lower portion, or so that a corner of the lower portion overlaps with a corner of the upper portion, it is not possible to detect the fold edge based solely on the orientation of each straight line. Therefore, when detecting the fold edge based solely on the orientation of each straight line, it is necessary to limit the method of folding the medium in half.

[0092] Next, the generating unit 156 generates a combined image by combining the front side included in the front side image and the back side included in the back side image along the folding side detected by the folding side detecting unit 155 (step S110).

[0093] The generation unit 156 detects edge pixels in each vertical line and each horizontal line in the front image and the back image in the same manner as in the process of step S105. Next, the generation unit 156 uses the least squares method or the Hough transform to detect a plurality of horizontal lines extending in a substantially horizontal direction and a plurality of vertical lines extending in a substantially vertical direction from the edge pixel groups detected in the vertical lines and the edge pixel groups detected in the horizontal lines, respectively.

[0094] Next, the generating unit 156 extracts a plurality of quadrangles each composed of four lines, two of which are substantially perpendicular to each other, from the plurality of detected lines. The generating unit 156 selects a first horizontal line from the plurality of horizontal lines, and extracts a second horizontal line from the plurality of horizontal lines, the second horizontal line having an angle with respect to the first horizontal line within a first range and a distance from the first horizontal line that is equal to or greater than a predetermined distance. The first range is set to the range of angles that can be formed between a folding edge and two sides that face each other across the folding edge in a diagonally folded medium (for example, within ±30°). Next, the generating unit 156 extracts a first vertical line from the plurality of vertical lines, the first vertical line having an angle with respect to the first horizontal line within a second range. The second range is set to the range of angles that can be formed between a folding edge and two sides that intersect with the folding edge in a diagonally folded medium (for example, within ±30° of 90°). Next, the generating unit 156 extracts a second vertical line having an angle with respect to the first horizontal line within a second range and a distance from the first vertical line that is equal to or greater than a predetermined distance. The predetermined distance is determined in advance according to the size of the medium to be read by the image reading device 100.

[0095] The generation unit 156 extracts all combinations of the first horizontal line, the second horizontal line, the first vertical line, and the second vertical line that satisfy the above conditions for all extracted horizontal and vertical lines, and extracts quadrilaterals formed from each extracted combination. Among the quadrilaterals extracted from the front image, the generation unit 156 detects the quadrilateral with two sides joined and the largest area as the front side of the medium. Furthermore, among the quadrilaterals extracted from the back image, the generation unit 156 detects the quadrilateral with two sides joined and the largest area as the back side of the medium. This allows the generation unit 156 to properly detect the upper portion of the diagonally folded medium that is located on the side of the imaging device 118 that generated the image.

[0096] Next, the generation unit 156 identifies the edges corresponding to the fold edge detected by the fold edge detection unit 155 on the extracted front and back sides. That is, for the same image as the image in which the fold edge was detected, the generation unit 156 identifies the edge closest to the fold edge of the diagonally folded medium among the top, bottom, left, and right edges of the front or back side of the medium as the edge corresponding to the fold edge. For an image different from the image in which the fold edge was detected, the generation unit 156 identifies the edges that match the edges identified as the edges corresponding to the fold edge in the same image as the image in which the fold edge was detected as the edge corresponding to the fold edge. That is, if the edges identified in the same image as the image in which the fold edge was detected are the top, bottom, left, and right edges, respectively, the generation unit 156 identifies the top, bottom, right, and left edges in the image different from the image in which the fold edge was detected as the edges corresponding to the fold edge.

[0097] Next, the generation unit 156 cuts out the extracted front and back sides from the front image and back image, respectively, and generates a combined image by combining the cut-out front and back sides at the edge identified as the edge corresponding to the folding edge.

[0098] Next, the output control unit 157 outputs the generated combined image by transmitting it to the information processing device 200 via the first interface device 132 (step S111). The second processing circuit 220 of the information processing device 200 receives the combined image from the image reading device 100 via the second interface device 203, and displays it on the second display device 202.

[0099] Next, control unit 151 determines whether or not a medium remains on mounting table 103 based on the first medium signal received from first medium sensor 111 (step S112). If a medium remains on mounting table 103, control unit 151 returns the process to step S104 and repeats the processes of steps S104 to S112.

[0100] On the other hand, if there are no media remaining on the mounting table 103, the control unit 151 stops the motor 131, the feed roller 112, the separation roller 113, and the first to fourth transport rollers 116, 117, 119, and 120 (step S113), and ends the series of steps.

[0101] In step S107, the determination unit 154 may determine that the medium is a diagonally folded medium only when the non-separation mode is set as the operation mode.

[0102] Alternatively, the determination unit 154 may determine that the medium is an obliquely folded medium only when the non-separation mode is set as the operating mode, or when the separation mode is set as the operating mode and the medium is being transported sandwiched between carrier sheets. The carrier sheet may have a predetermined mark printed on the side where the fold is located. The determination unit 154 uses known image processing technology to determine whether the predetermined mark is included in the front or back image, and if the predetermined mark is included in the medium, determines that the medium is being transported sandwiched between carrier sheets. When the generation unit 156 determines that the medium is an obliquely folded medium and is being transported sandwiched between carrier sheets, it joins the front and back sides of the medium at the fold edge detected by the fold edge detection unit 155, regardless of the position of the predetermined mark. On the other hand, when the generation unit 156 determines that the medium is not an obliquely folded medium and is being transported sandwiched between carrier sheets, it joins the front and back sides of the medium at the edge on the side where the predetermined mark is located.

[0103] The determination unit 154 may also determine whether or not overlapping occurs among the transported media based on the transmission information indicated in the ultrasonic signal output from the ultrasonic sensor 115, and determine that the media are diagonally folded only if overlapping occurs among the media. At locations where overlapping occurs among the media, the ultrasonic waves emitted by the ultrasonic sensor 115 are attenuated by the air gap between the multiple media, resulting in a decrease in the transmission information. The acquisition unit 152 periodically receives ultrasonic signals from the ultrasonic sensor 115 and acquires the transmission information indicated in the ultrasonic signal. The determination unit 154 determines that overlapping occurs among the media at locations where the calculated value based on the transmission information is less than the overlap threshold. The determination unit 154 calculates a statistical value (average, median, maximum, or minimum) of the transmission information detected within a predetermined period before and after each piece of transmission information is detected as a calculated value. The determination unit 154 may also use each piece of transmission information itself as a calculated value. The overlap threshold is set to a value between the transparency information detected when one PPC (Plain Paper Copier) sheet is conveyed and the transparency information detected when two PPC sheets are conveyed, for example.

[0104] Furthermore, the imaging device 118 may capture an image of only one side of the medium each time the medium is transported, rather than capturing images of both sides of the medium simultaneously as the medium is transported. In this case, in step S105, the acquisition unit 152 acquires the front side image and the back side image twice, and the image reading device 100 executes the processes from step S106 onwards when both the front side image and the back side image have been acquired.

[0105] As described above in detail, the image reading device 100 detects the fold edge from an image of a diagonally folded medium based on the characteristics of multiple straight lines in the image, and combines the front and back images at the detected fold edge. This makes it possible for the image reading device 100 to appropriately combine images of each side of a medium folded in half.

[0106] In conventional image reading devices, in order to generate an image in which each side of a folded medium is joined, it is necessary to restrict the side of the medium on which the fold is located or to use a carrier sheet with a predetermined mark printed on the side on which the fold is located. That is, the user must transport the folded medium so that the fold is located on a predetermined side, or transport the folded medium sandwiched between a carrier sheet so that the fold is located on the mark side. With image reading device 100, the user can obtain a joined image in which each side of the folded medium is properly joined by simply folding the medium diagonally, without having to consider the orientation of the folded medium or using a carrier sheet. Therefore, image reading device 100 reduces the user's effort and improves user convenience.

[0107] Furthermore, the image reading device 100 can prevent the faces of the medium from being joined at an edge different from the folding edge due to incorrect placement of the medium by the user, etc. Therefore, the image reading device 100 can reduce the effort required for the user to re-read the medium, thereby improving user convenience.

[0108] FIG. 8 is a flowchart showing an example of an image processing operation according to another embodiment.

[0109] The flowchart shown in Fig. 8 is executed in place of the flowchart shown in Fig. 5. The processes of steps S201 to S203, S207 to S209, S211, and S213 to S216 in Fig. 8 are the same as the processes of steps S101 to S103, S104 to S106, S108, and S110 to S113 in Fig. 5, and therefore descriptions thereof will be omitted. Only the processes of steps S204 to S206, S210, and S212 will be described below.

[0110] In step S204, the acquisition unit 152 receives an ultrasonic signal from the ultrasonic sensor 115. The acquisition unit 152 detects the transmission information indicated in the received ultrasonic signal as transmission information of ultrasonic waves passing through the medium at multiple positions of the medium transported by the transport unit, and stores the information in the first storage device 140 in association with the current time (step S204).

[0111] Next, acquisition unit 152 determines whether the trailing edge of the medium has passed the position of ultrasonic sensor 115 (step S205). For example, acquisition unit 152 determines that the trailing edge of the medium has passed the position of ultrasonic sensor 115 when a second predetermined time has elapsed since the trailing edge of the medium passed the position of second medium sensor 114. The second predetermined time is set to the time required for the medium to move from the position of second medium sensor 114 to the position of ultrasonic sensor 115. Note that acquisition unit 152 may also determine that the trailing edge of the medium has passed the position of ultrasonic sensor 115 when a predetermined time has elapsed since feeding of the medium began. If the trailing edge of the medium has not yet passed the position of ultrasonic sensor 115, acquisition unit 152 returns to step S204 and repeats the processes of steps S204 to S205.

[0112] On the other hand, when the trailing edge of the medium passes the ultrasonic sensor 115, the acquisition unit 152 acquires area information indicating overlapping areas where overlapping of media occurs and non-overlapping areas where overlapping of media does not occur within the transported medium (step S206). The acquisition unit 152 detects areas within the transported medium where the transmission information falls within a first range as overlapping areas where overlapping of media occurs, and detects areas within a second range as non-overlapping areas where overlapping of media does not occur. The acquisition unit 152 determines that overlapping of media occurs at a position where the calculated value based on the transmission information is less than the overlap threshold. The acquisition unit 152 calculates a statistical value (average, median, maximum, or minimum value) of the transmission information detected within a predetermined period before and after the detection of each piece of transmission information as a calculated value. Note that the acquisition unit 152 may use each piece of transmission information itself as a calculated value. The acquisition unit 152 refers to the transmission information stored in the first storage device 140, and detects, as an overlapping area, an area within the transported medium where the calculated value is less than the overlapping threshold.

[0113] Furthermore, the acquisition unit 152 determines that a medium is present and no overlapping of media occurs at a position where the calculated value based on the transparency information is equal to or greater than the overlap threshold and less than the medium threshold. The medium threshold is set to a value between the transparency information detected when no medium is being conveyed and the transparency information detected when a single sheet of PPC paper is being conveyed. The acquisition unit 152 references the transparency information stored in the first storage device 140 and detects, within the conveyed medium, an area where the calculated value is equal to or greater than the overlap threshold and less than the medium threshold as a non-overlapping area.

[0114] Note that the transmission information may indicate the magnitude of the phase shift of the ultrasonic waves received by the ultrasonic receiver 115b relative to the phase of the ultrasonic waves transmitted by the ultrasonic transmitter 115a, rather than the magnitude of the ultrasonic waves received by the ultrasonic receiver 115b. When the media overlap, the phase shift of the ultrasonic waves passing through the media becomes larger than when the media do not overlap. Therefore, the determination unit 154 determines that an overlap of the media occurs at a position where the calculated value based on the transmission information is greater than the overlap threshold value.

[0115] Figures 9(A), (B), and 10 are schematic diagrams showing examples of region information. Figures 9(A), (B), and 10 show region information R1, R2, and R3 generated for media M1, M2, and M3 included in surface images P1, P2, and P3 shown in Figures 6(A), (B), and 7, respectively.

[0116] As shown in FIGS. 9A, 9B, and 10, for example, the region information is generated as a region image including a plurality of pixels arranged two-dimensionally, similar to the front or back image. Each pixel included in the region information (region image) indicates whether the position corresponding to each pixel is included in an overlapping region, a non-overlapping region, or a non-medium region. The acquisition unit 152 sets each pixel in the region image based on the arrangement positions of the imaging device 118 and the ultrasonic sensor 115 so that each pixel in the region image indicates information about the portion captured by the corresponding pixel in the front or back image. Based on the transmission information stored in the first storage device 140, the acquisition unit 152 identifies whether each pixel corresponds to an overlapping region, a non-overlapping region, or a non-medium region, and sets the gradation value of each pixel to a value indicating the identified region.

[0117] 9(A), in the region information R1, the region W where the upper portion N1 and the lower portion N2 overlap within the medium M1 is detected as an overlapping region, and the four regions S1 to S4 are detected as non-overlapping regions. Regions S1 and S3 are regions where the upper portion N1 extends above and to the right of the lower portion N2, respectively, and regions S2 and S4 are regions where the lower portion N2 extends below and to the right of the upper portion N1, respectively.

[0118] 9(B), in region information R2, region W where upper portion N1 and lower portion N2 overlap within medium M2 is detected as an overlapping region, and three regions, S1, S2, and S4, are detected as non-overlapping regions. Region S1 is the region where upper portion N1 extends above lower portion N2, and regions S2 and S4 are the regions where lower portion N2 extends to the right and below upper portion N1, respectively. Note that if the medium is folded in half so that the bottom left corner is positioned inside the right edge of the medium, regions S2 and S4 are combined, and a total of two regions are detected as non-overlapping regions.

[0119] 10, in the region information R3, the region W where the upper portion N1 and the lower portion N2 overlap within the medium M3 is detected as an overlapping region, and the two regions S1 and S4 are detected as non-overlapping regions. The region S1 is the region where the upper portion N1 extends upward beyond the lower portion N2, and the region S4 is the region where the lower portion N2 extends downward (to the right) beyond the upper portion N1.

[0120] In step S210, the determination unit 154 determines whether the medium is a diagonally folded medium based on the region information (step S210). As shown in Figures 9(A), (B), and 10, overlapping regions are detected from the region information acquired when the diagonally folded medium is transported, and the number of detected non-overlapping regions is two, three, or four. Also, due to the influence of noise contained in the image, etc., five or more non-overlapping regions may be detected.

[0121] The determination unit 154 groups first connected regions that are adjacent to each other in the horizontal or vertical direction within the region information (region image) and are surrounded by pixels corresponding to non-overlapping regions, by labeling or the like. The determination unit 154 detects the grouped first connected regions as non-overlapping regions and calculates the number of detected non-overlapping regions. The determination unit 154 determines that the medium is a diagonally folded medium if there are overlapping regions within the medium and the number of non-overlapping regions is two or more. On the other hand, the determination unit 154 determines that the medium is not a diagonally folded medium if there are no overlapping regions within the medium or if the number of non-overlapping regions is less than two. This allows the determination unit 154 to accurately determine whether the medium is a diagonally folded medium.

[0122] The determination unit 154 may determine that the medium is a diagonally folded medium if there are overlapping areas within the medium and the number of non-overlapping areas is two or more and four or less. In this case, the determination unit 154 determines that the medium is not a diagonally folded medium if there are no overlapping areas within the medium, if the number of non-overlapping areas is less than two, or if the number of non-overlapping areas is more than four.

[0123] Furthermore, similar to the processing of step S107, the determination unit 154 may further determine whether the medium is an obliquely folded medium based on the front image or the back image. For example, if the determination unit 154 determines that the medium is an obliquely folded medium based on the region information and also determines that the medium is an obliquely folded medium based on the front image or the back image, the determination unit 154 determines that the medium is an obliquely folded medium. On the other hand, if the determination unit 154 determines that the medium is not an obliquely folded medium based on the region information or if the determination unit 154 determines that the medium is not an obliquely folded medium based on the front image or the back image, the determination unit 154 determines that the medium is not an obliquely folded medium. Alternatively, the determination unit 154 may determine that the medium is an obliquely folded medium based on the region information or if the determination unit 154 determines that the medium is an obliquely folded medium based on the front image or the back image. In this case, if the determination unit 154 determines that the medium is not an obliquely folded medium based on the region information and also determines that the medium is not an obliquely folded medium based on the front image or the back image, the determination unit 154 determines that the medium is not an obliquely folded medium.

[0124] Alternatively, the determination unit 154 may calculate a first evaluation value that is higher the more likely the medium is to be a diagonally folded medium, and determine whether the medium is a diagonally folded medium based on the calculated first evaluation value. In this case, the determination unit 154 calculates the first evaluation value so that it is higher the greater the number of straight lines corresponding to the edges of the medium. The determination unit 154 also calculates the first evaluation value so that it is higher when an overlapping area exists within the medium. The determination unit 154 also calculates the first evaluation value so that it is higher the greater the number of non-overlapping areas. If the first evaluation value is equal to or greater than a predetermined threshold, the determination unit 154 determines that the medium is a diagonally folded medium. If the first evaluation value is less than the threshold, the determination unit 154 determines that the medium is not a diagonally folded medium. This allows the determination unit 154 to more accurately determine whether the medium is a diagonally folded medium.

[0125] If the determination unit 154 determines that the medium is a diagonally folded medium, the fold edge detection unit 155 detects a fold edge from among the multiple straight lines detected by the line detection unit 153 based on the characteristics of the multiple straight lines detected by the line detection unit 153 (step S212).

[0126] For example, the fold edge detection unit 155 detects the relationship between the multiple straight lines detected by the straight line detection unit 153 and the overlapping area as a feature of each straight line, and detects the straight line corresponding to the longest straight line among the multiple detected straight lines surrounding the overlapping area as the fold edge.

[0127] As described above, when a medium having a standard size such as A or B is folded in half near the center of the longitudinal edge so that the folded edge is slightly inclined relative to the lateral edge, the length of the folded edge becomes greater than the length of each longitudinal edge of the folded medium.

[0128] As shown in FIG. 9A, of the lines E1 to E5 surrounding the overlapping region W, the lengths of the lines E1 and E4 on the folded edges extending in the longitudinal direction are approximately the same as the lengths of the folded edges themselves. Furthermore, of the lines E1 to E5 surrounding the overlapping region W, the lengths of the lines E2 and E3 on the folded edges extending in the lateral direction are shorter than the lengths of the folded edges themselves. Therefore, of the lines E1 to E5 surrounding the overlapping region W, the length of the line E5 on the folded edges is greater than the lengths of the other lines E1 to E4. Similarly, in FIG. 9B, of the lines E1, E2, E4, and E5 surrounding the overlapping region W, the length of the line E5 on the folded edges is greater than the lengths of the other lines E1, E2, and E4. Also, in FIG. 10, of the lines E1, E4, and E5 surrounding the overlapping region W, the length of the line E5 on the folded edges is greater than the lengths of the other lines E1 and E4.

[0129] The fold edge detection unit 155 groups second connected regions that are adjacent to each other in the horizontal or vertical direction in the region information (region image) and that are surrounded by pixels corresponding to overlapping regions, by labeling or the like. The fold edge detection unit 155 detects the grouped second connected regions as overlapping regions. The fold edge detection unit 155 detects straight lines that surround the overlapping region, i.e., straight lines that pass through the ends (outer edges) of the overlapping region, and regards the longest straight line among the detected straight lines as the fold edge. The fold edge detection unit 155 detects the positional relationship (distance) between the multiple straight lines detected by the line detection unit 153 and the straight lines that are regarded as fold edges in the region image as the relationship between the multiple straight lines detected by the line detection unit 153 and the overlapping region. The fold edge detection unit 155 calculates the distance between the end or center of each straight line detected by the line detection unit 153 and the position corresponding to the end or center of the straight line that is regarded as the fold edge in the region image, in the front or back image. The folding side detection unit 155 detects, as a folding side, the straight line with the shortest calculated distance from among the straight lines detected by the straight line detection unit 153. This allows the folding side detection unit 155 to detect the folding side with high accuracy.

[0130] In addition, the fold edge detection unit 155 may detect the relationship between the multiple straight lines detected by the straight line detection unit 153 and the non-overlapping area as a feature of each straight line, and detect, among the multiple detected straight lines, a straight line that does not correspond to a line surrounding the non-overlapping area as a fold edge.

[0131] 9(A), 9(B), and 10, when the medium is folded diagonally, one of the upper portion N1 and the lower portion N2 protrudes from the other around the straight lines surrounding the overlapping area W, other than the straight line E5 corresponding to the folding edge, creating a non-overlapping area. In other words, the straight lines surrounding the overlapping area W, other than the straight line E5 corresponding to the folding edge, are in contact with the non-overlapping area and overlap with the straight lines surrounding the non-overlapping area.

[0132] The fold edge detection unit 155 detects non-overlapping areas within the area information (area image) in the same manner as in step S210. The fold edge detection unit 155 detects the positional relationship (whether or not the lines are adjacent) between the multiple lines detected by the line detection unit 153 and the non-overlapping areas detected in the area image as the relationship between the multiple lines detected by the line detection unit 153 and the non-overlapping areas. The fold edge detection unit 155 determines whether each line detected by the line detection unit 153 is located in a position adjacent to any non-overlapping area detected in the area image, within the front or back image. The fold edge detection unit 155 determines whether all pixels on each line detected by the line detection unit 153 are located within a predetermined distance from an area corresponding to any non-overlapping area in the area image. The fold edge detection unit 155 detects, as a fold edge, any line detected by the line detection unit 153, if any pixel on the line is not located within a predetermined distance from an area corresponding to any non-overlapping area in the area image. This allows the fold side detection unit 155 to accurately determine whether the medium is an obliquely folded medium.

[0133] The fold edge detection unit 155 may detect two or more of the following as characteristics of the multiple lines: the length and orientation of each line detected in step S109 of FIG. 5; the positional relationship between each line and the detection area detected in step S212 of FIG. 8; and the positional relationship between each line and the non-overlapping area. For example, the fold edge detection unit 155 calculates a second evaluation value for each of the multiple lines, the higher the likelihood that each line is a fold edge, and detects the fold edge based on the calculated second evaluation value. In this case, the fold edge detection unit 155 calculates the second evaluation value so that it increases as each line is longer. The fold edge detection unit 155 also calculates the second evaluation value so that it increases as the angle between each line and another line increases. The fold edge detection unit 155 also calculates the second evaluation value so that it increases as the distance between each line and the position corresponding to the line in the area image deemed to be the fold edge decreases. The fold edge detection unit 155 also calculates the second evaluation value so that it increases as the distance between each line and the non-overlapping area increases. Then, the folding side detection unit 155 detects the straight line with the highest second evaluation value as the folding side, thereby enabling the folding side detection unit 155 to detect the folding side with higher accuracy.

[0134] As described above in detail, the image reading device 100 determines whether the medium being transported is a diagonally folded medium based on the area information, and even when detecting the fold edge, it is possible to appropriately combine images captured of each side of the medium folded in half.

[0135] FIG. 11 is a diagram for explaining a transport path inside an image reading device 300 according to another embodiment.

[0136] 11, the image reading device 300 has the same components as the image reading device 100. However, the image reading device 300 has a thickness sensor 315 instead of the ultrasonic sensor 115.

[0137] The thickness sensor 315 is located downstream of the feed roller 112 and the separation roller 113 and upstream of the first and second conveyance rollers 116 and 117. The thickness sensor 315 includes a light emitter 315a and a light receiver 315b. The light emitter 315a and the light receiver 315b are located near the medium conveyance path, facing each other across the conveyance path. The light emitter 315a emits light (infrared light or visible light) toward the light receiver 315b. The light receiver 315b receives the light emitted by the light emitter 315a and generates and outputs a thickness signal, which is an electrical signal corresponding to the intensity of the received light. When a medium is present at the thickness sensor 315, the light emitted by the light emitter 315a is attenuated by the medium, and the greater the thickness of the medium, the greater the attenuation. For example, the thickness sensor 315 generates a thickness signal such that the signal value increases as the medium becomes thicker. The thickness signal indicates thickness information of the medium at multiple positions within the medium being transported by the transport unit. The number of thickness sensors 315 is not limited to one and may be multiple. In this case, the multiple thickness sensors 315 are arranged side by side at intervals in the width direction A2.

[0138] The thickness sensor 315 may be a reflective light sensor, a pressure sensor, or a mechanical sensor. The reflective light sensor includes a pair of a light emitter and a light receiver provided on one side of the media transport path and a pair of a light emitter and a light receiver provided on the other side. The reflective light sensor detects the distance between each pair and each side of the media based on the time between one pair irradiating one side of the media with light and receiving the reflected light, and the time between the other pair irradiating the other side of the media with light and receiving the reflected light. The reflective light sensor subtracts each detected distance from the distance between the two pairs to generate a thickness signal indicating the subtracted value as thickness information. The pressure sensor detects pressure, which changes depending on the thickness of the media, and generates a thickness signal indicating the detected pressure as thickness information. The mechanical sensor detects the amount of movement of a roller in contact with the media and generates a thickness signal indicating the detected amount of movement as thickness information.

[0139] The image reading device 300, like the image reading device 100, executes the image processing shown in FIG.

[0140] However, in step S204, the acquisition unit 152 receives a thickness signal from the thickness sensor 315. The acquisition unit 152 detects the thickness information indicated in the received thickness signal as thickness information of the medium at multiple positions of the medium transported by the transport unit, and stores it in the first storage device 140 in association with the current time.

[0141] Furthermore, in step S205, the acquisition unit 152 determines whether the trailing edge of the medium has passed the position of the thickness sensor 315. If the trailing edge of the medium has not yet passed the position of the thickness sensor 315, the acquisition unit 152 returns the process to step S204 and repeats the processes of steps S204 and S205.

[0142] In step S206, the acquisition unit 152 detects an area within the transported medium where the detected thickness information falls within a first range as an overlapping area where media overlap occurs. The acquisition unit 152 detects an area within the transported medium where the detected thickness information falls within a second range as a non-overlapping area where media overlap does not occur. The acquisition unit 152 considers an overlapping area of ​​media to occur at a position where the thickness information is greater than the overlap threshold. The acquisition unit 152 calculates a statistical value (average, median, maximum, or minimum) of the thickness information detected within a predetermined period before and after each piece of thickness information is detected as a calculated value. The acquisition unit 152 may also use each piece of thickness information itself as a calculated value. The overlap threshold is set to a value between the thickness information detected when one piece of PPC paper is transported and the thickness information detected when two pieces of PPC paper are transported, for example. The acquisition unit 152 refers to the thickness information stored in the first storage device 140, and detects, within the transported medium, an area where the calculated value is greater than the overlap threshold as an overlap area.

[0143] The acquisition unit 152 also determines that a medium is present at a position where the calculated value based on the thickness information is equal to or less than the overlap threshold and greater than the medium threshold, and that no overlapping of the medium has occurred. The medium threshold is set to a value between the thickness information detected when no medium is being conveyed and the thickness information detected when a single sheet of PPC paper is being conveyed. The acquisition unit 152 references the thickness information stored in the first storage device 140 and detects, within the conveyed medium, an area where the calculated value is equal to or less than the overlap threshold and greater than the medium threshold as a non-overlapping area.

[0144] As described above in detail, the image reading device 300 determines whether the medium being transported is a diagonally folded medium based on thickness information, and even when detecting the folded edge, it is possible to appropriately combine images captured of each side of the medium folded in half.

[0145] 12 is a diagram showing a schematic configuration of a second storage device 410 and a second processing circuit 420 in an information processing device according to yet another embodiment. In this embodiment, the information processing device is an example of an image processing device.

[0146] 12, the second storage device 410 stores a control program 411, an acquisition program 412, a line detection program 413, a determination program 414, a fold edge detection program 415, a generation program 416, and an output control program 417. Each of these programs is a functional module implemented by software running on a processor. The second processing circuit 420 reads each program stored in the second storage device 410 and operates in accordance with the read program. As a result, the second processing circuit 420 functions as a control unit 421, an acquisition unit 422, a line detection unit 423, a determination unit 424, a fold edge detection unit 425, a generation unit 426, and an output control unit 427.

[0147] The acquisition unit 422, the straight line detection unit 423, the determination unit 424, the fold side detection unit 425, and the generation unit 426 have the same functions as the acquisition unit 152, the straight line detection unit 153, the determination unit 154, the fold side detection unit 155, and the generation unit 156 of the image reading device, respectively. The control unit 421 performs image correction and the like instead of controlling the drive of a motor or the imaging of an imaging device. The output control unit 427 displays the generated image on the second display device 202 instead of transmitting it to an external device.

[0148] The image reading device according to this embodiment executes the image processing shown in Fig. 5. However, the processes of steps S106 to S107 and S109 to S111 are omitted, and the output control unit 157 transmits the front and back images to the information processing device 200 via the first interface device 132 in step S108. Alternatively, the image reading device executes the image processing shown in Fig. 8. However, the processes of steps S209 to S210 and S212 to S214 are omitted, and the output control unit 157 transmits the front and back images and area information to the information processing device 200 via the first interface device 132 in step S211.

[0149] FIG. 13 is a flowchart showing an example of the image processing operation of the information processing device.

[0150] An example of the image processing operation of the information processing device will be described below with reference to the flowchart shown in Fig. 13. The flow of the operation described below is executed mainly by the second processing circuit 420 in cooperation with each element of the information processing device based on a program stored in advance in the second storage device 410. The flow of the operation shown in Fig. 13 is executed periodically.

[0151] First, the acquisition unit 422 acquires the front side image, the back side image, and / or the area information by receiving them from the image reading device via the second interface device 203 (step S301).

[0152] Next, the line detection unit 423 detects a plurality of lines from the front image or the back image in the same manner as in step S106 or S209 (step S302).

[0153] Next, the determination unit 424 determines whether the medium is a diagonally folded medium based on the detected multiple straight lines and / or area information, in the same manner as in the process of step S107 or S210 (step S303).

[0154] If it is determined that the medium is not a diagonally folded medium, the output control unit 427 outputs the front and back images by displaying them on the second display device 202 (step S304), and the series of steps ends.

[0155] On the other hand, if it is determined that the medium is a diagonally folded medium, the fold edge detection unit 425 detects the fold edge from among the detected multiple straight lines based on the characteristics of the detected multiple straight lines, in a manner similar to the processing of step S109 or S212 (step S305).

[0156] Next, the generating unit 426 generates a combined image by combining the front side included in the front side image and the back side included in the back side image along the detected folding edge, similar to the process in step S110 or S213 (step S306).

[0157] Next, the output control unit 427 outputs the generated combined image by displaying it on the second display device 202 (step S307), and the series of steps ends.

[0158] As described above in detail, even when the information processing device performs image processing, the information processing device can appropriately combine images captured on each side of a medium folded in half.

[0159] 14 is a diagram showing a schematic configuration of a first processing circuit 550 in an image reading device according to yet another embodiment. The first processing circuit 550 is used in place of the first processing circuit 150 of the image reading device 100 or 300, and performs image processing and the like in place of the first processing circuit 150. The first processing circuit 550 includes a control circuit 551, an acquisition circuit 552, a line detection circuit 553, a determination circuit 554, a fold side detection circuit 555, a generation circuit 556, and an output control circuit 557. Each of these components may be configured by an independent integrated circuit, microprocessor, firmware, or the like.

[0160] The control circuit 551 is an example of a control unit, and has the same functions as the control unit 151. The control circuit 551 receives an operation signal from the first operation device 105 or the first interface device 132, a first medium signal from the first medium sensor 111, and a second medium signal from the second medium sensor 114, and controls the motor 131 based on the received signals.

[0161] The acquisition circuit 552 is an example of an acquisition unit, and has the same function as the acquisition unit 152. The acquisition circuit 552 receives the front image and the back image from the imaging device 118 and stores them in the first storage device 140, and also receives the transmission information or the thickness information from the ultrasonic sensor 115 or the thickness sensor 315 to generate the region information and store it in the first storage device 140.

[0162] The line detection circuit 553 is an example of a line detection unit, and has the same function as the line detection unit 153. The line detection circuit 553 reads out the front image or the back image from the first storage device 140, detects multiple lines from the read front image or the back image, and stores the detection results in the first storage device 140.

[0163] The determination circuit 554 is an example of a determination unit, and has the same function as the determination unit 154. The determination circuit 554 reads the front image, the back image, the straight line detection results, and / or the area information from the first storage device 140, determines whether the medium is a diagonally folded medium based on the read information, and stores the determination result in the first storage device 140.

[0164] The fold edge detection circuit 555 is an example of a fold edge detection unit, and has the same function as the fold edge detection unit 155. The fold edge detection circuit 555 reads the front image, the back image, the straight line detection result, the diagonally folded medium determination result, and / or area information from the first storage device 140, detects the fold edge based on the read information, and stores the detection result in the first storage device 140.

[0165] The generation circuit 556 is an example of a generation unit, and has the same function as the generation unit 156. The generation circuit 556 reads the front image or back image and the folding edge detection results from the first storage device 140, generates a combined image based on the read information, and stores the combined image in the first storage device 140.

[0166] The output control circuit 557 is an example of an output control unit, and has the same function as the output control unit 157. The output control circuit 557 reads the front image, the back image, the combined image, and / or the area information from the first storage device 140, and outputs them to the first interface device 132.

[0167] As described above in detail, even when the first processing circuit 550 is used, the image reading device is able to appropriately combine images captured on each side of a medium folded in half.

[0168] 15 is a diagram showing a schematic configuration of a second processing circuit 620 in an information processing device according to another embodiment. The second processing circuit 620 is used in place of the second processing circuit 420 of the information processing device, and performs image processing and the like in place of the second processing circuit 420. The second processing circuit 620 includes a control circuit 621, an acquisition circuit 622, a line detection circuit 623, a determination circuit 624, a fold side detection circuit 625, a generation circuit 626, and an output control circuit 627. Each of these components may be configured by an independent integrated circuit, microprocessor, firmware, or the like.

[0169] The control circuit 621 is an example of a control unit, and has the same functions as the control unit 421. The control circuit 621 reads out the front image, the back image, or the combined image from the second storage device 210, corrects the read image, and stores it in the second storage device 210.

[0170] The acquisition circuit 622 is an example of an acquisition unit, and has the same function as the acquisition unit 422. The acquisition circuit 622 receives the front image, the back image, and / or the area information from the second interface device 203 and stores them in the second storage device 210.

[0171] The line detection circuit 623 is an example of a line detection unit, and has the same function as the line detection unit 423. The line detection circuit 623 reads out the front image or the back image from the second storage device 210, detects multiple lines from the read front image or the back image, and stores the detection results in the second storage device 210.

[0172] The determination circuit 624 is an example of a determination unit, and has the same function as the determination unit 424. The determination circuit 624 reads the front image, the back image, the straight line detection results, and / or the area information from the second storage device 210, determines whether the medium is a diagonally folded medium based on the read information, and stores the determination result in the second storage device 210.

[0173] The fold edge detection circuit 625 is an example of a fold edge detection unit, and has the same function as the fold edge detection unit 425. The fold edge detection circuit 625 reads the front image, the back image, the straight line detection results, the diagonally folded medium determination results and / or area information from the second storage device 210, detects the fold edge based on each piece of read information, and stores the detection results in the second storage device 210.

[0174] The generation circuit 626 is an example of a generation unit, and has the same functions as the generation unit 426. The generation circuit 626 reads the front image or back image and the folding edge detection results from the second storage device 210, generates a combined image based on the read information, and stores the combined image in the second storage device 210.

[0175] The output control circuit 627 is an example of an output control unit, and has the same function as the output control unit 427. The output control circuit 627 reads out the front image, the back image, or the combined image from the second storage device 210, and outputs it to the second display device 202.

[0176] As described above in detail, even when the information processing device uses the second processing circuit 620, the information processing device can appropriately combine images captured on each side of a medium folded in half. [Explanation of symbols]

[0177] 100 Image reading device, 200 Information processing device, 152, 422 Acquisition unit, 153, 423 Line detection unit, 154, 424 Determination unit, 155, 425 Fold edge detection unit, 156, 426 Generation unit, 157, 427 Output control unit

Claims

1. An acquisition unit that acquires a first image of a first side of a diagonally folded medium, a second image of a second side of the diagonally folded medium, and information indicating an overlapping area or a non-overlapping area of ​​the diagonally folded medium; a line detection unit that detects a plurality of lines corresponding to an edge of the diagonally folded medium from the first image or the second image; a folding edge detection unit that detects a folding edge based on a relationship between the detected line and the overlapping region or the non-overlapping region; a generating unit that generates a combined image by combining a first face image included in the first image and a second face image included in the second image such that edges corresponding to the detected folding edges are connected to each other; 1. An image processing device comprising:

2. The image processing device according to claim 1 , wherein the line detection unit detects straight lines from the first image or the second image.

3. the acquiring unit acquires information indicating an overlapping area of ​​the diagonally folded medium, 3. The image processing device according to claim 2, wherein the fold edge detection unit detects a relationship between the detected straight lines and the overlapping area, and detects, as the fold edge, a straight line corresponding to the longest straight line among the detected straight lines that surround the overlapping area.

4. The acquisition unit further acquires information indicating a non-overlapping area of ​​the diagonally folded medium, 3. The image processing device according to claim 2, wherein the fold edge detection unit detects a relationship between the detected straight lines and the non-overlapping region, and detects, as the fold edge, a straight line among the detected straight lines that does not correspond to a line surrounding the non-overlapping region.

5. The image processing device according to claim 2, wherein the fold edge detection unit detects the fold edge when the number of the detected straight lines is five or more.

6. An image processing device described in any one of claims 1 to 5, wherein the fold edge detection unit detects the fold edge when an overlapping area exists in a diagonally folded medium and the number of non-overlapping areas is two or more.

7. A control method for an image processing device, comprising: Obtaining a first image of a first side of the diagonally folded medium, a second image of a second side of the diagonally folded medium, and information indicating an overlapping area or a non-overlapping area of ​​the diagonally folded medium; detecting a plurality of lines corresponding to edges of the diagonally folded medium from the first image or the second image; Detecting a folding edge based on a relationship between the detected line and the overlapping region or the non-overlapping region; generating a combined image by combining the first side image included in the first image and the second side image included in the second image such that the edges corresponding to the detected folding edges are connected to each other; A control method comprising:

8. A control program for an image processing device, Obtaining a first image of a first side of the diagonally folded medium, a second image of a second side of the diagonally folded medium, and information indicating an overlapping area or a non-overlapping area of ​​the diagonally folded medium; detecting a plurality of lines corresponding to edges of the diagonally folded medium from the first image or the second image; Detecting a folding edge based on a relationship between the detected line and the overlapping region or the non-overlapping region; generating a combined image by combining the first side image included in the first image and the second side image included in the second image such that the edges corresponding to the detected folding edges are connected to each other; a control program for causing the image processing device to execute the above steps;

Images