Image processing system and image processing method

Abstract

To provide an image processing system and an image processing method for more accurately restore an original image from an image deteriorated through a printing process and a reading process.SOLUTION: An image processing system comprises: reading means that optically reads a printed material in which drawing information on an original image is multiplexed to acquire read image data; extraction means that extracts the drawing information from the read image data acquired by the reading means; acquisition means that, based on the drawing information extracted by the extraction means, acquires original image data representing the original image; correction means that compares the read image data and the original image data with each other and corrects the original image data based on a result of the comparison; and storage means that stores corrected original image data corrected by the correction means as a result of reading the printed material.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image processing system and an image processing method for processing an image. [Background technology]

[0002] A scanning function is known that reads an image printed by a printer based on an original image with a scanner and saves the image as read data. In this case, the image quality of the image represented by the read data generated through the printing and reading processes may be degraded compared to the original original image.

[0003] Patent document 1 describes a method of estimating the output image color by taking into account the color information in the original image and the output characteristics of the printer, embedding the estimated color in the printed image as multiplexing information, extracting the multiplexing information from the image read by a scanner, and restoring the degraded color. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5180551 Summary of the Invention [Problem to be solved by the invention]

[0005] Further accuracy is required for restoration from images that have been deteriorated through the printing and reading processes.

[0006] An object of the present invention is to provide an image processing system and an image processing method that can more accurately restore an image that has been deteriorated through a printing process and a reading process. [Means for solving the problem]

[0007] In order to solve the above problems, the image processing system according to the present invention includes a reading means for optically reading a printed matter on which drawing information of an original image is multiplexed, and acquiring read image data, an extraction means for extracting the drawing information from the read image data acquired by the reading means, an acquisition means for acquiring original image data representing the original image based on the drawing information extracted by the extraction means, and a combination of the read image data and the original image data. and dividing the image represented by each image data into a plurality of regions and comparing the images, and determining whether or not to use the read image data and the original image data for each of the plurality of regions based on the results of the comparison, and determining whether or not to use the read image data and the original image data for each of the plurality of regions for correcting the original image data based on the results of the determination. The image forming apparatus is characterized by comprising a correction unit that corrects the original image data, and a storage unit that stores the original image data corrected by the correction unit as a result of reading the printed matter. Effect of the Invention

[0008] According to the present invention, an image that has been deteriorated through a printing process and a reading process can be restored with higher accuracy. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a configuration of a printing system. [Diagram 2] 13 is a flowchart showing a PDL multiplexing print process and a multiplexed PDL extraction process. [Diagram 3] FIG. 2 is a diagram showing an original image represented by PDL data. [Figure 4] FIG. 1 is a diagram for explaining image degradation due to printing and reading. [Diagram 5] 13 is a flowchart showing a process of generating corrected PDL data. [Figure 6] FIG. 13 is a diagram showing mask data. [Figure 7] FIG. 2 is a diagram showing an image pattern represented by mask data. [Figure 8] FIG. 10 is a diagram for explaining an example of a multiplexing method. [Figure 9] 11A and 11B are diagrams for explaining processing performed on an image to which handwritten characters have been added; [Figure 10] FIG. 2 is a diagram showing an original image represented by PDL data. [Figure 11]FIG. 13 is a diagram showing an original image represented by corrected PDL data. [Figure 12] 13 is a flowchart showing a process of generating corrected PDL data. [Figure 13] 11A and 11B are diagrams for explaining processing for an image with additional handwritten characters; [Figure 14] FIG. 13 is a diagram showing the spatial frequency characteristics of a pattern used for embedding. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] [First embodiment] Fig. 1 is a block diagram showing the configuration of a printing system (image processing system) in this embodiment. As shown in Fig. 1, the printing system includes an MFP (Multi Function Printer) 110 and a personal computer (PC) 100 as its host device. The MFP 110 is a printer that has multiple functions integrated into its main body, such as a printer function and a scanner function, and has a copy function that performs both functions in cooperation with each other.

[0012] The PC 100 includes the following components. The CPU 101 reads and executes programs stored in the HDD 103 and the RAM 102. The RAM 102 is a volatile storage and temporarily stores programs and data. The HDD 103 is a non-volatile storage and stores programs and data used in this embodiment. The data transfer interface (I / F) 104 controls data transmission and reception with the MFP 110. Examples of connection methods for data transmission and reception include wired connections such as USB, IEEE1394, and LAN, wireless connections such as Bluetooth (registered trademark) and WiFi, and connections including both. The HID interface (I / F) 105 is an interface that controls a HID (Human Interface Device) such as a keyboard and a mouse, and accepts inputs by user operations. The display interface (I / F) 106 controls the display of a display (not shown).

[0013] The MFP 110 is configured to include the following elements: A CPU 111 reads and executes programs stored in a ROM 113 and a RAM 112. The RAM 112 is a volatile storage and temporarily stores programs and data. The ROM 113 is a non-volatile storage and stores programs and data used in this embodiment.

[0014] A data transfer interface (I / F) 114 controls the transmission and reception of data to and from the PC 100. A printhead controller 115 controls the heating operation of a heater mounted on the printhead based on print data converted so that it can be processed by a printhead (not shown), and causes ink droplets to be ejected from the nozzles. For example, the printhead controller 115 is configured to read control parameters and print data from a predetermined address in the RAM 112. When the CPU 111 writes the control parameters and print data to a predetermined address in the RAM 112, the printhead controller 115 is started, and the heating operation of the heater mounted on the printhead is performed.

[0015] The image processing accelerator 116 is configured with hardware, and can execute image processing faster than the CPU 111. For example, the image processing accelerator 116 is configured to read parameters and data required for image processing from a predetermined address in the RAM 112. When the CPU 111 writes the parameters and data to a predetermined address in the RAM 112, the image processing accelerator 116 is started and image processing is performed. Depending on the specifications of the printer, the parameter creation process and image processing may be executed only by the CPU 111, instead of the image processing accelerator 116.

[0016] Scanner controller 117 causes a scanner unit (not shown) to irradiate light onto a document placed on a document table. Scanner controller 117 instructs the scanner unit to transmit light quantity information acquired by an imaging element such as a CCD based on reflected light from the document to scanner controller 117. For example, when CPU 111 writes control parameters and a read data write address to a predetermined address in RAM 112, scanner controller 117 is started. Then, light emission control of an LED mounted on the scanner unit, acquisition of light quantity information from the scanner unit, and writing of the light quantity information to the read data write address and subsequent addresses in RAM 112 are performed.

[0017] Motor controller 118 controls the motor operation of multiple motor units (not shown). The motors controlled by motor controller 118 include a motor for moving the print head relative to the print medium and a motor for moving the scanner unit relative to a platen on which a document is placed. Other motors include a motor for maintaining the print head. PC 100 and MFP 110 are not limited to the configuration shown in FIG. 1, and may include blocks according to the functions that PC 100 and MFP 110 can respectively execute.

[0018] In this embodiment, the PC 100 controls the MFP 110 to print based on the image data to be printed. At that time, drawing information of an image (original image) represented by the image data is multiplexed. As a result, the printed matter printed by the MFP 110 is the result of multiplexing the drawing information of the original image. Note that in this embodiment, PDL will be described as an example of drawing information. Then, the printed matter is optically read by a scanner unit of the MFP 110 and generated as read image data. When generating the read image data, the multiplexed drawing information is extracted. In this embodiment, the original image data is corrected based on a comparison result between the original image data represented by the multiplexed drawing information and the read image data.

[0019] When the printing process and the reading process are performed, degradation of image quality may occur due to each process. On the other hand, when a case is assumed in which handwritten characters or the like are added to a printed matter by a user and then the printed matter is read, it is not possible to generate appropriate read image data by simply restoring the original image based on the multiplexed drawing information. In this embodiment, by correcting the original image data, it is possible to restore the image from the image deteriorated due to each process, and also to appropriately restore the image in the case assumed above.

[0020] Fig. 2 is a flowchart showing the PDL multiplexing print process and the multiplexed PDL extraction process. S201 to S205 in Fig. 2 are a flowchart showing the PDL multiplexing print process, which is a process up to the point where printing is performed by the MFP 110 based on the image data to be printed. S211 to S216 are a flowchart showing the multiplexed PDL extraction process, which is a process up to the point where the printed matter is read by the scanner unit of the MFP 110 and the read image data is saved.

[0021] First, the PDL multiplexing print process will be described. In S201, the CPU 101 of the PC 100 acquires PDL data as image data to be printed. The image data to be printed may be created by an application in the PC 100, or may be acquired from an external device such as a server.

[0022] Here, PDL is an abbreviation of Page Description Language, which is composed of a set of drawing commands for each page. The types of drawing commands are defined for each PDL specification, and for example, the following four types are defined:

[0023] Command 1: DOT drawing command (X, Y, color, dot size) Command 2: LINE drawing command (X1, Y1, X2, Y2, color, line type, line width) Command 3: BOX drawing command (X1, Y1, X2, Y2, color, fill shape) Command 4: TEXT drawing command (X1, Y1, color, font information, string information) Command 5: IMAGE drawing command (X1, Y1, X2, Y2, image file information) Other suitable drawing commands may be used depending on the application, such as a CIRCLE drawing command for drawing a circular arc, etc. Also, as the PDL, for example, PDF (Portable Document Format), XPS, HP-GL / 2, etc. may be used.

[0024] Fig. 3 is a diagram showing an example of an original image represented by PDL data. An original image 300 in Fig. 3 represents one page, and will be described here as an original image having 600 pixels in width and 900 pixels in height.

[0025] An example of a PDL corresponding to the original image 300 in FIG. 3 is shown below.

[0026] <PAGE=001> <box> 50, 75, 200, 300, GRAY, STRIPE< / box> <box> 420, 75, 550, 300, BLACK, FULL< / box> <box> 50,620,200,825,BLACK,NULL< / box> <text> 50,620, 200, 825, BLACK, STD-18, "ABCXYZ123"< / text> <line> 25, 25, 25, 700, BLACK, PLAIN, 3< / line> <line> 25, 25, 450, 25, BLACK, PLAIN, 3< / line> 250, 350, 550, 825, “PORTRAIT.jpg” First line<PAGE=001> is a tag that indicates the number of pages. Normally, PDL is designed to be able to describe multiple pages, so tags that indicate page breaks are described in the PDL. In this example, the first 9th line indicates that it is the first page, which corresponds to the document image 300 in FIG. 3. If there is a second page, the above PDL is followed by<PAGE=002> is described.

[0027] The second line <box> from< / box> The above is command 3, which corresponds to object 301 in Fig. 3. The first two coordinates indicate the upper left coordinates (X1, Y1) which are the drawing start point, and the next two coordinates indicate the lower right coordinates (X2, Y2) which are the drawing end point. GRAY (gray: R=128, G=128, B=128) is specified as the color, and STRIPE, which is a striped pattern, is specified as the fill shape. In this example, the direction of the stripes is always a line going in a lower right direction, but the angle and period of the lines may also be specified in the BOX command.

[0028] The BOX command on line 3 corresponds to object 302 in Fig. 3. Unlike the command on line 2, this command specifies the color as BLACK (black: R=0, G=0, B=0) and the fill shape as a solid FULL. The BOX command on line 4 and the TEXT command on line 5 correspond to object 303 in Fig. 3. First, the BOX command describes a frame around the text. In this example, NULL (blank) is specified as the fill shape inside the BOX.

[0029] The TEXT command on line 5 specifies "STD" (standard) as the character font, 18 points as the character size, and "AMCXYZ123" as the character string to be written. The LINE commands on lines 6 and 7 correspond to object 304 in FIG. 3. First, a vertical line is drawn by the LINE command on line 6. In this example, PLAIN (solid line) is specified as the type of line to be drawn, and 3 pixels is specified as the thickness of the line to be drawn. Similarly, a horizontal line is drawn by the LINE command on line 7.

[0030] The IMAGE command on line 8 corresponds to object 305 in Fig. 3. In this example, "PORTRAIT.jpg" is specified as the file name of the image in the area, indicating that it is a JPEG file, which is an image compression format. The statement on line 9 indicates that drawing of the page has been completed.

[0031] The PDL data may be in the form of a PDL file that includes the above-mentioned drawing commands, as well as "STD" font data and a "PORTRAIT.jpg" image file. If the font data and image files are managed separately, it may be possible that the text and image parts cannot be formed with drawing commands alone, but this can be prevented by using a PDL file format that includes font data and image files. The above-mentioned PDL data is acquired in S201 of FIG. 2.

[0032] In S202, the CPU 101 performs rendering processing of the PDL data acquired in S201. The rendering processing executes each drawing command described in the PDL, and generates bitmap image data composed of color information for each pixel. Hereinafter, the bitmap image data generated in S202 is also referred to as rendered image data.

[0033] 3 is composed of 600 pixels in width and 900 pixels in height, the bitmap image data generated in S202 is composed of 600 x 900 pixels. Each pixel is represented by 256 gradations with 8 bits for each of R / G / B.

[0034] In S203, CPU 101 generates multiplexed image data. Here, the PDL data acquired in S201 is superimposed as multiplexing information on the rendered image data generated in S202. This is to enable the multiplexed PDL data to be extracted after the printed matter is read by the scanner unit.

[0035] Handling information on an information processing device such as the PC100 means handling binary data. Binary data is information that is either "0" or "1", and when these "0" or "1" pieces of information are connected in succession, they have a specific meaning. For example, when handling the information "hello" as binary data, in the case of "Shift JIS", one of the character codes, "h" corresponds to the binary data "01101000". Similarly, "e" corresponds to the binary data "01100101", "l" corresponds to "01101100", and "o" corresponds to the binary data "01101111". In other words, the character "hello" can be expressed in binary data as "0110100001100101011011000110110001101110001101111". Conversely, if the binary data "0110100001100101011011000110110001101111" can be obtained, the character "hello" can be obtained. In this way, multiplexing can be performed by embedding data so that "0" or "1" can be determined. In this embodiment, two mask data as shown in FIG. 6(a) and FIG. 6(b) are used to generate "0" and "1".

[0036] 6(a) and 6(b) show mask data consisting of 8×8 pixels, and adding the value of the mask data to the image data can provide a periodic pattern to the 8×8 pixel region in the image data. For example, in digital image data, one color is expressed by 8 bits, and one of the values ​​from 0 to 255 is assigned. Values ​​outside the range of 0 to 255 cannot be treated as image data, so if the calculation result of the pixel value is less than 0 or is a value of 256 or more, for example, 0 or 255 is assigned to keep it within the valid range. In the mask data of FIG. 6(a) and FIG. 6(b), the pixel value is changed by ±10, but if the image data values ​​in the mask region are all 255, for example, the value of the region will be in the range from 245 to 255, not in the range from 245 to 265. In this embodiment, it is described that one color is expressed by 8 bits, but one color may be expressed by other than 8 bits. When dealing with digital image data, there is a valid range regardless of the number of bits used to express one color, and no change is given that goes outside that range.

[0037] Figure 7 is a visual diagram showing what kind of pattern is given to an image by the mask data of Figures 6(a) and 6(b). In Figure 7, the positions of "10" in the mask data of Figures 6(a) and 6(b) are expressed in black, the positions of "0" in grey, and the positions of "-10" in white, and a diagonal line appears in the image to be multiplexed.

[0038] An example of pseudocode for alternately applying the mask data of FIG. 6(a) and the mask data of FIG. 6(b) to the entire image is shown below. ------------------------------------------------------------------ 01: int i, j, k, l; 02: int width = 640, height=480; 03: unsigned char *data = image data; 04: int **maskA = mask data; 05: bool isMaskA = true; 06: for(j = 0; j < height; j+=8){ 07: for(i = 0; i < width; i+=8){ 08: for(k = 0; k < 8; k++) 09: for(l = 0; l < 8; l++) { 10: if(isMaskA == true) { 11: data[(i+k)+(j+l)*width] += maskA[k][l]; 12:} 13:} 14:} 15:} 16:} ------------------------------------------------------------------ In this embodiment, multiplexing of the multiplexing information is realized, for example, by the pseudo code as described above. Next, in S204, the CPU 101 generates print image data. In generating the print image data, for example, the following method is used.

[0039] In S204, four processes, namely, color conversion, ink color separation, output characteristic conversion, and quantization, are performed on each pixel of the multiplexed image data composed of RGB pixel values ​​generated in S203.

[0040] Color conversion processing is a process that converts the RGB information of the multiplexed image data so that it can be printed by the MFP 110. Generally, the colors described in the PDL drawing command are set using color values ​​that are expressed on the display, and if the values ​​are output as is by the printer, different colors will be output. For this reason, color conversion processing is performed.

[0041] For example, a three-dimensional lookup table (LUT) is used to calculate a combination of output pixel values ​​(Rout, Gout, Bout) for a combination of input pixel values ​​(Rin, Gin, Bin). For example, if the input values ​​Rin, Gin, and Bin each have 256 gradations, a table Table 1

[0256]

[0256]

[0256] [3] with a total of 16,777,216 sets of output values ​​of 256 x 256 x 256 is prepared in advance, and the output pixel value is calculated as follows.

[0042] Rout = Table1[Rin][Gin][Bin][0] Gout = Table1[Rin][Gin][Bin][1] Bout = Table1[Rin][Gin][Bin][2] Also, the number of grids in the lookup table may be reduced from 256 to, for example, 16 grids, and the table size may be reduced by determining the output value by interpolating table values ​​of a plurality of grids.

[0043] The ink color separation process is a process for converting the output values ​​Rout, Gout, and Bout of the color conversion process into output values ​​of each ink color to be recorded by the inkjet recording method. In this embodiment, four colors of ink, cyan, magenta, yellow, and black, are assumed as an example. In this embodiment, as in the color conversion process, a three-dimensional lookup table is used to calculate combinations of ink color pixel values ​​(C, M, Y, K) for combinations of output pixel values ​​(Rout, Gout, Bout). For example, Table2

[0256]

[0256]

[0256] [4] is prepared in advance, and the ink color pixel values ​​are calculated as follows.

[0044] C = Table2[Rout][Gout][Bout][0] M = Table2[Rout][Gout][Bout][1] Y = Table2[Rout][Gout][Bout][2] K = Table2[Rout][Gout][Bout][3] Also, the number of grids in the lookup table may be reduced from 256 to, for example, 16 grids, and the table size may be reduced by determining the output value by interpolating table values ​​of a plurality of grids.

[0045] The output characteristic conversion process converts the density of each ink color into a printing dot number rate. For example, the density of each color with 256 gradations is converted into the dot number rates Cout, Mout, Yout, and Kout with 1024 gradations. For this purpose, a one-dimensional lookup table is used in which the printing dot number rate for each ink color density is set. For example, Table 3[4]

[0256] is prepared in advance, and the printing dot number rate is calculated as follows.

[0046] Cout = Table3[0][C] Mout = Table3[1][M] Yout = Table3[2][Y] Kout = Table3[3][K] Also, the number of grids in the lookup table may be reduced from 256 to, for example, 16 grids, and the table size may be reduced by determining the output value by interpolating table values ​​of a plurality of grids.

[0047] The quantization process converts the recording dot count rates Cout, Mout, Yout, and Kout of each ink color into the On / Off of the actual recording dots of each pixel. For example, error diffusion or dithering can be used as a quantization method. In the case of dithering, for example, the On / Off of the recording dots of each ink color is output by comparing with a threshold value corresponding to each pixel position as shown below. For example, the occurrence probability of each recording dot is Cout / 1023, Mout / 1023, Yout / 1023, and Kout / 1023.

[0048] Cdot = Halftone[Cout][x][y] Mdot = Halftone[Mout][x][y] Ydot = Halftone[Yout][x][y] Kdot = Halftone[Kout][x][y] As described above, the print image data is generated in S204. In S205, the CPU 501 transmits the print image data generated in S204 to the MFP 110 via the data transfer interface 104, and the MFP 110 prints the image data on a recording medium.

[0049] In this embodiment, as described above, printing is performed on a recording medium based on multiplexed image data obtained by multiplexing rendered image data with PDL data as multiplexing information. In this embodiment, the process of printing based on multiplexed image data with PDL data multiplexed is called a "multiplexing encoding process." In a printed matter printed based on the multiplexed image data, for example, a pattern is drawn with a two-pixel diagonal line as shown in FIG. 7.

[0050] Next, the multiplexed PDL extraction process will be described.

[0051] 2, the CPU 111 of the MFP 110 controls the scanner unit to read a printed matter printed based on the multiplexed image data. For example, a user places a printed matter on the platen of the MFP 110 as a document to be read, and instructs scanning. By controlling the scanner unit by the CPU 111, LED light is irradiated onto the document, and the reflected light from the document is converted into an analog electrical signal by an imaging element such as a CCD corresponding to each pixel.

[0052] Next, in S212, the CPU 111 converts the analog electric signal into digital data to generate bitmap read image data composed of RGB pixel values. In the process of generating the bitmap read image data, for example, the following method is used.

[0053] In S212, four processes, namely, MTF (Modulation Transfer Function) correction, input correction, shading correction, and color conversion, are performed on each pixel of the bitmap read image data composed of the acquired RGB pixel values.

[0054] MTF correction processing is a process that corrects the resolution of the scanner unit's reading performance. For example, when reading with a scanner unit, the image becomes blurred due to deviation from the focus position or limitations of the optical characteristics of the lens itself, so a certain degree of restoration is performed using filter processing, etc. If strong emphasis processing is performed to completely restore the image, image defects such as blown-out highlights and emphasis of image noise and dust pixels will become more noticeable, so restoration is performed by balancing image quality improvement and defects. For example, the following edge emphasis filter is used, which multiplies the center of the image by 5 and multiplies the pixel values ​​of the top, bottom, left, and right by -1.

[0055] R'[x][y] = R[x][y]×5-R[x-1][y]-R[x+1][y]-R[x][y-1]-R[x][y+1] G'[x][y] = G[x][y]×5-G[x-1][y]-G[x+1][y]-G[x][y-1]-G[x][y+1] B'[x][y] = B[x][y]×5-B[x-1][y]-B[x+1][y]-B[x][y-1]-B[x][y+1] Input correction is a process that converts the CCD output value, which is originally a photon amount, into a brightness that matches the sensitivity of the human eye. For example, it is a process that converts the R'G'B' signal with 4096 gradations for each color into color intensity values ​​R", G", B" with 1024 gradations for each color. For this purpose, a one-dimensional lookup table Table 4[3]

[4096] is prepared in advance, which sets the recording dot count rate for each ink color density, and the color intensity value is calculated as follows:

[0056] R” = Table4[0][R'] G” = Table4[1][G'] B” = Table4[2][B'] Also, the number of grids in the lookup table may be reduced from 4096 grids to, for example, 256 grids, and the table size may be reduced by determining the output value by interpolating table values ​​of a plurality of grids.

[0057] Shading correction is a process that reduces color and density unevenness caused by differences in reading sensitivity at each pixel position due to manufacturing and assembly variations in the lenses, LEDs, and CCDs that make up the scanner unit. For example, R"G"B" signals with 1024 gradations for each color are converted into color intensity values ​​R"', G"', B"' with 256 gradations for each color. For this purpose, a one-dimensional lookup table Table5[x][3]

[1024] for density conversion for each X pixel position is prepared in advance for the direction in which the scanner lens is arranged (the X direction), and color intensity values ​​are calculated as follows:

[0058] R”’ = Table5[x][0][R”] G”’ = Table5[x][1][G”] B”' = Table5[x][2][B”] Also, the number of grids in the lookup table may be reduced from 1024 grids to, for example, 256 grids, and the table size may be reduced by determining the output value by interpolating table values ​​of a plurality of grids.

[0059] The color conversion process is the opposite of printing; the R"', G"', and B"' calculated up to this point are each scanner unit-specific values, so they are converted into Rout, Gout, and Bout values ​​for display on the display. To do this, just like with color conversion when printing, since the input values ​​R"', G"', and B"' each have 256 gradations, a table Table 6

[0256]

[0256]

[0256] [3] is prepared in advance, which has a total of 16,777,216 sets of output values ​​of 256 x 256 x 256, and the output values ​​are calculated as follows.

[0060] Rout = Table1[R”'][G”'][B”'][0] Gout = Table1[R”'][G”'][B”'][1] Bout = Table1[R”'][G”'][B”'][2] Also, the number of grids in the lookup table may be reduced from 256 to, for example, 16 grids, and the table size may be reduced by determining the output value by interpolating table values ​​of a plurality of grids.

[0061] As described above, the bitmap read image data is generated in S212.

[0062] Fig. 4 is a diagram for explaining image degradation due to printing and reading. Fig. 4(a) shows a PDL image 400 obtained by rendering the document 300 in Fig. 3. In other words, it corresponds to the upper left part of the object 301 in the rendered image data generated in S202 in Fig. 2. Fig. 4(b) shows a read image 401 obtained by reading a printout of the PDL image 400.

[0063] As can be seen by comparing the two, the straight lines in the PDL image 400 have a uniform density (R=128, G=128, B=128), but the straight lines in the scanned image 401 are non-uniform in density, with noticeable unevenness at the edges of the lines. In addition, there are some darker pixels around the lines, and the background image is a little dark overall. This is because the following image degradation occurs in the bitmap scanned image data generated in S212 by printing and scanning based on the PDL data, compared to the PDL image 400 (original document).

[0064] Deterioration of shape due to quantization performed in S204 (reduction in the number of gradations due to quantization) Deterioration of shape due to printing performed in S205 (printing accuracy, recording medium (printing paper)) Deterioration of the shape of the document due to scanning in S211 (image blurring caused by the scanner) Furthermore, deterioration of the shape due to deterioration such as soiling of the recording medium is expected.

[0065] In S213, the CPU 111 extracts multiplexed drawing information, that is, PDL data, from the bitmap read image data acquired in S212. An example of bitmap image analysis will be outlined below.

[0066] First, the position where the multiplexed information (also called additional information) is embedded from the image data is detected. Specifically, the embedding position is detected by analyzing the spatial frequency characteristics of the image data. Figure 14 shows the spatial frequency characteristics of the pattern used for embedding. The horizontal axis shows the horizontal frequency, and the vertical axis shows the vertical frequency, with the frequency increasing as the distance from the origin increases.

[0067] In this embodiment, as shown in Fig. 7(a) and Fig. 7(b), two patterns are embedded in an image. In the embedding example, 10 is added to and subtracted from the B component of RGB. As a result, the pattern in Fig. 7(a) generates a large power spectrum on line A01 in Fig. 14. Similarly, the pattern in Fig. 7(b) generates a large power spectrum on line A02. This power spectrum is detected, for example, for each 8 x 8 pixel region, to extract data of "0" or "1". Note that, as a pre-processing step of the detection, edge detection may be performed to emphasize the power spectrum.

[0068] In the extraction of data by frequency analysis, accurate cutting of the analysis area from the image data is required, so a process to correct the deviation of the coordinate position is performed. For example, cutting of an 8 pixel x 8 pixel area from the image and frequency analysis are repeated vertically and horizontally while shifting by one pixel in each direction. Then, for example, a method is to repeat this 64 times until it is shifted by 8 pixels horizontally and 8 pixels vertically, and the point where the spectrum is strongest is set as the reference position for cutting. In this case, the area to be subjected to frequency analysis is a 16 pixel x 16 pixel area, and if it is within an area where the additional information is multiplexed without gaps, at least one additional information will be included in that area. Therefore, the position of the reference area can be determined as described above. After the position detection is completed, the additional information is extracted according to the multiplexing rules using that position as the reference, and the embedded sequence of "0"s and "1"s is obtained.

[0069] If the head of the code region in which the additional information is embedded is selected as the position in the image where the reference position is to be specified by the above method, the specified reference position will indicate the head of the code region. Even if this is not the case, it is possible to roughly specify where the region in which the reference position is specified is located in the multiplexed code string, so that the head of the multiplexed code string can be specified based on that position. By analyzing the spatial frequency of the 8×8 pixel region specified as the head and the 8×8 pixel regions before and after it, the head of the region in which the additional information is multiplexed can be specified. Since the length of the additional information may be fixed, the entire region in which the additional information is multiplexed can be specified. The entire code region specified in this way can be decoded into binary data by the spatial frequency analysis shown in FIG. 14.

[0070] In S214, the CPU 111 renders the PDL data extracted in S213 to generate rendered original image data. The rendering process is the same as that in S202. In S215, the CPU 111 compares and combines the bitmap read image data generated in S212 and the rendered original image data rendered in S214 to generate corrected original image data.

[0071] 5 is a flowchart showing the process of generating corrected original image data in S215. In S501, the CPU 111 performs normalization processing on the bitmap read image data generated in S212. This is because the dynamic ranges of the bitmap read image data generated in S212 and the rendered original image data rendered in S214 are different.

[0072] For example, the brightest part of the Bitmap read image data is generally the color of the paper of the document to be read, and in principle, has a value with some density. On the other hand, the brightest part of the rendered original image data is a pixel of R=255, G=255, B=255, so there is originally a difference between the brightest colors of the two. Similarly, the darkest part of the Bitmap read image data is generally the black color of ink or toner, and in principle, has a value with some brightness due to reflected light. On the other hand, the brightest part of the rendered original image data is a pixel of R=0, G=0, B=0, so there is originally a difference between the darkest colors of the two. Also, even in the color of a color document, even the most vivid red color printed on the document to be read has a lower saturation than the most vivid red color (R=255, G=0, B=0) in the rendered original image data. In S501, the Bitmap read image data is normalized so that the dynamic ranges of the Bitmap read image data and the rendered original image data are matched.

[0073] For example, the following calculation is made for pixel values ​​R, G, and B of the Bitmap scanned image data: Rd, Gd, and Bd represent the minimum brightness of each color, and Rw, Gw, and Bw represent the maximum brightness of each color.

[0074] Rnorm = (R-Rd) / (Rw-Rd) x 255 Gnorm = (G-Gd) / (Gw-Gd) x 255 Bnorm = (B-Bd) / (Bw-Bd) x 255 By performing the above calculations, the brightest color of the bitmap read image data is normalized to R=255, G=255, B=255, and the darkest color of the bitmap read image data is normalized to R=0, G=0, B=0.

[0075] Next, in S502, the CPU 111 performs a filter process on the bitmap read image data normalized in S501. In S502, since the above-mentioned MTF correction has enhanced edges within a visually preferable range, a filter process is executed to perform stronger edge enhancement for use in image comparison.

[0076] Next, in S503, the CPU 111 performs a process of removing a multiplexed image on the bitmap read image data that has been subjected to the filter process in S502. By generating the multiplexed image data in S203 in FIG. 2, a difference occurs between the original image that is to be restored and the printed matter (original image). Therefore, this process is performed to remove the difference as much as possible. For example, since the multiplexed drawing information is extracted from the bitmap read image data acquired in S212, the bitmap read image data acquired in S212 can be made closer to the image data before multiplexing by subtracting the drawing information inversely. In this embodiment, for example, the multiplexed image is removed by further multiplexing a value obtained by multiplying a mask value that embeds 0 and 1 as shown in FIG. 6(a) and FIG. 6(b) by −1.

[0077] As described above, in this embodiment, the filter process of S502 is executed on the bitmap read image data generated by reading the document with a stronger correction strength (edge ​​emphasis) than the correction strength in S212. As a result, the image restored to a certain extent by the MTF correction is restored more completely, and the image deteriorated due to the document reading can be restored more effectively. In addition, the removal of the multiplexed image of S503 is executed on the bitmap read image data generated by reading the document. As a result, the image deteriorated due to the pattern (multiplexed image, for example the above-mentioned two-pixel diagonal line) that appears due to the multiplexing of drawing information can be restored.

[0078] Next, in S504, the CPU 111 compares the image data. In this embodiment, the rendered original image data rendered in S214 is compared with the multiplexed image removed scanned image data generated in S503 on a pixel-by-pixel basis. For example, in S504, the difference amount for each color is obtained by the following formula:

[0079] ΔR = |R[x][y] of image A - R[x][y] of image B| ΔG = |G[x][y] of image A - G[x][y] of image B| ΔB = |B[x][y] of image A - B[x][y] of image B| Next, in S505, the CPU 111 determines whether or not the difference amount of pixel values ​​exceeds a threshold value. In this embodiment, there are threshold values ​​Rth, Gth, and Bth for each of the R, G, and B channels, and the determination is made based on, for example, the following conditional expression.

[0080] If ((ΔR > Rth)|| (ΔG > Gth)|| (ΔB > Bth) ){”Yes”} Else {”No”} In this example, the thresholds are, for example, Rth=Gth=Bth=64, but may be set according to the characteristics of the scanner unit or printer unit. If it is determined that the difference is greater than the threshold, the process proceeds to S507. On the other hand, if it is determined that the difference is equal to or less than the threshold, the process proceeds to S506.

[0081] In S506, the CPU 111 overwrites pixel values. The CPU 111 overwrites pixel values ​​of the rendered original image data with pixel values ​​of the image data from which the multiplexed image has been removed. That is, the following process is performed on pixel values ​​of each color.

[0082] R[x][y] of the rendered original image data = R[x][y] of the demultiplexed image data G[x][y] of the rendered original image data = G[x][y] of the demultiplexed image data B[x][y] of the rendered original image data = B[x][y] of the demultiplexed image data For example, a separate plane of overwrite data is prepared for overwriting pixel values. The image format used is RGBα (8 bits per color, 32 bits per pixel) image data that includes an α value that indicates transparency in addition to RGB.

[0083] 5, the overwrite data for all pixels is α=255 (maximum transparency=no overwrite data). In S506, pixel values ​​are overwritten based on the following formula.

[0084] R[x][y] of overwritten data = R[x][y] of image data after multiplexing has been removed G[x][y] of overwritten data = G[x][y] of image data after multiplexing has been removed B[x][y] of overwritten data = B[x][y] of image data after multiplexing has been removed Overwrite data α[x][y] = 0 (lowest transparency = opaque) Next, in S507, the CPU 111 judges whether the judgment process of S505 has been performed for all pixels. If it is judged that the judgment process has not been performed for all pixels, the CPU 111 focuses on other pixels and repeats the process from S504. On the other hand, if it is judged that the judgment process has been performed for all pixels, the process of FIG. 5 is terminated.

[0085] As a result of the process in Fig. 5, corrected original image data is generated in which overwrite data is placed on top of the rendered original image data rendered in S214. In S216 in Fig. 2, the CPU 111 stores the corrected original image data generated in S215, and then ends the multiplexed PDL extraction process. After the multiplexed PDL extraction process, for example, the CPU 111 performs a print process based on the corrected original image data using the printhead controller 115.

[0086] By the above process, for example, a deteriorated image obtained by scanning an original as shown in Fig. 4(b) can be restored to an image close to that shown in Fig. 4(a). The process of calculating the difference in pixel values ​​for each pixel in S505 and overwriting in S506 will be further described below.

[0087] FIG. 9 is a diagram for explaining processing of an image to which handwritten characters have been added. A PDL image 900 in FIG. 9(a) is the same as the PDL image 400 in FIG. 4(a). FIG. 9(b) is a read image 901 obtained as a result of reading an original. Unlike FIG. 4(b), the read image 901 shows the result of reading an original in a state in which the character "Check" 902 has been added to the printout of the PDL image 900. The character "Check" 902 does not exist in the original original, so it is not rendered in S202 in FIG. 2, and is not included in the multiplexed image data generated in S203. Therefore, it is not included in the PDL data extracted in S213, and is not included in the rendered original image data generated in S214. Therefore, if the rendered original image data generated in S214 is preferentially used as restored image data, the character "Check" 902 in the read image 901 will be missing.

[0088] In this embodiment, for the portion of the character "Check" 902, it is determined in S504 to S505 that the difference in pixel value is greater than the threshold, and the pixel is set as a target pixel for overwriting in S506. That is, in this embodiment, as in the overwritten image 903 in Fig. 9, pixels in the scanned image are used for pixels with large differences, and pixels in the original image are used for pixels with small differences. As a result, it is possible to make the portion not added by hand closer to the original image, without missing any portion added by hand.

[0089] As described above, in this embodiment, PDL data is extracted from the bitmap read image data obtained by reading a printed matter, and is combined with the bitmap read image data (handwritten character portion). At this time, by performing the filter process of S502 on the bitmap read image data, it is possible to restore an image that has deteriorated due to document reading. In addition, by performing the removal of the multiplexed image of S503 on the bitmap read image data, it is possible to restore an image that has deteriorated due to the printing process of the multiplexed image. In addition, by combining the bitmap read image data with the PDL data (original image), it is possible to restore an image that has deteriorated due to dirt on the recording medium, and to prevent handwritten characters added to the printed matter from being lost.

[0090] 6 and 7, an example of multiplexing the multiplexing information in units of 8×8 pixels has been described. This is an example of multiplexing that is difficult for a user to see by multiplexing the multiplexing information in a high frequency region. However, any multiplexing method may be used other than multiplexing the multiplexing information in a high frequency region.

[0091] FIG. 8 is a diagram for explaining another example of the multiplexing method. FIG. 8(a) shows an example of a QR code pattern. In this example, PDL data is converted into a QR code, which is then multiplexed in a manner that makes it difficult to see. FIG. 8(b) shows an actual print pattern, in which only one dot is printed per 8×8 pixel unit. A dot corresponding to a black pixel 800 in FIG. 8(a) corresponds to one dot, dot 801, in FIG. 8(b). Also, no dot is formed in a position in FIG. 8(b) that corresponds to a blank pixel in FIG. 8(a).

[0092] By using such a method, a multiplexed pattern that is difficult to see can be formed on a recording medium. For example, in S203 of Fig. 2, the PDL data is converted into a QR code as shown in Fig. 8(a), and this is superimposed on the rendered image data as image data of a group of spaced dots as shown in Fig. 8(b). Furthermore, since yellow ink is the most difficult to see among the inks that make up the spaced dots, by forming the pattern of Fig. 8(b) with yellow ink, a multiplexed pattern that is even more difficult to see can be formed on a recording medium.

[0093] In the case of this multiplexing method, when extracting the PDL data in S213, it is necessary to determine whether yellow dots are recorded in 8 x 8 pixel units in the read Bitmap read image data, extract the QR code pattern equivalent to Figure 8(a), and decode it to extract the PDL data.

[0094] As another multiplexing method, for example, a method of multiplexing by performing threshold modulation during quantization processing in the print image data generation processing of S204 in Fig. 2 may be used. Furthermore, the removal of the multiplexed image in S503 can be performed by multiplexing a value obtained by multiplying the amount of change in pixel value during multiplexing by -1, in the same manner as described above according to the multiplexing method used.

[0095] Generally, when the amount of information to be multiplexed increases, it becomes necessary to increase the number of recording dots and the amount of modulation, which tends to increase image degradation of the entire printed image. Furthermore, image degradation also appears in the read image obtained by reading such a printed image. In this embodiment, even in such a case, the multiplexed image is removed in S503, so that it is possible to restore the image degradation that occurs when the amount of information to be multiplexed increases.

[0096] As mentioned above, the original image 300 in Fig. 3 is composed of 600 pixels in width and 900 pixels in height, so one page is composed of 540,000 pixels. If the color of each pixel is expressed in 256 gradations with 8 bits each for R / G / B, each pixel is 3 bytes, and the total size of one page is 1,620,000 bytes, which is approximately 1.6 MB. On the other hand, the file size in PDL state is, for example, 106,200 bytes, which is about 6.6%, as shown below. The breakdown of the file size estimate is shown below.

[0097] -Drawing command section: approx. 350 characters (approx. 700 bytes, assuming 2 bytes per character) "PORTRAIT.jpg" 300 x 475 pixels, each pixel is 3 Bytes, totaling 427,500 Bytes. If the compression rate is 20%, the total becomes 85,500 Bytes. "STD" font data is assumed to be 20,000 Bytes for the following reasons: For example, assuming a 20-page document, the PDF file size without font embedding is approximately 60KB, whereas the PDF file size with font embedding is 460KB, which is a nearly 900% increase in size (assuming 75% of the new characters per page). Therefore, the amount of font embedding per page is assumed to be 400kBytes / 20=20kBytes.

[0098] As described above, the size of the PDL data acquired in S201 is extremely small, about 6.6%, compared to the Bitmap image data rendered in S202. Therefore, when multiplexing, multiplexing PDL image data rather than Bitmap image data can greatly reduce image degradation of the entire print image that accompanies multiplexing.

[0099] It is also possible to completely restore the PDL data (original document) by reading a printout in which the PDL data is directly embedded. In that case, since all of the drawing commands associated with FIG. 3 can be restored, for example, the read image 401 in FIG. 4(b) can be made completely the same as the PDL image 400 in FIG. 4(a). Similarly, characters and thin lines such as objects 303 and 304 in FIG. 3 can also be restored. Since copy processing generally involves processes from reading to printing, image degradation is unavoidable, but if the PDL data of the original document can be extracted during scanning, a nearly perfect copy can be realized by printing based on the extracted PDL data.

[0100] Furthermore, when reduction or enlargement is involved during printing, thin lines and characters may become thinner or thicker, causing the characters and thin lines to become illegible or disappear; however, according to this embodiment, the multiplexed PDL data can be extracted, making it possible to avoid such cases.

[0101] In addition, when copying with reduced or enlarged size, the characters and thin lines are also reduced or enlarged, which may result in poor visibility, but if the multiplexed PDL data can be extracted as in this embodiment, it is possible to change the size of the TEXT drawing command, for example. Therefore, as shown in the following example, copying may be performed in which the size of only the characters is changed.

[0102] - "Reduced copy while maintaining font size" by setting 50% reduction + font size 200% - "Enlarged copy with font size maintained" by setting 200% enlargement + font size 50% - "Enlarged text copy" with normal size + text size 200% - "Reduced text copy" with normal size + 50% text size Similarly, in this embodiment, since multiplexed PDL data can be extracted, in the copy process, it is also possible to change the line thickness, line type, color, and fill pattern for other commands such as a LINE drawing command.

[0103] In addition, in this embodiment, the display control may be performed using the corrected original image data saved in S216. As described above, when handwritten characters are added on the scanned document, the corrected original image data is saved by combining the image of the handwritten part with the original image represented by the multiplexed PDL data. At that time, the image data part to which the handwritten characters are added (for example, the character 902 in FIG. 9(b)) may be saved as individual image data. Then, the display control on the display unit may be performed using the individual image data. For example, the handwritten character part may be displayed in a identifiable manner, such as by displaying it in red or blinking. Also, the handwritten character part may be erased and the image before the handwritten characters are added (i.e., the original image) may be displayed. Also, the image before the handwritten characters are added and the image after the handwritten characters are added may be displayed in parallel. Also, a selection from the user may be accepted in the parallel display, and the selected image may be printed.

[0104] 2 is executed by the PC 100, and the multiplexed PDL extraction process is executed by the MFP 110. However, the PDL multiplexed print process and the multiplexed PDL extraction process may be executed within one device. For example, both processes may be executed by the MFP 110.

[0105] [Second embodiment] The second embodiment will be described below with respect to the differences from the first embodiment. In the first embodiment, the configuration in which PDL data is multiplexed onto rendered image data has been described. In this embodiment, only a portion of the PDL data is multiplexed onto rendered image data.

[0106] There may be cases where the multiplexed drawing information is difficult to extract because information is overwritten on the printed document image with handwritten characters, stamps, etc. There may also be cases where multiplexing is difficult because the size of the image file in the document image is too large, or where multiplexing cannot be performed because image degradation during multiplexing is significant. In this embodiment, only a portion of the PDL data is subject to multiplexing, making it possible to avoid such cases.

[0107] Fig. 10 is a diagram showing an example of an original image represented by PDL data in this embodiment. An original image 1000 and objects 1001 to 1004 in Fig. 10 correspond to the original image 300 and objects 301 to 304 in Fig. 3. The description contents of the multiplexing PDL in this embodiment are shown below.

[0108] <PAGE=001> <box> 50, 75, 200, 300, GRAY, STRIPE< / box> <box> 420, 75, 550, 300, BLACK, FULL< / box> <box> 50,620,200,825,BLACK,NULL< / box> <text>50,620, 200, 825, BLACK, STD-18, "ABCXYZ123" <line> 25, 25, 25, 700, BLACK, PLAIN, 3< / line> <line> 25, 25, 450, 25, BLACK, PLAIN, 3< / line> In this embodiment, the following described in the first embodiment is A drawing command for the image 250, 350, 550, 825, "PORTRAIT.jpg" is not written. In the first embodiment, the following premise has been described.

[0109] -Drawing command section: approx. 350 characters (approx. 700 bytes, assuming 2 bytes per character) "PORTRAIT.jpg" 300 x 475 pixels, each pixel is 3 Bytes, totaling 427,500 Bytes If the compression rate is 20%, the result is 85,500 Bytes. "STD" font data is assumed to be 20,000 Bytes for reasons explained later. In other words, as will be shown below, if images and fonts are not embedded, the amount of multiplexing data can be reduced accordingly, and the degree of image degradation due to multiplexing can be reduced.

[0110] Drawing instructions + images + fonts 106,200Bytes (100%) Drawing instructions + fonts 20,700Bytes (19%) Drawing command 700Bytes (<1%) Therefore, in this embodiment, when multiplexed image data is generated in S203 of the PDL multiplexing print process in Fig. 2, the multiplexed image data is generated using the above-mentioned multiplexing PDL data. Therefore, the PDL data extracted in S213 becomes the above-mentioned multiplexing PDL data. Moreover, the rendered original image data rendered in S214 is an image that does not contain a human image, as shown in Fig. 10.

[0111] Then, when generating the corrected original image data in S215, the bitmap read image data obtained by reading the printed matter on which the person image of object 305 in Fig. 3 is printed is compared with the rendered original image data in which no person image exists. In S505 and S506 in Fig. 5, the part containing the person image is processed as the "part to which handwritten characters have been added" described in the first embodiment. That is, the part containing the person image in the bitmap read image data is generated as overwrite data for the rendered original image data in which no person image exists, which was rendered in S214. Then, the corrected original image data is generated with the overwrite data placed at the top.

[0112] As described above, according to this embodiment, it is possible to restore the original image from the read image, and also to reduce the load of multiplexed information.

[0113] [Third embodiment] The following describes the present embodiment with respect to differences from the first and second embodiments. In the first and second embodiments, a configuration was described in which overwrite data of the same image size as the PDL data is overwritten on rendered image data. For example, when the PDL image 300 is composed of 600 pixels in width and 900 pixels in height, one page is composed of 540,000 pixels. When the color of each pixel is expressed in 256 gradations with 8 bits each for R / G / B / α, each pixel is 4 bytes, and one page is 2,160,000 bytes ≒ 2.2 MBytes. In this embodiment, the amount of data can be reduced by dividing the image area into multiple blocks (i.e., unit areas) and generating overwrite data for each block.

[0114] Fig. 11 is a diagram showing an example of corrected original image data in this embodiment. A corrected PDL image 1100 and objects 1101 to 1105 in Fig. 11 correspond to the PDL image 300 and objects 301 to 305 in Fig. 3. Broken lines are shown on the corrected PDL image 1100 in Fig. 11, dividing it into five parts in the horizontal and vertical directions. In this embodiment, the entire PDL image is divided into 25 parts, and overwrite data corresponding to necessary parts is overwritten on the rendered image data.

[0115] Fig. 12 is a flowchart showing the process of generating corrected original image data in this embodiment. Since S1201 to S1203 in Fig. 12 are the same as S501 to S503 in Fig. 5, the description thereof will be omitted.

[0116] In S1204, the CPU 111 compares the image tiles. In this example, the image tiles are each image tile (120 pixels, 180 pixels) obtained by dividing the entire image (600 pixels wide, 900 pixels high) vertically and horizontally into five. The CPU 111 compares the rendered image data with the image data from which the multiplexing information has been removed for each image tile. The CPU 111 measures the number of pixels with a difference as a comparison result for each image tile.

[0117] In S1205, the CPU 111 judges whether or not the number of pixels with difference in the image tile exceeds a predetermined threshold. In other words, in S1205, it is judged whether or not there is a "portion to which handwritten characters or the like are added" in the image tile. The predetermined threshold may be the number of pixels or a ratio. For example, 216 pixels, which is about 1%, is used as the threshold. Here, the threshold may be set according to any standard. For example, 216 pixels, which is 1%, is set as a negligible number of pixels due to dust on the paper or the scanner glass, for example. If it is judged in S1205 that the threshold has been exceeded, the process proceeds to S1206. On the other hand, if it is judged that the threshold has not been exceeded, overwrite data is not generated for the image tile, and the process from S1204 is repeated with attention focused on the next image tile.

[0118] In S1206, the CPU 111 places the image data of the image tile of the image data from which the multiplexing information has been removed on the rendered image data. In this case, when the image tile is overwritten, the image data may be 3 bytes with 8 bits for each R / G / B pixel, so the image data amount of each tile is 64,800 bytes. In other words, the amount of data can be reduced more than when it is determined whether to overwrite the image on a pixel-by-pixel basis by determining the difference in pixel value in the first embodiment. The amount of image data of each image tile is reduced as the number of pixels of the image tile is reduced. Therefore, the possibility of including unnecessary data other than the portion that needs to be overwritten is reduced. In addition, the image data may be compressed in order to reduce the amount of image data.

[0119] FIG. 13 is a diagram for explaining the process for an image with additional handwritten characters in this embodiment. FIG. 13(a) shows an image represented by rendered image data. FIG. 13(b) shows an image represented by read image data obtained by reading an original. In FIG. 13, each image is divided into 4×4 image tiles, and the CPU 111 assigns identification information such as 13a00 to 13a33 and 13b00 to 13b33 to the tiles and manages them. For example, a rule is set for the identification information in image tile 13-a-xy, where x represents the horizontal tile position number and y represents the vertical tile position number. The same applies to tiles 13-bxy and 13-cxy.

[0120] Fig. 13(c) shows the result of overwriting the eight image tiles 13a11, 13a12, 13a13, 13a14, 13a21, 13a22, 13a23, and 13a24, which contain the added handwritten character string "Check", as performed in steps S1205 and S1206. As shown in Fig. 13(c), the scanned image data is overwritten only on the above eight image tiles.

[0121] As described above, according to this embodiment, the rendered image data is overwritten with data only in the area that needs to be overwritten, and this configuration makes it possible to reduce the size of the corrected original image data.

[0122] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0123] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0124] 100 PC: 101, 111 CPU: 110 MFP: 116 Image processing accelerator< / text>

Claims

1. a reading means for optically reading a printout on which drawing information of an original image is multiplexed, and acquiring read image data; an extracting means for extracting the drawing information from the read image data acquired by the reading means; an acquisition means for acquiring original image data representing the original image based on the drawing information extracted by the extraction means; a correction means for comparing the read image data and the original image data by dividing an image indicated by each image data into a plurality of regions, determining whether or not to use the read image data and the original image data for each of the plurality of regions based on a result of the comparison, and correcting the original image data based on the result of the determination; a storage unit that stores the corrected original image data corrected by the correction unit as a result of reading the printed matter; An image processing system comprising:

2. 2. The image processing system according to claim 1, wherein the correction means compares pixel values ​​of the read image data with pixel values ​​of the original image data, and corrects the original image data based on the result of the decision based on the result of the comparison.

3. The image processing system of claim 2, characterized in that when the difference between the pixel values ​​of the read image data and the pixel values ​​of the original image data in the target image area is greater than a threshold value, the correction means corrects the original image data by using the pixel values ​​of the read image data as the result of the reading instead of the pixel values ​​of the original image data.

4. 4. The image processing system according to claim 3, wherein said correction means does not correct said original image data for said target image region when said difference is not greater than said threshold value.

5. a first processing unit that performs a normalization process on the read image data acquired by the reading unit before the comparison; 5. The image processing system according to claim 1, wherein the comparison is performed using the read image data on which the normalization process has been performed.

6. a second processing unit that performs a correction process for the optical characteristics of the reading unit on the read image data acquired by the reading unit before the comparison; 6. The image processing system according to claim 1, wherein the comparison is performed using the read image data that has been subjected to a correction process for the optical characteristics.

7. 7. The image processing system according to claim 6, wherein a correction intensity of the correction process for the optical characteristics is greater than a correction intensity of the correction process for the optical characteristics performed when the reading unit obtains the read image data.

8. a third processing unit that performs a removal process for removing the drawing information from the read image data acquired by the reading unit before the comparison; 8. The image processing system according to claim 1, wherein the comparison is performed using the scanned image data on which the removal process has been performed.

9. a multiplexing means for multiplexing the drawing information onto image data of the original image to generate multiplexed image data; a generating unit that generates print image data for printing the printed matter based on the multiplexed image data generated by the multiplexing unit; The image processing system according to claim 1 , further comprising:

10. 10. The image processing system according to claim 9, wherein the drawing information multiplexed by the multiplexing means is a part of the drawing information of the original image.

11. The image processing system includes an information processing device and an image processing device, The information processing device includes the multiplexing means and the generating means, The image processing device includes the reading means, the extracting means, the acquiring means, the correcting means, and the storing means.

11. The image processing system according to claim 9, wherein the first and second inputs are input to the image processing apparatus.

12. 11. The image processing system according to claim 9, wherein the image processing system is an image processing apparatus capable of executing a printer function and a scanner function.

13. 1. An image processing method executed in an image processing system, comprising: a reading step of optically reading a printout on which drawing information of an original image is multiplexed, and acquiring read image data; an extraction step of extracting the drawing information from the read image data acquired in the reading step; an acquisition step of acquiring original image data representing the original image based on the drawing information extracted in the extraction step; a correction step of comparing the read image data and the original image data by dividing an image indicated by each image data into a plurality of regions, determining whether or not to use the read image data and the original image data for each of the plurality of regions based on a result of the comparison, and correcting the original image data based on the result of the determination; a storing step of storing the corrected original image data corrected in the correcting step as a result of reading the printed matter; 13. An image processing method comprising:

Images