Information processing apparatus, method, and program

Abstract

To extract additional information embedded in a metallic area and a color area.SOLUTION: An information processing device includes a light source, an adjustment unit that adjusts the amount of light from the light source, an imaging unit that captures an image formed on a medium and acquires image data and an extraction unit that extracts additional information multiplexed in the image captured by the imaging unit. The information processing device adjusts the amount of light from the light source to the amount of light corresponding to the reflection intensity of the medium on which the image is formed, captures the image, and extracts additional information from the image. Here, the medium includes an area on which an image of metallic ink is formed and an area on which an image of metallic ink is not formed.SELECTED DRAWING: Figure 25

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program for extracting additional information embedded in a recorded image.

Background Art

[0002] An electronic watermarking technique in which other information related to an image is multiplexed with respect to image information is known. In the electronic watermarking technique, for example, additional information such as the name of the author and the permission to use is multiplexed in image information such as a photograph or a painting so that it is difficult to visually discriminate. Further, as an application field, with the improvement in image quality of image output devices such as copiers and printers, in order to prevent forgery of banknotes, stamps, securities, etc., a technique of embedding additional information in an image for identifying an output device and its machine number from an image output on paper is known. Further, in order to extract additional information embedded in an image output on paper, a technique of photographing an object output with an imaging device such as a scanner or a built-in camera of a mobile terminal and analyzing the photographed image to extract the embedded information is also known (Patent Document 1).

[0003] On the other hand, in printing by an inkjet recording apparatus using a metallic ink containing metal particles, a metallic luster can be imparted to a printed matter. By overlaying a conventional color ink on such a metallic ink, it is possible to output a metallic color printed matter.

[0004] Patent Document 2 discloses a method of realizing a metallic color expression by discharging ink while moving a recording head in a main scanning direction to apply a metallic ink as a base on a recording medium and applying a color ink thereon.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] When extracting additional information embedded in an image output on paper, the additional information embedded is extracted by analyzing a captured image obtained by irradiating the printed matter with light and imaging the reflected light. However, the reflection characteristics of each of a region formed with only color ink on the paper surface (hereinafter referred to as a color region) and a region formed with only metallic ink or formed with metallic ink and color ink (hereinafter referred to as a metallic region) are different. More specifically, the reflection intensity of the color region is smaller than the reflection intensity of the metallic region.

[0007] Therefore, when extracting the additional information embedded in the color region and the additional information embedded in the metallic region respectively, it has been difficult to extract both from an image captured by irradiating a light beam with the same amount of light.

[0008] The present invention has been made in view of the above conventional examples, and an object thereof is to make it easier to extract additional information embedded in a color region by increasing the amount of light for photographing the color region compared to the amount of light for photographing the metallic region and performing photographing.

Means for Solving the Problems

[0009] In order to achieve the above object, the present invention has the following configuration. That is, an adjustment means for adjusting the amount of light of a light source that irradiates light on an image formed on a medium to an amount of light corresponding to the reflection intensity of the medium, Determined by at least one of the presence or absence of an image formed with metallic ink and the type of the medium an acquisition means for acquiring image data obtained by photographing the image, and an extraction means for extracting additional information multiplexed in the image from the acquired image data. has and After the extraction means extracts the additional information from the image data when the light quantity of the light source is a first light quantity, the adjustment means adjusts the light quantity of the light source to be smaller than the first light quantity and to a second light quantity at which image data can be acquired from the image formed on the medium and ​An information processing apparatus is provided, which is characterized by...

Effects of the Invention

[0010] According to the present invention, by increasing the amount of light for photographing the color area compared to the amount of light for photographing the metallic area and performing photographing, additional information embedded in the color area can be extracted.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, all of these plurality of features are not necessarily essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0013] (First Embodiment) The information processing system according to the first embodiment of the present invention includes a multiplexing encoding processing unit that embeds additional information in image information, and a multiplexing decoding processing unit that extracts additional information from a captured image. In the following, the basic configuration of the information processing system and the characteristic configuration of the information processing system (particularly, the configuration of the multiplexing decoding processing unit) will be described separately.

[0014] (1) Basic configuration (1-1) Hardware of the multiplexing encoding processing unit and the multiplexing decoding processing unit FIGS. 1(a) and 1(b) are explanatory diagrams of a configuration example of hardware (multiplexing encoding processing unit) for multiplexing encoding for embedding additional information (also referred to as "multiplexing information" and "embedded information") in image information in the information processing system. This hardware acquires image data (image information) A and additional data (additional information) B, and generates a recording medium (printed matter) C in which the information A and B are embedded. The hardware in FIG. 1(a) is configured to perform a process of embedding additional information B in image data A by a device separate from the printer (recording device). The hardware in FIG. 1(b) is configured to perform a process of embedding additional information B in image data A within the printer (recording device).

[0015] In the configuration of FIG. 1(a), the image data A input from the input terminal 100 is multi-tone image data including color components. The additional information B input from the input terminal 101 is text document data, audio data, video data, or data obtained by compressing text document information, audio information, images, video information, and data converted into other binary values. The additional information multiplexing device 102 performs a process of embedding additional information B in image data A (also referred to as "multiplexing process" or "embedding process") as will be described later. The printer (recording device) 103 generates a recording medium C by a recording operation based on the image data A in which the additional information B is embedded. The additional information multiplexing device 102 can be realized, for example, by executing a multiplexing process program on a general-purpose computer.

[0016] In the configuration of FIG. 1(b), the additional information multiplexing unit 105 corresponding to the multiplexing device 102 in FIG. 1(a) is included in the printer 103. Similar to the configuration of FIG. 1(a), image data A is input from the input terminal 100, and additional information B is input from the input terminal 101. The additional information multiplexing unit 105 in the printer 103 performs a process of embedding the additional information B into the image data A. Also, the recording unit 106 in the printer 103 generates a recording medium C by a recording operation based on the image data A embedded with the additional information B. The process of generating the recording medium C based on the image data A embedded with the additional information B in this way is also called "multiplexing encoding process". Alternatively, the process of embedding additional information into image data and printing the image may be called multiplexing encoding process (or encoding process). Note that the additional information multiplexing unit 105 can be realized, for example, by a processor executing a multiplexing process program.

[0017] FIG. 2 is an explanatory diagram of a configuration example of hardware (multiplexing decoding processing unit) for multiplexing decoding to extract the additional information B from the image information recorded on the recording medium C in an information processing system. This hardware captures the multiplexing encoded recording medium C with an imaging device such as a camera and analyzes the captured image to extract the additional information B embedded in the image. Here, these series of processes are also called "reading process", "separation process", and "extraction process".

[0018] FIG. 2 is an explanatory diagram of a configuration example of hardware (multiplexing decoding processing unit) for multiplexing decoding to extract the additional information embedded in the recording medium in an information processing system. Here, a mobile terminal with a camera is adopted as an example of the information processing system.

[0019] In FIG. 2, a mobile terminal (information processing device) 201 with a camera that holds an imaging sensor 202 has a function of photographing a recording object C. An additional information separation device 203 extracts additional information B by analyzing an image photographed by the imaging sensor 202 which is a light receiving element unit, as will be described later. A CPU (central processing unit, also called a processor) 204 executes an information processing method according to a program. A program executed by the CPU 204 is stored in a ROM 205. A RAM 206 functions as a memory for temporarily storing various information when the program is executed by the CPU 204. A secondary storage device 207 such as a hard disk stores a database including image files and image analysis results. A display 208 presents the processing results of the CPU 204 and the like to the user. A key input device 209 uses the display 208 having a touch panel function to give instructions for processing and input characters by operating the touch panel.

[0020] A wireless LAN (Local Area Network) connection unit 210 is connected to the Internet via a wireless LAN, accesses a site connected to the Internet, and causes the display 208 to display a screen of the site and the like. The wireless LAN connection unit 210 is also used for sending and receiving data. A speaker 211 outputs sound when the extracted additional information is voice data or video data with voice. Also, when there is video data at the Internet connection destination, sound is output during playback of the video data. A light 212 irradiates the recording object C with light when photographing the recording object C. The amount of this light irradiation (or luminous flux) can be adjusted by control by the CPU 204 and the like. For example, when an LED is used as a light source, the amount of light can be controlled by controlling the input voltage by pulse width modulation or the like. The light amount control may be performed in a plurality of stages (for example, two stages) or continuously.

[0021] The mobile terminal 201 with a camera is not limited to a configuration that incorporates an imaging sensor 202. For example, it may be configured such that an imaging sensor 202 is controlled by a device separate from the mobile terminal 201 and the captured image is transmitted to the additional information separation device 203. As the imaging sensor 202, a digital camera, a video camera, etc. can be used, and as the additional information separation device 203, a personal computer, a smartphone, etc. can be used, as long as it can extract the additional information B from the recording medium C. In the following, the process of extracting the additional information B from the recording medium C is also referred to as "multiplex decoding process". Or it may simply be called decoding process.

[0022] (1-2) Firmware Configuration for Multiplex Encoding Process FIG. 3 is a block diagram of a basic firmware configuration for multiplex encoding process. The image data is subjected to the following processes, converted to a resolution and the number of gradations that can be received by the recording engine connected to the recording head, and then transmitted to the recording engine.

[0023] (1-2-1) Attached Information Acquisition Unit The attached information acquisition unit 301 acquires various parameters used when compressing the image data. The various parameters are stored in association with the compressed image data, and may be included, for example, in the file of the compressed image data, or attached as attribute information or the like. The acquired various parameters are sent to the image data restoration unit 302 and used for the process of extracting (restoring) the image data from the compressed image. Also, the acquired various parameters are used for the process of calculating the compression degree. For example, the image data A which is the input image is irreversible image data obtained by compressing document data in the JPEG format and is recorded on a recording medium. The irreversible image data includes the quantization table and the image data size used when compressing. If the compressed image data is a JPEG image, the acquisition unit 301 acquires the image data size and the quantization table. The acquired image data size information, codebook, and quantization table are sent to the image data restoration unit 302.

[0024] (1-2-2) Image Data Restoration Unit The image data restoration unit 302 decodes the encoded image data and extracts the image data. In the following, the input image will be described as a JPEG image.

[0025] Figure 4 is a flowchart for explaining the process in which the restoration unit 304 decodes the encoded image data. For example, assume that an image in the JPEG data format is divided into N 8-pixel square blocks (8×8 pixel blocks). First, the Huffman code of the first block is decoded in units of 8-pixel square blocks (S41, S42), inverse quantization is performed using the obtained quantization table (S43), and then it is inverse DCT-transformed (S44). This is executed for all N blocks of the target screen (S45, S46). Since the JPEG decoding process uses a method that is already known, in the following, the details of Huffman coding, inverse quantization, and inverse DCT transformation will be omitted, and only an overview of them will be described.

[0026] Huffman coding is a compression method that reduces the total number of bits by assigning short-bit-number codes to frequently occurring data. Huffman decoding decodes the original data according to a specification in which the Huffman code is defined in advance. For example, a codebook is defined as the specification, and Huffman coding and decoding are performed using it. In inverse quantization, it is a process of expanding the image data by inverse quantization using the quantization table (the quantization table used when compressing the image data) obtained by the attached information acquisition unit 301. Inverse DCT transformation is a process of performing an inverse transformation to return the image data DCT-transformed into a direct current component (DC component) and an alternating current component (AC component) to the data of the original image density component. JPEG compression is often performed in the luminance Y, color difference Cb, Cr format. In that case, the data subjected to the inverse DCT process also becomes in the YCbCr format. The values in the YCbCr format are converted to RGB format image signal values by the following formula 1.

[0027] Formula 1: R = Y + 1.402×Cr G = Y - 0.344×Cb - 0.714×Cr B = Y + 1.772×Cb (1-2-3) Image correction unit The image correction unit 303 performs image correction processing on the RGB data decoded by the image data restoration unit 302. Image correction includes brightness adjustment to lighten or darken the overall color, contrast adjustment, and color balance adjustment, as well as various corrections such as backlight correction and red-eye correction assuming photo recording. By performing these corrections uniformly in the image correction unit 303, processing independent of the recording device can be realized.

[0028] (1-2-4) Resolution conversion unit The resolution conversion unit 304 converts the image data to a resolution corresponding to the recording device. Based on the input image data, the resolution of the recording device, and the magnification factor derived accordingly, enlargement or reduction processing is performed. There are methods such as nearest neighbor, bilinear, and bicubic for the magnification process, and an appropriate selection can be made considering the characteristics of the process and the processing speed. In the example of FIG. 5, multiplexing of additional information is performed after the resolution conversion process, and color correction is performed on the image data with the additional information multiplexed. However, here, first, color correction will be described, and the additional information 309 and the additional information multiplexing unit 310 will be described after the tone conversion unit 308.

[0029] (1-2-5) Color correction unit The color correction unit 305 performs conversion processing on the image data so that the recorded image by the recording device has a suitable color. For example, when recording an image displayed on a display device, the color reproduction ranges in the display device and the recording device do not necessarily match. For a certain color, the reproduction range of the recording device may be narrower, and for another color, the reproduction range of the recording device may be wider. Therefore, it is necessary to appropriately compress and expand the color to minimize image degradation.

[0030] In this example, those processes are performed in the RGB format. That is, considering the reproducibility of the recording device, the RGB values input to the color correction unit 305 are converted into RGB values for the recording device (hereinafter also referred to as "recording device RGB"). This conversion can also be performed by operations such as a matrix. Generally, a three-dimensional color correction table 311 is used. When the input RGB values are 8 bits (256 gradations) for each color, it is not realistic to hold all combinations from the perspective of storage capacity. Therefore, as the color correction table 311, a table decimated at a predetermined interval is used.

[0031] FIG. 5 is an explanatory diagram of an example of the color correction table 311. The color correction table 311 in this example is a table in which 256 gradations for each color are set as 17 grid points, and the corresponding RGB values for the recording device are described (17 * 17 * 17 = 4913 grid points). The values between the grid points are calculated using interpolation processing. The interpolation method can be selected and used from several methods. In this example, the tetrahedron interpolation method is used. The tetrahedron interpolation method is a linear interpolation using four lattice points with the division unit of the three-dimensional space being a tetrahedron. In this tetrahedron interpolation method, first, as shown in FIG. 6(a), the three-dimensional space is divided into tetrahedrons with grid points as the four vertices, and then it is determined to which of the divided tetrahedrons the target point p belongs. Let the four vertices of the tetrahedron be p0, p1, p2, and p3 respectively. As shown in FIG. 6(b), the tetrahedron is divided into four smaller tetrahedrons with the point p as one of the vertices and each face of the original tetrahedron as the bottom surface. When the conversion values of each point given in the color correction table are f(p0), f(p1), f(p2), and f(p3) respectively, the interpolation value f(p) can be obtained by the following formula 2. Formula 2: f(p)=Σ i=0 3 (wi×f(pi)) =w0·f(p0)+w1·f(p1)+w2·f(p2)+w3·f(p3) Here, w0, w1, w2, and w3 are the volume ratios of the small tetrahedrons at positions opposite to each vertex pi (i.e., the small tetrahedrons that do not have vertex pi). In this way, the RGB values for the recording device corresponding to the target RGB values are calculated. In that case, considering gradation, the output may be 8 bits or more. Also, since the color correction table depends on the color reproduction range of the recording device, for example, when the recording paper (recording medium) used for recording is different, it is desirable to prepare a corresponding table.

[0032] (1-2-6) Ink Color Conversion Unit The ink color conversion unit 306 converts the RGB values for the recording device converted by the color correction unit 305 into the values of the ink colors. For this conversion, a color separation table 312 in which combinations of RGB values for the recording device and the values of the ink colors are pre-associated is used. Similar to the color correction unit 305, the table at grid point 17 is also used in the ink color conversion unit 306.

[0033] FIG. 7 is an explanatory diagram of an example of the color separation table 312. In the color separation table 312 of this example, four colors, cyan (C), magenta (M), yellow (Y), and black (K), are assumed as the ink colors, and the values of the four colors corresponding to each grid point are described. These values are determined considering that the ink does not overflow on the recording surface of the recording paper (recording medium) and that the inks do not bleed when adjacent to each other. Therefore, when the recording paper (recording medium) used for recording is different, it is desirable to prepare a corresponding color separation table 312. Also, similar to the above-described color correction unit 305, the values of the ink colors corresponding to the RGB values for the recording device can be interpolated by tetrahedron interpolation processing.

[0034] (1-2-7) Density Correction Unit 307 In an inkjet recording apparatus, as the amount of ink applied to form dots on a recording paper (recording medium) increases, dot overlap increases, and it becomes difficult to increase the recording density of an image. The density correction unit 307 corrects the density in order to make such density responsiveness uniform. Such density correction makes it easier to ensure the creation accuracy of the color correction table 311 and the color separation table 312. In a recording apparatus using inks of C (cyan), M (magenta), Y (yellow), and K (black), density correction is performed for those ink colors. In this example, a one-dimensional density correction table 313 is used. As the table, a table corresponding to 8-bit (256 gradations) input for each ink color may be prepared, and in particular, a table associating the input signal value and the output signal value after correction can be used without thinning.

[0035] (1-2-8) Tone conversion unit 308 The tone conversion unit 308 converts the multi-bit data that has been converted for each ink color and subjected to density correction into the number of tones that can be recorded by the recording apparatus. In this example, it is converted into 2 tones (1 bit) of recording "1" / non-recording "0", and as a tone conversion method, an error diffusion method that eliminates the low-frequency components of the image and enables reproduction of tones suitable for visual perception is used. Also, 8-bit data from 0 to 255 is assumed as the input signal.

[0036] FIG. 8 is an explanatory diagram of an error distribution method in the error diffusion method. The signal value L of the target pixel is compared with the threshold TH. In this example, the threshold is set to 127 in order to binarize 0 to 255, and it is determined whether the target pixel is "1" (recording) or "0" (non-recording) as follows.

[0037] L>TH ······ 1 (recording) L≦TH ······ 0 (non-recording) According to this determination result, the quantization representative value V is set as follows.

[0038] 1 (recording) ······ 255 0 (Non-recording) ······ 0 By setting the quantized representative value V in this way, the generated error E (= L - V) is distributed to the surrounding pixels according to the distribution coefficient in FIG. 8.

[0039] The value La obtained by adding the distributed error Ea to the signal value L of the next target pixel is compared with the threshold value, and it is determined whether the target pixel is "1" (recording) or "0" (non-recording) as follows.

[0040] La > TH ······ 1 (recording) La ≤ TH ······ 0 (non-recording) By performing such processing for all pixels and all ink colors C, M, Y, K, recording data for each recordable 1-bit ink color can be obtained.

[0041] (1-2-9) Additional information The additional information 309 is the additional information B embedded in the image data A in the multiplexing device 102 in FIG. 1(a) or the multiplexing unit 105 in FIG. 1(b), and is, for example, text document data (text data). The text document data is, for example, numerical data in which numbers and characters are assigned numerical values using a character code that is already known, and this numerical data is transmitted to the multiplexing unit 310 as the additional information 309.

[0042] As a specific example, the text document data corresponding to the character "hello" will be described. The text document data is numerical data, so-called binary data. Binary data is information of "0" or "1", and the continuous connection of this "0" or "1" information has a specific meaning. The correspondence between binary data and characters is defined by the "character code". In the case of "Shift JIS", which is one of the character codes, "h" corresponds to the binary data "01101000". Similarly, "e" corresponds to the binary data "01100101", "l" corresponds to "01101100", and "o" corresponds to "01101111". Therefore, the character "hello" can be expressed as "0110100001100101011011000110110001101111" in binary data. Conversely, if the binary data "0110100001100101011011000110110001101111" can be obtained, the character "hello" can be obtained. The additional information 309 corresponds to the numerical data converted into such binary data.

[0043] (1-2-10) Additional Information Multiplexing Unit The multiplexing unit 310 of the additional information receives the image data converted in the resolution conversion unit 304 and the additional information 309, and embeds the additional information 309 into the image data. In this embedding process (multiplexing process), the additional information 309 (such as a text document converted into binary data of "0" and "1") is embedded into the image data so that the additional information 309 can be read from the recorded image of the image data. For example, a masking process is performed on the image data so that the binary data of "0" and "1" of the additional information 309 can be read, and the information of "0" and "1" corresponding to the binary data is embedded. In this example, by performing a masking process on the image data, different periodicities corresponding to the binary data of "0" and "1" are given to the image data in a predetermined area.

[0044] Figs. 9(a) and 9(b) are explanatory diagrams of mask data corresponding to binary data of "0" and "1", respectively. These mask data are sized to correspond to a 5 px (pixel) × 5 px (pixel) area. By combining patterns with different periodicities in Figs. 9(a) or 9(b) into the image for the image data corresponding to the 5 px (pixel) × 5 px (pixel) area, binary data of "0" and "1" are embedded. When reading the recorded image, the binary data of "0" and "1" can be read by recognizing the periodicity corresponding to the binary data of "0" and "1" through frequency analysis of the read data, etc.

[0045] In the additional information multiplexing unit 310, based on the binary data (numerical data) of the additional information 309, the additional information 309 is embedded by giving the image data a periodicity corresponding to the binary data of "0" and "1". As an example of a method for embedding the additional information 309 into the image data, a method of using the image data as single-color grayscale image data and embedding the binary data of "0" and "1" throughout the image data will be described.

[0046] The size of the multiplexed image shall be 640 px in vertical width and 480 px in horizontal width, and the mask data shall be 5 px × 5 px in size as shown in FIGS. 9(a) and 9(b). It is assumed that the binary data of "0" is represented by the mask data in FIG. 9(a), and the binary data of "1" is represented by FIG. 9(b). In the mask data of FIGS. 9(a) and 9(b), the 5×5 pixel block is divided into a black block 901, a white block 902, and a hatched block 903. The black block 901 corresponds to the numerical value "+2", the white block 902 corresponds to the numerical value "0", and the hatched block 903 corresponds to the numerical value "-1". In the following pseudo-code, when the logical variable ismaskA is true, the mask data stored in maskA is superimposed on the image. At this time, if the additional information is "0", the mask data in FIG. 9(a) is used as maskA, and if the additional information is "1", the mask data in FIG. 9(b) is used as maskA. The pseudo-code for applying the numerical values corresponding to the black, white, and hatched blocks in FIGS. 9(a) and 9(b) to the entire image data is shown below. Note that the bit length of the additional information is codelen, and the additional information is embedinfo.

[0047] Pseudo-code: ―――――――――――――――――――――――――――――― 1 int i, j, k, l; 2 int width = 640, height=480, codelen=length of additional information, m=0; 3 unsigned char *data = image data; 4 int **maskA = mask data; 5 bool *embedinfo = additional information; 6 bool isMaskA = true; 7 for(j = 0; j < height; j+=5){ 8 for(i = 0; i < width; i+=5){ 9 if(embedinfo[m] == true){ 10 maskA = pattern corresponding to "1"; 11} 12 else{ 13 maskA = the pattern corresponding to "0"; 11} 12 m = (m + 1) mod codelen; 13 for(k = 0; k < 5; k++){ 14 for(l = 0; l < 5; l++){ 15 if(isMaskA == true){ 16 data[(i + k)+(j + l)*width] += maskA[k][l]; 17} 18} 19} 20} 21} ----------------------------------------

[0048] As shown in the above pseudo - code, the entire image is divided into 5px×5px blocks, and for each block, by adding the data of maskA, patterns in FIGS. 9(a) and (b) are formed according to the value of the additional information. Here, maskA is the array data of either FIG. 9(a) or (b), and if the value of the additional information is 0, the pattern in FIG. 9(a) may be selected, and if it is 1, the pattern in FIG. 9(b) may be selected. ismaskA may be one bit of bitmap information that is true for the area where the additional information is multiplexed in the target image and false otherwise. In the above pseudo - code, ismaskA is true, and the entire image is the target of multiplexing. Also, in the above code, an example of repeatedly multiplexing the pattern corresponding to the additional information is shown, but for one bit of the additional information, a plurality of corresponding patterns may be embedded continuously two - dimensionally (for example, over an m×n block).

[0049] As a pattern corresponding to the binary data (numerical data) of the additional information 309, there may be a case where a pattern that is difficult for the human visual system to notice is formed as much as possible. As in this example, when the image data is monochromatic grayscale image data, since the pattern must be formed by the luminance component, it is visually prominent. Although it depends on the shape and frequency components of the pattern, changes in the color component tend to be less visually prominent than changes in the luminance component. For example, a color image having RGB components is converted into a color space such as YCbCr, Lab, or Yuv and separated into a luminance component and a color component. Then, by applying the mask data to the color component instead of the luminance component, a pattern that is difficult for the human visual system to notice can be formed. For example, if the additional information is multiplexed with the Y (yellow) component, it is difficult for the human visual system to notice. Also, for example, in a color image having RGB components, when red is dominant in a 5px × 5px area where the pattern is applied, it is desirable to form the pattern with the red component.

[0050] In this example, the mask data is divided into 5px × 5px blocks and added to the image data. However, the unit of the block size and the mask shape are arbitrary, and as a method of incorporating the mask data into the image data, combinations of addition, subtraction, multiplication, and division may be used. Any method may be used as long as the pattern of the mask data can be distinguished when photographing a recording of the image in which the additional information 309 is embedded. Therefore, the multiplexing unit 310 for the additional information is a processing unit for embedding the additional information 309 into the image data so that the additional information 309 can be extracted when photographing a recording of the image in which the additional information 309 is embedded.

[0051] (1-3) Multiplexing encoding process Figure 10 is a flowchart for explaining the multiplex encoding process in this example. First, the acquisition unit 301 for attached information and the restoration unit 302 for image data in FIG. 3 are used to acquire the image data A for recording (step S1001). For example, the image data A is data that has been previously captured by a mobile terminal with a camera (smartphone) and stored in the memory of the mobile terminal in the JPEG format. The acquired JPEG image data is decompressed to generate RGB image data of 3 colors and 8 bits for still images. Also, if necessary, the acquired image data is corrected or processed by the image correction unit 303 in FIG. 3.

[0052] Next, based on the arbitrarily selected paper size (size of the recording medium) and the resolution of the recording device, resolution conversion processing is performed on the acquired image data A (step S1002). For example, when the selected paper size is 2L, the resolution of the image data A is converted according to the number of pixels of the input resolution in the recording device. Specifically, when the input resolution in the recording device is 600 dpi (dots per inch), the number of pixels of the paper size 2L is set to 3000 pixels × 4000 pixels. In this case, for the image data A with 1500 pixels × 2000 pixels, resolution conversion is performed so that the number of pixels in the vertical and horizontal directions is doubled. When it is not desired to change the aspect ratio of the input image, resolution conversion is performed with the same magnification and reduction ratio in the vertical and horizontal directions.

[0053] Next, the multiplexing unit 310 for additional information is used to acquire the additional information 309 to be embedded in the image data A (step S1003). For example, text document data input via a key on a smartphone is acquired. The text document data is, for example, numerical data in which numbers and characters are assigned numerical values using the well-known character code shift JIS. The numerical data is transmitted to the multiplexing unit 310 as the additional information 309.

[0054] Next, the multiplexing unit 310 of the additional information in FIG. 3 performs multiplexing processing of the additional information for embedding the additional information 309 into the image data A (step S1004). FIG. 11 is a block diagram for explaining the firmware configuration of the multiplexing unit 310 in this example. Hereinafter, each processing unit in the multiplexing unit 310 will be described.

[0055] (1-3-1) Color Space Conversion Unit The color space conversion unit 1101 is a processing unit that converts the color space of the image data whose size has been changed in the resolution conversion unit 304 into a color space for information multiplexing. For example, as shown in Equation 3 below, the color space for performing information multiplexing is set to U of YUV, and the RGB color space of the image data is converted into the YUV color space.

[0056] Equation 3: Y = 0.299×R + 0.587×G + 0.114×B U = -0.169×R - 0.331×G + 0.500×B V = 0.500×R - 0.419×G - 0.081×B.

[0057] (1-3-2) Block Position Setting Unit In this example, the image data is divided into a plurality of block areas, and the density of each pixel is modulated in block units to form a pattern corresponding to the mask data in FIGS. 9(a) and 9(b), thereby embedding the additional information 309. The block position setting unit 1102 acquires the image data after color space conversion, and sets the block position coordinates for a specified color plane image according to the size of a specified one block. For example, the size of the U color plane image of YUV is set to 640 px in vertical width and 480 px in horizontal width, and the block size is set to 5 px in vertical width and 5 px in horizontal width. In this case, the number of vertical blocks is 128 (=640÷5), the number of horizontal blocks is 96 (=480÷5), and the total number of blocks is 12288 (=128×96). For example, the upper left coordinates of each block can be defined as the block position and set.

[0058] (1-3-3) Quantization Unit The digitization unit 1103 converts the received additional information 309 into digitized data. For example, assume the additional information 309 is a Shift-JIS character string. In this case, a conversion map in which characters and numerical values are associated according to the Shift-JIS format is held in advance, and the character string is converted into a numerical sequence using this conversion map. For example, in the case of the character string "hello", the resulting numerical sequence is "0110100001100101011011000110110001101111".

[0059] (1-3-4) Pattern Selection Unit The pattern selection unit 1104 has mask patterns registered for performing density modulation of each pixel in block units, and selects a mask pattern to be applied to the additional information 309 digitized by the digitization unit 1103.

[0060] Figures 12(a) and (b) are diagrams obtained by digitizing the patterns of Figures 9(a) and (b) with different frequency characteristics. As described above, the patterns of Figures 9(a) and 12(a) correspond to "0" in the binary data of the additional information 309, and the patterns of Figures 9(b) and 12(b) correspond to "1" in the binary data of the additional information 309.

[0061] (1-3-5) Information Multiplexing Unit The information multiplexing unit 1105 acquires the image data that has undergone color space conversion in the color space conversion unit 1101, the positions of the respective blocks set in the block position setting unit 1102, and the mask pattern selected in the pattern selection unit 1104. The information multiplexing unit 1105 generates image data obtained by applying the mask pattern to the image data from these acquired pieces of information. Application of the pattern may be, for example, addition on a pixel-by-pixel basis. The addition may be the addition of the mask pattern itself, or may be addition with weighting.

[0062] As described above, when the image size is set to 640 px in vertical width and 480 px in horizontal width, and the size of one block is 5 px in vertical width and 5 px in horizontal width, the total number of blocks is 12,288. When photographing the recorded image of the recording medium C, it is not always possible to photograph the entire image. Therefore, the same additional information is embedded in multiple locations in the recorded image so that the additional information can be extracted even by photographing only a part of the recorded image of the recording medium C. For example, when 96 blocks are regarded as one piece of additional information, the same additional information is divided and embedded in 128 (= 12,288 ÷ 96) areas with respect to the total number of blocks 12,288. Therefore, the image data is divided into 128 areas, and for one of those areas, additional information of 96 blocks with a vertical width of 5 px and a horizontal width of 5 px per block is embedded. By treating 96 blocks as one piece of additional information, 96-bit additional information can be set. However, in order to know the start position of the 96 bits, 8 bits of "11111111" that are not expressed as characters in Shift-JIS are included at the beginning of the additional information. The block immediately after detecting "11111111" corresponds to the leading bit of the substantial additional information to be multiplexed. Therefore, 88 (= 96 - 8) bits of data become the additional information.

[0063] The data that fits within 96 bits is a numerical sequence of "0" and "1" of the additional information digitized in the digitizing unit 1103. A numerical value is defined for each block of 5 px × 5 px, and a mask pattern corresponding to that numerical value is selected. The mask pattern corresponding to the additional information is embedded in the 5 px × 5 px block in the image data. For example, the image data is set as the U color plane of YUV, and it is processed for each block (5 px × 5 px), and the values of the mask patterns in FIGS. 12(a) and (b) are applied to the values of the U color plane of the YUV. For example, as follows, the value of the U color plane (U value) of the YUV is subjected to addition and subtraction processing according to the numerical values of those mask patterns. In the formula, the reference value is a weight that multiplies the value of the mask pattern.

[0064] Formula 4: Applied U value = U value of YUV + reference value × numerical value of mask pattern For example, when the U value of one pixel within one block is "20", the reference value is "10", and the numerical value of the mask pattern to be applied is "0", the U value is processed as shown in Equation 5 below.

[0065] Equation 5: U value after application = 20 + 10 × 0 = 20 Also, when the U value of one pixel within one block is "30" and the numerical value of the mask pattern to be applied is "2", the U value is processed as shown in Equation 6 below.

[0066] Equation 6: U value after application = 30 + 10 × 2 = 50.

[0067] In this example, multiplexing is achieved by adding the value obtained by multiplying the numerical value of the mask pattern to be applied for each pixel by the reference value. The method of applying the mask pattern only needs to be able to embed the mask pattern on the U color plane and is not limited to the method of this example. For example, the numerical value of the mask pattern may be multiplied by the U value of YUV. Also, the size of the block is not necessarily 5 pixels × 5 pixels and may be larger or smaller. Also, it does not have to be a square block, and any shape that can be arranged continuously is acceptable. However, for the multiplexing pattern, the one with a higher spatial frequency is less visually conspicuous and more mask patterns can be multiplexed, so a size of about 5 pixels × 5 pixels is preferable.

[0068] Such multiplexing encoding processing is executed by the multiplexing device 102 for additional information in FIG. 1(a) or the multiplexing unit 105 for additional information in FIG. 1(b). The multiplexing encoding processing may not be included in the printer 103 or may be included in the printer 103. The image data after the multiplexing encoding processing generated in the multiplexing device 102 or the multiplexing unit 105 is transmitted to the printer 103 or the recording unit 106.

[0069] (1-4) Recording processing of image data FIG. 13 is a flowchart for explaining the recording processing of the image data after the multiplexing encoding processing.

[0070] First, in the additional information multiplexing unit 310 of FIG. 3, image data (multiplexed image data) in which additional information is embedded is acquired (step S1301). Next, color correction is appropriately performed on the multiplexed image data by the color correction unit 305 of FIG. 3 (step S1302). Next, in the ink color conversion unit 306, density correction unit 307, and gradation conversion unit 308 of FIG. 3, the color-corrected image data is converted into ink color values, then the density is corrected, and the density-corrected image data is converted into the number of gradations, thereby generating recording data (step S1303). The recording data is transmitted to the recording engine of FIG. 3 (for example, the printer 103 of FIG. 1), and the recording engine generates a recording object C by applying inks of respective colors to a recording medium based on the recording data.

[0071] (1-5) Basic firmware for multiplexing decoding processing Next, a configuration and procedure for multiplexing information extraction that decodes an image in which additional information is multiplexed according to the above description and extracts the additional information will be described. FIG. 14(a) is an explanatory diagram of the basic firmware configuration for multiplexing decoding processing in this example, and extracts additional information embedded in the recorded image of the recording object C.

[0072] The imaging sensor 202 (see FIG. 2) in this example includes an imaging unit 1401 and a color adjustment unit 1402. Also, the additional information separation device 203 (see FIG. 2) in this example includes a multiplexing position detection unit 1403, an additional information separation unit 1404, and an analysis unit 1405 for extracted data. Also, in this example, digitized additional information data such as text document data, audio data, and video data is embedded in the recorded image of the recording object C. In the following, the additional information will be described as being repeatedly embedded in different regions for each predetermined region throughout the recorded image of the recording object C.

[0073] (1-5-1) Imaging unit The imaging unit 1401 captures a recorded image of the recording object C by an imaging element in the imaging sensor 202 and converts it into image data. The image formed on the imaging sensor 202 by an engineering system such as a lens is read by the imaging unit 1401.

[0074] FIG. 14(b) is an explanatory diagram of the case where a recorded image of the recording object C is photographed by the mobile terminal 201 with a camera. An image subjected to multiplex encoding processing is recorded in the recording area 1407 of the recording medium 1406 corresponding to the recording object C. The area 1409 is an area photographed by the device 1408 corresponding to the mobile terminal 201 with a camera in FIG. 2. The imaging unit 1401 built in the device 1408 photographs an image of the area 1409 in the recording area 1407 of the recording medium 1406. As the imaging element in the imaging unit 1401, a CCD or a CMOS can be used. The CCD and the CMOS sense light by a photodiode (light receiving element) and change the light into a voltage. At this time, the light can be converted into color data by a color filter such as RGB or CMY arranged for each imaging element. In addition, there is a range of brightness of light that can be reproduced by the imaging element, and this is called the dynamic range. The dynamic range is defined by the ratio of the minimum luminance and the maximum luminance that can distinguish gradations. The detection signal of the photodiode is sent to the color tone adjustment unit 1402.

[0075] (1-5-2) Color Tone Adjustment Unit The color tone adjustment unit 1402 converts the output data of the photodiode in the imaging unit 1401 into image data with 1 pixel being RGB 8-bit data. Before converting it into image data, color interpolation processing such as RGB is performed on the output data of the photodiode according to the light source at the time of photographing. The interpolation processing is to adjust so that a white subject can be photographed white when photographing using a digital camera and a mobile terminal 201 with a camera. Since the imaging unit 1401 is irradiated with light from a light source such as the sun or a light and the light reflected from the subject is detected by the photodiode, the color of the image varies depending on the light source. Therefore, the color tone adjustment unit 1402 performs interpolation processing according to the light source.

[0076] As a general interpolation method, there is a method that uses the unit of Kelvin (K), which quantifies the color of light expressed by the color temperature indicating the color tone of the light source. Generally, daylight in the daytime is 5500K, and an incandescent bulb is 3000K. When the color temperature is high, it appears blue, and when it is low, it appears red. Therefore, the color of the captured image varies depending on the light source. Generally, digital cameras and mobile terminals 201 with cameras are equipped with a so-called auto white balance adjustment function that automatically adjusts the color temperature detected by a sensor during shooting so that a white subject can be photographed white. Also, the white balance can be manually adjusted according to the light source such as sunlight or an incandescent bulb.

[0077] The color tone adjustment unit 1402 generates image data with the white balance adjusted for the output data of the photodiode. The image data (captured image) is sent to the additional information separation device 203.

[0078] (1-5-3) Detection unit for multiplexing position The multiplexing position detection unit 1403 inputs the image data whose color tone has been adjusted by the color tone adjustment unit 1402, and detects the position (multiplexing position) where the additional information is embedded by determining the frequency characteristics of the image data.

[0079] FIG. 15(a) is an explanatory diagram of the difference in frequency characteristics in the two-dimensional frequency domain. The horizontal axis represents the horizontal frequency, the vertical axis represents the vertical frequency, the central origin indicates the DC component, and the high-frequency region is formed as the distance from the origin increases. In this example, the frequency characteristics change due to the multiplexing process.

[0080] For example, as described above, due to the change in the frequency characteristics when the mask pattern of FIG. 9(a) is applied, a large power spectrum occurs on the straight line 1501 in FIG. 15(a). Also, due to the change in the frequency characteristics when the mask pattern of FIG. 9(b) is applied, a large power spectrum occurs on the straight line 1502 in FIG. 15(a). When separating the additional information, in order to determine the multiplexed signal, a frequency vector at which such a large power spectrum occurs is detected. Therefore, it is necessary to individually emphasize and extract each frequency vector.

[0081] For this purpose, it is effective to use an HPF (high-pass filter) having frequency characteristics similar to the mask patterns of FIGS. 12(a) and (b). The spatial filter corresponding to the mask pattern of FIG. 12(a) can emphasize the frequency vector on the straight line 1501 in FIG. 15(a). The spatial filter corresponding to the mask pattern of FIG. 12(b) can emphasize the frequency vector on the straight line 1502 in FIG. 15(a). For example, assume that due to the quantization condition for applying the mask pattern of FIG. 12(b), a large power spectrum occurs on the frequency vector of the straight line 1501 in FIG. 15(a). In this case, although the change amount of the power spectrum is amplified by the spatial filter corresponding to the mask pattern of FIG. 12(a), it is hardly amplified by the spatial filter corresponding to the mask pattern of FIG. 12(b). That is, when filtering using a plurality of spatial filters in parallel, the power spectrum is amplified only by the spatial filter whose frequency vector matches, and is hardly amplified by other spatial filters. Therefore, by specifying the spatial filter that amplified the power spectrum, it is possible to determine on which frequency vector a large power spectrum has occurred. By determining the frequency characteristics in this way, it is possible to extract the additional information. At that time, if the extraction position of the additional information is shifted, it becomes difficult to extract it correctly.

[0082] FIG. 15(b) is an explanatory diagram of a recording area on the recording object C. The recording medium 1503 as the recording object C includes an area 1504 multiplexed in a plurality of block units, and additional information is embedded in the area 1504 of the block unit so as to have specific frequency characteristics.

[0083] FIGS. 16(a) and (b) are explanatory diagrams of the relationship between the multiplexed blocks and the frequency characteristic determination area. The recording medium 1503 is multiplexed in four blocks. In FIG. 16(a), the determination area 1602 for determining the frequency characteristics of the block unit is shifted from the position of the block. In FIG. 16(b), the determination area 1603 for determining the frequency characteristics of the block unit coincides with the position of the block. In the case of FIG. 16(b), the frequency characteristics can be correctly determined in the determination area 1603. On the other hand, in the case of FIG. 16(a), in the determination area 1602, the power spectrum of a specific frequency vector decreases, and it is difficult to correctly determine the frequency characteristics.

[0084] The multiplexing position detection unit 1403 uses a spatial filter to determine the frequency characteristics of the multiplexed block units. At that time, it is necessary to specify the position of the multiplexed block. The position of the multiplexed block can be specified based on the intensity of the power spectrum of a specific frequency vector. Therefore, in the multiplexing position detection unit 1403, the frequency characteristics are detected while shifting the determination area of the frequency characteristics of the block unit with respect to the captured image, and the position of the multiplexed block is specified by determining the frequency characteristics. The amount of shift of the determination area may be one pixel unit in each of the vertical and horizontal directions. For example, if one block is 5 pixels × 5 pixels, it may be shifted by up to 4 pixels from the original position in each of the vertical and horizontal directions. That is, the position where the maximum spatial filter is applied is shifted 5 × 5 = 25 times to detect the frequency characteristics, and the position where the intensity of the power spectrum is the strongest may be determined as the position of the block where the pattern is multiplexed.

[0085] (1-5-4) Additional Information Separation Unit The frequency characteristics in block units are determined based on the positions of the blocks detected by the multiplexing position detector 1403, and the additional information separator 1404 extracts the multiplexed additional information based on the determination result of the frequency characteristics in block units.

[0086] As shown in Fig. 15(b), when the total number of multiplexed blocks is 96 blocks (8 blocks horizontally × 12 blocks vertically), for each of these blocks, "0" and "1" of the additional information are embedded by the multiplexing encoding process. Based on the frequency vector for each block, the additional information embedded in each block is determined. That is, for the blocks where the frequency vector of the straight line 1501 in Fig. 15(a) exceeds a predetermined threshold, it is determined that the additional information embedded in that block is "0". Also, for the blocks where the frequency vector of the straight line 1502 in Fig. 15(a) exceeds a predetermined threshold, it is determined that the additional information embedded in that block is "1".

[0087] By shifting the frequency characteristic determination region in block units with reference to the positions of the blocks detected by the detector 1403, the frequency characteristics of the total 96 blocks in Fig. 15(b) are determined, and the additional information embedded in each block can be extracted. In this case, since 1 bit of additional information of "0" or "1" can be extracted for each block, a total of 96 bits of data can be extracted from the 96 blocks. In this way, by determining the frequency characteristics while shifting the frequency characteristic determination region, the multiplexed additional information can be extracted from a plurality of blocks.

[0088] (1-5-5) Analysis Unit for Extracted Data The extracted data analysis unit 1405 analyzes the numerical sequence, which is the extraction result separated as additional information by the additional information separator 1404, and converts the numerical sequence into the original format of the additional information before embedding.

[0089] For example, in advance, the additional information to be multiplexed is used as text document data, and its character code is digitized to "Shift JIS". In the 1-byte code (half-width characters) of Shift JIS, conversion (digitization) corresponding to numerical values and characters can be performed by combining the upper 4 bits and the lower 4 bits. For example, when the upper 4 bits are "0100" and the lower 4 bits are "0001", those numerical sequences are determined to be the character "A". In this way, by holding the conversion map in advance and associating the conversion map with the numerical sequence, the numerical sequence can be converted into a character. For example, the numerical sequence separated as additional information can be temporarily held in the RAM 206 of FIG. 2, and the "Shift JIS" conversion map can be held in advance in the secondary storage device 207 so as to be referable.

[0090] Assume a case where the numerical sequence "0110100001100101011011000110110001101111" is separated as additional information by the additional information separation unit 1404. This numerical sequence is converted as follows by the conversion map. The upper 4 bits "0110" and the lower 4 bits "1000" are converted into the character "h". The upper 4 bits "0110" and the lower 4 bits "0101" are converted into the character "e". The upper 4 bits "0110" and the lower 4 bits "1100" are converted into the character "l". The upper 4 bits "0110" and the lower 4 bits "1100" are converted into the character "l". The upper 4 bits "0110" and the lower 4 bits "1111" are converted into the character "o". Therefore, it will be converted into the character string "hello".

[0091] The character string extracted as additional information in this way can be displayed, for example, on the display 208 in FIG. 2. Also, when the extracted character string is a URL (Uniform Resource Locator), it is possible to connect to the network via the wireless LAN 210 in FIG. 2 and use a browser to display the screen of the URL destination on the display 208. Further, when the URL is a video site, it is possible to display a video on the display 208 and play audio through the speaker 211. Furthermore, the character string can be interpreted as a numerical value and used as a numerical value with a predetermined meaning.

[0092] (1-6) Multiplex decoding process FIG. 17 is a flowchart for explaining the multiplex decoding process in this example.

[0093] First, the imaging unit 1401 of the imaging sensor 202 in FIG. 14(a) of the camera-equipped mobile terminal 201 (see FIG. 2) captures a recorded image of the recording object C (step S1701). The captured light is converted into color data and then transmitted to the color adjustment unit 1402 in FIG. 14(a). The color adjustment unit 1402 generates image data by performing white balance adjustment on the output data of the photodiode (step S1702). The generated image data is transmitted to the additional information separation device 203 in FIGS. 2 and 8, or stored in the secondary storage device 207 in FIG. 2. The detection unit 1403 at the multiplexing position in FIG. 14(a) detects the multiplexed position based on the image data with adjusted white balance as described above (step S1703). In step S1704, it is determined by the detection unit 1403 at the multiplexing position whether the position of the multiplexed block has been detected. If it has been detected, the process proceeds to the next additional information separation process (step S1705), and if it has not been detected, the process returns to the previous step S1701.

[0094] In step S1705, the additional information separation unit 1404 in FIG. 14(a) determines the frequency characteristics for each block based on the image data generated by the color adjustment unit 1402 and the position of the block detected by the multiplexing position detection unit 1403. Then, based on the determination result, the multiplexed additional information is extracted as numerical data. The extracted numerical data is transmitted to the extraction data analysis unit 1405 in FIG. 14(a), or temporarily stored in the RAM 206 in FIG. 2 and then notified to the extraction data analysis unit 1405 in FIG. 14(a).

[0095] Next, as described above, the extraction data analysis unit 1405 in FIG. 14(a) analyzes the numerical data extracted as additional information and converts the numerical data into additional information such as characters (step S1706). In the next step S1707, it is determined whether or not the conversion of all the extracted numerical data into additional information by such an analysis unit 1405 has been completed. If it is completed, the multiplexing decoding process in FIG. 18 is terminated. If it is not completed, the process returns to the previous step S1701. The additional information extracted as characters or the like can be displayed on the display 208 in FIG. 2, etc., and the network can be accessed based on the additional information. Or it can be used for other purposes as well.

[0096] When the additional information cannot be completely extracted from the recording medium C, for example, it is conceivable that the photographed area of the recording medium C contains only a part of the area where the additional information is embedded. In this case, since only a part of the additional information can be extracted, it is necessary to photograph the recording medium C again. The determination of whether or not the additional information has been extracted can be made, for example, by including in advance a value indicating the data amount of the additional information in the additional information itself, and determining the data amount of the additional information from that value. In order to distinguish whether the data as additional information relates to the data amount or character data, for example, a combination of numerical sequences is determined in advance, and several bits immediately following the numerical sequence are used as data related to the data amount. In this example, as described above, the binary sequence "11111111" is defined as data for distinction.

[0097] Also, when only a part of the additional information can be extracted, for example, only the extracted content may be stored in the secondary storage device 207 of FIG. 2, and the additional information extracted by subsequent processing may be combined with a part of the stored additional information. In this way, the additional information may be extracted in multiple times. Also, the additional information extracted in multiple times may be sequentially displayed on the display 208 of FIG. 2 or the like.

[0098] (1-7) Recording process using metallic ink In the first embodiment of the present invention, the image data after the multiplex encoding process is recorded by a recording device equipped with metallic ink. Here, the recording process using metallic ink will be described. The recording process using metallic ink will also be referred to as "metallic printing".

[0099] (1-7-1) Recording by an inkjet printer FIG. 18 is a diagram schematically showing an inkjet printer according to the first embodiment of the present invention. As shown in FIG. 18, the printer 1800 includes a recording head 1801 on a frame that forms the structural material of the printer. The recording head 1801 is equipped with a plurality of nozzles for ejecting ink. In this example, a nozzle 1802C for cyan ink, a nozzle 1802M for magenta ink, a nozzle 1802Y for yellow ink, and a nozzle 1802Me for metallic ink are mounted.

[0100] The printer 1800 is of a so-called serial recording type in which nozzles are arranged in a direction (Y direction) perpendicular to the width direction of the recording paper 1805, and the recording head 1801 is scanned in the X direction along the guide 1803 for recording. The resolution of the nozzle arrangement of each ink color nozzle row is 1200 dpi in this example.

[0101] The recording paper 1805 as a recording medium is conveyed in the Y direction in the figure by the rotation of the conveyance roller 1804 (and other rollers not shown) by the driving force of a motor (not shown). After the recording paper 1805 is fed, ink is ejected from a plurality (predetermined number) of nozzles of the recording head 1801 according to the recording data, so that an image for one scanning width corresponding to the nozzle array of the recording head is recorded. After the recording, the recording paper 1805 is conveyed in the direction of the arrow in the figure by the width corresponding to the nozzle array again, and an image for one scanning width is recorded again. Note that the feed amount of the storage paper can be made smaller than the recording width of the recording head, and the recording head 1801 can perform multiple scans to form an image.

[0102] By repeating such conveyance of the recording paper and the ink ejection operation from each recording head with respect to the recording paper, for example, an image for one page can be recorded. Note that the recording apparatus to which the present invention is applicable is not limited to the serial recording type apparatus described above. For example, the present invention can also be applied to a so-called full line type recording apparatus in which recording heads are arranged in the conveyance direction of the recording paper and recording is performed in synchronization with the conveyance of the recording paper.

[0103] (1-7-2) Hardware Configuration of Inkjet Printer FIG. 19 is a block diagram showing the hardware configuration of an inkjet printer according to the first embodiment of the present invention.

[0104] The printer 1800 mainly consists of the following elements. The CPU 1901 executes processing according to programs held in the ROM 1903 and the RAM 1902. The RAM 1902 is a volatile storage that temporarily holds programs and data. Also, the ROM 1903 is a non-volatile storage that can hold table data and programs created in each process. The data transfer I / F 1904 controls the transmission and reception of data between a personal computer (PC) (not shown), a mobile terminal such as a smartphone or a tablet as shown in FIG. 2, and a server (not shown). As the connection method for this data transmission and reception, USB, IEEE 1394, LAN, etc. can be used according to the connection partner. The head controller 1905 supplies recording data to the recording head 1801 shown in FIG. 18 and controls the ejection operation of the recording head. Specifically, the head controller 1905 can be configured to read control parameters and recording data from a predetermined address in the RAM 1902. Then, when the CPU 1901 writes the control parameters and the recording data to the above-mentioned predetermined address in the RAM 1902, the process is started by the head controller 1905, and ink is ejected from the recording head 1801. The image processing accelerator 1906 is configured by hardware and executes image processing faster than the CPU 1901. Specifically, the image processing accelerator 1906 can be configured to read parameters and data necessary for image processing from a predetermined address in the RAM 1902. Then, when the CPU 1901 writes the above parameters and data to the above-mentioned predetermined address in the RAM 1902, the image processing accelerator 1906 is started, and predetermined image processing is performed. Note that the image processing accelerator 1906 is not necessarily a necessary element, and of course, the CPU 1901 may execute only the above-mentioned table parameter creation process and image processing according to the specifications of the printer and the like.

[0105] (1-7-3) Regarding the image data of the metallic print There are two types of image data input for the recording process using metallic ink. One is, for example, image data for three-color color ink (referred to as color image data), and the other is image data for metallic ink (referred to as metallic image data). Color image data is an image in which each pixel has values of a plurality of color components for expressing a standardized color space such as sRGB. Metallic image data is a grayscale image having the same size as the color image. Fig. 20 is an example of the input image data. In the image data of Fig. 20, both color ink and metallic ink are printed in the area where the color image and the metallic image overlap (the area of the central circle). Only color ink is printed in the other areas. In this case, the former is classified as the metallic color area, and the latter is classified as the color area. Hereinafter, it is assumed that the color image data is an image in which each pixel has values of 3 channels, specifically, 8-bit values for each of the RGB channels, and the metallic image data is an image in which each pixel has an 8-bit value.

[0106] (1-7-4) Recording operation of metallic printing Fig. 21 is a diagram schematically showing the arrangement of the nozzle rows of each ink in the recording head 1801 and the state in which the recording operation is being performed by the nozzles that actually eject the ink. When forming an image using both metallic ink and color ink within the same area on the paper surface, it is necessary to pay attention to the timing of ejecting each ink. Specifically, the metallic ink must be ejected first, and then, after providing a time difference of a certain value or more, the color ink must be ejected. Here, for simplicity of explanation, an example will be given in which the number of nozzles in each nozzle row 1802 is 16 and image formation is performed in one pass (one scan of the forward or return path).

[0107] First, when discharging the metallic ink, among the nozzles included in the nozzle row 1802Me, the ones that actually discharge the ink are the 4 nozzles at the tip shown in black in the figure. Next, when discharging each of the C, M, and Y color inks, among the nozzles included in each of the nozzle rows 1802C, 1802M, and 1802Y, the ones that actually discharge the ink are the 4 nozzles at the rear end shown by hatching. Here, in each nozzle row, the nozzles existing on the tip side from the center are called upstream nozzles in the conveyance direction (simply also called upstream nozzles). On the other hand, the nozzles existing on the rear end side from the center are called downstream nozzles in the conveyance direction (simply also called downstream nozzles). In this embodiment, by setting the conveyance amount of the recording medium to be equivalent to 4 nozzles, it is possible to discharge the color ink after discharging the metallic ink first.

[0108] In FIG. 21, the blackened portion on the recording medium 1805 represents the area recorded with the metallic ink. On the other hand, the portion shown by hatching represents the area recorded with the color ink after being recorded with the metallic ink. By discharging the metallic ink from the tip nozzles and discharging the color ink from the rear end nozzles, it can be seen that two types of inks are applied at different timings on the same area of the recording medium 1805.

[0109] Also, in this embodiment, as shown in FIG. 21, eight nozzles existing between the nozzles (the four nozzles at the tip) that actually eject metallic ink and the nozzles (the four nozzles at the rear end) that actually eject color ink are controlled not to eject ink. In this way, the area where neither metallic ink nor color ink is ejected is called the "blank nozzle area". By providing the blank nozzle area, it is possible to apply metallic ink and color ink to the paper surface with a sufficient time difference. In the case shown in FIG. 21, a time difference corresponding to at least three main scans is provided between the application of metallic ink and the application of color ink. Thereby, sufficient time can be ensured for the metallic ink applied on the recording medium to dry. As a result, it is possible to surely form a metallic ink layer and a color ink layer on the recording medium, and realize a metallic color expression that achieves both high glossiness and chroma.

[0110] Note that the number of nozzles used and the conveyance amount are not limited to those described above. For example, the nozzles that actually eject metallic ink in the nozzle row of metallic ink may be set as the six nozzles at the tip, and the nozzles that actually eject each color ink in the nozzle row of each color ink may be set as the six nozzles at the rear end. Alternatively, the nozzles that actually eject metallic ink in the nozzle row of metallic ink may be set as the three nozzles at the tip, and the nozzles that actually eject each color ink in the nozzle row of each color ink may be set as the six nozzles at the rear end. At this time, by setting the conveyance amount of the recording medium to be equivalent to that for three nozzles, the number of scans of the nozzles that can eject color ink can be increased, and more color ink can be ejected in the same area.

[0111] (1-7-5) Regarding the recorded matter of metallic printing Figs. 22(a), 22(b), and 22(c) are diagrams for explaining a recording medium output when the recording operation described in Fig. 21 is performed based on the image data shown in Fig. 20. Fig. 22(a) shows the recording medium output by metallic printing. In region 2201, imaging is performed only with color ink. In region 2202, imaging is performed with both color ink and metallic ink. Fig. 22(b) is a cross-sectional view of region 2202. Fig. 22(c) is a cross-sectional view of region 2201. In Fig. 22(b), a metallic ink layer 2204 is first formed on a recording medium 2203, and a color ink layer 2205 is formed thereon. In Fig. 22(c), a color ink layer 2205 is formed on the recording medium 2203 without a metallic ink layer 2204.

[0112] (2) Characteristic Configuration In the present embodiment, a characteristic configuration is added to the basic configuration of the multiplex decoding processing unit described above.

[0113] (2-1) Confirmation of the Problem of the Present Invention Here, the problem of the present invention will be confirmed. In the present invention, additional information embedded in a recording medium by metallic printing is extracted. Here, a case where additional information is embedded in the color image shown in Fig. 20 will be described. When the recording process of metallic printing is performed with additional information embedded in the color image of Fig. 20, the additional information is embedded with color ink over the entire surface of the recording medium of metallic printing shown in Fig. 22(a). However, in region 2201, a pattern for directly embedding the additional information with color ink on the paper surface is formed, whereas in region 2202, a pattern for embedding the additional information with color ink is formed on the metallic ink layer formed on the paper surface.

[0114] Figures 23(a) and (b) illustrate the multiplex decoding process of extracting additional information embedded in a recorded object 2304 by metallic printing using a mobile terminal 201 with a camera. In Fig. 23(a), the mobile terminal 201 with a camera has an illumination means (light) 212 and a lens unit 2303 on its outer side. The light 212 irradiates light toward the recorded object 2304 that is the subject, and the reflected light is received by the lens unit 2303. An imaging sensor 202 (see Fig. 2) is included in the lens unit 2303, and the light condensed by the lens is imaged by the imaging sensor 202. In Fig. 23(b), the area 2305 is the area photographed by the mobile terminal 201 with a camera. In the present embodiment, it is assumed that the photographed area 2305 includes both an area 2306 formed by only color ink and an area 2307 formed by color ink and metallic ink. The area 2306 may be called a non-metallic area or a color area, and the area 2307 may be called a metallic area.

[0115] Figure 24 is a graph showing the relationship between the metallic ink density (the amount of metallic ink per unit area on the paper surface) and the glossiness of the recorded object. The graph 2401 shows the case where the recording medium is glossy paper. The glossiness of the metallic ink increases dramatically according to the amount of ink printed on the paper surface. Depending on the type of metallic ink and the amount of ink printed, the glossiness when effective glossiness is obtained by the metallic ink is several times to several tens of times the value compared to the glossiness of the paper white (the state where no ink is printed on the paper surface). On the other hand, regarding the color ink, even if the amount of ink printed on the paper surface is increased, the glossiness of the recorded object hardly changes from the glossiness of the paper white (not shown). The graph 2402 in Fig. 24 will be described in the second embodiment.

[0116] Therefore, the glossiness of the area 2306 formed with only color ink and the glossiness of the area 2307 formed with color ink and metallic ink are significantly different. As a result, when taking an image for extracting additional information, when the recording material 2304 is irradiated with light by the light 212, the reflection intensities are significantly different between the area 2306 and the area 2307. Specifically, the reflection intensity of the color area 2306 is smaller than the reflection intensity of the metallic area 2307. As a result, in an image taken by irradiating light with a light beam value (or light amount) appropriate for extracting the additional information embedded in the metallic area 2307, the signal intensity of the additional information embedded in the area 2306 is small and its extraction is difficult.

[0117] (2-2) Configuration and processing flow for solving the problems of the present invention FIG. 25 shows a firmware configuration of multiplexed decoding processing for solving the problems of the present invention. It is assumed that hardware necessary for control by firmware, such as hardware for illumination light control, is provided. The difference from FIG. 14(a) is the light 212 that irradiates the recording material C with light. The light 212 is composed of an illumination unit 2501 and an illumination light control unit 2502. The illumination light control unit 2502 controls the light beam value (or light amount) of the light irradiated on the recording material during imaging so that the additional information embedded in each of the area 2306 and the area 2307 can be extracted.

[0118] FIG. 26(a) is a flowchart of multiplexed decoding processing for solving the problems of the present invention. The procedure in FIG. 26(a) is executed by, for example, the CPU 204 of the camera-equipped mobile terminal 201 or the CPU of an information processing apparatus to which a scanner is connected. Alternatively, it may be a copying machine equipped with a scanner. This flow, for example, when the application program of the procedure in FIG. 26(a) is started and an execution start is instructed with the camera facing the target image, is executed from S2601. Alternatively, the procedure in FIG. 26(a) may be executed as part of an application that uses additional information, and the extracted additional information may be used by that application. The same applies to the flowcharts in FIGS. 26(b), 27, and 28.

[0119] In S2601, image capturing is performed in a shooting mode for extracting additional information embedded in the color area 2306. In S2602, the additional information embedded in the area 2306 is extracted from the image captured in S2601. In S2603, it is determined whether the extraction process of S2602 is completed. If not, the process returns to the process of S2601. If completed, the process proceeds to the process of S2604.

[0120] In S2604, image capturing is performed in a shooting mode for extracting additional information embedded in the area 2307. In S2605, the additional information embedded in the metallic area 2307 is extracted from the image captured in S2604. In S2606, it is determined whether the extraction process of S2605 is completed. If not, the process returns to the process of S2604. If completed, the process ends.

[0121] Here, the illumination light control unit 2502 controls the light of the illumination unit 2501 so that the luminous flux value of the light irradiated in the shooting mode of S2601 is larger than the luminous flux value of the light irradiated in the shooting mode of S2604. Alternatively, the light of the illumination unit 2501 is controlled so that the luminous flux value of the light irradiated in the shooting mode of S2604 is smaller than the luminous flux value of the light irradiated in the shooting mode of S2601. This is to raise the signal intensity of the additional information embedded in the color area 2306 to an extractable magnitude in the captured image captured in S2601. Also, if the signal intensity of the additional information embedded in the metallic area 2307 remains weak in the image captured in S2601, this may become noise in the extraction process of S2602. That is, the additional information in the metallic area 2307 is detected in an incomplete state, preventing correct extraction processing. As a result, the reliability of the additional information extracted in S2602 is lowered.

[0122] In order to reduce this problem, it is preferable to control the luminous flux value of the light irradiated in the shooting mode of S2601 so as to cause the pixels included in the metallic region 2307 in the captured image to overflow as much as possible. Overflow (saturation) refers to a state that deviates from the dynamic range of the image sensor in the imaging unit in the bright direction. It is also called blooming. As a result, in the extraction process of S2602, it becomes difficult to detect the additional information of the metallic region 2307 in the image captured in S2601.

[0123] From the above, by irradiating with the amount of light corresponding to the presence or absence of the image formed by the metallic ink, it becomes possible to extract the additional information embedded in each of the color region and the metallic region with different reflection intensities. Note that the amount of light of the light source 212 may be adjusted and determined in advance so that the color region 2306 and the metallic area 2307 can be captured respectively. Further, the amount of light for capturing the color region 2306 may be determined so that the luminance of the metallic region 2307 is saturated. Also, the amount of light for capturing the metallic region 2307 may be determined so that the luminance of the metallic region 2307 is not saturated. Here, the adjustment of the amount of light of the light source 212 is performed by the illumination light control unit 2502. As described in the explanation of FIG. 2, for example, this is performed by controlling the applied voltage of the LED by PWM control or the like. For example, by setting the parameter for determining the duty ratio of PWM by the CPU 204 to the amount of light determined for each of the metallic region and the color region, the adjustment of the light source 212 becomes possible.

[0124] Alternatively, the luminous flux of the light 212 may be determined based on the average luminance value of the captured image. For example, before S2601 in FIG. 26(a), the luminous flux of the light 212 is set to a predetermined value, and an image is captured in that state. The light 212 is adjusted so that the average of the luminance values of the image becomes the intermediate value of the dynamic range, and the execution is started from S2601 in FIG. 26(a). At this time, the image is divided into several regions, a region having an average luminance exceeding the average luminance of the entire image is determined, and it is estimated as a metallic region and excluded, and the light 212 may be adjusted so that the excluded average luminance becomes the intermediate value of the dynamic range. In S2605, the image may be captured by reducing the luminous flux value by a value determined in advance by measurement or the like with reference to the luminous flux in S2601.

[0125] In this embodiment, the case where additional information is embedded in the color image of the metallic print has been described, but additional information may be embedded in the metallic image. However, when additional information is embedded in the metallic image, since the additional information is embedded depending on the pattern of the density of the metallic ink, it is necessary to analyze the luminance component of the captured image during the extraction process. Also, additional information may be embedded in both the color image and the metallic image. In this case, in order to extract the additional information embedded in the color image, the color difference component of the captured image may be analyzed, and in order to extract the additional information embedded in the metallic image, the luminance component of the captured image may be analyzed. Also, when additional information is embedded in the same region of the color image and the metallic image, if the additional information is embedded in the same frequency band, interference may occur during extraction. Therefore, it is preferable that the additional information of the color image and the additional information of the metallic image are embedded in different frequency bands from each other. If the frequencies of both are known, the additional information can be extracted by analyzing that frequency band during extraction.

[0126] Also, in Fig. 26(a), additional information is first extracted from the color area and then from the metallic area, but this order may be reversed. The same applies to Fig. 27. Furthermore, for a printed matter whose entire surface is a metallic area, the procedure of Fig. 26(a) may be executed. In this way, the image is irradiated and photographed with a light beam value (light amount) corresponding to the reflection intensity (or reflectance) of the medium on which the image is formed by a light source such as light. By doing so, it becomes easier to extract the additional information and the extraction accuracy is improved. (Modification example of the first embodiment) In the first embodiment, both the color area and the metallic area were included in the photographed image for extracting the additional information, but the present invention is not limited to this. The same effect can be obtained even when only the color area is included in the photographed image or when the metallic area is included in the photographed image. The same applies when the extraction process of the additional information is separately performed from the recording material having only the color area and the recording material having only the metallic area.

[0127] The flowchart of the multiplex decoding process at that time is shown in Fig. 26(b). The differences from Fig. 26(a) are S2607, S2608, and S2609. By adding these steps, in Fig. 26(b), the number of trials is counted by the counter i, and when the additional information embedded in the color area cannot be extracted after a predetermined number of trials, the process proceeds to the extraction process of the additional information embedded in the metallic area (S2608 - No). As a result, even when only one of the color area and the metallic area is included in the photographed image, the embedded additional information can be extracted. Note that when the extraction process of the additional information is separately performed from the recording material having only the color area and the recording material having only the metallic area, the designation of the photographing mode may be accepted from the user in advance (not shown).

[0128] (Second embodiment) As shown by graph 2401 in FIG. 24, the glossiness changes significantly according to the amount of metallic ink per unit area on the paper surface. Further, graph 2402 in FIG. 24 shows the relationship between the metallic ink density and the glossiness in semi-gloss paper, where the smoothness of the paper surface is lower than that of glossy paper. Since the smoothness of the paper surface is lower than that of glossy paper, the degree of increase in glossiness with respect to the amount of ink on the paper surface in graph 2402 of semi-gloss paper is smaller than that in graph 2401 of glossy paper. Thus, the glossiness changes significantly according to the amount of metallic ink printed on the paper surface and the type of recording medium used. When there are multiple types of recording media to which additional information is to be embedded, or when there are multiple amounts of metallic ink to be printed on the recording medium, it is preferable to have multiple shooting modes for the metallic area. In the first embodiment, there was one shooting mode for the metallic color area, but in this embodiment, an example of having multiple shooting modes for the metallic color area will be described.

[0129] In order to extract the embedded additional information from each of a plurality of metallic areas with different reflection intensities, it is necessary to set the light flux value of the light irradiated during shooting to an appropriate value according to the reflection intensity. The appropriate value is a value for detecting the additional information embedded in the target metallic area from the captured image with sufficient intensity for extraction. In this embodiment, a light flux value suitable for each metallic area with each reflection intensity to be subjected to multiplexed decoding processing is obtained in advance and assigned to each shooting mode for the metallic area. Then, when each shooting mode k is activated, the illumination light control unit 2502 controls the light flux value of the light irradiated by the illumination unit 2501 to be the value assigned in advance in association with the shooting mode k.

[0130] Figure 27 is a flowchart of the multiplexed decoding process in this embodiment. The differences from the aforementioned Figure 26 are S2701 to S2707. By adding S2705, S2706, and S2707, shooting in the shooting mode for one metallic area will be repeated a predetermined number of times. Here, if additional information can be extracted, the multiplexed decoding process ends. If the additional information cannot be extracted, in S2701, S2702, S2703, and S2704, the shooting mode k for the metallic area is changed to a setting according to the value of k and shooting is performed. Here, if the additional information can be extracted, the multiplexed decoding process ends. If the additional information cannot be extracted, the process of further changing the shooting mode for the metallic area and performing shooting and extraction is repeated.

[0131] As described above, in addition to the color area, it becomes possible to appropriately extract the additional information embedded from a plurality of metallic areas with different reflection intensities.

[0132] (Third Embodiment) In the second embodiment, in addition to the color area, it was possible to appropriately extract the additional information embedded from a plurality of metallic areas with different reflection intensities. However, at the same time, the number of shootings for the multiplexed decoding process increases, and the processing time becomes long.

[0133] To solve this problem, at the time of the multiplexed encoding process, the shooting conditions of the related metallic area may be included in the additional information embedded in the color area. That the color area and the metallic area are related means the case where the color area and the metallic area exist on the same recording medium, or the case where a recording medium having only a color area and a recording medium having only a metallic area form a set.

[0134] In this embodiment, an example of determining the shooting mode of the metallic area based on the information regarding the shooting conditions of the metallic area included in the additional information extracted from the color area at the time of the multiplexed decoding process will be described.

[0135] FIG. 28 is a flowchart of the multiplexing decoding process in the present embodiment. The differences from FIG. 26(b) are S2801 and S2802. In S2801, based on the additional information extracted in S2602, the shooting mode for the metallic area is determined. Also, when the additional information embedded in the color area cannot be extracted, in S2802, the user is instructed to move the terminal to change the extraction target area.

[0136] As described above, based on the information regarding the shooting conditions of the metallic area included in the additional information extracted from the color area, the shooting mode of the metallic area is determined. By doing so, the additional information embedded in the color area and the metallic area can be extracted respectively without making the processing time redundant.

[0137] Since the movement of the terminal is encouraged in S2801, the procedure of FIG. 28 can be applied only to portable devices such as mobile terminals with cameras. In the case of a device with a scanner, since it is possible to attempt to extract the additional information for the entire sheet surface, for such a device, it may be a procedure that does not perform S2608 and S2802. That is, it may be a procedure in which S2801 is added to FIG. 26(a).

[0138] In the first to third embodiments described above, the luminous flux of the light 212 is controlled according to the reflectance of the medium on which the image is formed. However, if the exposure of the camera 202 can be controlled, the exposure may be controlled. In this case, contrary to the luminous flux of the light, the exposure is reduced when shooting the metallic area, and the exposure is increased more when shooting the non-metallic area.

[0139] [Other Examples] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0140] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the claims are attached to disclose the scope of the invention.

Explanation of Signs

[0141] 201: Mobile terminal with camera, 202: Imaging sensor, 203: Additional information separation device, 212: Light

Claims

1. Adjusting means for adjusting the amount of light of a light source that irradiates light onto an image formed on a medium to an amount of light corresponding to the reflection intensity of the medium, which is determined by at least either the presence or absence of an image formed with metallic ink and the type of the medium; Acquisition means for acquiring image data obtained by photographing the image; Extraction means for extracting additional information multiplexed in the image from the acquired image data; comprising: After the extraction means extracts the additional information from the image data when the amount of light of the light source is a first amount of light, the adjusting means reduces the amount of light of the light source to a second amount of light smaller than the first amount of light and that enables acquisition of image data from the image formed on the medium. An information processing apparatus characterized by the above.

2. The information processing apparatus according to Claim 1, wherein the adjusting means adjusts the amount of light of the light source to an amount of light corresponding to each of a plurality of reflection intensities, and the acquisition means acquires a plurality of pieces of image data obtained by photographing the image with an amount of light corresponding to each of the plurality of reflection intensities. An information processing apparatus characterized by the above.

3. The information processing apparatus according to Claim 1 or 2, wherein the adjusting means sets the first amount of light such that the luminance of the image saturates in the region of the image formed with metallic ink and does not saturate in other regions, and sets the second amount of light such that the luminance of the image does not saturate in the region of the image formed with metallic ink. An information processing apparatus characterized by the above.

4. The information processing apparatus according to any one of Claims 1 to 3, wherein the adjusting means determines the second amount of light based on the additional information extracted from the image data obtained by photographing with the first amount of light. An information processing apparatus characterized by the above.

5. The information processing apparatus according to any one of Claims 1 to 4, wherein the adjusting means adjusts the amount of light of the light source to the second amount of light when the additional information can be extracted from the image data obtained by photographing with the first amount of light, or when the additional information could not be extracted after a predetermined number of attempts. An information processing apparatus characterized by the above.

6. A program for causing a computer to function as the information processing apparatus according to any one of Claims 1 to 5.

7. An adjustment step of adjusting the amount of light of a light source that irradiates light on an image formed on a medium to an amount of light corresponding to the reflection intensity of the medium determined by at least either the presence or absence of an image formed with metallic ink and the type of the medium; An acquisition step of acquiring image data obtained by photographing the image; An extraction step of extracting additional information multiplexed on the image from the acquired image data; comprising: In the adjustment step, after extracting the additional information from the image data when the amount of light of the light source is a first amount of light in the extraction step, the amount of light of the light source is made smaller than the first amount of light and adjusted to a second amount of light capable of acquiring image data from the image formed on the medium. An information processing method characterized by the above.

Images