Information processing device, method, program, and system

Abstract

To provide an information processing apparatus that prevents omission of multiplexed additional information due to print processing.SOLUTION: On condition that a monochrome mode is set and additional information is multiplexed for an image to be printed, an information processing apparatus sets a color mode in place of the monochrome mode as the setting of printing based on input image data. The information processing apparatus generates, on the basis of the input image data, color image data corresponding to printing in the color mode in which a color obtained by converting the image to be printed into a monochrome image is indicated by the value of a color signal and performs, on the color image data, processing for multiplexing the additional information for the image to be printed. The information processing apparatus causes a printer to print, in the color mode, a multiplexed image in which the additional information is multiplexed.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present invention relates to an information processing device, a method, a program, and a system for multiplexing additional information onto image data. [Background technology]

[0002] A technology known as digital watermarking technology, which multiplexes additional information onto printed matter so that it is difficult to visually distinguish it, has been known for some time (Patent Document 1). "Multiplexing" is a general term for the superimposition of data with a different meaning onto other data, such as steganography and watermarks, or for the technical field in question. Examples of multiplexing include the multiplexing of identification information to prevent fraudulent counterfeiting of banknotes, revenue stamps, securities, etc., and the multiplexing of audio information onto photographs. By reading the multiplexed information with, for example, a smartphone, it is possible to provide various added values ​​to users. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-174556 A Summary of the Invention [Problem to be solved by the invention]

[0004] When print settings are configured to print image data in which additional information has been multiplexed by changing color components in monochrome, the additional information is lost due to monochrome printing processing in the printer.

[0005] An object of the present invention is to provide an information processing apparatus, method, program, and system that prevent loss of multiplexed additional information due to printing processing. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing apparatus according to the present invention includes: an acquisition unit for acquiring input image data indicating an image to be printed; When the print setting is set to monochrome mode, the input image data is converted into converted image data representing a monochrome image of a plurality of channels, and a process for multiplexing additional information on the image to be printed is performed on the converted image data. A multiplexing means for performing of The present invention is characterized by being equipped with: Effect of the Invention

[0007] According to the present invention, it is possible to prevent loss of multiplexed additional information due to printing processing. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a system configuration including an information processing device. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of an information processing device. [Diagram 3] FIG. 2 illustrates an example of a hardware configuration of an image processing apparatus. [Figure 4] FIG. [Diagram 5] FIG. 2 is a visual representation of the pattern imparted to an image by a mask. [Figure 6] FIG. 1 shows a grayscale image. [Figure 7] FIG. 13 is a diagram showing the result of performing multiplexing. [Figure 8] FIG. 13 is a diagram showing the result of performing multiplexing. [Figure 9] FIG. 13 is a diagram showing the result of applying FFT. [Figure 10] FIG. 13 is a diagram showing the result of applying FFT. [Figure 11] FIG. 13 is a diagram showing the result of applying FFT. [Figure 12] FIG. 13 is a diagram showing the result of applying FFT. [Figure 13] FIG. 13 is a diagram showing a pattern for determining a start position. [Figure 14] 4 is a flowchart showing a process executed in the information processing device. [Figure 15] FIG. 13 is a diagram showing a user interface screen. [Figure 16] 10 is a flowchart showing a process before a print process is started. [Figure 17] FIG. [Figure 18] 13 is a flowchart showing a color conversion process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] 1 is a diagram showing an example of a system configuration including information processing devices 101 and 103 in this embodiment. The information processing devices 101 and 103 are mobile terminals such as smartphones and PCs, and the information processing devices 101 and 103 will be described as having a common configuration. However, the configurations of the information processing devices 101 and 103 may be different.

[0011] In the system, a user can internally store image data captured by an imaging device. In the system, the router 104 and the image processing device 102, and the router 104 and the information processing device 103 are connected to each other so that they can communicate with each other by wired communication such as a LAN cable. Furthermore, the information processing device 101 and the router 104 are connected to each other so that they can communicate with each other by wireless communication such as Wifi. Note that the router 104 and the image processing device 102 or the information processing device 103 may be connected by wireless communication. Furthermore, the wireless communication may be Wifi, Bluetooth, NFC, or the like.

[0012] In this embodiment, in a system as shown in FIG. 1, a user can select image data to be printed by an application on the information processing device 101 and multiplex any additional information such as a watermark onto the image data. That is, the information processing device 101 can operate as an additional information multiplexing device that multiplexes additional information onto image data. In addition, the user can print the image data onto which such additional information has been multiplexed, using the image processing device 102. In this embodiment, text information is described as an example of additional information, but other types of information may be used, for example, audio information. Then, another user captures the printed matter with the information processing device 101, and the multiplexed additional information is extracted. That is, the information processing device 101 can operate as an additional information extraction device. For example, an operation in which audio is reproduced by capturing an image of a printed matter is realized.

[0013] FIG. 2 is a diagram showing an example of a hardware configuration of the information processing device 101. The CPU 201 is a central processing unit, and executes, for example, a process of multiplexing additional information according to a program. The ROM 202 stores the program executed by the CPU 201. The RAM 203 provides a memory for temporarily storing various information when the CPU 201 executes the program. The secondary storage device 204 is a storage medium for saving image files, additional information to be multiplexed, and the like. The display unit 205 displays a user interface screen, processing contents, and the like. The display unit 205 may have a touch panel function, and accepts processing instructions, settings, character input, and the like by user operation of the touch panel. The speaker 206 outputs sound. For example, the speaker 206 outputs sound when the decoded and extracted additional information is sound data.

[0014] The network interface (IF) 207 is connected to a network 220 such as a LAN (Local Area Network). The network 220 may include the Internet. For example, the CPU 201 accesses a site provided by a server 212 connected to the Internet through the network interface 207 and displays a screen of the site on the display unit 205. The internal imaging device 209 is a device having an imaging function, such as a built-in camera. The external imaging device 210 is an external device having an imaging function connected to the information processing device 101 via the device interface (IF) 208, such as a digital camera or a video camera.

[0015] In this embodiment, the information processing device 101 will be described as an additional information multiplexing device. However, the information processing device 101 may also operate as the additional information multiplexing device.

[0016] FIG. 3 is a diagram showing an example of a hardware configuration of the image processing device 102. In this embodiment, the image processing device 102 is described as an MFP (Multifunctional Peripheral), but the image processing device 102 does not have to be configured as an MFP as long as it is a printing device (printer) having a printing function. The control unit 301 is connected to a scanner unit 311 which is an image input device and a printer unit 312 which is an image output device, and controls input and output of image information between each device. The control unit 301 can also communicate with the information processing device 101, the router 104, and the information processing device 103 via a network I / F 306. For example, the control unit 301 can receive print jobs and image data from the information processing device 101 and the information processing device 103.

[0017] The CPU 302 comprehensively controls the operation of the image processing device 102, and operates based on a program stored in, for example, the RAM 303. The ROM 304 is, for example, a boot ROM, and stores a system boot program. The storage unit 305 stores system software, image data, programs for controlling the operation of the image processing device 102, and the like. The CPU 302 loads the program stored in the storage unit 305 into the RAM 303, and controls each unit of the image processing device 102 based on the program.

[0018] The network I / F 306 connects the control unit 301 to the network 220, communicates with devices on the network 220, and inputs and outputs various types of information. The device I / F 307 connects the control unit 301 to a scanner unit 311 and a printer unit 312, which are image input / output devices, and performs synchronous / asynchronous conversion of image data. The operation unit I / F 308 is an interface that connects the operation unit 313 to the control unit 301, and outputs data to be displayed on the operation unit 313 to the operation unit 313. The operation unit 313 includes a panel and hardware keys, and can accept instructions and setting operations from the user. The operation unit I / F 308 also transmits information input by the user from the operation unit 313 to the CPU 302.

[0019] The image processing unit 309 performs image processing such as correction, interpolation, and conversion on image data to be printed that is received via the network in accordance with the settings and characteristics of the input / output device. The image processing unit 309 also performs image processing on image data that is input to the device I / F 307 or output from the device I / F 307. The image memory 310 is a memory for temporarily expanding the image data processed by the image processing unit 309.

[0020] A description will now be given of the multiplexing embedding process (multiplexing encoding process) performed in the information processing device 101. As an example, a method of multiplexing information "hello" onto an input image will be described.

[0021] Handling information on an information processing device such as a PC means handling binary data. Binary data is information that is either "0" or "1", and when these "0" or "1" pieces of information are connected in succession, they have a specific meaning. Each character has a corresponding binary data. Which character corresponds to which binary data is defined by something called a "character code". For example, in "Shift JIS", one of the character codes, "h" corresponds to the binary data "01101000". Similarly, "e" corresponds to the binary data "01100101", "l" corresponds to "01101100", and "o" corresponds to the binary data "01101111".

[0022] In other words, the character "hello" can be expressed in binary data as "0110100001100101011011000110110001101111". Conversely, if the binary data "0110100001100101011011000110110001101111" can be obtained, the character "hello" can be obtained. In other words, multiplexing can be achieved by embedding data so that it can be determined as "0" or "1". When multiplexing, the data input is "hello", but the data actually embedded is "0110100001100101011011000110110001101110001101111". In this embodiment, in order to distinguish between the two, "hello" is called additional information (multiplexed information), and "0110100001100101011011000110110001101111" converted for embedding is called embedded data.

[0023] Next, a method of multiplexing information "0" and "1" into an image will be described. In this embodiment, embedding of data indicating "0" and "1" will be described, but the multiplexing method is not limited to embedding of "0" and "1", and is not limited to embedding of "0" and "1" as long as it is possible to uniquely associate data on the encoding side and the decoding side.

[0024] The size of the multiplexed image is explained as 640 pixels (px) in height and 480 pixels (px) in width. Here, to generate "0" and "1", two masks are prepared as shown in Fig. 4(a) and Fig. 4(b).

[0025] Figures 4(a) and 4(b) show masks consisting of 8px x 8px, and by adding the mask value to an image, a periodic pattern can be given to an 8px x 8px area in the image. Figures 5(a) and 5(b) visually show what kind of pattern is given to an image by a mask. In Figure 5, the positions of "10" in the masks of Figures 4(a) and 4(b) are represented as black, the positions of "0" as gray, and the positions of "-10" as white, and the diagonal lines shown in Figures 5(a) and 5(b) appear in the image.

[0026] Pseudo code for applying the masks of Figures 4(a) and 4(b) alternately to the entire image is given below. 01: int i, j, k, l; 02: int width = 640, height=480; 03: unsigned char *data = image data; 04: int **maskA = mask data; 05: bool isMaskA = true; 06: for(j = 0; j < height; j+=8){ 07: for(i = 0; i < width; i+=8){ 08: for(k = 0; k < 8; k++) 09: for(l = 0; l < 8; l++) { 10: if(isMaskA == true) { 11: data[(i+k)+(j+l)*width] += maskA[k][l]; 12:} 13:} 14:} 15:} 16:} The image data in the third row is assumed to be a grayscale image. Here, we assume a grayscale image as shown in Fig. 6. Fig. 7 shows an example of multiplexing the grayscale image in Fig. 6 using the 8px x 8px mask in Fig. 4(a) and Fig. 4(b). As shown in Fig. 7, the image has alternating diagonal lines (corresponding to "0" and "1").

[0027] Figure 8 shows an example where the "hello" data "0110100001100101011011000110110001101111" is applied, rather than a simple repetition of "0" and "1". "hello" itself only requires 40 8px x 8px blocks, so an area of ​​320px x 8px is sufficient, but in Figure 8 "hello" is multiplexed repeatedly. By repeatedly multiplexing the same data in this way, it is possible to increase the resistance to the pattern being erased due to scratches or dirt on the photo. Also, having the same data multiple times makes it easier to search for when decoding.

[0028] In this example, the image size is 640px x 480px, so the total number of pixels is 307,200. Also, the number of pixels required to multiplex "hello" is 320px x 8px = 2,560 pixels, as shown above. In other words, "hello" is repeated 307,200÷2560=120 times.

[0029] In multiplexing additional information, data is embedded as described above. In general, additional information is multiplexed in areas to which the human eye is less likely to react. For example, additional information is multiplexed by varying the ab component after an image is decomposed into Lab components, the UV component after decomposition into YUV components, or the CbCr component of YCbCr. For example, in the case of YUV components, the U component alone may be varied, or both the UV components may be varied.

[0030] When varying a single component, the algorithm is simpler than when varying multiple components. As a result, the program size is smaller, which means faster application downloads, and the circuit area is smaller, which means lower manufacturing costs.

[0031] In addition, when multiple components are varied, the effect of not being able to embed data can be reduced. For example, when a digital image is expressed in 8 bits per color, a value between 0 and 255 is assigned. Since values ​​outside this range cannot be used as image data, when the calculation result of the pixel value is less than 0 or is 256 or more, generally, 0 or 255 is assigned to keep it within the valid range. In the masks of Fig. 4(a) and Fig. 4(b), the pixel values ​​are varied by ±10, but if, for example, all the U components of the YUV components in the mask area are 255, the values ​​of that area will be in the range of 245 to 255, not 245 to 265. In that case, as a result, only a smaller variation than the predetermined value is given, resulting in multiplexing that is vulnerable to noise and difficult to decode. However, even if the U component of the YUV components is 255, if the V component is 128, for example, even if the change in value of ±10 falls within the valid range, noise resistance can be maintained by using the V component. In this way, it is possible to increase the robustness of the multiplexing (robustness of decoding) by using both UV components. When converting an image into RGB format for use, the UV components may be changed while checking whether the RGB components after converting the YUV components into RGB components are also within the range of 0 to 255.

[0032] As described above, data embedding (multiplexed encoding) is achieved by superimposing binary information by applying periodic fluctuations.

[0033] In this embodiment, there are cases where the information processing device 101 operates as an additional information extraction device. The process of decoding the embedded data in that case will be described below.

[0034] The mask shown in FIG. 4(a) has a periodic pattern, and in order to extract the features, FFT (Fast Fourier Transform) is applied to the grayscale image in FIG. 8. FIG. 9 shows an example of the result of applying FFT to an image in which data with a periodic pattern is embedded. Meanwhile, FIG. 10 shows an example of the result of applying FFT to the grayscale image in FIG. 6 in which no data is embedded. Compared to FIG. 10, it can be seen that a spectrum (white dot) appears at a specific position in FIG. 9. FIG. 11 shows an example of the result of applying FFT to the grayscale image in FIG. 6 by multiplexing only "0". Also, FIG. 12 shows an example of the result of applying FFT to the grayscale image in FIG. 6 by multiplexing only "1". FFT is a method used for frequency analysis of an image, and the fluctuation of the periodic pattern of the mask changes the frequency of the image. Therefore, by applying FFT, a spectrum appears at a specific position. Therefore, by reading that state, it is possible to recognize and decode the embedded data.

[0035] As shown in Figure 9, it is necessary to be able to recognize even if the properties of both "0" and "1" are included. Therefore, in order to determine whether it is "0" or "1", the size of the FFT analysis is performed at 8px x 8px, the same size as the multiplexed block, so that the size at the time of embedding and the size at the time of analysis are the same. By performing the FFT only within this range, only one of the spectra will appear, making it possible to determine whether it is "0" or "1".

[0036] On the decoding side, when decoding data, it is possible to specify the position from which the data starts. In that case, for example, by preparing a "pattern for determining the start position" different from the pattern of "0" and "1", it is possible to determine the start of the data. FIG. 13 is a diagram showing an example of a pattern for determining the start position. As shown in FIG. 13, the pattern for determining the start position is a mask pattern different from those in FIG. 5(a) and FIG. 5(b). In the mask pattern in FIG. 13, if the top and bottom of the image are the y-axis and the left and right are the x-axis, a spectrum peak appears from the center of the image in the direction of the x-axis. If two patterns for determining the start position are detected, additional information can be extracted by decoding the "0" and "1" between them.

[0037] Furthermore, on the decoding side, processing may be performed to enable FFT analysis. When an image of a printed matter on which additional information is multiplexed is captured through an imaging sensor (e.g., a camera), it is not always captured in the correct orientation. For example, the image sensor may be close to the printed matter or far away from it, and the data may not be decoded even if the captured image is subjected to FFT analysis in 8px x 8px block units. In the above-mentioned multiplexing, the image is changed so that a strong spectrum appears at a specific frequency, and it is known where the spectrum should appear. Based on this, on the decoding side, image conversion may be performed before FFT analysis so that the spectrum obtained from the image captured by the imaging sensor approaches the known spectrum.

[0038] The operation of the information processing device 101 in this embodiment will be described below.

[0039] The user can configure various print settings in the information processing device 101 for printing in the image processing device 102, for example, to set "monochrome printing" or "color printing."

[0040] When multiplexing additional information, as described above, the color components of the image are varied. On the other hand, when the user has set "monochrome printing" as the print setting, the image processing device 102 performs processing in two tones of monochrome (monochrome conversion). That is, even if the additional information is multiplexed, the color components that are multiplexed are lost due to the processing in two tones of monochrome in the image processing device 102. As a result, the additional information multiplexed on the multiplexed image cannot be decoded. Therefore, in this embodiment, when multiplexing additional information, if the user has set "monochrome printing", if the image data to be printed is color image data, it is converted to grayscale, and the setting is changed from "monochrome printing" to "color printing". With such a configuration, it is possible to maintain image quality close to the color tone of monochrome printing desired by the user, and to prevent the loss of color components due to the processing in two tones of monochrome in the image processing device 102.

[0041] Fig. 14 is a flowchart showing processing executed in the information processing device 101 up to printing of image data to be printed. The processing in Fig. 14 is realized, for example, by the CPU 201 reading a program stored in the ROM 202 into the RAM 203 and executing it. The program for executing the processing in Fig. 14 is installed in the information processing device 101 as, for example, a print application. The processing in Fig. 14 is started, for example, when the user starts the print application on the information processing device 101.

[0042] In S1401, the CPU 201 acquires image data (input image data) selected by the user. FIG. 15(a) is a diagram showing an example of a main screen 1501 displayed on a print application. As shown in FIG. 15(a), a button 1502 is displayed on the main screen 1501. When the user presses the button 1502, an image selection screen (not shown) is displayed, and the user can select any image data. As a method for acquiring an image, image data in the information processing device 101 may be acquired, or image data may be acquired from a server (not shown) via a network. Also, the image may be acquired from image data prepared in advance in the print application. In this embodiment, image data will be described, but it is not limited to image data, and document-based data such as PDF may be used, and any data that can be converted into two-dimensional RGB data may be used. The image data selected by the user as a print target becomes image data to be multiplexed with additional information. The CPU 201 acquires the selected image data as data to be multiplexed, and temporarily stores the data in the RAM 203 so that it can be reused in the print application.

[0043] Furthermore, an image of the image data selected by the user is displayed in a display area 1504 in the main screen 1501. If the storage area of ​​the RAM 203 in the information processing device 101 is insufficient, the image may be saved as a file in the secondary storage device 204, and a reduced image (thumbnail image) of the designated image may be displayed on the screen of the print application.

[0044] In S1402, the CPU 201 acquires additional information to be multiplexed onto the image data selected by the user in S1401. As shown in Fig. 15(a), a check box 1503 and an input area 1505 are provided in a main screen 1501. The user can instruct multiplexing onto the selected image data by checking the check box 1503. The user can also input any text information into the input area 1505. The CPU 201 acquires the text information input into the input area 1505 as additional information to be multiplexed onto the image data.

[0045] In this embodiment, text information is described as the multiplexed information, but other types of information that can be handled as binary data, such as audio, photos, videos, etc., may be used. The CPU 201 saves the text information input by the user in the RAM 203 so that it can be reused within the print application, or in the secondary storage device 204 as a file.

[0046] In S1403, the CPU 201 executes a search for the image processing device 102. As shown in FIG. 15A, a button 1506 is displayed in the main screen 1501. When the user presses the button 1506, the CPU 201 transmits a printer search broadcast to the network 220. Information is sent to all hosts sharing the same network by broadcasting. When the image processing device 102 replies to the information processing device 101 that is the source of the broadcast, the information processing device 101 can grasp the IP address of the image processing device 102, and one-to-one communication is possible thereafter. In this embodiment, the identification information of the printer that has responded is automatically registered in the print application, and in this example, for the sake of explanation, the printer in the network 220 is one of the image processing device 102. The information registered in the print application is, for example, the printer name and the printer IP address, and the printer name is displayed as a printer name 1507 on the main screen 1501.

[0047] In S1404, the CPU 201 acquires information on print settings from the image processing device 102 found in S1403, and accepts the print settings from the user. Normally, a printer can acquire information on the media (recording medium) and size that the printer itself supports, print settings, and the like.

[0048] In this embodiment, communication between the print application of the information processing device 101 and the image processing device 102 is performed using XML (Extensible Markup Language). XML is one method of expressing information, but data transmission and reception is not limited to this, and other protocols and communication methods may be used. An example of XML for requesting the image processing device 102 to obtain information related to print settings is shown below. 001:<?xml version=”1.0” encoding=”UTF-8”?> 002: <cmd> 003: <operation> getInformation< / operation> 004:< / cmd> The first line says that XML is version 1.0 and that the character encoding is UTF-8. <cmd> indicates the start of a command that the printer will receive, and the fourth line< / cmd> indicates the end of the command. A tag is added within the cmd tag and sent to the image processing device 102 by the print application, causing the image processing device 102 to perform an operation according to the content of the tag. For example, the operation type of the printer is described in the operation on the third line, and getInformation indicates an operation of returning the acquired information to the application that made the request. An example of XML returned by the image processing device 102 to the print application is shown below. 001:<?xml version=”1.0” encoding=”UTF-8”?> 002: <res> 003: <name> printer001< / name> 004: <medialist> 005: <media> 006: <name> A4< / name> 007: <width> 2000< / width> 008: <height> 3000< / height> 009: < / media> 010: <media> 011: <name> L< / name> 012: <width> 1000< / width> 013: <height> 1500< / height> 014: < / media> 015: <media> 016: <name> Posted by< / name> 017: <width> 1500< / width> 018: <height> 2250< / height> 019: < / media> 020: < / medialist> 021: <colormode> 022: <mode> Color< / mode> 023: <mode> Mono< / mode> 024: < / colormode> 025: < / res> The "res" in the second line means a response from the printer. In this embodiment, the information sent from the print application to the printer is <cmd>The information goes from the printer to the print application. <res>The command line starts with, but is not limited to, "printer name". The third line, "name", indicates the printer name. Lines 4 to 20 are a list of the supported paper sizes. <medialist>The tag lists the media that the printer can handle. Here, <media>The tag contains name, width, and height tags, which respectively indicate the name of the paper, the vertical width (pixels) of the image, and the horizontal width (pixels) of the image. From this information, for example, for A4 size (210 mm vertical, 297 mm horizontal), an image size of 2,000px horizontal and 3,000px vertical is required. Generally, such image size depends on the printer's dpi (dots per inch). dpi is a unit of how finely ink can be arranged, and the higher the dpi, the more detailed the expression, or in other words, the higher the printing resolution. In other words, even for the same A4 size, the higher the dpi, the more pixels of image data required for printing. On the other hand, the image data specified by the printing application does not necessarily have the number of pixels required by the printer. In that case, the input image is corrected to the size required by the printer by enlarging, reducing, or trimming it inside the printer or in the printer driver, and then it is printed. For example, if you try to print an image with a height of 4,000px and a width of 6,000px on A4 size using the image processing device 102, the data contains more pixels than necessary for A4 size, and printing cannot be performed as is. Therefore, the image data is converted into a height of 2,000px and a width of 3,000px, for example, inside the printer or by the printer driver, and then printing is performed.

[0049] The 21st to 24th lines list selectable print color setting modes. In this embodiment, two types can be selected: "color" which is a color mode and "mono" which is a monochrome mode. The color mode is a print mode in which printing is performed using multiple colors of printer ink. A printer normally expresses colors by subtractive color mixing. Normally, subtractive color mixing can express all colors with three colors, C (cyan), M (magenta), and Y (yellow), and the image processing device 102 is provided with three colors of ink, C, M, and Y. Note that the types of ink are not limited to these, and ink colors other than CMY, such as gray ink, which is a special color ink, and RGB ink, may be provided.

[0050] On the other hand, the monochrome mode is a printing mode in which the color information of the input image is replaced with gray information and printed. The monochrome mode may be used, for example, to reduce the amount of ink used or to improve printing speed. In the monochrome mode, one color of black ink (K) or multiple achromatic color inks are used. Also, process black may be formed using CMYK. Note that in this embodiment, any mode that realizes monotone color tones, such as sepia mode or grayscale mode, is considered to be included in the monochrome mode.

[0051] In this embodiment, when the user has set the monochrome mode, which uses only black ink, and additional information is to be multiplexed, the print setting is changed from "monochrome mode" to "color mode." As a result, color printing is performed using color inks in the image processing device 102. Even if the print setting is changed to "color mode," the color of the output is output in a color close to gray, from the perspective of the image quality expected in monochrome mode. In other words, the output is an image in which gradation is expressed by the density of dots printed in black.

[0052] Fig. 15(b) is a diagram showing an example of a print setting screen 1510 displayed on a print application. The print setting screen 1510 is displayed when a print setting button 1508 on a main screen 1501 is pressed. The print setting screen 1510 is displayed based on information related to print settings acquired from the image processing device 102. As shown in Fig. 15(b), the print setting screen 1510 displays items such as a paper size 1511 and a print color mode 1513 so that selection can be accepted.

[0053] Here, it is assumed that the user has checked check box 1503 for executing multiplexing processing, set the paper size to "A4" indicated by selection button 1512, and set the print color mode to "monochrome" indicated by selection button 1513. When the user presses OK button 1514, CPU 201 temporarily stores the items set by the user on print setting screen 1510 in a storage area such as RAM 203, making them available for later reference. When OK button 1514 is pressed, the display returns to main screen 1501. After the user presses print button 1509 on main screen 1501 is received, print processing is started in image processing device 102.

[0054] Fig. 16 is a flowchart showing the process up to the start of print processing in the image processing device 102. The process in Fig. 16 is realized, for example, by the CPU 201 reading a program stored in the ROM 202 into the RAM 203 and executing it. The process in Fig. 16 is started when the print button 1509 on the main screen 1501 is pressed, subject to the settings of the check box 1503 and the selection button 1513 as described above.

[0055] In S1601, the CPU 201 acquires image data to be printed that the user selected on the main screen 1501 from the RAM 203. In S1602, the CPU 201 acquires additional information, such as text data, that the user input on the main screen 1501. In S1603, the CPU 201 acquires print settings that the user set.

[0056] In S1604, the CPU 201 determines whether or not the monochrome mode is set based on the print setting acquired in S1603. If it is determined that the monochrome mode is set, in S1605, the CPU 201 changes the print setting from the "monochrome mode" to the "color mode". As described above, when the monochrome mode is set as the print setting, the monochrome conversion is executed inside the printer or in the printer driver. For example, when the monochrome conversion is executed inside the printer, the monochrome conversion is executed inside the printer after the multiplexed image data is transmitted to the image processing device 102. Here, the monochrome conversion is a conversion for expressing an image by setting one color as a reference in the image and only changing the shade of the color, and the image data is expressed as the density of data in a certain area. In other words, there is no change other than the so-called luminance component such as a and b of Lab or U and V of YUV, and in the case of a method of executing multiplexing by varying the components other than the luminance, additional information is lost and decoding becomes impossible. Therefore, in this embodiment, when the monochrome mode is set by the user, the print setting is changed to the color mode so that the monochrome conversion is not executed inside the printer or in the printer driver. At that time, a predetermined color conversion process is executed so that even in the color mode, a color tone equivalent to the monochrome mode desired by the user can be output. With such a configuration, the operation appears to the user as being in the print mode set, and multiplexing of additional information is also possible. In this embodiment, as a color conversion process to replace the monochrome conversion using black ink, grayscale conversion is executed as the above-mentioned predetermined color conversion process.

[0057] In the branching of the process after S1604, for example, the following configuration may be used. For example, a blacklist may be stored in advance in the secondary recording device 204 or in a program, and the blacklist may be compared with information on print settings acquired from the image processing device 102. A blacklist is a list that describes the names of services and software that should not be used. For example, the blacklist describes the character string "mono" as a text file. If the print settings match the description in the blacklist, the process proceeds to S1606. Note that if it is desired to store multiple pieces of information in the blacklist, or if it is desired to describe color conversion processing to be executed when a setting matches the blacklist, CSV (comma-separated values) or a general database may be used.

[0058] Also, the print application may obtain information from a server on the network. The print application can obtain the printer name by communicating with the image processing device 102. This printer name may be used to make an inquiry to the server, and the process from S1604 onwards may be branched. In this case, the server holds the printer name, print settings, and information on whether or not monochrome conversion is performed for these combinations. In other words, the configuration has a blacklist on the network side.

[0059] Alternatively, information obtained from the image processing device 102 may be used. Information on whether multiplexing is possible or not may be added to the colormode of the XML including the printer information described above. For example, the XML may look like the following. 001:<?xml version=”1.0” encoding=”UTF-8”?> 002: <res> 003:~ One part omitted~ 004: <colormode> 005: <mode> 006: <modename> Color< / modename> 007: <steganography>true< / steganography > 008:< / steganography> < / mode> 009: <mode> 010: <modename> Mono< / modename> 011: <steganography>false< / steganography > 012: <alternate> grayscaletransform< / alternate> 013: <colormode> Color< / colormode> 014:< / steganography> < / mode> 015:< / colormode> 016:< / res> In the above XML, compared to the previous XML, <colormode>of <mode>First, the 11th line <steganography>contains information on whether or not the multiplexed additional information will be lost depending on the color mode setting. If this tag is false, it means that the multiplexed additional information will be lost. <alternate>The tag describes a proxy method to prevent loss of additional information, and is written on line 13. <colormode>describes the print settings after using the proxy method. The above notation represents information that "direct multiplexing is not possible in monochrome mode, but instead grayscale conversion is applied to the image and the print color mode is set to color." For example, if the information acquired from the image processing device 102 represents the above information, the processing from S1605 onwards is performed. <alternate>The tag's contents, grayscaletransform, is used to determine the processing to be performed by the printing application. Alternatively, the processing content itself may be written in the tag. For example, the image conversion program may be written directly in the XML as shown below, and the printing application may interpret and execute it. ~Omitted~ 001: <alternate> 002: function transform(r, g, b){ 003: var gray = 0.299*r + 0.587*g + 0.114*b; 004: return {r: gray, g:gray, b:gray}; 005:} 006:< / alternate> ~Omitted~ The second to fifth lines are functions that follow the JavaScript notation. By giving pixel data of R (Red), G (Green), and B (Blue) as arguments, the function returns the changed R, G, and B, and the third line corresponds to the process of converting R, G, and B to grayscale. This function is acquired by an application, and the result interpreted by an interpreter running on the CPU 201 is executed from the application, thereby enabling dynamic program execution. For example, by using the eval function provided by JavaScript, the above function can be interpreted as a program and made executable from the application. In addition, other script languages ​​may be used, or an interpreter that interprets and executes the processing contents may be prepared independently. By using the above notation, even if a print setting other than monochrome conversion, which would cause the multiplexed additional information to be lost, is added to the image processing device 102, it is possible to respond at any time.

[0060] In this embodiment, as an example, a case will be described in which a blacklist is stored in a print application and only monochrome print settings are supported. The blacklist may be incorporated in the print application (stored in the secondary storage device 204) or may be directly written as a program.

[0061] When the print setting is changed from "monochrome mode" to "color mode" in S1605, the CPU 201 performs grayscale conversion on the image data to be multiplexed in S1606. In monochrome conversion, printing is normally performed using one color of ink, and the gradation is expressed by light and dark, but in this embodiment, the monochrome conversion of black ink is replaced with grayscale conversion. That is, the multiplexing process executed in the subsequent step S1609 is performed based on the grayscale converted image data. For example, the multiplexing process executed in the subsequent step S1609 may be performed on the grayscale converted image data having gray gradations of 0 to 255, or may be performed on a halftone image in which the pixel values ​​of the image are expressed only in two values ​​of coarseness and density of 255 (white) and 0 (black). However, in this embodiment, the grayscale converted image data and the image data expressed only in two values ​​of coarseness and density are converted into image data of multiple channels such as RGB, and as a result, multiplexing by variation of color components is possible.

[0062] FIG. 17 is a diagram showing an example of an image 1701 expressed by binary density. An image 1703 is an enlarged view of a portion 1702 of the image 1701, and it can be seen that the density of black and white varies depending on the region. Here, for example, a mask 1704 of 4px x 4px is used. The mask 1704 is arranged so as not to overlap with the image region of FIG. 6, and the value of the mask 1704 is compared with the pixel value of the image of FIG. 6. If the pixel value is smaller than the value of the mask 1704, it is converted to a pixel value of 0 (black), and if it is larger, it is converted to a pixel value of 255 (white). Note that, although a pixel value of 255 is expressed as white in the image data, in the image processing device 102, if the recording medium is white, ink is not discharged into the region of the pixel value of 255.

[0063] FIG. 18 is a flowchart showing an example of the process of S1606. In S1801, the CPU 201 judges whether the image data input to the process of S1606 is in a multi-channel format, for example, image data composed of multiple channels such as RGB or YCbCr. If it is judged that the image data is composed of multiple channels, in S1802, the CPU 201 converts the image data into image data composed of a single channel (grayscale conversion). In S1802, for example, RGB image data is converted into monochrome image data or into grayscale image data represented by Y of YCbCr. Then, in S1803, the CPU 201 executes halftone processing using a dither method, an error diffusion method, or the like. On the other hand, if it is judged in S1801 that the image data is composed of a single channel, the halftone processing of S1803 is executed thereafter.

[0064] After S1803, CPU 201 converts the image data generated in S1803 into multi-channel data indicated by color signal values. For example, the image data generated in S1803 is converted into RGB image data corresponding to black (pixel value "0") and white (pixel value "255"). By converting the image data into multi-channel image data in this way, it becomes possible to multiplex additional information by varying color components, which is executed in a later stage. After S1804, the processing of FIG. 18 ends.

[0065] 18, the input image data can be converted into image data expressed only in binary (black and white) density. The size of the ink dots printed by the image processing device 102 is sufficiently smaller than the resolution of the human eye. Therefore, the actual printed matter is perceived by the human eye as black and white shading, not as dot sparseness as in image 1703.

[0066] After S1606, the process proceeds to S1607. Also, if the mode is determined to be "color mode" in S1604, the process of S1607 is executed. In S1607, the CPU 201 determines whether the image size of the image data matches the size required for printing (for example, A4). If it is determined that the image size does not match the size required for printing, in S1608, the CPU 201 executes resizing processing to resize to the size required for printing. This is because if the image is resized in the printer driver or inside the printer, the period of the spectrum intended by the multiplexing embedding changes, which affects the decoding. In particular, if reduction is executed in the resizing, there is a possibility that the multiplexed additional information will be lost because pixels are thinned out. In this embodiment, the image is resized before the multiplexing processing, so that it is possible to prevent the multiplexed additional information from being lost due to the resizing of the image. For the resizing, linear interpolation such as nearest neighbor, bilinear, and bicubic may be used. Also, in this embodiment, the resizing is executed after the grayscale conversion, but the processing may be executed in the reverse order. After S1608, or if it is determined in S1607 that the image size matches the size required for printing, the process proceeds to S1609.

[0067] In S1609, the CPU 201 multiplexes the additional information acquired in S1603, for example, text data, onto the image data processed up to S1607 or S1608. As described above, the multiplexing process is performed on color components, for example, CbCr components after conversion from RGB to YCbCr. By performing the multiplexing process according to the process in FIG. 16, it becomes possible to print in a state that can be decoded by the image processing device 102 while ensuring the print settings intended by the user.

[0068] In S1610, the CPU 201 transmits a print execution instruction to the image processing apparatus 102. An example of XML instructing the print execution is shown below. 001:<?xml version=”1.0” encoding=”UTF-8”?> 002: <cmd> 003: <operation> print< / operation> 004: <data> jpeg base64 data< / data> 005: < / cmd> Third line <operation>The tag "print" indicates printing. <data>Image data is described in the tag. Here, as an example, it is described that a jpeg image in base64 format is to be sent.

[0069] After S1610, the process of Fig. 16 ends. After the process of Fig. 16 ends, the image processing apparatus 102 that has received the transmitted information starts printing.

[0070] As described above, in this embodiment, in the case of print settings that may cause loss of multiplexed additional information, a process that substitutes for the process performed in the print settings is performed, and then multiplexing of the additional information is performed. This prevents loss of multiplexed additional information, and enables printing in a decodable state while maintaining the image quality desired by the user.

[0071] 14 and 16 have been described as being executed in the information processing device 101 in the present embodiment, but they may be executed in the image processing device 102. In that case, for example, image data captured by the information processing device 101 such as a smartphone is acquired as image data to be printed and multiplexed.

[0072] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0073] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0074] 101 information processing device: 102 image processing device: 201 CPU: 202 ROM: 203 RAM< / data> < / operation> < / alternate> < / colormode> < / alternate> < / steganography> < / mode> < / colormode> < / media> < / medialist> < / res> < / cmd>

Claims

1. An acquisition means for acquiring input image data representing an image to be printed; a multiplexing means for converting the input image data into converted image data representing a monochrome image of a plurality of channels when the print setting of the input image data is set to a monochrome mode, and performing a process on the converted image data for multiplexing additional information onto the image to be printed; An information processing device comprising:

2. a first conversion means for converting the input image data into single-channel image data when the input image data is multi-channel image data; a generating means for generating the converted image data based on the single-channel image data converted by the first converting means; The information processing apparatus according to claim 1 , further comprising:

3. a second conversion means for converting the single channel image data into multiple channel image data; The converted image data to be multiplexed by the multiplexing means is the image data of the plurality of channels converted by the second conversion means.

3. The information processing apparatus according to claim 2.

4. 4. The information processing apparatus according to claim 3, wherein said multiplexing means multiplexes said additional information by varying a color component.

5. The image processing apparatus further comprises a first determination means for determining whether or not to convert the input image data into the converted image data when a print setting of the input image data is set to a monochrome mode, When the first determination means determines that the conversion to the converted image data is to be executed, the multiplexing means converts the input image data into the converted image data.

5. The information processing apparatus according to claim 1,

6. 6. The information processing apparatus according to claim 5, wherein the first determination means determines to execute conversion of the input image data into the converted image data on the basis that processing corresponding to a monochrome mode is executed in a printing device.

7. 7. The information processing apparatus according to claim 6, wherein the first determination means determines to execute conversion of the input image data into the converted image data when a user's setting includes a monochrome mode.

8. The information processing device according to claim 6, characterized in that the first determination means determines to perform conversion of the input image data to the converted image data when it determines, based on information acquired from the printing device, that processing corresponding to the monochrome mode is to be performed in the printing device.

9. 9. The information processing apparatus according to claim 8, wherein the first determination unit determines whether or not the printing device is to execute processing corresponding to the monochrome mode, based on the information acquired from the printing device and a blacklist.

10. a second determination unit that determines whether or not to resize the converted image data in accordance with a size of a recording medium on which printing based on the input image data is to be performed; An information processing device according to any one of claims 1 to 9, characterized in that when the second determination means determines that resizing of the converted image data is to be performed, resizing of the converted image data is performed before multiplexing of the additional information by the multiplexing means.

11. An information processing device according to any one of claims 1 to 10, characterized in that when the print setting of the input image data is set to color mode, the additional information is multiplexed onto the input image data without converting the input image data into the converted image data.

12. A display means for displaying a user interface screen, the input image data acquired by the acquisition means is image data selected by a user on the user interface screen; 12. The information processing apparatus according to claim 1,

13. 13. The information processing apparatus according to claim 12, wherein the additional information is information input by a user on the user interface screen.

14. 14. The information processing apparatus according to claim 1, further comprising a printing means.

15. A method executed in an information processing device, comprising: acquiring input image data representing an image to be printed; a multiplexing step of converting the input image data into converted image data representing a monochrome image of a plurality of channels when the print setting of the input image data is set to a monochrome mode, and performing a process on the converted image data for multiplexing additional information onto the image to be printed; The method according to claim 1, further comprising:

16. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 14.

17. A system including an information processing device and a printing device, The information processing device includes: An acquisition means for acquiring input image data representing an image to be printed; a multiplexing means for converting the input image data into converted image data representing a monochrome image of a plurality of channels when the print setting of the input image data is set to a monochrome mode, and performing a process on the converted image data for multiplexing additional information onto the image to be printed; a transmission means for transmitting the image data onto which the additional information has been multiplexed by the multiplexing means to the printing device; Equipped with The printing device includes: printing a multiplexed image on which the additional information is multiplexed based on the image data transmitted by the transmitting means; A system characterized by:

Images