IMAGE PROCESSING APPARATUS, IMAGE PROCESSING SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM

Abstract

To provide an image processing device, an image processing system, an image processing method, and a program capable of accurately converting a captured image to a colorimetric value image.SOLUTION: The image processing device comprises: a specifying part which specifies a color characteristic of a subject in a region included in a captured image captured by an imaging device for imaging the subject; an acquisition part which on the basis of the color characteristic specified by the specifying part, acquires conversion information indicating correspondence between a pixel value of the captured image and a colorimetric value of the subject; an assignment part which assigns the conversion information acquired by the acquisition part to the region included in the captured image; and a conversion part which on the basis of the conversion information assigned to the region included in the captured image, converts the pixel value of the captured image to the colorimetric value.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing system, an image processing method, and a program. [Background technology]

[0002] Various technologies have been proposed to convert the pixel values ​​of an image captured by an imaging device into colorimetric values ​​(coordinate values ​​in a color space) that indicate the color of the subject, thereby making the color of the subject in the captured image closer to the appearance of the actual subject.

[0003] For example, Patent Document 1 listed below discloses a technology in which a transformation matrix for converting pixel values ​​into colorimetric values ​​is calculated based on image data of a standard plate on which a plurality of color chips with different spectral reflectances are arranged, and the transformation matrix is ​​used to convert image data of a subject into colorimetric data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4715288 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional technology, color conversion that converts a captured image into a colorimetric image is applied uniformly to the entire captured image, so the range of possible color characteristics cannot be narrowed, and the accuracy of conversion from pixel values ​​to colorimetric values ​​is reduced. The accuracy of conversion from pixel values ​​to colorimetric values ​​increases as the range of the color characteristics of the subject becomes narrower (more specific).

[0006] In view of the above-mentioned problems, an object of the present invention is to provide an image processing device, an image processing system, an image processing method, and a program capable of converting a captured image into a colorimetric image with high accuracy. [Means for solving the problem]

[0010] Books An image processing device according to one embodiment of the invention comprises an identification unit that identifies the color characteristics of the subject in an area included in an captured image captured by an imaging device that images the subject based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristics identified by the identification unit; an allocation unit that assigns the conversion information acquired by the acquisition unit to an area included in the captured image; and a conversion unit that converts the pixel values ​​of the captured image to the colorimetric values ​​based on the conversion information assigned to the area included in the captured image.When the subject is a printed matter, the design information is a platemaking image, and the identification unit identifies the position of the material of the subject included in the captured image and the color characteristics of the material by aligning the captured image with the platemaking image. An image processing system according to one embodiment of the present invention has an imaging unit that images a subject, an identification unit that identifies the color characteristics of the subject in an area included in an image captured by the imaging unit based on design information indicating color characteristics for each material of the subject, an acquisition unit that acquires conversion information indicating a correspondence between pixel values ​​of the image and colorimetric values ​​of the subject based on the color characteristics identified by the identification unit, an assignment unit that assigns the conversion information acquired by the acquisition unit to an area included in the image, a conversion unit that converts the pixel values ​​of the image into colorimetric values ​​based on the conversion information assigned to the area included in the image, and a display unit that displays a display image based on the colorimetric values, and when the subject is a printed matter, the design information is a platemaking image, and the identification unit identifies the position of the material of the subject included in the image and the color characteristics of the material by aligning the image and the platemaking image. An image processing method according to one embodiment of the present invention includes an identification unit identifying color characteristics of the subject in an area included in an captured image captured by an imaging device that images the subject based on design information indicating color characteristics for each material of the subject; an acquisition unit acquiring conversion information indicating a correspondence between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristics identified by the identification unit; an assignment unit assigning the conversion information acquired by the acquisition unit to an area included in the captured image; and a conversion unit converting the pixel values ​​of the captured image to the colorimetric values ​​based on the conversion information assigned to the area included in the captured image, wherein when the subject is a printed matter, the design information is a platemaking image, and the identification unit aligns the captured image with the platemaking image to identify the position of the material of the subject included in the captured image and the color characteristics of the material. A program according to one aspect of the present invention causes a computer to function as an identification unit that identifies color characteristics of the subject in an area included in an captured image captured by an imaging device that images the subject based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristics identified by the identification unit; an allocation unit that assigns the conversion information acquired by the acquisition unit to an area included in the captured image; and a conversion unit that converts the pixel values ​​of the captured image into the colorimetric values ​​based on the conversion information assigned to the area included in the captured image, and when the subject is a printed matter, the design information is a platemaking image, and the identification unit identifies the position of the material of the subject included in the captured image and the color characteristics of the material by aligning the captured image with the platemaking image. Effect of the Invention

[0011] According to the present invention, a captured image can be converted into a colorimetric image with high accuracy. [Brief description of the drawings]

[0012] [Figure 1]1 is a diagram illustrating an example of a configuration of an image processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of a configuration of an image processing device according to a first embodiment. [Diagram 3] 4 is a diagram showing an example of a configuration of a profile table in a profile DB according to the first embodiment; FIG. [Figure 4] 5 is a flowchart showing an example of a flow of conversion of a captured image according to the first embodiment. [Diagram 5] FIG. 11 is a diagram showing an example of identification information according to the second embodiment. [Figure 6] FIG. 11 is a diagram showing an example of the configuration of a profile table in a profile DB according to the second embodiment. [Figure 7] FIG. 13 is a diagram showing an example of identification information according to the third embodiment. [Figure 8] FIG. 13 is a diagram showing an example of the configuration of a profile table in a profile DB according to the third embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a relationship between a captured image and design information according to the fourth embodiment. [Figure 10] FIG. 13 is a diagram showing an example of the configuration of a profile table in a profile DB according to the fourth embodiment. [Figure 11] 13 is a flowchart showing an example of a flow of conversion of a captured image according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present invention relates to an image processing system capable of converting pixel values ​​of an image of a subject into colorimetric values ​​of the subject with high accuracy. In the following embodiments, an example in which the subject is a printed matter will be described as an example. Note that the subject is not limited to this example. Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings.

[0014] <<1. First embodiment>> A first embodiment of the present invention will be described with reference to Figures 1 to 4. In the first embodiment, an example will be described in which color characteristics of a printed material are specified based on information preset by a user.

[0015] <1-1. Image processing system configuration> The configuration of an image processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an image processing system according to the first embodiment. As shown in FIG. 1, the image processing system 1 includes an imaging device 10, an image processing device 20, and a display device 30.

[0016] The imaging device 10 and the image processing device 20 are connected to each other so as to be able to communicate with each other. For example, the imaging device 10 and the image processing device 20 are connected to each other via a local area network (LAN). The LAN may be realized by a wired connection, or may be connected by a wireless connection such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). The image processing device 20 and the display device 30 are also connected to each other so as to be able to communicate with each other.

[0017] The imaging device 10 is a device that captures an image of a subject. One example of the imaging device 10 is a camera. As shown in FIG. 1 , the imaging device 10 of this embodiment captures an image of a printed matter 51 placed on an imaging stand 50. The printed matter 51 is an example of a "subject." The printed matter 51 is illuminated by a light 40. The imaging device 10 transmits the captured image obtained by capturing an image of the printed matter 51 to the image processing device 20.

[0018] The printed matter 51 of this embodiment is a medium such as paper on which characters, numbers, symbols, figures, images, etc. are printed with ink. Printed area 52a, printed area 52b, printed area 52c, printed area 52d, and printed area 52e in the printed matter 51 indicate areas printed with ink.

[0019] The image processing device 20 is a device that performs image processing. Examples of the image processing device 20 include a PC (Personal Computer), a server, a smartphone, and a tablet terminal. The image processing device 20 of this embodiment performs a conversion process for converting pixel values ​​into colorimetric values ​​for a captured image received from the imaging device 10. In this way, the image processing device 20 converts the captured image into a colorimetric image. Here, the colorimetric image may be an image in which pixel values ​​directly indicate colorimetric values ​​such as XYZ or CIELAB, or an image in a color space in which the correspondence between pixel values ​​and colorimetric values ​​is uniquely determined, such as sRGB or AdobeRGB. The image processing device 20 transmits to the display device 30 a display image that has been converted from the colorimetric image obtained by the conversion process according to the display characteristics of the display device 30.

[0020] The display device 30 is a device that displays a display image based on color measurement values. The display device 30 is a display unit in the image processing system 1. An example of the display device 30 is a display. The display device 30 may be a display included in the image processing device 20, or may be an external display connected to the image processing device 20 by wire or wirelessly. When the display device 30 is an external display, the location where the display device 30 is provided is not particularly limited. For example, the display device 30 may be provided in a remote location different from the location where the subject is imaged.

[0021] <1-2. Configuration of image processing device> The configuration of the image processing device 20 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the image processing device 20 according to the first embodiment. As shown in FIG. 2, the image processing device 20 includes a communication unit 200, a control unit 210, and a storage unit 220.

[0022] (1) Communications Department 200 The communication unit 200 communicates with other devices. The communication unit 200 transmits and receives various information through the communication. For example, in communication with the imaging device 10, the communication unit 200 receives a captured image from the imaging device 10. In communication with the display device 30, the communication unit 200 transmits a colorimetric value image to the display device 30.

[0023] (2) Control unit 210 The control unit 210 has a function of controlling the overall operation of the image processing device 20. The control unit 210 is realized by causing a CPU (Central Processing Unit) provided as hardware in the image processing device 20 to execute a program. As shown in FIG. 2, the control unit 210 includes an identifying unit 2102, an acquiring unit 2104, an allocating unit 2106, a converting unit 2108, and an output processing unit 2110.

[0024] (2-1)Specific section 2102 The identification unit 2102 identifies color characteristics of the printed matter 51 in an area included in the captured image. The color characteristics of the printed matter 51 are, for example, the spectral reflectance of the material of the printed matter 51. Examples of the material of the printed matter 51 include a medium on which printing is performed, ink, etc. The color characteristics of the printed matter 51 can change depending on a combination of the type of ink, printing method, etc. Examples of the type of ink include spot color ink, process ink, etc. Examples of printing methods include letterpress printing, offset printing, gravure printing, screen printing, inkjet method, etc. Hereinafter, information indicating a combination of the type of ink, printing method, etc. of the printed matter 51 is also referred to as "color characteristic information."

[0025] In this embodiment, the specifying unit 2102 refers to color characteristic information set in advance by a user to specify the color characteristics of the printed matter 51. The specifying unit 2102 inputs the specified color characteristics of the printed matter 51 to the acquiring unit 2104.

[0026] (2-2) Acquisition section 2104 The acquisition unit 2104 acquires a profile (conversion information). The profile is information for converting a captured image into a colorimetric image, and includes information indicating a correspondence relationship between pixel values ​​of the captured image and colorimetric values ​​of the printed matter 51. The correspondence relationship between pixel values ​​of the captured image and colorimetric values ​​of the printed matter 51 depends on, for example, the color characteristics of the printed matter 51. Therefore, the profile is prepared according to the color characteristics of the printed matter 51.

[0027] The correspondence between the pixel values ​​of the captured image and the colorimetric values ​​of the printed matter 51 may also depend on the characteristics of the imaging device 10. The characteristics of the imaging device 10 (e.g., spectral sensitivity) may vary depending on the type of imaging device 10, for example. Furthermore, the characteristics of imaging devices 10 may vary from device to device even if they are of the same type. For this reason, it is preferable that a profile is prepared according to the characteristics of the imaging device 10. A profile prepared according to the characteristics of the imaging device 10 will hereinafter also be referred to as a "camera profile." There may be as many camera profiles as there are combinations of the characteristics of the imaging device 10 and the color characteristics of the printed matter 51.

[0028] The correspondence between pixel values ​​and colorimetric values ​​indicated by the camera profile is acquired based on pixel values ​​of an image of a color chart captured by the imaging device 10 in advance in the same imaging environment as the imaging environment in which the imaging device 10 captures the printout 51. The camera profile may include correspondence between pixel values ​​and colorimetric values ​​that are not included when the color chart is captured. When a CLUT (Color Look-Up Table) is used as the format of the camera profile, interpolation using a model formula may be performed on the colorimetric values ​​at each RGB grid point. When color conversion is performed, the interpolation process is performed by referring to the CLUT values.

[0029] When the colorimetric image is displayed on the display device 30, the color of the displayed colorimetric image may depend on the characteristics of the display device 30. Therefore, if the colorimetric image converted by the camera profile is displayed on the display device 30 as it is, a difference may occur between the appearance of the printed matter 51 in the displayed colorimetric image and the appearance of the actual printed matter 51. Therefore, by further preparing a profile according to the characteristics of the display device 30 and applying the profile to the colorimetric image before displaying it on the display device 30, the difference can be reduced. The characteristics of the display device 30 may differ, for example, depending on the type of the display device 30. Furthermore, the characteristics of the display device 30 may differ for each device even if they are the same type. Therefore, it is preferable that the profile is prepared according to the characteristics of the display device 30. The profile prepared according to the characteristics of the display device 30 is hereinafter also referred to as a "display profile". There may be as many display profiles as there are characteristics of the display device 30.

[0030] In this embodiment, each profile is prepared in advance by a user and stored in the storage unit 220 described later. The acquisition unit 2104 acquires a profile from the storage unit 220 based on, for example, color characteristics input from the specification unit 2102. The acquisition unit 2104 inputs the acquired profile to the allocation unit 2106.

[0031] (2-3) Allocation section 2106 The allocation unit 2106 allocates a profile to a target image. For example, when a camera profile is input from the acquisition unit 2104, the allocation unit 2106 allocates the camera profile to an area included in the captured image. The allocation unit 2106 allocates the display profile input from the acquisition unit 2104 to the colorimetric image. The allocation unit 2106 inputs the image to which each profile is allocated to the conversion unit 2108. The allocation unit 2106 may allocate one camera profile to the entire captured image, or may allocate a camera profile to each area in the captured image. When the allocation unit 2106 allocates a camera profile to each area in the captured image, the allocation unit 2106 may allocate different camera profiles to each area, or may allocate the same camera profile to some areas of each area. Moreover, the area in the captured image may include the entire captured image.

[0032] (2-4) Conversion unit 2108 The conversion unit 2108 performs image conversion processing based on the profile. For example, when a captured image to which a camera profile is assigned is input, the conversion unit 2108 converts pixel values ​​of the captured image into colorimetric values ​​based on the camera profile assigned to an area included in the captured image. In this way, the conversion unit 2108 converts the captured image into a colorimetric value image.

[0033] The conversion unit 2108 converts the colorimetric values ​​of the colorimetric image into colorimetric values ​​suitable for the display device 30 based on the display profile assigned to the colorimetric image. In this way, the conversion unit 2108 converts the colorimetric image into a display image. The conversion unit 2108 inputs the display image to the output processing unit 2110.

[0034] The conversion unit 2108 performs conversion processing using the profile, thereby making it possible to reproduce the color of the subject in the image closer to the appearance of the actual subject. When the display device 30 displays the captured image captured by the imaging device 10 as in this embodiment, the image processing device 20 performs conversion processing using the camera profile and conversion processing using the display profile on the captured image. This allows the display device 30 to display an image of the printed matter 51 in which the color characteristics of the imaging device 10 and the display device 30 are taken into consideration.

[0035] (2-5) Output processing unit 2110 The output processing unit 2110 controls the output of a display image. For example, the output processing unit 2110 transmits the display image input from the conversion unit 2108 from the image processing device 20 to the display device 30 via the communication unit 200. When the image processing device 20 has a display unit, the output processing unit 2110 may display the display image input from the conversion unit 2108 on the display unit.

[0036] (3) Storage section 220 The storage unit 220 is configured by a storage medium, such as a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 220 can use, for example, a non-volatile memory.

[0037] The storage unit 220 has a function of storing various information. The storage unit 220 stores, for example, various profiles. The various information is stored in a DB (database). For example, the various profiles are stored in a profile DB.

[0038] Here, an example of a profile stored in the profile DB will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of a configuration of a profile table in the profile DB according to the first embodiment. As shown in Fig. 3, a record in the profile table includes a profile name, a color characteristic, a pixel value, and a colorimetric value. Note that the information included in the record in the profile table is not limited to this example.

[0039] The profile name is information indicating the name of the profile for each imaging device 10 or display device 30. The color characteristics is information indicating the color characteristics of the printout 51. The pixel value is information indicating the pixel value in the captured image and is indicated by (R, G, B). The colorimetric value is information indicating the value after conversion of the pixel value of the captured image and is indicated by (X, Y, Z). Note that the R, G, and B pixel values ​​correspond to the X, Y, and Z colorimetric values, respectively. FIG. 3 shows, as examples of profiles, camera profile 1 and camera profile 2, which are profiles according to the characteristics of the camera.

[0040] The camera profile 1 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristics of the printed matter 51 are "ink 1" and "printing method 1." In this correspondence relationship, 1-1 ,G 1-1 ,B 1-1 ) is the colorimetric value (X 1-1 ,Y 1-1 ,Z 1-1 ), pixel value (R 1-2 ,G 1-2 ,B 1-2 ) is the colorimetric value (X 1-2 ,Y 1-2 ,Z 1-2 ) respectively.

[0041] The camera profile 2 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristics of the printed matter 51 are "ink 2" and "printing method 2." In this correspondence relationship, 2-1 ,G 2-1 ,B 2-1 ) is the colorimetric value (X 2-1 ,Y2-1 ,Z 2-1 ), pixel value (R 2-2 ,G 2-2 ,B 2-2 ) is the colorimetric value (X 2-2 ,Y 2-2 ,Z 2-2 ) respectively.

[0042] <1-3. Processing flow> A flow of captured image conversion according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of a flow of captured image conversion according to the first embodiment.

[0043] As shown in FIG. 4, first, the user places the printed matter 51 on the photographing stand 50 (S102).

[0044] Next, the image processing device 20 acquires a captured image of the printed matter 51 placed on the photographing stand 50 (S104). The printed matter 51 is captured by, for example, operating the imaging device 10 by the user.

[0045] Next, the identifying unit 2102 of the image processing device 20 identifies the color characteristics of the printed matter 51 included in the captured image (S106). For example, the identifying unit 2102 identifies the color characteristics set by the user as the color characteristics of the printed matter 51.

[0046] Next, the acquisition unit 2104 of the image processing device 20 acquires a profile from the storage unit 220 based on the identified color characteristics (S108). For example, when the color characteristics are “ink 1” and “printing method 1”, the acquisition unit 2104 acquires camera profile 1 from the profile DB of the storage unit 220.

[0047] Next, the allocation unit 2106 of the image processing device 20 allocates the acquired profile to the captured image (S110).

[0048] Next, the conversion unit 2108 of the image processing device 20 performs conversion processing based on the assigned profile (S112).

[0049] Finally, the display device 30 displays the display image obtained by the conversion process of the conversion unit 2108 (S114).

[0050] <<2. Second embodiment>> The first embodiment of the present invention has been described above. Next, a second embodiment of the present invention will be described with reference to Figs. 5 and 6. In the second embodiment, an example will be described in which the color characteristics of a printed matter included in a captured image are identified based on identification information associated with the color characteristics of the printed matter. Note that the identification information in the second embodiment is identification information (first identification information) that is assigned to the printed matter. Note that, in the following, descriptions that overlap with the first embodiment described above will be omitted.

[0051] Here, an example of the identification information according to the second embodiment will be described with reference to FIG. 5. FIG. 5 is a diagram showing an example of the identification information according to the second embodiment. As shown in FIG. 5, identification information 53 is printed (applied) on the printed matter 51. Examples of the identification information printed on the printed matter 51 include a one-dimensional code, a two-dimensional code, etc. that can be read by the imaging device 10. The one-dimensional code is, for example, a one-dimensional barcode. The two-dimensional code is, for example, a two-dimensional barcode. The identification information is associated with the color characteristics of the printed matter 51. As an example, the identification information is associated with the color characteristics of each material of the printed matter 51 and the position of the material in the captured image. Here, the material refers to the medium or ink used in the printed matter 51.

[0052] <2-1. Image processing system configuration> The configuration of the image processing system according to the second embodiment is similar to the configuration of the image processing system 1 according to the first embodiment described above, and therefore detailed description will be omitted. Hereinafter, the same reference numerals as those of the image processing system 1 according to the first embodiment will be used for the configuration of the image processing system according to the second embodiment.

[0053] <2-2. Configuration of image processing device> The configuration of the image processing device 20 according to the second embodiment is similar to the configuration of the image processing device 20 according to the first embodiment described above. Hereinafter, the same reference numerals as those of the image processing device 20 according to the first embodiment are used for the configuration of the image processing device 20 according to the second embodiment. Note that in the image processing device 20 according to the second embodiment, part of the functions of the identification unit 2102 and part of the configuration of the profile table of the profile DB are different from those of the first embodiment. The differences from the first embodiment will be described below.

[0054] The identifying unit 2102 of the second embodiment identifies color characteristics of the printed matter 51 included in the captured image based on the identification information 53 included in the captured image. For example, the identifying unit 2102 identifies color characteristics associated with the identification information 53 as color characteristics of the printed matter 51. When color characteristics for each material of the printed matter 51 are associated with the identification information 53, the identifying unit 2102 can identify color characteristics for each material of the printed matter 51. Furthermore, when the position of the material of the printed matter 51 is associated with the identification information 53, the identifying unit 2102 can identify the position of the material of the printed matter 51.

[0055] The identification unit 2102 identifies the color characteristics for each material of the printed matter 51, and the acquisition unit 2104 can acquire a profile for each material of the printed matter 51. Furthermore, the allocation unit 2106 can allocate a profile for each material of the printed matter 51 to the captured image. This allows the conversion unit 2108 to convert the pixel values ​​of the captured image for each material in accordance with the color characteristics of the material. This allows the conversion unit 2108 of the second embodiment to perform conversion processing with higher accuracy than the conversion unit 2108 of the first embodiment.

[0056] Here, an example of a profile stored in the profile DB will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a configuration of a profile table in the profile DB according to the second embodiment. As shown in Fig. 6, a record in the profile table includes a profile name, identification information, color characteristics, pixel values, and colorimetric values. Note that the information included in the record in the profile table is not limited to this example.

[0057] The identification information of the profile table is information indicating the identification information associated with the color characteristics. In Fig. 6, camera profile 3 and camera profile 4, which are profiles according to the characteristics of the camera, are shown as examples of the profile.

[0058] Camera profile 3 is associated with identification information "2D barcode 1," color characteristics "ink 3," and "printing method 3." Furthermore, the camera profile 3 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristics of the printed matter 51 are "ink 3" and "printing method 3." In this correspondence relationship, 3-1 ,G 3-1 ,B 3-1 ) is the colorimetric value (X 3-1 ,Y 3-1 ,Z 3-1 ), pixel value (R 3-2 ,G 3-2 ,B 3-2 ) is the colorimetric value (X 3-2 ,Y 3-2 ,Z 3-2 ) respectively.

[0059] The camera profile 4 is associated with the identification information "2D barcode 2", the color characteristics "ink 4", and "printing method 4". Furthermore, the camera profile 4 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristics of the printed matter 51 are "ink 4" and "printing method 4." In this correspondence relationship, 4-1 ,G 4-1 ,B 4-1) is the colorimetric value (X 4-1 ,Y 4-1 ,Z 4-1 ), pixel value (R 4-2 ,G 4-2 ,B 4-2 ) is the colorimetric value (X 4-2 ,Y 4-2 ,Z 4-2 ) respectively.

[0060] <2-3. Flow of conversion of captured images> The flow of conversion of a captured image according to the second embodiment is similar to the flow of conversion of a captured image according to the first embodiment described above, and therefore a detailed description thereof will be omitted.

[0061] <<3. Third embodiment>> The second embodiment of the present invention has been described above. Next, a third embodiment of the present invention will be described with reference to Figs. 7 and 8. In the third embodiment, an example will be described in which the color characteristics of a printed matter included in a captured image are identified based on identification information associated with the color characteristics of the printed matter. Note that the identification information in the third embodiment is identification information (second identification information) that is given to the imaging environment in which the imaging device 10 captures the printed matter 51. An example of the imaging environment is a shooting stand. Note that, in the following, descriptions that overlap with the first and second embodiments described above will be omitted.

[0062] Here, an example of the identification information according to the third embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of the identification information according to the third embodiment. In the example shown in FIG. 7, the shooting stand 50 is divided into an area 58 and an area 59. A label 56 indicated as "printed matter storage area" is attached to the area 58 as an example of the identification information. A printed matter 51 is placed in the area 58 to which the label 56 is attached. A label 57 indicated as "sample storage area" is attached to the area 59 as an example of the identification information. A sample 54 of the printed matter is placed in the area 59 to which the label 57 is attached.

[0063] In the third embodiment, the identification information associates the color characteristics of the subject with the position of the subject in the captured image. The subject includes both the printed matter 51 and the sample 54 shown in Fig. 7. Therefore, when a captured image includes multiple subjects, the image processing device 20 can distinguish between the subjects and identify the color characteristics corresponding to each subject based on the identification information.

[0064] <3-1. Image processing system configuration> The configuration of the image processing system according to the third embodiment is similar to the configuration of the image processing system 1 according to the first and second embodiments described above, and therefore detailed description thereof will be omitted. Hereinafter, the same reference numerals as those of the image processing system 1 according to the first and second embodiments will be used for the configuration of the image processing system according to the third embodiment.

[0065] <3-2. Configuration of image processing device> The configuration of the image processing device 20 according to the third embodiment is similar to the configuration of the image processing device 20 according to the first and second embodiments described above. Hereinafter, the same reference numerals as those of the image processing device 20 according to the first and second embodiments are used for the configuration of the image processing device 20 according to the third embodiment. Note that in the image processing device 20 according to the third embodiment, part of the function of the identification unit 2102 and part of the configuration of the profile table of the profile DB are different from those of the first and second embodiments. The differences from the first and second embodiments will be described below.

[0066] The identifying unit 2102 of the third embodiment identifies the position and color characteristics of the subject included in the captured image based on the identification information (second identification information) included in the captured image. For example, in the example shown in Fig. 7, the label 56 is associated with the color characteristics of the printed matter 51 and the position of the printed matter 51 (for example, the area 58 to which the label 56 is attached). This allows the image processing device 20 to recognize the label 56 included in the captured image, determine that the subject placed in the area 58 to which the label 56 is attached is the printed matter 51, and identify the color characteristics of the printed matter 51.

[0067] 7, the label 57 is associated with the color characteristic of the sample 54 and the position of the sample 54 (for example, the area 59 to which the label 57 is attached). As a result, the image processing device 20 can recognize the label 57 included in the captured image, determine that the subject placed in the area 59 to which the label 57 is attached is the sample 54, and identify the color characteristic of the sample 54.

[0068] In the third embodiment, a captured image including both the printed matter 51 and the sample 54 may be divided into an image including only the printed matter 51 and an image including only the sample 54, and then the color characteristics of each of the printed matter 51 and the sample 54 may be identified by the above-mentioned method. The conversion process may also be performed on each image.

[0069] Here, an example of a profile stored in the profile DB will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the configuration of a profile table in the profile DB according to the third embodiment. As shown in Fig. 8, the records of the profile table include a profile name, identification information, position information, color characteristics, pixel values, and colorimetric values. Note that the information included in the records of the profile table is not limited to this example.

[0070] The position information in the profile table is information that indicates the position of the subject in the captured image. Camera profile 5 and camera profile 6, which are profiles according to the characteristics of the camera, are shown as examples of the profile in Fig. 8.

[0071] Camera profile 5 is associated with the identification information "printed matter storage label" (label 56), the location information "area 58", and the color characteristics "ink 5" and "printing method 5". This indicates that the color characteristics of the subject placed in "area 58" to which the "printed matter storage label" is attached are "ink 5" and "printing method 5". Furthermore, the camera profile 5 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristics of the subject are "ink 5" and "printing method 5." In this correspondence relationship, 5-1 ,G 5-1 ,B 5-1 ) is the colorimetric value (X 5-1 ,Y 5-1 ,Z 5-1 ), pixel value (R 5-2 ,G 5-2 ,B 5-2 ) is the colorimetric value (X 5-2 ,Y 5-2 ,Z 5-2 ) respectively.

[0072] The camera profile 6 is associated with the identification information "sample location label" (label 57), the location information "area 59", and the color characteristics "ink 6" and "printing method 6". This indicates that the color characteristics of the subject placed in "area 59" to which the "sample location label" is attached are "ink 6" and "printing method 6". Furthermore, the camera profile 6 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristics of the subject are "ink 6" and "printing method 6." In this correspondence relationship, 6-1 ,G 6-1 ,B 6-1 ) is the colorimetric value (X 6-1 ,Y 6-1 ,Z 6-1 ), pixel value (R 6-2 ,G 6-2 ,B 6-2 ) is the colorimetric value (X 6-2 ,Y 6-2 ,Z 6-2 ) respectively.

[0073] <3-3. Flow of conversion of captured images> The flow of conversion of a captured image according to the third embodiment is similar to the flow of conversion of a captured image according to the first and second embodiments described above, and therefore a detailed description thereof will be omitted.

[0074] <<4. Fourth embodiment>> The third embodiment of the present invention has been described above. Next, a fourth embodiment of the present invention will be described with reference to Figs. 9 to 11. In the fourth embodiment, an example will be described in which the color characteristics of a subject included in a captured image are identified based on design information indicating the color characteristics of each material of the subject. When the subject is a printed matter 51, the design information is, for example, a platemaking image of the printed matter 51. Note that the design information of the subject is not limited to this example. Note that, hereinafter, descriptions that overlap with the first to third embodiments described above will be omitted.

[0075] Here, an example of the relationship between a captured image and design information according to the fourth embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the relationship between a captured image and design information according to the fourth embodiment. Fig. 9 shows a platemaking image 70 of a printed matter 51 as an example of design information.

[0076] The platemaking image 70 shows a printing area when printing the printed matter 51. The printing area is associated with, for example, the color characteristics of the material used in the printing area when printing the printed matter 51. As an example, as shown in Fig. 9, the printing area 71a of the platemaking image 70 is associated with spot ink 1, the printing area 71b with spot ink 2, the printing area 71c with process ink 3, the printing area 71d with process ink 2, and the printing area 71e with process ink 3.

[0077] The image processing device 20 aligns the captured image 60 including the printed matter 51 with the platemaking image 70, and, assuming that the color characteristics at corresponding positions are the same, identifies the color characteristics for each material of the printed matter 51. For example, in the example shown in Fig. 9, the printing area 52a, the printing area 52b, the printing area 52c, the printing area 52d, and the printing area 52e of the printed matter 51 correspond to the printing area 71a, the printing area 71b, the printing area 71c, the printing area 71d, and the printing area 71e of the platemaking image 70, respectively. Therefore, the image processing device 20 can identify that the color characteristics of printing area 52a, printing area 52b, printing area 52c, printing area 52d, and printing area 52e are, respectively, the color characteristics of spot ink 1, the color characteristics of spot ink 2, the color characteristics of process ink 1, the color characteristics of process ink 2, and the color characteristics of process ink 3.

[0078] <4-1. Image processing system configuration> The configuration of the image processing system according to the fourth embodiment is similar to the configuration of the image processing system 1 according to the first to third embodiments described above, and therefore detailed description will be omitted. Hereinafter, the same reference numerals as those of the image processing system 1 according to the first to third embodiments will be used for the configuration of the image processing system according to the fourth embodiment.

[0079] <4-2. Configuration of image processing device> The configuration of the image processing device 20 according to the fourth embodiment is similar to the configuration of the image processing device 20 according to the first to third embodiments described above. Hereinafter, the same reference numerals as those of the image processing device 20 according to the first to third embodiments will be used for the configuration of the image processing device 20 according to the fourth embodiment. Note that in the image processing device 20 according to the fourth embodiment, part of the functions of the identification unit 2102 and part of the configuration of the profile table of the profile DB are different from those of the first to third embodiments. The differences from the first to third embodiments will be described below.

[0080] The identifying unit 2102 of the fourth embodiment identifies color characteristics of a subject included in a captured image based on a platemaking image of the printed matter 51. For example, the identifying unit 2102 aligns the position of the printed matter 51 included in the captured image 60 with the platemaking image 70 to identify the position of the material of the printed matter 51 and the color characteristics of the material.

[0081] The identification unit 2102 identifies the color characteristics for each material of the printed matter 51, and the acquisition unit 2104 can acquire a profile for each material of the printed matter 51. In addition, the identification unit 2102 also identifies the position of each material of the printed matter 51 in the captured image 60, and the allocation unit 2106 can allocate a profile with high accuracy to each material of the printed matter 51 in the captured image based on the position. This also improves the accuracy of the conversion process in the conversion unit 2108.

[0082] Here, an example of a profile stored in the profile DB will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the configuration of a profile table in the profile DB according to the fourth embodiment. As shown in Fig. 10, the records of the profile table include a profile name, a print area, color characteristics, pixel values, and colorimetric values. Note that the information included in the records of the profile table is not limited to this example.

[0083] The position information in the profile table is information that indicates the position of each material in the platemaking image 70. In Fig. 10, as examples of profiles, camera profile 7, camera profile 8, camera profile 9, camera profile 10, and camera profile 11, which are profiles according to the characteristics of the camera, are shown.

[0084] The position information "printing area 71a" and the color characteristic "spot ink 1" are associated with the camera profile 7. This indicates that the color characteristic of the "printing area 71a" of the platemaking image 70 is "spot ink 1." Furthermore, the camera profile 7 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristic is "spot color ink 1." In this correspondence relationship, 7-1 ,G 7-1 ,B 7-1 ) is the colorimetric value (X 7-1 ,Y 7-1 ,Z 7-1 ) has been shown to be converted to

[0085] The position information "printing area 71b" and the color characteristic "spot ink 2" are associated with the camera profile 8. This indicates that the color characteristic of the "printing area 71b" of the platemaking image 70 is "spot ink 2." Furthermore, the camera profile 8 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristic is "spot color ink 2." In this correspondence relationship, 8-1 ,G 8-1 ,B 8-1 ) is the colorimetric value (X 8-1 ,Y 8-1 ,Z 8-1 ) has been shown to be converted to

[0086] The position information "print area 71c" and the color characteristic "process ink 1" are associated with the camera profile 9. This indicates that the color characteristic of the "print area 71c" of the platemaking image 70 is "process ink 1." Furthermore, the camera profile 9 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristic is "Process Ink 1." In this correspondence relationship, 9-1 ,G 9-1 ,B 9-1 ) is the colorimetric value (X 9-1 ,Y 9-1 ,Z 9-1 ) has been shown to be converted to

[0087] The position information "printing area 71d" and the color characteristic "process ink 2" are associated with the camera profile 10. This indicates that the color characteristic of the "printing area 71d" of the platemaking image 70 is "process ink 2." Furthermore, the camera profile 10 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristic is "Process Ink 2". In this correspondence relationship, 10-1 ,G 10-1 ,B 10-1 ) is the colorimetric value (X 10-1 ,Y 10-1 ,Z 10-1 ) has been shown to be converted to

[0088] The position information "printing area 71e" and the color characteristic "process ink 3" are associated with the camera profile 11. This indicates that the color characteristic of the "printing area 71e" of the platemaking image 70 is "process ink 3." Furthermore, the camera profile 11 is associated with a correspondence relationship between pixel values ​​and colorimetric values ​​when the color characteristic is "Process Ink 3". In this correspondence relationship, 11-1 ,G 11-1 ,B 11-1 ) is the colorimetric value (X 11-1 ,Y 11-1 ,Z 11-1 ) has been shown to be converted to

[0089] <4-3. Flow of conversion of captured images> A flow of captured image conversion according to the fourth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of a flow of captured image conversion according to the fourth embodiment.

[0090] As shown in FIG. 11, first, the user places the printed matter 51 on the photographing stand 50 (S202).

[0091] Next, the image processing device 20 acquires a captured image of the printed matter 51 placed on the photographing stand 50 (S204). The printed matter 51 is photographed, for example, by the user operating the imaging device .

[0092] Next, the specifying unit 2102 of the image processing device 20 aligns the captured image 60 with the platemaking image 70 (S206). Based on the alignment result, the specifying unit 2102 specifies color characteristics for each printing area of ​​the printout 51 included in the captured image (S208).

[0093] Next, the acquisition unit 2104 of the image processing device 20 acquires a profile from the storage unit 220 based on the identified color characteristics (S210). For example, when the color characteristics are “spot color ink 1”, the acquisition unit 2104 acquires camera profile 7 from the profile DB of the storage unit 220.

[0094] Next, the allocation unit 2106 of the image processing device 20 allocates the acquired profile to the captured image (S212).

[0095] Next, the conversion unit 2108 of the image processing device 20 performs conversion processing based on the assigned profile (S214).

[0096] Finally, the display device 30 displays the display image obtained by the conversion process of the conversion unit 2108 (S216).

[0097] <<5. Modifications>> The fourth embodiment of the present invention has been described above. Next, modified examples of the present invention will be described. Note that the modified examples described below may be applied in place of the configurations described in the respective embodiments of the present invention, or may be applied in addition to the configurations described in the respective embodiments of the present invention.

[0098] In the above-described embodiments, the color characteristics of the printed matter 51 in the area included in the captured image are identified by the function of the identification unit 2102, but the present invention is not limited to such an example. For example, the color characteristics of the printed matter 51 may be identified based on machine learning. An example of an algorithm used for machine learning is semantic segmentation. Semantic segmentation is a deep learning algorithm that associates a label or a category with every pixel in an image. By using this algorithm, for example, when a plurality of subjects exist in an area included in the captured image, it is possible to simultaneously perform area division for each subject and identification of the material of each subject. In addition, by using this algorithm, it is possible to convert pixel values ​​to colorimetric values ​​with high accuracy even in the case of an image of the material of clothing or an image captured in a room where various subjects may exist.

[0099] The above describes the modified examples of the present invention. As described above, the specifying unit 2102 of the image processing device 20 according to each embodiment specifies color characteristics of the printout 51 in an area included in a captured image captured by the imaging device 10 that captures the printout 51. The acquiring unit 2104 of the image processing device 20 acquires a profile (conversion information) indicating the correspondence between the pixel values ​​of the captured image and the colorimetric values ​​of the subject, based on the color characteristics identified by the identifying unit 2102. The allocation unit 2106 of the image processing device 20 allocates the profile acquired by the acquisition unit 2104 to an area included in the captured image. The conversion unit 2108 of the image processing device 20 converts pixel values ​​of the captured image into colorimetric values ​​based on a profile assigned to an area included in the captured image.

[0100] With this configuration, the image processing device 20 according to each embodiment can convert pixel values ​​of a captured image into colorimetric values ​​after specifying (narrowing) the range of color characteristics of the subject, the printed matter 51. As a result, the accuracy of conversion from pixel values ​​to colorimetric values ​​in the present invention is improved compared to the accuracy of conventional methods in which color conversion is applied uniformly to the entire captured image.

[0101] Therefore, the image processing device 20 according to each embodiment can convert a captured image into a colorimetric image with high accuracy.

[0102] The above describes the embodiments of the present invention. Note that each of the above-described embodiments may be implemented independently as an embodiment of the present invention, or may be implemented in combination as one embodiment of the present invention.

[0103] The image processing device 20 in each of the above-mentioned embodiments may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0104] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0105] 1. Image Processing System 10. Imaging device 20 Image processing device 30 Display device 40 Lighting 50 Photo stand 51 Printed matter 53 Identification information 54 Sample 56 Label 57 Label 60 Captured images 70 Plate making images 200 Communications Department 210 Control section 220 Storage section 2102 Specific section 2104 Acquisition Department 2106 Allocation Department 2108 Conversion unit 2110 Output processing section

Claims

1. an identification unit that identifies color characteristics of the subject in a region included in a captured image captured by an imaging device that images the subject, based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence relationship between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristic identified by the identification unit; an allocation unit that allocates the conversion information acquired by the acquisition unit to an area included in the captured image; a conversion unit that converts the pixel values ​​of the captured image into the colorimetric values ​​based on the conversion information assigned to an area included in the captured image; Equipped with When the subject is a printed matter, the design information is a platemaking image. The identification unit identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image.

2. The specifying unit specifies the color characteristic for each of a plurality of regions to be converted that are included in the region of the subject, the acquisition unit acquires the conversion information for each of the plurality of regions based on the color characteristics identified for each of the plurality of regions; the allocating unit allocates the conversion information acquired for each of the plurality of regions to each of the plurality of regions; the conversion unit converts the pixel values ​​of the captured image into the colorimetric values ​​based on the conversion information assigned to each of the plurality of regions. The image processing device according to claim 1 .

3. The identification unit identifies color characteristics of the subject included in the captured image based on identification information associated with the color characteristics of the subject. The image processing device according to claim 1 .

4. the identification information is first identification information given to the subject, The identification unit identifies color characteristics of a subject included in the captured image based on the first identification information included in the captured image. The image processing device according to claim 3 .

5. the identification information is second identification information assigned to an imaging environment in which the imaging device images the subject, The identification unit identifies color characteristics of a subject included in the captured image based on the second identification information included in the captured image. The image processing device according to claim 3 .

6. the identification information is further associated with a color characteristic of the object and a position of the object in the captured image; The identification unit identifies color characteristics and a position of a subject included in the captured image based on the identification information. The image processing device according to any one of claims 3 to 5.

7. the identification unit identifies a set of spectral reflectances to which the material of the subject belongs as the spectral reflectance of the material of the subject. The image processing device according to claim 1 .

8. An imaging unit that captures an image of a subject; an identification unit that identifies color characteristics of the subject in a region included in a captured image captured by the imaging unit based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence relationship between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristic identified by the identification unit; an allocation unit that allocates the conversion information acquired by the acquisition unit to an area included in the captured image; a conversion unit that converts the pixel values ​​of the captured image into the colorimetric values ​​based on the conversion information assigned to an area included in the captured image; a display unit that displays a display image based on the colorimetric values; having When the subject is a printed matter, the design information is a platemaking image. The identification unit identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image.

9. The identification unit identifies color characteristics of the subject in a region included in a captured image captured by an imaging device that images the subject, based on design information indicating color characteristics for each material of the subject; an acquisition unit acquires conversion information indicating a correspondence relationship between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristic identified by the identification unit; an allocating unit allocating the conversion information acquired by the acquiring unit to an area included in the captured image; a conversion unit converting the pixel values ​​of the captured image into the colorimetric values ​​based on the conversion information assigned to an area included in the captured image; Including, When the subject is a printed matter, the design information is a platemaking image. The identification unit identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image.

10. Computer, an identification unit that identifies color characteristics of the subject in a region included in a captured image captured by an imaging device that images the subject, based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence relationship between pixel values ​​of the captured image and colorimetric values ​​of the subject based on the color characteristic identified by the identification unit; an allocation unit that allocates the conversion information acquired by the acquisition unit to an area included in the captured image; a conversion unit that converts the pixel values ​​of the captured image into the colorimetric values ​​based on the conversion information assigned to an area included in the captured image; Function as a When the subject is a printed matter, the design information is a platemaking image. The identification unit is a program that identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image.

Images