Image processing apparatus, image processing system, image processing method, and program

Abstract

To maintain a hue of a fixed image by adjusting quality of a variable image without depending on manual work.SOLUTION: An analysis unit analyzes characteristics related to a color of a first image as analytic characteristics, and a selection unit selects a color profile corresponding to the analytic characteristics as a selected color profile from a storage unit in which color profiles including conversion information indicating correspondences between colors before and after conversion and correction information for compensating for changes in color due to color correction based on the conversion information are stored in association with each other with respect to each of reference characteristics being characteristics of a reference image, and a correction unit corrects a color of a second image by using correction information corresponding to the selected color profile, and an output processing unit performs output processing based on the selected color profile, on an output image into which the first image and the corrected second image are combined.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present application relates to an image processing device, an image processing system, an image processing method, and a program for adjusting the color tone of a variable image, such as a face image of a person, and outputting the image to an output device, such as a printer or a display monitor, so that the color tone of a fixed image, such as a logo mark, is maintained constant when displayed on a display device. [Background technology]

[0002] Display media that display multicolor images including a person's facial image and logo mark are widespread and used as personal authentication media (ID). Examples of such display media include personal identification cards, passports, driver's licenses, and membership cards. A variety of output devices, such as printers and display monitors, are used to form multicolor images. On the display medium, display areas for the facial image and logo mark are set according to the size of the recording or display surface. A display image is formed by arranging the facial image and logo mark according to the size of the display area. Then, a distribution of shading for each color component is superimposed on the display area to form a multicolor image.

[0003] For example, the image processing device described in Patent Document 1 converts the profile by referring to a profile that is referenced when converting color space data of input image information into color space data of output image information that can be interpreted by a printer, a history storage unit that stores a history of input and output color image information, and the history of stored image information. This allows the profile to be converted in accordance with changes over time in the printer. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-060924 Summary of the Invention [Problem to be solved by the invention]

[0005] However, static images such as logos are displayed in the same pattern regardless of the display medium or output device. Therefore, it is relatively easy to display static images with consistent quality by adjusting settings such as the shading of each color in advance. However, image data representing facial images varies depending on the display medium. Facial images can be captured under various conditions, including devices, lighting, and backgrounds. The attributes of the subjects, such as race and age, can vary from person to person. Therefore, it is relatively difficult to maintain a consistent texture in facial images displayed on display media. For example, the facial image as a whole may be too dark or too bright, or some color components of the skin may be stronger than others. Furthermore, color washout may become apparent in dark or bright areas.

[0006] To maintain the quality of facial images, image data containing facial images may be corrected for brightness, contrast, and other parameters when they are input or read. Correction may involve reselecting a color profile or lookup table (LUT) to change the color tone after correction. However, correcting facial images requires significant effort, making it difficult to rely on manual processing. Furthermore, improving texture requires even greater skill. Inexperienced workers may encounter color gamut limitations or unevenness in the image. For example, adjusting the facial image of a person with brown skin may result in areas other than the face being too bright, making the color distribution unclear. The present application has been made in consideration of the above-mentioned problems, and one of its objectives is to provide an image processing device, an image processing system, an image processing method, and a program that can adjust the quality of a variable image and maintain the color tone of a fixed image without relying on manual work. [Means for solving the problem]

[0007] An image processing device according to a first aspect includes an analysis unit that analyzes color characteristics of a first image as analysis characteristics; a selection unit that selects a color profile corresponding to the analysis characteristics as a selected color profile from a memory unit that stores color profiles including conversion information indicating the correspondence between colors before and after conversion and correction information for compensating for color changes due to color correction based on the conversion information, in correspondence with each reference characteristic that is a characteristic of a reference image; a correction unit that corrects the color of a second image using correction information corresponding to the selected color profile; and an output processing unit that performs output processing based on the selected color profile on an output image that combines the first image and the second image after correction.

[0008] An image processing method according to a second aspect is an image processing method in an image processing device, in which the image processing device executes an analysis step of analyzing the color characteristics of a first image as analysis characteristics, a selection step of selecting a color profile corresponding to the analysis characteristics as a selected color profile from a memory unit that stores color profiles including conversion information indicating the correspondence between colors before and after conversion and correction information for compensating for color changes due to color correction based on the conversion information, in correspondence with each reference characteristic that is a characteristic of a reference image, a correction step of correcting the color of a second image using correction information corresponding to the selected color profile, and an output processing step of performing output processing based on the selected color profile on an output image that combines the first image and the corrected second image. [Effects of the Invention]

[0009] According to the present invention, the quality of the variable image can be adjusted and the color tone of the fixed image can be maintained without relying on manual work. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic block diagram showing an example of the functional configuration of an image processing system according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of a display medium that displays a display image. [Figure 3]FIG. 2 is a diagram illustrating an example of the configuration of an analysis feature according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of the organization of color profiles according to the first embodiment. [Figure 5] 4 is a flowchart illustrating image processing according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of the organization of color profiles according to the second embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of an organization of profile groups according to an embodiment of the present application. [Figure 8] 1 is a schematic block diagram illustrating an example of the configuration of a computer system according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present application will be described with reference to the drawings. First Embodiment An image processing system 1 according to a first embodiment of the present invention will be described. 1 is a schematic block diagram showing an example of the functional configuration of an image processing system 1 according to this embodiment. First, an overview of the image processing system 1 will be described. The image processing system 1 includes an image processing device 10 and an output unit 20. The image processing device 10 detects an image representing a specific subject (sometimes referred to herein as a "first subject") from among display images to be displayed, and defines the first image as the first image. Each display image includes a first image and a second image. The image processing device 10 analyzes analysis features representing the extracted color characteristics of the first image, and selects a color profile corresponding to the analysis features from pre-stored color profiles as a selected color profile. The color profile and the corresponding reference features may be stored in a record as a pair of fields. The reference features may be used as an identifier. A dataset of multiple color profiles may be a data table with records including reference features and corresponding color profiles as fields. From this data table, the reference features most similar to the analysis features of the captured image are selected, and the selected color profile can be retrieved using the reference features as an identifier, thereby selecting the selected profile corresponding to the analysis features.

[0012] The image processing device 10 stores a data set including multiple color profiles in advance. The data set is configured by associating each color profile and correction information with a reference feature representing the color characteristics of a predetermined reference image. The color profile is a data set including conversion information used for image color conversion, and the conversion information indicates the correspondence between the pre-conversion color and the converted color. The correction information is used to correct the color characteristics of the image displayed on the display medium and is stored in association with each reference feature representing the color characteristics. For example, a separate profile may be set as the correction information to prevent excessive or insufficient shading on the skin. The image processing device 10 uses the correction information to perform corrections to match the profile corresponding to the analysis feature and the color profile of the captured facial image of the individual to differences in brightness, such as median brightness and variance. This allows the image processing device 10 to adjust skin tone, etc., to prevent blown-out highlights in dark areas with low brightness or bright areas with high brightness in the first image.

[0013] The image processing device 10 corrects the color characteristics of the second image using correction information corresponding to the selected profile, and composes an output image by combining the first image and the second image. The image processing device 10 performs an output process of the output image composed with the selected color profile to the output unit 20. In the output process, the image processing device 10 corrects the color tone of the first image using, for example, the correction information included in the selected color profile, and outputs an output image including the first image and the second image obtained by the correction to the output unit 20.

[0014] The image processing device 10 may be configured as a dedicated image processing device, or may be configured as an information processing device including a general-purpose computer system that executes a program (e.g., an application program) to cause a computer to function as an image processing device. The information processing device may take any form, such as a personal computer, a tablet terminal device, or a mobile phone. In this application, executing a process instructed by commands written in a program may be referred to as "executing a program" or "running a program." An application program may also be referred to as an "app."

[0015] The output unit 20 acquires an output image input from the image processing device 10 and presents the acquired output image. The output unit 20 may be any device capable of presenting an output image, such as a printer or a display monitor. A printer displays an output image by applying a pigment to the surface of a display medium. A display monitor functions as a display medium in which multiple pixels are arranged on the surface and a pattern in which brightness and color differ from position to position due to the distribution of luminance for each pixel is expressed as an output image. Note that in this application, brightness and color may be collectively referred to simply as "color."

[0016] In this embodiment, the display image to be processed includes a first image and a second image. The arrangement of the first image and the second image may be predetermined. The first image is a variable image in which the subject (sometimes referred to as the "first subject" in this application) differs for each display image, and thus the color distribution may vary. The second image is a fixed image that has a constant shape and brightness regardless of the display image. The second image may be a pattern including shapes, symbols, characters, colors, or any combination of these. When the display medium is a printed material, the first image may differ for each display medium, and the second image may be constant regardless of the display medium.

[0017] FIG. 2 is a diagram illustrating an example of a display medium for displaying a display image. The display medium illustrated in FIG. 2 is a printed material in which a display image is printed on the surface of a plastic sheet and is used as an identification card (ID). The ID card has a facial image (VI) arranged as a first image in a portion of the lower left area, and a logo mark (FI) arranged as a second image in another portion of the area extending from the lower center to the lower right. The facial image (VI) is different for each person who is the first subject and is typically used for identity verification. The logo mark (FI) is used consistently regardless of the person who is the first subject. However, the logo mark (FI) may be set to a pattern with a different color distribution for each issuer of the display medium. The logo mark (FI) is typically used to identify the issuer, such as a company or government agency, or the purpose of the issuer or the display medium, such as for service usage control, access control, and proof of qualifications.

[0018] In this embodiment, the display image is displayed on a display medium so that the color of the second image remains constant even if the color of the first image is adjusted. In the example of Figure 2, the color of the face image vi can be adjusted depending on the person's skin color, but the color of the logo mark fi indicating the issuer or purpose is fixed. This improves the quality of the overall display image and supports the functionality of the display medium.

[0019] An example of the functional configuration of an image processing system 1 according to this embodiment will be described with reference to Fig. 1. The following description will mainly focus on a case where the output medium is an identification card made of a plastic sheet, the first image is a person's face image, and the second image is the logo mark of the issuing organization, as exemplified in Fig. 2.

[0020] The image processing device 10 includes a control unit 110, a storage unit 130, an operation unit 140, a display unit 150, and an input / output I / F unit 160. The control unit 110 performs processing to enable the functions of the image processing device 10. The functional configuration of the control unit 110 will be described later. The storage unit 130 stores various data used in processing by the control unit 110 and various data acquired by the control unit 110. The storage unit 130 includes a storage medium such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive). The operation unit 140 receives an operation from a user and outputs an operation signal in accordance with the received operation to the control unit 110. The operation unit 140 is, for example, a mouse, a touch sensor, or a keyboard.

[0021] The display unit 150 displays various display information indicated in the display data input from the control unit 110. The display unit 150 may be any monitor display, such as a liquid crystal display (LCD) or an organic light emitting diode (OLED) display. The input / output I / F unit 160 is connected to other devices so as to be able to send and receive various data via wire or wirelessly. The input / output I / F unit 160 may be connected to a communication network and connected to other devices using the communication network.

[0022] Next, an example of the functional configuration of the control unit 110 will be described. The control unit 110 includes an image acquisition unit 112 , a detection unit 114 , an analysis unit 116 , a selection unit 118 , a correction unit 120 , and an output processing unit 122 . The image acquisition unit 112 acquires display image data that indicates a display image to be processed. Image acquisition unit 112 acquires display image data specified by the operation information from another device via input / output I / F unit 160. The other device may be an information processing device that stores display image data in advance, or an input device such as a scanner or camera for reading an image from a display medium. Image acquisition unit 112 may read out display image data specified by the operation information from display image data indicating various display images that are stored in advance in storage unit 130. Image acquisition unit 112 outputs the acquired display image data to detection unit 114.

[0023] The detection unit 114 detects an area where the first image appears (sometimes referred to herein as a "first area") from the display image shown in the display image data input from the image acquisition unit 112. The detection unit 114, for example, performs a known face detection process on the display image data to detect an area where a person's face appears as a face area. The detection unit 114 may specify, as the first area, a rectangular area that includes margins with a certain width extending outward from both vertical and horizontal ends of the detected face area. The detection unit 114 outputs first region information indicating the detected first region and the display image data to the analysis unit 116 and the correction unit 120. Information about the first image is transmitted based on the first region information and the display image data.

[0024] The first image may include a facial region and a surrounding background region. The background region may be colored in a color (e.g., blue, gray, etc.) that is clearly distinguishable from the person's skin color and the surrounding region. Furthermore, the color tone and brightness may be nearly constant within the background region. Therefore, the detection unit 114 may use a known edge detection method to detect edges surrounding the facial region from the displayed image, and define the region surrounded by the edges as the first region. In some cases, the position and area where the first image is arranged in the display image appearing on the display medium are predetermined. In such cases, layout information indicating the layout of the first area in the display image may be set in advance in the detection unit 114, and the first area may be identified from the acquired display image data by referring to the layout information.

[0025] The analysis unit 116 identifies the display image shown in the display image data and the first region shown in the first region information input from the detection unit 114. The analysis unit 116 analyzes, as an analysis feature, a feature that represents the color characteristics of the color distribution that constitutes the first image that appears in the first region of the display image. The analysis unit 116, for example, performs a known part detection technique on the face appearing in the first image and divides the face region into multiple parts. The multiple parts mainly include parts on the skin, such as the forehead, nose, cheeks, chin, and ears. These parts on the skin change little over time and are important for identifying individuals. The analysis unit 116 calculates an analysis feature amount including a color feature value representing the color of each part. Color feature values representing parts that do not show skin (e.g., hair, accessories such as glasses) are not used in the analysis feature amount and may be excluded. The features of these parts may differ even for the same person, are relatively less dependent on the individual, and may actually hinder individual identification.

[0026] The display image or the first image is expressed, for example, as RGB (Red, Green, Blue) values as color signal values for each pixel. The analysis unit 116 converts the RGB values for each pixel of each part into color space values and calculates a representative value among the pixels of the color space values. 3 has five color space values 1 to 5 as elements. Color space values 1, 2, 3, 4, and 5 are color space values that indicate colors that represent different parts of the face: the forehead, nose, cheeks, chin, and ears. The color space values may be, for example, CIE (Commission International de l'Eclairage) color space values, XYZ color space values, Lab color space values, etc. The analysis unit 116 determines, as the analysis feature, a vector containing a representative value for each part as an element. The representative value may be any statistical value that represents the color for each part, such as the mean, median, or mode. The analysis unit 116 outputs the analysis feature obtained by the analysis to the selection unit 118.

[0027] Returning to FIG. 1, the selection unit 118 selects one group of color profiles (sometimes referred to herein as a "profile group") from a data set previously stored in the storage unit 130, based on the analysis feature input from the analysis unit 116. The data set includes a profile group for each feature (sometimes referred to herein as a "reference feature") that represents a predetermined reference color characteristic. One profile group includes multiple different color profiles. Correction information is further associated with each color profile. The correction information is used to compensate for the conversion characteristics in color conversion using each color profile, as will be described later.

[0028] In the data set stored in the storage unit 130, multiple color profiles are classified by reference feature amount to form profile groups. The data set includes correction information for each individual color profile, and these are associated to form a database. In the example of FIG. 4, eight color profiles are each classified into two profile groups, either group A or group B. Group A has four profiles A01 to A04. Correction information A01 to A04 is associated with the profiles A01 to A04, respectively. Group B has four profiles B01 to B04. Correction information B01 to B04 is associated with the profiles B01 to B04, respectively. Note that, although the reference feature values are set separately from the profile group in the example of Fig. 4, this is not limiting. The reference feature values may be described in metadata included in each color profile. In that case, the reference feature values set separately from the profile group may be omitted.

[0029] 1, the selection unit 118 selects, for example, a profile group in which the representative value of the color space values for each part indicated in the analysis feature is closest to the color space value indicated in the reference feature. The representative value of the color profile may be, for example, the mean value, median value, or variance of the color profile, or a combination thereof. The selection unit 118 selects one color profile from multiple color profiles included in the selected profile group. For example, the selection unit 118 selects one color profile from multiple color profiles included in the profile group, based on a reference feature similar to the analysis feature or a representative value of the color profile itself. The selection unit 118 calculates converted color signal values using each color profile for color signal values indicating color feature values for each part indicated in the analysis feature input from the analysis unit 116. The selection unit 118 may prioritize color profiles in descending order of variance between parts of the converted color signal values, and may select color profiles with higher priorities with higher priority. In this case, color profiles with high correction capabilities for high-contrast colors are selected with higher priority. The selection unit 118 outputs profile selection information indicating the selected color profile as the selected color profile to the correction unit 120 .

[0030] Each color profile is used for pixel-by-pixel color correction and includes conversion information that indicates conversion characteristics from color signal values representing pre-conversion colors to color signal values representing post-conversion colors. The color profile associated with the reference feature may include conversion information with conversion characteristics that expand the range of color signal values that represent a color gamut within a predetermined range, in color space, from the chromaticity of the reference color represented by the reference feature. In this case, the change in the color signal values after conversion is greater than the change in the color signal values before conversion, thereby enhancing the contrast of colors that approximate the reference color.

[0031] Furthermore, the color profile associated with the reference feature may include conversion information having conversion characteristics that emphasize changes in color signal values in a color space as the color approaches the reference color indicated by the reference feature. In this case, the converted color signal values change nonlinearly with changes in the pre-conversion color signal values. Here, the entire range of the pre-conversion color signal values and the entire range of the post-conversion color signal values may be the same.

[0032] With these conversion characteristics, the area of the image to be processed is approximated as a whole to the complementary color of the reference color, and the closer the color gamut is to the reference color, the more detailed the color distribution is expressed. For example, a dark face image is brightened as a whole and the light-dark contrast of the face image is emphasized. A reddish face image is greenish as a whole and the red-green contrast of the face image is emphasized.

[0033] Color profiles belonging to the same profile group, i.e., different color profiles, have different conversion characteristics from input values to output values. The conversion characteristics may differ, for example, in the ratio of the amount of change in color signal values after conversion to the amount of change in color signal values before conversion, or in the strength of nonlinearity. The higher the nonlinearity, the more closely a color resembles the reference color, the greater the degree of emphasis on color changes relative to changes in color signal values. The lower the nonlinearity, the less closely a color resembles the reference color, the greater the degree of emphasis on color changes relative to changes in color signal values.

[0034] An example of a color profile is an ICC (International Color Consortium) profile. An ICC profile is a data set configured in accordance with standards defined by the ICC. An ICC profile includes conversion information and metadata indicating its attributes. The metadata includes a profile identifier, profile classification, etc. The profile classification indicates the type of profile, such as an input profile or output profile.

[0035] An input profile contains conversion information for converting color signal values (e.g., RGB values) used when capturing an image to be processed into Profile Connection Space (PCS) values. PCS values are reference values that quantitatively represent color (including hue and brightness). The color space values described above may be used as PCS values. An input profile may be used, for example, to compensate for color changes due to the spectral characteristics of an input device (e.g., a camera or scanner) or the display medium of the image to be input, or to convert color signal values (e.g., RGB values) in other color systems that make up the image data into PCS values. An output profile contains conversion information for converting PCS values into color signal values (e.g., CMYK (Cyan, Magenta, Yellow, Black) values) to be used for output.

[0036] An output profile may be used, for example, to compensate for color changes due to the spectral characteristics of a display medium presented by an output device (e.g., output unit 20), to convert PCS values constituting image data into color signal values in another color system, etc. The display means may be the output device (e.g., a display monitor, a printer) itself, or may be combined with another object (e.g., a plastic sheet, paper, ink, etc.) that is acted upon by the output device. Similarly, the acquisition means may be the input device (e.g., a camera, a scanner) itself, or may be combined with another object (e.g., a plastic sheet, paper, ink, a display monitor, etc.) that presents the image to be acquired.

[0037] The conversion information may be expressed in any format, such as a lookup table (LUT), a conversion matrix, or a color conversion model (for example, a TRC (Tone Reproduction Curve) model). The conversion information may specify a colorimetric rendering intent defined by the ICC, such as ICC absolute colorimetry, media-relative colorimetry, perceptual colorimetry, and saturation-oriented colorimetry.

[0038] The correction unit 120 identifies the display image represented by the display image data input from the detection unit 114 and the first region represented by the first region information. The analysis unit 116 performs known brightness and contrast correction processing on the first image appearing in the first region of the display image. In the brightness and contrast correction processing, the correction unit 120 linearly converts the color signal values of each pixel representing the first image so that the converted color signal values fall within a predetermined value range. As the predetermined value range, a range of color signal values that can clearly represent the face of a person as a subject with all attributes is set in advance. Correction of the first image at this stage can be considered preprocessing performed separately from color correction based on the profile and is not required in this embodiment.

[0039] The correction unit 120 extracts, from the display image, an image that appears in a second area different from the first area as a second image. As described above, the data set stored in storage unit 130 includes, for each color profile, correction information for compensating for color changes due to color correction using that color profile, and is configured in association with that color profile.

[0040] Therefore, correction unit 120 reads correction information corresponding to the selected color profile indicated in the selected profile information input from selection unit 118 from the data set stored in storage unit 130. Correction unit 120 corrects the colors of the extracted second image using the read correction information. Color correction based on the correction information compensates for the conversion characteristics of the color conversion using the selected color profile. The correction unit 120 outputs to the output processing unit 122 first image data indicating the corrected first image, second image data indicating the corrected second image, and selected profile information indicating the selected color profile.

[0041] The output processing unit 122 performs processing to output the output image to the output unit 20. The output processing unit 122 combines the corrected first image indicated in the first image data input from the correction unit 120 with the corrected second image indicated in the second image data to synthesize the output image. The output processing unit 122 associates the output image data indicating the synthesized output image with the selected profile information input from the correction unit 120, and outputs the result to the output unit 20 via the input / output I / F unit 160.

[0042] The output unit 20 receives the output image data and the selected profile information from the output processing unit 122. The output unit 20 performs color correction on the output image indicated in the input output image data using the color profile indicated in the selected profile information, and presents the corrected output image. The output processing unit 122 determines whether the selected color profile is appropriate. For example, the output processing unit 122 can determine whether the selected color profile is appropriate based on whether operation information indicating that the image quality of the output image is good is input from the operation unit 140 within a certain period after the output image is presented.

[0043] The output processing unit 122 may cause a camera connected to the image processing device 10 to capture the presented output image. The output processing unit 122 acquires output image data representing the captured output image from the camera via the input / output I / F unit 160. The output processing unit 122 then calculates a change amount indicating the magnitude of the difference between the output image data and display image data representing the original display image, and determines whether the selected color profile is appropriate depending on whether the change amount exceeds a predetermined upper limit. This allows output images whose color or brightness has changed significantly due to correction to be eliminated. Indices such as the sum of squared differences (SSD) and the sum of absolute differences (SAD) can be used as the change amount.

[0044] If the output processing unit 122 determines that the selected color profile is appropriate, it ends the series of processes. If the output processing unit 122 determines that the selected color profile is inappropriate, it returns the process to the correction unit 120. The correction unit 120 then selects a different, unselected color profile from the selected profile group as the selected color profile. Then, the correction process on the second image based on the newly selected color profile by the correction unit 120, the output process to the selected color profile by the output processing unit 122, and the determination of the appropriateness of the selected color profile are repeated.

[0045] Next, an example of image processing according to this embodiment will be described below with reference to Fig. 5, which is a flowchart illustrating the image processing according to this embodiment. (Step S102) The image acquisition unit 112 acquires display image data indicating a display image to be processed. (Step S104) The detection unit 114 detects an area in which a face image appears as a first image from the display image shown in the display image data. (Step S106) The analysis unit 116 analyzes the color characteristics of the first image and calculates an analysis feature amount indicating the color characteristics.

[0046] (Step S108) The selection unit 118 identifies, from the data set stored in the storage unit 130, a profile group corresponding to a reference feature that indicates color characteristics that are most similar to the color characteristics indicated by the analysis feature. (Step S110) The selection unit 118 selects one color profile from the unselected profiles in the identified profile group as a selected color profile.

[0047] (Step S112) The correction unit 120 performs brightness and contrast correction processing on the face image, which is the first image. In the brightness and contrast correction processing, the correction unit 120 refers to histogram data indicating the color distribution of the face region as the analysis feature calculated in step S106, for example, and performs processing such as color correction using a known tone curve according to the brightness of the skin color appearing in the first image. (Step S114) Correction unit 120 extracts a fixed image as a second image that appears in an area different from the face image. (Step S116) The correction unit 120 performs color correction on the fixed image using the correction information corresponding to the selected color profile. The color correction compensates for the conversion characteristics due to the color conversion (color correction) of the fixed image using the selected color profile.

[0048] (Step S118) The output processing unit 122 combines the facial image, which is the first image after correction, with the fixed image, which is the second image after correction, to synthesize an output image. (Step S120) The output processing unit 122 outputs the combined output image data and the selected color profile to the output unit 20. The output unit 20 performs color correction on the output image indicated in the output image data input from the output processing unit 122, using the input selected color profile. The output unit 20 presents the corrected output image.

[0049] (Step S122) Output processing unit 122 causes the camera to capture an output image. Output processing unit 122 determines the appropriateness of the selected color profile based on the difference between the captured output image and the display image. If the selected color profile is determined to be appropriate (YES in step S122), the processing of FIG. 5 ends. If the selected color profile is determined to be inappropriate (NO in step S122), the processing returns to step S110, and steps S112 to S122 are repeated.

[0050] Second Embodiment Next, an image processing system 1 according to a second embodiment of the present invention will be described, focusing mainly on the differences from the first embodiment. For points in common with the first embodiment, the description of the first embodiment will be used unless otherwise specified. In the image processing device 10 according to this embodiment, the analysis unit 116 analyzes attributes of the subject of the first image (sometimes referred to herein as "analysis characteristics") instead of or in addition to the above-described analysis features as analysis characteristics related to the color of the first image. The analysis unit 116 can estimate attributes of people appearing in the first image by performing known image recognition processing on the first image.

[0051] If the subject of the first image is a person, for example, one or both of age and race can be applied as the attribute. This is because the characteristics of the skin color of the person depicted in the first image are highly dependent on these attributes. In general, skin color tends to change with age. More specifically, skin color tends to become darker and duller with age. Darkness and dullness are highly correlated with lightness and saturation, which are elements of color, respectively. In addition, differences in skin color depending on race are well known.

[0052] The storage unit 130 stores a set of color profiles and correction information for each attribute of a subject as reference characteristics of a predetermined reference image, with the set included in a data set. More specifically, for each attribute of the subject, a profile group including multiple color profiles with different conversion characteristics is set, and correction information for each color profile is associated with the data set. For each attribute, color profiles with multiple different color conversion characteristics that enable color correction with a high degree of improvement in subjective quality are created in advance for facial images of people with average or typical color distribution.

[0053] 6 illustrates two profile groups, group I and group II. Groups I and II are associated with attributes I and II, respectively, as attributes of the subject of the first image. Group I includes profiles I01 to I04, which are associated with correction information I01 to I04, respectively. Group II includes profiles II01 to II04, which are associated with correction information II01 to II04, respectively.

[0054] The selection unit 118 selects a profile group that corresponds to the analysis attribute analyzed by the analysis unit 116 from among the profile groups included in the data set. The selection unit 118, the correction unit 120, and the output processing unit 122 execute the processes of steps S110 to S122 (FIG. 5) for the selected profile group.

[0055] The above embodiment may be modified as follows: For example, among the multiple profile groups included in the data set, color profiles including input profiles may be classified by type of image acquisition means (i.e., one or both of an input device and a display medium from which a display image is acquired). Color profiles including output profiles may be classified by type of display means (i.e., one or both of an output device and a display medium to which a display image is presented).

[0056] Fig. 7 illustrates an example in which multiple profile groups are associated with each model of output device that constitutes display means. In the example of Fig. 7, profile groups A to D are associated with device X, which is an output device, and groups E to H are associated with device Y. Each profile group includes multiple color profiles. Identification information (device ID) that identifies each device is described as part of metadata associated with each color profile.

[0057] In this way, when multiple color profiles are associated with each acquisition means or display means, the control unit 110 detects an acquisition means or display means that can be used by the control unit 110. The control unit 110 enables the color profile associated with the detected acquisition means or display means and disables the other color profiles. For example, the control unit 110 can identify the input device and output device connected to the control unit 110 as the acquisition means and display means, respectively, by executing predetermined firmware (e.g., BIOS (Basic Input Output System)) at startup. The control unit 110 may also identify a display medium specified by an operation signal from the operation unit 140 as the acquisition means or display means.

[0058] As described above, the image processing device 10 includes an analysis unit 116, a selection unit 118, a correction unit 120, and an output processing unit 122. The analysis unit 116 analyzes color characteristics of the first image as analysis characteristics. The selection unit 118 selects a color profile corresponding to the analyzed analysis characteristics as a selected color profile from a storage unit 130 that stores color profiles including conversion information indicating the correspondence between colors before and after conversion and correction information for compensating for color changes due to color correction based on the conversion information, in association with each reference characteristic that is a characteristic of the reference image. The correction unit 120 corrects the color of the second image using the correction information corresponding to the selected color profile. The output processing unit 122 performs output processing based on the selected color profile on an output image obtained by combining the first image and the corrected second image.

[0059] According to this configuration, a color profile corresponding to the color characteristics of the first image is selected, and correction information corresponding to the selected color profile is used in the second image to change the colors with characteristics opposite to those caused by the selected color profile. The colors of the entire output image including the first and second images are then corrected using the selected color profile. Therefore, the colors of the first image are corrected using the selected color profile, and the colors of the second image, which have changed with the opposite characteristics, are compensated for using the selected color profile. Therefore, the colors of the first image, which is a variable image, are adjusted while the colors of the second image, which is a fixed image, are maintained.

[0060] In the above description, an example has been given in which image acquisition unit 112 acquires a display image including both the first image and the second image, but the first image and the second image may be acquired separately. Image acquisition unit 112 instructs a camera connected via input / output I / F unit 160 to take a photograph based on, for example, operation information input from operation unit 140. Image acquisition unit 112 acquires first image data indicating a facial image captured by the camera. The image acquisition unit 112 may acquire existing first image data designated by the operation information from another device via the input / output I / F unit 160.

[0061] The image acquisition unit 112 is instructed by the operation information to read out second image data indicating a second image pre-stored in the storage unit 130. The second image may be included as part of a predetermined form (e.g., a certificate template). In this case, the second image data may be data representing the form including the second image. The image acquisition unit 112 outputs the acquired first image data to the analysis unit 116, and outputs the acquired second image data to the correction unit 120. Therefore, the detection unit 114 may be omitted.

[0062] In the above description, the subject of the first image is primarily a human face, but this is not limited to this. The first image may be any image with a different color distribution for each displayed image, such as a palm, fingers, or other body part, a living object other than a human, an inanimate object, or a landscape. The second image may be any image with a fixed color distribution for a specific purpose or issuer.

[0063] In the above description, the output unit 20 is mainly a printer, but the output unit 20 may also be a display monitor. In this case, the display unit 150 included in the image processing device 10 may be used as the output unit 20. The image processing device 10 may be realized as a general-purpose electronic device or an electronic device having another primary function, such as a mobile phone, a tablet terminal device, or a personal computer. As an application example, the monitor display forming the display unit 150 may display a face image as the first image and a membership card or identification card showing the logo mark of a specific service as the second image.

[0064] In the above description, an example was given in which output processing based on the selected color profile selected by selection unit 118 involves output processing unit 122 outputting the selected color profile to output unit 20, and output unit 20 performing color correction on the output image represented by the output image data using the selected color profile input from output processing unit 122. However, depending on the functionality of output unit 20, it may not have the function to perform color correction. In that case, output processing unit 122 corrects the color of the output image, which is a composite of the first image and the second image, using the selected color profile as output processing based on the selected color profile. Output processing unit 122 outputs output image data representing the corrected output image to output unit 20, and omits outputting the selected color profile.

[0065] In the above description, an example has been given in which a data set including multiple color profiles is stored in advance in the storage unit 130, but this is not limiting. The data set may also be stored in a device separate from the image processing device 10. The separate device may be a device connectable to the image processing device 10 via a communication network, such as a database server. In this case, the selection unit 118 reads the color profile or profile group from the separate device instead of reading it from the storage unit 130 of its own device.

[0066] The image processing device 10 may be configured to include a general-purpose computer system. 8 is a schematic block diagram showing an example configuration of a computer system 200 according to an embodiment of the present application. The computer system 200 includes, for example, a processor 202, an input device 204, an output device 206, a ROM (Read Only Memory) 208, a RAM (Random Access Memory) 210, an auxiliary storage unit 212, and an interface unit 214. The processor 202, the input device 204, the output device 206, the ROM 208, the RAM 210, the auxiliary storage unit 212, and the interface unit 214 are connected to one another using a bus BS.

[0067] The processor 202, for example, reads out programs and various data stored in the ROM 208, executes the programs, and controls the operation of the computer system 200. In this application, "executing a program" or "executing a program" includes the meaning of executing processing instructed by commands written in the program. The processor 202 is, for example, a CPU (Central Processing Unit). There may be multiple processors 202. The multiple processors 202 may include a GPU (Graphic Processing Unit) in addition to a CPU.

[0068] The input device 204 receives a user operation, receives operation information corresponding to the received operation, and outputs operation data to the processor 202. The operation unit 140 described above is realized using the input device 204. The output device 206 outputs the output data input from the processor 202 to various devices as output destinations. The display unit 150 is realized using the output device 206. The output unit 20 built into the image processing device 10 can also be an example of the output device 206.

[0069] The ROM 208 stores, for example, a program to be executed by the processor 202 . The RAM 210 is used as a main storage medium that functions as a work area for temporarily storing various data and programs used by the processor 202, for example. The auxiliary storage unit 212 is an auxiliary storage device such as an HDD (Hard Disk Drive), a flash memory, etc. The storage unit 130 includes the auxiliary storage unit 212.

[0070] The interface unit 214 is connected to other devices and enables input and output of various data wirelessly or via a wired connection. The interface unit 214 includes, for example, a communication module that connects to a network via a wired or wireless connection. The input / output I / F unit 160 described above is realized using the interface unit 214.

[0071] Although the embodiments of the present application have been described in detail above with reference to the drawings, the specific configurations are not limited to the above-described configurations and include designs within the scope of the gist of the present embodiment. The above-described configurations can be combined in any manner, and some of them may be omitted. [Explanation of symbols]

[0072] 1...image processing system, 10...image processing device, 20...output unit, 110...control unit, 112...image acquisition unit, 114...detection unit, 116...analysis unit, 118...selection unit, 120...correction unit, 122...output processing unit, 130...storage unit, 140...operation unit, 150...display unit, 160...input / output I / F unit, 200...computer system, 202...processor, 204...input device, 206...output device, 208...ROM, 210...RAM, 212...auxiliary storage unit, 214...interface unit

Claims

1. an analysis unit that analyzes color characteristics of the first image as analysis characteristics; a storage unit that stores a color profile including conversion information indicating a correspondence relationship between colors before and after conversion and correction information for compensating for a color change due to color correction based on the conversion information, in association with each reference characteristic that is a characteristic of a reference image; a selection unit that selects a color profile corresponding to the analysis characteristics as a selected color profile; a correction unit that corrects the color of the second image using correction information corresponding to the selected color profile; an output processing unit that performs output processing based on the selected color profile on an output image obtained by combining the first image and the corrected second image. Image processing device.

2. the analysis unit analyzes an analysis feature amount representing a color characteristic of the first image as the analysis characteristic; the storage unit stores the color profile and the correction information in association with each other for each reference feature amount that represents a color characteristic of the reference image as the reference characteristic; The selection unit selects a color profile corresponding to the analysis feature as a selected color profile. The image processing device according to claim 1 .

3. the analysis unit analyzes an attribute of a subject in the first image as an analysis attribute as the analysis characteristic; the storage unit stores the color profile and the correction information in association with each other as the reference characteristics for each attribute of the subject; The selection unit selects a color profile corresponding to the analysis attribute as a selected color profile. The image processing device according to claim 1 .

4. a subject of the first image is a person's face, The analytical attributes include at least one of the person's age and race. The image processing device according to claim 3 .

5. the storage unit stores a profile group including a plurality of color profiles with different conversion characteristics in association with each of the reference characteristics, and stores the correction information for each of the color profiles; The selection unit selects, as the selected color profile, one color profile included in a selected profile group that is a profile group corresponding to the analysis characteristics. The image processing device according to claim 1 .

6. The output processing unit An output image displayed on a display medium is acquired from a photographing unit, determining whether the selected color profile is appropriate based on a difference between the acquired output image and a display image including the first image and the second image; When the selected color profile is determined to be inappropriate, the selection unit selects another color profile included in the selected profile group. The image processing device according to claim 5 .

7. The first image is a face image of a person. The second image includes a mark having a color distribution. The image processing device according to claim 1 .

8. To the computer A program for causing the image processing device according to claim 1 to function.

9. an output device to which the output image is to be output; The image processing device according to claim 1 Image processing system.

10. An image processing method in an image processing device, comprising: The image processing device an analyzing step of analyzing color-related characteristics of the first image as the analysis characteristics; a storage unit that stores a color profile including conversion information indicating a correspondence relationship between colors before and after conversion and correction information for compensating for a color change due to color correction based on the conversion information, in association with each reference characteristic that is a characteristic of a reference image; a selection step of selecting a color profile corresponding to the analytical characteristics as a selected color profile; a correcting step of correcting the color of the second image using correction information corresponding to the selected color profile; an output processing step of performing output processing based on the selected color profile on an output image obtained by combining the first image and the corrected second image. Image processing methods.

Images