Display control device, display control method, image processing system, and program

Abstract

To provide a technology that enables color matching of both image areas in an image having an image area corresponding to an area where an external display device displays an image and an image area corresponding to a subject.SOLUTION: A display device 100 acquires image data having image areas corresponding to an external display device on which an image is displayed and a subject different from the external display device. A detection unit 104 detects a first area corresponding to an image display area of the external display device from the image data. A parameter generation unit 103 generates color adjustment parameters for performing color matching between the detected first area and a second area corresponding to the subject. A color conversion unit 105 applies color conversion based on the generated color adjustment parameters to the first area to perform color adjustment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for acquiring an image having an image area corresponding to an area where an external display device displays an image and an image area corresponding to a subject, and performing color conversion to match the colors of both image areas. [Background technology]

[0002] In-camera VFX is a technique in which a real subject is placed in front of an external display device and filmed. Virtual production allows for a production environment in which a background image from an external display device and a real subject are simultaneously filmed and composited with a camera. An example of an external display device is an LED wall, which consists of a grid of LEDs (Light Emitting Diodes). The image captured by the camera has a first area derived from the external display device and a second area derived from the real subject. For example, consider a situation in which performers and a set are placed between the external display device and the camera. The external display device displays the background image, and the camera films the external display device, performers, and set together. The first area is the image area derived from the external display device that displays the background image, and is hereafter referred to as the "external display device area." The second area is the image area derived from the performers and set, and is hereafter referred to as the "natural image area."

[0003] In the shooting process, for color matching between the area of the natural picture and the area of the external display device, lighting adjustment and color adjustment of the external display device are performed. Regarding the color adjustment of the area of the natural picture, by setting the white balance of the camera according to the lighting, shooting can be done with colors close to those seen by the human eye. However, when using an LED wall as the external display device, the spectral characteristics of the LEDs used in the LED wall are steep, and the area of the external display device has a large difference from the colors seen by the human eye compared to the area of the natural picture. The reason is that the steeper the spectral characteristics of the light source, the more easily it is affected by the difference between the standard spectral luminous efficiency (characteristics of the human eye) and the spectral sensitivity characteristics of the camera. This effect finally surfaces as a color difference. Also, since there are subtle differences in spectral sensitivity characteristics for each camera, the colors are different for each camera. Therefore, in order to match the area of the external display device to the same color as that seen by the human eye, it is necessary to perform color adjustment on the area of the external display device.

[0004] Methods for video color adjustment include those using CDL (Color Decision List) and LUT (Look Up Table). The content of the color adjustment performed at the shooting site can be digitized and the data can be recorded, and in the shooting process and editing process etc. carried out afterwards, the color adjustment can be easily reproduced based on the recorded data. Patent Document 1 discloses a technique for making it possible to simply and accurately reproduce the color adjustment process performed in the previous process in the subsequent process for the purpose of improving the accuracy of color adjustment with CDL. In the extended meta-information area of the CDL file, image quality adjustment parameters and attention area information are recorded for each attention area.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, in the conventional color adjustment method using a CDL or LUT, since a shooting target in which an external display device and an actual subject are mixed is not considered, color adjustment cannot be performed limited to the area of the external display device. An object of the present invention is to provide a technique that enables color matching between two image regions in an image having an image region corresponding to a region where an external display device displays an image and an image region corresponding to a subject.

Means for Solving the Problems

[0007] A display control device according to an embodiment of the present invention includes an acquisition unit that acquires image data having a first region corresponding to a region where an external display device displays an image and a second region corresponding to a subject different from the external display device, a detection unit that detects the first region from the image data, a generation unit that generates a first parameter for performing color matching between the first region and the second region, and a color conversion unit that performs color adjustment of an image using the first parameter. The color conversion unit performs color adjustment on the first region detected by the detection unit using the first parameter.

Effects of the Invention

[0008] According to the present invention, it is possible to provide a technique that enables color matching between two image regions in an image having an image region corresponding to a region where an external display device displays an image and an image region corresponding to a subject.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Best Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the embodiment, an example in which a display device incorporates a display control device will be described. However, the display control device according to the present invention may be applied as a device separate from the display device (such as a personal computer). An image processing system that applies color conversion to an area of an external display device for color matching between the area of the external display device and the area of a natural image in an image including an image area corresponding to the external display device and an image area of a natural image obtained by an in-camera VFX or the like will be described.

[0011] [First Embodiment] FIG. 1 is a block diagram showing a display device 100 as a configuration example of a display control device according to the present embodiment. The display device 100 includes a receiving unit 101, a switch unit 102, a parameter generation unit 103, a detection unit 104, a color conversion unit 105, a display unit 106, a control unit 107, and a memory unit 108.

[0012] The receiving unit 101 acquires video data from an external device such as an imaging device or a playback device, and outputs the video data to the switch unit 102. For example, the receiving unit 101 acquires frame data from the external device for each frame of the video, and outputs the frame data to the switch unit 102. The receiving unit 101 can acquire video data by a method compliant with a predetermined standard. Examples of the predetermined standard include standards such as SDI (Serial Digital Interface) and HDMI (High-Definition Multimedia Interface: registered trademark). The receiving unit 101 can also acquire video data recorded in the memory unit 108.

[0013] The switch unit 102 is arranged downstream of the receiving unit 101, and outputs the video data input from the receiving unit 101 to either the parameter generation unit 103 or the detection unit 104. The display device 100 switches whether to generate color adjustment parameters from the video data or to perform color conversion based on a user's operation instruction. The user's operation instruction can be given by a signal output from an operation unit (not shown) to the control unit 107. The control unit 107 interprets the content of the user's operation instruction and controls the output destination of the switch unit 102. When it is determined that the user's operation instruction is an instruction to generate color adjustment parameters, the control unit 107 controls to switch the output destination of the switch unit 102 to the parameter generation unit 103. When it is determined that the user's operation instruction is an instruction for color conversion, the control unit 107 controls to switch the output destination of the switch unit 102 to the detection unit 104.

[0014] Based on the video data input from the switch unit 102, the parameter generation unit 103 generates color adjustment parameters for color matching the area of the external display device and the area of the natural picture. This color adjustment parameter is a parameter for reducing the difference between the pixel value of the display image displayed on the external display device and the pixel value of the image captured by the imaging device for the display image. The parameter generation unit 103 outputs the generated color adjustment parameters to the control unit 107 and performs a process of storing them in the memory unit 108 via the control unit 107.

[0015] Based on the metadata of the video data input from the switch unit 102, the detection unit 104 detects the area of the natural picture and the area of the external display device, and outputs the information of the detected area (hereinafter referred to as "area information") and the video data to the color conversion unit 105. The metadata is, for example, ANC data standardized by SDI or data stored in an Info Frame standardized by HDMI (registered trademark). The metadata indicates whether the target pixel is a pixel in the area of the external display device or a pixel in the area of the natural picture, and is assumed to be given by an external device. Regarding the area information, for example, based on a distance map acquired by a distance measuring sensor, an area including pixels corresponding to the vicinity of a predetermined distance is set as the area of the external display device, and an area other than the area is set as the area of the natural picture.

[0016] The color conversion unit 105 performs color conversion based on the area information and the video data input from the detection unit 104. This color conversion is performed to color match the area of the external display device and the area of the natural picture. Regarding the color adjustment parameters used at that time, they are generated by the parameter generation unit 103 and stored in the memory unit 108. The control unit 107 reads out the color adjustment parameters from the memory unit 108 and outputs them to the color conversion unit 105. The color conversion unit 105 outputs the processed video data to the display unit 106.

[0017] The display unit 106 displays an image based on the video data input from the color conversion unit 105 on the display screen. For example, the display unit 106 includes a liquid crystal display device having a liquid crystal display panel and a backlight unit, an organic EL (Electro Luminescence) display panel, or the like.

[0018] The control unit 107 controls each component of the display device 100. For example, the control unit 107 includes a CPU (Central Processing Unit), reads and executes a program stored in the memory unit 108, and performs various controls. The memory unit 108 stores programs, parameters, video data, and the like. The control unit 107 controls the writing and reading of information stored in the memory unit 108 and outputs it to each component of the display device 100 as necessary.

[0019] Next, a method for calculating color adjustment parameters for color matching between the area of the external display device and the area of the natural image will be described. FIG. 2 shows a schematic arrangement example of the display device 100, the external display device 200, and the imaging device 210 as a system configuration for generating color adjustment parameters. The video data at the time of imaging by the imaging device 210 facing the external display device 200 is input to the display device 100.

[0020] The external display device 200 displays a white patch 201. The area indicated by the rectangular frame on the screen of the external display device 200 is the area where the white patch 201 is displayed. In the display process of the white patch 201, for example, the pixel values (RGB values) in the bright area including the center of the screen are set to (255, 255, 255), and the pixel values in the dark area outside the area are set to (0, 0, 0). The imaging device 210 acquires the image in which the white patch 201 is displayed and outputs the imaging data to the display device 100.

[0021] FIG. 3 is a flowchart showing an example of a process for generating color adjustment parameters for color matching between the area of an external display device and the area of a natural picture. After the external display device 200 displays a white patch 201 and the imaging device 210 performs imaging and image data is input to the display device 100, when the frame image data acquired by the receiving unit 101 of the display device 100 is updated, the process of FIG. 3 starts.

[0022] In S301, image data is input from the receiving unit 101 to the parameter generation unit 103 via the switch unit 102. The parameter generation unit 103 acquires the pixel value (patch measurement value) of the white patch from the input image data. As a method for specifying the area of the white patch from the image data, for example, there is a method of performing binarization processing on the image and setting the brighter of the two areas as the area of the white patch. A process of calculating the average value of the pixel values in the area of the white patch (bright area) is performed, and the calculated average value is used as the pixel value of the white patch. The pixel value of the white patch is used in the next S302.

[0023] In S302, the parameter generation unit 103 calculates color adjustment parameters for color matching between the area of the external display device and the area of the natural image from the patch measurement values in S301. The control unit 107 acquires the calculated color adjustment parameters from the parameter generation unit 103 and performs control to store them in the memory unit 108. Specifically, the pixel values of the white patch in the image data acquired by shooting are denoted as w_r, w_g, w_b, and the minimum pixel value among them is denoted as w_min. "_r", "_g", and "_b" respectively represent that they are the components of red, green, and blue. The parameter generation unit 103 calculates w_min / w_r, w_min / w_g, and w_min / w_b from the pixel values w_r, w_g, w_b of the white patch and the minimum pixel value w_min. The color adjustment parameters for each color component are denoted as p_r, p_g, and p_b. The calculated w_min / w_r, w_min / w_g, and w_min / w_b are stored in the memory unit 108 as p_r, p_g, and p_b respectively. For example, when the patch measurement value is (230, 230, 255), the values of the color adjustment parameters are (1.0, 1.0, 0.9). The color adjustment parameters stored in the memory unit 108 are read from the memory unit 108 and used when the color conversion unit 105 executes color conversion processing. After the processing of S302, a series of processing is terminated.

[0024] Next, referring to FIG. 4, a method for implementing color matching will be described. FIG. 4 shows a schematic arrangement example of a display device 100, an external display device 200, an imaging device 210, a distance measuring device 401, and a subject 402 as a system configuration when performing color matching between the area of the external display device and the area of the natural image and displaying the video on the display device. The subject 402 is, for example, a person and is located between the external display device 200 and the imaging device 210. The distance measuring device 401 has a distance measuring sensor, and the imaging device 210 and the distance measuring device 401 are arranged adjacent to each other.

[0025] The metadata attached to the captured image acquired by the imaging device 210 includes information indicating that the pixel of the captured image is in the area of the external display device and information indicating that the pixel of the captured image is in the area of a natural picture. The distance measuring device 401 measures the distances to the external display device 200 and the subject 402 respectively. For example, before shooting, the distance measuring device 401 measures the distance between the external display device 200 and the imaging device 210. The distance measuring device 401 creates a distance map of the external display device 200 based on the measured distance information. During shooting, the distance measuring device 401 measures the distance between the imaging device 210 and the subject 402. The distance measuring device 401 collates with the created distance map of the external display device 200 and determines the area with the same distance as the area of the external display device 200. Regarding the area where the person who is the subject 402 is imaged, the distance between the subject 402 and the imaging device 210 is smaller compared to the distance between the external display device 200 and the imaging device 210. Therefore, the distance measuring device 401 determines the area where the subject 402 is imaged as the area of a natural picture.

[0026] In the example of FIG. 4, the imaging device 210 and the distance measuring device 401 are separate devices, and the captured image and the metadata attached to the image have been described, but it is not limited thereto. The imaging device 210 may have a configuration including a distance measuring device or a distance measuring function. For example, by using an imaging device equipped with an imaging element of the imaging surface phase difference detection method, the amount of image shift can be detected from the phase difference detection signals output by a plurality of photoelectric conversion units in the pixel unit to obtain depth information (such as a distance map or a defocus map). Alternatively, a configuration capable of stereo shooting and distance measurement using a plurality of imaging devices can be used.

[0027] FIG. 5 is a schematic diagram for explaining whether the pixel in the moving image is a pixel in the area of the external display device or the area of a natural picture. The area 501 is an image area where the person who is the main subject is imaged and is the area of a natural picture. The area 502 is the area of the external display device and is the area where the video serving as the background is displayed during shooting.

[0028] Referring to FIG. 6, the processing executed by the display device 100 will be described. FIG. 6 is a flowchart showing an example of processing when an image is displayed by performing color matching between the area of the external display device and the area of the natural picture. When the frame image data acquired by the receiving unit 101 of the display device 100 is updated, the processing of FIG. 6 starts.

[0029] In S601, the detection unit 104 acquires video data and metadata from the receiving unit 101 via the switch unit 102. Based on the area information included in the metadata, the detection unit 104 detects the area of the external display device and the area of the natural picture, and outputs the video data and the area information to the color conversion unit 105. Next, the process proceeds to the process of 602.

[0030] In S602, the color conversion unit 105 acquires the video data and area information from the detection unit 104 and the color adjustment parameters read from the memory unit 108. The color conversion unit 105 applies color conversion to the area of the external display device using the color adjustment parameters based on the area information and the color adjustment parameters. The color conversion unit 105 outputs the video data after the color conversion is applied to the display unit 106. Here, the pixel value before color adjustment is expressed as (r_in, g_in, b_in), and the pixel value after color adjustment is expressed as (r_out, g_out, b_out).

[0031] The color conversion from the pixel value (r_in, g_in, b_in) before color adjustment to the pixel value (r_out, g_out, b_out) after color adjustment is expressed as the following formula (1) using the color adjustment parameters (p_r, p_g, p_b).

Equation

[0032] As can be seen from Equation (1), color adjustment is not performed on the area of the natural image, and color adjustment is performed on the area of the external display device based on color adjustment parameters. In the color conversion for the area of the external display device, a process of correcting the deviation of white balance for the input pixel value and outputting it is performed. By correcting the deviation of white balance for the area imaged by the imaging device in the image display area of the external display device having steep spectral characteristics, the color of the area of the external display device can be made closer to the color of the area of the natural image. Next, the process proceeds to 603. In S603, the display unit 106 displays an image on the screen according to the video data input from the color conversion unit 105. Then, a series of processes are terminated.

[0033] FIG. 7 is a diagram schematically explaining the difference in images before and after applying color conversion to the area of the external display device. FIG. 7(A) shows the image before color conversion. The area 701 of the natural image corresponds to the area where the main subject, a person, is photographed, and the area 702 of the external display device corresponds to the area where the background image is photographed. FIG. 7(B) shows the image after color conversion. Regarding the area 701 of the natural image, it is the same as in FIG. 7(A), but the area 703 of the external display device is different from FIG. 7(A).

[0034] FIG. 7(A) shows that there is a deviation in white balance between the area 701 of the natural image and the area 702 of the external display device. FIG. 7(B) shows that by performing color adjustment on the area 703 of the external display device, the white balance between the area 703 and the area 701 of the natural image has approached. For example, assuming the pixel value of the area 701 is (255, 255, 255) and the pixel value of the area 702 is (230, 230, 255). When applying (1.0, 1.0, 0.9) as the color adjustment parameter, the pixel value of the area 703 after color conversion becomes (230, 230, 230). At this time, if the pixel values of the area 701 and the area 703 are divided by the maximum value of their respective pixel values and normalized, the pixel values of both areas become (1.0, 1.0, 1.0), so it can be seen that the white balance between the area 701 and the area 703 approaches.

[0035] In this embodiment, color conversion processing based on color adjustment parameters is performed on the area of the external display device detected from image data (video data), and color matching is performed between the area of the external display device and the area of the natural picture. There is a problem in the color adjustment means based on the conventional technology that it is impossible to adjust the color of only the area of the external display device for a shooting target in which the actual subject corresponding to the natural picture and the external display device are mixed. On the other hand, according to this embodiment, it is possible to perform color matching between the area of the external display device and the area of the natural picture by adjusting the color of the area of the external display device. In this embodiment, an example is shown in which the white balance is adjusted using the pixel values (RGB values) of the white patch when calculating the color adjustment parameters. Without being limited to this, instead of the pixel values of the white patch, the color adjustment parameters can be calculated based on the luminance or chrominance of the white patch or both the luminance and chrominance.

[0036] [Second Embodiment] Next, the second embodiment will be described. The display device according to this embodiment acquires data output from a plurality of imaging devices and identifies each imaging device using predetermined information. For example, assume a case where data is input from the first and second imaging devices to the display device. The predetermined information is identification information such as the manufacturer name, model number, and serial number of the imaging device, which is acquired from the ANC data of SDI. The display device performs color adjustment on the area of the external display device using individual color adjustment parameters for each imaging device based on the identification information of the imaging device. In this embodiment, detailed descriptions of the same matters, symbols, etc. as those in the first embodiment are omitted, and mainly the differences will be described. Such an omission method of description is the same in the embodiments described later.

[0037] FIG. 8 is a block diagram showing a configuration example of a display device 800 according to the present embodiment. In the display device 800, the difference from the configuration shown in FIG. 1 is that an identification unit 801 is provided between the reception unit 101 and the switch unit 102. The identification unit 801 is under the control of the control unit 107, acquires video data and its metadata from the reception unit 101, and performs a process of identifying which imaging device the input is from using the metadata. The control unit 107 performs control to store the color adjustment parameters in the memory unit 108 separately, together with data for managing which imaging device the color adjustment parameters generated by the parameter generation unit 103 are for.

[0038] FIG. 9 is a flowchart showing an example of a process related to generation of color adjustment parameters for matching the colors of the external display device area and the natural image area. In S901, the identification unit 801 acquires input image data from the reception unit 101, and identifies which imaging device the input image data is input from based on the metadata of the input image data. Next, the process proceeds to the process of S301, and a process of acquiring the pixel values of the white patch from the input image data is executed.

[0039] In the next S902, the parameter generation unit 103 calculates color adjustment parameters from the pixel values of the white patch. The memory unit 108 stores the color adjustment parameters together with information for associating them with the imaging device for use in matching the colors of the external display device area and the natural image area. Then, a series of processes ends.

[0040] FIG. 10 is a flowchart showing an example of a process of performing color matching between the external display device area and the natural image area and displaying an image. Note that the details of the processes of S601 and S603 described in FIG. 6 are omitted.

[0041] In S1001, the identification unit 801 acquires input image data from the reception unit 101, and identifies which imaging device the input image data is input from based on the metadata of the input image data. Next, after the process of S601 is performed, the process proceeds to the process of S1002.

[0042] In S1002, the color conversion unit 105 applies color conversion using the video data and region information input from the detection unit 104 and the color adjustment parameters associated with the imaging device stored in the memory unit 108. The color conversion unit 105 outputs the video data after color conversion is applied to the region of the external display device using the color adjustment parameters associated with the imaging device to the display unit 106. Next, the process proceeds to S603, and the image after color conversion is output to the screen. Then, a series of processes is terminated.

[0043] In the present embodiment, the imaging device is identified by the metadata of the video signal, and color matching with the region of the natural image is performed based on the color adjustment parameters associated with the imaging device for the region of the external display device detected from the image. It is possible to suppress the influence of subtle differences in the sensitivity of the imaging elements provided for each imaging device and perform color adjustment only for the region of the external display device to match the color with the region of the natural image.

[0044] [Third Embodiment] Next, the third embodiment will be described. In the present embodiment, the display control device can change the overall appearance color matching of the screen based on color editing parameters such as CDL and LUT prepared by the user. At this time, color conversion with different processing contents is applied to the region of the natural image and the region of the external display device. In the present disclosure, "color editing" refers to color conversion using CDL, LUT, etc. prepared by the user, and "color adjustment" refers to color conversion using color adjustment parameters calculated by the display control device from the patch measurement values.

[0045] Regarding the display control device, the processing content of the color conversion unit 105 is different from that in the above embodiment. The color conversion unit 105 performs color editing of the entire screen based on the region information and video data input from the detection unit 104, and performs color adjustment for the region of the external display device. The parameters used by the color conversion unit 105 for color conversion are the color editing parameters specified by the user and the color adjustment parameters generated by the parameter generation unit 103. The color conversion unit 105 performs color conversion processing using different parameters for the region of the external display device and the region of the natural image, and outputs the processed video data to the display unit 106.

[0046] Specifically, the color conversion unit 105 performs color conversion represented by the following formula (2) to calculate the pixel values (r_out, g_out, b_out) after color conversion from the pixel values (r_in, g_in, b_in) before color conversion.

Equation

[0047] M in formula (2) represents color editing parameters such as CDL or LUT prepared by the user. M is applied to both the area of the natural image and the area of the external display device in order to change the overall appearance of the screen. On the other hand, the color adjustment parameters (p_r, p_g, p_b) are applied only to the area of the external display device.

[0048] Regarding the order of color adjustment and color editing, color adjustment is performed first, and then color editing is performed. For example, assume a color conversion represented by "(r_out, g_out, b_out)=(0.9×r_in, g_in, b_in)" as color editing, and set the color adjustment parameters to (1.0, 1.0, 0.9). The total color conversion applied to the area of the external display device is represented by "(r_out, g_out, b_out) =(0.9×r_in, g_in, 0.9×b_in)".

[0049] FIG. 11 is a schematic diagram for explaining an example of the input to the color conversion unit 105 and the corresponding output. FIG. 11(A) shows the image before color conversion, and shows the area 1101 of the natural image corresponding to the person and the area 1102 of the external display device corresponding to the background. There is a deviation in white balance between the area 1101 of the natural image and the area 1102 of the external display device, and it represents that the image is before color editing for the whole image.

[0050] FIG. 11(B) shows the image after color conversion. In the image where color adjustment and color editing have been performed, the area 1103 of the natural picture corresponding to the person and the area 1104 of the external display device corresponding to the background are shown. Color conversion is applied to the area 1103 of the natural picture based on the color editing parameters prepared by the user. Color conversion is applied to the area 1104 of the external display device based on the color adjustment parameters for color matching between the area of the external display device generated by the display control device and the area of the natural picture, and the color editing parameters prepared by the user.

[0051] For example, assume that the pixel value of the area 1101 of the natural picture shown in FIG. 11(A) is (255, 255, 255), and the pixel value of the area 1102 of the external display device is (230, 230, 255). In this case, for the area 1101, “(r_out, g_out, b_out) = (0.9×r_in, g_in, b_in)”, which corresponds to the conversion of color editing, is applied. For the area 1102, “(r_out, g_out, b_out) = (0.9×r_in, g_in, 0.9×b_in)”, which corresponds to the total color conversion of color adjustment and color editing, is applied. The pixel value of the area 1103 after color conversion shown in FIG. 11(B) becomes (230, 255, 255), and the pixel value of the area 1104 becomes (207, 230, 230). At this time, when the pixel values of the area 1103 and the area 1104 are normalized by dividing each pixel value by the maximum value of each pixel value, the pixel values of both areas become (0.9, 1.0, 1.0). It can be seen that the white balance is approaching between the area 1103 and the area 1104, and the appearance of the entire screen has been changed by applying the color conversion using the color editing parameters prepared by the user.

[0052] In the present embodiment, color editing is applied to the area of the natural picture using predetermined color editing parameters. For the area of the external display device, color editing and color adjustment are applied by color conversion using the predetermined color editing parameters and the color adjustment parameters generated by the display control device. The predetermined color editing parameters are, for example, color editing parameters prepared by the user and specified by the operation unit. Alternatively, the predetermined color editing parameters are the color editing parameters specified by the user via the operation unit among a plurality of color editing parameters pre-held by the display control device. According to the present embodiment, it is possible to adjust the overall appearance of the display image while suppressing the influence on the image caused by the color difference due to the difference in spectral characteristics between the area of the external display device and the area of the natural picture.

[0053] [Fourth Embodiment] Next, the fourth embodiment will be described. The display control device of the present embodiment outputs color conversion parameters for application to the area of the natural picture and the area of the external display device as a color conversion file. Information on the imaging device to be applied, the application frame, and the application area is given as metadata information in the color conversion file. Hereinafter, an example of outputting CDL data as the color conversion file will be shown. In the present embodiment, the "color conversion process" refers to any one of the processes of "color editing", "color adjustment", and "color adjustment and color editing".

[0054] FIG. 12 is a block diagram showing a configuration example of the display device according to the present embodiment. In the display device 1200, a file generation unit 1201 is added to the configuration of the display device 800 shown in FIG. 8. The file generation unit 1201 calculates different color conversion parameters for the area of the external display device and the area of the natural picture based on the color editing parameters specified by the user and the color adjustment parameters generated by the parameter generation unit 103. The file generation unit 1201 generates a color conversion file according to the command of the control unit 107. Region information is given not only to the color conversion parameters but also to the color conversion file generated by the file generation unit 1201.

[0055] Furthermore, the file generation unit 1201 receives, from the control unit 107 via the detection unit 104, the identification information and time code of the imaging device, which are obtained from the metadata of the signal at the timing when the color conversion unit 105 starts color conversion. The file generation unit 1201 attaches the acquired identification information (camera identifier) and time code of the imaging device to the color conversion file as metadata.

[0056] FIG. 13 is a diagram for explaining an example of a file generated by the file generation unit 1201. FIG. 13(A) shows a file for the area of the external display device, and FIG. 13(B) shows a file for the area of the natural picture. Slope, Offset, and Power are parameters applied individually for each RGB color. Slope is used for adjusting the white level of the image, Offset is used for adjusting the black level of the image, and Power is used for gamma adjustment of the image. For example, in the case of "(r_out, g_out, b_out)=(0.9×r_in, g_in, 0.9×b_in)" corresponding to color conversion, Slope=(0.9, 1.0, 0.9). Also, Saturation is a parameter applied equally to RGB and is used for adjusting the saturation of the image. As extended meta information, for example, the following information is described. · Camera identifier: Information indicating to which camera the color conversion file is applied. · Time code: Information indicating from which frame the color conversion file is applied. · Area information: Information indicating whether the color conversion file is applied to the area of the external display device or the area of the natural picture.

[0057] For example, when the set value of the area information is zero, the area to which the color conversion file is applied is the area of the external display device. When the set value of the area information is 1, the area to which the color conversion file is applied is the area of the natural picture.

[0058] The color conversion file generated by the file generation unit 1201 is output to an external device. For example, a case where a video processing device as an external device uses the color conversion file will be described. Time code and area information are acquired from the video signal input from the display device 1200 to the video processing device. The video processing device determines whether the color conversion file describes the acquired time code and area information. When the video processing device determines that it is the color conversion file, color conversion is applied to the area corresponding to the information described in the color conversion file.

[0059] The display control device of the present embodiment outputs color conversion parameters for applying to the area of the natural image and the area of the external display device as a color conversion file. Identification information of the imaging device to be applied, information of the applicable frame, and information of the applicable area are given as metadata to the color conversion file. Since the content of the color conversion performed for each area to be processed can be recorded as a numerical value, it is possible to easily reproduce on-site color adjustment based on the recorded data of the color conversion file in a subsequent shooting process or editing process.

[0060] In the present embodiment, a CDL file is described as an example of the color conversion file. However, the file generated by the file generation unit 1201 may be a 1D-LUT file or a 3D-LUT file.

[0061] [Other Embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0062] Further, without being limited to the above-described embodiments, embodiments obtained by deformation or modification within the scope of the gist of the present invention are also included in the technical scope of the present invention. Embodiments of the present disclosure include the following configurations, methods, and programs.

[0063] [Configuration 1] An acquisition unit that acquires image data having a first region corresponding to a region where an external display device displays an image and a second region corresponding to a subject different from the external display device; A detection unit that detects the first region from the image data; A generation unit that generates a first parameter for color matching between the first region and the second region; A color conversion unit that performs color adjustment of an image using the first parameter, and the color conversion unit performs color adjustment on the first region detected by the detection unit using the first parameter. A display control device characterized by the above. [Configuration 2] The detection unit detects, as the first region, a region corresponding to the distance of the external display device from information acquired by a distance measurement unit that measures the distance of the external display device, and detects, as the second region, a region having a different distance from the region. The display control device according to Configuration 1, characterized by the above. [Configuration 3] The distance from the distance measurement unit to the subject is smaller than the distance from the distance measurement unit to the external display device. The display control device according to Configuration 2, characterized by the above. [Configuration 4] The generation unit generates the first parameter that reduces the difference between the pixel value of the image displayed on the external display device and the pixel value of the image captured by the imaging device. The display control device according to any one of Configurations 1 to 3, characterized by the above. [Configuration 5] It has an identification unit that acquires identification information of the imaging device, the generation unit generates the first parameter in association with the identification information, and the color conversion unit performs color adjustment using the first parameter associated with the identification information. The display control device according to any one of Configurations 1 to 4, characterized by the above. [Configuration 6] The color conversion means performs color editing for the entire screen using the specified second parameter. The display control device according to any one of Configurations 1 to 5, characterized in that. [Configuration 7] The color conversion means performs color adjustment using the first parameter and color editing using the second parameter for the first region, and performs color editing using the second parameter for the second region. The display control device according to Configuration 6, characterized in that. [Configuration 8] It has file generation means for generating a file having the first parameter. The file generation means assigns one or more pieces of information among the identification information of the imaging device to which the color conversion is applied to the image, the information indicating the frame of the image to which the color conversion is applied, and the information indicating the image area to which the color conversion is applied, to the first parameter. The display control device according to any one of Configurations 1 to 7, characterized in that. [Configuration 9] The generation means generates the first parameter from the pixel value or luminance or chromaticity of an image obtained by photographing a plurality of image areas displayed on the external display device. The display control device according to any one of Configurations 1 to 8, characterized in that. [Configuration 10] A display control device according to any one of Configurations 1 to 9, Imaging means for performing imaging related to the first and second regions and outputting the image data to the display control device, Display means for displaying the image subjected to the color adjustment, comprising An image processing system characterized in that. [Method 1] A display control method executed by a display control device for controlling the display of an image, A step of acquiring image data having a first region corresponding to a region where an external display device displays an image and a second region corresponding to a subject different from the external display device. A detection step of detecting the first region from the image data; A generation step of generating parameters for color matching between the first region and the second region; A color conversion step of performing color adjustment of an image using the generated parameters, and having: In the color conversion step, color adjustment is performed on the first region detected by the detection step using the parameters generated by the generation step. A display control method characterized by the above. [Program] A program for causing a computer to execute each step described in Method 1.

Explanation of Signs

[0064] 100, 800, 1200 Display device 103 Parameter generation unit 105 Color conversion unit 801 Identification unit 1201 File generation unit

Claims

1. an acquiring means for acquiring image data having a first region corresponding to a region in which an external display device displays an image and a second region corresponding to a subject different from the external display device; a detection means for detecting the first region from the image data; a generating means for generating a first parameter for performing color matching between the first region and the second region; a color conversion means for adjusting the color of an image using the first parameter; The color conversion means performs color adjustment on the first region detected by the detection means using the first parameter. A display control device comprising:

2. The detection means detects an area corresponding to the distance of the external display device as the first area from information acquired by a distance measurement means that measures the distance to the external display device, and detects an area having a different distance from the first area as the second area.

2. The display control device according to claim 1.

3. The distance from the distance measuring means to the subject is shorter than the distance from the distance measuring means to the external display device.

3. The display control device according to claim 2.

4. The generating means generates the first parameter that reduces a difference between pixel values of an image displayed on the external display device and pixel values of an image captured by an imaging device.

2. The display control device according to claim 1.

5. an identification unit for acquiring identification information of the imaging device; the generating means generates the first parameter by linking it with the identification information; The color conversion means performs color adjustment using the first parameter associated with the identification information.

2. The display control device according to claim 1.

6. The color conversion means performs color editing for the entire screen using the designated second parameters.

2. The display control device according to claim 1.

7. The color conversion means performs color adjustment using the first parameter and color editing using the second parameter for the first region, and performs color editing using the second parameter for the second region.

7. The display control device according to claim 6,

8. a file generating means for generating a file having the first parameters; The file generation means assigns, to the first parameters, one or more pieces of information selected from the group consisting of identification information of an image capture device to which the color conversion is applied, information indicating a frame of the image to which the color conversion is applied, and information indicating an image region to which the color conversion is applied.

2. The display control device according to claim 1.

9. The generating means generates the first parameters from pixel values, luminance, or chromaticity of images captured from a plurality of image areas displayed on the external display device.

2. The display control device according to claim 1.

10. A display control device according to any one of claims 1 to 9; an imaging means for imaging the first and second areas and outputting the image data to the display control device; a display means for displaying the color-adjusted image. An image processing system comprising:

11. A display control method executed by a display control device that controls the display of an image, acquiring image data having a first region corresponding to a region in which an external display device displays an image and a second region corresponding to a subject different from the external display device; a detecting step of detecting the first region from the image data; a generating step of generating parameters for performing color matching between the first region and the second region; a color conversion step of performing color adjustment of the image using the generated parameters, In the color conversion step, color adjustment is performed on the first region detected in the detection step using the parameters generated in the generation step. A display control method comprising:

12. A program that causes a computer to execute the steps according to claim 11.

Images