Information processing device, information processing method, and information processing program

Abstract

An information processing device according to the present invention is provided with an interpolation target color difference pixel generation unit which generates interpolation target color difference pixels corresponding to first color difference pixels in an interpolation target pixel location on the basis of the relationship between: brightness pixels in the interpolation target pixel location, which is a pixel location where the first color difference pixels are not present, in image data that has been generated on the basis of three-primary-color pixels which can include values greater than a prescribed white clipping value and in which the number of brightness pixels is greater than the number of the first color difference pixels and the number of second color difference pixels; and brightness pixels in at least one adjacent pixel location among the plurality of pixel locations adjacent to the interpolation target pixel location.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Traditionally, there are two methods for recording images: one is to develop the output of an imager and save the resulting image (such as JPG), and the other is to record the information contained in the imager output as is (RAW recording) so that the gradation, color, etc. can be adjusted and developed later.

[0003] In RAW recording, in addition to the method of recording the imager output (for example, RGB signals in a Bayer array) as is, there is a method of performing some signal processing in the camera to shorten the development processing time, separating the signal into a luminance signal and a color difference signal, and recording YCbCr. In the case of YCbCr, the color difference signal CbCr may be thinned out to reduce the amount of data (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-5245 A Summary of the Invention [Problem to be solved by the invention]

[0005] When the color difference signals CbCr are thinned out, the thinned color difference signals are interpolated before development, but conventionally, the interpolation was done using only the color difference signals, without using the luminance signal, which resulted in problems such as unintended coloring, which reduced the quality of the developed image.

[0006] The present technology has been developed in consideration of these points, and aims to provide an information processing device, an information processing method, and an information processing program that can interpolate thinned pixels without degrading the image quality of the image. [Means for solving the problem]

[0007] In order to solve the above-described problem, a first technique includes an interpolation target chrominance pixel generation unit that generates an interpolation target chrominance pixel corresponding to the first chrominance pixel at an interpolation target pixel position based on a relationship between a luminance pixel at an interpolation target pixel position, which is a pixel position where no first chrominance pixel is present, and a luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the interpolation target pixel position, in image data in which the number of luminance pixels is greater than the number of first chrominance pixels and the number of second chrominance pixels, the image data being generated based on three primary color pixels that may include a value greater than a predetermined white clip value; The chrominance pixel generation unit for interpolation calculates an average value of a ratio between a value of a luminance pixel at the position of the pixel to be interpolated and a value of a luminance pixel at a neighboring pixel position, and a multiplication value of chrominance pixels at a plurality of neighboring pixel positions, to generate a chrominance pixel for interpolation. It is an information processing device.

[0008] The second technique is a technique for generating a luminance pixel at an interpolation target pixel position where no first chrominance pixel is present in image data in which the number of luminance pixels is greater than the number of first chrominance pixels and the number of second chrominance pixels, the image data being generated based on three primary color pixels that may include a value greater than a predetermined white clip value. Value of and a luminance pixel at at least one adjacent pixel position among a plurality of pixel positions adjacent to the pixel position to be interpolated. The ratio of the values ​​of , Calculate the average value of the multiplication values ​​of the chrominance pixels at the adjacent pixel positions; The information processing method generates a chrominance pixel to be interpolated, which corresponds to a first chrominance pixel, at a position of a pixel to be interpolated.

[0009] Furthermore, a third technique includes a method for interpolating a luminance pixel at an interpolation target pixel position where a first chrominance pixel is not present in image data in which the number of luminance pixels is greater than the number of first chrominance pixels and the number of second chrominance pixels, the image data being generated based on three primary color pixels that may include a value greater than a predetermined white clip value. Value of and a luminance pixel at at least one adjacent pixel position among a plurality of pixel positions adjacent to the pixel position to be interpolated. The ratio of the values ​​of , Calculate the average value of the multiplication values ​​of the chrominance pixels at the adjacent pixel positions; An information processing program that causes a computer to execute an information processing method for generating an interpolation target chrominance pixel corresponding to a first chrominance pixel at an interpolation target pixel position. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the configurations of an imaging device 10 and a developing device 20. FIG. [Diagram 2] FIG. 1 is an explanatory diagram of a mechanism of image quality degradation. [Diagram 3] 1 is a block diagram showing a configuration of an information processing device 100 according to a first embodiment. [Figure 4] FIG. 11 is an explanatory diagram of an interpolation target pixel position and an adjacent pixel position. [Diagram 5] FIG. 13 is an explanatory diagram of the necessity of an offset value Δ. [Figure 6] FIG. 13 is an explanatory diagram of an offset value Δ. [Figure 7] 4A to 4C are diagrams illustrating an interpolation process according to the first embodiment. [Figure 8] FIG. 11 is a block diagram showing a configuration of an information processing device 200 according to a second embodiment. [Figure 9] FIG. 11 is an explanatory diagram of a first technique of the interpolation process in the second embodiment. [Figure 10] FIG. 13 is a diagram illustrating a first technique of an interpolation process in the second embodiment. [Figure 11] 13 is a table showing a second technique for the interpolation process according to the second embodiment. [Figure 12] FIG. 13 is a block diagram showing a configuration of an information processing device 300 according to a third embodiment. [Figure 13] FIG. 13 is an explanatory diagram of an interpolation process according to the third embodiment. [Figure 14] FIG. 13 is a block diagram showing a configuration of an information processing device 400 according to a modified example. [Figure 15] FIG. 13 is an explanatory diagram of adjacent pixel position selection in a modified example. [Figure 16] FIG. 13 is an explanatory diagram of an arrangement of pixel positions according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order. <1. First embodiment> [1-1. Configuration of imaging device 10 and developing device 20] [1-2. Mechanism of image quality degradation] [1-3. Configuration of information processing device 100] [1-4. Processing by Information Processing Device 100] <2. Second embodiment> [2-1. Configuration of information processing device 200] [2-2. Processing by Information Processing Device 200] <3. Third embodiment> [3-1. Configuration of information processing device 300] [3-2. Processing by Information Processing Device 300] <4. Modifications>

[0012] <1. First embodiment> [1-1. Configuration of imaging device 10 and developing device 20] 1, the configuration of an imaging device 10 that generates image data by photographing, and a developing device 20 that develops the photographed image data will be described. The imaging device 10 is configured to include a control unit 11, an imaging unit 12, a white balance processing unit 13, a color separation processing unit 14, a resizing processing unit 15, a signal conversion unit, a thinning processing unit 17, and an interface 18.

[0013] The control unit 11 is composed of a CPU (Central Processing Unit), a RAM (Random access Memory), a ROM (Read Only Memory), etc. The CPU executes various processes according to a program stored in the ROM and issues commands to control the entire imaging device 10 and each part thereof.

[0014] The imaging unit 12 is capable of capturing an image corresponding to three primary color pixels. The imaging unit 12 includes a lens, an imaging element that photoelectrically converts incident light from a subject obtained through the lens into an electric charge and outputs an imaging signal, a lens driver that drives the lens, and a processing unit that performs A / D (Analog / Digital) conversion on the imaging signal to generate image data. Examples of imaging elements include a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor).

[0015] The white balance processor 13 performs white balance processing on the image data based on an arbitrary white balance setting set by the user.

[0016] The color separation processing unit 14 performs, for example, demosaic processing as a color separation process on the image data that has been subjected to the white balance process.

[0017] A resizing processor 15 performs resizing processing on the image data to adjust the size by changing the number of pixels, for example.

[0018] The data conversion processing unit 16 converts the RGB image data into YC image data consisting of a luminance signal and a color difference signal.

[0019] The thinning processing unit 17 thins out chrominance pixels from the image data. The image data before thinning is YC 4:4:4 (luminance signal Y: chrominance signal Cb: chrominance signal Cr = 4:4:4), and the luminance pixels are thinned out by the thinning process, so that the image data becomes, for example, YC 4:2:2 (luminance Y: chrominance Cb: chrominance Cr = 4:2:2). The thinning process reduces the file size of the image data, thereby shortening the transfer time, etc.

[0020] The interface 18 is an interface between the developing device 20 and other devices, networks, etc. The interface 18 may include a wired or wireless communication interface. More specifically, the wired or wireless communication interface may include cellular communication such as 3TTE, 4G, 5G, Wi-Fi, Bluetooth (registered trademark), NFC (Near Field Communication), Ethernet (registered trademark), HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), etc. In addition, when the imaging device 10 and the developing device 20 are all or partly realized in the same device, the interface 18 may include a bus in the device, data reference in a program module, etc. (hereinafter, these are also referred to as interfaces in the device). In addition, when the imaging device 10 and the developing device 20 are realized in a distributed manner in a plurality of devices, the interface 18 may include different types of interfaces for each device. For example, the interface 18 may include both a communication interface and an interface in the device.

[0021] The image data that has been subjected to the thinning process is transferred from the imaging device 10 to the developing device 20 via the interface 18 for development. This image data is the image data to be processed in the information processing device 100. The image data to be processed is image data that has been generated based on three primary color pixels that may include values ​​greater than a predetermined white clip value, and in which the number of luminance pixels (Y) is greater than the number of first chrominance pixels (one of Cb or Cr) and the number of second chrominance pixels (the other of Cb or Cr) due to the thinning process.

[0022] Although not shown in the figure, the camera is equipped with a memory unit for storing image data, an input unit including a shutter button operated by the photographer, and a display unit for displaying a through image, captured images, a UI, etc.

[0023] The imaging device 10 is configured as described above. Examples of the imaging device 10 include devices specialized for camera functions, such as digital cameras, single-lens reflex cameras, camcorders, commercial cameras, and professional-grade imaging equipment, as well as smartphones, tablet terminals, wearable devices, and portable game consoles that are equipped with camera functions.

[0024] The developing device 20 includes a control unit 21, an interface 22, an information processing device 100, a data conversion processing unit 23, a white balance processing unit 24, a white clip processing unit 25, and a developing processing unit .

[0025] The control unit 21 is realized by the CPU executing various processes according to a program and issuing commands, and controls the entire developing device 20 and each of its components.

[0026] The interface 22 is similar to that provided in the imaging device 10, and serves as an interface between the imaging device 10 and other devices, networks, etc. Image data to be processed by the information processing device 100 is transferred from the imaging device 10 to the developing device 20 via the interface 22.

[0027] The information processing device 100 performs the interpolation process by generating chrominance pixels to be interpolated at pixel positions where chrominance pixels have been thinned out. The configuration of the information processing device 100 will be described later.

[0028] The data conversion processing unit 23 converts the YC image data, which is made up of a luminance signal and a color difference signal, into RGB image data.

[0029] The white balance processor 24 performs white balance processing on the RGB image data.

[0030] The white clip processing unit 25 performs white clip processing on the image data that has been subjected to the white balance processing. The white clip value that serves as the reference for processing in the white clip processing unit corresponds to a predetermined white clip value, and the image data to be processed in this technology is assumed to be image data made up of three primary color pixels that may include values ​​larger than the predetermined white clip value.

[0031] The development processing unit 26 performs processes such as edge emphasis, resolution conversion, and image compression, and generates image data that is ultimately saved and output as a captured image.

[0032] The developing device 20 is configured as described above. The developing device 20 may be configured as a standalone device, or may operate on a terminal device such as a personal computer, a tablet terminal, or a smartphone. A program for performing the processing of the developing device 20 may be pre-installed in the terminal device, or may be distributed by download or storage medium, and installed by the photographer himself. The developing device 20 may also be configured as a dedicated device with hardware having the function.

[0033] [1-2. Causes of deterioration in image quality] Here, with reference to Fig. 2, we will explain the mechanism of image quality degradation that occurs when performing interpolation processing using a method that uses only chrominance pixels after thinning out chrominance pixels from image data. Here, we will explain using an example where three pixels, white (bloated highlights), black 1, and black 2, are the targets of processing. Black 1 and black 2 have low RGB values. On the other hand, white (bloated highlights) has high RGB values, and G and R are saturated beyond the dynamic range of the imaging element. This is due to the difference in the characteristics of the RGB color filters.

[0034] As shown in Figure 2A, when white balance processing is performed on the white, black 1, and black 2 pixels, the R and B signal levels become higher than the G signal level. In order to retain as much color information as possible as RAW data so that the photographer can adjust RGB by performing white balance processing again during development, white clip processing is not performed before interpolation processing, and the R and B signal levels are allowed to be higher than the G signal level. Then, as shown in Figure 2B, the R and B of the white pixel are raised and emphasized, becoming magenta (a bright and vivid reddish purple). Also, let us assume that the color difference of the central black 1 out of the white, black 1, and black 2 pixels is thinned out by thinning processing. Therefore, the black 1 pixel becomes the pixel position to be interpolated, and the white pixel and the black pixel become the adjacent pixel positions used for interpolation. Note that the luminance signal of the thinned out black 1 pixel remains without being thinned out.

[0035] In this state, if the thinned out black 1 pixel is interpolated using only the color difference signals of the adjacent white and black 2 pixels, a non-black color will appear at the pixel position to be interpolated due to the magenta white, as shown in Figure 2C. Note that the magenta white returns to white as the R and B values ​​raised by the white clip process after the interpolation process are suppressed.

[0036] In this way, the interpolation process adds a color other than the original color of the pixel, resulting in unnatural colors in the output image and a decrease in image quality. Note that although black and white pixels are used as an example here, this technology is not limited to black and white.

[0037] [1-3. Configuration of information processing device 100] Next, the configuration of the information processing device 100 in the first embodiment will be described with reference to Fig. 3. The information processing device 100 includes a neighboring pixel position selection unit 101, an interpolation target chrominance pixel generation unit 102, and a switch unit 103.

[0038] The information processing device 100 receives image data to be processed, which is generated based on three primary color pixels that may include values ​​greater than a predetermined white clip value, and in which the number of luminance pixels (Y) is greater than the number of first chrominance pixels (one of Cb or Cr) and the number of second chrominance pixels (the other of Cb or Cr). This image data is assumed to be YC4:2:2 image data in which chrominance signals have been thinned out by a thinning process in the imaging device 10. It is assumed that the control unit 21 or the like knows which pixels are being processed among the pixels constituting the image data.

[0039] The adjacent pixel position selection unit 101 selects the position of an adjacent pixel to be used for generating an interpolation target chrominance pixel from among a plurality of pixel positions, based on a luminance signal and / or a chrominance signal constituting the image data to be processed. The signal input to the adjacent pixel position selection unit 101 may be a luminance signal or a chrominance signal.

[0040] The neighboring pixels are not necessarily adjacent to the target pixel position, but are pixels located within a predetermined range around the target chrominance pixel. For example, they may be pixels two or three pixels away from the target chrominance pixel. From the viewpoint of the accuracy of the interpolation, it is more preferable that the neighboring pixels are adjacent to the target chrominance pixel. The target pixel position is a pixel position where chrominance pixels are thinned out by the thinning process, that is, a pixel position where at least one of the first chrominance pixel and the second chrominance pixel is not present, and is a pixel position to be interpolated by the target chrominance pixel generation unit 102.

[0041] Two or more pixels adjacent to the pixel position to be interpolated may be selected as the adjacent pixel positions, or an edge may be extracted and the adjacent pixel positions may be selected based on the results, or the adjacent pixel positions may be selected based on the format of the image data to be processed (YC4:2:2, YC4:2:0, etc.), or the adjacent pixel positions may be selected based on the correlation of luminance or color difference.

[0042] The chrominance pixel generation unit 102 for interpolation calculates the chrominance at the pixel position for interpolation using the luminance and chrominance at adjacent pixel positions based on the positive correlation between the magnitudes (absolute values) of the luminance and chrominance values ​​constituting the image data, and generates a chrominance pixel for interpolation. By generating the chrominance pixel for interpolation at the pixel position for interpolation, the thinned-out pixels are interpolated. The positive correlation may be linear, or may be expressed by a nonlinear function such as a quadratic function, a cubic function, or an exponential function.

[0043] The switch unit 103 outputs the chrominance pixels to be interpolated generated by the chrominance pixel to be interpolated generation unit 102 and all chrominance pixels other than those at the positions of the chrominance pixels to be interpolated in the image data in an appropriate order as all chrominance pixels constituting the image data.

[0044] Since all of the thinned-out chrominance pixels are interpolated using the chrominance pixels to be interpolated, the image data output from the information processing apparatus 100 has a YC4:4:4 format.

[0045] The information processing device 100 is configured as described above. The information processing device 100 may be configured by a program, and execution of the program may cause the developing device 20 to have the functions of the information processing device 100. The program may be installed in advance in a terminal device on which the developing device 20 operates, or may be distributed by download or storage medium, etc., and installed by the user in the terminal device. The information processing device 100 may also be configured as a dedicated device using hardware having that function.

[0046] Furthermore, the information processing device 100 may be configured to have the functions of the white balance processing unit 13, the imaging unit 12, and the imaging device .

[0047] [1-4. Processing by Information Processing Device 100] Next, the process of the information processing device 100 in the first embodiment will be described.

[0048] First, the adjacent pixel position selection unit 101 selects an adjacent pixel position to be used for generating an interpolation target chrominance pixel at an interpolation target pixel position from among a plurality of pixel positions constituting the image data. Note that the number of adjacent pixel positions is not limited to one, and may be one or three or more.

[0049] When the adjacent pixel position selection unit 101 selects the adjacent pixel position, adjacent pixel position information is supplied to an interpolation target chrominance pixel generation unit 102 .

[0050] 4, the value of the chrominance pixel at the pixel to be interpolated is defined as chrominance CC, and the value of the luminance pixel at the pixel to be interpolated is defined as luminance YC. The value of the chrominance pixel at the first neighboring pixel used for the interpolation process is defined as chrominance CL, and the value of the luminance pixel at the first neighboring pixel is defined as luminance YL. The value of the chrominance pixel at the second neighboring pixel used for the interpolation process is defined as chrominance CR, and the value of the luminance pixel at the second neighboring pixel is defined as luminance YR.

[0051] In the first embodiment, the chrominance pixel generation unit 102 calculates the chrominance CC at the pixel position to be interpolated using the chrominance CL and luminance YL at the first adjacent pixel position and the chrominance CR and luminance YR at the second adjacent pixel position according to the following formula 1 to generate a chrominance pixel to be interpolated. This corresponds to "generating a chrominance pixel to be interpolated based on the relationship between the value of the luminance pixel at the pixel position to be interpolated and the value of the luminance pixel at the adjacent pixel position."

[0052] [Formula 1] TIFF0007626135000001.tif15126

[0053] Equation 1 is an equation that specifies that luminance and chrominance have a linear relationship among the positive correlations, and uses the ratio of chrominance CL to luminance YL at the first neighboring pixel position and the ratio of chrominance CR to luminance YL at the second neighboring pixel position to calculate chrominance pixel CC at the pixel to be interpolated, thereby generating a chrominance pixel to be interpolated.

[0054] In formula 1, the offset value Δ is used as an offset for the luminance in order to prevent noise from being amplified due to either the luminance YL or the luminance YR being small.

[0055] For example, consider a case where the luminance YC at the pixel position to be interpolated, the luminance YL and chrominance CL at the first neighboring pixel position, and the luminance YR and chrominance CR at the second neighboring pixel position have the values ​​shown in Fig. 5A. In this case, the first neighboring pixel position and the second neighboring pixel position are both dark because the luminance is small, and are also closer to the vertical axis side where chrominance C = 0, so the color is close to black. On the other hand, the pixel position to be interpolated has a large luminance YC, so it is brighter than the first neighboring pixel position and the second neighboring pixel position.

[0056] In this state, if there is noise in either the chrominance CL at the first adjacent pixel position or the chrominance CR at the second adjacent pixel position, and the first adjacent pixel position or the second adjacent pixel position is used to perform interpolation of the pixel position to be interpolated, the chrominance with noise is amplified by the luminance YC, and the noise becomes noticeable. Therefore, by using the offset value △, when the luminance YL and / or the luminance YR is large or small, the chrominance CL and / or the chrominance CR become too large, and as a result, the noise in the chrominance CC at the pixel position to be interpolated is prevented from being amplified. FIG. 5A shows the chrominance noise, but when there is noise in the luminance as shown in FIG. 5B, even if the amount of noise in the luminance is the same when the luminance is small and when the luminance is large, the calculated amount of noise becomes large when the luminance is small. Therefore, the offset value △ is also effective in preventing the luminance noise from being amplified.

[0057] The offset value Δ may be defined as a constant relative to the luminance Y as shown in FIG. 6A.

[0058] In addition, the offset value Δ can be defined as a function of luminance YC by the following equation 2.

[0059] [Formula 2] TIFF0007626135000002.tif1088

[0060] This formula 2 determines the offset value Δ based on the minimum value of the luminance YC, luminance YL, and luminance YR, and the relationship between the luminance Y and the offset value Δ can be expressed, for example, as shown in Fig. 6B. Note that the relationship between the luminance Y and the offset value Δ shown in Fig. 6B is merely an example, and the relationship between the luminance Y and the offset value Δ may be linear or nonlinear.

[0061] The offset value △ is necessary when the luminance Y is small, in other words, the offset value △ is effective when the luminance Y is small, so the offset value △ may be defined as a function of Y. When Y is small, the offset value △ has a value, and as Y increases, the offset value △ decreases, and when Y reaches a certain value, the offset value △ may be set to 0.

[0062] The offset value Δ may be determined according to the functions and settings of the imaging device 10 .

[0063] Although the offset value Δ is not essential for the interpolation process, it is preferable to use it in order to obtain better interpolation results.

[0064] Fig. 7 shows a schematic diagram of a process for generating a chrominance pixel to be interpolated by calculating the chrominance CC at the position of a pixel to be interpolated using Equation 1, with the luminance on the vertical axis (Y axis) and the chrominance on the horizontal axis (C axis). Fig. 7A shows a first example, and Fig. 7B shows a second example.

[0065] The color difference CC at the pixel position to be interpolated is the average of the value of color difference C at the intersection of a line LL, which connects a point corresponding to the color difference CL and luminance YL at the first neighboring pixel position from a point shifted by the origin on the Y axis by the offset value △, and a line RR, which connects a point corresponding to the color difference CR and luminance YR at the second neighboring pixel position from a point shifted by the origin on the Y axis by the offset value △, and the value of color difference C at the intersection of a line RR and a line Y=YC (dash line).

[0066] For example, as shown in the following formula 3, a different offset value Δ can be defined for each of the luminance YC, luminance YL, and luminance YR to calculate the chrominance CC at the pixel position to be interpolated.

[0067] [Formula 3] TIFF0007626135000003.tif22164

[0068] This involves assuming that the positive correlation between luminance Y and color difference C is a proportional relationship, and calculating the color difference CC at the pixel position to be interpolated by averaging the product of the ratio of the luminance YC at the pixel position to be interpolated and the luminance YL at the first neighboring pixel position and the color difference CL at the first neighboring pixel position, and the product of the ratio of the luminance YC at the pixel position to be interpolated and the luminance YR at the second neighboring pixel position and the color difference CR at the second neighboring pixel position.

[0069] Note that, because YC4:2:2 only thins out chrominance pixels in the horizontal direction, the luminance of adjacent pixels to the left and right of the pixel to be interpolated is used when the image data to be processed is YC4:2:2. On the other hand, because YC4:2:0 thins out chrominance pixels in both the horizontal and vertical directions, the luminance of adjacent pixels to the left and right, as well as adjacent pixels above and below, is used when the image data to be processed is YC4:2:0.

[0070] In the above manner, the interpolation process in the first embodiment of the present technology is performed. Note that the above description is for the case where there are two adjacent pixel positions, and when there is one adjacent pixel position, the color difference value of the adjacent pixel position becomes the color difference CC of the pixel position to be interpolated. When there are three adjacent pixel positions, the average of the three color difference values ​​becomes the color difference CC of the pixel position to be interpolated. Formulas 1, 2, and 3 in the first embodiment are all formulas when there are two adjacent pixel positions, and when there are three or more adjacent pixel positions, the formulas can be applied to any number of adjacent pixel positions by increasing the number of terms in each formula accordingly.

[0071] <2. Second embodiment> [2-1. Configuration of information processing device 200] Next, a second embodiment will be described. Note that the configurations of the imaging device 10 and the developing device 20 are similar to those of the first embodiment, so the description will be omitted.

[0072] 8, an information processing device 200 is configured to include a neighboring pixel position selection unit 101 and a switch unit 103, similarly to the first embodiment, and in the second embodiment, the processing in the chrominance pixel generation unit 201 to be interpolated differs from that in the chrominance pixel generation unit 102 to be interpolated in the first embodiment. The chrominance pixel generation unit 201 to be interpolated generates chrominance pixels to be interpolated based on the fact that there is a correlation between the magnitudes of luminance and chrominance constituting image data, in which when the luminance value is close, the chrominance also takes a close value. By generating the chrominance pixel to be interpolated at the position of this pixel to be interpolated, the thinned-out pixel is interpolated.

[0073] As in the first embodiment, in the second embodiment, thinned-out YC 4:2:2 image data is input as a processing target. Moreover, the information processing device 200 interpolates chrominance at all thinned-out pixel positions to be interpolated, and therefore, YC 4:4:4 image data is output from the information processing device 200.

[0074] The information processing device 200 is configured as described above. The information processing device 200 is similar to the first embodiment in that it may be configured by a program or as a dedicated device using hardware.

[0075] [2-2. Processing by Information Processing Device 200] The following describes the processing in the information processing device 200. Note that the processing in the adjacent pixel position selection unit 101 and the switch unit 103 is similar to that in the first embodiment.

[0076] Like the first embodiment, the second embodiment will be described using chrominance CC and luminance YC at the pixel to be interpolated, chrominance CL and luminance YL at the first adjacent pixel, and chrominance CR and luminance YR at the second adjacent pixel.

[0077] In the second embodiment, interpolation is performed by generating chrominance pixels to be interpolated by taking a weighted average using the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YL of the first adjacent pixel, and the luminance difference between the luminance YC and the luminance YR of the second adjacent pixel. This uses a weighted average by utilizing the fact that the chrominance value of the pixel to be interpolated is close to the chrominance of the adjacent pixel, which has a luminance value closer to the luminance value of the pixel to be interpolated. This luminance difference corresponds to the "relationship between the value of the luminance pixel at the pixel to be interpolated and the value of the luminance pixel at the adjacent pixel."

[0078] First, a first method of processing in the interpolation target chrominance pixel generation unit 201 will be described. In the first method, the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YL of the first adjacent pixel, and the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YR of the second adjacent pixel are compared with a threshold value th. Then, depending on whether the luminance difference is equal to or greater than the threshold value th, the interpolation process is performed.

[0079] 9A to 9G are diagrams showing the relationship between the luminance YC at the pixel to be interpolated, the luminance YL at the first adjacent pixel position, and the luminance YR at the second adjacent pixel position, with the vertical direction being the magnitude of luminance.

[0080] 9A, consider a case where the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YL of the first neighboring pixel is equal to or less than the threshold th, and the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YR of the second neighboring pixel is equal to or more than the threshold th. In this case, the chrominance CL of the first neighboring pixel, whose luminance difference is equal to or less than the threshold th, is used, and the chrominance CR of the second neighboring pixel is not used, and the chrominance CL is set as the chrominance CC of the pixel to be interpolated (chrominance CC = chrominance CL).

[0081] FIG. 10A shows a schematic representation of the interpolation process in the case of FIG. 9A, with the vertical axis (Y axis) representing luminance and the horizontal axis (C axis) representing color difference.

[0082] As a result of generating chrominance pixels to be interpolated in the case of FIG. 9A, the chrominance CC at the position of the interpolation target pixel becomes the same value as the chrominance CL at the position of the first adjacent pixel, and is located on the axis of chrominance CL, as shown in FIG. 10A.

[0083] 9B, consider a case where the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YL of the first neighboring pixel is equal to or greater than the threshold th, and the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YR of the second neighboring pixel is equal to or less than the threshold th. In this case, the chrominance CR of the second neighboring pixel, whose luminance difference is equal to or less than the threshold th, is used, and the chrominance CL of the first neighboring pixel is not used, and the chrominance CR is set as the chrominance CC of the pixel to be interpolated (chrominance CC = chrominance CR).

[0084] In Fig. 9A, the luminance YL of the first adjacent pixel position used for interpolation and the luminance YC of the pixel position to be interpolated are shown as the same luminance value, but the luminance YL and the luminance YC do not necessarily have to be the same, and it is sufficient if the luminance difference is equal to or less than the threshold value th. The same is true for the luminance YR of the second adjacent pixel position and the luminance YC of the pixel position to be interpolated in Fig. 9B.

[0085] As shown in Figures 9A and 9B, if the brightness difference between the pixel position to be interpolated and the first adjacent pixel position and / or the second adjacent pixel position is equal to or greater than the threshold th, there is an edge between the pixel position to be interpolated and the first adjacent pixel position and / or the second adjacent pixel position, and it is considered that they constitute a different subject, so it is better not to use them for interpolation.

[0086] 9C, consider a case where the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YL of the first adjacent pixel, and the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YR of the second adjacent pixel are both equal to or greater than a threshold value th. In this case, neither the chrominance CL of the first adjacent pixel nor the chrominance CR of the second adjacent pixel is used, and the chrominance CC of the pixel to be interpolated is set to 0 (achromatic color).

[0087] The luminance difference between the luminance YC and the luminance YL and the case where the luminance YC and the luminance YR are both equal to or greater than the threshold value th may be either the case where the luminance YL and the luminance YR are greater than or smaller than the luminance YC, as shown in FIG. 9D and FIG. 9E.

[0088] 10B shows an example of how the chrominance pixels to be interpolated in this case are generated, with the luminance on the vertical axis (Y axis) and the chrominance on the horizontal axis (C axis). The chrominance CC at the position of the pixel to be interpolated is 0.

[0089] 9F, consider the case where the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YL of the first adjacent pixel, and the luminance difference between the luminance YC of the pixel to be interpolated and the luminance YR of the second adjacent pixel are both equal to or less than the threshold th. In this case, the weighted average of the chrominances CL and CR is set as the chrominance CC of the pixel to be interpolated using the following formula 4.

[0090] [Formula 4] CC=(CL+CR) / 2

[0091] This is because, since the luminance difference is small, it is considered that the deviation between the chrominance CL and chrominance CR and the chrominance CC at the pixel position to be interpolated will be small even if the luminance difference is not used and only the chrominance CL and chrominance CR are used for interpolation.

[0092] In addition, the luminance difference between the luminance YC at the pixel position to be interpolated and the luminance YL at the first adjacent pixel position, and the case where the luminance YC at the pixel position to be interpolated and the luminance YR at the second adjacent pixel position are both below the threshold th, may be either when the luminance YL and luminance YR are greater than or less than the luminance YC, as shown in Figures 9F and 9G.

[0093] In this case, the generation of the chrominance pixel to be interpolated can be represented diagrammatically as shown in Figure 10C, with the vertical axis (Y axis) representing luminance and the horizontal axis (C axis) representing chrominance. When the luminance YL and chrominance CL at the first neighboring pixel position, the luminance YR and chrominance CR at the second neighboring pixel position, and the luminance YC at the pixel to be interpolated have the values ​​shown in Figure 10C, the chrominance CC at the pixel to be interpolated is the weighted average of the chrominance CL at the first neighboring pixel position and the chrominance CR at the second neighboring pixel position, and the chrominance CC is located at the average value of the chrominance CL and chrominance CR.

[0094] Note that the values ​​of the luminance YL and color difference CL at the first adjacent pixel position, the luminance YR and color difference YR at the second adjacent pixel position, and the luminance YC at the pixel position to be interpolated in Figure 10 are merely examples, and the present technology is not limited to these values.

[0095] Next, a second method of the processing in the interpolation target color difference pixel generation unit 102 will be described. In the second method, a plurality of thresholds for the luminance difference are used. Here, an example will be described in which two thresholds, a first threshold th1 and a second threshold th2, are used.

[0096] The relationship among the luminance difference, the threshold, and the color difference CC is shown in the table of FIG. 11. It is assumed that the first threshold th1 and the second threshold th2 have a relationship of "th2 < th1". Also, let the difference between the luminance YC and the luminance YL be DL (DL = abs(YC - YL)), and the difference between the luminance YC and the luminance YR be DR (DR = abs(YC - YR)).

[0097] Also, using DL, DR, the first threshold th1, and the second threshold th2, α and β are defined as in the following Expressions 5 and 6.

[0098] [Expression 5] α = (DL - th2) / (th1 - th2)

[0099] [Expression 6] β = (DR - th2) / (th1 - th2)

[0100] However, in Expressions 5 and 6, it is assumed that they are clipped to 0 ≤ α ≤ 1 and 0 ≤ β ≤ 1.

[0101] When DL ≤ th2 (where α = 0) and DR ≤ th2 (where β = 0), the color difference CC at the interpolation target pixel position is calculated by the expression shown in (1) of the table in FIG. 11.

[0102] Also, when th2 ≤ DL ≤ th1 (where 0 ≤ α ≤ 1) and DR ≤ th2 (where β = 0), the color difference CC at the interpolation target pixel position is calculated by the expression shown in (2) of the table in FIG. 11.

[0103] Also, when th1 ≤ DL (where α = 1) and DR ≤ th2 (where β = 0), the color difference CR at the second closest pixel position is set as the color difference CC at the interpolation target pixel position as shown in (3) of the table in FIG. 11.

[0104] If DL≦th2 (where α=0) and th2≦DR≦th1 (where 0≦β≦1), the color difference CC at the pixel position to be interpolated is calculated by the formula shown in (4) of the table in FIG.

[0105] If th2≦DL≦th1 (where 0≦α≦1) and th2≦DR≦th1 (where 0≦β≦1), the color difference CC at the pixel position to be interpolated is calculated using the formula (5) in the table of FIG.

[0106] If th1≦DL (where α=1) and th2≦DR≦th1 (where 0≦β≦1), the color difference CC at the pixel position to be interpolated is calculated using the formula shown in table (6) of FIG.

[0107] Furthermore, if DL≦th2 (where α=0) and th1≦DR (where β=1), the color difference CL at the first adjacent pixel position is set as the color difference CC at the pixel position to be interpolated, as shown in (7) of the table in FIG.

[0108] If th2≦DL≦th1 (where 0≦α≦1) and th1≦DR (where β=1), the color difference CC at the pixel position to be interpolated is calculated by the formula shown in (8) of the table in FIG.

[0109] Furthermore, if th1≦DL (where α=1) and th1≦DR (where β=1), the color difference CC at the pixel position to be interpolated is set to 0 (achromatic color) as shown in (8) of the table in FIG.

[0110] All of (1) to (9) shown in the table of FIG. 11 can be expressed by the following formula 7.

[0111] [Formula 7] TIFF0007626135000004.tif61158

[0112] In addition, since YC4:2:2 thins out chrominance pixels only in the horizontal direction, if the image data to be processed is YC4:2:2, a comparison is made with the luminance of adjacent pixels located to the left and right of the pixel to be interpolated. On the other hand, since YC4:2:0 thins out chrominance pixels in both the horizontal and vertical directions, if the image data to be processed is YC4:2:0, a comparison is made with the luminance of adjacent pixels located to the left and right, and above and below.

[0113] The processing in the second embodiment is carried out as described above.

[0114] <3. Third embodiment> [3-1. Configuration of information processing device 300] Next, a configuration of an information processing device 300 in the third embodiment will be described with reference to Fig. 12. Note that the configurations of the imaging device 10 and the developing device 20 are similar to those in the first embodiment, and therefore description thereof will be omitted.

[0115] The third embodiment differs from the first and second embodiments in that it includes a generation method determination unit 301 and that an interpolation target chrominance pixel generation unit 302 can perform the processes in the first and second embodiments. The adjacent pixel position selection unit 101 and the switch unit 103 are the same as those in the first embodiment.

[0116] The generation method determination unit 301 determines whether the interpolation target chrominance pixel generation unit 302 will execute a generation method of the interpolation target chrominance pixels of the first embodiment or the second embodiment, based on luminance information of the image data to be processed. Information indicating the determined generation method is supplied from the generation method determination unit 301 to the interpolation target chrominance pixel generation unit 302. Note that, since the generation method determination unit 301 determines the generation method using luminance at the first neighboring pixel position and the second neighboring pixel position, it is supplied with neighboring pixel position information from the neighboring pixel position selection unit 101.

[0117] The information processing device 300 is configured as described above. The information processing device 300 is similar to the first embodiment in that it may be configured by a program or may be configured as a dedicated device using hardware.

[0118] [3-2. Processing by Information Processing Device 300] Next, determination of a generation method for the chrominance pixels to be interpolated by the generation method determination unit 301 will be described.

[0119] Similar to the first embodiment, the third embodiment will be described using chrominance CC and luminance YC at the pixel to be interpolated, chrominance CL and luminance YL at the first adjacent pixel, and chrominance CR and luminance YR at the second adjacent pixel.

[0120] The generation method determination unit 301 determines a generation method for the chrominance pixel to be interpolated by comparing the luminance difference between the luminance YC of the pixel to be interpolated, which is included in the image data to be processed, and the luminance difference between the luminance YL of the first adjacent pixel position and the luminance difference between the luminance YR of the second adjacent pixel position with a threshold value th.

[0121] 13A to 13E, when the luminance difference between the luminance YC at the pixel position to be interpolated and the luminance YL at the first adjacent pixel position and / or the luminance difference between the luminance YR at the second adjacent pixel position is equal to or greater than a threshold value th, the method for generating the chrominance pixel to be interpolated is determined to be the method in the second embodiment. The method in the second embodiment is a method for generating the chrominance pixel to be interpolated based on the difference (luminance difference) between the value of the luminance pixel at the pixel position to be interpolated and the value of the luminance pixel at the adjacent pixel position.

[0122] Specifically, in the case of Fig. 13A, the chrominance CL of the first adjacent pixel position whose luminance difference is equal to or less than the threshold th is set as the chrominance CC of the pixel to be interpolated, and in the case of Fig. 13B, the chrominance CR of the second adjacent pixel position whose luminance difference is equal to or less than the threshold th is set as the chrominance CC of the pixel to be interpolated.

[0123] 13C, 13D, and 13E, the color differences CL and CR are not used, and the color difference CC at the pixel position to be interpolated is set to 0 (achromatic color). This is because when the luminance difference is large, the color difference at the adjacent pixel position is not used for interpolation. When the luminance difference is large, there is an edge between the pixel position to be interpolated and the first adjacent pixel position and / or the second adjacent pixel position, and it is considered that they constitute a different subject.

[0124] 13F and 13G show cases where the luminance difference between luminance YC and luminance YL and the luminance difference between luminance YC and luminance YR are both equal to or less than the threshold value th. In this case, it is confirmed whether the magnitude relationship between luminance YL, luminance YC, and luminance YR is a monotonically increasing (decreasing) relationship. If it is a monotonically increasing (decreasing) relationship as shown in FIG. 13F, the method of generating the chrominance pixel to be interpolated is determined to be the same as that of FIG. 9F in the second embodiment, and the weighted average of the chrominance CL and chrominance CR is set as the chrominance CC at the pixel position to be interpolated using the above-mentioned formula 4.

[0125] This monotonic increase (decrease) corresponds to "when the luminance pixel value that is the maximum and minimum among the luminance pixel value of the pixel position to be interpolated and the luminance pixel value of the adjacent pixel position is the luminance pixel value of one of the adjacent pixel positions" in the claims.

[0126] On the other hand, when the magnitude relationship between the luminance YL, luminance YC, and luminance YR is not monotonically increasing (decreasing) as shown in FIG. 13G, the method of generating chrominance pixels to be interpolated is determined to be the method in the first embodiment.

[0127] In addition, in FIG. 13D, the luminance YL, luminance YC, and luminance YR can be said to have a monotonically increasing (decreasing) relationship. However, since the luminance difference is large, the chrominance CL and chrominance CR are not used, and the chrominance CC at the pixel position to be interpolated is set to 0 (achromatic color).

[0128] This is because in the case of a monotonically increasing (decreasing) luminance at the pixel position to be interpolated is sandwiched between the luminance at adjacent pixel positions, and the method of the second embodiment calculates the internal division point, thereby improving accuracy, whereas in the case of a monotonically increasing (decreasing) luminance, when the relationship between the luminance CC, luminance CL, and luminance CR is concave (or convex), the method of the first embodiment improves accuracy.

[0129] In addition, when the luminance difference between luminance YC and luminance YL, and the luminance difference between luminance YC and luminance YR are below a threshold value, the method of generating the chrominance pixels to be interpolated may be the method in the second embodiment, regardless of whether they are monotonically increasing (decreasing).

[0130] The processing in the third embodiment is carried out as described above.

[0131] According to the present technology, it is possible to interpolate the thinned chrominance pixels without degrading the image quality due to coloring, etc. As a result, it is possible to obtain image data in which the luminance and chrominance are consistent for all pixels even after the thinning process has been performed.

[0132] Conventionally, in order to prevent degradation of image quality due to interpolation, it was necessary to perform white clip processing before the interpolation processing by generating chrominance pixels to be interpolated. However, in the present technology, it is not necessary to perform white clip processing before the interpolation processing by generating chrominance pixels to be interpolated, and it is possible to perform thinning processing without performing white clip processing in the imaging device 10, and to perform white clip processing after the interpolation processing in the developing device 20.

[0133] In addition, the size of the image data can be reduced by thinning out the image data, shortening the transfer time from the imaging device 10 to the developing device 20, and an image with high image quality can be obtained by performing interpolation before development.

[0134] <4. Modifications> Although the embodiments of the present technology have been specifically described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical ideas of the present technology are possible.

[0135] 14 is a block diagram showing the configuration of an information processing device 400 according to the modified example. The information processing device 400 includes a memory 401, a neighboring pixel position selection unit 402, and an interpolation target chrominance pixel generation unit 403. The neighboring pixel position selection unit 402 and the interpolation target chrominance pixel generation unit 403 are similar to the neighboring pixel position selection unit 101 and the interpolation target chrominance pixel generation unit 102 in the first embodiment.

[0136] A memory 401 stores image data (for example, YC 4:2:2) that has been subjected to thinning processing, and a luminance signal and a chrominance signal are supplied from the memory 401 to a neighboring pixel position selection unit 402 and an interpolation target chrominance pixel generation unit 403 .

[0137] After generating the chrominance pixels to be interpolated, the chrominance pixel generation unit 403 supplies the chrominance pixels to be interpolated to the memory 401. As a result, the YC 4:4:4 image data in which chrominance has been interpolated is stored in the memory 401. This modification example can be applied to any of the first to third embodiments.

[0138] Fig. 15 is a diagram showing a modified example of adjacent pixel position selection by the adjacent pixel position selection unit 101. As shown in Fig. 15A, from among a plurality of pixel positions (e.g., eight pixel positions within a 3 × 3 frame) existing around the pixel position to be interpolated, two adjacent pixel positions may be selected based on the correlation of luminance and color as shown in Fig. 15B. Then, as shown in Fig. 15C, an interpolation target chrominance pixel is generated at the pixel position to be interpolated using the adjacent pixel positions, and an interpolation process is performed.

[0139] Fig. 16 is a diagram showing the configuration of pixel arrangement constituting image data. An example of the YC4:4:4 configuration in image data is shown in Fig. 16A. When the chrominance pixels are thinned out horizontally from this YC4:4:4, it becomes YC4:2:2, which is an example of a pixel arrangement as shown in Fig. 16B. When the chrominance signals are further thinned out vertically, it becomes YC4:2:0, which is an example of a pixel arrangement as shown in Fig. 16C1. This type of pixel arrangement is often adopted in digital image data / video data.

[0140] There is also an example where there is a phase difference between the luminance signal and the color difference signal. In the example shown in Fig. 16B2, two pixel positions surrounded by a dashed line are treated as one combination, and a luminance corresponding to the center position of the two pixel positions shown in P in Fig. 16B2 is generated from the luminance at the two pixel positions, and the present technology is applied to generate a chrominance pixel to be interpolated at the pixel position to be interpolated. In the example shown in Fig. 16C2, four pixel positions surrounded by a dashed line are treated as one combination, and a luminance corresponding to the center position of the four pixel positions shown in P in Fig. 16C2 is generated from the luminance at the four pixel positions, and the present technology is applied to generate a chrominance pixel to be interpolated at the pixel position to be interpolated.

[0141] The present technology can also be configured as follows. (1) In image data generated based on three primary color pixels that may include a value greater than a predetermined white clip value, the number of luminance pixels is greater than the number of first color difference pixels and the number of second color difference pixels, an interpolation target chrominance pixel generating unit that generates an interpolation target chrominance pixel corresponding to the first chrominance pixel at an interpolation target pixel position, the luminance pixel being a pixel position where the first chrominance pixel is not present, based on a relationship between the luminance pixel at the interpolation target pixel position, and the luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the interpolation target pixel position; An information processing device comprising: (2) The information processing device according to (1), wherein the interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel by using the first chrominance pixel at the adjacent pixel position. (3) The information processing device according to (2), wherein the chrominance pixel generation unit for interpolation calculates a value of the chrominance pixel to be interpolated from a value of the luminance pixel at the pixel position to be interpolated, a value of the luminance pixel at the adjacent pixel position, and a value of the first chrominance pixel at the adjacent pixel position. (4) The information processing device according to (2) or (3), wherein the interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel based on a relationship between a value of the luminance pixel at the interpolation target pixel position and a value of the luminance pixel at the adjacent pixel position. (5) The information processing device according to (4), wherein the interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel by using a positive correlation between the value of the luminance pixel at the interpolation target pixel position and the magnitude of the value of the first chrominance pixel based on a relationship between the value of the luminance pixel at the interpolation target pixel position and the value of the luminance pixel at the adjacent pixel position. (6) The information processing device according to (5), wherein the interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel based on a relationship between a value of the luminance pixel at the interpolation target pixel position and a value of the luminance pixel at the adjacent pixel position, using a linear relationship as the positive correlation. (7) The information processing device according to (6), wherein the chrominance pixel generation unit for interpolation calculates an average value of a ratio between a value of the luminance pixel at the pixel position for interpolation and a value of the luminance pixel at the adjacent pixel position, and a multiplication value of the chrominance pixel values ​​at a plurality of the adjacent pixel positions, to generate the chrominance pixel for interpolation. (8) The information processing device according to (5), wherein the value of the chrominance pixel to be interpolated is calculated using a value obtained by adding an offset value to the value of the luminance pixel at the position of the pixel to be interpolated and the value of the luminance pixel at the adjacent pixel position. (9) The information processing device according to (8), wherein the offset value is a value determined based on the value of the luminance pixel at the adjacent pixel position. (10) The information processing device according to any one of (4) to (9), wherein the interpolation target chrominance pixel generation unit generates the chrominance pixel to be interpolated based on a luminance difference which is a relationship between a value of a luminance pixel at the interpolation target pixel position and a value of a luminance pixel at the adjacent pixel position, and which is a difference between the value of the luminance pixel at the interpolation target pixel position and the value of the luminance pixel at the adjacent pixel position. (11) The information processing device according to (10), wherein the interpolation target chrominance pixel generation unit calculates the value of the interpolation target chrominance pixel by a weighted average using the luminance difference. (12) The information processing device according to any one of (1) to (11), further comprising a generation method determination unit that determines a generation method of the chrominance pixel to be interpolated based on a relationship between a value of the luminance pixel at the pixel position to be interpolated and a value of the luminance pixel at the adjacent pixel position. (13) The information processing device according to (12), wherein the generation method determination unit determines the generation method based on a magnitude of the luminance difference. (14) The information processing device according to (12) or (13), wherein the generation method determination unit determines the method to be a method of generating the interpolation target chrominance pixels based on the luminance difference when the magnitude of the absolute value of the luminance difference is greater than a predetermined value. (15) The information processing device according to any one of (12) to (14), wherein the generation method determination unit determines a method for generating the chrominance pixel to be interpolated based on a magnitude relationship between a value of the luminance pixel at the pixel position to be interpolated and a value of the luminance pixel at the adjacent pixel position. (16) The information processing device according to any one of (1) to (15), further comprising an adjacent pixel position selection unit that selects one or more adjacent pixel positions from among a plurality of pixel positions adjacent to the pixel position to be interpolated. (17) The information processing device according to any one of (1) to (16), further comprising a white balance processing unit that performs white balance on image data in which the first color difference pixel at the interpolation target pixel position is generated by the interpolation target color difference pixel generation unit, based on a user setting. (18) The information processing device according to any one of (1) to (17), further comprising an imaging unit configured to capture an image corresponding to the three primary color pixels. (19) In image data generated based on three primary color pixels that may include a value greater than a predetermined white clip value, the number of luminance pixels is greater than the number of first color difference pixels and the number of second color difference pixels, generating an interpolation target chrominance pixel corresponding to the first chrominance pixel at the interpolation target pixel position based on a relationship between the luminance pixel at the interpolation target pixel position, which is a pixel position where the first chrominance pixel is not present, and the luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the interpolation target pixel position; Information processing methods. (20) In image data generated based on three primary color pixels that may include a value greater than a predetermined white clip value, the number of luminance pixels is greater than the number of first color difference pixels and the number of second color difference pixels, generating an interpolation target chrominance pixel corresponding to the first chrominance pixel at the interpolation target pixel position based on a relationship between the luminance pixel at the interpolation target pixel position, which is a pixel position where the first chrominance pixel is not present, and the luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the interpolation target pixel position; An information processing program that causes a computer to execute an information processing method. [Explanation of symbols]

[0142] 12 Imaging unit 100, 200, 300, 400... Information processing device 102, 201, 302, 403: Interpolation target chrominance pixel generation unit 301...Generation method determination section

Claims

1. In image data generated based on three primary color pixels that may include a value greater than a predetermined white clip value, the number of luminance pixels is greater than the number of first color difference pixels and the number of second color difference pixels, an interpolation target chrominance pixel generating unit that generates an interpolation target chrominance pixel corresponding to the first chrominance pixel at an interpolation target pixel position, the luminance pixel being a pixel position where the first chrominance pixel is not present, based on a relationship between the luminance pixel at the interpolation target pixel position and the luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the interpolation target pixel position; The chrominance pixel to be interpolated generating unit calculates an average value of a ratio between a value of the luminance pixel at the chrominance pixel to a value of the luminance pixel at the adjacent pixel, and a multiplication value of chrominance pixels at a plurality of the adjacent pixel positions, to generate the chrominance pixel to be interpolated. Information processing device.

2. The interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel by using the first chrominance pixel at the adjacent pixel position. The information processing device according to claim 1 .

3. The chrominance pixel generation unit calculates a value of the chrominance pixel to be interpolated from a value of the luminance pixel at the chrominance pixel position to be interpolated, a value of the luminance pixel at the adjacent pixel position, and a value of the first chrominance pixel at the adjacent pixel position. The information processing device according to claim 2 .

4. The chrominance pixel to be interpolated generating unit generates the chrominance pixel to be interpolated based on a relationship between a value of the luminance pixel at the position of the pixel to be interpolated and a value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 2 .

5. The interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel by using a positive correlation between the value of the luminance pixel at the interpolation target pixel position and the value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 4.

6. The interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixels using a linear relationship as the positive correlation based on a relationship between a value of the luminance pixel at the interpolation target pixel position and a value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 5 .

7. The value of the chrominance pixel to be interpolated is calculated by adding an offset value to the value of the luminance pixel at the position of the pixel to be interpolated and the value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 5 .

8. The offset value is a value that is determined based on the value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 7.

9. The interpolation target chrominance pixel generation unit generates the interpolation target chrominance pixel based on a luminance difference which is a relationship between a value of a luminance pixel at the interpolation target pixel position and a value of a luminance pixel at the adjacent pixel position, the luminance difference being a difference value between the value of the luminance pixel at the interpolation target pixel position and the value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 4.

10. The interpolation target chrominance pixel generation unit calculates the value of the interpolation target chrominance pixel by weighted averaging using the luminance difference. The information processing device according to claim 9.

11. a generation method determination unit that determines a generation method of the chrominance pixel to be interpolated based on a relationship between a value of the luminance pixel at the pixel to be interpolated and a value of the luminance pixel at the adjacent pixel position; The information processing device according to claim 1 .

12. The generation method determination unit determines the generation method based on a magnitude of a luminance difference, which is a difference between a value of the luminance pixel at the interpolation target pixel position and a value of the luminance pixel at the adjacent pixel position. The information processing device according to claim 11.

13. The generation method determination unit determines the method to be a method of generating the chrominance pixel to be interpolated based on the luminance difference when an absolute value of a luminance difference, which is a difference between a value of the luminance pixel at the pixel to be interpolated and a value of the luminance pixel at the adjacent pixel, is greater than a predetermined value. The information processing device according to claim 11.

14. The generation method determination unit determines a method for generating the chrominance pixel to be interpolated based on a magnitude relationship between a value of the luminance pixel at the pixel to be interpolated and a value of the luminance pixel at the adjacent pixel. The information processing device according to claim 11.

15. a neighboring pixel position selection unit that selects one or more neighboring pixel positions from among a plurality of pixel positions neighboring the pixel position to be interpolated; The information processing device according to claim 1 .

16. a white balance processing unit that performs white balance on the image data in which the first color difference pixel at the interpolation target pixel position is generated by the interpolation target color difference pixel generation unit, based on a user setting. The information processing device according to claim 1 .

17. an imaging unit that captures an image corresponding to the three primary color pixels; The information processing device according to claim 1 .

18. In image data generated based on three primary color pixels that may include a value greater than a predetermined white clip value, the number of luminance pixels is greater than the number of first color difference pixels and the number of second color difference pixels, a chrominance pixel to be interpolated corresponding to the first chrominance pixel is generated at the pixel position to be interpolated by calculating an average value of a ratio between a value of the luminance pixel at a pixel position to be interpolated, which is a pixel position where the first chrominance pixel is not present, and a value of the luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the pixel position to be interpolated, and a value of the chrominance pixels at the neighboring pixel positions; Information processing methods.

19. In image data generated based on three primary color pixels that may include a value greater than a predetermined white clip value, the number of luminance pixels is greater than the number of first color difference pixels and the number of second color difference pixels, a chrominance pixel to be interpolated corresponding to the first chrominance pixel is generated at the pixel position to be interpolated by calculating an average value of a ratio between a value of the luminance pixel at a pixel position to be interpolated, which is a pixel position where the first chrominance pixel is not present, and a value of the luminance pixel at at least one neighboring pixel position among a plurality of pixel positions neighboring the pixel position to be interpolated, and a value of the chrominance pixels at the neighboring pixel positions; An information processing program that causes a computer to execute an information processing method.

Images