IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

To provide an image processing apparatus which performs optimized image processing in accordance with color components, an image processing method, and an image processing program.SOLUTION: An image processing apparatus 3 determines a blending ratio for blending first interpolation data which is generated by first interpolation processing by means of a Gr / Gb component interpolation processing unit 240 which interpolates first and second color components as the same color component in one map and second interpolation data which is generated by second interpolation processing by means of a Gr component interpolation processing unit 220 and a Gb component interpolation processing unit 230 which interpolate the first and the second color components as different color components in multiple maps, and generates reduced data of image data including the first and the second color components on the basis of synthetic data blended in accordance with the blending ratio.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The disclosure of this specification relates to an image processing device, an image processing method, and an image processing program. [Background technology]

[0002] For example, Patent Document 1 describes a video compression device for a single frame image of a dual green system having a Bayer array sampling structure. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-63289 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional image processing devices including those in Patent Document 1 have the following features: Corresponding to the G component There is room for improvement in terms of providing optimized image processing.

[0005] In light of the above-mentioned circumstances, an object of one aspect of the present invention is to Corresponding to the G component An object of the present invention is to provide an image processing device, an image processing method, and an image processing program capable of performing optimized image processing. [Means for solving the problem]

[0006] The image processing device according to a first aspect of the present invention is a device for processing an image having a first interpolation data generated by a first interpolation process that interpolates a G component in Bayer data using one map, and a second interpolation data generated by a second interpolation process that uses separate maps to interpolate between a G component in a row in which an R component is arranged and a G component in a row in which a B component is arranged, the device being configured to set a blending ratio for mixing the first interpolation data and a second interpolation data, the blending ratio being set to a reduction ratio for reducing the Bayer data. Value of The control means is provided to set the and the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data.It is characterized by: In addition, in the image processing device according to the second aspect of the present invention, when pixels are interpolated for each color component based on Bayer data, the G component is separated into a Gr component as the G component of the row in which the R component is arranged and a Gb component as the G component of the row in which the B component is arranged, and pixels are generated by separately interpolating the Gr component and the Gb component. Second The interpolation data is generated by interpolating pixels for the G component without separating the G component into the Gr component and the Gb component. No. 1 The blend ratio for mixing the interpolated data and the Bayer data is set to the reduction ratio Value of A control means for setting the the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. It is characterized by: Further, an image processing method according to a first aspect of the present invention is an image processing method executed by an image processing device, in which a blending ratio for mixing first interpolated data generated by a first interpolation process that interpolates a G component in Bayer data using one map, and second interpolated data generated by a second interpolation process that interpolates a G component in a row in which an R component is arranged and a G component in a row in which a B component is arranged using separate maps, is set to a reduction ratio when reducing the Bayer data. Value of Control processing based on the control process sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. It is characterized by: Further, an image processing method according to a second aspect of the present invention is an image processing method executed by an image processing device, and when interpolating pixels for each color component based on Bayer data, the G component is separated into a Gr component as the G component of a row in which an R component is arranged and a Gb component as the G component of a row in which a B component is arranged, and pixels are generated by interpolating the Gr component and the Gb component separately. Second The interpolation data is generated by interpolating pixels for the G component without separating the G component into the Gr component and the Gb component. No. 1 The blend ratio for mixing the interpolated data and the Bayer data is set to the reduction ratio Value of Control processing based on the control process sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. It is characterized by: Further, an image processing program according to a first aspect of the present invention causes a computer to set a blending ratio for mixing first interpolation data generated by a first interpolation process that interpolates a G component in Bayer data using one map, and second interpolation data generated by a second interpolation process that interpolates a G component in a row in which an R component is arranged and a G component in a row in which a B component is arranged using separate maps, the blending ratio being a reduction ratio when reducing the Bayer data. Value of It serves as a control means to set and the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is less than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. It is characterized by: Further, an image processing program according to a second aspect of the present invention causes a computer to, when interpolating pixels for each color component based on Bayer data, separate a G component into a Gr component as the G component of a row in which an R component is arranged and a Gb component as the G component of a row in which a B component is arranged, and to generate pixels by separately interpolating the Gr component and the Gb component. Second The interpolation data is generated by interpolating pixels for the G component without separating the G component into the Gr component and the Gb component. No. 1 The blend ratio for mixing the interpolated data and the Bayer data is set to the reduction ratio Value of It serves as a control means to set and the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is less than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. It is characterized by: Effect of the Invention

[0007] According to the above aspect, Corresponding to the G component It is possible to provide an image processing device, an image processing method, and an image processing program capable of performing optimized image processing. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an external view showing an example of a digital camera 1X. [Diagram 2] FIG. 2 is a diagram showing an example of a functional configuration block of a digital camera 1X. [Diagram 3] 2 is a functional block diagram showing an example of the internal configuration of a RAW image reducing unit 3. FIG. [Figure 4] FIG. 2 is a diagram illustrating an example of division processing by a data division unit 100. [Diagram 5] 13 is a diagram showing an example of the interpolation process of the R component divided image D2R by the R component interpolation processing unit 210. FIG. [Figure 6] 13 is a diagram showing an example of the interpolation process of the Gr component divided image D2Gr by the Gr component interpolation processing unit 220. FIG. [Figure 7] 13 is a diagram showing an example of the interpolation process of the Gb component divided image D2Gb by the Gb component interpolation processor 230. FIG. [Figure 8] 13 is a diagram showing an example of the interpolation process of the Gr component divided image D2Gr and the Gb component divided image D2Gb by the Gr·Gb component interpolation processing unit 240. FIG. [Figure 9] 13 is a diagram showing an example of the interpolation process of the B component divided image D2B by the B component interpolation processing unit 250. FIG. [Figure 10] 13 is a diagram showing an example of a synthesis process performed by a synthesis processor 300. FIG. [Figure 11] FIG. 13 is a diagram illustrating an example of a reduction process when the reduction ratio is 1 / 2. [Figure 12] FIG. 13 is a diagram illustrating an example of resampling processing when the reduction ratio is 1 / 2. [Figure 13] FIG. 13 is a diagram showing an example of a combining process when the reduction ratio is 1 / 2. [Figure 14] FIG. 11 is a first diagram showing an example of the effect of image processing according to the present embodiment. [Figure 15] FIG. 11 is a second diagram showing an example of the effect of the image processing according to the present embodiment. [Figure 16] 5 is a flowchart illustrating an example of image processing according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The image processing device of this embodiment will be described in detail with reference to Figures 1 to 16. Note that the image processing method and image processing program of this embodiment are realized by causing a computer (CPU: Central Processing Unit) of the image processing device to execute various processing steps.

[0010] The image processing device may be mounted on a fixed photographing device such as a security camera or a surveillance camera, or may be mounted on a portable photographing device such as a digital camera or various terminals. The image processing device may be specialized for performing image processing on images transmitted by wire or wirelessly from a photographing device or other device, or images stored in various storage media. In other words, the configuration of the image processing device has a degree of freedom, and various design changes are possible.

[0011] FIG. 1 is an external view showing an example of a digital camera (photographing device) 1X. In the digital camera 1X, light from a subject is guided into a camera body 11X via a lens 10X, reflected by a mirror 12X, and a real image of the subject is formed on a focusing glass 13X. This real image of the subject is guided to a viewfinder 15X by a pentaprism 14X so that it can be observed. A display device 16X is disposed between the focusing glass 13X and the pentaprism 14X, and displays various information superimposed on the subject image reflected on the focusing glass 13X. The broken lines in FIG. 1 indicate the optical path of the light from the subject that is guided to the viewfinder 15X. In the digital camera 1X, the mirror 12X is raised, the shutter 17X is opened, and the subject light is guided to an image sensor 18X to perform photography, so that the subject image does not reach the display device 16X in the mirror-up state.

[0012] The digital camera 1X may incorporate an image processing device according to an embodiment of the present invention that performs image processing on an image captured by the image sensor 18X, or may have a communication unit (such as a connection terminal or an antenna) that transmits the image captured by the image sensor 18X to an external image processing device according to an embodiment of the present invention by wire or wirelessly. That is, the present invention is not limited to the digital camera 1X having a photographing function, but can also be applied to any device that can obtain an image captured by an external device and apply image processing to it. In other words, the present invention can also be applied to a system composed of multiple devices.

[0013] FIG. 2 is a diagram showing an example of a functional configuration block of a digital camera (photographing device) 1X. The digital camera 1X has a RAW image correction processing section 2 and a RAW image reduction section 3. The RAW image correction processing section 2 performs correction processing (e.g. pre-processing such as scratch removal) on an image D1 captured by an image sensor 18X to generate a RAW image D2, and outputs the RAW image D2 to the RAW image reduction section 3. The RAW image D2 is, for example, Bayer data including an R component, a G component (Gr component, Gb component), and a B component. Bayer data means image data in a Bayer array. The RAW image reduction section 3 performs division processing, interpolation processing, blending ratio B rate Taking these factors into consideration, the image processing device performs synthesis (mixing), reduction, resampling, and joining processes to output a reduced RAW image D3. The reduced RAW image D3 is also Bayer data including, for example, an R component, a G component (Gr component, Gb component), and a B component. The RAW image reduction unit 3 constitutes the image processing device of this embodiment, and the details of the RAW image reduction unit (image processing device) 3 will be described later.

[0014] The digital camera 1X has a main image generating section 4, a CPU (Central Processing Unit) 5, a key IF section 6, a non-volatile memory I / F section 7, and an external high-speed memory I / F section 8. The RAW image reducing section 3, the main image generating section 4, the CPU 5, the key IF section 6, the non-volatile memory I / F section 7, and the external high-speed memory I / F section 8 are connected to each other by a bus 9.

[0015] The main image generating unit 4 generates a main image based on the reduced RAW image D3 by the RAW image reducing unit 3. The CPU 5 controls the internal components of the digital camera 1X including the RAW image reducing unit 3, the main image generating unit 4, the Key IF unit 6, the non-volatile memory I / F unit 7, and the external high-speed memory I / F unit 8. The Key IF unit 6 executes operation control (operation restriction) according to the Key 6A provided from an external device. The non-volatile memory I / F unit 7 controls writing of various data (e.g., the reduced RAW image D3 by the RAW image reducing unit 3 and the main image by the main image generating unit 4) to the non-volatile memory (SD card) 7A inserted as an external storage medium. The external high-speed memory I / F unit 8 controls writing of various data (e.g., the reduced RAW image D3 by the RAW image reducing unit 3 and the main image by the main image generating unit 4) to the external high-speed memory (DDR memory) 8A.

[0016] 3 is a functional block diagram showing an example of the internal configuration of the RAW image reduction unit (image processing device) 3. As described above, the RAW image reduction unit 3 receives the RAW image D2 from the RAW image correction processing unit 2, and performs division processing, interpolation processing, and blending processing on the RAW image D2. rate A reduced RAW image D3 that has been subjected to synthesis processing (mixing processing), reduction processing, resampling processing, and combination processing taking into consideration the above is output from the RAW image reduction unit 3. Both the RAW image D2 and the reduced RAW image D3 are Bayer data that includes, for example, an R component, a G component (Gr component, Gb component), and a B component.

[0017] The RAW image reducing unit 3 includes a data dividing unit 100, an interpolation processing unit 200, a synthesis processing unit (mixing processing unit) 300, a reduction processing unit 400, a resampling processing unit 500, and a data combining unit 600.

[0018] The data dividing unit 100 divides the RAW image (Bayer data including R, G (Gr, Gb) and B components) D2 from the RAW image correction processing unit 2 into pixel maps for each color component.

[0019] Fig. 4 is a diagram showing an example of the division process by the data division unit 100. As shown in Fig. 4, the RAW image D2 is composed of a total of 16 pixels, including four pixels R00, R02, R20, and R22 of the R component, four pixels Gr01, Gr03, Gr21, and Gr23 of the Gr component, four pixels Gb10, Gb12, Gb30, and Gb32 of the Gb component, and four pixels B11, B13, B31, and B33 of the B component, arranged in a Bayer array.

[0020] The data division unit 100 extracts four pixels R00, R02, R20, and R22 of the R component from the RAW image D2 and divides them into an R component divided image D2R, extracts four pixels Gr01, Gr03, Gr21, and Gr23 of the Gr component and divides them into a Gr component divided image D2Gr, extracts four pixels Gb10, Gb12, Gb30, and Gb32 of the Gb component and divides them into a Gb component divided image D2Gb, and extracts four pixels B11, B13, B31, and B33 of the B component and divides them into a B component divided image D2B.

[0021] The interpolation processing section 200 has an R component interpolation processing section 210 , a Gr component interpolation processing section 220 , a Gb component interpolation processing section 230 , a Gr·Gb component interpolation processing section 240 , and a B component interpolation processing section 250 .

[0022] The R component interpolation processing unit 210 performs an interpolation process on the R component divided image D2R from the data dividing unit 100 to generate a post-interpolation processed image H2R.

[0023] Fig. 5 is a diagram showing an example of the interpolation process of the R component divided image D2R by the R component interpolation processor 210. In Fig. 5, 16 pixels R00, R01, R02, R03, R10, R11, R12, R13, R20, R21, R22, R23, R30, R31, R32, and R33 of the R component after interpolation are calculated by the following formula based on four pixels R00, R02, R20, and R22 of the R component of the R component divided image D2R before interpolation. [After interpolation] [Before interpolation] R00 = R00 R01 = (R00+R02) / 2 R02 = R02 R03 = (R02+R04) / 2 R10 = (R00+R20) / 2 R11 = (R00+R02+R20+R22) / 4 R12 = (R02+R22) / 2 R13 = (R02+R04+R22+R24) / 4 R20 = R20 R21 = (R20+R22) / 2 R22 = R22 R23 = (R22+R24) / 2 R30 = (R20+R40) / 2 R31 = (R20+R22+R40+R42) / 4 R32 = (R22+R42) / 2 R33 = (R22+R24+R42+R44) / 4

[0024] The Gr component interpolation processing unit 220 performs an interpolation process on the Gr component divided image D2Gr from the data dividing unit 100, thereby generating a post-interpolation processed image H2Gr.

[0025] Fig. 6 is a diagram showing an example of the interpolation process of the Gr component divided image D2Gr by the Gr component interpolation processor 220. In Fig. 6, 16 pixels Gr00, Gr01, Gr02, Gr03, Gr10, Gr11, Gr12, Gr13, Gr20, Gr21, Gr22, Gr23, Gr30, Gr31, Gr32, and Gr33 of the Gr component after interpolation are calculated by the following formula based on four pixels Gr01, Gr03, Gr21, and Gr23 of the Gr component of the Gr component divided image D2Gr before interpolation. [After interpolation] [Before interpolation] Gr00 = (Gr01+Gr01) / 2 Gr01 = Gr01 Gr02 = (Gr01+Gr03) / 2 Gr03 = Gr03 Gr10 = (Gr01+Gr01+Gr21+Gr21) / 4 Gr11 = (Gr01+Gr21) / 2 Gr12 = (Gr01+Gr03+Gr21+Gr23) / 4 Gr13 = (Gr03+Gr23) / 2 Gr20 = (Gr21+Gr21) / 2 Gr21 = Gr21 Gr22 = (Gr21+Gr23) / 2 Gr23 = Gr23 Gr30 = (Gr21+Gr21+Gr41+Gr41) / 4 Gr31 = (Gr21+Gr41) / 2 Gr32 = (Gr21+Gr23+Gr41+Gr43) / 4 Gr33 = (Gr23+Gr43) / 2

[0026] The Gb component interpolation processing unit 230 performs interpolation processing on the Gb component divided image D2Gb from the data dividing unit 100 to generate a post-interpolation processing image H2Gb.

[0027] Fig. 7 is a diagram showing an example of the interpolation process of the Gb component divided image D2Gb by the Gb component interpolation processor 230. In Fig. 7, 16 pixels Gb00, Gb01, Gb02, Gb03, Gb10, Gb11, Gb12, Gb13, Gb20, Gb21, Gb22, Gb23, Gb30, Gb31, Gb32, and Gb33 of the Gb component after interpolation are calculated by the following formula based on the four pixels Gb10, Gb12, Gb30, and Gb32 of the Gb component of the Gb component divided image D2Gb before interpolation. [After interpolation] [Before interpolation] Gb00 = (Gb10+Gb10) / 2 Gb01 = (Gb10+Gb12+Gb10+Gb12) / 4 Gb02 = (Gb12+Gb12) / 2 Gb03 = (Gb12+Gb14+Gb12+Gb14) / 4 Gb10 = Gb10 Gb11 = (Gb10+Gb12) / 2 Gb12 = Gb12 Gb13 = (Gb12+Gb14) / 2 Gb20 = (Gb10+Gb30) / 2 Gb21 = (Gb10+Gb12+Gb30+Gb32) / 4 Gb22 = (Gb12+Gb32) / 2 Gb23 = (Gb12+Gb14+Gb32+Gb34) / 4 Gb30 = Gb30 Gb31 = (Gb30+Gb32) / 2 Gb32 = Gb32 Gb33 = (Gb32+Gb34) / 2

[0028] The Gr / Gb component interpolation processing unit 240 performs an interpolation process on the Gr component divided image D2Gr and the Gb component divided image D2Gb from the data dividing unit 100 to generate a post-interpolation image H2GrGb.

[0029] Fig. 8 is a diagram showing an example of the interpolation process of the Gr component divided image D2Gr and the Gb component divided image D2Gb by the Gr·Gb component interpolation processor 240. In Fig. 8, 16 pixels G00, G01, G02, G03, G10, G11, G12, G13, G20, G21, G22, G23, G30, G31, G32, and G33 of the G component (GrGb component) after interpolation are calculated by the following formula based on four pixels Gr01, Gr03, Gr21, and Gr23 of the Gr component of the Gr component divided image D2Gr before interpolation and four pixels Gb10, Gb12, Gb30, and Gb32 of the Gb component of the Gb component divided image D2Gb before interpolation. [After interpolation] [Before interpolation] G00 = (Gr01+Gr01) / 2 G01 = Gr01 G02 = (Gr01+Gr03) / 2 G03 = Gr03 G10 = Gb10 G11 = (Gb10+Gb12) / 2 G12 = Gb12 G13 = (Gb12+Gb14) / 2 G20 = (Gr21+Gr21) / 2 G21 = Gr21 G22 = (Gr21+Gr23) / 2 G23 = Gr23 G30 = Gb30 G31 = (Gb30+Gb32) / 2 G32 = Gb32 G33 = (Gb32+Gb34) / 2

[0030] In the above example, the Gr·Gb component interpolation process is performed on the pixels to the left and right of the interpolated pixel, but it may be performed on the pixels above, below, left, or right. For example, in the above example, G11=(Gb10+Gb12) / 2, but it may be G11=(Gr01+Gr21+Gb10+Gb12) / 4.

[0031] The B component interpolation processing unit 250 performs an interpolation process on the B component divided image D2B from the data dividing unit 100 to generate a post-interpolation processed image H2B.

[0032] Fig. 9 is a diagram showing an example of the interpolation process of the B component divided image D2B by the B component interpolation processor 250. In Fig. 9, 16 pixels B00, B01, B02, B03, B10, B11, B12, B13, B20, B21, B22, B23, B30, B31, B32, and B33 of the B component after interpolation are calculated by the following formula based on four pixels B11, B13, B31, and B33 of the B component of the B component divided image D2B before interpolation. [After interpolation] [Before interpolation] B00 = (B11+B11+B11+B11) / 4 B01 = (B11+B11) / 2 B02 = (B11+B13+B11+B13) / 4 B03 = (B13+B13) / 2 B10 = (B11+B11) / 2 B11 = B11 B12 = (B11+B13) / 2 B13 = B13 B20 = (B11+B11+B31+B31) / 4 B21 = (B11+B31) / 2 B22 = (B11+B13+B31+B33) / 4 B23 = (B13+B33) / 2 B30 = (B31+B31) / 2 B31 = B31 B32 = (B31+B33) / 2 B33 = B33

[0033] In addition, in the interpolation process, there may be cases where pixel components are not present around the interpolated pixel at the end of the input image. In this case, pixel components of other peripheral parts of the interpolated pixel, or pixel components adjacent to the part where no pixel components are present, may be used. For example, when there is no pixel to the left of the interpolated pixel, the pixel to the right of the interpolated pixel may be used; when there is no pixel to the right of the interpolated pixel, the pixel to the left of the interpolated pixel may be used; when there is no pixel above the interpolated pixel, the pixel below the interpolated pixel may be used; when there is no pixel below the interpolated pixel, the pixel above the interpolated pixel may be used; when there is no pixel to the top left of the interpolated pixel, the pixel to the top right of the interpolated pixel may be used; when there is no pixel to the bottom left of the interpolated pixel, the pixel to the bottom right of the interpolated pixel may be used; when there is no pixel to the top left of the interpolated pixel, the pixel to the bottom left of the interpolated pixel may be used; when there is no pixel to the top right of the interpolated pixel, the pixel to the bottom right of the interpolated pixel may be used. Furthermore, when there are no pixels to the top left, top right, or bottom left of the interpolated pixel, the pixel to the bottom right of the interpolated pixel may be used.

[0034] In this manner, in the image processing of this embodiment, when one of the Gr component and the Gb component of the G component and the other are defined as the "first and second color components," a "first interpolation process (see FIG. 8)" is executed in which the first and second color components are interpolated as the same color component using one (common) map, and a "second interpolation process (see FIG. 6 and FIG. 7)" is executed in which the first and second color components are interpolated as different color components using multiple (separate) maps.

[0035] Returning to FIG. 3, the synthesis processing section (mixing processing section) 300 has a Gr component synthesis processing section (mixing processing section) 310 and a Gb component synthesis processing section (mixing processing section) 320.

[0036] The Gr component synthesis processing unit 310 and the Gb component synthesis processing unit 320 are provided with a blending ratio B determined (calculated, calculated) based on the relationship between the reduction ratio A and the threshold value S. rate Specifically, the RAW image reducing unit 3 has a reduction ratio threshold comparison unit 700, and in response to the comparison result of the reduction ratio threshold comparison unit 700, the blend ratio B rate The value of fluctuates.

[0037] In this embodiment, as an example, when the reduction ratio A is 1 to 3 / 4, the blending ratio B rate is determined as 100%, and the blending ratio B when the reduction ratio A is 1 / 4 to 0 rate is determined to be 0%, and the blending ratio B when the reduction ratio A is 3 / 4 to 1 / 4 rate The case where the blending ratio B is determined based on the following formula (1) will be described. For example, when the reduction ratio A is 1 / 2, rate In this example, the blending ratio B rate The threshold values ​​S are 3 / 4 and 1 / 4 of the reduction ratio A, which are the boundaries between different methods of determining (calculating, calculating) the reduction ratio A. For example, 3 / 4 can be set as the first threshold value S1, and 1 / 4 can be set as the second threshold value S2. (1) Blend ratio B rate =(Reduction rate A-1 / 4) / (3 / 4-1 / 4)×100[%]

[0038] The Gr component synthesis processing unit 310 uses the determined blend ratio B rate In response to this, the post-interpolation process image H2Gr (second interpolation data) from the Gr component interpolation processor 220 and the post-interpolation process image H2GrGb (first interpolation data) from the Gr·Gb component interpolation processor 240 are synthesized (mixed) at a predetermined ratio. The predetermined ratio related to synthesis (mixing) here is used as a concept including 0% and 100%.

[0039] The Gb component synthesis processing unit 320 uses the determined blend ratio B rateIn response to this, the post-interpolation process image H2Gb (second interpolation data) from the Gb component interpolation processor 230 and the post-interpolation process image H2GrGb (first interpolation data) from the Gr·Gb component interpolation processor 240 are synthesized (mixed) at a predetermined ratio. The predetermined ratio for synthesis (mixing) here is used as a concept including 0% and 100%.

[0040] Blend ratio B rate The synthesis (mixing) process when is 100% is a process in which, when one of the Gr component and the Gb component of the G component and the other one are the "first and second color components", the first interpolation data (interpolated image H2GrGb) generated by the "first interpolation process (see Figure 8)" which interpolates the first and second color components as the same color component in one (common) map is output as is.

[0041] Blend ratio B rate The synthesis (mixing) process when is 0% is a process of outputting, as it is, second interpolation data (interpolated image H2Gr or interpolated image H2Gr) generated by a "second interpolation process (see Figs. 6 and 7)" in which the first and second color components are interpolated in multiple (separate) maps as different color components when one of the Gr component and the Gb component of the G component and the other one are the "first and second color components."

[0042] Blend ratio B rate When the blend ratio B is greater than 0% and less than 100%, the blending (mixing) process is a process in which the first interpolation data (interpolated image H2GrGb) generated by the "first interpolation process" and the second interpolation data (interpolated image H2Gr or interpolated image H2Gr) generated by the "second interpolation process" are blended (mixed) at a predetermined ratio that is not zero and output. rate is an index indicating which of the first and second interpolation data should have a larger (smaller) utilization ratio in the synthesis (mixing).

[0043] Blend ratio B rateis determined according to the reduction ratio A, and when the reduction ratio A is greater than a threshold value S (e.g., 3 / 4, 1 / 4), the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the reduction ratio A is smaller than the threshold value S (e.g., 3 / 4, 1 / 4), the utilization ratio of the first interpolation data is lower than the utilization ratio of the second interpolation data. Alternatively, the greater the reduction ratio A, the higher the utilization ratio of the first interpolation data is than the utilization ratio of the second interpolation data, and the smaller the reduction ratio A, the lower the utilization ratio of the first interpolation data is than the utilization ratio of the second interpolation data.

[0044] Here, the reduction ratio A may be determined by manual setting by the user, or may be automatically determined based on a predetermined shooting situation (e.g., subject, brightness, etc.). Moreover, the threshold value S may be set as a fixed value when the device is manufactured, may be determined by manual setting by the user, or may be automatically determined based on a predetermined shooting situation (e.g., subject, brightness, etc.). For example, in a mode in which at least one of the reduction ratio A and the threshold value S is automatically determined, the blending ratio B is automatically set without receiving an instruction from the user during shooting with the digital camera (shooting device) 1X. rate Furthermore, image processing control can be executed according to the degree of synthesis (degree of mixing) of the first interpolation data and the second interpolation data.

[0045] For example, in an embodiment of a security or surveillance camera, when no abnormality is detected (normally), memory usage can be saved by executing the second interpolation process (interpolation process when the image size after reduction is smaller than when the first interpolation process is executed), and when an abnormality is detected, the execution can be automatically switched to the first interpolation process (interpolation process when the image size after reduction is larger than when the second interpolation process is executed). Alternatively, when no abnormality is detected (normally), the proportion of the second interpolation process can be increased (the blending rate can be decreased), while when an abnormality is detected, the proportion of the first interpolation process can be increased (the blending rate can be increased).

[0046] 10A and 10B are diagrams showing an example of the synthesis process by the synthesis processing unit 300. Fig. 10A is a diagram showing an example of the synthesis process by the Gr component synthesis processing unit 310, and Fig. 10B is a diagram showing an example of the synthesis process by the Gb component synthesis processing unit 320.

[0047] In FIG. 10A, the 16 pixels Gr00', Gr01', Gr02', Gr03', Gr10', Gr11', Gr12', Gr13', Gr20', Gr21', Gr22', Gr23', Gr30', Gr31', Gr32', and Gr33' of the Gr component after synthesis are the same as the 16 pixels Gr00, Gr01, Gr02, Gr03 of the interpolated image H2Gr before synthesis. , Gr10, Gr11, Gr12, Gr13, Gr20, Gr21, Gr22, Gr23, Gr30, Gr31, Gr32, Gr33, and G00, G01, G02, G03, G10, G11, G12, G13, G20, G21, G22, G23, G30, G31, G32, G33 of the post-interpolation image H2GrGb before synthesis, using the following formula. In the following formula, D is the blending ratio B rate The image consisting of the 16 pixels Gr00', Gr01', Gr02', Gr03', Gr10', Gr11', Gr12', Gr13', Gr20', Gr21', Gr22', Gr23', Gr30', Gr31', Gr32', Gr33' of the Gr component after synthesis is called the synthesized image H2Gr' after interpolation processing. [After synthesis] [Before synthesis] After synthesis Gr00' = (Gr00*(100-D)+G00*D) / 100 After synthesis Gr01' = (Gr01*(100-D)+G01*D) / 100 After synthesis Gr02' = (Gr02*(100-D)+G02*D) / 100 Gr03' after synthesis = (Gr03*(100-D)+G03*D) / 100 After synthesis Gr10' = (Gr10*(100-D)+G10*D) / 100 After synthesis Gr11' = (Gr11*(100-D)+G11*D) / 100 After synthesis Gr12' = (Gr12*(100-D)+G12*D) / 100 After synthesis Gr13' = (Gr13*(100-D)+G13*D) / 100 After synthesis Gr20' = (Gr20*(100-D)+G20*D) / 100 After synthesis Gr21' = (Gr21*(100-D)+G21*D) / 100 After synthesis Gr22' = (Gr22*(100-D)+G22*D) / 100 After synthesis Gr23' = (Gr23*(100-D)+G23*D) / 100 After synthesis Gr30' = (Gr30*(100-D)+G30*D) / 100 After synthesis Gr31' = (Gr31*(100-D)+G31*D) / 100 After synthesis Gr32' = (Gr32*(100-D)+G32*D) / 100 After synthesis Gr33' = (Gr33*(100-D)+G33*D) / 100

[0048] In FIG. 10B, the 16 pixels Gb00', Gb01', Gb02', Gb03', Gb10', Gb11', Gb12', Gb13', Gb20', Gb21', Gb22', Gb23', Gb30', Gb31', Gb32', and Gb33' of the Gb component after synthesis correspond to the 16 pixels Gb00, Gb01, Gb02, Gb03 of the interpolated image H2Gb before synthesis. , Gb10, Gb11, Gb12, Gb13, Gb20, Gb21, Gb22, Gb23, Gb30, Gb31, Gb32, Gb33, and G00, G01, G02, G03, G10, G11, G12, G13, G20, G21, G22, G23, G30, G31, G32, G33 of the post-interpolation image H2GrGb before synthesis, using the following formula. In the following formula, D is the blending ratio B rateAn image consisting of the 16 Gb component pixels after synthesis, Gb00', Gb01', Gb02', Gb03', Gb10', Gb11', Gb12', Gb13', Gb20', Gb21', Gb22', Gb23', Gb30', Gb31', Gb32', and Gb33', is called the post-interpolation synthetic image H2Gb'. [After synthesis] [Before synthesis] After synthesis Gb00' = (Gb00*(100-D)+G00*D) / 100 After synthesis Gb01' = (Gb01*(100-D)+G01*D) / 100 After synthesis Gb02' = (Gb02*(100-D)+G02*D) / 100 Gb03' after synthesis = (Gb03*(100-D)+G03*D) / 100 After synthesis Gb10' = (Gb10*(100-D)+G10*D) / 100 After synthesis Gb11' = (Gb11*(100-D)+G11*D) / 100 After synthesis Gb12' = (Gb12*(100-D)+G12*D) / 100 After synthesis Gb13' = (Gb13*(100-D)+G13*D) / 100 After synthesis Gb20' = (Gb20*(100-D)+G20*D) / 100 After synthesis Gb21' = (Gb21*(100-D)+G21*D) / 100 After synthesis Gb22' = (Gb22*(100-D)+G22*D) / 100 After synthesis Gb23' = (Gb23*(100-D)+G23*D) / 100 After synthesis Gb30' = (Gb30*(100-D)+G30*D) / 100 After synthesis Gb31' = (Gb31*(100-D)+G31*D) / 100 After synthesis Gb32' = (Gb32*(100-D)+G32*D) / 100 After synthesis Gb33' = (Gb33*(100-D)+G33*D) / 100

[0049] Returning to FIG. 3, the reduction processing section 400 has an R component reduction processing section 410 , a Gr component reduction processing section 420 , a Gb component reduction processing section 430 , and a B component reduction processing section 440 .

[0050] The R component reduction processing unit 410, the Gr component reduction processing unit 420, the Gb component reduction processing unit 430, and the B component reduction processing unit 440 perform reduction processing in accordance with the reduction ratio A. For example, the R component reduction processing unit 410, the Gr component reduction processing unit 420, the Gb component reduction processing unit 430, and the B component reduction processing unit 440 perform reduction processing using an averaging operation method based on the area ratio of pixels after reduction.

[0051] 11A to 11D are diagrams showing an example of the reduction process when the reduction ratio is 1 / 2.

[0052] FIG. 11A shows the reduction process of the interpolated image H2R by the R component reduction processing unit 410. In FIG. 11A, a map of 16 pixels (4×4) is reduced to a map of 4 pixels (2×2). In FIG. 11A, the four pixels R00, R01, R10, and R11 of the R component after reduction are calculated by the following formula based on the 16 pixels R00, R01, R02, R03, R10, R11, R12, R13, R20, R21, R22, R23, R30, R31, R32, and R33 of the R component of the interpolated image H2R before reduction. An image composed of the four pixels R00, R01, R10, and R11 of the R component after reduction is called a reduced image S2R. [After reduction] [Before reduction] R00 = (R00+R01+R10+R11) / 4 R01 = (R02+R03+R12+R13) / 4 R10 = (R20+R21+R30+R31) / 4 R11 = (R22+R23+R32+R33) / 4

[0053] FIG. 11B shows the reduction process of the interpolated composite image H2Gr' by the Gr component reduction processing unit 420. In FIG. 11B, a map of 16 pixels (4×4) is reduced to a map of 4 pixels (2×2). In FIG. 11B, the four pixels Gr00', Gr01', Gr10', and Gr11' of the Gr component after reduction are calculated by the following formula based on the 16 pixels Gr00', Gr01', Gr02', Gr03', Gr10', Gr11', Gr12', Gr13', Gr20', Gr21', Gr22', Gr23', Gr30', Gr31', Gr32', and Gr33' of the Gr component of the interpolated composite image H2Gr' before reduction. An image composed of the four pixels Gr00', Gr01', Gr10', and Gr11' of the Gr component after reduction is called the reduced image S2Gr'. [After reduction] [Before reduction] Gr00' = (Gr00'+Gr01'+Gr10'+Gr11') / 4 Gr01' = (Gr02'+Gr03'+Gr12'+Gr13') / 4 Gr10' = (Gr20'+Gr21'+Gr30'+Gr31') / 4 Gr11' = (Gr22'+Gr23'+Gr32'+Gr33') / 4

[0054] Fig. 11C shows the reduction process of the post-interpolation synthesis image H2Gb' by the Gb component reduction processing unit 430. In Fig. 11C, a map of 16 pixels (4 x 4) is reduced to a map of 4 pixels (2 x 2). In Fig. 11C, the four pixels Gb00', Gb01', Gb10', and Gb11' of the Gb component after reduction are calculated by the following formula based on the 16 pixels Gb00', Gb01', Gb02', Gb03', Gb10', Gb11', Gb12', Gb13', Gb20', Gb21', Gb22', Gb23', Gb30', Gb31', Gb32', and Gb33' of the Gb component of the post-interpolation synthesis image H2Gb' before reduction. An image formed of the four pixels Gb00', Gb01', Gb10', and Gb11' of the Gb components after reduction is called a post-reduction processing image S2Gb'. [After reduction] [Before reduction] Gb00' = (Gb00'+Gb01'+Gb10'+Gb11') / 4 Gb01' = (Gb02'+Gb03'+Gb12'+Gb13') / 4 Gb10' = (Gb20'+Gb21'+Gb30'+Gb31') / 4 Gb11' = (Gb22'+Gb23'+Gb32'+Gb33') / 4

[0055] FIG. 11D shows the reduction process of the interpolated image H2B by the B component reduction processing unit 440. In FIG. 11D, a map of 16 pixels (4×4) is reduced to a map of 4 pixels (2×2). In FIG. 11D, the four pixels B00, B01, B10, and B11 of the B component after reduction are calculated by the following formula based on the 16 pixels B00, B01, B02, B03, B10, B11, B12, B13, B20, B21, B22, B23, B30, B31, B32, and B33 of the B component of the interpolated image H2B before reduction. An image composed of the four pixels B00, B01, B10, and B11 of the B component after reduction is called a reduced image S2B. [After reduction] [Before reduction] B00 = (B00+B01+B10+B11) / 4 B01 = (B02+B03+B12+B13) / 4 B10 = (B20+B21+B30+B31) / 4 B11 = (B22+B23+B32+B33) / 4

[0056] Returning to FIG. 3, the resampling processor 500 has an R component resampling processor 510 , a Gr component resampling processor 520 , a Gb component resampling processor 530 , and a B component resampling processor 540 .

[0057] The R component resampling processor 510, the Gr component resampling processor 520, the Gb component resampling processor 530, and the B component resampling processor 540 execute resampling processing in accordance with the reduction ratio A.

[0058] 12A to 12D are diagrams showing an example of the resampling process when the reduction ratio is 1 / 2.

[0059] In FIG. 12A, the R component resampling processor 510 extracts pixel R00 from among four pixels R00, R01, R10, and R11 of the R component of the post-reduction processing image S2R as a resampled image RS2R.

[0060] In FIG. 12B, the Gr component resampling processor 520 extracts the pixel Gr01' from among the four pixels Gr00', Gr01', Gr10', and Gr11' of the Gr component of the post-reduction processing image S2Gr' as a resampled image RS2Gr'.

[0061] In FIG. 12C, the Gb component resampling processor 530 extracts pixel Gb10' from the four pixels Gb00', Gb01', Gb10', and Gb11' of the Gb component of the post-reduction processing image S2Gb' as a resampled image RS2Gb'.

[0062] In FIG. 12D, the B component resampling processor 540 extracts pixel B11 from among four pixels B00, B01, B10, and B11 of the B component of the post-reduction processing image S2B as a resampled image RS2B.

[0063] The data combining unit 600 combines the resampled images RS2R, RS2Gr', RS2Gb', and RS2B input from the resampling processing unit 400 to generate a reduced RAW image D3.

[0064] FIG. 13 is a diagram showing an example of the combining process when the reduction ratio is 1 / 2.

[0065] In Figure 13, pixel R00 extracted by the R component resampling processing unit 510 as the resampled image RS2R, pixel Gr01' extracted by the Gr component resampling processing unit 520 as the resampled image RS2Gr', pixel Gb10' extracted by the Gb component resampling processing unit 530 as the resampled image RS2Gb', and pixel B11 extracted by the B component resampling processing unit 540 as the resampled image RS2B are combined.

[0066] In the image processing of this embodiment, a blend ratio B is used to mix first interpolation data generated by a first interpolation process in which the Gr and Gb components (first and second color components) of the G components are interpolated in one (common) map as the same color component, and second interpolation data generated by a second interpolation process in which the Gr and Gb components (first and second color components) of the G components are interpolated in multiple (separate) maps as different color components. rate Determine the blend ratio B rate Based on the composite data (interpolated composite image H2Gr', interpolated composite image H2Gb'), reduced data of image data including Gr and Gb components (first and second color components) is generated. This makes it possible to perform image processing optimized for the Gr and Gb components (first and second color components).

[0067] That is, when the reduction ratio A is larger than a first threshold value S1 (e.g., 3 / 4), the first interpolation data generated by the first interpolation process in which the Gr component and the Gb component (first and second color components) are interpolated as the same color component in one (common) map is reduced, whereas when the reduction ratio A is smaller than a second threshold value S2 (e.g., 1 / 4), the second interpolation data generated by the second interpolation process in which the Gr component and the Gb component (first and second color components) are interpolated as different color components in multiple (separate) maps is reduced.

[0068] In other words, the threshold value S includes a first threshold value S1 and a second threshold value S2, and when the reduction ratio A is greater than the first threshold value S1, the second interpolation process is not performed and the second interpolated data is not generated, and when the reduction ratio A is smaller than the second threshold value S2, the first interpolation process is not performed and the first interpolated data is not generated.

[0069] Furthermore, when the reduction ratio A is smaller than the first threshold value S1 (for example, 3 / 4) and larger than the second threshold value S2 (for example, 1 / 4), the blend ratio B determined based on the reduction ratio A is rate The first interpolation data and the second interpolation data are combined (mixed) and reduced according to the above. More specifically, the larger the reduction rate A (closer to unity), the higher the usage rate of the first interpolation data becomes compared to the usage rate of the second interpolation data, and the smaller the reduction rate A, the lower the usage rate of the first interpolation data becomes compared to the usage rate of the second interpolation data.

[0070] In this way, when the reduction ratio A is greater than the first threshold value S1 (e.g., 3 / 4) (close to life size), a high-quality image with reduced jaggies and improved resolution can be obtained, while when the reduction ratio A is less than the second threshold value S2 (e.g., 1 / 4), a high-quality image with suppressed false colors and moire can be obtained. Also, when the reduction ratio A is less than the first threshold value S1 (e.g., 3 / 4) and greater than the second threshold value S2 (e.g., 1 / 4), the blending ratio B determined based on the reduction ratio A is used. rate Since the first interpolation data and the second interpolation data are synthesized (mixed) and then reduced according to the above, it is possible to obtain a high-quality image that achieves a balance between reducing jaggies, improving resolution, and suppressing false colors and moire.

[0071] If binary image processing control is performed to determine whether the first or second interpolation process (first or second interpolation data) is performed using a certain threshold as a reference, there is a risk that satisfactory reduction may not be achieved when reducing at a reduction ratio close to the threshold. In other words, when the reduction ratio is close to the threshold, the choice is between prioritizing improved resolution and reduced jaggies (tolerating the occurrence of false colors and moiré), or prioritizing suppression of false colors and moiré (tolerating reduced resolution and the occurrence of jaggies), and satisfactory reduction may not be achieved.

[0072] In contrast, in the present embodiment, the blending ratio B is determined based on the reduction ratio A. rate The blend ratio B rate By reducing the result of combining (mixing) the first interpolation data and the second interpolation data in accordance with the above, it is possible to achieve a balance between reducing jaggies and improving resolution, and suppressing false colors and moire.

[0073] Here, the respective advantages of the first and second interpolation processes (first and second interpolation data) will be further described with reference to Figs. 14A and 14B. Figs. 14A and 14B are first diagrams showing an example of the effect of the image processing of this embodiment. Fig. 14A corresponds to the first interpolation process in which the Gr and Gb components (first and second color components) of the G components are interpolated as the same color components in one (common) map, and Fig. 14B corresponds to the second interpolation process in which the Gr and Gb components (first and second color components) of the G components are interpolated as different color components in multiple (separate) maps. Fig. 14B illustrates the Gr component, but the same applies to the Gb component.

[0074] In the first interpolation process of FIG. 14A, of the 16 pixels of 4×4, eight pixels (G01, G03, G21, G23, G10, G12, G30, G32) are included as the source data of interpolation (before interpolation). These eight pixels (G01, G03, G21, G23, G10, G12, G30, G32) as the source data of interpolation (before interpolation) are enclosed in a thick frame and indicated with the code GA. In contrast, in the second interpolation process of FIG. 14B, of the 16 pixels of 4×4, four pixels (Gr01, Gr03, Gr21, Gr23) are included as the source data of interpolation (before interpolation). These four pixels (Gr01, Gr03, Gr21, Gr23) as the source data of interpolation (before interpolation) are enclosed in a thick frame and indicated with the code GrA. In this way, the first interpolation process, which has a larger amount of source (pre-interpolation) data, is more advantageous in terms of reducing jaggies and improving resolution.

[0075] On the other hand, in the second interpolation process of FIG. 14B, the Gr component and the Gb component of the G component are made into separate maps, so that the interpolation can be performed by referring to a wider range. That is, in the first interpolation process of FIG. 14A, the eight pixels GA (G01, G03, G21, G23, G10, G12, G30, G32) as the data to be interpolated (before interpolation) are close to each other, whereas in the second interpolation process of FIG. 14B, the four pixels GrA (Gr01, Gr03, Gr21, Gr23) as the data to be interpolated (before interpolation) are far from each other. For example, when performing interpolation by referring to the left and right pixels, the number of pixels increases by +1, but this actually corresponds to referring to a wider range, and it is possible to obtain a stronger low-pass filter effect as a countermeasure against false colors and moire. When performing the interpolation and the low-pass filter process simultaneously, the number of reference pixels can be reduced as the pixels are farther apart. If the buffer to hold the input data is the same, the smaller the number of reference pixels, the wider the range of the low-pass filter can be applied.

[0076] 15A, 15B, 15C, and 15D are second diagrams showing an example of the effect of the image processing of this embodiment. FIG. 15A and FIG. 15B correspond to a first interpolation process in which the Gr component and the Gb component (first and second color components) of the G component are interpolated as the same color components in one (common) map, and FIG. 15C and FIG. 15D correspond to a second interpolation process in which the Gr component and the Gb component (first and second color components) of the G component are interpolated as different color components in multiple (separate) maps. Although FIG. 15C and FIG. 15D show the Gr component as an example, the same applies to the Gb component. In FIG. 15A to FIG. 15D, the area surrounded by a thick solid line indicates the range LPFA of the low-pass filter, and the area surrounded by a thick dashed line indicates the range Din of the input data referred to in the interpolation process and the low-pass filter process.

[0077] In FIG. 15A, the interpolation process is performed by referring to the input data (Din) of 12 pixels, and the low-pass filter process (LPFA) of 9 pixels of 3×3 is performed. In FIG. 15B, the interpolation process is performed by referring to the input data (Din) of 24 pixels, and the low-pass filter process (LPFA) of 25 pixels of 5×5 is performed. In FIG. 15C, the interpolation process is performed by referring to the input data (Din) of 9 pixels, and the low-pass filter process (LPFA) of 9 pixels of 3×3 is performed. In FIG. 15D, the interpolation process is performed by referring to the input data (Din) of 16 pixels, and the low-pass filter process (LPFA) of 25 pixels of 5×5 is performed. In this way, the second interpolation process in FIG. 15C and FIG. 15D achieves a wide range of low-pass filter effects with a smaller number of reference pixels.

[0078] Fig. 16 is a flowchart showing an example of image processing according to the present embodiment. The processing steps in the flowchart of Fig. 16 are executed by, for example, functional components of a CPU (computer) constituting the RAW image reduction unit (image processing device) 3.

[0079] In step ST1, the data division unit 100 divides the RAW image D2 into an R component divided image D2R, a Gr component divided image D2Gr, a Gb component divided image D2Gb, and a B component divided image D2B.

[0080] In step ST2, a magnitude relationship between the reduction ratio A and a threshold value S (for example, a first threshold value S1 or a second threshold value S2) is determined, and a blend ratio B based on the reduction ratio A is calculated. rate Determine the blend ratio B rate If it is 100%, proceed to step ST3 and blend ratio B rate If it is 0%, proceed to step ST7 and rate If is greater than 0% and smaller than 100%, the process proceeds to step ST11.

[0081] In step ST3, the interpolation processing unit 200 performs a first interpolation process in which the Gr component and the Gb component (first and second color components) of the G component are interpolated in one (common) map as the same color component. In step ST4, the reduction processing unit 400 performs a reduction process according to the reduction ratio A. In step ST5, the resampling processing unit 500 performs a resampling process according to the reduction ratio A. In step ST6, the data combination unit 600 combines the resampled images input from the resampling processing unit 500 to generate a reduced RAW image D3.

[0082] In step ST7, the interpolation processing unit 200 performs a second interpolation process in which the Gr component and Gb component (first and second color components) of the G component are interpolated in multiple (separate) maps as different color components. In step ST8, the reduction processing unit 400 performs a reduction process according to the reduction ratio A. In step ST9, the resampling processing unit 500 performs a resampling process according to the reduction ratio A. In step ST10, the data combination unit 600 combines the resampled images input from the resampling processing unit 500 to generate a reduced RAW image D3.

[0083] In step ST11, the synthesis processing unit (mixing processing unit) 300 blends the first interpolated data by the first interpolation process and the second interpolated data by the second interpolation process at a blend ratio B rate In step ST12, the reduction processing unit 400 performs reduction processing in accordance with the reduction ratio A. In step ST13, the resampling processing unit 500 performs resampling processing in accordance with the reduction ratio A. In step ST14, the data combination unit 600 combines the resampled images input from the resampling processing unit 500 to generate a reduced RAW image D3.

[0084] In addition, the present invention is not limited to the above-mentioned embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. In addition, the functions executed in the above-mentioned embodiment may be implemented in appropriate combinations as much as possible. The above-mentioned embodiment includes various steps, and various inventions can be extracted by appropriate combinations of the multiple components disclosed. For example, if an effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention.

[0085] For example, in the above embodiment, the Gr component and Gb component of the G component for reducing Bayer data including R, G, and B components are described as the "first and second color components," but the "first and second color components" may be color components other than the Gr component and Gb component.

[0086] The invention as described in the claims of the original application is set forth below. [Appendix 1] determining a blend ratio for blending first interpolation data generated by a first interpolation process in which the first and second color components are interpolated in one map as the same color component, and second interpolation data generated by a second interpolation process in which the first and second color components are interpolated in a plurality of maps as different color components; and generating reduced data of image data including the first and second color components based on composite data mixed in accordance with the blend ratio. [Appendix 2] 2. The image processing device according to claim 1, wherein the first and second color components are a Gr component and a Gb component of the G component in Bayer data including an R component, a G component, and a B component. [Appendix 3] 3. The image processing device according to claim 1, further comprising: a processor for processing the image data based on the blend ratio; [Appendix 4] 4. The image processing device of claim 3, wherein when the reduction ratio is greater than a threshold value, a utilization ratio of the first interpolation data is higher than a utilization ratio of the second interpolation data, and when the reduction ratio is smaller than the threshold value, a utilization ratio of the first interpolation data is lower than a utilization ratio of the second interpolation data. [Appendix 5] the threshold value includes a first threshold value and a second threshold value; The image processing device according to claim 4, wherein, when the reduction ratio is greater than the first threshold value, the second interpolation process is not executed and the second interpolated data is not generated. [Appendix 6] The image processing device according to claim 5, wherein, when the reduction ratio is smaller than the second threshold value, the first interpolation process is not executed and the first interpolated data is not generated. [Appendix 7] A method for causing a computer to execute each process of the image processing device according to any one of appendixes 1 to 6. [Appendix 8] A program for causing a computer to execute each process of the image processing device according to any one of appendixes 1 to 6. [Explanation of symbols]

[0087] 3. RAW image reduction unit (image processing device) 100 Data division section 200 Interpolation processing unit 210 R component interpolation processing unit 220···Gr component interpolation processing section 230···Gb component interpolation processing section 240···Gr·Gb component interpolation processing section 250···B component interpolation processing section 300···Synthesis processing section (mixing processing section) 310···Gr component synthesis processing unit (mixing processing unit) 320···Gb component synthesis processing unit (mixing processing unit) 400 Reduction processing unit 410: R component reduction processing unit 420···Gr component reduction processing section 430···Gb component reduction processing section 440 B component reduction processing unit 500···Resampling processing section 510 R component resampling processing unit 520···Gr component resampling processing section 530···Gb component resampling processing section 540 B component resampling processing unit 600 Data connection section 700...Reduction ratio threshold comparison section

Claims

1. a control means for setting a blending ratio for blending first interpolation data generated by a first interpolation process for interpolating a G component in the Bayer data with one map and second interpolation data generated by a second interpolation process for interpolating a G component in a row in which an R component is arranged and a G component in a row in which a B component is arranged, using separate maps, based on a reduction ratio value for reducing the Bayer data; the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. Image processing device.

2. When pixels are interpolated for each color component based on Bayer data, a blending ratio for mixing second interpolation data generated by separating a G component into a Gr component as the G component of a row in which an R component is arranged and a Gb component as the G component of a row in which a B component is arranged and performing pixel interpolation for the Gr component and the Gb component separately, and first interpolation data generated by interpolating pixels for the G component without separating the G component into the Gr component and the Gb component, a control means for setting the Bayer data based on a value of a reduction ratio when the Bayer data is reduced; the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. Image processing device.

3. An image processing method executed by an image processing device, comprising: a control process for setting a blending ratio for blending first interpolation data generated by a first interpolation process that interpolates a G component in the Bayer data with one map, and second interpolation data generated by a second interpolation process that interpolates a G component in a row in which an R component is arranged and a G component in a row in which a B component is arranged, using separate maps, based on a reduction ratio value for reducing the Bayer data; the control process sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. Image processing methods.

4. An image processing method executed by an image processing device, comprising: When interpolating pixels for each color component based on Bayer data, a blending ratio for mixing second interpolation data generated by separating a G component into a Gr component as the G component of a row in which an R component is arranged and a Gb component as the G component of a row in which a B component is arranged and performing pixel interpolation for the Gr component and the Gb component separately, and first interpolation data generated by interpolating pixels for the G component without separating the G component into the Gr component and the Gb component, a control process for setting the Bayer data based on a value of a reduction ratio when the Bayer data is reduced; the control process sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. Image processing methods.

5. A computer comprising: functioning as a control means for setting a blending ratio for blending first interpolation data generated by a first interpolation process for interpolating the G component in the Bayer data with one map, and second interpolation data generated by a second interpolation process for interpolating the G component in a row in which the R component is arranged and the G component in a row in which the B component is arranged, using separate maps, based on a reduction ratio value for reducing the Bayer data; the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. Image processing program.

6. Computer, When interpolating pixels for each color component based on Bayer data, a blending ratio for mixing second interpolation data generated by separating a G component into a Gr component as the G component of a row in which an R component is arranged and a Gb component as the G component of a row in which a B component is arranged and performing pixel interpolation for the Gr component and the Gb component separately, and first interpolation data generated by interpolating pixels for the G component without separating the G component into the Gr component and the Gb component, a control unit that sets the Bayer data based on a value of the reduction ratio when the Bayer data is reduced; the control means sets the blending ratio so that when the degree of reduction is small and the value of the reduction rate is greater than a threshold value, the utilization ratio of the first interpolation data is higher than the utilization ratio of the second interpolation data, and when the degree of reduction is large and the value of the reduction rate is smaller than a threshold value, the utilization ratio of the second interpolation data is higher than the utilization ratio of the first interpolation data. Image processing program.

Images