Imaging device, imaging method, program, and image processing method and program

Abstract

The present invention relates to an imaging device, an imaging method, a program, and an image processing method and program that make it possible to generate a RAW image by compressing a captured image using a compression method for decomposing the image into a plurality of spatial frequency components and compressing the spatial frequency components while suppressing image quality degradation of a developed image. And the data size of the RAW image is reduced. An input unit of a touch panel acquires a development size that is an image size during development of a RAW image. The irreversible compression unit generates a RAW image by compressing the captured image using a compression method for decomposing the image into a plurality of spatial frequency components and compressing the spatial frequency components at a compression ratio corresponding to the development size. The larger the compression ratio is, the smaller the developing size is. The present technology can be applied to, for example, an imaging device that captures an image and records a RAW image.

Description

Technical Field

[0001] The present technology relates to an imaging device, an imaging method, a program, and an image processing method and program, and more particularly, to an imaging device, an imaging method, a program, and an image processing method and program as follows: when a captured image is compressed by a compression method that compresses an image by decomposing the image into multiple spatial frequency components to generate a RAW image, the data size of the RAW image can be reduced while suppressing the degradation of the image quality of the developed image. Background Art

[0002] Imaging devices that capture, compress, and record images are becoming more and more common. Imaging elements that capture moving images, reduce the moving images for simple compression, and record the compressed moving images are also designed (for example, see Patent Document 1).

[0003] Meanwhile, in such an imaging device, there is a case where a RAW image generated from a captured image is recorded. A RAW image is an image that is regarded as an undeveloped image by development software. Typically, a RAW image is a captured image (hereinafter, also referred to as a captured image) captured by an image sensor and is recorded as it is without applying a development process in the imaging device. A developed image is obtained from the RAW image by performing a development process using development software. In addition, various development parameters are provided in the development software, allowing the user to freely adjust the developed image. In addition, an image generated by performing a visually lossless transformation process on the captured image from the perspective of color reproduction is also regarded as a RAW image. Examples of transformation processes include processes for adjusting white balance, nonlinear transformation processes using a logarithmic (Log) curve, and the like. Since such a transformation process can be offset by performing an inverse transformation process during development, it can be regarded as equivalent to a captured image that has not undergone a transformation process. Thus, in the development software, not only is a captured image regarded as a RAW image, but a captured image that has undergone a lossless transformation process is also regarded as a RAW image.

[0004] Note that a developed image is usually generated by performing many lossy processes on a RAW image, which makes it impossible to restore an image close enough to the captured image from the developed image. Thus, the developed image cannot be regarded as a RAW image.

[0005] As a method of reducing the data size of such a RAW image, various methods have been proposed. For example, in a case where the number of pixels of a developed image may be low, there is a method of reducing the data size (file size) of a RAW image by generating a RAW image having a developed size, which is an image size indicating the number of pixels of the developed image. According to this method, for example, a captured size, which is an image size of a captured image, is reduced to a developed size by a reduction process, a demosaic process, etc. that does not affect color reconstruction, and the captured image is compressed by a lossless Joint Photographic Experts Group (JPEG) method, etc., to generate a RAW image. Thus, both the data size of the RAW image and the development load can be reduced.

[0006] In addition, there is also a method of generating a RAW image by lossy compressing a captured image by a compression method using a wavelet transform without changing the image size, thereby reducing the data size of the RAW image. Compression by a compression method using a wavelet transform (hereinafter referred to as wavelet compression) corresponds to compression using spatial correlation in an image. Specifically, in wavelet compression, an image is decomposed into a plurality of spatial frequency components for compression.

[0007] This method maintains the image size of the RAW image the same as the captured size. In addition, by applying a mechanism for decomposing spatial frequency components, a RAW image having fewer pixels than the original RAW image can be extracted at the time of decompression of wavelet compression, and development processing (hereinafter referred to as reduced-size development) with fewer pixels than the original RAW image can be performed. That is, the data size of the RAW image can be reduced regardless of whether development is performed at a reduced size or at a full size.

[0008] Citation List

[0009] Patent Literature

[0010] Patent Document 1: Japanese Patent Application Publication No. 2017-135760 Summary of the invention

[0011] Problems to be solved by the present invention

[0012] However, according to this method, the captured image is lossy compressed, so the larger the compression ratio, the more the image quality of the developed image deteriorates. Therefore, it is difficult to reduce the data size of the RAW image by increasing the compression ratio.

[0013] Therefore, in the case of compressing a captured image to generate a RAW image by using a compression method that compresses an image by decomposing it into multiple spatial frequency components (such as a compression method using a wavelet transform), it is desired to provide a method for reducing the data size of the RAW image while suppressing the degradation of the image quality of the developed image, but this demand has not been fully met.

[0014] The present technology has been proposed in view of such situations, and its purpose is to reduce the data size of a RAW image while suppressing degradation in the image quality of a developed image when compressing a captured image to generate a RAW image by a compression method that compresses the image by decomposing the image into multiple spatial frequency components.

[0015] Solution to the problem

[0016] An imaging device or program according to a first aspect of the present technology is an imaging device or a program that causes a computer to function as the imaging device, including: an acquisition unit that acquires a development size corresponding to an image size of a RAW image when the image is developed; and a compression unit that compresses a captured image by a compression method that compresses the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired by the acquisition unit to generate a RAW image, wherein the compression ratio increases as the development size decreases.

[0017] An imaging method according to a first aspect of the present technology includes: using an imaging device to acquire a development size corresponding to an image size of a RAW image when it is developed; and compressing a captured image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired in the acquisition to generate a RAW image, wherein the compression ratio increases as the development size decreases.

[0018] In a first aspect of the present technology, a development size corresponding to an image size of a RAW image at the time of development is acquired, and a captured image is compressed by a compression method that compresses the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size to generate a RAW image. The compression ratio increases as the development size decreases.

[0019] According to a second aspect of the present technology, an image processing method includes: using an image processing device, acquiring size information and a RAW image, the size information indicating a development size corresponding to the image size of the RAW image when it is developed, and the RAW image corresponds to a captured image compressed by a compression method that compresses the image by decomposing the image into multiple spatial frequency components at a compression ratio corresponding to the development size; and using the image processing device, developing the RAW image by decompressing the RAW image acquired in the acquisition based on the development size indicated by the size information acquired in the acquisition to generate a developed image having the development size, wherein the compression ratio increases as the development size decreases.

[0020] A program according to a second aspect of the present technology is a program that causes a computer to function as an image processing device, the image processing device including: an acquisition unit that acquires size information and a RAW image, the size information indicating a development size corresponding to an image size of the RAW image at the time of development, the RAW image corresponding to a captured image compressed by a compression method that compresses the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size; and a developing unit that decompresses the RAW image acquired by the acquisition unit based on the development size indicated by the size information acquired by the acquisition unit to generate a developed image having the development size, wherein the compression ratio increases as the development size decreases.

[0021] In a second aspect of the present technology, the size information indicates a development size corresponding to an image size at the time of development of a RAW image, and the RAW image corresponds to a captured image compressed by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size, and the RAW image is decompressed based on the development size indicated by the size information to generate a developed image having the development size. The compression ratio increases as the development size decreases. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a block diagram illustrating a configuration example of an imaging device according to an embodiment to which the present technology is applied.

[0023] Figure 2 is a block diagram illustrating an example of the configuration of a lossy compression unit.

[0024] Figure 3 is a diagram illustrating an example of a compression ratio table.

[0025] Figure 4 is a diagram illustrating a first example of the recording method setting screen.

[0026] Figure 5 is a diagram illustrating a second example of the recording method setting screen.

[0027] Figure 6 is a flowchart for describing the setting process.

[0028] Figure 7 is a flowchart for describing the lossy compression process.

[0029] Figure 8 is a block diagram illustrating a configuration example of an embodiment of a developing apparatus as an image processing apparatus to which the present technology is applied.

[0030] Fig. 9is a block diagram illustrating a configuration example of a lossy compression RAW processing unit.

[0031] Fig.10 : is a diagram illustrating the relationship between the RAW image recording method and the presence or absence of reduction at the time of recording and at the time of development.

[0032] Fig.11 are diagrams illustrating the processes of the image forming apparatus and the developing apparatus.

[0033] Fig.12 are diagrams illustrating examples of processing of other image forming apparatuses and other developing apparatuses.

[0034] Fig.13 are diagrams illustrating examples of processing of still further other image forming apparatuses and other developing apparatuses.

[0035] Fig.14 Is used for comparison Figure 11 to Figure 13 Schematic diagram of the processing and processing results.

[0036] Fig.15 is a flowchart for describing the update process.

[0037] Fig.16 is a flowchart for describing lossy compressed RAW development processing.

[0038] Fig.17 This is a block diagram illustrating an example of the configuration of hardware of a computer. DETAILED DESCRIPTION

[0039] Hereinafter, a mode for carrying out the present technology (hereinafter referred to as an embodiment) will be described. Note that the description will be made in the following order.

[0040] 1. An embodiment

[0041] 2. Computer

[0042] <One embodiment>

[0043] <Configuration Example of Imaging Apparatus>

[0044] Figure 1 is a block diagram illustrating a configuration example of an imaging device according to an embodiment to which the present technology is applied.

[0045] Figure 1The imaging device 10 in the embodiment includes an image sensor 11, a selection unit 12, a development processing unit 13, a YC codec 14, a non-compression unit 15, a lossless compression unit 16, a lossy compression unit 17, a recording control unit 18, a control unit 19, a storage unit 20, and a touch panel 21. The imaging device 10 captures an image and records a developed image obtained by developing the captured image or a RAW image generated from the captured image on a recording medium 30.

[0046] The image sensor 11 (imaging unit) captures an image of a subject to obtain (an analog signal of) a captured image as a Bayer image (Bayer Image), and supplies the captured image to the selection unit 12 .

[0047] The selection unit 12 supplies the captured image supplied from the image sensor 11 to the development processing unit 13. The selection unit 12 supplies the captured image supplied from the image sensor 11 to the development processing unit 13, the non-compression unit 15, the lossless compression unit 16, or the lossy compression unit 17 based on a selection signal supplied from the control unit 19.

[0048] The development processing unit 13 performs development processing on the captured image supplied from the selection unit 12 to generate a developed image. The development processing is, for example, a process of converting the Bayer image into a YCbCr image. The development processing unit 13 supplies the developed image to the YC codec 14.

[0049] The YC codec 14 quantizes and JPEG-encodes the developed image supplied from the development processing unit 13 to generate a JPEG image. The YC codec 14 supplies the JPEG image to the recording control unit 18. Note that the encoding method applied to the YC codec 14 may be H.265. In this case, the YC codec 14 generates a High Efficiency Image File Format (HEIF) image from the developed image and supplies the HEIF image to the recording control unit 18.

[0050] The non-compression unit 15 performs non-compression processing including quantization and encoding without compression on the captured image supplied from the selection unit 12 to generate a RAW image. The non-compression unit 15 supplies the RAW image to the recording control unit 18.

[0051] The lossless compression unit 16 performs lossless compression processing on the captured image supplied from the selection unit 12 based on the development size supplied from the control unit 19, including reducing the image size to the development size as needed, and compressing the captured image by a lossless compression method such as a lossless JPEG method to generate a RAW image.

[0052] Specifically, the lossless compression unit 16 generates a captured image having a development size by performing a reduction process, a demosaic process, etc. on the captured image based on a development size that is less than or equal to the capture size supplied from the control unit 19. Then, the lossless compression unit 16 compresses the captured image by a lossless compression method to generate a RAW image having a development size. The lossless compression unit 16 supplies the RAW image to the recording control unit 18.

[0053] Note that the lossless compression unit 16 may convert the captured image as a Bayer image into a YCbCr image before the reduction process. At this time, the resolutions of the Y signal as brightness information and the Cb signal and Cr signal as color information of the YCbCr image may be made different. For example, the resolution of the Y signal is the development size, but the resolutions of the Cb signal and the Cr signal may be made smaller than the development size.

[0054] The lossy compression unit 17 (compression unit) performs lossy compression processing including wavelet compression of the captured image supplied from the selection unit 12 based on a compression ratio corresponding to the development size supplied from the control unit 19 to generate a RAW image. The details of the configuration of the lossy compression unit 17 will be referred to in detail later. Figure 2 The lossy compression unit 17 supplies the RAW image to the recording control unit 18 .

[0055] The recording control unit 18 supplies the information indicating the development size supplied from the control unit 19 to the recording medium 30 as the recording-time size information indicating the development size set when recording the RAW image, to record the information as a metadata file. In the case where the RAW image is not supplied from the non-compression unit 15, the lossless compression unit 16, or the lossy compression unit 17, the recording control unit 18 supplies the JPEG image supplied from the YC codec 14 to the recording medium 30, to record the JPEG image as a JPEG file. The recording control unit 18 supplies the RAW image supplied from the non-compression unit 15 to the recording medium 30, to record the RAW image as an uncompressed RAW file.

[0056] The recording control unit 18 supplies the RAW image supplied from the lossless compression unit 16 to the recording medium 30 to record the RAW image as a lossless compression RAW file. The recording control unit 18 supplies the RAW image supplied from the lossy compression unit 17 to the recording medium 30 to record the RAW image as a lossy compression RAW file. At this time, the recording control unit 18 associates a metadata file with the lossy compression RAW file.

[0057] In the case where a RAW image is supplied from the non-compression unit 15, the lossless compression unit 16, or the lossy compression unit 17, the recording control unit 18 supplies the JPEG image supplied from the YC codec 14 as a thumbnail image to the recording medium 30 to record the JPEG image as a thumbnail file. The thumbnail file is associated with the uncompressed RAW file, the lossless compression RAW file, or the lossy compression RAW file.

[0058] The control unit 19 controls each unit of the imaging device 10. For example, the control unit 19 (display control unit) controls the touch panel 21 to display various screens such as a recording setting screen for setting the presence or absence of RAW image recording and a recording method setting screen for setting a recording method indicating a development size and a compression method.

[0059] The control unit 19 acquires the recording method set by the user on the recording method setting screen from the touch panel 21. The control unit 19 provides the selection unit 12 with a selection signal indicating the selection of a compression method suitable for the recording method from among the non-compression unit 15, the lossless compression unit 16, and the lossy compression unit 17. In the case where the selection signal indicating the selection of the lossless compression unit 16 is provided to the selection unit 12, the control unit 19 provides the development size indicated by the recording method to the lossless compression unit 16. In the case where the selection signal indicating the selection of the lossy compression unit 17 is provided to the selection unit 12, the control unit 19 calculates the ratio of the development size to the development size indicated by the recording method. The control unit 19 reads the compression ratio corresponding to the ratio from the storage unit 20, provides the compression ratio to the lossy compression unit 17, and provides the development size to the recording control unit 18.

[0060] The storage unit 20 stores a compression ratio table in which the ratio of the development size to the capture size is associated with the compression ratio. The storage unit 20 provides the control unit 19 with the compression ratio registered in the compression ratio table according to the ratio of the development size to the capture size calculated by the control unit 19.

[0061] The touch panel 21 includes a display unit that performs display and an input unit that receives a touch operation performed on the display unit. The display unit of the touch panel 21 displays various screens provided from the control unit 19. The input unit of the touch panel 21 receives operations performed by the user on various screens displayed on the display unit, and acquires information corresponding to the operations. For example, the input unit of the touch panel 21 (acquisition unit) receives a touch operation of setting a recording method performed by the user on a recording method setting screen, and acquires the recording method. The input unit provides the recording method to the control unit 19.

[0062] The recording medium 30 includes a semiconductor memory, a memory card, etc., and is attachable to and detachable from the imaging device 10. The recording medium 30 records the RAW image supplied from the recording control unit 18 as an uncompressed RAW file, a lossless compressed RAW file, or a lossy compressed RAW file. The recording medium 30 records the recording time size information supplied from the recording control unit 18 as a metadata file. The metadata file is associated with the lossy compressed RAW file. The recording medium 30 records the JPEG image supplied from the recording control unit 18 as a JPEG file or a thumbnail file. The thumbnail file is associated with the uncompressed RAW file, the lossless compressed RAW file, or the lossy compressed RAW file.

[0063] <Configuration Example of Lossy Compression Unit>

[0064] Figure 2 It is a graphic description Figure 1 A block diagram of an example of the configuration of the lossy compression unit 17 in FIG.

[0065] Figure 2 The lossy compression unit 17 in FIG. 4 includes a spatial frequency transform unit 41 , a quantization unit 42 , and an encoding unit 43 .

[0066] The spatial frequency conversion unit 41 converts the Figure 1 The spatial frequency transform unit 41 performs wavelet transform on the captured image supplied from the selection unit 12 in order to decompose the captured image into a plurality of spatial frequency components (resolution components). The spatial frequency transform unit 41 supplies the spatial frequency components to the quantization unit 42.

[0067] The quantization unit 42 is based on Figure 1 The quantization unit 42 compresses the captured image at the compression ratio provided by the control unit 19 in the encoding unit 43, and quantizes each of the plurality of spatial frequency components provided from the spatial frequency transform unit 41 to compress the captured image at the compression ratio. The quantization unit 42 provides the compressed captured image to the encoding unit 43.

[0068] The encoding unit 43 encodes the compressed captured image supplied from the quantization unit 42 for each spatial frequency component to generate a RAW image. The encoding unit 43 supplies the RAW image to Figure 1 The recording control unit 18 in.

[0069] <Compression ratio table example>

[0070] Figure 3 is a diagram illustrating an example of a compression ratio table.

[0071] like Figure 3 As shown in the solid line table in FIG. 1 , the compression ratio table is a table in which the ratio of the developed size to the captured size is associated with the compression ratio. Figure 3In the compression ratio table of , a compression ratio of "3:1" is registered in association with a case where the ratio of the development size to the capture size is "greater than 0.75 times and not more than 1 times (full size)". In this case, Figure 3 As shown by the dashed line in the table, for example, the file size of a lossy compressed RAW file is 33.3MB.

[0072] In association with the case where the ratio of the development size to the capture size is "greater than 0.5 times and not more than 0.75 times", a compression ratio of "6:1" is registered. In this case, Figure 3 As shown by the dashed line in the table, for example, the file size of a lossy compressed RAW file is 16.6MB.

[0073] In association with the case where the ratio of the development size to the capture size is "greater than 0.25 times and not more than 0.5 times", a compression ratio of "9:1" is registered. Figure 3 As shown by the dashed line in the table, for example, the file size of a lossy compressed RAW file is 11.1MB.

[0074] In association with the case where the ratio of the development size to the capture size is "less than or equal to 0.25 times", a compression ratio of "12:1" is registered. Figure 3 As shown in the dashed line table, for example, the file size of the RAW file is 8.33MB.

[0075] As described above, in the image forming apparatus 10 , the smaller the ratio of the development size to the capture size set by the user, that is, the smaller the development size, the larger the compression ratio is set.

[0076] Specifically, since wavelet compression is lossy compression, the greater the compression ratio of the RAW image, the more the image quality of the developed image deteriorates. However, the smaller the development size, the less obvious the deterioration of the image quality of the developed image caused by the increase in the compression ratio. Therefore, the smaller the development size set by the user, the greater the compression ratio of the RAW image applied by the imaging device 10. Therefore, the data size of the RAW image can be reduced while suppressing the deterioration of the image quality of the developed image.

[0077] <First Example of Recording Method Setting Screen>

[0078] Figure 4 is a diagram illustrating a first example of the recording method setting screen.

[0079] exist Figure 4On the recording method setting screen 60, "Non-compression", "Lossless compression (L)", "Lossless compression (M)", "Lossless compression (S)", "Compressed RAW (L)", "Compressed RAW (M)" and "Compressed RAW (S)" are displayed as recording method information indicating the recording method of the RAW image.

[0080] "Non-compression" indicates a recording method indicating a compression method that does not perform compression. "Lossless compression (L)" indicates a recording method indicating a lossless compression method as a compression method and a capture size as a development size. "Lossless compression (M)" indicates a recording method indicating a lossless compression method as a compression method and a 1 / 2 times (50%) capture size as a development size. "Lossless compression (S)" indicates a recording method indicating a lossless compression method as a compression method and a 1 / 4 times (25%) capture size as a development size.

[0081] "Compressed RAW (L)" indicates a recording method indicating a lossy compression method as a compression method and a capture size as a development size. "Compressed RAW (M)" indicates a recording method indicating a lossy compression method as a compression method and a 1 / 2 times (50%) capture size as a development size. "Compressed RAW (S)" indicates a recording method indicating a lossy compression method as a compression method and a 1 / 4 times (25%) capture size as a development size.

[0082] The user touches a display position of recording method information indicating a desired recording method in the recording method setting screen 60 displayed on the touch panel 21 to set the recording method. A cursor 61 is displayed in the recording method information indicating the recording method being set. Figure 4 In the example in which the user sets a recording method indicating a lossless compression method as a compression method and a capture size as a development size, a cursor 61 is displayed in the recording method information "lossless compression (L)" of the recording method.

[0083] As described above, when the lossless compression method or the lossy compression method is set as the compression method, three options of the same size as the capture size, 1 / 2 times the capture size, and 1 / 4 times the capture size are displayed on the recording method setting screen 60. The user can set a desired development size from among these three options.

[0084] <Second Example of Recording Method Setting Screen>

[0085] Figure 5 is a diagram illustrating a second example of the recording method setting screen.

[0086] exist Figure 5On the recording method setting screen 80 in Figure 4 The parts corresponding to the recording method setting screen 60 in FIG. 6 are denoted by the same reference numerals. Therefore, the description of these parts will be omitted appropriately, and the following description will focus on the parts different from the recording method setting screen 60.

[0087] The recording method setting screen 80 is different from the recording method setting screen 60 in that, as the recording method information, “Compressed” is displayed instead of “Compressed RAW (L)”, and “Developed pixel number specifies the RAW pixel number [] Mpix” is displayed instead of “Compressed RAW (M)” and “Compressed RAW (S)”, while the other parts are similar to those of the recording method setting screen 60.

[0088] "Compression" indicates the recording method indicated by "Compressed RAW (L)" on the recording method setting screen 60. "Development pixel number specifies RAW pixel number [] Mpix" indicates a recording method that indicates a lossy compression method as a compression method and a size input by a user as a development size. In a case where the user touches the display position of the recording method information "Development pixel number specifies RAW pixel number [] Mpix" to set the recording method indicated by the recording method information, the user inputs the number of pixels as a desired development size through a touch operation or the like. As a result, the recording method indicates that the lossy compression method is set as the compression method and the development size input by the user is set as the development size.

[0089] As described above, on the recording method setting screen 80 , in the case where the lossy compression method is set as the compression method, any desired size can be set as the development size.

[0090] <Description of Setting Process>

[0091] Figure 6 is used to describe the Figure 1 Flowchart of a setting process for setting a recording method by the imaging device 10 in FIG. This setting process is started, for example, in response to a command from a user to display the recording method setting screen 60 ( 80 ).

[0092] exist Figure 6 In step S10, the control unit 19 causes the display unit of the touch panel 21 to display the recording method setting screen 60 (80). The user touches the display position of the recording method information indicating the desired recording method on the recording method setting screen 60 (80) to set the recording method.

[0093] In step S11 , the input unit of the touch panel 21 receives a touch operation, acquires a recording method set by a user, and provides the recording method to the control unit 19 .

[0094] In step S12, the control unit 19 determines whether the recording method supplied from the touch panel 21 indicates the non-compression method. In the case where it is determined in step S12 that the recording method indicates the non-compression method, the process proceeds to step S13.

[0095] In step S13, the control unit 19 supplies a selection signal indicating selection of the non-compression unit 15 to the selection unit 12, and terminates the processing.

[0096] In the case where it is determined in step S12 that the recording method does not indicate the non-compression method, the process proceeds to step S 14. In step S14, the control unit 19 determines whether the recording method supplied from the touch panel 21 indicates the lossless compression method.

[0097] In the case where it is determined in step S14 that the recording method indicates the lossless compression method, the process proceeds to step S 15. In step S15, the control unit 19 supplies a selection signal indicating selection of the lossless compression unit 16 to the selection unit 12.

[0098] In step S16, the control unit 19 supplies the development size indicated by the recording method to the lossless compression unit 16, and terminates the process.

[0099] On the other hand, in step S14, if the recording method does not indicate the lossless compression method, that is, in step S14, the process proceeds to step S17. In step S17, the control unit 19 supplies a selection signal indicating selection of the lossy compression unit 17 to the selection unit 12.

[0100] In step S18, the control unit 19 calculates the ratio of the development size to the captured size based on the development size indicated by the recording method. In step S19, the control unit 19 provides the ratio to the storage unit 20 and reads the compression ratio registered in the compression ratio table in association with the ratio. The control unit 19 provides the compression ratio to the lossy compression unit 17.

[0101] In step S20, the control unit 19 supplies the development size indicated by the recording method to the recording control unit 18, and records the recording-time size information indicating the development size as a metadata file on the recording medium 30. Then, the process is ended.

[0102] <Description of lossy compression processing>

[0103] Figure 7 17 is a flowchart describing the lossy compression process of the lossy compression unit 17. The lossy compression process is based on, for example, Figure 6 The selection signal supplied in the process of step S17 in starts from when the selection unit 12 supplies the captured image to the lossy compression unit 17.

[0104] exist Figure 7 In step S41 in FIG. 1 , the spatial frequency transform unit 41 of the lossy compression unit 17 performs wavelet transform on the captured image supplied from the selection unit 12 to decompose the captured image into a plurality of spatial frequency components. The spatial frequency transform unit 41 supplies the spatial frequency components to the quantization unit 42 .

[0105] In step S42, the quantization unit 42 generates Figure 6 The quantization unit 42 quantizes the plurality of spatial frequency components generated by the decomposition in the process of step S41 at the compression ratio supplied from the control unit 19 in the process of step S19 in order to obtain a captured image compressed at the compression ratio. The quantization unit 42 supplies the compressed captured image to the encoding unit 43.

[0106] In step S43 , the encoding unit 43 encodes the captured image compressed in the process of step S42 to generate a RAW image. The encoding unit 43 supplies the RAW image to the recording control unit 18 .

[0107] In step S44, the recording control unit 18 stores the RAW image generated in the process of step S43 as a lossy compressed RAW file. Figure 6 The metadata file recorded in the process of step S20 in the above is recorded in association with the recording medium 30. Then, the process is ended.

[0108] <Configuration Example of Developing Device>

[0109] Figure 8 is a block diagram illustrating a configuration example of an embodiment of a developing apparatus as an image processing apparatus to which the present technology is applied.

[0110] Figure 8 The developing device 100 includes a readout unit 101, an uncompressed RAW processing unit 102, a lossless compression RAW processing unit 103, a lossy compression RAW processing unit 104, a developing processing unit 105, a YC codec 106, a storage unit 107, a control unit 108, an input unit 109 and a recording control unit 110.

[0111] The recording medium 30 on which an uncompressed RAW file, a lossless compressed RAW file, or a lossy compressed RAW file is recorded by the imaging device 10 is attachable to or detachable from the developing device 100. The developing device 100 reads the uncompressed RAW file, the lossless compressed RAW file, or the lossy compressed RAW file recorded on the attached recording medium 30, and develops and stores the RAW image.

[0112] Specifically, the readout unit 101 reads a RAW image recorded as an uncompressed RAW file from the recording medium 30, and supplies the RAW image to the uncompressed RAW processing unit 102. The readout unit 101 reads a RAW image recorded as a lossless compressed RAW file from the recording medium 30, and supplies the RAW image to the lossless compressed RAW processing unit 103. The readout unit 101 reads a RAW image recorded as a lossy compressed RAW file from the recording medium 30, and supplies the RAW image to the lossy compressed RAW processing unit 104.

[0113] The reading unit 101 reads and acquires a metadata file recorded in association with a lossy compression RAW file from the recording medium 30. The reading unit 101 supplies the recording-time size information included in the metadata file or the post-recording size information indicating the development size set after recording of the lossy compression RAW file to the lossy compression RAW processing unit 104.

[0114] The uncompressed RAW processing unit 102 performs uncompressed RAW processing to decode and inversely quantize the RAW image supplied from the readout unit 101 , thereby generating a Bayer image having a captured size. The uncompressed RAW processing unit 102 supplies the Bayer image to the development processing unit 105 .

[0115] The lossless compression RAW processing unit 103 performs lossless compression RAW processing to generate a Bayer image having a development size set at the time of recording by decompressing the RAW image supplied from the readout unit 101 according to a lossless compression method. The lossless compression RAW processing unit 103 supplies the Bayer image to the development processing unit 105.

[0116] The RAW image and the size information at the time of recording or the size information after recording are supplied from the readout unit 101 to the lossy compression RAW processing unit 104. The lossy compression RAW processing unit 104 performs lossy compression RAW processing by, for example, decompressing the RAW image corresponding to wavelet compression based on the development size indicated by the size information at the time of recording or the size information after recording to generate a Bayer image having the development size. The details of the configuration of the lossy compression RAW processing unit 104 will be referred to in detail later. Fig. 9 The lossy compression RAW processing unit 104 supplies the Bayer image to the development processing unit 105. Note that the lossy compression RAW processing unit 104 may generate an RGB image instead of the Bayer image.

[0117] The development processing unit 105 performs development processing on the Bayer image supplied from the uncompressed RAW processing unit 102, the lossless compression RAW processing unit 103, or the lossy compression RAW processing unit 104 to generate a developed image as a YCbCr image. At this time, in the case where a development size is supplied from the control unit 108, the development processing unit 105 performs reduction processing to generate a developed image having the development size. This reduction processing is performed not in the Bayer image but in the domain of the RGB image. As a result, the image quality of the developed image having the development size can be improved. The development processing unit 105 supplies the thus generated developed image to the YC codec 106.

[0118] The YC codec 106 quantizes and JPEG-encodes the developed image supplied from the development processing unit 105 to generate a JPEG image. The YC codec 106 supplies the JPEG image to the storage unit 107 to store it.

[0119] The storage unit 107 stores the JPEG image supplied from the YC codec 106 .

[0120] The control unit 108 controls each unit. For example, the control unit 108 supplies the development size of the RAW image of the uncompressed RAW file or the lossless compressed RAW file supplied from the input unit 109 to the development processing unit 105. The control unit 108 supplies information indicating the development size of the RAW image of the lossy compressed RAW file supplied from the input unit 109 to the recording control unit 110 as post-recording development size information.

[0121] The input unit 109 receives input of a development size of a RAW image of an uncompressed RAW file, a lossless compression RAW file, or a lossy compression RAW file input by a user, and supplies the development size to the control unit 108 .

[0122] The recording control unit 110 provides the post-recording development size information provided from the control unit 108 to the recording medium 30 to record the post-recording development size information in the metadata file recorded on the recording medium 30. At this time, in the case where the post-recording development size information is already included in the metadata file, the recording control unit 110 updates the post-recording development size information to the new post-recording development size information provided from the control unit 108. The post-recording development size information is read and acquired by the readout unit 101.

[0123] Note that in the developing device 100, the JPEG image is stored in the built-in storage unit 107, but may be recorded on the recording medium 30. The recording medium 30 may be built in the imaging device 10. In this case, the imaging device 10 and the developing device 100 are connected by a cable or the like, so that the developing device 100 can read various files from the recording medium 30.

[0124] <Configuration Example of Lossy Compression RAW Processing Unit>

[0125] Fig. 9 It is a graphic description Figure 8 1 is a block diagram of a configuration example of the lossy compression RAW processing unit 104 in FIG.

[0126] Fig. 9 The lossy compression RAW processing unit 104 in FIG. 1 includes a decoding unit 121 , an inverse quantization unit 122 , and an inverse spatial frequency transform unit 123 .

[0127] The decoding unit 121 of the lossy compression RAW processing unit 104 decodes the Figure 8 The RAW image supplied by the readout unit 101 is decoded, and the decoded RAW image is supplied to the inverse quantization unit 122.

[0128] The inverse quantization unit 122 inversely quantizes (re-quantizes) the RAW image supplied from the decoding unit 121. The inverse quantization unit 122 supplies the inversely quantized RAW image to the inverse spatial frequency transform unit 123.

[0129] The spatial frequency inverse transform unit 123 performs spatial frequency inverse transform on the RAW image supplied from the inverse quantization unit 122 based on the development size indicated by the recording size information or the post-recording size information supplied from the readout unit 101 to decompress the RAW image. Specifically, the spatial frequency inverse transform unit 123 performs spatial frequency inverse transform on a predetermined spatial frequency component corresponding to the development size among a plurality of spatial frequency components as the inverse quantized RAW image to generate a Bayer image having the development size. The spatial frequency inverse transform unit 123 supplies the Bayer image to the Figure 1 The development processing unit 105 in the development processing unit 105. Note that since the extractable Bayer image size is discrete, the spatial frequency inverse transform unit 123 can generate a Bayer image having a size required for generating a developed image having a development size, instead of generating a Bayer image having a development size, and the development processing unit 105 can reduce the Bayer image to the development size during the development process. For example, in the case where the capture size is 50MP and the development size is 10MP, the spatial frequency inverse transform unit 123 can generate a 12.5-MP Bayer image that is easy to extract, and the development processing unit 105 can generate a developed image reduced to 10MP.

[0130] <Relationship between the Recording Method and the Presence or Absence of Reduction during Recording and Development of RAW Images>

[0131] Fig.10 : is a diagram showing the relationship between the RAW image recording method and the presence or absence of reduction during recording and development.

[0132] exist Fig.10 In the table, except for the first row, information indicating the RAW image compression method when the recording method indicated by the recording method information is set, whether the RAW image is reduced when recording, and whether the RAW image is reduced when developing is recorded is recorded in association with the recording method information.

[0133] exist Fig.10 In the first row of the table of , information indicating the RAW image compression method, the presence or absence of reduction when recording the RAW image, and the presence or absence of reduction when developing the RAW image in the case of "no RAW image recording" is recorded in association with "no RAW image recording" instead of the recording method information. "No RAW image recording" indicates a case in which the user sets no RAW image recording on the recording setting screen. In this case, since no RAW image is recorded, "not applicable" (N / A) is recorded as the information indicating the RAW image compression method, the presence or absence of reduction when recording the RAW image, and the presence or absence of reduction when developing the RAW image.

[0134] As in Fig.10 As shown in the second row of the table in , when the recording method indicated by the recording method information "non-compression" is set, the RAW image compression method is a method of not compressing, and the image size is not reduced when the RAW image is recorded. In this case, the image size is not reduced when developing unless instructed by the user.

[0135] As in Fig.10 As shown in the third row of the table in , in the case where the recording method indicated by the recording method information "lossless compression (L)" is set, the RAW image compression method is, for example, the lossless JPEG method, and the image size is not reduced at the time of recording the RAW image. In this case, the image size is not reduced at the time of development unless instructed by the user.

[0136] As in Fig.10 As shown in the fourth row of the table in , in the case where the recording method indicated by the recording method information "Compression (L)" or "Compression" is set, the RAW image compression method is, for example, a method using wavelet transform, and the image size is not reduced when the RAW image is recorded. In this case, the image size is not reduced when developing unless instructed by the user.

[0137] As in Fig.10As shown in the fifth row of the table in , in the case where the recording method indicated by the recording method information "lossless compression (M)" or "lossless compression (S)" is set, the RAW image compression method is, for example, the lossless JPEG method, and the image size is reduced at the time of recording of the RAW image. In this case, the image size is not reduced at the time of development unless instructed by the user.

[0138] As in Fig.10 As shown in the sixth row of the table in , in the case where the recording method indicated by the recording method information "Compressed RAW (M)", "Compressed RAW (S)" or "Developed pixel number specifies the number of RAW pixels [] Mpix" is set, the RAW image compression method is, for example, a method using wavelet transform. In this case, the image size is not reduced at the time of recording of the RAW image, but is reduced at the time of development based on the size information at the time of recording or the size information after recording.

[0139] <Description of Processing of Image Forming Apparatus and Developing Apparatus>

[0140] Fig.11 1 is a diagram illustrating the processing of the imaging device 10 and the developing device 100 in the case where the recording method indicated by the recording method information “Compressed RAW (S)” is set.

[0141] When recording a RAW image, the user sets the recording method indicated by the recording method information "Compressed RAW (S)" to the imaging device 10. Fig.11 As shown in the diagram, in the case where the captured image is a 50MP Bayer image, the imaging device 10 compresses the captured image by wavelet compression at a compression ratio corresponding to 0.25 times the ratio of the development size to the capture size indicated by the recording method. Then, the imaging device 10 generates a 50MP wavelet-compressed Bayer image as a RAW image. The imaging device 10 records the RAW image as a lossy compressed RAW file on the recording medium 30. The imaging device 10 records the recording size information indicating 12.5MP (=50×0.25) as the development size indicated by the recording method as a metadata file on the recording medium 30 in association with the lossy compressed RAW file.

[0142] In the case where the user does not set a new development size after recording of the RAW image, the development device 100 reads the RAW image and the size information at the time of recording from the recording medium 30. The development device 100 develops the RAW image at a reduced size of 12.5 MP as indicated by the size information at the time of recording, thereby generating a YCbCr image such as YC422 of 12.5 MP as a developed image.

[0143] On the other hand, when the user sets a new development size to the developing device 100 after recording of the RAW image, the developing device 100 includes the size information at the time of recording indicating the development size in the metadata file corresponding to the RAW image, and records it on the recording medium 30. When the new development size is set multiple times after recording of the RAW image, the size information at the time of recording is updated each time the new development size is set, and finally the size information at the time of recording indicating the latest development size is recorded on the recording medium 30. The developing device 100 reads the RAW image and the size information after recording from the recording medium 30.

[0144] For example, when the recorded size information indicates 50 MP, the developing device 100 develops the RAW image at full size, thereby generating a YCbCr image such as 50 MP YC422 as a developed image. As described above, since the image size of the RAW image is the captured size, the developed size can be changed to the captured size during development.

[0145] <Example of Overview of Processing of Other Image Forming Apparatuses and Other Developing Apparatuses>

[0146] Fig.12 is a diagram illustrating an example of processing of an imaging device that generates a RAW image by compressing a captured image having a development size smaller than a captured size by a lossless compression method, and a developing device that generates a developed image from the RAW image.

[0147] The user sets the development size to the image forming apparatus 201. Fig.12 In the example in , the developed size is 0.25 times the captured size. Fig.12 As shown in the diagram, in the case where the captured image is a 50MP Bayer image, the imaging device 201 converts the captured image into a YC422 YCbCr image, reduces the captured size to the developed size, and compresses the image by a lossless compression method. The imaging device 201 records the resulting 12.5MP (=50×0.25) compressed YCbCr image as a RAW image. As described above, since the imaging device 201 generates a RAW image by converting the captured image into a YC422 YCbCr image, the reduction in the brightness resolution of the RAW image can be suppressed. The developing device 202 develops the RAW image without changing the image size to generate a 12.5MP RGB image as a developed image.

[0148] Note that the imaging device 201 can generate a RAW image by reducing the image size to an intermediate size that is larger than the development size and smaller than the capture size and compressing the capture image by a lossless compression method without converting the capture image into a YCbCr image. Fig.12As shown in the diagram, the imaging device 201 generates a RAW image as a Bayer image of 25MP (=50×0.5) by, for example, reducing a captured image as a Bayer image of 50MP to 0.5 times the captured size, i.e., an intermediate size, and compressing it by a lossless compression method. The imaging device 201 records the RAW image and 12.5MP as a development size on a recording medium in association with each other. The developing device 202 develops the RAW image recorded on the recording medium at a reduced size of 12.5MP as a development size associated with the RAW image, thereby generating a 12.5MP RGB image as a developed image.

[0149] <Example of Overview of Processing of Other Image Forming Apparatus and Other Developing Apparatus>

[0150] Fig.13 is a diagram illustrating an example of an overview of processing of an imaging device that does not have a setting function for a development size, generates a RAW image by compressing a captured image with wavelet compression, and a developing device that develops the RAW image.

[0151] like Fig.13 As shown in the diagram, in the case where the captured image is a 50MP Bayer image, for example, the imaging device 221 compresses the captured image using wavelet compression at a predetermined compression ratio. The compression ratio may be preset or may be selected by the user. For example, in the case where the captured image is a moving image, the compression ratio is in the range of 3:1 to 5:1, and in the case where the captured image is a still image, the compression ratio is approximately 4:1. The imaging device 221 records the wavelet-compressed 50MP Bayer image as a RAW image.

[0152] In the case where the user does not set the development size for the development device 222, the development device 202 develops the RAW image at full size, thereby generating a 50MP RGB image as a developed image. On the other hand, in the case of developing the RAW image at a reduced size, the user sets the development size for the development device 222. In the case where the user sets 12.5MP as the development size, the development device 202 develops the RAW image at a reduced size of 12.5MP, thereby generating a 12.5MP RGB image as a developed image.

[0153] <Description of Effects of the Present Technology>

[0154] Fig.14 Is used for comparison Figure 11 to Figure 13 Schematic diagram of the processing and processing results.

[0155] like Fig.14As shown in the table in , in the imaging device 10 and the development process 100, the development size is set at the time of capture, that is, at the time of recording of the RAW image. The development size can be changed after the recording of the RAW image, such as at the time of development.

[0156] On the other hand, Fig.12 In the imaging device 201 and the developing device 202, the developing size can be set only at the time of capturing, that is, at the time of recording the RAW image. Fig.13 In the imaging device 221 and the developing device 222, the developing size can be set only when the RAW image is developed.

[0157] When the development size set at the time of capture is smaller than the capture size, the file size (data size) of the lossy compression RAW file generated by the processing of the imaging device 10 is smaller than the file size of the RAW file generated by the processing of the imaging device 221 .

[0158] Specifically, since wavelet compression is lossy compression, the greater the compression ratio of wavelet compression, the more the image quality of the developed image deteriorates. However, the smaller the development size, the less obvious the deterioration of the image quality of the developed image. Therefore, the imaging device 10 performs compression so that the compression ratio increases as the development size set by the user decreases, thereby reducing the file size while suppressing the deterioration of the image quality of the developed image.

[0159] On the other hand, since the imaging device 221 does not have a setting function for the development size, the imaging device 221 compresses the captured image by wavelet compression at a predetermined compression ratio regardless of the development size. Therefore, the imaging device 221 needs to set the compression ratio to be smaller in consideration of the image quality in the case where the development size is the capture size, that is, the image quality in the case where the degradation of the image quality of the developed image caused by the increase in the compression ratio is most significant. Therefore, the file size of the RAW image generated by the imaging device 221 is larger than the file size of the lossy compressed RAW file generated by the imaging device 10.

[0160] The imaging device 201 generates a RAW image having a development size smaller than the capture size. Thus, the file size of the RAW image generated by the imaging device 201 is larger than the file size of the RAW file generated by the processing of the imaging device 221.

[0161] As described above, regarding the file size of the RAW image, the imaging apparatus 10 and the imaging apparatus 201 are superior to the imaging apparatus 221 .

[0162] The development size of the developed image generated by the processing of the developing device 100 is automatically set to the development size set at the time of capture and smaller than or equal to the capture size. Note that, in the case where a new development size is set after recording of the RAW image (such as at the time of development), the development size of the developed image is set to the newly set development size.

[0163] The developed size of the developed image generated by the processing of the developing device 202 is the developed size set at the time of capture and smaller than the captured size. The developed size of the developed image generated by the processing of the developing device 222 is the developed size set at the time of development and smaller than or equal to the captured size.

[0164] As described above, the developing device 100 and the developing device 222 are superior to the developing device 202 because the development size of the developed image can be set at the time of development. The developing device 100 is also superior to the developing device 222 because no setting is required at the time of development even when developing at a reduced size.

[0165] Since the developing device 100 performs so-called layered decoding when developing at a reduced size, in which only the spatial frequency components of the RAW image required to generate the developed image having the development size are subjected to spatial frequency inverse transformation, the processing load can be reduced compared to developing at full size. Thus, the generation speed of the developed image when developing at a reduced size is fast. On the other hand, when developing at full size, it is necessary to perform spatial frequency inverse transformation on all spatial frequency components of the RAW image, which makes the generation speed of the developed image slow. The same applies to the generation speed of the developed image of the developing device 222. Since the developing device 202 develops the RAW image having a development size smaller than the captured size, the generation speed of the developed image of the developing device 202 is fast.

[0166] Since the degradation of the image quality of the developed image generated by the process of the developing device 100 due to the large compression ratio is not noticeable in the case of development at a reduced size, the image quality is a standard level similar to that of the developed image generated by the developing device 202. In the case of development at a reduced size, the image quality of the developed image generated by the developing device 222 is a standard level similar to that of the developed image generated by the process of the developing device 100.

[0167] In the case where the development size set at the time of capture is smaller than the capture size, the compression ratio in the imaging device 100 is greater than the compression ratio in the imaging device 221. Thus, in the case of development at full size, the image quality of the developed image generated by the processing of the developing device 222 is higher than the image quality of the developed image generated by the processing of the developing device 100.

[0168] As described above, in the imaging device 10, in the case where the user develops the RAW image in a reduced size, a development size smaller than the capture size is set at the time of capture. Thus, there is an advantage that the imaging device 100 can reduce the file size of the RAW image while suppressing the degradation of the image quality of the developed image, as compared with the imaging device 221, by performing wavelet compression at a compression ratio corresponding to the development size. Note that, in the case where the user changes the development size to a larger development size at the time of development of the RAW image, for example, in the case of developing the RAW image in full size, there is a possibility that the image quality of the developed image generated by the processing of the developing device 100 is degraded compared with the developed image generated by the processing of the developing device 222. In the developing device 100, even in the case of developing the RAW image in a reduced size, there is an advantage that the user does not need to specify the development size at the time of development.

[0169] On the other hand, the imaging device 201 reduces the image size to the development size when recording the RAW image. Therefore, there is a disadvantage that the development device 202 cannot generate a development image with a development size larger than the development size set when recording the RAW image. That is, for example, there is a disadvantage that the development device 222 cannot develop the RAW image in full size.

[0170] Since the imaging device 221 does not have a setting function for the development size, there is a disadvantage that the file size of the RAW image cannot be reduced even when the RAW image is developed in a reduced size. In the developing device 222, when the RAW image is developed in a reduced size, there is a disadvantage that the user needs to set the development size at the time of development.

[0171] <Description of Processing of Developing Device>

[0172] Fig.15 2 is a flowchart for describing an update process for updating the development size of the development device 100. This update process is performed, for example, when an uncompressed RAW file for updating the development size is selected.

[0173] exist Fig.15 In step S51, the input unit 109 of the developing device 100 determines whether the user has input the developing size. In the case where it is determined in step S51 that the developing size has not been input, the input unit 109 waits for the input.

[0174] On the other hand, in the case where it is determined in step S51 that the development size is input, the input unit 109 supplies the development size to the control unit 108, and the process proceeds to step S52. In step S52, the control unit 108 determines whether the recorded size information already exists in the metadata file corresponding to the uncompressed RAW file being selected.

[0175] If it is determined in step S52 that the recorded size information does not exist, the process proceeds to step S53. In step S53, the control unit 108 includes the information indicating the development size supplied from the input unit 109 as the recorded size information in the metadata file corresponding to the selected uncompressed RAW file, and records it on the recording medium 30. Then, the process ends.

[0176] In the case where it is determined in step S52 that the recorded size information already exists, the process proceeds to step S54. In step S54, the control unit 108 updates the recorded size information contained in the metadata file corresponding to the selected uncompressed RAW file recorded on the recording medium 30, using the information indicating the development size supplied from the input unit 109 as the recorded size information. Then, the process ends.

[0177] Fig.16 1 is a flowchart for describing a lossy compression RAW development process for developing a lossy compression RAW file by the development device 100. For example, when a RAW image recorded on the recording medium 30 as a lossy compression RAW file is read as a development target by the readout unit 101 and supplied to the lossy compression RAW processing unit 104, the lossy compression RAW development process starts.

[0178] exist Fig.16 In step S61 in , the decoding unit 121 of the lossy compression RAW processing unit 104 decodes the RAW image supplied from the readout unit 101 , and supplies the decoded RAW image to the inverse quantization unit 122 .

[0179] In step S62, the inverse quantization unit 122 inversely quantizes the decoded RAW image obtained by the process of step S61. The inverse quantization unit 122 supplies the inversely quantized RAW image to the inverse spatial frequency transform unit 123.

[0180] In step S63 , the reading unit 101 determines whether or not there is post-recording size information in the metadata file corresponding to the lossy compressed RAW file to be developed recorded on the recording medium 30 .

[0181] In the case where it is determined in step S63 that there is no post-recording size information, the process proceeds to step S64. In step S64, the readout unit 101 determines whether there is recording size information in the metadata file corresponding to the lossy compressed RAW file recorded on the recording medium 30 to be developed.

[0182] In the case where it is determined in step S64 that the size information at the time of recording does not exist, the process proceeds to step S65. For example, the process proceeds to step S65 in the following case: the development target is a lossy compression RAW file generated by an imaging device other than the imaging device 10, and no metadata file including the size information at the time of recording is recorded in association with the lossy compression RAW file.

[0183] In step S65, the spatial frequency inverse transform unit 123 performs spatial frequency inverse transform on all the multiple spatial frequency components of the inverse quantized RAW image obtained by the process of step S62 to generate a Bayer image with a captured size. Then, the spatial frequency inverse transform unit 123 supplies the Bayer image to the development processing unit 105, and the process proceeds to step S70.

[0184] On the other hand, in the case where it is determined in step S64 that the size information at the time of recording exists, the readout unit 101 reads the size information at the time of recording from the recording medium 30, and supplies the size information at the time of recording to the spatial frequency inverse transform unit 123, and the process proceeds to step S66. In step S66, the spatial frequency inverse transform unit 123 determines whether the development size indicated by the size information at the time of recording is smaller than the capture size.

[0185] In the case where it is determined in step S66 that the development size indicated by the size information at the time of recording is not smaller than the capture size, that is, in the case where it is determined that the development size is equal to the capture size, the process proceeds to step S65. As a result, as described above, a Bayer image having the capture size is generated and supplied to the development processing unit 105, and then the process proceeds to step S70.

[0186] On the other hand, in the case where it is determined in step S66 that the development size indicated by the size information at the time of recording is smaller than the capture size, the process proceeds to step S67. In step S67, the spatial frequency inverse transform unit 123 performs spatial frequency inverse transform on only a predetermined spatial frequency component corresponding to the development size among a plurality of spatial frequency components as the inversely quantized RAW image obtained by the process of step S62 to generate a Bayer image having the development size. Then, the spatial frequency inverse transform unit 123 supplies the Bayer image to the development processing unit 105, and the process proceeds to step S70.

[0187] On the other hand, in the event that it is determined in step S63 that the recorded size information exists, the readout unit 101 reads the recorded size information from the recording medium 30 and supplies the recorded size information to the spatial frequency inverse transform unit 123, and the process proceeds to step S68.

[0188] In step S68, the spatial frequency inverse transform unit 123 determines whether the development size indicated by the post-recording size information is smaller than the capture size. In the case where it is determined in step S68 that the development size indicated by the post-recording size information is not smaller than the capture size, that is, in the case where it is determined that the development size is equal to the capture size, the process proceeds to step S65. As a result, as described above, a Bayer image having the capture size is generated and supplied to the development processing unit 105, and then the process proceeds to step S70.

[0189] On the other hand, in the case where it is determined in step S68 that the development size indicated by the post-recording size information is smaller than the capture size, the process proceeds to step S69. In step S69, the spatial frequency inverse transform unit 123 performs spatial frequency inverse transform on a predetermined spatial frequency component corresponding to the development size among a plurality of spatial frequency components as the inversely quantized RAW image obtained in step S62 to generate a Bayer image having the development size. Then, the spatial frequency inverse transform unit 123 supplies the Bayer image to the development processing unit 105, and the process proceeds to step S70.

[0190] In step S70 , the development processing unit 105 performs development processing on the Bayer image generated by the processing of step S65 , S67 or S69 to generate a developed image as a YCbCr image. The development processing unit 105 supplies the obtained developed image to the YC codec 106 .

[0191] In step S71, the YC codec 106 quantizes and JPEG-encodes the developed image generated in step S70 to generate a JPEG image. The YC codec 106 supplies the JPEG image to the storage unit 107 for storage. Then, the processing is ended. Note that the processing of the above-mentioned steps S61 and S62 can be performed immediately before the processing of steps S65, S67 and S69. In this case, for example, the decoding unit 121 and the inverse quantization unit 122 decode and inverse quantize only the spatial frequency components of the RAW image corresponding to the development size. With this configuration, the processing speed of decoding and inverse quantization when developing with a reduced size can be improved.

[0192] As described above, the imaging device 10 acquires the development size, and generates a RAW image by performing wavelet compression on the captured image at a compression ratio set to increase as the development size decreases. As a result, in a case where the development size is smaller than the capture size, the imaging device 10 can generate a RAW image having a smaller data size than a RAW image generated by the imaging device 221 that does not have a setting function for the development size. In a case where the compression ratio is large, the image quality of the developed image deteriorates, but in a case where the development size is small, the degradation is less noticeable. Thus, the imaging device 10 can reduce the data size of the RAW image while suppressing the degradation of the image quality of the developed image. As a result, the user can capture more images more smoothly using the imaging device 10.

[0193] In the imaging device 10, since the user can set the development size at the time of capturing, it is not necessary to set the development size at the time of developing. Thus, the user's workload is reduced during the development process. As a result, the development time can be saved.

[0194] The metadata file may include both size information at the time of recording and size information after recording. That is, information indicating the development size may be redundantly maintained in the metadata file. Thus, the user can change the development size at the time of development. Thus, a RAW image that was intended to be developed in a reduced size at the time of capture can be developed in full size. As a result, even in the case where an incorrect development size is set at the time of capture, in the case where the RAW image is used for a purpose other than the intended use at the time of capture, etc., the RAW image can be developed in the desired development size. Thus, user convenience is improved.

[0195] Note that in the present embodiment, in the case where the compression method indicates the lossless compression method, and the development size indicates an image size smaller than the captured size, reduction is not performed at the time of capture unless instructed by the user, but at the time of recording of the RAW image, the captured image may be reduced to an intermediate size, and the RAW image may be further reduced to the development size at the time of development. In this case, a metadata file including information indicating the development size set at the time of recording of the lossless compression RAW file is recorded on the recording medium 30 in association with the lossless compression RAW file. Then, after the lossless compression process, the lossless compression RAW processing unit 103 reduces the image size of the Bayer image to the development size indicated by the information included in the metadata file.

[0196] In the present embodiment, the post-recording development size information is recorded in the developing device 100 , but may be recorded in the image forming device 10 .

[0197] In the present embodiment, the development process is a process of converting a Bayer image into a YCbCr image, but may be a process of converting a Bayer image into an RGB image. The imaging device 10 and the development device 100 may be combined in one device. The compression method in the lossy compression unit 17 may be a compression method other than a compression method using a wavelet transform, such as a compression method using a discrete cosine transform (DCT), as long as the compression method compresses a captured image by decomposing the captured image into a plurality of spatial frequencies.

[0198] <Computer>

[0199] <Computer Configuration Example>

[0200] The above series of processing can be performed by hardware or software. In the case of performing a series of processing by software, a program configuring the software is installed in a computer. Here, examples of computers include computers incorporated into dedicated hardware, general-purpose personal computers that can perform various functions by installing various programs, etc.

[0201] Fig.17 It is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processes in accordance with a program.

[0202] In the computer, a central processing unit (CPU) 301 , a read only memory (ROM) 302 , and a random access memory (RAM) 303 are connected to each other via a bus 304 .

[0203] The bus 304 is also connected to an input / output interface 305. The input / output interface 305 is connected to an imaging unit 306, an input unit 307, an output unit 308, a storage unit 309, a communication unit 310, and a drive 311.

[0204] The imaging unit 306 includes the image sensor 11, etc. The input unit 307 includes a keyboard, a mouse, a microphone, an input unit of a touch panel, etc. The output unit 308 includes a display, a speaker, a display unit of a touch panel, etc. The storage unit 309 includes a hard disk, a nonvolatile memory, etc. The communication unit 310 includes a network interface, etc. The drive 311 drives a removable medium 312 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc.

[0205] The computer configured as described above causes the CPU 301 to load a program stored in, for example, the storage unit 309 into the RAM 303 via the input / output interface 305 and the bus 304 and execute the program, so as to perform the above-described series of processing.

[0206] The program executed by the computer (CPU 301) can be provided by being recorded on the removable medium 312 as a package medium or the like, for example. Furthermore, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0207] In the computer, by inserting the removable medium 312 into the drive 311, the program can be installed on the storage unit 309 via the input / output interface 305. In addition, the program can be received by the communication unit 310 via a wired or wireless transmission medium and installed on the storage unit 309. In addition, the program can be pre-installed on the ROM 302 and the storage unit 309.

[0208] Note that the program executed by the computer may be a program that is processed in the time series order described herein, or may be a program that is processed in parallel or at necessary timing such as when a call is made.

[0209] Software that performs a series of processes of the developing device 100 may be provided as, for example, developing software.

[0210] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the scope of the present technology.

[0211] For example, a mode obtained by combining all or some of the above-described plural embodiments may be adopted.

[0212] For example, the present technology may be embodied in cloud computing, where functions are shared and executed by multiple devices via a network.

[0213] In addition, each step described in the above flowchart may be performed by one device, or may be shared and performed by a plurality of devices.

[0214] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in the one step may be performed by one device, or may be performed by a plurality of devices in a shared manner.

[0215] Note that the effects described herein are merely examples and are not limiting, and there may be effects other than those described herein.

[0216] Note that the present technology can have the following configurations.

[0217] (1) An imaging device comprising:

[0218] an acquisition unit that acquires a development size corresponding to an image size when the RAW image is developed; and

[0219] a compression unit that compresses the captured image by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired by the acquisition unit to generate a RAW image,

[0220] in

[0221] The compression ratio increases as the development size decreases.

[0222] (2) The imaging device according to (1) above,

[0223] in

[0224] The compression method includes a compression method using wavelet transform.

[0225] (3) The imaging device according to (1) or (2) above,

[0226] Also includes:

[0227] A recording control unit records the RAW image generated by the compression unit on a recording medium in association with size information indicating a development size.

[0228] (4) The imaging device according to any one of (1) to (3) above,

[0229] Also includes:

[0230] a display control unit that controls display of a setting screen for developing size,

[0231] in

[0232] The acquisition unit acquires a development size set by a user on the setting screen.

[0233] (5) The imaging device according to (4) above,

[0234] in

[0235] The setting screen includes a screen for setting one option selected by a user from among a plurality of development size options.

[0236] (6) The imaging device according to (4) above,

[0237] in

[0238] The setting screen includes a screen in which a development size input by a user is set.

[0239] (7) The imaging device according to any one of (1) to (6),

[0240] Also includes:

[0241] An imaging unit is configured to acquire the captured image.

[0242] (8) An imaging method comprising:

[0243] Using imaging equipment,

[0244] acquiring a development size corresponding to an image size when the RAW image is developed; and

[0245] compressing the captured image by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired in the acquiring to generate a RAW image,

[0246] in

[0247] The compression ratio increases as the development size decreases.

[0248] (9) A program for causing a computer to function as an imaging device, the imaging device comprising:

[0249] an acquisition unit that acquires a development size corresponding to an image size when the RAW image is developed; and

[0250] a compression unit that compresses the captured image by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired by the acquisition unit to generate a RAW image,

[0251] in

[0252] The compression ratio increases as the development size decreases.

[0253] (10) An image processing method, comprising:

[0254] Using image processing equipment,

[0255] acquiring size information indicating a development size corresponding to an image size of the RAW image at the time of development, and a RAW image corresponding to a captured image compressed by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size; and

[0256] developing the RAW image by decompressing the RAW image acquired in the acquiring based on the development size indicated by the size information acquired in the acquiring to generate a developed image having the development size,

[0257] in

[0258] The compression ratio increases as the development size decreases.

[0259] (11) The image processing method according to (10) above,

[0260] in

[0261] The compression method includes a compression method using wavelet transform.

[0262] (12) The image processing method according to (10) or (11) above,

[0263] in

[0264] The size information includes size information at the time of recording and size information after recording, the size information at the time of recording indicating a development size set at the time of recording of the RAW image, and the size information after recording indicating a development size set after recording of the RAW image,

[0265] compressing the RAW image at a compression ratio corresponding to the development size indicated by the size information at the time of recording,

[0266] The recorded size information in the size information is obtained in the obtaining, and

[0267] In the developing, the RAW image is decompressed based on the developed size indicated by the recorded size information acquired in the acquiring.

[0268] (13) The image processing method according to (12) above further includes:

[0269] Receive input of development size,

[0270] Here, in the acquiring, information indicating the development size whose input has been received in the receiving is acquired as the post-recording size information.

[0271] (14) The image processing method according to claim 13, further comprising:

[0272] performing recording control so as to record information indicating the development size whose input has been received in the receiving as the recorded size information on the recording medium in association with the RAW image,

[0273] in

[0274] in the recording control, in a case where the recorded size information has been recorded in association with the RAW image, updating the recorded size information to information indicating a development size whose input has been received in the reception, and

[0275] In the acquisition, the recorded size information recorded in association with the RAW image is read from the recording medium for acquisition.

[0276] (15) A program for causing a computer to function as an image processing apparatus, the image processing apparatus comprising:

[0277] an acquisition unit that acquires size information indicating a development size corresponding to an image size at the time of development of the RAW image, and a RAW image corresponding to a captured image compressed by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size; and

[0278] a developing unit that decompresses the RAW image acquired by the acquiring unit based on the development size indicated by the size information acquired by the acquiring unit to generate a developed image having the development size,

[0279] in

[0280] The compression ratio increases as the development size decreases.

[0281] Reference numerals list

[0282] 10 Imaging equipment

[0283] 11 Image Sensor

[0284] 17 Lossy Compression Unit

[0285] 18 Recording control unit

[0286] 19 Control Unit

[0287] 21 Touch Panel

[0288] 30 Recording medium

[0289] 100 Image processing equipment

[0290] 101 Readout unit

[0291] 104 Lossy compression RAW processing unit

[0292] 105 Development unit

[0293] 109 Input unit

[0294] 110 Recording control unit

Claims

1. An imaging device, comprising: an acquisition unit, the acquisition unit acquiring a development size corresponding to an image size of the RAW image when it is developed; and a compression unit that compresses the captured image by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired by the acquisition unit to generate a RAW image, in The compression ratio increases as the development size decreases.

2. The imaging device according to claim 1, in The compression method includes a compression method using wavelet transform.

3. The imaging device according to claim 1, Also includes: A recording control unit records the RAW image generated by the compression unit on a recording medium in association with size information indicating a development size.

4. The imaging device according to claim 1, Also includes: a display control unit that controls display of a setting screen for developing size, in The acquisition unit acquires a development size set by a user on the setting screen.

5. The imaging device according to claim 4, in The setting screen includes a screen for setting one option selected by a user from among a plurality of development size options.

6. The imaging device according to claim 4, in The setting screen includes a screen in which a development size input by a user is set.

7. The imaging device according to claim 1, Also includes: An imaging unit is configured to acquire the captured image.

8. An imaging method comprising: Using imaging equipment, Acquire a development size corresponding to an image size when the RAW image is developed; as well as compressing the captured image by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired in the acquiring to generate a RAW image, in The compression ratio increases as the development size decreases.

9. A program for causing a computer to function as an imaging device, the imaging device comprising: an acquisition unit, the acquisition unit acquiring a development size corresponding to an image size of the RAW image when it is developed; and a compression unit that compresses the captured image by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size acquired by the acquisition unit to generate a RAW image, in The compression ratio increases as the development size decreases.

10. An image processing method, comprising: Using image processing equipment, acquiring size information indicating a development size corresponding to an image size of the RAW image at the time of development, and a RAW image corresponding to a captured image compressed by a compression method of compressing the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size; as well as developing the RAW image by decompressing the RAW image acquired in the acquiring based on the development size indicated by the size information acquired in the acquiring to generate a developed image having the development size, in The compression ratio increases as the development size decreases.

11. The image processing method according to claim 10, in The compression method includes a compression method using wavelet transform.

12. The image processing method according to claim 10, in The size information includes size information at the time of recording and size information after recording, the size information at the time of recording indicating a development size set at the time of recording of the RAW image, and the size information after recording indicating a development size set after recording of the RAW image, compressing the RAW image at a compression ratio corresponding to the development size indicated by the size information at the time of recording, The recorded size information in the size information is obtained in the obtaining, and In the developing, the RAW image is decompressed based on the developed size indicated by the recorded size information acquired in the acquiring.

13. The image processing method according to claim 12, further comprising: Receive input of development size, Here, in the acquiring, information indicating the development size whose input has been received in the receiving is acquired as the post-recording size information.

14. The image processing method according to claim 13, further comprising: performing recording control so as to record information indicating the development size whose input has been received in the receiving as the recorded size information on the recording medium in association with the RAW image, in in the recording control, in a case where the recorded size information has been recorded in association with the RAW image, updating the recorded size information to information indicating a development size whose input has been received in the reception, and In the acquisition, the recorded size information recorded in association with the RAW image is read from the recording medium for acquisition.

15. A program for causing a computer to function as an image processing apparatus, the image processing apparatus comprising: an acquisition unit that acquires size information indicating a development size corresponding to an image size of the RAW image at the time of development, and a RAW image corresponding to a captured image compressed by a compression method that compresses the image by decomposing the image into a plurality of spatial frequency components at a compression ratio corresponding to the development size; and a developing unit that decompresses the RAW image acquired by the acquiring unit based on the development size indicated by the size information acquired by the acquiring unit to generate a developed image having the development size, in The compression ratio increases as the development size decreases.

Images