Imaging control device, imaging device, and imaging control method

Abstract

To increase the number of consecutive shots and continuous imaging time without improving the performance of a circuit.SOLUTION: An imaging control device has: processing means 103-106, 108-110 that generate, from output from an image pick-up device that performs imaging, first image data to be temporarily recorded in non-volatile storage means 107; and control means 101 that controls first processing from the imaging to the temporary recording. The processing means perform second processing on the first image data read out from the non-volatile storage means to generate second image data for permanent recording. The control means controls the first processing so that the average writing rate of the first image data in the temporary recording becomes equal to or more than a rate based on an imaging rate of the image pick-up device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to control of an imaging device that performs continuous still image shooting and video shooting. [Background technology]

[0002] When still images are continuously shot and recorded on a recording medium, the speed and number of continuous shots may be limited due to insufficient image processing and compression processing speed and insufficient capacity of the intermediate buffer. If the circuit is made more powerful (larger in scale) to increase the processing speed or the capacity of the intermediate buffer, the power consumption and cost will increase. In addition, in the case of video shooting, the resolution and frame rate may be limited due to insufficient processing speed, and the continuous shooting time may be limited due to insufficient capacity of the intermediate buffer.

[0003] Patent Document 1 discloses an imaging device that compresses a RAW image and a reduced RAW image, records the compressed RAW image on a storage medium, reads the recorded RAW image and performs development processing in the background, and develops and displays the reduced RAW image when the development processing is not yet completed. This imaging device can relax the maximum development processing speed during continuous shooting. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-63252 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, Patent Document 1 does not disclose conditions such as the imaging rate (continuous shooting speed and frame rate), the write rate to the intermediate buffer, and the write compression rate, and therefore the number of continuous shots and continuous imaging time may not necessarily be improved.

[0006] The present invention provides an imaging control device and an imaging device that are capable of increasing the number of continuous shots and the continuous imaging time without increasing the performance of the circuit. [Means for solving the problem]

[0007] An imaging control device according to one aspect of the present invention includes a processing means for generating first image data for temporary recording in non-volatile storage means from an output of an imaging element that captures an image, and a control means for controlling a first process from imaging to temporary recording. The processing means generates second image data for main recording by performing a second process on the first image data read from the non-volatile storage means. The control means controls the first process so that an average write rate of the first image data in the temporary recording is equal to or greater than a speed based on the imaging rate of the imaging element.

[0008] An imaging control device according to another aspect of the present invention includes a processing means for generating first image data for temporary recording in non-volatile storage means from the output of an imaging element that captures an image, and a control means for controlling a first process from imaging to temporary recording. The processing means generates second image data for main recording by performing a second process on the first image data read from the non-volatile storage means. The control means controls the imaging rate of the imaging element based on an average write rate of the first image data in the temporary recording. An imaging device having the imaging control device also constitutes another aspect of the present invention.

[0009] According to another aspect of the present invention, there is provided an imaging control method, which includes the steps of generating first image data for temporary recording in non-volatile storage means from an output of an imaging element that captures an image, and generating second image data for main recording by performing a second processing on the first image data read from the non-volatile storage means. The method is characterized in that the first processing from imaging to temporary recording is controlled so that an average write rate of the first image data in the temporary recording is equal to or higher than a speed based on an imaging rate of the imaging element.

[0010] Furthermore, an imaging control method according to another aspect of the present invention includes a step of generating first image data for temporary recording in a non-volatile storage means from an output of an imaging element that captures an image, and a step of generating second image data for main recording by performing a second process on the first image data read from the non-volatile storage means. The method is characterized in that the imaging rate of the imaging element is controlled based on an average writing rate of the first image data in the temporary recording. Note that a program for causing a computer to execute a process according to the imaging control method also constitutes another aspect of the present invention. Effect of the Invention

[0011] According to the present invention, the number of continuous shots and the continuous imaging time can be increased without increasing the performance of the circuit. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing a temporary recording area, a main recording area, and a non-volatile storage medium in an embodiment. [Diagram 3] 11 is a flowchart showing a process for a user to specify a temporary recording area in the embodiment. [Figure 4] FIG. 11 is a diagram showing a demosaic process in the embodiment. [Diagram 5] FIG. 4 is a diagram showing the structure of a temporary recording file in the embodiment. [Figure 6] FIG. 4 is a diagram showing a temporarily recorded image file in the embodiment. [Figure 7] 5A to 5C are diagrams showing data compression utilizing correlation between images in an embodiment. [Figure 8] 4 is a flowchart showing an imaging process in the embodiment. [Figure 9] 6A to 6C are diagrams showing a process for setting a temporary recording area and a regular recording area in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0014] 1 shows the configuration of a digital still camera as an imaging device including an imaging control device according to an embodiment. The imaging control device according to the embodiment can be mounted not only on a digital still camera but also on various imaging devices such as a digital video camera, a smartphone, a camera phone, and a car-mounted camera. Furthermore, the imaging control device according to the embodiment can be provided as an external device (such as a personal computer) separate from a camera (imaging element).

[0015] The imaging device includes a control unit 101, a sensor unit 102, a sensor correction unit 103, a DRAM 104, a compression unit 105, an encryption unit 106, a non-volatile storage unit 107, a decryption unit 108, an expansion unit 109, a main image processing unit 110, a compression unit 111, a display image processing unit 112, and a display unit 113. The imaging device also includes a UI unit 114, and an AD conversion unit that performs AD conversion on the output from the imaging lens and the sensor unit 102 (not shown).

[0016] In the imaging device of this embodiment, image data for temporary recording (first image data) is generated from the output of the sensor unit 102 by the sensor correction unit 103, DRAM 104, compression unit 105 and encryption unit 106, and this is temporarily recorded in the nonvolatile storage unit 107. Then, image data for main recording (second image data) is generated from the image data read out from the nonvolatile storage unit 107 by the decoding unit 108, decompression unit 109, main image processing unit 110 and compression unit 111, and this is main recorded in the nonvolatile storage unit 107.

[0017] The control unit 101, the sensor correction unit 103, the DRAM 104, the compression unit 105, the encryption unit 106, the decryption unit 108, the decompression unit 109, the main image processing unit 110, the compression unit 111, and the display image processing unit 112 constitute an imaging control device. The control unit 101 corresponds to a control means. The sensor correction unit 103, the compression unit 105, and the encryption unit 106 constitute a first processing means. The processing from the imaging performed by the sensor unit 102 to the temporary recording performed by the first processing means corresponds to a first processing (processing for temporary recording). The decryption unit 108, the decompression unit 109, the main image processing unit 110, and the compression unit 111 constitute a second processing means, and the processing performed by these from reading out the image data from the non-volatile storage unit 107 to the main recording corresponds to a second processing (processing for main recording). The first processing means and the second processing means constitute a processing means.

[0018] The nonvolatile storage unit 107 may be detachable (insertable) from the imaging device (imaging control device), or may be built into the imaging control device. In the following description, nonvolatile is abbreviated as NV.

[0019] The control unit 101 is configured with a microcomputer including a CPU and the like, and issues various instructions and executes various processes to the above-mentioned sensor unit 102, display unit 113, imaging lens, AD converter, etc., in accordance with a program stored in the DRAM 104. The various processes include still image capturing process and video image capturing process.

[0020] The sensor unit 102 is configured with an imaging element such as a CCD sensor or a CMOS sensor that photoelectrically converts a subject image (optical image) formed by an imaging lens. The output (imaging signal) of the sensor unit 102 is input to a sensor correction unit 103.

[0021] The sensor correction unit 103 generates RAW image data from the output of the sensor unit 102, and further performs sensor correction such as pixel correction, black level correction, and shading correction on the RAW image data (original image data). The RAW image data that has undergone sensor correction is written to and temporarily stored in a DRAM 104, which is a volatile storage means.

[0022] Compression unit 105 performs a process of compressing the sensor-corrected image data read from DRAM 104 (hereinafter referred to as temporary compression) before temporarily recording the image data in NV storage unit 107. This temporary compression aims to save the temporary recording area of ​​NV storage unit 107 and reduce the required write rate of image data for temporary recording (first image data: hereinafter referred to as temporary recording data). Temporary recording refers to temporarily recording temporary recording data until image data for main recording (second image data: hereinafter referred to as main recording data) described later is recorded, and in this case NV storage unit 107 functions as a buffer.

[0023] If there is no need to save the temporary recording area or reduce the required write rate of the temporary recording data, the compression unit 105 does not need to be provided.

[0024] The sensor correction unit 103 described above is provided before the NV storage unit 107 for the purpose of making the temporarily recorded data sensor-corrected. However, if the temporarily recorded data does not need to be sensor-corrected, the sensor correction unit 103 may be provided between the decompression unit 109 and the main image processing unit 110.

[0025] The encryption unit 106 encrypts the temporarily recorded data that is temporarily recorded in the NV storage unit 107. The encryption unit 106 is provided to enhance the confidentiality of the image data recorded in the NV storage unit 107, but is not necessarily required.

[0026] The NV storage unit 107 is a non-volatile storage medium (non-volatile storage means) that stores the temporary recording data and the main recording data. The decryption unit 108 decrypts the temporary recording data encrypted by the encryption unit 106 and temporarily recorded in the NV storage unit 107. As described above, if the encryption unit 106 is not provided, the decryption unit 108 is not necessary.

[0027] The decompression unit 109 decompresses the temporarily recorded data temporarily compressed by the compression unit 105 into RAW image data by performing decompression processing on the temporarily recorded data. As described above, if the compression unit 105 is not provided, the decompression unit 109 is not necessary.

[0028] The main image processing unit 110 generates data for main recording by performing a process for main recording on the RAW image data from the decompression unit 109. The process for main recording includes various image processes other than the sensor correction performed by the sensor correction unit 103, such as noise reduction, white balance adjustment, magnification chromatic aberration correction, demosaic processing, gamma correction, brightness / color generation processing, geometric transformation, and enlargement / reduction. It is also possible to perform demosaic processing on the temporarily recorded data before the temporary recording. However, as will be described later, the amount of data of the temporarily recorded data increases as a result of demosaic processing compared to before demosaic processing, so it is preferable to perform demosaic processing on the RAW image data after temporary recording.

[0029] The compression unit 111 performs a main compression process such as JPEG compression (in the case of still image data) on the main recording data. The main recording data after the main compression process is written (mainly recorded) in the NV storage unit 107.

[0030] 1 shows a case where the temporary recording data and the actual recording data are stored in the same NV storage unit 107, but the temporary recording data and the actual recording data may be stored in separate NV storage units. This will be described in detail later. Furthermore, the size of the temporary recording area in the same NV storage unit 107 and the NV storage unit when the temporary recording data and the actual recording data are stored in separate NV storage units may be automatically managed by the control unit 101, or may be specified by the user as described later.

[0031] A display image processing unit 112 performs image processing for display (hereinafter, referred to as display image processing) on ​​the RAW image data read out from the DRAM 104.

[0032] The display unit 113 displays the display image data that has been subjected to the display image processing on a display device such as an electronic viewfinder or a rear panel (not shown). The user can check the image composition and the like while looking at the image displayed on the display unit 113. In this embodiment, the display image processing is performed on the RAW image data directly and displayed without going through the NV storage unit 107, so that the display image processing and display can be performed without depending on the access bandwidth limit of the NV storage unit 107.

[0033] The UI unit 114 is a user interface that enables the user to perform various operations such as power ON / OFF operation, image capture mode selection operation, image capture condition setting operation, image capture operation, image playback operation, etc. The UI unit 114 includes operation members such as button switches, a cross key, and a dial, and a touch sensor provided on the display unit 113 that displays menu images, etc. Through the UI unit 114, the user can specify the size of the temporary recording area of ​​the NV storage unit 107 and the NV storage unit in which the temporary recording data and the actual recording data are to be stored.

[0034] 2(a) and (b) show configuration examples of the NV storage unit 107. Fig. 2(a) shows an example in which one NV storage unit 107 is divided by physical or logical partitions into a temporary recording area 107a in which temporarily recorded data is stored and a main recording area 107b in which main recorded data is stored. As described above, the size of the temporary recording area is managed by the control unit 101 or specified by the user depending on the number of still images shot continuously (number of still image shots) and the continuous shooting time of a moving image (moving image shooting time).

[0035] It is not necessary to clearly divide the NV storage unit 107 into a temporary recording area and a main recording area. In this case, the size of the temporary recording area is logically managed by the control unit 101, but the size and number of the temporary recording areas may be dynamically changed in the NV storage unit 107. However, by clearly dividing the temporary recording area and the main recording area as shown in FIG. 2(a), the size of the temporary recording area can be secured as a fixed amount compared to the case where the areas are not divided. This makes it possible to secure a certain number of continuous shots of still images and a certain amount of continuous shooting time of videos, and makes it easier to avoid problems such as accidentally deleting temporary recording data due to a user's erroneous operation.

[0036] FIG. 2(b) shows an example in which the NV storage unit 107A for the temporary recording area and the NV storage unit 107B for the main recording area are provided separately. In this example, compared to the case in which a temporary recording area and a main recording area are provided in one NV storage unit 107, the two NV storage units 107A and 107B can be accessed simultaneously in parallel. This makes it possible to ensure high write and read rates for the temporary recording data relative to the upper limit of the access bandwidth of the NV storage unit 107A. The temporary recording area of ​​the NV storage unit 107A may be divided into multiple areas, and the sizes of these areas may be managed by the control unit 101.

[0037] The flowchart in Fig. 3 shows the process in which the user selects the NV storage unit in which to provide a temporary recording area and specifies the size of the temporary recording area. "S" stands for step. When a menu related to the temporary recording area is selected by the user, the control unit 101 starts this process.

[0038] First, in S301, control unit 101 displays information about NV storage unit 107 in which a temporary recording area can be provided, on display unit 113. The displayed information is, for example, information identifying NV storage unit 107, such as whether NV storage unit 107 is a built-in memory or a memory inserted in a media slot, or in the case of multiple media slots, which slot the memory is inserted in, and information such as the storage capacity of NV storage unit 107.

[0039] Next, in S302, the control unit 101 prompts the user to specify the NV storage unit 107 in which the temporary storage area is to be allocated.

[0040] Next, in S303, the control unit 101 displays size information of the temporary recording area in the NV storage unit 107, for example, size information when the maximum amount of the temporary recording area is secured, on the display unit 113. The unit of the size information can be selected from the number of bytes, the number of recordable still images, continuous image capture time, etc.

[0041] Next, in S305, the control unit 101 determines whether or not the user has performed an operation to change (select) the unit of the size information, and if the user has performed a change operation, proceeds to S304, and if the user has not performed a change operation, proceeds to S306.

[0042] In S304, the control unit 101 changes the unit of the size information to the unit selected by the user, and displays the size information of the temporary recording area on the display unit 113 again in S303.

[0043] In S306, the control unit 101 prompts the user to specify the size of the temporary recording area. Size information indicating the specified size is displayed on the display unit 113 in the unit set at that time.

[0044] Next, in S307, the control unit 101 secures a temporary recording area of ​​a size designated by the user in the NV storage unit 107 designated by the user, and then ends this process.

[0045] Next, the average write rate WR of the temporarily recorded data to the NV storage unit 107, the imaging rate SR of the sensor unit 102, the temporary recorded data size IS which is the data size of the temporarily recorded data (image data), and the temporary compression rate CR which is the compression rate of the compression unit 105 will be described. The temporary recorded data size IS is the size of the image data before temporary compression by the compression unit 105. The imaging rate SR is the average imaging rate during continuous still image shooting or the frame rate during video shooting. Hereinafter, continuous still image shooting and video shooting will be collectively referred to as continuous shooting.

[0046] The relationship between the average writing rate (hereinafter simply referred to as the writing rate) WR [Byte / sec], the imaging rate SR [FPS], the temporary recording data size IS [Byte], and the temporary compression rate CR (1 / 2, 1 / 4, 1 / 5, 0.23, etc.) is expressed by the following equation (1).

[0047]

number

[0048] The control unit 101 controls the first process from image capture to temporary recording so as to obtain a writing rate WR at which the formula (1) holds, that is, a writing rate WR equal to or higher than a speed determined based on the image capture rate SR, the temporary recording data size IS, and the temporary compression rate CR. When the formula (1) holds, even if continuous image capture is performed at the image capture rate SR, the temporary recording data can be continuously written until the temporary recording area of ​​the NV storage unit 107 becomes full. In this case, the sensor correction unit 103, the compression unit 105, and the encryption unit 106 need to have processing performance equal to or higher than the image capture rate SR.

[0049] In contrast, the processing after writing the temporarily recorded data, that is, reading the temporarily recorded data from the NV storage unit 107, the actual image processing, the actual compression, and other main recording processing, does not require processing performance equal to or higher than the image capture rate SR. Therefore, it is possible to perform processing at a predetermined (arbitrary) rate or timing that is independent of the image capture rate SR.

[0050] The following equation (2) is obtained from equation (1).

[0051]

number

[0052] That is, the temporary compression rate CR that allows continuous imaging until the temporary recording area becomes full can be set based on the imaging rate SR, the temporary recording data size IS, and the writing rate WR. For example, it is possible to control the setting of the temporary compression rate CR according to the continuous shooting speed (imaging rate SR), or to set the temporary compression rate CR after checking the writing rate WR of the removable NV storage unit 107.

[0053] Furthermore, the following equation (3) can be obtained from equation (1).

[0054]

number

[0055] That is, the image capture rate SR at which continuous image capture is possible until the temporary recording area becomes full can be set based on the temporary recording data size IS, the temporary compression rate CR, and the writing rate WR. For example, it is possible to control the maximum continuous shooting speed (maximum image capture rate SR) to be set by the temporary compression rate CR at which a certain image quality can be guaranteed, or to set the maximum continuous shooting speed after checking the writing rate WR of the removable NV storage unit 107.

[0056] On the other hand, under the condition that continuous shooting is not possible until the temporary recording area becomes full, the relationship between the number of continuous shots BN, the continuous shooting time BT [sec], the buffer size BS [Byte] of the DRAM 104, the write rate WR, the imaging rate SR, the temporary recording data size IS, and the temporary compression rate CR is expressed by the following equations (4) to (7).

[0057]

number

[0058]

number

[0059]

number

[0060]

number

[0061] When formula (4) is satisfied, even if continuous imaging is performed at the imaging rate SR, the temporary recording data cannot be written continuously until the temporary recording area becomes full. In this case, the maximum continuous imaging time BT can be calculated from the buffer size BS, the temporary recording data size IS, the temporary compression rate CR, the imaging rate SR, and the writing rate WR, as shown in formula (5).

[0062] From the relationship between the number of continuous shots BN and the continuous image capture time BT shown in equation (7), the number of continuous shots BN is expressed by equation (6) from equation (5), and can be calculated from the buffer size BS, the temporary recording data size IS, the temporary compression rate CR, the image capture rate SR, and the write rate WR. Note that the sensor correction unit 103, the compression unit 105, and the encryption unit 106 need to have processing performance equal to or higher than the image capture rate SR.

[0063] In contrast, the processing after writing the temporarily recorded data, that is, reading the temporarily recorded data from the NV storage unit 107, the actual image processing, the actual compression, and other main recording processing, does not require processing performance equal to or higher than the image capture rate SR. Therefore, it is possible to perform processing at a predetermined (arbitrary) rate or timing that is independent of the image capture rate SR.

[0064] From equations (5) and (6), the following equations (8) and (9) are obtained.

[0065]

number

[0066]

number

[0067] That is, the temporary compression rate CR that allows continuous imaging until the continuous imaging time reaches BT can be set based on the buffer size BS, the imaging rate SR, the temporary recording data size IS, and the writing rate WR. For example, it is possible to control the temporary compression rate CR by setting it based on the continuous shooting speed (imaging rate SR), by checking the writing rate WR of the removable NV storage unit 107, or by setting it based on the continuous imaging time BT specified by the user.

[0068] Also, a temporary compression rate CR that allows still images to be continuously shot until the number of continuous shots reaches BN can be set based on the buffer size BS, the image capture rate SR, the temporary recording data size IS, and the writing rate WR. For example, it is possible to control the temporary compression rate CR by setting it based on the continuous shooting speed (image capture rate SR), by checking the writing rate WR of the removable NV storage unit 107, or by setting it based on the number of continuous shots BN specified by the user.

[0069] Furthermore, the following equations (10) and (11) are obtained from equations (5) and (6).

[0070]

number

[0071]

number

[0072] That is, the maximum continuous shooting speed (image capture rate SR) at which continuous shooting can be performed until the continuous image capture time reaches BT can be set based on the buffer size BS, the temporary recording data size IS, the temporary compression rate CR, and the writing rate WR. For example, it is possible to control the maximum continuous shooting speed (image capture rate SR) by the temporary compression rate CR that can guarantee a certain image quality, to set the maximum continuous shooting speed after checking the writing rate WR of the removable NV storage unit 107, or to set the maximum continuous shooting speed by the user-specified continuous image capture time BT.

[0073] In addition, the maximum continuous shooting speed (image capture rate SR) at which continuous shooting can be performed until the number of continuous shots reaches BN can be set based on the buffer size BS, the temporary recording data size IS, the temporary compression rate CR, and the writing rate WR. For example, it is possible to control the maximum continuous shooting speed by setting the temporary compression rate CR at which a certain image quality can be guaranteed, by checking the writing rate WR of the removable NV storage unit 107, or by setting the maximum continuous shooting speed based on the number of continuous shots BN specified by the user.

[0074] Next, a case will be described in which the access rate AR [Byte / sec] of the NV storage unit 107 has a margin for the temporary recording data size IS, the temporary compression rate CR, and the image capture rate SR, as shown in the following formula (12).

[0075]

number

[0076] In this case, since the access rate AR of the NV storage unit 107 has a sufficient margin even during continuous image capture, main image processing, main compression, and main recording can be performed at a processing rate according to the read rate RR [Byte / sec] of the temporarily recorded data while continuous image capture is being performed.

[0077] When the NV storage unit 107A used for the temporarily recorded data and the NV storage unit 107B used for the actual recorded data are different as shown in FIG. 2(b), the read rate RR of the temporarily recorded data can be expressed by the following formula (13) based on formula (12).

[0078]

number

[0079] At this time, the maximum read rate RR of the temporary recording data can be set based on the maximum access rate AR of the temporary recording data, the temporary recording data size IS, the temporary compression rate CR, and the image capture rate SR. For example, it is possible to control the process rate of the main image processing and the main compression by the temporary compression rate CR that can guarantee a certain image quality, or to set the process rate of the main image processing and the main compression after checking the access rate AR of the removable NV storage unit 107A.

[0080] Here, the image processing unit 110 and the compression unit 111 are assumed to have processing performance higher than the read rate RR, and the writing performance of the recording data to the NV memory unit 107B is assumed to be high enough not to limit the read rate RR.

[0081] As shown in Fig. 2(a), when the NV storage unit 107 used for the temporary recording data and the actual recording data is the same, the read rate RR of the temporary recording data can be expressed by the following formula (14) from formula (12). FIS [Byte] is the size of the image data which is the actual recording data (hereinafter referred to as the actual recording data size), and FCR is the compression rate of the actual compression by the compression unit 111 (hereinafter referred to as the actual compression rate). The actual recording data size FIS is the size of the actual recording image data before the actual compression by the compression unit 111. The actual compression rate FCR is 1 / 8, 1 / 10, 1 / 16, 0.07, etc.

[0082]

number

[0083] At this time, the maximum read rate RR of the temporary record data can be set based on the maximum access rate AR of the temporary record data, the temporary record data size IS, the temporary compression rate CR, the imaging rate SR, the regular record data size and the regular compression rate FCR.

[0084] Also, assume that the main image processing unit 110 and the compression unit 111 have a processing performance higher than the read rate RR. In this case, it is possible to control the processing rates of the main image processing and main compression by, for example, a temporary compression rate CR that can guarantee a certain image quality, or to set the processing rates of the main image processing and main compression after checking the access rate AR of the removable NV storage unit 107.

[0085] As a result, it is possible to eliminate the dependency of the write rate and read rate of the temporarily recorded data on the NV storage unit 107 on the image capture rate, the number of continuous shots, or the continuous image capture time. As a result, there is less need to increase the processing rate in the main image processing and main compression after the temporarily recorded data is read from the NV storage unit 107, and it is possible to suppress the increase in performance (large scale) of circuits including the main image processing unit 110 and compression unit 111 and the associated increase in power consumption.

[0086] Fig. 4 shows an example of demosaic processing. Reference numeral 401 denotes image data in a Bayer array (hereinafter referred to as a Bayer image). In odd-numbered rows such as row 1 and row 3, red (R) and green (G) pixels are repeatedly arranged. In even-numbered rows such as row 2 and row 4, G and blue (B) pixels are repeatedly arranged.

[0087] Reference numeral 402 indicates the interpolation process of the R pixel value. In the odd-numbered rows of the Bayer image, an R pixel is arranged at every other pixel, and these R pixel values ​​are output as is. In addition, the R pixel value at the position of a G pixel in the Bayer image is calculated from the average of the pixel values ​​of the two R pixels to the left and right or above and below the G pixel. Furthermore, the R pixel value at the position of a B pixel in the Bayer image is calculated from the average of the pixel values ​​of the four R pixels at the top left, top right, bottom left, and bottom right of the B pixel.

[0088] Reference numeral 403 denotes an interpolation process for G pixel values. The pixel values ​​of the G pixels of the Bayer image are output as is. The G pixel values ​​at the positions of the R and B pixels of the Bayer image are calculated from the average of the pixel values ​​of the G pixels to the left and right of the R pixel.

[0089] 404 shows the interpolation process of B pixel values. In the even rows of the Bayer image, B pixels are arranged every other pixel, and these B pixel values ​​are output as is. In addition, the B pixel value at the position of a G pixel in the Bayer image is calculated from the average of the pixel values ​​of the two B pixels to the left and right or above and below the G pixel. Furthermore, the B pixel value at the position of an R pixel in the Bayer image is calculated from the average of the pixel values ​​of the four B pixels located at the top left, top right, bottom left, and bottom right of the R pixel.

[0090] Reference numeral 405 denotes output image data consisting of R image data, G image data and B image data after the above-mentioned interpolation processes (402, 403, 404).

[0091] In this way, the data size of the output image data 405 after demosaic processing is three times larger than that of the Bayer image 401 before demosaic processing.

[0092] 5(a) to 5(c) will be used to explain the format in which the temporarily recorded data is stored in the NV storage unit 107. Here, an example will be explained in which the temporarily recorded data is stored in the NV storage unit 107 in a file format.

[0093] 5(a) shows a case where imaging information 502 and RAW image data 503 are stored in one temporary recording file 501. The imaging information 502 includes various information related to imaging for obtaining the RAW image data 503, such as an imaging mode and parameters at the time of imaging. The imaging information 502 is attached to the RAW image data 503 as information required for setting parameters when performing the main image processing and main compression. The RAW image data 503 is captured image data that has been subjected to sensor correction, compression, and encryption and has not yet been subjected to the main recording processing.

[0094] In this way, the imaging information 502 is recorded in the NV storage unit 107 in association with the RAW image data 503. As a result, even if an incident occurs in which the main image processing or main compression is interrupted due to a power interruption or the like and data being processed is lost, the main image processing, main compression, and main recording can be resumed and completed from the RAW image data 503 and the imaging information 502. As a result, even in the event of a low battery or an unintended power interruption by the user, the risk of image data loss can be minimized.

[0095] Fig. 5(b) shows a case where a temporary recording file (first area) 504 in which imaging information storage destination information 505 and RAW image data 506 are stored, and a temporary recording file (second area) 507 in which imaging information 508 is stored are provided separately. The storage destination information 505 includes file information (information indicating the second area) such as path information to the temporary recording file 507 and the file name of the temporary recording file 507. The RAW image data 506 is the same as the RAW image data 503 shown in Fig. 5(a). The imaging information 508 is also the same as the imaging information 502 shown in Fig. 5(a).

[0096] In this case as well, the imaging information 508 is recorded in the NV storage unit 107 in association with the RAW image data 506. As a result, even if an incident occurs in which the main image processing or main compression is interrupted due to a power interruption or the like and data being processed is lost, the main image processing, main compression, and main recording can be resumed and completed from the RAW image data 506 and the imaging information 508. As a result, even in the event of a low battery or an unintended power interruption by the user, the risk of image data loss can be minimized.

[0097] Fig. 5(c) shows a case where a temporary recording file (first area) 509 in which storage destination information 510 of RAW image data and imaging information 508 are stored, and a temporary recording file (second area) 512 in which RAW image data 513 is stored are provided separately. The storage destination information 510 includes file information (information indicating the second area) such as path information to the temporary recording file 512 and the file name of the temporary recording file 512. The imaging information 511 is the same as the imaging information 502 shown in Fig. 5(a). The RAW image data 513 is also the same as the RAW image data 503 shown in Fig. 5(a).

[0098] In this case as well, the imaging information 511 is recorded in the NV storage unit 107 in association with the RAW image data 513. As a result, even if an incident occurs in which the main image processing or main compression is interrupted due to a power interruption or the like and data being processed is lost, the main image processing, main compression, and main recording can be resumed and completed from the RAW image data 513 and the imaging information 511. As a result, even in the event of a low battery or an unintended power interruption by the user, the risk of image data loss can be minimized.

[0099] Although an example has been shown in which imaging information is managed and stored in association with RAW image data, imaging information relating to a plurality of pieces of RAW image data may be collectively managed in a dedicated management file on the NV storage unit 107 .

[0100] In addition, although the RAW image data, which is the temporarily recorded data, is the captured image data before the main image processing that has been subjected to sensor correction, compression and encryption, the RAW image data is not limited to this. For example, it is sufficient if the image processing that has been completed before the RAW image data is temporarily recorded as the temporarily recorded data in association with the imaging information and the image processing that has not been completed after that are clear. In other words, even if the main image processing or the main compression is interrupted due to a power cut or the like, and the data being processed is lost, it is sufficient if the uncompleted image processing can be resumed. However, if the captured image data is Bayer array image data, it is better to perform demosaic processing in the main image processing to reduce the data size of the temporarily recorded data and the usage of the temporary recording area.

[0101] Also, although the above description has been given of the case where the temporary recording data is stored in the NV memory unit 107 in a file format, the temporary recording data does not necessarily have to be in a file format, and the temporary recording data may be stored in the NV memory unit 107 in another storage format or management format.

[0102] Fig. 6 shows a case where captured image data (temporary recording data) as a plurality of pieces of RAW image data obtained by continuous still image shooting is not stored in individual temporary recording files, but is collectively stored in one temporary recording file and recorded in the NV storage unit 107. Fig. 6 shows the passage of time t from left to right. S2 shows the timing at which still image shooting is performed. In the first two S2s, single shooting (single shooting) 601, 602 is performed, and in the third S2, continuous shooting 603 is performed.

[0103] In the single shot 601 , one piece of RAW image data is stored in a temporary recording file 605 , and in the single shot 602 , one piece of RAW image data is stored in another temporary recording file 606 , and both are recorded in the NV storage unit 107 .

[0104] On the other hand, when continuous shooting 603 starts, four pieces of RAW image data are collectively stored in a temporary recording file 607, and further four pieces of RAW image data from the middle of continuous shooting 604 are collectively stored in another temporary recording file 608, and each of these is recorded in the NV storage unit 107. In this manner, an arrangement may be made in which a predetermined number of pieces of RAW image data are stored in one temporary recording file, or an arrangement may be made in which a plurality of pieces of RAW image data generated within a predetermined continuous time are stored in one temporary recording file.

[0105] Fig. 7 shows an example of a method for reducing data to be recorded using correlation between captured image data in the continuous shooting 603 (604) in Fig. 6. 701 to 705 show one still image capture during continuous shooting. In each of the still image captures 701 to 705, images 1 to 5 are obtained as captured image data.

[0106] As the data to be recorded, a first image 1 (711) obtained in the first imaging, which is basic image data, is selected, and difference data (n=1 to 5) 712 to 715 between image (n+1) and image n are generated. The difference data may be between captured image data before sensor correction, or between captured image data after sensor correction. By using the difference data of captured image data as the data to be recorded in this way, the redundancy of the data to be recorded can be reduced and compressed efficiently, thereby reducing the data size of the temporary recording data, compared to when all captured image data is used as the data to be recorded.

[0107] Here, the case where the data to be recorded is reduced by utilizing the correlation between images acquired by capturing consecutive (preceding and succeeding) still images has been described, but the correlation between images other than those acquired by capturing preceding and succeeding images may be utilized, such as the correlation between the first image and another image. Furthermore, in addition to the difference between images, the movement of the subject in a common area between the images may be detected, and the amount of that movement may be extracted to increase the compression efficiency.

[0108] Although continuous still image shooting has been described as an example here, temporary recording can also be performed in the same manner when shooting moving images.

[0109] 7 also shows a case where the time interval between image capture 704 and image capture 705 is different from the time interval between other images. Even when the time interval between images changes in this way, as long as the width of the change is within a predetermined time, compression using the correlation between images may be performed or the images may be stored together in one temporary recording file.

[0110] In this way, by storing multiple captured image data together in a single temporary recording file based on a predetermined standard, the overhead of file creation is reduced, enabling efficient processing, and also increasing the flexibility when performing the actual image processing, compression, and recording.

[0111] The flowcharts in FIGS. 8(a) and 8(b) show the process executed by the control unit 101 for acquiring and releasing a temporary recording area.

[0112] The flowchart in FIG. 8(a) shows the imaging process.

[0113] In S801, the control unit 101 determines whether or not the user has performed an imaging operation (SW2 operation) on the UI unit 114, and if the SW2 operation has not been performed, repeats this determination, and if the SW2 operation has been performed, starts imaging (continuous still image shooting or video shooting) and proceeds to S802.

[0114] In S802, the control unit 101 acquires a temporary recording area in the NV storage unit 107.

[0115] In S803, the control unit 101 temporarily records the captured image data (temporarily recorded data) in the temporary recording area acquired in S802, and then the process returns to S801.

[0116] The flowchart in FIG. 8B shows a process for performing main recording after the temporary recording of the temporary recording data and releasing the temporary recording area.

[0117] In S811, the control unit 101 determines whether or not temporarily recorded data exists in the NV storage unit 107. If temporarily recorded data does not exist, the process returns to S811, and if temporarily recorded data exists, the process proceeds to S812.

[0118] In S812, the control unit 101 reads out the temporary recording data from the NV memory unit 107, causes the main image processing unit 110 to perform main image processing on the temporary recording data to generate main recording data, and further causes the compression unit 111 to perform main compression on the main recording data.

[0119] Next, in S813, the control unit 101 actually records the actual compressed image data to the actual recording area of ​​the NV storage unit 107.

[0120] Next, in S814, the control unit 101 deletes the temporary recording data corresponding to the actually recorded data (the data from which the actually recorded data was created) from the NV storage unit 107, and releases the temporary recording area in the NV storage unit 107. Then, the process returns to step S811.

[0121] The above imaging process ensures that the captured image data is always temporarily or permanently recorded in the NV storage unit 107. This reduces the risk of the captured image data being lost in the event of a low battery or an unintended power outage by the user.

[0122] Note that, due to physical characteristics, some NV storage units 107 have a limit to the number of times data can be written. In such NV storage units 107, if the writing frequency in a biased portion of the recording area becomes high, there is a risk that only that recording area will become unusable. To avoid this, it is preferable to control the writing frequency for each recording area so that it is as uniform as possible (in particular, so that the recording frequency of temporary recording data in multiple temporary recording areas is averaged).

[0123] 9 shows the process of setting a temporary recording area and a regular recording area in the NV storage unit 107. In steps S901 to S912, recording areas for temporary recording data and regular recording data in the NV storage unit 107 are set in sequence.

[0124] In S901, the control unit 101 records the first obtained temporary recording data 1 in the first temporary recording area. At this time, a recording area is left above the first temporary recording area so that the main recording data can be recorded.

[0125] In S902, the control unit 101 records the main record data 1 generated by performing the main image processing and the main compression on the temporary record data 1 in the recording area left vacant in S901.

[0126] In S903, the control unit 101 deletes the temporary recording data 1, and makes the recording area in which it was recorded free area.

[0127] In S904, the control unit 101 maintains the free area of ​​S903 as it is, and records the next acquired temporary recording data 2 in the second temporary recording area below the free area.

[0128] In S905, the control unit 101 records the main record data 2 generated by performing the main image processing and the main compression on the temporary record data 2 in the free space maintained in S904. That is, the newly generated free space is used preferentially as the recording area for the main record data.

[0129] In S906, the control unit 101 deletes the temporary recording data 2, and makes the recording area in which it was recorded free area.

[0130] In S907, the control unit 101 maintains the free area of ​​S906 as it is, and records the next acquired temporary recording data 3 in the third temporary recording area below the free area.

[0131] In S908, the control unit 101 records the main record data 3 generated by performing the main image processing and the main compression on the temporary record data 3 in the free area maintained in S907.

[0132] In S909, the control unit 101 deletes the temporary recording data 3, and makes the recording area where it was recorded free area.

[0133] In S910, the control unit 101 maintains the free area of ​​S909 as it is, and records the next acquired temporary recording data 4 in the fourth temporary recording area below the free area.

[0134] In S911, the control unit 101 records the main record data 4, which has been generated by performing the main image processing and the main compression on the temporary record data 4, in the free area maintained in S910.

[0135] In S912, the control unit 101 deletes the temporary recording data 4, and makes the recording area in which it was recorded free area.

[0136] By using the recording areas as described above, it is possible to prevent the writing frequency from being biased toward some of the recording areas in the NV storage unit 107, and to equalize the writing frequency for each recording area.

[0137] In addition, in Fig. 9, the temporary recording data and the actual recording data are shown to have the same data size, but if they have different data sizes, it is preferable to set the recording area according to the larger data size. Also, it is sufficient that the above-mentioned processing can be performed on the logical recording area, and the association between the physical recording area setting and the logical recording area setting is managed separately. Therefore, the physical recording area does not need to be linked to the recording area setting shown in Fig. 9.

[0138] In addition, FIG. 9 shows a case where the imaging rate and the main image processing and compression are executed at the same processing rate, but a similar recording area can also be set when the imaging rate is high and imaging and writing of temporary recording data are performed consecutively and in advance.

[0139] Also, unlike the example of FIG. 9, other methods may be used, such as managing the number of times writing (the number of times recording) is performed on a plurality of recording areas, and a recording area with a smaller number of times writing is preferentially assigned as a temporary recording area.

[0140] As described above, according to the above embodiment, the number of consecutive shots and the continuous image capturing time can be increased without increasing the performance of the circuit.

[0141] The above embodiment includes the following configurations.

[0142] (Configuration 1) A processing means for generating first image data for temporary recording in a non-volatile storage means from an output of an image pickup element that captures an image; a control means for controlling a first process from the imaging to the temporary recording, the processing means performs a second processing on the first image data read from the non-volatile storage means to generate second image data for main recording; The imaging control device, wherein the control means controls the first process so that an average writing rate of the first image data in the temporary recording is equal to or higher than a speed based on an imaging rate of the imaging element. (Configuration 2) 2. The imaging control device according to configuration 1, wherein the speed based on the imaging rate is a speed obtained from the imaging rate, a size of the first image data, and a compression rate when the first image data is compressed. (Configuration 3) 3. The imaging control device according to configuration 1 or 2, wherein the imaging rate is a maximum imaging rate or an average imaging rate. (Configuration 4) A processing means for generating first image data for temporary recording in a non-volatile storage means from an output of an image pickup element that captures an image; a control means for controlling a first process from the imaging to the temporary recording, the processing means performs a second processing on the first image data read from the non-volatile storage means to generate second image data for main recording; The imaging control device, wherein the control means controls an imaging rate of the imaging element based on an average writing rate of the first image data in the temporary recording. (Configuration 5) 5. The imaging control device according to configuration 4, wherein the imaging rate is a maximum imaging rate or an average imaging rate. (Configuration 6) 6. The imaging control device according to any one of configurations 1 to 5, wherein the second processing includes a demosaic processing. (Configuration 7) The control means providing an area for temporary recording and an area for main recording in said non-volatile storage means; 7. The imaging control device according to any one of configurations 1 to 6, wherein the size of the area for temporary recording is set based on a user's designation. (Configuration 8) 8. The imaging control device according to claim 7, wherein the user designates the number of still image continuous shots or the video imaging time. (Configuration 9) The imaging control device according to any one of configurations 1 to 8, characterized in that the control means performs the temporary recording by attaching, to the first image data, information regarding the imaging when the first image data is generated. (Configuration 10) The control means providing a first area and a second area for the temporary recording in the non-volatile storage means; recording one of the first image data and information related to the imaging when generating the first image data in the first area, and recording the other in the second area; 11. The imaging control device according to any one of configurations 1 to 10, wherein information indicating the second area is recorded in the first area. (Configuration 11) 11. The imaging control device according to any one of configurations 1 to 10, wherein the first processing includes at least one of sensor correction, compression, and encryption. (Configuration 12) the first processing includes at least two of sensor correction, compression, and encryption; An imaging control device as described in any one of configurations 1 to 11, characterized in that when the first processing includes the sensor correction, the sensor correction is performed before other processing included in the first processing, and when the first processing includes the encryption, the encryption is performed after the other processing included in the first processing. (Configuration 13) the first processing includes compression; 13. The imaging control device according to any one of configurations 1 to 12, wherein the control means sets a compression rate in the compression based on a size of the first image data, the average writing rate, and the imaging rate. (Configuration 14) the first process further includes temporarily storing the information in a volatile storage means before the temporary recording; The imaging control device according to configuration 13, characterized in that the control means sets a compression rate for the compression based on the size of the first image data, the average writing rate, the imaging rate, and the number of continuous still image captures specified by a user. (Configuration 15) the first processing includes compression; 15. The imaging control device according to any one of configurations 1 to 14, wherein the control means sets the imaging rate based on a compression ratio in the compression, a size of the first image data, and the average writing rate. (Configuration 16) the first process further includes temporarily storing the information in a volatile storage means before the temporary recording; The imaging control device according to configuration 15, characterized in that the control means sets the imaging rate based on a compression ratio in the compression, a size of the first image data, the average writing rate, and a number of continuous still image shootings specified by a user. (Configuration 17) the second processing includes decompressing the first image data that has been compressed and then temporarily stored in the non-volatile storage means, performing a main image processing on the decompressed first image data, and storing the second image data generated by the main image processing in the non-volatile storage means, The imaging control device described in any one of configurations 1 to 15, characterized in that the control means sets the read rate of the first image data from the non-volatile storage means, the processing rate of the main image processing, and the average write rate of the second image data to the non-volatile storage means based on the size of the first image data, the average write rate in the temporary recording, the imaging rate, and the compression rate in the compression. (Configuration 18) the second processing includes decompressing the first image data that has been compressed and temporarily stored in the nonvolatile storage means after being read out, performing a main image processing on the decompressed first image data, and recording the second image data generated by the main image processing in a nonvolatile storage means different from the nonvolatile storage means, The imaging control device described in any one of configurations 1 to 15, characterized in that the control means sets the read rate of the first image data from the non-volatile storage means and the processing rate of the main image processing based on the size of the first image data, the average write rate in the temporary recording, the imaging rate, and the compression rate in the compression. (Configuration 19) An imaging control device according to any one of configurations 1 to 18, characterized in that in the first processing, a predetermined number of the first image data generated by continuous shooting of still images or a plurality of the first image data generated within a predetermined continuous time in continuous shooting of still images is stored in one temporary recording file to perform the temporary recording. (Configuration 20) the first processing includes compression; 20. The imaging control device according to configuration 19, wherein the compression of the plurality of first image data is performed by utilizing correlation between the plurality of first image data. (Configuration 21) An imaging control device as described in any one of configurations 1 to 20, characterized in that the first image data on which the temporary recording has been performed is deleted from the non-volatile storage means after the main recording of the second image data corresponding to the first image data has been performed. (Configuration 22) An imaging control device as described in any one of configurations 1 to 21, characterized in that in the first processing, the recording frequency of the first image data in multiple temporary recording areas in the non-volatile storage means is averaged. (Configuration 23) The imaging control device according to configuration 22, characterized in that in the first processing, an area among the plurality of temporary recording areas that has been recorded on less frequently than the others is used preferentially as an area for temporary recording of the first image data. (Configuration 24) The imaging control device described in configuration 21 is characterized in that an area in which the first image data corresponding to the second image data for which the actual recording has been completed was recorded is made free space by deleting the first image data, and the free space is used preferentially as an area for the actual recording of other second image data. (Configuration 25) An imaging control device according to any one of configurations 1 to 24; An imaging device comprising the imaging element. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0143] The embodiments described above are merely representative examples, and various modifications and alterations are possible for each embodiment when implementing the present invention. [Explanation of symbols]

[0144] 101 Control section 102 Sensors 107 Non-volatile memory unit 110 Image Processing Unit 111 Compression section

Claims

1. a processing means for generating first image data for temporary recording in a non-volatile storage means from an output of an image pickup element that captures an image; a control unit that controls a first process from the imaging to the temporary recording, the processing means performs a second processing on the first image data read from the non-volatile storage means to generate second image data for main recording; The imaging control device according to claim 1, wherein the control means controls the first process so that an average writing rate of the first image data in the temporary recording is equal to or higher than a speed based on an imaging rate of the imaging element.

2. 2. The imaging control device according to claim 1, wherein the speed based on the imaging rate is a speed obtained from the imaging rate, a size of the first image data, and a compression rate when the first image data is compressed.

3. 2. The imaging control device according to claim 1, wherein the imaging rate is a maximum imaging rate or an average imaging rate.

4. a processing means for generating first image data for temporary recording in a non-volatile storage means from an output of an image pickup element that captures an image; a control unit that controls a first process from the imaging to the temporary recording, the processing means performs a second processing on the first image data read from the non-volatile storage means to generate second image data for main recording; The imaging control device according to claim 1, wherein the control means controls an imaging rate of the imaging element based on an average writing rate of the first image data in the temporary recording.

5. 5. The imaging control device according to claim 4, wherein the imaging rate is a maximum imaging rate or an average imaging rate.

6. 5. The imaging control device according to claim 1, wherein the second processing includes a demosaic processing.

7. The control means providing an area for temporary recording and an area for main recording in said non-volatile storage means; 5. The imaging control device according to claim 1, wherein the size of the area for temporary recording is set based on a user's designation.

8. 8. The image capture control device according to claim 7, wherein the user designates the number of still image continuous shots or the moving image capture time.

9. 5. The imaging control device according to claim 1, wherein the control means performs the temporary recording by adding, to the first image data, information relating to the imaging performed when the first image data was generated.

10. The control means providing a first area and a second area for the temporary recording in the non-volatile storage means; one of the first image data and information related to the imaging when generating the first image data is recorded in the first area, and the other of the first image data and information related to the imaging when generating the first image data is recorded in the second area; 5. The imaging control device according to claim 1, wherein information indicating the second area is recorded in the first area.

11. 5. The imaging control device according to claim 1, wherein the first processing includes at least one of a sensor correction, a compression, and an encryption.

12. the first processing includes at least two of sensor correction, compression, and encryption; An imaging control device as described in claim 1 or 4, characterized in that when the first processing includes the sensor correction, the sensor correction is performed before other processing included in the first processing, and when the first processing includes the encryption, the encryption is performed after the other processing included in the first processing.

13. the first processing includes compression; 5. The imaging control device according to claim 1, wherein the control means sets a compression rate in the compression based on a size of the first image data, the average writing rate, and the imaging rate.

14. the first process further includes temporarily storing the information in a volatile storage means before the temporary recording; 14. The imaging control device according to claim 13, wherein the control means sets a compression rate in the compression based on a size of the first image data, the average writing rate, the imaging rate, and a number of continuous still image captures designated by a user.

15. the first processing includes compression; 5. The imaging control device according to claim 1, wherein the control means sets the imaging rate based on a compression ratio in the compression, a size of the first image data, and the average writing rate.

16. the first process further includes temporarily storing the information in a volatile storage means before the temporary recording; 16. The imaging control device according to claim 15, wherein the control means sets the imaging rate based on a compression ratio in the compression, a size of the first image data, the average writing rate, and a number of continuous still image captures designated by a user.

17. the second processing includes decompressing the first image data that has been compressed and then temporarily stored in the non-volatile storage means, performing a main image processing on the decompressed first image data, and storing the second image data generated by the main image processing in the non-volatile storage means, 5. The imaging control device according to claim 1, wherein the control means sets a read rate of the first image data from the non-volatile storage means, a processing rate of the main image processing, and an average write rate of the second image data to the non-volatile storage means based on a size of the first image data, the average write rate in the temporary recording, the imaging rate, and a compression rate in the compression.

18. the second processing includes decompressing the first image data that has been compressed and then temporarily stored in the nonvolatile storage means, performing a main image processing on the decompressed first image data, and recording the second image data generated by the main image processing in a nonvolatile storage means different from the nonvolatile storage means, 5. The imaging control device according to claim 1, wherein the control means sets a read rate of the first image data from the non-volatile storage means and a processing rate of the main image processing based on a size of the first image data, the average write rate in the temporary recording, the imaging rate, and a compression ratio in the compression.

19. The imaging control device according to claim 1 or 4, characterized in that in the first processing, a predetermined number of the first image data generated by continuous shooting of still images or a plurality of the first image data generated within a predetermined continuous time in continuous shooting of still images is stored in a single temporary recording file to perform the temporary recording.

20. the first processing includes compression; 20. The imaging control device according to claim 19, wherein the compression of the plurality of first image data is performed by utilizing correlation between the plurality of first image data.

21. 5. The imaging control device according to claim 1, wherein the first image data on which the temporary recording has been performed is deleted from the non-volatile memory means after the main recording of the second image data corresponding to the first image data has been performed.

22. 5. The imaging control device according to claim 1, wherein in the first process, the frequency of recording the first image data in a plurality of temporary recording areas in the non-volatile storage means is averaged.

23. 23. The imaging control device according to claim 22, wherein in the first process, an area among the plurality of temporary recording areas that has been recorded on less frequently than the other areas is used preferentially as the area for temporary recording of the first image data.

24. The imaging control device according to claim 21, characterized in that an area in which the first image data corresponding to the second image data for which the actual recording has been completed was recorded is made free space by deleting the first image data, and the free space is used preferentially as an area for the actual recording of other second image data.

25. The imaging control device according to claim 1 or 24, An imaging device comprising the imaging element.

26. generating first image data for temporary recording in a non-volatile storage means from an output of an image pickup element that captures an image; generating second image data for main recording by performing a second process on the first image data read from the non-volatile storage means; An imaging control method comprising: controlling a first process from the imaging to the temporary recording so that an average writing rate of the first image data in the temporary recording is equal to or greater than a speed based on an imaging rate of the imaging element.

27. generating first image data for temporary recording in a non-volatile storage means from an output of an image pickup element that captures an image; generating second image data for main recording by performing a second process on the first image data read from the non-volatile storage means; an imaging control method for controlling an imaging rate of the imaging element based on an average writing rate of the first image data in the temporary recording;

28. 28. A program for causing a computer to execute a process according to the imaging control method according to claim 26.

Images