Imaging apparatus and imaging system
The imaging device addresses the challenge of missed photo opportunities by performing dark correction at convenient times or using a server device, ensuring effective image quality improvement without immediate dark image acquisition.
Patent Information
- Application Number
- JP2023079508
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-22
AI Technical Summary
Existing imaging devices fail to perform dark correction without missing photo opportunities, as they require acquiring a dark image immediately after a main image, which can change imaging conditions, leading to ineffective image quality improvement.
The imaging device performs normal imaging operations, records image and condition data, and executes dark correction at predetermined times or with a server device, using stored dark images or calculated corrections based on recorded conditions.
Enables dark correction without missing photo opportunities, improving image quality by performing corrections at convenient times or through remote processing, reducing power consumption and heat generation.
Smart Images

Figure 2024163684000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an imaging device and an imaging system. [Background technology]
[0002] When taking pictures of night scenes, fireworks, starry skies, etc. with a digital camera, long exposure is often used. When taking pictures with long exposure, shading due to spot scratches and dark current on the image sensor increases, causing degradation of the image quality.
[0003] One known countermeasure to this degradation in image quality is the so-called "dark correction." In general, "dark correction" refers to a process in which, after capturing an image of a subject to obtain a main image, another image is captured with the shutter closed for the same exposure time to obtain a dark image, and the dark image is then subtracted from the main image.
[0004] If a dark image is captured immediately after the main image is captured, there is a possibility that a shutter opportunity will be missed during the period until the dark image is captured. Therefore, Patent Document 1 discloses a technique for capturing a dark image before capturing the main image, and performing dark correction immediately after capturing the main image. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2009-76978 A Summary of the Invention [Problem to be solved by the invention]
[0006] In Patent Document 1, if the main image is captured after a certain time has passed since the capture of the dark image, dark correction is not performed because the imaging conditions have changed. In other words, the deterioration of image quality is not eliminated.
[0007] An object of the present disclosure is to provide an imaging device and an imaging system that are capable of performing dark correction without missing a photo opportunity. [Means for solving the problem]
[0008] In the present disclosure, the imaging device is capable of performing a normal imaging operation in which an image of a subject is captured to generate image data, and a dark correction operation in which image data is subjected to dark correction to generate restored image data. The imaging device includes an imaging section, a recording section, and a control section. The imaging section performs a normal imaging operation by opening a shutter under a predetermined shooting condition to generate image data. The recording section records the image data and shooting condition data indicating the shooting conditions on a recording medium. The control section controls the imaging section. Furthermore, the control section performs a dark correction operation on each image data using a dark image corresponding to each shooting condition after performing the normal imaging operation multiple times.
[0009] In the present disclosure, the imaging system is composed of an imaging device and a server device that can communicate with each other. The imaging device includes an imaging section, a recording section, a control section, a user interface, and a first communication circuit. The imaging device opens the shutter under a predetermined shooting condition to perform a normal imaging operation and generate image data. The recording section records the image data and shooting condition data indicating the shooting conditions in a recording medium. The control section controls the imaging section. The user interface accepts a designation from a user to perform a dark correction operation in cooperation with the server device. The first communication circuit transmits the shooting conditions and the designation. The server device includes a second communication circuit and a server control section. The second communication circuit receives the shooting conditions and the designation. The server control section determines a dark image to be used for the image data based on the shooting conditions. The server control section determines a dark image based on the shooting conditions from the imaging device. Then, one of the imaging device and the server device performs a dark correction operation to generate restored image data by performing a dark correction on the image data. Effect of the Invention
[0010] According to an exemplary embodiment of the present disclosure, an imaging device and an imaging system are provided that are capable of performing dark correction without missing a shutter opportunity. [Brief description of the drawings]
[0011] [Figure 1] Hardware configuration diagram of a digital camera according to a first exemplary embodiment of the present disclosure. [Diagram 2] FIG. 13 is a diagram showing an example of a menu setting screen displayed on the display monitor of a digital camera. [Diagram 3] 1 is a flowchart illustrating a procedure of a dark correction operation in the digital camera of the first embodiment. [Figure 4] FIG. 1 is a diagram for explaining the timing for executing a dark calibration operation. [Diagram 5] FIG. 1 is a diagram showing an example of a data structure of an image file. [Figure 6] FIG. 13 is a diagram for explaining the procedure of a dark correction operation when normal imaging operations are performed multiple times under the same shooting conditions; [Figure 7] FIG. 1 is a diagram for explaining a method of performing dark photography when the photography conditions do not match and when they match; [Figure 8] Graph showing temperature characteristics of image sensor dark current [Figure 9] FIG. 1 is a diagram showing an example of the configuration of an imaging system according to a second exemplary embodiment of the present disclosure. [Figure 10] FIG. 11 is a diagram showing an example of a menu setting screen displayed on a display monitor of a digital camera according to an exemplary embodiment 2 of the present disclosure. [Figure 11] Server hardware configuration diagram [Figure 12] FIG. 1 is a diagram for explaining the operation of a digital camera and a server device that constitute an imaging system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, the embodiments will be described with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.
[0013] The inventor(s) provide the accompanying drawings and the following description so that those skilled in the art can fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims.
[0014] 1. Embodiment 1 In the first embodiment, a digital camera capable of capturing an image of a subject and recording moving image data will be described as an example of an imaging device according to the present disclosure.
[0015] 1-1. Digital camera configuration examples Fig. 1 is a hardware configuration diagram of a digital camera 100 according to this embodiment. The digital camera 100 shown in Fig. 1 is, for example, a single-lens digital camera or a compact digital camera. Below, an overview of the components of the digital camera 100 will be given.
[0016] Digital camera 100 of this embodiment includes an image sensor 115, a temperature sensor 116, an image processing engine 120, a display monitor 130, and a controller 135. Digital camera 100 also includes a buffer memory 125, a card slot 140, a flash memory 145, an operation unit 150, and a communication module 160. Digital camera 100 also includes an optical system 110 and a lens driving unit 112. Digital camera 100 further includes a microphone 161, an analog / digital (A / D) converter 165 for the microphone, and an audio processing engine 170.
[0017] The optical system 110 includes a focus lens, a zoom lens, an aperture, a shutter, etc. The focus lens is a lens for changing the focus state of the subject image formed on the image sensor 115. The zoom lens is a lens for changing the magnification of the subject image formed by the optical system. The focus lens, etc. are each composed of one or more lenses.
[0018] Lens driving unit 112 drives a focus lens and the like in optical system 110. Lens driving unit 112 includes a motor, and moves the focus lens along the optical axis of optical system 110 under the control of controller 135. The configuration for driving the focus lens in lens driving unit 112 can be realized by a DC motor, a stepping motor, a servo motor, an ultrasonic motor, or the like.
[0019] When the digital camera 100 is a single-lens digital camera, the optical system 110 and the lens drive unit 112 are housed in a lens barrel (not shown). The lens barrel may be detachable from the digital camera 100.
[0020] The image sensor 115 captures an image of a subject formed via the optical system 110 to generate imaging data. The imaging data constitutes image data indicative of an image captured by the image sensor 115. The image sensor 115 generates new frame image data at a predetermined frame rate (e.g., 60 frames / second). The timing of generating imaging data and electronic shutter operation in the image sensor 115 are controlled by the controller 135. The image sensor 115 can be any of various image sensors, such as a CMOS image sensor, a CCD image sensor, or an NMOS image sensor.
[0021] The image sensor 115 performs imaging operations of moving images, still images, through images, etc. The through images are mainly moving images, and are displayed on the display monitor 130 so that the user can determine, for example, a composition for capturing a still image. The through images, moving images, and still images are each an example of a captured image in this embodiment. The image sensor 115 is an example of an imaging section in this embodiment.
[0022] Digital camera 100 receives external light incident via an optical system such as lens 110 at image sensor 115, acquires an image with, for example, 4K resolution, and can process the image by displaying, recording, etc. An "image with 4K resolution" is, for example, an image with 4096 pixels x 2160 pixels or 3840 pixels x 2160 pixels. In this specification, "image" can include both moving images and still images.
[0023] The temperature sensor 116 is provided in contact with or in the vicinity of the image sensor 115 , and detects the temperature of the image sensor 115 .
[0024] The image processing engine 120 performs various processes on the imaging data output from the image sensor 115 to generate image data, and performs various processes on the image data to generate an image to be displayed on the display monitor 130. The various processes include, but are not limited to, white balance correction, gamma correction, YC conversion processing, electronic zoom processing, compression processing, and expansion processing. The image processing engine 120 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer, processor, etc. using a program.
[0025] The display monitor 130 is a device capable of displaying various information, and may be, for example, a liquid crystal display device or an organic EL device.
[0026] Operation unit 150 is a general term for hard keys such as operation buttons and operation levers provided on the exterior of digital camera 100, and accepts operations by the user. Operation unit 150 includes, for example, a release button, a mode dial, a touch panel, a cursor button, and a joystick. When operation unit 150 accepts an operation by the user, it transmits an operation signal corresponding to the user operation to controller 135.
[0027] The controller 135 includes a CPU and the like, and the CPU executes a program (software) to realize the processes and functions according to the present disclosure. Instead of a CPU, the controller 135 may include a processor configured with a dedicated electronic circuit designed to realize a predetermined function. That is, the controller 135 can be realized by various processors such as a CPU, an MPU, a GPU, a DSU, an FPGA, an ASIC, etc. The controller 135 may be configured with one or more processors. The controller 135 may also be configured on a single semiconductor chip together with the image processing engine 120 and / or the audio processing engine 170, etc. In this specification, the controller 135 may be referred to as a "control unit."
[0028] The buffer memory 125 is a recording medium that functions as a work memory for the image processing engine 120 and the controller 135. The buffer memory 125 is realized by a dynamic random access memory (DRAM) or the like. The flash memory 145 is a non-volatile recording medium. Although not shown, the controller 135 may have various types of internal memory, for example, a built-in ROM. The ROM stores various programs executed by the controller 135. The controller 135 may also have a built-in RAM that functions as a work area for the CPU.
[0029] The card slot 140 is a means for inserting a removable memory card 142. The card slot 140 is capable of electrically and mechanically connecting the memory card 142. The memory card 142 is an external memory equipped with an internal recording element such as a flash memory. The memory card 142 can store data such as image data generated by the image processing engine 120.
[0030] The communication module 160 is a communication circuit that performs communication in compliance with communication standards such as IEEE802.11 or Wi-Fi standards, and / or communication in compliance with standards such as 4G or 5G. The digital camera 100 can communicate with other devices via the communication module 160. The digital camera 100 may communicate with other devices directly via the communication module 160, or may communicate with other devices via an access point. The communication module 160 may be connectable to a communication network such as the Internet.
[0031] Microphone 161 includes one or more microphone elements built into digital camera 100. Microphone 161 converts the sound picked up by the microphone elements into an analog signal, which is an electrical signal, and outputs the analog signal. Note that microphone 161 may also include a microphone element that is external to digital camera 100.
[0032] The microphone A / D converter 165 converts an analog signal from the microphone 161 into digital audio data. The audio processing engine 170 receives audio data output from an audio acquisition unit such as the microphone A / D converter 165, and performs various types of audio processing on the received audio data.
[0033] 1-2.Operation 1-2-1. Examples of capturing dark images according to individual shooting conditions Digital camera 100 according to the present disclosure is capable of performing a normal imaging operation in which an image of a subject is captured to generate image data, and a dark correction operation in which dark correction is performed on the image data to generate restored image data.
[0034] Dark correction is a technique for removing noise caused by spot or white scratches on the image sensor 115, unevenness in brightness caused by dark current, that is, shading. Specifically, dark correction is performed by subtracting dark image data containing only noise from image data obtained by capturing an image of a subject through normal shooting operations. The noise increases as the ISO sensitivity becomes relatively higher, the shutter speed becomes slower, and the sensor temperature becomes higher. Therefore, dark correction is effective in improving the image quality of an image captured by performing a long exposure time of 4 to 60 seconds in a dark place such as a night view or a starry sky.
[0035] In the digital camera 100 according to the present disclosure, the user can set the execution timing of dark correction via a menu. For example, Fig. 2 shows an example of a menu setting screen displayed on the display monitor 130 of the digital camera 100. In the example of Fig. 2, dark correction is shown as "long exposure noise reduction".
[0036] The user operates the operation unit 150 to select the item 2b indicating "ON, execute later" from the item 2a indicating "OFF" of the long second noise removal. Here, "execute later" means that the dark correction operation is performed after the digital camera 100 has finished the normal imaging operation, for example, during non-imaging operation after the shooting mode has ended. For example, the dark correction operation is performed when the power is turned off after multiple normal imaging operations have been completed. In this embodiment, the digital camera 100 does not take the same amount of time to acquire a dark image each time the normal imaging operation is performed. This prevents the user from missing a shutter opportunity, and improves the convenience of the digital camera 100.
[0037] After the normal imaging operation is completed, the controller 135 of the digital camera 100 acquires dark image data under the same or sufficiently close imaging conditions as those of the image data acquired by the normal imaging operation, and performs a dark correction operation. Since the dark correction operation is not performed or not performed depending on the conditions, the user can obtain image data whose image quality has been improved by the dark correction operation.
[0038] Next, the operation of digital camera 100 will be described in detail with reference to Figures 3 to 5. As a premise for the description, it is assumed that the user operates operation unit 150 to select option 2b indicating "Execute after turning ON" for long second noise removal.
[0039] Fig. 3 is a flowchart illustrating the procedure of the dark correction operation in the digital camera 100 of this embodiment. Each process in this flowchart is executed by the controller 135 and by the image engine and the like that receives instructions from the controller 135. Fig. 4 is a diagram for explaining the timing of executing the dark correction operation.
[0040] First, the controller 135 executes a normal imaging operation in response to a user operation such as pressing a shutter button received via the operation unit 150 (S1). As a result, the image processing engine 120 generates image data. The controller 135 associates the image data with the shooting condition data and stores them in the memory card 142 (S2). The controller 135 repeats steps S1 and S2 until the normal imaging operation is completed (S3).
[0041] 4, the normal imaging operation is shown in the period up to time T. For example, normal exposure is performed in the first shooting, and an image file F including shooting condition data 10 and image data 11 is obtained.
[0042] 5 shows an example of the data structure of the image file F. In this embodiment, the shooting condition data 10 describes the ISO sensitivity during normal image capture, the shutter speed, and the temperature of the image sensor 115 measured by the temperature sensor 116. The shutter speed indicates the speed at which the shutter closes, in other words, the exposure time. The shooting condition data 10 may further include shooting conditions such as the shooting date and time, the manufacturer name of the digital camera 100, the overall image resolution, the resolution per unit in the horizontal and vertical directions, the aperture (F-number), and the focal length.
[0043] The shooting condition data 10 is written in compliance with, for example, the Exif (Exchangeable image file format) standard. The image data 11 is data in compliance with a well-known image format such as JPEG, TIFF, PNG, etc. In this embodiment, embedding the shooting condition data 10 in the image data 11 in compliance with the Exif standard is called linking the image data and the shooting condition data. Note that if the image data and the shooting condition data are associated one-to-one, it is not essential to use the Exif standard, and it is also not essential to embed the shooting condition data 10 in the image data 11.
[0044] 4, an image file F acquired by normal imaging operation is stored in memory card 142. Similarly, an image file G acquired by subsequent normal imaging operations is also stored in memory card 142, for example, each time an image is captured. Since normal imaging operations can be performed continuously, the user will not miss a photo opportunity.
[0045] 3 again, when the controller 135 detects the end of the normal imaging operation, it proceeds to the next process (S4). For example, the controller 135 detects the end of the normal imaging operation when the user turns off the power of the digital camera 100 or when the operation mode is changed to a non-imaging mode, such as a playback mode.
[0046] The controller 135 reads the associated image data and shooting condition data from the memory card 142 (S4), and extracts images that require dark correction by referring to the shooting conditions in the shooting condition data (S5).
[0047] For example, the flash memory 145 of the digital camera 100 holds in advance threshold conditions of ISO sensitivity, shutter speed, and / or temperature of the image sensor 115 for performing the dark correction operation. When the controller 135 determines that at least one of the shooting conditions in the shooting condition data 10 satisfies the threshold condition by referring to the threshold conditions in the flash memory 145, it extracts an image taken under the shooting condition as an image requiring dark correction. The controller 135 assigns an uncorrected flag to the extracted image file indicating that the image is a target for the dark correction operation. In addition to or instead of extracting an image based on the threshold conditions, the controller 135 may accept an image requiring dark correction from a user and assign an uncorrected flag to the image.
[0048] Next, the controller 135 acquires a dark image based on the shooting conditions of the image to be subjected to dark correction (S6). For example, the controller 135 refers to the shooting condition data of the image file to which the uncorrected flag is added, and performs an image capturing operation with the shutter closed for an exposure time corresponding to the same ISO sensitivity and shutter speed as the shooting conditions. This acquires a dark image for correcting the image.
[0049] Then, the controller 135 performs a dark correction operation on the image using the dark image (S7). Specifically, the controller 135 subtracts the luminance value of each pixel constituting the dark image from the luminance value of each pixel constituting the image. The image and image data obtained by the subtraction are hereinafter referred to as the "restored image" and "restored image data".
[0050] The controller 135 stores the restored image data obtained by the dark correction operation in a recording medium (S8).
[0051] The above-mentioned steps S4 to S8 will be described with reference to Fig. 4. The targets of the dark correction operation are image files F and G. Note that in image file G, shooting condition data 12 and image data 13 are linked.
[0052] The controller 135 detects the end of the normal imaging operation at time T. The controller 135 refers to the imaging condition data 10 and 12, respectively, and performs imaging for acquiring a dark image, that is, dark imaging.
[0053] During the dark shooting operation after time T, the digital camera 100 is in a non-shooting state after the normal shooting operation has ended. For example, the power of the digital camera 100 is off, or the digital camera 100 is in an image playback mode. Therefore, it can be said that the user has no intention of shooting. Therefore, there is no need to assume that a shutter opportunity will be lost due to dark shooting.
[0054] In the above example, it has been described that the photographing condition data 10 and the like are referenced and the photographing operation is performed with the shutter closed for an exposure time corresponding to the same ISO sensitivity and shutter speed as the photographing conditions. However, if more photographing conditions are matched to perform dark photographing, the improvement of image quality by dark correction is more remarkable. For example, if the photographing condition data 10 and the like describes the temperature value of the image sensor 115, it is preferable to perform dark photographing under conditions that are the same as or sufficiently close to that temperature. The sufficiently close conditions are, for example, within the range of ±1 degree of the temperature value of the image sensor 115 at the time of photographing.
[0055] The controller 135 suspends the dark shooting operation until the temperature of the controller 135, which rises as a result of operating the digital camera 100, reaches, for example, the temperature value of the image sensor 115 described in the shooting condition data 10. When the temperature of the controller 135 matches the shooting condition data 10, the dark correction operation is executed at that point. This makes it possible to achieve high correction accuracy.
[0056] Dark image data D1 and D2 are sequentially obtained by dark photography. The controller 135 obtains restored image data 16 by subtracting image data 14 of the dark image D1 from image data 11. The controller 135 stores the restored image data 16 in the memory card 142. Similarly, the controller 135 obtains restored image data 17 by subtracting image data 15 of the dark image D2 from image data 13, and stores the restored image data 17 in the memory card 142.
[0057] The image data 11 may remain stored in the memory card 142 or may be deleted. Also, by embedding the shooting condition data 10 in the restored image data 16 and embedding the shooting condition data 12 in the restored image data 17, one image file each may be stored in the memory card 142.
[0058] 1-2-2. Example of common dark image 4, for each normal exposure, a dark image is generated by performing a dark photograph after time T. However, if the photographing conditions for multiple normal exposures are the same or sufficiently close, a single dark photograph may suffice.
[0059] 6 is a diagram for explaining the procedure of the dark correction operation when normal imaging operations are performed multiple times under the same shooting conditions. For example, the normal imaging operations performed multiple times under the same shooting conditions may be images of a starry sky captured multiple times with fixed ISO sensitivity and shutter speed.
[0060] By capturing images multiple times, images Pa3 to Pa5 are acquired, each of which is associated with the shooting condition data and stored in the memory card 142. After that, at time T, the controller 135 detects the end of the normal imaging operation.
[0061] Next, the controller 135 determines whether the other shooting conditions except for the shooting time, such as the ISO sensitivity and the shutter speed, are all the same. FIG. 6 shows that the shooting condition data 20 is common between the images Pa3 to Pa5. If it is determined that the other shooting conditions except for the shooting time are the same, the controller 135 performs dark shooting once after the time T to obtain a dark image D3. Thereafter, the controller 135 performs dark correction on the images Pa3 to Pa5 using the obtained dark image D3 to obtain restored images Pb3 to Pb5.
[0062] Even if the multiple shooting conditions are not perfectly the same but are close enough, one dark shooting can be performed. For example, if the exposure time corresponding to the shutter speed is within ±10% of the median value, the controller 135 can perform one dark shooting with the median exposure time to obtain a dark image D3.
[0063] According to this example, the dark image D3 acquired by one dark photographing is used for the dark correction, which shortens the time required for the dark photographing and prevents the heat generation and the consumption of the battery (not shown) due to the operation.
[0064] 1-2-3. A mixture of examples of taking dark images according to individual shooting conditions and examples of sharing dark images The controller 135 of the digital camera 100 can also operate in a combination of the examples of items 1-2-1 and 1-2-2 above.
[0065] 7 is a diagram for explaining a method of performing dark photography when the shooting conditions do not match and match. An example is given in which a subject image is captured by performing normal exposure under three shooting conditions C1, C2, and C3 during normal imaging operation. Of these, it is assumed that one normal exposure is performed under each of the shooting conditions C1 and C2, and four normal exposures are performed under the shooting condition C3. After each normal exposure, the image data and the shooting condition data are linked to each other and saved in the memory card 142.
[0066] An outline of the dark correction operation is shown within the dashed line in Fig. 7. Please note that for convenience of explanation, it differs from the descriptions in Figs. 4 and 6.
[0067] For the images obtained in the first two normal imaging operations, the controller 135 performs dark imaging under imaging conditions C1 and C2 to obtain separate dark images, and performs dark correction using each dark image. On the other hand, for the four images obtained in the third and subsequent normal imaging operations, the controller 135 performs dark imaging under imaging condition C3 to obtain one dark image, and performs dark correction using that dark image. The controller 135 stores the restored image data after each dark correction in the memory card 142.
[0068] As in the examples of items 1-2-1 to 1-2-3 above, dark photography may be performed multiple times, once, or in combination depending on the shooting conditions. This makes it possible to reduce the time required for dark photography, the processing power used, heat generation, and battery consumption while performing dark correction suitable for images obtained by normal imaging operations.
[0069] The controller 135 may associate the data of the dark image acquired by dark photography with the photography condition data, and store it, for example, in the memory card 142 and / or the flash memory 145. When dark photography is performed later under matching photography conditions, the stored dark image data can be read out and dark correction can be performed. Since dark photography can be omitted, the dark correction operation can be performed quickly.
[0070] Instead of performing the above-mentioned dark photography and storing dark images, the manufacturer of the digital camera 100 may store dark images under various shooting conditions in advance. This eliminates the need to perform dark photography, thereby improving user convenience.
[0071] However, continued use of the digital camera 100 may cause deterioration of the image sensor 115 due to ultraviolet rays and the like. For example, the image sensor 115 may develop several tens of white scratches in a year. If dark images are captured and stored in advance, the effects of white scratches that develop over time cannot be eliminated. Therefore, dark photography may be performed periodically, for example, once a year, to update the dark images. This makes it possible to perform dark correction on images captured in normal imaging operations and improve image quality.
[0072] 1-2-4. Generation of dark images by calculation In the examples described so far, the digital camera 100 basically captures dark images by performing dark photography. Dark photography requires time and power, so it would be preferable to omit it. The inventors of the present application therefore investigated how to do this, and have come up with a method for generating dark images by calculation.
[0073] FIG. 8 is a graph showing the temperature characteristics of the dark current of the image sensor 115. The horizontal axis indicates temperature, and the vertical axis indicates the amount of dark current. As shown in FIG. 8, the amount of dark current of the image sensor 115 such as a photodiode increases exponentially with temperature. For example, it is said that the dark current doubles every 8° C., or that it doubles every 10° C. FIG. 8 shows four types of curves corresponding to four types of temperature coefficient Tc values. Specifically, each curve is expressed by the following formula with the temperature coefficient Tc as a parameter.
number
[0074] The dark current-temperature characteristics of the image sensor 115 differ depending on the characteristics of each device, the voltage used, etc. If a more accurate temperature coefficient is required, it is preferable to measure the temperature characteristics in the actual usage environment.
[0075] As an example, the solid curve in FIG. 8 will be described. When the temperature of the image sensor 115 is 30 degrees, the dark current amount I 30 The amount of dark current at 40 degrees is I 40 It is expressed as follows.
[0076] When two dark images are captured under the same shooting conditions except for the temperature of the image sensor 115, the difference between them, that is, the difference in the level of noise, is the amount of dark current I 30 and I 40 It can be said that the difference is between the two. Therefore, for example, a rule for obtaining a dark image of the image sensor 115 at 40 degrees can be derived in advance from a dark image acquired at 30 degrees (hereinafter referred to as the "reference dark image"). The rule specifies a calculation method for adding the effect (gain) of the difference in the amount of dark current on noise, based on the reference dark image. The rule for angles lower than 30 degrees specifies a calculation method for subtracting the effect (gain) of the difference in the amount of dark current on noise, based on the reference dark image. It is easy to set up such a rule in advance, and the calculation method is relatively simple. Implementation is easy because no complicated control is required.
[0077] Alternatively, a single or pixel-by-pixel conversion rule may be used to calculate a dark image at 40 degrees from a dark image at 30 degrees using dark images actually acquired when the temperatures of the image sensor 115 are 30 degrees and 40 degrees. Similarly, by preparing a rule for each temperature according to the measurement resolution of the temperature sensor 116, for example, it is possible to calculate a dark image at that temperature from the reference dark image by referring to the rule based on the temperature of the image sensor 115 acquired as a shooting condition. Furthermore, it is possible to set a rule according to temperature for each temperature coefficient and / or for each combination of ISO sensitivity and shutter speed as a shooting condition.
[0078] As an example of the rule, a case where a plurality of dark images are used is also considered. For example, a dark image where the temperature of the controller 135 is 30 degrees and a dark image where the temperature of the controller 135 is 40 degrees are prepared in advance, and the temperature of the controller 135 at the time of shooting is described as 37 degrees in the shooting condition data. The controller 135 may calculate a dark image of 37 degrees by weighting and averaging the dark image of 30 degrees and the dark image of 40 degrees. This method can also shorten the time required for dark correction compared to the case of performing dark imaging, and achieve high correction accuracy.
[0079] The manufacturer of the digital camera 100 stores the reference dark image and the rule in, for example, the flash memory 145. When the timing for performing the dark correction operation arrives, the controller 135 can generate a dark image from the reference dark image based on the shooting conditions of at least one image data generated by the normal imaging operation and the pre-stored rule. This eliminates the need for the digital camera 100 to shoot and store dark images for each of various shooting conditions. Since there is no need to perform dark shooting, the convenience of the user can be improved.
[0080] 1-3. Effects etc. As described above, in this embodiment, the digital camera 100, which is an example of an imaging device, can perform a normal imaging operation of capturing an image of a subject and generating image data, and a dark correction operation of performing dark correction on the image data to generate restored image data. The digital camera 100 includes an image sensor 115 (an example of an imaging unit) that opens a shutter under a predetermined shooting condition to perform a normal imaging operation and generate image data, a card slot 140 (an example of a recording unit) that records the image data and shooting condition data indicating the shooting conditions on a memory card 142, which is an example of a recording medium, and a controller 135 (an example of a control unit) that controls the imaging unit. The controller 135 performs the normal imaging operation multiple times, and then performs a dark correction operation on each image data using a dark image corresponding to each shooting condition.
[0081] With this configuration, the user of the digital camera 100 will not miss a photo opportunity. Since the digital camera 100 performs dark correction, degradation of the image quality of the captured image can be eliminated.
[0082] In this embodiment, the digital camera 100 further includes an operation unit 150 (an example of a user interface) that accepts a user's designation of the execution timing of the dark correction operation. The execution timing includes a timing different from the timing immediately after the end of the normal imaging operation. Since the dark correction operation is not performed immediately after the end of the normal imaging operation, dark photography is not performed, and the user will not miss a photo opportunity.
[0083] In this embodiment, when the execution timing arrives, the controller 135 of the digital camera 100 captures a dark image under the same shooting conditions as at least one of the multiple image data generated by multiple normal imaging operations, and performs a dark correction operation on the at least one image data using the captured dark image.
[0084] The flash memory 145 (an example of a recording unit) holds a reference dark image and rules for generating a dark image from the reference dark image. When the timing for performing the dark correction operation arrives, the controller 135 generates a dark image from the reference dark image based on the shooting conditions of at least one of the multiple pieces of image data generated by multiple normal imaging operations and the rules. Then, the controller 135 performs the dark correction operation on the at least one piece of image data using the generated dark image. Since there is no need to perform dark shooting, the convenience of the user can be improved.
[0085] In this embodiment, the flash memory 145 holds image data of a plurality of dark images captured under a plurality of shooting conditions. When the timing for performing the dark correction operation arrives, the controller 135 selects one dark image from the plurality of dark images based on the shooting conditions of at least one image data among the plurality of image data generated by the plurality of normal imaging operations, and performs the dark correction operation on at least one image data using the selected dark image. Since it is no longer necessary to perform dark shooting, the convenience of the user can be improved.
[0086] In this embodiment, the controller 135 determines whether or not the shooting conditions of at least one piece of image data correspond to conditions requiring dark correction, and if it is determined that the conditions correspond to conditions requiring dark correction, performs a dark correction operation.
[0087] In this embodiment, when the shooting conditions of the multiple image data are the same, the controller 135 performs a dark correction operation on the multiple image data using a common dark image. Since it is not necessary to perform dark shooting for all captured images, the processing speed is increased and the user's convenience can be improved.
[0088] (Embodiment 2) In the first embodiment, the digital camera performed a normal shooting operation and a dark correction operation. In the present embodiment, an imaging system including a digital camera and a server device will be described. The configuration of the digital camera according to this embodiment is the same as the configuration of the digital camera 100 according to the first embodiment. Therefore, the digital camera according to this embodiment will be described using the same reference numeral 100.
[0089] 9 shows an example of the configuration of an imaging system 200 according to this embodiment. The imaging system 200 includes a digital camera 100 and a server device 220. The digital camera 100 communicates with an external device using a communication module 160. For example, the digital camera 100 can communicate with the server device 220 via a 5G communication base station 212 and a telecommunication line 214 such as the Internet.
[0090] As in the first embodiment, the digital camera 100 of this embodiment also opens the shutter under predetermined shooting conditions to perform normal imaging operation and generate image data. Then, the shooting condition data and the image data are recorded on the memory card 142 in association with each other.
[0091] In this embodiment, the digital camera 100 accepts a command from the user to perform a dark correction operation in cooperation with the server device 220 via the operation unit 150. In response to this command, the communication module 160 transmits the shooting conditions and the command to the server device 220.
[0092] 10 shows an example of a menu setting screen displayed on the display monitor 130 of the digital camera 100 according to this embodiment. When the user selects the item 2b indicating "ON, execute later", two further submenus, "Execute on camera" and "Execute on server", are displayed.
[0093] "Execute on camera" is the same as the operation of the digital camera 100 according to the first embodiment. "Execute on Server" is a new option introduced in the digital camera 100 according to this embodiment. When option 2c indicating "Execute on Server" is selected, in this embodiment, the server device 220 executes dark correction. The digital camera 100 transmits shooting condition data and image data to the server device 220 every time one imaging operation is completed. This allows the server device 220 to execute dark correction without a large time lag in the server device 220. Alternatively, the digital camera 100 transmits multiple sets of shooting condition data and image data collectively to the server device 220 after multiple normal imaging operations are completed.
[0094] The server device 220 determines a dark image by referring to the received shooting condition data. The dark image may be prepared in advance or may be calculated by calculation. An example of calculating a dark image by calculation will be described below.
[0095] FIG. 11 is a diagram showing the hardware configuration of the server device 220. As shown in FIG.
[0096] Server device 220 includes processing circuitry 222 , storage device 224 , and communications circuitry 226 .
[0097] The processing circuitry 222 includes a CPU and the like, and the CPU executes a program (software) to realize the processing and functions of the server device 220 according to this embodiment. Instead of a CPU, the processing circuitry 222 may include a processor configured with a dedicated electronic circuit designed to realize a predetermined function. In other words, the processing circuitry 222 can be realized by various processors such as a CPU, MPU, GPU, DSU, FPGA, ASIC, etc. The processing circuitry 222 may be configured with one or more processors.
[0098] The storage device 224 is a recording medium that functions as a work memory of the server device 220. The storage device 224 may include a DRAM and / or a flash ROM. In this embodiment, the storage device 224 stores a dark correction database (DB) 224a in advance. The dark correction DB 224a includes a reference dark image group and a generation rule for a dark image. The reference dark image group is a collection of reference dark images prepared for various shooting conditions. To give a more specific example, one dark image is prepared for each of various combinations of the shutter speed, the ISO sensitivity, and the temperature of the image sensor 115. It is considered that the greater the number of dark images, the higher the accuracy of the dark correction. However, as a trade-off, the total data size of the dark images increases. A person skilled in the art can appropriately set the number of steps of the shutter speed, the ISO sensitivity, and the temperature of the image sensor 115, taking into account requirements such as the total data size of the dark images. The generation rule for a dark image is a rule for generating a dark image from a given shooting condition and a reference dark image. An example of the rule is the same as that described in the first embodiment.
[0099] The communication circuit 226 is a communication interface such as an Ethernet (registered trademark) communication terminal or a USB (registered trademark) terminal. The communication circuit 226 may perform communication in compliance with standards such as IEEE802.11, 4G, or 5G. The communication circuit 226 receives from the digital camera 100 shooting conditions and an instruction specifying that the digital camera 100 and the server device 220 operate in cooperation with each other.
[0100] FIG. 12 is a diagram illustrating the operation of the digital camera 100 and the server device 220 that constitute the image capture system 200. As shown in FIG.
[0101] After the digital camera 100 has performed a normal imaging operation, it transmits one or more sets of shooting condition data and image data to the server device 220 .
[0102] The processing circuit 222 of the server device 220 refers to the dark correction DB 224a based on the shooting conditions and determines a dark image to be used for image data captured by the digital camera 100. Specifically, the processing circuit 222 refers to the dark correction DB 224a based on the shooting conditions received from the digital camera 100 and determines a dark image to be used for dark correction. The processing circuit 222 uses the image data of the dark image to perform a dark correction operation on the image received from the digital camera 100. The processing circuit 222 transmits restored image data obtained by the dark correction operation to the digital camera 100 via the communication circuit 226.
[0103] Digital camera 100 receives the restored image data from server device 220 and stores it, for example, in memory card 142. Digital camera 100 no longer needs to perform dark shooting or dark correction, and the processing load is significantly reduced. This eliminates the need to reserve a processing program and buffer memory for the dark correction, reducing manufacturing costs and improving user convenience.
[0104] In the above example, the server device 220 performed the dark correction operation, but the digital camera 100 may also perform the dark correction. In this case, the digital camera 100 transmits shooting condition data to the server device 220, but does not need to transmit image data. On the other hand, the server device 220 determines a dark image according to the shooting conditions and transmits it to the digital camera 100. The digital camera 100 then performs dark correction using the received dark image data. Since the digital camera 100 still does not need to perform dark shooting, processing is sped up and user convenience can be improved.
[0105] Aspects of the present disclosure As described above, the present disclosure includes the following aspects.
[0106] <Aspect 1> An imaging apparatus capable of performing a normal imaging operation of capturing an image of a subject and generating image data, and a dark correction operation of performing dark correction on the image data and generating restored image data, an imaging unit that opens a shutter under a predetermined photographing condition to perform the normal photographing operation and generate image data; a recording unit that records the image data and imaging condition data indicating the imaging conditions on a recording medium; A control unit that controls the imaging unit, The control unit is After performing the normal imaging operation a plurality of times, the imaging device performs a dark correction operation on each image data by using a dark image corresponding to each imaging condition.
[0107] <Aspect 2> a user interface that receives a specification of a timing for performing the dark correction operation from a user; 2. The imaging device according to aspect 1, wherein the execution timing includes a timing different from a timing immediately after the end of the normal imaging operation.
[0108] <Aspect 3> The imaging device of aspect 2, wherein when the execution timing arrives, the control unit captures the dark image under the same shooting conditions as the image data of at least one of the multiple image data generated by the multiple normal imaging operations, and performs the dark correction operation on the image data of the at least one image using the captured dark image.
[0109] <Aspect 4> the recording unit holds a reference dark image and a rule for generating the dark image from the reference dark image; The imaging device described in aspect 2, wherein when the execution timing arrives, the control unit generates the dark image from the reference dark image based on the shooting conditions of at least one piece of image data among the multiple image data generated by the multiple normal imaging operations and the rule, and performs the dark correction operation on the at least one piece of image data using the generated dark image.
[0110] <Aspect 5> The recording unit holds image data of a plurality of dark images captured under a plurality of shooting conditions, The imaging device described in aspect 2, wherein when the execution timing arrives, the control unit selects one dark image from the multiple dark images based on the shooting conditions of at least one piece of image data among the multiple image data generated by the multiple normal imaging operations, and performs the dark correction operation on the at least one piece of image data using the selected dark image.
[0111] <Aspect 6> The imaging device according to any one of aspects 3 to 5, wherein the control unit determines whether or not the shooting conditions of the at least one image data satisfy conditions under which the dark correction is required, and performs the dark correction operation if it determines that the shooting conditions satisfy conditions under which the dark correction is required.
[0112] <Aspect 7> 6. The imaging device according to any one of aspects 3 to 5, wherein, when the shooting conditions for each of the plurality of image data are the same, the control unit performs a dark correction operation on the plurality of image data by using the common dark image.
[0113] <Aspect 8> An imaging system including an imaging device and a server device that can communicate with each other, The imaging device includes: an imaging unit that opens a shutter under a predetermined shooting condition to perform a normal imaging operation and generate image data; a recording unit that records the image data and imaging condition data indicating the imaging conditions on a recording medium; A control unit that controls the imaging unit; a user interface that accepts, from a user, a designation for performing the dark calibration operation in cooperation with the server device; a first communication circuit for transmitting the photographing conditions and the designation; having The server device includes: a second communication circuit for receiving the photographing conditions and the designation; a server control unit that determines a dark image to be used for the image data based on the photographing conditions; Equipped with The server control unit determines the dark image based on the shooting conditions from the imaging device; An imaging system, wherein one of the imaging device and the server device performs a dark correction operation to generate restored image data by performing dark correction on the image data.
[0114] <Aspect 9> the server device includes a server storage device that stores a reference dark image and a rule for generating the dark image from a given shooting condition and the reference dark image, or that stores a plurality of dark images for each of a plurality of shooting conditions of the imaging device; The server control unit The imaging system of aspect 8, wherein the dark image is determined to be a dark image generated using the reference dark image stored in the server storage device based on the shooting conditions from the imaging device and the rules, or a dark image selected from the multiple dark images stored in the server storage device. [Industrial Applicability]
[0115] The present disclosure is applicable to imaging devices. [Explanation of symbols]
[0116] 100 Digital Camera 115 Image Sensor 116 Temperature Sensor 130 Display Monitor 135 Controller 140 Card Slots 142 Memory Card 145 Flash Memory 150 Operation section 160 Communication Module 200 Imaging System 220 Server device 222 Processing Circuit 224 Storage device 226 Communication Circuit
Claims
1. An imaging apparatus capable of performing a normal imaging operation of capturing an image of a subject and generating image data, and a dark correction operation of performing dark correction on the image data and generating restored image data, an imaging unit that opens a shutter under a predetermined photographing condition to perform the normal photographing operation and generate image data; a recording unit that records the image data and imaging condition data indicating the imaging conditions on a recording medium; A control unit that controls the imaging unit, The control unit is After performing the normal imaging operation a plurality of times, the imaging device performs a dark correction operation on each image data by using a dark image corresponding to each imaging condition.
2. a user interface that receives a specification of a timing for performing the dark correction operation from a user; The imaging device according to claim 1 , wherein the execution timing includes a timing different from a timing immediately after the end of the normal imaging operation.
3. 3. The imaging device according to claim 2, wherein, when the execution timing arrives, the control unit captures the dark image under the same shooting conditions as those of at least one of the multiple image data sets generated by the multiple normal imaging operations, and performs the dark correction operation on the at least one image data set using the captured dark image.
4. the recording unit holds a reference dark image and a rule for generating the dark image from the reference dark image; 3. The imaging device according to claim 2, wherein, when the execution timing arrives, the control unit generates the dark image from the reference dark image based on the shooting conditions of at least one of the multiple image data generated by the multiple normal imaging operations and the rule, and performs the dark correction operation on the at least one image data using the generated dark image.
5. The recording unit holds image data of a plurality of dark images captured under a plurality of shooting conditions, 3. The imaging device according to claim 2, wherein, when the execution timing arrives, the control unit selects one dark image from the plurality of dark images based on shooting conditions of at least one image data among the plurality of image data generated by the plurality of normal imaging operations, and performs the dark correction operation on the at least one image data using the selected dark image.
6. 6. The imaging device according to claim 3, wherein the control unit determines whether or not the shooting conditions of the at least one image data correspond to conditions under which the dark correction is necessary, and performs the dark correction operation if it is determined that the shooting conditions correspond to conditions under which the dark correction is necessary.
7. The imaging device according to claim 3 , wherein the control unit performs a dark correction operation on the plurality of image data by using the common dark image when the shooting conditions of each of the plurality of image data are the same.
8. An imaging system including an imaging device and a server device that can communicate with each other, The imaging device includes: an imaging unit that opens a shutter under a predetermined shooting condition to perform a normal imaging operation and generate image data; a recording unit that records the image data and imaging condition data indicating the imaging conditions on a recording medium; A control unit that controls the imaging unit; a user interface that accepts, from a user, a designation for performing the dark calibration operation in cooperation with the server device; a first communication circuit for transmitting the photographing conditions and the designation; having The server device includes: a second communication circuit for receiving the photographing conditions and the designation; a server control unit that determines a dark image to be used for the image data based on the photographing conditions; Equipped with The server control unit determines the dark image based on the shooting conditions from the imaging device; An imaging system, wherein one of the imaging device and the server device performs a dark correction operation to generate restored image data by performing dark correction on the image data.
9. the server device includes a server storage device that stores a reference dark image and a rule for generating the dark image from a given shooting condition and the reference dark image, or that stores a plurality of dark images for each of a plurality of shooting conditions of the imaging device; The server control unit The imaging system of claim 8, wherein the dark image is determined to be a dark image generated using the reference dark image stored in the server storage device and the rules based on the shooting conditions from the imaging device, or a dark image selected from the multiple dark images stored in the server storage device.
Citation Information
Patent Citations
Electronic camera and its photographing method
JP2009076978A
