Image capture device, image capture method, and image processing device

Abstract

The purpose of the present invention is to provide technology for correcting a defective pixel even if a video output inversion function is used. For this purpose, one image capture device according to the present invention comprises: an imaging unit which captures an image and is capable of outputting a captured positive image and an inverted image obtained by inverting the positive image; a defective pixel detection unit which detects first defective pixel coordinates contained in the positive image and second defective pixel coordinates contained in the inverted image; a defective pixel information management unit which calculates third defective pixel coordinates obtained by inverting the second defective pixel coordinates; a storage unit which stores the first defective pixel coordinates and the third defective pixel coordinates; and a defective pixel correction execution unit which corrects a defective pixel in the inverted image on the basis of the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit.

Description

Imaging device, imaging method, and image processing device

[0001] The present invention relates to an imaging device, an imaging method, and an image processing device.

[0002] In the imaging elements of imaging devices, there is a risk of defective pixels being generated that cause the captured image to output signals with abnormal levels due to the manufacturing process or deterioration over time, etc. Therefore, imaging devices are equipped with an image correction function to prevent defective pixels from affecting the captured image.

[0003] For example, Patent Document 1 discloses the following technology: "The CMOS image sensor 200 outputs an image signal that is inverted upside down or leftside to right in response to a user's instruction. The defect correction circuit 110 identifies defective pixels in the image signal using defect information (defect location, defect level) stored in the defect information storage RAM 120, and corrects these defective pixels. The microcomputer 140 determines whether the CMOS image sensor 200 is inverted upside down or leftside to right, and converts the addresses of the defect information in the defect information storage RAM 120 in accordance with the determination result. This allows processing to be performed to match the addresses of the image signal and the defect information without changing the data order of the image signal read out from the CMOS image sensor 200, thereby enabling smooth defect correction."

[0004] Japanese Patent Application Laid-Open No. 2003-163842

[0005] However, when the image output is inverted using the image sensor's image output inversion function, defective pixels may appear in positions different from the positions of the defective pixels in the image before inversion. While the invention described in Patent Document 1 makes it possible to correct the image according to the positions of the defective pixels in the image before inversion, it is unable to correct defective pixels that appear only in the inverted image as described above. Therefore, an object of the present invention is to provide a technology for correcting defective pixels even when the image output inversion function is used.

[0006] In order to solve the above-mentioned problems, one representative imaging device of the present invention has an imaging unit that can capture an image and output the captured normal image and an inverted image obtained by inverting the normal image; a defective pixel detection unit that detects first defective pixel coordinates contained in the normal image and second defective pixel coordinates contained in the inverted image; a defective pixel information management unit that calculates third defective pixel coordinates obtained by inverting the second defective pixel coordinates; a memory unit that stores the first defective pixel coordinates and the third defective pixel coordinates; and a defective pixel correction execution unit that corrects defective pixels in the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the memory unit.

[0007] According to the present invention, it is possible to provide a technique for correcting defective pixels even when a video output inversion function is used. Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention.

[0008] Fig. 1 shows an image captured by an imaging device with an image inversion function and the image inverted upside down. Fig. 2 shows an example of an imaging system. Fig. 3 is a flowchart showing a defective pixel correction process in a conventional method. Fig. 4 is a flowchart showing a defective pixel correction process in the present invention.

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to these embodiments. In the drawings, identical parts are denoted by the same reference numerals. When multiple components having the same or similar functions are present, they may be described using the same reference numerals with different subscripts. Furthermore, when it is not necessary to distinguish between these multiple components, the subscripts may be omitted. Furthermore, although terms such as "first," "second," and "third" may be used to describe various elements or components in this disclosure, it should be understood that these elements or components should not be limited by these terms. These terms are used only to distinguish one element or component from another. Therefore, a first element or component discussed below may also be referred to as a second element or component without departing from the teachings of the inventive concept. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. A normal image refers to an image captured by an imaging device as is, whereas an inverted image refers to an image that has been inverted by flipping the normal image upside down, flipping it left to right, rotating it 90 degrees to the right, rotating it 90 degrees to the left, or the like.

[0010] First, referring to Fig. 1, defective pixels when a video output inversion function is used will be described. Fig. 1(A) is a diagram showing a normal image 10A captured by an imaging device having a video inversion function. Fig. 1(B) is a diagram showing an inverted image 10B obtained by inverting the image of Fig. 1(A) upside down. The normal image 10A captured by the imaging device includes defective pixels 11a and 12a. In this case, the imaging device corrects the defective pixels 11a and 12a and outputs the image, thereby preventing the defective pixels from affecting the normal image 10A.

[0011] Furthermore, in an inverted image 10B obtained by vertically inverting the normal image 10A, defective pixels 11b and 12b corresponding to the defective pixels included in the normal image 10A are present at coordinates obtained by vertically inverting the defective pixels 11a and 12a. Here, a new defective pixel 13 may appear in the inverted image 10B due to the inversion of the image.

[0012] In this case, the defective pixels of the imaging device that are to be corrected are defective pixels 11a and 12a contained in the captured normal image 10A, and the defective pixels corrected in the inverted image 10B obtained by inverting the normal image 10A are only defective pixels 11b and 12b corresponding to the defective pixels contained in the normal image 10A. In other words, the newly generated defective pixel 13 is not corrected, and the inverted image 10B is affected by the defective pixel. Therefore, the imaging system 1 of the embodiment of the present disclosure corrects the newly generated defective pixel 13 caused by the upside-down inversion, and outputs an image that is not affected by the defective pixel even when the video output inversion function is used.

[0013] [Embodiment] First, a video system 1 according to an embodiment will be described with reference to Fig. 2. Fig. 2 shows an example of the video system 1. The video system 1 displays video captured by an imaging device 100 on a PC 200. The video system 1 includes the imaging device 100 and the PC 200, which are interconnected.

[0014] <Imaging Device> The imaging device 100 is a device that captures video. The imaging device 100 is connected to a PC 200, corrects the captured video based on operations from the PC 200, and transmits the corrected video to the PC 200. For this reason, the imaging device 100 mainly includes a video output unit 110, a command analysis unit 120, an image sensor control unit 130, an image sensor 140, a defective pixel correction unit 150, and a storage unit 160.

[0015] <<Video Output Unit>> The video output unit 110 outputs a video signal to be displayed on a monitor 210 of the PC 200, which will be described later.

[0016] <<Command Analysis Unit>> The command analysis unit 120 receives commands sent from an application 220 (described later), and sends operation information corresponding to the commands to the image sensor control unit 130 .

[0017] <<Image Sensor Control Unit>> The image sensor control unit 130 controls the image sensor 140 to perform the operation indicated by the operation information transmitted from the command analysis unit 120. Specifically, the image sensor control unit 130 sets the values ​​of the registers of the image sensor 140 so as to use the function of the image sensor 140 specified by the command analysis unit 120, for example, the ROI function or the FLIP function.

[0018] <<Image Sensor>> The image sensor 140 is also referred to as an imaging unit that can capture an image and output a normal image and an inverted image obtained by inverting the captured normal image, and is, for example, a solid-state imaging element equipped with a CMOS (Complementary Metal-Oxide-Semiconductor). The image sensor 140 captures an image by arranging a large number of photodiodes, which are minute sensors, on a planar substrate and forming an image on the plane using a lens or the like. The image sensor 140 also has an ROI function that outputs an image of a specified range and a FLIP function that inverts the image output, and can be used by setting values ​​in registers.

[0019] Here, the pixel of the image sensor 140 refers to the smallest unit of an image detected by each photodiode, and is also called a pixel. If a photodiode arranged in the image sensor 140 malfunctions due to some cause during the manufacturing process or thereafter, it will output a signal of an abnormal level. In other words, it will become a defective pixel.

[0020] <<Defective Pixel Correction Unit>> Defective pixel correction unit 150 detects and corrects defective pixels in the image output from image sensor 140, and outputs the result to image output unit 110. To this end, defective pixel correction unit 150 mainly includes defective pixel detection unit 151, defective pixel information management unit 152, and defective pixel correction execution unit 153.

[0021] <<<Defective Pixel Detection Unit>>> Defective pixel detection unit 151 detects the coordinates of a first defective pixel included in the normal image output from image sensor 140, and similarly detects the coordinates of a second defective pixel included in the inverted image output from image sensor 140. Defective pixel detection unit 151 transmits the detected coordinates of the first defective pixel and coordinates of the second defective pixel to defective pixel information management unit 152.

[0022] <<<<Defective Pixel Information Management Unit>>> The defective pixel information management unit 152 calculates third defective pixel coordinates that are the inverse of the second defective pixel coordinates and stores them in the memory unit 160, and similarly stores the first defective pixel coordinates in the memory unit 160.

[0023] Furthermore, when storing the first defective pixel coordinates and the third defective pixel coordinates in the storage unit 160, the defective pixel information management unit 152 performs a merging process. Here, the merging process refers to a process in which, if there are overlapping coordinates, only one of the coordinates is stored. Furthermore, the merging process may involve rearranging the first defective pixel coordinates and the third defective pixel coordinates in ascending order. This merging process reduces the amount of data for the defective pixel coordinates, thereby reducing the capacity of the storage unit 160 and reducing costs.

[0024] <<<<Defective Pixel Correction Execution Unit>>> Defective pixel correction execution unit 153 corrects the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in storage unit 160.

[0025] <<Storage Unit>> The storage unit 160 stores the first defective pixel coordinates and the third defective pixel coordinates transmitted from the defective pixel information management unit 152. The storage unit 160 can also store the output level of the defective pixel corresponding to the first defective pixel coordinates and the output level of the defective pixel corresponding to the third defective pixel coordinates.

[0026] <PC> The PC 200 is connected to the image capture device 100 and functions as an operation terminal unit for operating the image capture device 100. Therefore, the PC 200 mainly includes a monitor 210 and an application 220.

[0027] <<Monitor>> The monitor 210 is a display unit that displays the video transmitted from the imaging device 100. Therefore, the monitor 210 displays the video based on the video signal transmitted from the video output unit 110.

[0028] <<Application>> The application 220 is a program for specifying commands to be sent to the command analysis unit 120. For this reason, the application 220 sends a command to execute a function of the imaging device 100 specified by the user, for example, a function to correct an inverted image. The application 220 may also send a command to execute other functions, for example, a command to set a value of a register of the image sensor 140 that can execute an ROI function or a FLIP function.

[0029] <Conventional Defective Pixel Correction Process> Next, a conventional method for correcting defective pixels will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the conventional method for correcting defective pixels. The following description is based on the assumption that a value relating to whether or not an image inversion function is to be used, for example, whether or not an ROI function or a FLIP function is to be used, is set in advance in a register of image sensor 140.

[0030] (Step S101) In step S101, the command analysis unit 120 determines whether or not a command has been received from the application 220. If the command analysis unit 120 has received a command, the process proceeds to S102, and if not, the correction process ends.

[0031] (Step S102) In step S102, the image sensor control unit 130 initializes the ON / OFF setting value of the ROI function of the image sensor. Specifically, the image sensor control unit 130 saves the register values ​​related to the ROI function of the image sensor 140. The saved register values ​​are stored in, for example, a memory, so that they can be read in the processing described below.

[0032] (Step S103) In step S103, the image sensor control unit 130 turns off the ROI function of the image sensor 140. Specifically, the image sensor control unit 130 sets the value of the register related to the ROI function of the image sensor 140 to a value that turns it off.

[0033] (Step S104) In step S104, similar to step S102, the image sensor control unit 130 initializes the ON / OFF setting value of the flip function of the image sensor. Specifically, the image sensor control unit 130 saves register values ​​related to the flip function of the image sensor 140. The saved register values ​​are stored in, for example, a memory, so that they can be read in the processing described below.

[0034] (Step S105) In step S105, similarly to step S103, the image sensor control unit 130 turns off the flip function of the image sensor. Specifically, the image sensor control unit 130 sets the value of the register related to the flip function of the image sensor 140 to a value that turns off the flip function.

[0035] (Step S106) In step S106, the defective pixel detection unit 151 detects the coordinates of a first defective pixel included in the normal image. At this time, the defective pixel detection unit 151 also detects the output level of the defective pixel corresponding to the first defective pixel coordinates.

[0036] (Step S107) In step S107, the defective pixel information management unit 152 stores the coordinates of the detected first defective pixel in the storage unit 160. At this time, the defective pixel information management unit 152 also stores the output level of the defective pixel corresponding to the coordinates of the first defective pixel.

[0037] (Step S108) In step S108, the image sensor 140 reads the saved value of the ON / OFF setting value register for the ROI function and sets it in the register.

[0038] (Step S109) In step S109, the image sensor 140 determines whether to use the ROI function based on the set register value. If the image sensor 140 uses the ROI function, the process proceeds to step S110. If the image sensor 140 does not use the ROI function, the process proceeds to step S111.

[0039] (Step S110) In step S110, the defective pixel information management unit 152 subtracts the coordinates of the first defective pixel at the video output start position based on the ROI function, and calculates only the coordinates of the first defective pixel within the range of the video output width to be corrected.

[0040] (Step S111) In step S111, similarly to step S108, the image sensor 140 reads the saved register value of the ON / OFF setting value of the FLIP function and sets it in the register.

[0041] (Step S112) In step S112, similar to step S109, the image sensor 140 determines whether to use the flip function based on the set register value. If the image sensor 140 uses the flip function, the process proceeds to step S113, and if not, the process proceeds to step S114.

[0042] (Step S113) In step S113, the defective pixel information management unit 152 calculates third defective pixel coordinates by flipping the first defective pixel coordinates upside down.

[0043] (Step S114) In step S114, the defective pixel correction execution unit 153 corrects the inverted image based on the coordinates of the third defective pixel.

[0044] <Defective Pixel Correction Process of the Present Invention> Next, the defective pixel correction process of the present invention will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the defective pixel correction process of the present invention. In the following description, as with conventional defective pixel correction processes, it is assumed that values ​​for an image inversion function, such as an ROI function or a FLIP function, have been set in advance in a register of image sensor 140.

[0045] (Steps S101 to S106) Steps S101 to S106 are the same as the conventional defective pixel correction process, and therefore a description thereof will be omitted.

[0046] (Step S201) In step S201, the defective pixel information management unit 152 stores the coordinates of the detected first defective pixel in the storage unit 160. At this time, the defective pixel information management unit 152 also stores the output level of the defective pixel corresponding to the coordinates of the first defective pixel.

[0047] (Step S202) In step S202, the image sensor control unit 130 turns on the flip function of the image sensor 140 and flips the output normal image upside down. Specifically, the image sensor control unit 130 sets the value of the register of the image sensor 140 to flip the normal image and output the flipped image.

[0048] (Step S203) In step S203, the defective pixel detector 151 detects the coordinates and output level of the second defective pixel from the inverted image.

[0049] (Step S204) In step S204, the defective pixel information management unit 152 calculates third defective pixel coordinates by flipping the second defective pixel coordinates upside down.

[0050] (Step S205) In step S205, the defective pixel information management unit 152 performs merging processing on the first defective pixel coordinates and the third defective pixel coordinates, and stores the merged results in the storage unit 160.

[0051] (Steps S108 to S113) Steps S108 to S113 are the same as the defective pixel correction process in the conventional method, and therefore a description thereof will be omitted.

[0052] (Step S206) In step S206, the defective pixel correction execution unit 153 corrects the inverted image based on the coordinates of the second defective pixel and the coordinates of the third defective pixel.

[0053] <Actions and Effects> The video system 1 according to the embodiment has been described above. The video system 1 of the present disclosure mainly includes a defective pixel detection unit 151, a defective pixel information management unit 152, and a defective pixel correction execution unit 153, and detects the coordinates of a second defective pixel included in an inverted image and corrects the defective pixel based on the coordinates of a third defective pixel obtained by inverting the coordinates of the second defective pixel included in the inverted image. As a result, even if a defective pixel that does not occur in the normal image appears in the inverted image, it is possible to output an inverted image that is not affected by the defective pixel.

[0054] The present invention can also take the following aspects. (Aspect 1) An imaging device comprising: an imaging unit capable of capturing an image and outputting a normal image and an inverted image obtained by inverting the captured normal image; a defective pixel detection unit that detects first defective pixel coordinates included in the normal image and second defective pixel coordinates included in the inverted image; a defective pixel information management unit that calculates third defective pixel coordinates obtained by inverting the second defective pixel coordinates; a memory unit that stores the first defective pixel coordinates and the third defective pixel coordinates; and a defective pixel correction execution unit that corrects a defective pixel in the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the memory unit. (Aspect 2) The imaging device according to Aspect 1, wherein the memory unit stores coordinates obtained by merging the first defective pixel coordinates and the third defective pixel coordinates. (Aspect 3) A method for correcting defective pixels in an inverted image obtained by inverting a captured normal image, the method comprising: a first step of detecting coordinates of a first defective pixel included in the normal image; a second step of detecting coordinates of a second defective pixel included in the inverted image; a third step of calculating coordinates of a third defective pixel obtained by inverting the coordinates of the second defective pixel; a fourth step of storing the coordinates of the first defective pixel and the coordinates of the third defective pixel; and a fifth step of correcting the defective pixel in the inverted image based on the stored coordinates of the first defective pixel and the coordinates of the third defective pixel. (Aspect 4) An image processing device comprising: an image acquisition unit that acquires a captured normal image and an inverted image obtained by inverting the normal image, a defective pixel detection unit that detects first defective pixel coordinates included in the normal image and second defective pixel coordinates included in the inverted image, a defective pixel information management unit that calculates third defective pixel coordinates obtained by inverting the second defective pixel coordinates, a memory unit that stores the first defective pixel coordinates and the third defective pixel coordinates, and a defective pixel correction execution unit that corrects defective pixels in the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the memory unit. (Aspect 5) The image processing device according to Aspect 4, wherein the memory unit stores coordinates obtained by merging the first defective pixel coordinates and the third defective pixel coordinates.

[0055] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present invention.

[0056] 1: Video system, 10A: normal image, 10B: inverted image, 11A, 11B, 12A, 12B, 13: defective pixel, 100: imaging device, 110: image output unit, 120: command analysis unit, 130: image sensor control unit, 140: image sensor, 150: defective pixel correction unit, 151: defective pixel detection unit, 152: defective pixel information management unit, 153: defective pixel correction execution unit, 200: PC, 210: monitor, 220: application

Claims

1. An imaging device having: an imaging unit capable of capturing an image and outputting a normal image and an inverted image obtained by inverting the captured normal image; a defective pixel detection unit that detects a first defective pixel coordinate contained in the normal image and a second defective pixel coordinate contained in the inverted image; a defective pixel information management unit that calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate; a memory unit that stores the first defective pixel coordinate and the third defective pixel coordinate; and a defective pixel correction execution unit that corrects a defective pixel in the inverted image based on the first defective pixel coordinate and the third defective pixel coordinate stored in the memory unit.

2. An imaging device according to claim 1, wherein the storage unit stores coordinates obtained by merging the coordinates of the first defective pixel and the coordinates of the third defective pixel.

3. A method for correcting defective pixels in an inverted image obtained by inverting a captured normal image, comprising: a first step of detecting a first defective pixel coordinate contained in the normal image; a second step of detecting a second defective pixel coordinate contained in the inverted image; a third step of calculating a third defective pixel coordinate obtained by inverting the second defective pixel coordinate; a fourth step of storing the first defective pixel coordinate and the third defective pixel coordinate; and a fifth step of correcting the defective pixel in the inverted image based on the stored first defective pixel coordinate and the third defective pixel coordinate.

4. An image processing device having: an image acquisition unit that acquires a captured normal image and an inverted image obtained by inverting the normal image; a defective pixel detection unit that detects a first defective pixel coordinate contained in the normal image and a second defective pixel coordinate contained in the inverted image; a defective pixel information management unit that calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate; a memory unit that stores the first defective pixel coordinate and the third defective pixel coordinate; and a defective pixel correction execution unit that corrects a defective pixel of the inverted image based on the first defective pixel coordinate and the third defective pixel coordinate stored in the memory unit.

5. An image processing device according to claim 4, wherein the storage unit stores coordinates obtained by merging the coordinates of the first defective pixel and the coordinates of the third defective pixel.

Images