Imaging device, imaging method, image processing device

Abstract

The purpose of the present invention is to provide technology for correcting a defective pixel even if a video output inversion function is used. For this purpose, one image capture device according to the present invention comprises: an imaging unit which captures an image and is capable of outputting a captured positive image and an inverted image obtained by inverting the positive image; a defective pixel detection unit which detects first defective pixel coordinates contained in the positive image and second defective pixel coordinates contained in the inverted image; a defective pixel information management unit which calculates third defective pixel coordinates obtained by inverting the second defective pixel coordinates; a storage unit which stores the first defective pixel coordinates and the third defective pixel coordinates; and a defective pixel correction execution unit which corrects a defective pixel in the inverted image on the basis of the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit.

Description

Technical Field

[0001] The present invention relates to an imaging device, an imaging method, and an image processing device.

Background Art

[0002] In an imaging element of an imaging device, due to the manufacturing process or aging deterioration, etc., there may be defective pixels that output signals indicating abnormal levels for the imaged pixels. Therefore, in an imaging device, by implementing a function to correct an image, it is ensured that defective pixels do not affect the captured image.

[0003] For example, in Patent Document 1, "The CMOS image sensor 200 outputs an image signal that is vertically or horizontally flipped according to a user's operation instruction. The defect correction circuit 110 uses the defect information (defect location, defect level) stored in the defect information storage RAM 120 to identify defective pixels in the image signal and correct these defective pixels. Due to the function of the microcomputer 140, it determines whether there is vertical or horizontal flipping in the CMOS image sensor 200 and performs address conversion of the defect information in the defect information storage RAM 120 according to this determination result. Thereby, without changing the data order of the image signal read from the CMOS image sensor 200, a process is performed to match the addresses of the image signal and the defect information, and smooth defect correction is performed" is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when the image output is inverted using the image sensor's image output inversion function, defective pixels may appear in positions different from those in the image before inversion. In the invention described in Patent Document 1, it is possible to correct the image according to the position of the defective pixels in the image before inversion, but it is not possible to correct defective pixels that only appear in the inverted image as described above. Therefore, the present invention aims to provide a technique for correcting defective pixels even when using the video output inversion function. [Means for solving the problem]

[0006] To solve the above problems, one representative imaging device of the present invention includes: an imaging unit capable of capturing an image and outputting the captured positive image and an inverted image obtained by inverting the positive image; a defective pixel detection unit that detects a first defective pixel coordinate included in the positive image and a second defective pixel coordinate included in the inverted image; a defective pixel information management unit that calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate; a storage unit that stores the first defective pixel coordinate and the third defective pixel coordinate; and a defective pixel correction execution unit that corrects the defective pixels in the inverted image based on the first defective pixel coordinate and the third defective pixel coordinate stored in the storage unit. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a technique for correcting defective pixels even when using the video output inversion function. Other issues, configurations, and effects not mentioned above will be clarified by the description of the embodiments for carrying out the invention below. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an image captured by an imaging device with an image inversion function, and an image that has been inverted vertically. [Figure 2] Figure 2 shows an example of a video system. [Figure 3]Figure 3 is a flowchart showing the defective pixel correction process in a conventional method. [Figure 4] Figure 4 is a flowchart showing the defective pixel correction process in the present invention. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, in the drawings, identical parts are denoted by the same reference numerals. When there are multiple components with the same or similar function, they may be described using the same symbol but with different subscripts. Furthermore, when it is not necessary to distinguish between these multiple components, the subscripts may be omitted in the description. Furthermore, while terms such as "first," "second," and "third" may be used in this disclosure to describe various elements or components, it will be understood that these elements or components should not be limited by these terms. These terms are used solely to distinguish one element or component from another. Accordingly, the first element or component discussed below may also be called the second element or component without departing from the teaching of the concept of the present invention. The positions, sizes, shapes, and ranges of the components shown in the drawings may not represent their actual positions, sizes, shapes, and ranges in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, and ranges disclosed in the drawings. A "correct image" refers to the image as captured by the imaging device. Furthermore, a reversed image refers to an image that has been flipped from a normal image by methods such as vertical inversion, horizontal inversion, rotation 90 degrees to the right, or rotation 90 degrees to the left.

[0010] First, referring to Figure 1, we will explain the defective pixels when using the video output inversion function. Figure 1(A) shows the normal image 10A captured by an imaging device with an image inversion function. Figure 1(B) shows the inverted image 10B obtained by inverting the image of Figure 1(A) vertically. The positive image 10A captured by the imaging device includes defective pixels 11a and 12a. At this time, the imaging device corrects the defective pixels 11a and 12a and outputs the image so that the good image 10A is not affected by the defective pixels.

[0011] Furthermore, in the inverted image 10B, which is obtained by vertically inverting the normal image 10A, the defective pixels 11b and 12b corresponding to the defective pixels in the normal image 10A exist at coordinates that are vertically inverted from those of the defective pixels 11a and 12a. In this case, the inverted image 10B may have new defective pixels 13 due to the inversion of the image.

[0012] In this case, the defective pixels to be corrected by the imaging device are the defective pixels 11a and 12a included in the captured positive image 10A, and the defective pixels corrected by the inverted image 10B, which is obtained by flipping the positive image 10A vertically, are only the defective pixels 11b and 12b that correspond to the defective pixels included in the positive image 10A. In other words, since the newly created defective pixels 13 are not corrected, the inverted image 10B is affected by the defective pixels. Therefore, in the video system 1 of the embodiment of this disclosure, defective pixels 13 newly created by vertical inversion are corrected, and an image unaffected by defective pixels is output even when the video output inversion function is used.

[0013] [Embodiment] First, with reference to Figure 2, the video system 1 of the embodiment will be described. Figure 2 shows an example of video system 1. The video system 1 displays the video captured by the imaging device 100 on the PC 200. The video system 1 comprises an imaging device 100 and a PC 200, which are interconnected.

[0014] <Imaging device> The imaging device 100 is a device that captures images. The imaging device 100 is connected to the PC 200, and based on the operations from the PC 200, corrects the captured image and transmits the corrected image to the PC 200. Therefore, the imaging device 100 mainly includes a video output unit 110, a command analysis unit 120, an image sensor control unit 130, an image sensor 140, a defective pixel correction unit 150, and a storage unit 160.

[0015] <<Video output unit>> The video output unit 110 outputs a video signal for display on the monitor 210 of the PC 200 described later.

[0016] <<Command analysis unit>> The command analysis unit 120 receives the commands transmitted from the application 220 described later and transmits the operation information corresponding to the commands to the image sensor control unit 130.

[0017] <<Image sensor control unit>> The image sensor control unit 130 controls the image sensor 140 to execute the operations of the operation information transmitted from the command analysis unit 120. Specifically, the image sensor control unit 130 sets the values of the registers of the image sensor 140 so as to use the functions of the image sensor 140 specified by the command analysis unit 120, for example, the ROI function or the FLIP function.

[0018] <<Image sensor>> The image sensor 140 is also referred to as an imaging unit that can capture an image and output the captured normal image and the inverted image obtained by inverting the normal image. For example, it is a solid-state imaging device equipped with a CMOS (Complementary Metal-Oxcide-Semiconductor). The image sensor 140 arranges a large number of photodiodes, which are fine sensors, on a planar substrate, and forms an image on that plane using a lens or the like to perform photography. Furthermore, the image sensor 140 has an ROI function that outputs video within a specified range, and a FLIP function that inverts the video output, which can be used by setting values ​​in the registers.

[0019] Here, the pixels of the image sensor 140 refer to the smallest unit of the image detected by each photodiode, and are also called pixels. Furthermore, if a photodiode in the image sensor 140 malfunctions due to the manufacturing process or some other reason, it will output an abnormally high level of signal. In other words, it will become a defective pixel.

[0020] <<Defective Pixel Correction Unit>> The defective pixel correction unit 150 detects defective pixels in the video output from the image sensor 140, corrects them, and outputs them to the video output unit 110. Therefore, the defective pixel correction unit 150 mainly includes a defective pixel detection unit 151, a defective pixel information management unit 152, and a defective pixel correction execution unit 153.

[0021] <<<Defective Pixel Detection Unit>>> The defective pixel detection unit 151 detects the coordinates of a first defective pixel included in the positive image output from the image sensor 140, and similarly detects the coordinates of a second defective pixel included in the inverted image output from the image sensor 140. The defective pixel detection unit 151 transmits the detected first defective pixel coordinates and second defective pixel coordinates to the defective pixel information management unit 152.

[0022] <<<Defective Pixel Information Management Unit>>> The defective pixel information management unit 152 calculates the third defective pixel coordinates by inverting the second defective pixel coordinates and stores them in the storage unit 160, and similarly stores the first defective pixel coordinates in the storage unit 160.

[0023] Furthermore, the defective pixel information management unit 152 performs a merge process when storing the first defective pixel coordinates and the third defective pixel coordinates in the storage unit 160. Here, merging is the process of saving only one of the coordinates if there are duplicates. Furthermore, during the merging process, the coordinates of the first defective pixel and the coordinates of the third defective pixel may be rearranged in ascending order of their coordinates before processing. This merging process reduces the amount of data for defective pixel coordinates, thereby reducing the capacity of the storage unit 160 and lowering costs.

[0024] <<<Defective Pixel Correction Execution Unit>>> The defective pixel correction execution unit 153 corrects the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit 160.

[0025] <<Storage section>> The memory unit 160 stores the coordinates of the first defective pixel and the coordinates of the third defective pixel transmitted from the defective pixel information management unit 152. Furthermore, the memory unit 160 can also store the output level of the defective pixel corresponding to the first defective pixel coordinate and the output level of the defective pixel corresponding to the third defective pixel coordinate.

[0026] <pc> The PC200 is connected to the imaging device 100 and functions as an operating terminal for controlling the imaging device 100. Therefore, PC200 mainly includes a monitor 210 and an application 220.

[0027] <<Monitor>> The monitor 210 is a display unit that displays the video transmitted from the imaging device 100. Therefore, the monitor 210 displays the video based on the video signal transmitted from the video output unit 110.

[0028] <<Application>> Application 220 is a program for specifying the commands to be sent to the command analysis unit 120. Therefore, application 220 sends a command to execute a function of the imaging device 100 specified by the user, for example, a function to correct inverted images. The application 220 may also send commands to perform other functions, such as setting the values ​​of registers in the image sensor 140 that can perform ROI or FLIP functions.

[0029] <Conventional defective pixel correction process> Next, with reference to Figure 3, the conventional method for correcting defective pixels will be explained. Figure 3 is a flowchart showing the defective pixel correction process in a conventional method. In the following explanation, it is assumed that the image sensor 140 has a register set in advance with a value indicating whether or not to use the image inversion function, for example, the ROI function or the FLIP function.

[0030] (Step S101) In step S101, the command analysis unit 120 determines whether it has received a command from the application 220. If the command analysis unit 120 receives a command, it proceeds to the process in S102; otherwise, it terminates the correction process.

[0031] (Step S102) In step S102, the image sensor control unit 130 initializes the ON / OFF setting value of the ROI function of the image sensor. Specifically, the image sensor control unit 130 saves the values ​​of the registers related to the ROI function of the image sensor 140. The saved register values ​​are stored, for example, in memory, so that they can be read in the processing described later.

[0032] (Step S103) In step S103, the image sensor control unit 130 turns off the ROI function of the image sensor 140. Specifically, the image sensor control unit 130 sets the value of the register related to the ROI function of the image sensor 140 to a value that turns it OFF.

[0033] (Step S104) In step S104, similar to step S102, the image sensor control unit 130 initializes the ON / OFF setting value of the image sensor's FLIP function. Specifically, the image sensor control unit 130 saves the values ​​of the registers related to the FLIP function of the image sensor 140. The saved register values ​​are stored, for example, in memory, so that they can be read in the processing described later.

[0034] (Step S105) In step S105, similar to step S103, the image sensor control unit 130 turns off the FLIP function of the image sensor. Specifically, the image sensor control unit 130 sets the value of the register related to the FLIP function of the image sensor 140 to a value that turns it OFF.

[0035] (Step S106) In step S106, the defective pixel detection unit 151 detects the coordinates of a first defective pixel included in the positive image. At the same time, the defective pixel detection unit 151 also detects the output level of the defective pixel corresponding to the first defective pixel coordinates.

[0036] (Step S107) In step S107, the defective pixel information management unit 152 stores the detected first defective pixel coordinates in the storage unit 160. At the same time, it also stores the output level of the defective pixel corresponding to the first defective pixel coordinates.

[0037] (Step S108) In step S108, the image sensor 140 reads the value of the saved ON / OFF setting value register for the ROI function and sets it in the register.

[0038] (Step S109) In step S109, the image sensor 140 determines whether to use the ROI function based on the value of the set register. If the image sensor 140 uses the ROI function, the process proceeds to S110; otherwise, the process proceeds to S111.

[0039] (Step S110) In step S110, the defective pixel information management unit 152 subtracts the first defective pixel coordinates at the video output start position based on the ROI function, and calculates only the first defective pixel coordinates within the range of the video output width to be corrected.

[0040] (Step S111) In step S111, similar to step S108, the image sensor 140 reads the value of the saved ON / OFF setting value register for the FLIP function and sets it in the register.

[0041] (Step S112) In step S112, similar to step S109, the image sensor 140 determines whether to use the FLIP function based on the value of the set register. If the image sensor 140 uses the FLIP function, the process proceeds to S113; otherwise, the process proceeds to S114.

[0042] (Step S113) In step S113, the defective pixel information management unit 152 calculates a third defective pixel coordinate by inverting the first defective pixel coordinate vertically.

[0043] (Step S114) In step S114, the defective pixel correction execution unit 153 corrects the inverted image based on the third defective pixel coordinates.

[0044] <Defective pixel correction process in the present invention> Next, with reference to Figure 4, the defect pixel correction process in the present invention will be described. Figure 4 is a flowchart showing the defective pixel correction process in the present invention. In the following explanation, it is assumed that, similar to conventional defective pixel correction processing, the image sensor 140 has values ​​pre-set in its registers for image inversion functions, such as ROI function or FLIP function.

[0045] (Steps S101 to S106) Steps S101 to S106 are the same as the conventional defective pixel correction process, so their explanation will be omitted.

[0046] (Step S201) In step S201, the defective pixel information management unit 152 stores the detected first defective pixel coordinates in the storage unit 160. At the same time, it also stores the output level of the defective pixel corresponding to the first defective pixel coordinates.

[0047] (Step S202) In step S202, the image sensor control unit 130 turns on the FLIP function of the image sensor 140, inverting the output positive image vertically. Specifically, the image sensor control unit 130 sets the register value of the image sensor 140, inverts the normal image, and outputs the inverted image.

[0048] (Step S203) In step S203, the defective pixel detection unit 151 detects the coordinates of the second defective pixel and the output level of the defective pixel from the inverted image.

[0049] (Step S204) In step S204, the defective pixel information management unit 152 calculates a third defective pixel coordinate by inverting the second defective pixel coordinate vertically.

[0050] (Step S205) In step S205, the defective pixel information management unit 152 performs a merge process on the first defective pixel coordinates and the third defective pixel coordinates and stores them in the storage unit 160.

[0051] (Steps S108 to S113) Steps S108 to S113 are the same as the defective pixel correction process in the conventional method, so their explanation will be omitted.

[0052] (Step S206) In step S206, the defective pixel correction execution unit 153 corrects the inverted image based on the second defective pixel coordinates and the third defective pixel coordinates.

[0053] <Effects and Actions> The video system 1 according to this embodiment has been described above. The video system 1 of this disclosure mainly comprises a defective pixel detection unit 151, a defective pixel information management unit 152, and a defective pixel correction execution unit 153. It detects the coordinates of a second defective pixel included in the inverted image and corrects the defective pixel based on a third defective pixel coordinate obtained by inverting the coordinates of the second defective pixel included in the inverted image. This allows for the output of an unaffected inverted image, even if a defective pixel that does not exist in the original image appears in the inverted image.

[0054] Furthermore, the present invention can also take the following forms. (Aspect 1) An imaging unit capable of capturing an image and outputting the captured image and an inverted image obtained by flipping the image, A defect pixel detection unit that detects the coordinates of a first defect pixel included in the positive image and the coordinates of a second defect pixel included in the inverted image, A defective pixel information management unit calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate, A storage unit that stores the first defective pixel coordinates and the third defective pixel coordinates, A defective pixel correction execution unit corrects the defective pixels of the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit. An imaging device having (Aspect 2) The imaging device described in Embodiment 1, The storage unit is an imaging device that stores coordinates obtained by merging the first defective pixel coordinates and the third defective pixel coordinates. (Aspect 3) This is a method for correcting defective pixels in an inverted image obtained by flipping an captured image. A first step of detecting the coordinates of a first defective pixel included in the positive image, A second step of detecting the coordinates of a second defective pixel included in the inverted image, A third step of calculating a third defective pixel coordinate obtained by inverting the second defective pixel coordinate, A fourth step of storing the first defective pixel coordinates and the third defective pixel coordinates, The process includes a fifth step of correcting the defective pixels in the inverted image based on the stored first and third defective pixel coordinates. Method for correcting defective pixels. (Aspect 4) An image acquisition unit that acquires an image of the captured original image and an inverted image obtained by flipping the original image, A defect pixel detection unit that detects the coordinates of a first defect pixel included in the positive image and the coordinates of a second defect pixel included in the inverted image, A defective pixel information management unit calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate, A storage unit that stores the first defective pixel coordinates and the third defective pixel coordinates, A defective pixel correction execution unit corrects the defective pixels of the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit. An image processing device having (Appendix 5) An image processing apparatus according to Embodiment 4, The storage unit is an image processing device that stores coordinates obtained by merging the first defective pixel coordinates and the third defective pixel coordinates.

[0055] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the present invention. [Explanation of Symbols]

[0056] 1: Video system, 10A: Normal image, 10B: Inverted image, 11A, 11B, 12A, 12B, 13: Defective pixels, 100: Imaging device, 110: Video output unit, 120: Command analysis unit, 130: Image sensor control unit, 140: Image sensor, 150: Defective pixel correction unit, 151: Defective pixel detection unit, 152: Defective pixel information management unit, 153: Defective pixel correction execution unit, 200: PC, 210: Monitor, 220: Application< / pc>

Claims

1. An imaging unit capable of capturing an image and outputting the captured image and an inverted image obtained by flipping the image, A defective pixel detection unit detects a first defective pixel coordinate included in the positive image and a second defective pixel coordinate included in the inverted image. A defective pixel information management unit calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate, A storage unit that stores the first defective pixel coordinates and the third defective pixel coordinates, A defective pixel correction execution unit corrects the defective pixels of the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit. An imaging device having

2. The imaging apparatus according to claim 1, The storage unit is an imaging device that stores coordinates obtained by merging the first defective pixel coordinates and the third defective pixel coordinates.

3. This is a method for correcting defective pixels in an inverted image obtained by flipping an captured image. A first step of detecting the coordinates of a first defective pixel included in the positive image, A second step of detecting the coordinates of a second defective pixel included in the inverted image, A third step of calculating a third defective pixel coordinate obtained by inverting the second defective pixel coordinate, A fourth step of storing the first defective pixel coordinates and the third defective pixel coordinates, A fifth step of correcting the defective pixels in the inverted image based on the stored first defective pixel coordinates and third defective pixel coordinates, A method for correcting defective pixels, including [specific type of pixel].

4. An image acquisition unit that acquires an image of the captured original image and an inverted image obtained by flipping the original image, A defective pixel detection unit detects a first defective pixel coordinate included in the positive image and a second defective pixel coordinate included in the inverted image. A defective pixel information management unit calculates a third defective pixel coordinate obtained by inverting the second defective pixel coordinate, A storage unit that stores the first defective pixel coordinates and the third defective pixel coordinates, A defective pixel correction execution unit corrects the defective pixels of the inverted image based on the first defective pixel coordinates and the third defective pixel coordinates stored in the storage unit. An image processing device having

5. An image processing apparatus according to claim 4, The storage unit is an image processing device that stores coordinates obtained by merging the first defective pixel coordinates and the third defective pixel coordinates.

Images