Imaging system, method for controlling imaging system, and program

Abstract

To provide an imaging system capable of more effectively reducing a correction data capacity of shading correction.SOLUTION: An imaging system (1000) includes: an optical part (10); an imaging part (20) for photoelectrically converting an optical image formed by the optical part and outputting video data; a correction part (30) for performing the shading correction to the video data; and a storage part (40) for storing correction data used for the shading correction. The optical part forms the optical image having first resolution in a first area in a light-receiving surface of the imaging part and forms the optical image having second resolution lower than the first resolution in a second area in the light-receiving surface of the imaging part. The correction data includes the same correction value in a plurality of vicinity pixels in the second area of the imaging part.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an imaging system, a control method for an imaging system, and a program. [Background technology]

[0002] Due to the influence of uneven transmission of a lens, etc., the brightness of the periphery of an image is lower than that of the center, which is called a shading phenomenon. To correct the shading phenomenon, a shading correction method is known in which the brightness is increased by multiplying each pixel of an image sensor by a correction gain value. The correction gain value for each pixel is stored in advance in a memory, and correction is performed by reading the correction gain value from the memory. In recent years, there has been a trend toward higher resolution of image sensors, and the capacity of the correction data that stores the correction gain value for each pixel has been increasing accordingly. If the capacity of the correction data is large, the memory capacity needs to be increased, which increases costs.

[0003] Patent Document 1 discloses an imaging device that reduces the amount of correction data by dividing a screen into a plurality of correction blocks such that the size of the correction blocks becomes smaller toward the periphery of the screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-121612 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the imaging device disclosed in Patent Document 1, the amount of correction data is reduced without taking into consideration the optical characteristics of the lens, and therefore the effect of reducing the amount of correction data is not sufficient.

[0006] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide an imaging system capable of more effectively reducing the amount of correction data required for shading correction. [Means for solving the problem]

[0007] An imaging system as one aspect of the present invention has an optical unit, an imaging unit that photoelectrically converts an optical image formed by the optical unit to output video data, a correction unit that performs shading correction on the video data, and a memory unit that stores correction data used for the shading correction, wherein the optical unit forms the optical image of a first resolution in a first region on a light receiving surface of the imaging unit, and forms the optical image of a second resolution lower than the first resolution in a second region of the imaging unit, and the correction data includes the same correction value for multiple adjacent pixels in the second region of the imaging unit.

[0008] Other objects and features of the present invention will be described in the following embodiments. Effect of the Invention

[0009] According to the present invention, it is possible to provide an imaging system capable of more effectively reducing the amount of correction data required for shading correction. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram of an imaging system according to a first embodiment. [Diagram 2] 3A and 3B are explanatory diagrams of imaging ranges of a first angle of view and a second angle of view in the first embodiment. [Diagram 3] FIG. 3 is an explanatory diagram of the characteristics of an optical portion in the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram of shading correction in the first embodiment. [Diagram 5] 11A to 11C are diagrams illustrating shading correction characteristics when a correction block is enlarged in an area imaged at a second angle of view in the first embodiment. [Figure 6]11A and 11B are explanatory diagrams of shading correction characteristics in which the size of a correction block is made smaller as the image height increases in an area imaged at a second angle of view in the first embodiment. [Figure 7] 4 is a flowchart showing the operation of the imaging system in the first embodiment. [Figure 8] FIG. 11 is a block diagram of an imaging system according to a second embodiment. [Figure 9] 13 is an explanatory diagram of imaging ranges of a third angle of view and a fourth angle of view in the second embodiment. FIG. [Figure 10] FIG. 11 is an explanatory diagram of the characteristics of the optical portion in the second embodiment. [Figure 11] 13A to 13C are diagrams illustrating shading correction characteristics when a correction block is enlarged in an area imaged at a third angle of view in the second embodiment. [Figure 12] 10 is a flowchart showing the operation of an imaging system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In each drawing, the same reference numerals are used to refer to the same components, and duplicated explanations will be omitted.

[0012] (First embodiment) First, a first embodiment of the present invention will be described. In this embodiment, a correction data reduction method for shading correction that takes into account the optical characteristics of a lens will be described using an image capturing system for rear monitoring that is installed at the rear of a vehicle and has optical characteristics in which an image is formed at a high resolution on the central side of the optical axis and at a low resolution on the peripheral side.

[0013] 1 is a block diagram of an image capturing system 1000 according to this embodiment. The image capturing system 1000 captures and displays an image of the rear of the vehicle. The image capturing system 1000 is a camera-type rear-view mirror, generally called an electronic rear-view mirror, which captures an image of the rear of the vehicle using a camera installed at the rear of the vehicle and allows the image to be viewed on a monitor inside the vehicle. The image capturing system 1000 includes an optical unit 10, an image capturing unit 20, a shading correction unit (correction unit) 30, a storage unit 40, an image clipping processing unit 50, a rear monitor 60, an electronic rear-view mirror display unit 70, and a backing detection unit 80.

[0014] The optical unit 10 has a plurality of lenses, and forms two different angles of view on the light receiving surface of the imaging element of the imaging unit 20. One angle of view is an angle of view formed at a high resolution on the optical center side. This angle of view is defined as a first angle of view R1. The first angle of view R1 is an angle of view for checking the rear of the vehicle during normal driving. Since it serves as a substitute for a rearview mirror, it is imaged at a high resolution. The other angle of view is an angle of view formed at a low resolution and wide angle on the peripheral side of the imaging surface. This angle of view is defined as a second angle of view R2. The second angle of view R2 is an angle of view for checking the situation behind the vehicle at a wide angle when backing up. It is necessary to image at a wide angle to check whether there are people or obstacles behind the vehicle, but it is not necessary to have high resolution. FIG. 2 is an explanatory diagram of the imaging range of the first angle of view R1 and the second angle of view R2.

[0015] That is, the optical unit 10 forms an optical image with a first resolution (high resolution) in a first region on the light receiving surface of the imaging unit 20, and forms an optical image with a second resolution (low resolution) lower than the first resolution in a second region on the light receiving surface of the imaging unit 20. The first region corresponds to an imaging range of a first angle of view R1, and the second region corresponds to an imaging range of a second angle of view R2. In this embodiment, the first region is a central region including the center of the light receiving surface (imaging surface) of the imaging unit 20, and the second region is a peripheral region around the central region of the imaging unit 20.

[0016] Here, the optical characteristics of the optical unit 10 in this embodiment will be described with reference to Fig. 3(a) and (b). Fig. 3(a) is a diagram showing the image height y at each half angle of view on the imaging surface (light receiving surface) of the imaging unit 20 in the form of contour lines. Fig. 3(b) is a diagram showing the relationship between the half angle of view θ and the image height y (projection characteristics of the optical unit 10) in the first quadrant of Fig. 3(a).

[0017] As shown in FIG. 3(b), the optical unit 10 is configured such that its projection characteristic y(θ) differs between angles of view less than a predetermined half angle of view θa and angles of view equal to or greater than the half angle of view θa. Therefore, the optical unit 10 is configured such that the resolution differs depending on the angle of view (area on the light receiving surface of the imaging unit 20) when the increase in the image height y per unit half angle of view θ is taken as the resolution. This local resolution can be expressed as the differential value dy(θ) / dθ of the projection characteristic y(θ) at the half angle of view θ. For example, the greater the inclination of the projection characteristic y(θ) in FIG. 3(b), the higher the resolution. Also, in FIG. 3(a), the greater the interval between the contour lines of the image height y at each half angle of view, the higher the resolution.

[0018] The optical section 10 of this embodiment has a projection characteristic in which the rate of increase of image height y (the slope of the projection characteristic y(θ) in FIG. 3(b)) is large in the central region near the optical axis, and the rate of increase of image height y becomes smaller in the peripheral region outside the central region as the angle of view becomes larger.

[0019] 3A, a first region 301a (first imaging region) including the center corresponds to an angle of view less than a half angle of view θa, and a second region 301b (second imaging region) outside the first region 301a corresponds to an angle of view equal to or greater than the half angle of view θa. The angle of view less than the half angle of view θa corresponds to a first angle of view R1, and the angle of view equal to or greater than the half angle of view θa corresponds to a second angle of view R2.

[0020] As described above, the first region 301a is a relatively high-resolution region, and the second region 301b is a relatively low-resolution region. Also, the first region 301a is a low-distortion region with relatively little distortion, and the second region 301b is a high-distortion region with relatively much distortion. Therefore, in this embodiment, the first region 301a may be called a high-resolution region or a low-distortion region, and the second region 301b may be called a low-resolution region or a high-distortion region.

[0021] 3(a) and (b) are merely examples, and are not limited thereto. For example, the low-resolution region and the high-resolution region of the optical system do not have to be configured in a concentric shape, and each region may have a distorted shape. In addition, the center of gravity of the low-resolution region and the center of gravity of the high-resolution region do not have to coincide. In addition, the center of gravity of the low-resolution region and the center of gravity of the high-resolution region may be shifted from the center of the light receiving surface of the imaging unit 20.

[0022] In the optical section 10 of this embodiment, it is sufficient that the high resolution region is formed in the vicinity of the optical axis, and the low resolution region is formed on the peripheral side in the vicinity of the optical axis.

[0023] The optical unit 10 is configured so that the projection characteristic y(θ) in the first region 301a (high resolution region) is greater than f×θ (f is the focal length of the optical unit 10). Moreover, the projection characteristic y(θ) in the first region 301a (high resolution region) is set to be different from the projection characteristic in the second region 301b (low resolution region).

[0024] When θmax is the maximum half angle of view of the optical section 10, the ratio θa / θmax of θa to θmax is preferably equal to or greater than a predetermined lower limit, and for example, the predetermined lower limit is preferably 0.15 to 0.16. Also, the ratio θa / θmax of θa to θmax is preferably equal to or less than a predetermined upper limit, and for example, 0.25 to 0.35. For example, when θa is 90°, the predetermined lower limit is 0.15, and the predetermined upper limit is 0.35, it is preferable to determine θa in the range of 13.5 to 31.5°. However, the above is an example, and is not limited thereto.

[0025] Furthermore, the optical section 10 is configured so that its projection characteristic y(θ) satisfies the following formula (1).

[0026]

number

[0027] Here, f is the focal length of the optical unit 10 as described above, and A is a predetermined constant. By setting the lower limit to 1, it is possible to increase the central resolution compared to a fisheye lens of the orthogonal projection type (y=f×sinθ) having the same maximum imaging height, and by setting the upper limit to A, it is possible to obtain an angle of view equivalent to that of a fisheye lens while maintaining good optical performance. The predetermined constant A may be determined in consideration of the balance between the resolution of the high-resolution region and the low-resolution region, and is preferably set to 1.4 to 1.9. However, the above is just an example and is not limited to this.

[0028] By configuring the optical unit 10 as described above, a high resolution can be obtained in the first region 301a (high resolution region), while the increase in image height y per unit half angle of view θ can be reduced in the second region 301b (low resolution region), making it possible to capture a wider angle of view. Therefore, while the imaging range is a wide angle of view equivalent to that of a fisheye lens, high resolution can be obtained in the high resolution region.

[0029] The imaging unit 20 is an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. The imaging unit 20 photoelectrically converts an optical subject image (optical image) formed by the optical unit 10, and outputs the video data to the shading correction unit 30.

[0030] The shading correction unit 30 performs shading correction on the video data output from the imaging unit 20. Due to the influence of uneven transmission of the optical unit 10, etc., the amount of light in the peripheral area of ​​the imaging surface is lower than in the center, resulting in a shading phenomenon that becomes darker. Correcting the video data by applying a gain to the luminance of each pixel of the imaging unit 20 to cancel out such a shading phenomenon is called shading correction.

[0031] FIG. 4 is an explanatory diagram of shading correction, showing the gain characteristics of shading correction according to image height, the light amount characteristics due to the shading phenomenon, and the luminance after shading correction. In FIG. 4, the horizontal axis shows image height, and the vertical axis shows light amount / gain / luminance. As shown in FIG. 4, the shading phenomenon causes the light amount to decrease as the image height increases. Therefore, by performing shading correction so as to increase the gain as the image height increases, the luminance becomes approximately constant regardless of the image height.

[0032] In this way, the shading correction unit 30 corrects the image data so as to cancel out the peripheral light loss caused by the shading phenomenon. The shading correction unit 30 reads out the correction data (correction value) stored in the storage unit 40, and performs shading correction using the read out correction data. In addition, as correction processing to make it easier for the user to view the captured image, processing such as WDR (Wide Dynamic Range) correction, gamma correction, LUT processing, distortion correction, etc. may be performed. The image processed by the shading correction unit 30 is output to the image cut-out processing unit 50.

[0033] The storage unit 40 stores correction data used when performing shading correction in the shading correction unit 30. The storage unit 40 is, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory, but is not limited to these.

[0034] The correction data stored in the storage unit 40 is preferably assigned a correction gain value for each pixel of the imaging unit 20. However, if a correction gain value is assigned for each pixel, the capacity of the correction data increases. Therefore, in this embodiment, the same gain value is applied to multiple pixels (neighboring pixels). In this embodiment, a collection of multiple neighboring pixels (unit of the number of pixels of multiple neighboring pixels) to which the same gain value is applied is called a correction block. By making the correction block larger, the number of pixels to which the same gain value is applied increases. As a result, the number of gain values ​​stored in the storage unit 40 decreases, and the capacity of the storage unit 40 required to store the correction data can be reduced.

[0035] However, when the correction block is made larger, the change in gain value becomes larger at the boundary between adjacent correction blocks. In this regard, in the area imaged at the second angle of view R2 with low resolution, the MTF is low and the difference in luminance becomes unclear. Therefore, even if the change in gain value becomes large at the boundary between adjacent correction blocks, the effect on the corrected image data is small. Therefore, the correction block can be made larger than the area imaged at high resolution, and the effect of reducing the capacity of the memory unit 40 can be improved.

[0036] Fig. 5 is an explanatory diagram of shading correction characteristics when the correction block is enlarged in the area imaged at the second angle of view R2. In Fig. 5, the horizontal axis indicates image height, and the vertical axis indicates gain. Fig. 5 shows an example in which the correction block (unit of pixel count of multiple neighboring pixels to which the same correction value is applied) is constant (the size of the correction block is constant) regardless of the image height. However, this embodiment is not limited to the case where the unit of pixel count is constant, and the unit of pixel count may be within a predetermined range regardless of the image height (for example, the unit of pixel count may be a partially different unit such as 5, 6, 5, 5, 5, 5).

[0037] In the shading correction characteristic shown in Fig. 5, the change in gain value at the boundary between adjacent correction blocks increases as the image height increases. Therefore, even in an area imaged at the second angle of view R2 with low resolution, there are cases where the influence on the image data after shading correction, such as the occurrence of a luminance difference at the boundary between adjacent correction blocks, cannot be ignored.

[0038] In this case, by changing the size of the correction block (unit of the number of pixels of multiple neighboring pixels to which the same correction value is used) according to the image height, it is possible to reduce the capacity of the storage unit 40 that stores the correction data while further reducing the influence on the image data after shading correction. FIG. 6 is an explanatory diagram of the shading correction characteristic in which the correction block is made smaller as the image height increases in the area imaged at the second angle of view R2 with low resolution. In FIG. 6, the horizontal axis indicates the image height, and the vertical axis indicates the gain. Note that in this embodiment, the gain value applied to each correction block can be the average value of the gain values ​​assigned to each pixel, but is not limited to this.

[0039] 6, the correction block (unit of the number of pixels of multiple neighboring pixels using the same correction value) becomes smaller as the image height increases, but this embodiment is not limited to this. For example, the unit of the number of pixels of multiple neighboring pixels using the same correction value may be a first unit at a first image height, and may be a second unit smaller than the first unit at a second image height higher than the first image height. In other words, the unit of the number of pixels may be 5, 4, 4, 3, 2, 1, with some of the same units continuing.

[0040] The image cutout processing unit 50 cuts out a part of the image data processed by the shading correction unit 30. The image data is cut out by storing the image data processed by the shading correction unit 30 in a memory such as a RAM, and reading out only the image data of the part to be cut out from the memory. The position to be cut out is an image for observing a vehicle behind the vehicle during normal driving, among the area imaged at the first angle of view R1 with high resolution. The image cutout processing unit 50 outputs the cut-out image data for observing a vehicle behind the vehicle during normal driving to the electronic rear-view mirror display unit 70. In addition, when the backing driving detection unit 80 detects backing driving, the image cutout processing unit 50 cuts out the image data for observing the rear of the vehicle at a wide angle during backing driving from the image data processed by the shading correction unit 30, and outputs it to the rear monitor 60.

[0041] The rear monitor 60 is a display unit such as a liquid crystal display. The rear monitor 60 displays an image cut out by the image cut-out processing unit 50 for observing the rear of the vehicle at a wide angle when backing up. The electronic rear-view mirror display unit 70 is a display unit such as a liquid crystal display. The electronic rear-view mirror display unit 70 displays an image cut out by the image cut-out processing unit 50 for observing a vehicle behind the vehicle when backing up. The backing up detection unit 80 detects when the vehicle has switched to backing up, specifically, when the shift has been put into reverse.

[0042] Next, the operation of the rear monitoring imaging system 1000 in this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the operation of the imaging system 1000.

[0043] First, in step S1, the imaging unit 20 transmits captured video data (imaging data) to the shading correction unit 30. Then, in step S2, the shading correction unit 30 reads out correction data (correction values) used during shading correction from the storage unit 40. Then, in step S3, the shading correction unit 30 performs shading correction on the video data transmitted from the imaging unit 20 using the correction data, and transmits the corrected video data to the video cut-out processing unit 50.

[0044] Next, in step S4, the video clipping processor 50 determines whether or not the backing detector 80 has detected backing. If it is determined that the backing detector 80 has detected backing, the process proceeds to step S5 and step S7. On the other hand, if it is determined that the backing detector 80 has not detected backing, the process proceeds to step S5.

[0045] In step S5, the image cut-out processing unit 50 cuts out an image (high-resolution image, image with a first angle of view R1) for observing a vehicle behind the host vehicle in high resolution, and transmits the high-resolution image to the electronic rear-view mirror display unit 70. Step S5 is performed regardless of whether or not the backing detection unit 80 detects backing in step S4. Then, in step S6, the image cut-out processing unit 50 displays the image cut out in step S5 on the electronic rear-view mirror display unit 70. Step S6 is performed regardless of whether or not the backing detection unit 80 detects backing in step S4.

[0046] In step S7, the image cutout processing unit 50 cuts out an image for observing the rear of the vehicle at a wide angle when the vehicle is backing up (wide-angle image, image with the second angle of view R2), and transmits the wide-angle image to the rear monitor 60. Subsequently, in step S8, the image cutout processing unit 50 displays the image cut out in step S7 on the rear monitor 60.

[0047] In this embodiment, when the backing detection unit 80 detects backing, the image cut-out processing unit 50 cuts out the image to observe the rear of the vehicle at a wide angle. However, this embodiment is not limited to this, and the image captured by the imaging unit 20 may be displayed on the rear monitor 60 without being cut out.

[0048] In the present embodiment, the electronic rearview mirror display unit 70 and the rear monitor 60 are described as being configured separately. However, the present embodiment is not limited to this, and the image displayed on the rear monitor 60 may be displayed on the electronic rearview mirror display unit 70 in PinP (Picture in Picture) or PbyP (Picture by Picture).

[0049] In this embodiment, as a method for reducing correction data for shading correction that takes into account the optical characteristics of the lens, an image capturing system for rear monitoring that is installed at the rear of a vehicle and has optical characteristics in which an image is formed at a high resolution on the optical axis center side and at a low resolution on the peripheral side has been described as an example. According to this embodiment, by making the correction block for the area that is imaged at a low resolution on the peripheral side of the imaging surface larger, it is possible to reduce the amount of correction data for shading correction while reducing the effect on the video data.

[0050] Second embodiment Next, a second embodiment of the present invention will be described. In this embodiment, a correction data reduction method for shading correction that takes into account the optical characteristics of a lens will be described using an image pickup system for monitoring the rear side, which is installed on the side of a vehicle and has optical characteristics such that an image is formed at a low resolution on the central side of the optical axis and at a high resolution on the peripheral side.

[0051] 8 is a block diagram of an imaging system 2000 in this embodiment. The imaging system 2000 captures and displays images of the rear side of the vehicle. The imaging system 2000 is a camera-type side mirror, generally called an electronic side mirror, which captures images of the rear side of the vehicle using a camera installed on the side of the vehicle and allows the image to be confirmed on a monitor inside the vehicle. The imaging system 2000 has an optical unit 10, an imaging unit 20, a shading correction unit (correction unit) 30, a storage unit 40, an image clipping processing unit 50, an electronic side mirror display unit 90, a front side display unit 100, and a front side display detection unit 110.

[0052] The optical unit 10 has a plurality of lenses, and forms two different angles of view on the imaging unit 20. One angle of view is an angle of view formed at a low resolution on the optical center side. This angle of view is defined as the first angle of view R3. The third angle of view R3 is an angle of view for checking the state of the side of the vehicle, although a specific use example is not described in this embodiment. It is not necessary to have high resolution because it is sufficient to check whether there are people or obstacles on the side of the vehicle. The other angle of view is an angle of view formed at a high resolution on the peripheral side of the imaging surface. This angle of view is defined as the fourth angle of view R4. The fourth angle of view R4 is an angle of view for checking the rear side of the vehicle during normal driving. Since it serves as a substitute for a side mirror, it is imaged at high resolution. FIG. 9 is an explanatory diagram of the imaging range of the third angle of view R3 and the fourth angle of view R4. In FIG. 9, the camera of the imaging system 2000 is arranged only on the right side of the vehicle, but a camera is also arranged on the left side of the vehicle. These are symmetrical, so they perform the same operation.

[0053] That is, the optical unit 10 forms an optical image with a first resolution (high resolution) in a first region of the imaging unit 20, and forms an optical image with a second resolution (low resolution) lower than the first resolution in a second region of the imaging unit 20. The first region corresponds to an imaging range of a first angle of view R4, and the second region corresponds to an imaging range of a third angle of view R3. In this embodiment, the second region is a central region including the center of the light receiving surface (imaging surface) of the imaging unit 20, and the first region is a peripheral region surrounding the central region of the imaging unit 20.

[0054] Here, the optical characteristics of the optical system in the optical unit 10 in this embodiment will be described with reference to Fig. 10(a) and (b). Fig. 10(a) is a diagram showing the image height y at each half angle of view on the imaging surface (light receiving surface) of the imaging unit 20 in the form of contour lines. Fig. 10(b) is a diagram showing the relationship between the half angle of view θ and the image height y (projection characteristics of the optical unit 10) in the first quadrant of Fig. 10(a).

[0055] As shown in FIG. 10(b), the optical unit 10 is configured such that its projection characteristic y(θ) differs between angles of view less than a predetermined half angle of view θa and angles of view equal to or greater than the half angle of view θa. Therefore, the optical unit 10 is configured such that, when the increase in image height y per unit of half angle of view θ is taken as the resolution, the resolution differs depending on the angle of view (area on the light receiving surface of the imaging unit 20). This local resolution can be expressed as a differential value dy(θ) / dθ of the projection characteristic y(θ) at the half angle of view θ. For example, the greater the inclination of the projection characteristic y(θ) in FIG. 10(b), the higher the resolution. Also, FIG. 10(a) shows that the greater the interval between the contour lines of the image height y at each half angle of view, the higher the resolution.

[0056] The optical section 10 of this embodiment has a projection characteristic in which the rate of increase of image height y (the slope of the projection characteristic y(θ) in FIG. 10(b)) is small in the central region near the optical axis, and the rate of increase of image height y becomes larger as the angle of view becomes larger in the peripheral region outside the central region.

[0057] 10(a), a first region 1001a including the center corresponds to an angle of view less than a half angle of view θa, and a second region 1001b outside the first region 1001a corresponds to an angle of view equal to or greater than the half angle of view θa. The angle of view less than the half angle of view θa corresponds to the central angle of view (third angle of view R3) of the imaging region, and the angle of view equal to or greater than the half angle of view θa corresponds to the peripheral angle of view (fourth angle of view R4) of the imaging region.

[0058] As described above, the first region 1001a is a relatively low-resolution region, and the second region 1001b is a relatively high-resolution region. Also, the first region 1001a is a high-distortion region with relatively more distortion, and the second region 1001b is a low-distortion region with relatively less distortion. Therefore, in this embodiment, the first region 1001a may be called a low-resolution region or a high-distortion region, and the second region 1001b may be called a high-resolution region or a low-distortion region.

[0059] The characteristics shown in Figs. 10(a) and (b) are merely examples, and are not limited thereto. For example, the low-resolution region and the high-resolution region of the optical section 10 may not be concentrically configured, and each region may have a distorted shape. The center of gravity of the low-resolution region and the center of gravity of the high-resolution region may not coincide. The center of gravity of the low-resolution region and the center of gravity of the high-resolution region may be shifted from the center of the light receiving surface of the image sensor. In the optical section 10 of this embodiment, it is sufficient that the low-resolution region is formed near the optical axis, and the high-resolution region is formed on the peripheral side near the optical axis.

[0060] The optical unit 10 is configured to satisfy the following formula (2), where f is the focal length, θ is the half angle of view, y is the image height on the image plane, y(θ) is a projection characteristic that indicates the relationship between the image height y and the half angle of view θ, and θmax is the maximum half angle of view that the optical system has. That is, the optical unit 10 is configured so that the projection characteristic y(θ) is different from 2ftan(θ / 2) (stereoscopic projection method).

[0061] 0.2<2×f×tan(θmax / 2) / y(θmax) <0.92 ···(2) In the optical section 10 having such optical characteristics, the magnification in the radial direction with respect to the optical axis can be adjusted by adjusting the projection characteristic y(θ). This makes it possible to control the aspect ratio in the radial direction and the circumferential direction with respect to the optical axis, so that, unlike conventional fisheye lenses, it is possible to obtain a high-resolution image with little distortion in the peripheral area even with a wide angle of view.

[0062] Furthermore, by satisfying formula (2), the resolution in the second region 1001b can be made higher than that of a stereoscopic projection optical system. If the upper limit of formula (2) is exceeded, the resolution in the second region 1001b decreases, and the difference with the resolution in the first region 1001a becomes small, which is not preferable. If the lower limit of formula (2) is exceeded, it becomes difficult to satisfactorily correct various aberrations such as field curvature, which is also not preferable.

[0063] It should be noted that the above-mentioned formula (2) is merely an example, and the optical system in this embodiment is not limited to this.

[0064] By configuring the optical system as described above, high resolution can be obtained in the high resolution region, while the increase in image height y per unit half angle of view θ can be reduced in the low resolution region, making it possible to capture a wider angle of view. Therefore, while providing an imaging range with a wide angle of view equivalent to that of a fisheye lens, high resolution can be obtained in the high resolution region.

[0065] In this embodiment, the projection characteristics in the high-resolution region (low-distortion region) are approximate to the central projection method (y=f×tanθ) and equidistant projection method (y=f×θ), which are the projection characteristics of a general imaging optical system. Therefore, in the high-resolution region (low-distortion region), optical distortion is small, making it possible to generate a fine image.

[0066] The storage unit 40 stores correction data used when performing shading correction in the shading correction unit 30. The storage unit 40 is, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory, but is not limited to these.

[0067] The correction data stored in the storage unit 40 is preferably assigned a correction gain value for each pixel of the imaging unit 20. However, if a correction gain value is assigned for each pixel, the capacity of the correction data increases. Therefore, in this embodiment, the same gain value is applied to multiple pixels (neighboring pixels). In this embodiment, a collection of multiple neighboring pixels to which the same gain value is applied is called a correction block. By making the correction block larger, the number of pixels to which the same gain value is applied increases. As a result, the number of gain values ​​stored in the storage unit 40 decreases, and the capacity of the storage unit 40 required to store the correction data can be reduced.

[0068] However, when the correction block is made larger, the change in gain value becomes larger at the boundary between adjacent correction blocks. In this regard, in the area imaged at the third angle of view R3 with low resolution, the MTF is low and the difference in brightness becomes unclear. Therefore, even if the change in gain value becomes large at the boundary between adjacent correction blocks, the effect on the corrected image data is small. Therefore, the correction block can be made larger than the area imaged at high resolution, and the effect of reducing the capacity of the storage unit 40 can be improved.

[0069] Fig. 11 is an explanatory diagram of shading correction characteristics when the correction block is enlarged in an area imaged at the third angle of view R3 with low resolution. In Fig. 11, the horizontal axis indicates image height, and the vertical axis indicates gain. Note that in this embodiment, the gain value applied to each correction block can be the average value of the gain values ​​assigned to each pixel, but is not limited to this.

[0070] The image cutout processing unit 50 cuts out a part of the image data processed by the shading correction unit 30. The image data is cut out by storing the image data processed by the shading correction unit 30 in a memory such as a RAM, and reading out only the image data of the part to be cut out from the memory. The position to be cut out is an image for observing the rear side of the vehicle during normal driving, among the area imaged at the high-resolution fourth angle of view R4. The image cutout processing unit 50 outputs the cut-out image data for observing the rear side of the vehicle during normal driving to the electronic side mirror display unit 90. Furthermore, when the front side display detection unit 110 determines that the driver wants to display the front side of the vehicle, the image cutout processing unit 50 cuts out an image for observing the front side of the vehicle, among the fourth angle of view R4 imaged at high resolution on the periphery of the imaging surface. Then, the image cutout processing unit 50 outputs the cut-out image to the front side display unit 100.

[0071] The electronic side mirror display unit 90 is a display unit such as a liquid crystal display. The electronic side mirror display unit 90 displays an image cut out by the image cut-out processing unit 50 for observing the rear side of the vehicle during normal driving. The front side display unit 100 is a display unit such as a liquid crystal display. The front side display unit 100 displays an image cut out by the image cut-out processing unit 50 for observing the front side of the vehicle.

[0072] The front side display detection unit 110 detects that the driver wishes to display the front side of the vehicle by operating a button or the like. For example, when the driver is moving to the side of the vehicle, the driver may wish to view the front side of the vehicle, such as the area around the front tires, which is a blind spot for the driver. In this case, the driver operates a button, and the front side display detection unit 110 detects that the driver wishes to display the front side of the vehicle. In addition to the driver's intention, the front side display detection unit 110 may also be triggered to display the front side of the vehicle by detecting the approach of a gutter or an obstacle to the side of the vehicle. The imaging unit 20 and the shading correction unit 30 are the same as those in the first embodiment, and their description will be omitted.

[0073] Next, the operation of the imaging system 2000 for monitoring the rear side in this embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the operation of the imaging system 2000. Note that steps S1 to S3 in Fig. 12 are common to steps S1 to S3 in Fig. 7, respectively, and therefore their description will be omitted.

[0074] In step S9, the image clipping processor 50 judges whether the front side display detector 110 has detected that the driver wishes to display the front side of the vehicle. If the front side display detector 110 has detected that the driver wishes to display the front side of the vehicle, the process proceeds to step S10 and step S12. On the other hand, if the front side display detector 110 has not detected that the driver wishes to display the front side of the vehicle, the process proceeds to step S10.

[0075] In step S10, the image clipping processing unit 50 clips out an image (rear side image) for observing a vehicle behind the vehicle at high resolution, and transmits the rear side image to the electronic side mirror display unit 90. Step S10 is performed regardless of whether the front side display detection unit 110 detects in step S9 that the driver wishes to display the front side of the vehicle. Then, in step S11, the image clipping processing unit 50 displays the image clipped in step S10 on the electronic side mirror display unit 90. Step S11 is performed regardless of whether the front side display detection unit 110 detects in step S9 that the driver wishes to display the front side of the vehicle.

[0076] In step S12, the image cutout processing unit 50 cuts out an image (front-side image) for observing a vehicle in front of the vehicle at high resolution, and transmits the front-side image to the front-side display unit 100. Subsequently, in step S13, the image cutout processing unit 50 displays the image cut out in step S12 on the front-side display unit 100.

[0077] In the present embodiment, the electronic side mirror display unit 90 and the front side display unit 100 are described as being configured separately. However, the present embodiment is not limited to this, and the image displayed on the front side display unit 100 may be displayed on the electronic side mirror display unit 90 in PinP (Picture in Picture) or PbyP (Picture by Picture).

[0078] In this embodiment, as a method for reducing correction data for shading correction that takes into account the optical characteristics of the lens, an image pickup system for monitoring the rear side is described as an example, which is installed on the side of a vehicle and has optical characteristics such that an image is formed at a low resolution on the optical axis center side and at a high resolution on the peripheral side. According to this embodiment, by making the correction block of the area that is imaged at a low resolution on the optical center side of the imaging surface larger, it is possible to reduce the amount of correction data for shading correction while reducing the effect on the video data.

[0079] Although the shading correction in each embodiment has been described as correction for luminance, shading correction may be performed for each RGB color depending on the optical characteristics of the lens and the size of the image sensor. In that case, the correction data reduction method in each embodiment may be applied to the correction data for each RGB color.

[0080] In each embodiment, a method for reducing the amount of correction data while minimizing the effect on the corrected image data by enlarging the correction block in an area imaged at a low resolution has been described. However, each embodiment is not limited to this, and the amount of correction data may be reduced by applying the method of each embodiment to an area imaged at a high resolution.

[0081] In each embodiment, the high-resolution angle of view and the angle of view from which the image is cropped are described as being the same, but this is not limited thereto. For example, a part of the high-resolution angle of view may be cropped and displayed.

[0082] In each embodiment, when illustrating the characteristics of the shading phenomenon and shading correction in a diagram, for convenience, the light amount / gain is described as being rotationally symmetric with respect to the optical axis center. However, each embodiment is not limited to lenses having rotationally symmetric optical characteristics, and can also be applied to lenses having rotationally asymmetric optical characteristics.

[0083] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0084] In each embodiment, the correction data for shading correction is reduced by taking into account the optical characteristics of the lens. Therefore, according to each embodiment, it is possible to provide an imaging system, a control method for an imaging system, and a program that can more effectively reduce the amount of correction data for shading correction.

[0085] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) An optical section; an imaging unit that photoelectrically converts the optical image formed by the optical unit and outputs video data; a correction unit that performs shading correction on the video data; a storage unit that stores correction data used for the shading correction, the optical unit forms the optical image with a first resolution in a first region on the light receiving surface of the imaging unit, and forms the optical image with a second resolution lower than the first resolution in a second region on the light receiving surface of the imaging unit; The imaging system, wherein the correction data includes the same correction value for a plurality of adjacent pixels in the second region of the imaging section. (Configuration 2) the first region is a central region including a center of the light receiving surface of the imaging unit, 2. The imaging system according to configuration 1, wherein the second region is a peripheral region surrounding the central region of the imaging unit. (Configuration 3) 3. The imaging system according to configuration 2, wherein the unit of the number of pixels of the plurality of neighboring pixels for which the same correction value is used is within a predetermined range regardless of image height. (Configuration 4) 4. The imaging system according to configuration 3, wherein the unit of the number of pixels of the plurality of neighboring pixels is constant regardless of the image height. (Configuration 5) The unit of the number of pixels of the plurality of neighboring pixels to which the same correction value is used is is the first unit at the first image height, 3. The imaging system according to configuration 2, wherein at a second image height higher than the first image height, the second unit is smaller than the first unit. (Configuration 6) 6. The imaging system according to configuration 5, wherein the unit of the number of pixels of the plurality of neighboring pixels becomes smaller as the image height becomes higher. (Configuration 7) the second region is a central region including a center of a light receiving surface of the imaging unit, 2. The imaging system according to configuration 1, wherein the first region is a peripheral region surrounding the central region of the imaging unit. (Configuration 8) 8. The imaging system according to configuration 7, wherein the unit of the number of pixels of the plurality of neighboring pixels for which the same correction value is used is within a predetermined range regardless of image height. (Configuration 9) 9. The imaging system according to configuration 8, wherein the unit of the number of pixels of the plurality of neighboring pixels is constant regardless of the image height. (Configuration 10) 10. The imaging system according to any one of configurations 1 to 9, wherein the correction data includes a different correction value for each pixel in the first region. (Configuration 11) 11. The imaging system according to any one of configurations 1 to 10, wherein the correction data is a gain value to be applied to the video data. (Method 1) using the imaging unit to photoelectrically convert the optical image formed by the optical unit and output video data; reading out correction data used for shading correction from a storage unit; performing the shading correction on the image data using the correction data; The optical image is an optical image having a first resolution formed in a first region on a light receiving surface of the imaging unit; an optical image having a second resolution lower than the first resolution, the optical image being formed in a second region on the light receiving surface of the imaging unit; the correction data includes a same correction value for a plurality of neighboring pixels in the second region of the imaging unit; 23. A method for controlling an imaging system comprising: (Configuration 12) A program for causing a computer to execute the control method for an imaging system according to Method 1.

[0086] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. Parts of the above-described embodiments may be combined as appropriate. [Explanation of symbols]

[0087] 1000, 2000 Imaging System 10 Optics Department 20 Imaging unit 30 Shading correction section (correction section) 40 Storage section

Claims

1. An optical section; an imaging unit that photoelectrically converts the optical image formed by the optical unit and outputs video data; a correction unit that performs shading correction on the video data; A storage unit that stores correction data used for the shading correction, the optical unit forms the optical image with a first resolution in a first region on the light receiving surface of the imaging unit, and forms the optical image with a second resolution lower than the first resolution in a second region on the light receiving surface of the imaging unit; The imaging system, wherein the correction data includes the same correction value for a plurality of adjacent pixels in the second region of the imaging section.

2. the first region is a central region including a center of a light receiving surface of the imaging unit, The imaging system according to claim 1 , wherein the second region is a peripheral region surrounding the central region of the imaging unit.

3. 3. The imaging system according to claim 2, wherein the unit of the number of pixels of the plurality of neighboring pixels for which the same correction value is used is within a predetermined range regardless of image height.

4. 4. The imaging system according to claim 3, wherein the unit of the number of pixels of the plurality of neighboring pixels is constant regardless of the image height.

5. The unit of the number of pixels of the plurality of neighboring pixels to which the same correction value is used is a first unit at a first image height, 3. The imaging system according to claim 2, wherein at a second image height higher than the first image height, the second unit is smaller than the first unit.

6. 6. The imaging system according to claim 5, wherein the unit of the number of pixels of the plurality of neighboring pixels decreases as the image height increases.

7. the second region is a central region including a center of a light receiving surface of the imaging unit, The imaging system according to claim 1 , wherein the first region is a peripheral region surrounding the central region of the imaging unit.

8. 8. The imaging system according to claim 7, wherein the unit of the number of pixels of the plurality of neighboring pixels for which the same correction value is used is within a predetermined range regardless of image height.

9. 9. The imaging system according to claim 8, wherein the unit of the number of pixels of the plurality of neighboring pixels is constant regardless of the image height.

10. The imaging system according to claim 1 , wherein the correction data includes a different correction value for each pixel in the first region.

11. 10. The imaging system according to claim 1, wherein the correction data is a gain value applied to the video data.

12. using the imaging unit to photoelectrically convert the optical image formed by the optical unit and output video data; reading out correction data used for shading correction from a storage unit; performing the shading correction on the image data using the correction data; The optical image is an optical image having a first resolution formed in a first region on a light receiving surface of the imaging unit; an optical image having a second resolution lower than the first resolution, the optical image being formed in a second region on the light receiving surface of the imaging unit; the correction data includes a same correction value for a plurality of neighboring pixels in the second region of the imaging unit; 23. A method for controlling an imaging system comprising:

13. A program for causing a computer to execute the method for controlling an image pickup system according to claim 12.

Images