Imaging setting device, imaging device, and imaging setting method

Abstract

To allow an imaging apparatus to simultaneously perform imaging with an appropriate resolution and frame rate and imaging with an appropriate angle of view, while preventing an increase in the load on image processing.SOLUTION: Of a first pixel area 12a where signal reading with a first frame rate can be performed on a single image pick-up device 12 picking up a subject image formed by a single optical system 11, and a second pixel area 12b where signal reading with a second frame rate faster than the first frame rate can be performed, an imaging setting device sets at least the second pixel area. When the optical system has characteristics in which a resolution that is the number of pixels of the subject image per unit angle of view on the image pick-up device is different depending on the angle of view, the imaging setting device sets the second pixel area such that the lowest resolution in one pixel area of the first and second pixel areas becomes higher than the lowest resolution in the other pixel area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device used for an in-vehicle camera or the like.

Background Art

[0002] In an in-vehicle camera monitoring system, for example, it is required to capture and display a room mirror image at a high resolution and a high frame rate, and to capture and display a back monitor image at a wide angle of view.

[0003] Patent Document 1 discloses a camera capable of simultaneously generating two images by reading signals from two pixel regions in a single image sensor at different frame rates. Patent Document 2 discloses a camera capable of simultaneously performing high-resolution imaging and wide-angle imaging using one optical system and one image sensor having different characteristics depending on the angle of view.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document 1 discloses that a pixel region for reading a signal at a high frame rate among two pixel regions in an image sensor is set in a region where a moving subject is detected. However, there is no disclosure or the like regarding a combination with an optical system having different characteristics depending on the angle of view as in Patent Document 2.

[0006] On the one hand, when the camera of Patent Document 2 is optimized for the requirements of the room mirror image, since it is necessary to process the wide-angle back monitor image at the same high frame rate as the room mirror image, the load of image processing increases. On the other hand, when the same camera is optimized for the requirements of the back monitor image, the high resolution and high frame rate required for the room mirror image cannot be obtained.

[0007] The present invention provides an imaging setting device that enables an imaging device to perform imaging with appropriate resolution and frame rate while suppressing an increase in the load of image processing.

Means for Solving the Problems

[0008] The imaging Image mounting arrangement according to one aspect of the present invention includes at least a first pixel region capable of signal readout at a first frame rate and a second pixel region capable of signal readout at a second frame rate faster than the first frame rate on a single imaging element that images a subject image formed by a single optical system. The foregoing Setting means for setting the second pixel region and the optical system, the imaging element, and image generation means for generating a first image at the first frame rate using the signal read from the first pixel region and generating a second image at the second frame rate using the signal read from the second pixel region, and the foregoing The setting means The foregoing When the optical system has the characteristic that the resolution, which is the number of pixels of the subject image on the imaging element per unit angle of view, varies depending on the angle of view, The foregoing on the imaging element The foregoing the second pixel region is set so that the lowest resolution in one of the first and second pixel regions is higher than the lowest resolution in the other pixel region. The foregoing This is characterized by The foregoing setting the second pixel region and the image generation means generates the first image and the second image simultaneously. Note that a moving body equipped with the above Recording imaging device also constitutes another aspect of the present invention.

[0009] Also, as another aspect of the present invention, the imaging Image sideThe method sets at least the second pixel region out of a first pixel region capable of signal readout at a first frame rate and a second pixel region capable of signal readout at a second frame rate higher than the first frame rate, on a single image sensor that captures a subject image formed by a single optical system. Setting Step and an image generation step of generating a first image at the first frame rate using the signal read from the first pixel region and generating a second image at the second frame rate using the signal read from the second pixel region, and the setting In the step, when the optical system has characteristics in which the resolution, which is the number of pixels of the subject image on the image sensor per unit angular field of view, varies depending on the angular field of view, the second pixel region is set so that the lowest resolution in one of the first and second pixel regions is higher than the lowest resolution in the other pixel region. and the image generation step generates the first image and the second image simultaneously. It is characterized by this. Note that the above imaging Image side A program for causing a computer to execute the processing according to the method also constitutes another aspect of the present invention.

Effect of the Invention

[0010] According to the present invention, it is possible to cause the imaging device to perform imaging with appropriate resolution and frame rate while suppressing an increase in the load of image processing.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [Example 1] FIG. 1 shows the configuration of an in-vehicle camera monitoring system according to Example 1 of the present invention. The system is mounted on a vehicle as a moving body, and includes an in-vehicle camera 10, an electronic rearview mirror 16, and an in-dash monitor 17. The in-vehicle camera 10 includes an optical system 11, an image sensor 12, an image processing unit 13, an output unit 14, and a setting unit 15 as an imaging setting device (setting means). Note that the camera monitoring system may be mounted on a moving body other than a vehicle.

[0013] As shown in FIG. 2(a), the in-vehicle camera 10 fixed to the rear part of the vehicle body of the vehicle C images the rear (directly behind and left and right rear) of the vehicle C through a single optical system 11 and image sensor 12, respectively. The optical system 11 of the present embodiment has a maximum viewing angle of 180°, and enables imaging of the entire rear with the first viewing angles 11a and 11b corresponding to the maximum viewing angle, and enables imaging directly behind with the second viewing angle 11b.

[0014] The optical system 11 has a relationship (θ-y projection characteristic) between the half viewing angle θ and the image height y shown by a solid line in FIG. 3(a). The optical system 11 has a maximum half viewing angle of 90°. In a general equidistant projection (y = fθ) shown by a broken line, y on the imaging surface of the image sensor 12 increases in proportion to θ. On the other hand, the optical system 11 of the present embodiment has a projection characteristic in which y is higher than y = fθ between the center on the optical axis and the maximum half viewing angle, the increase rate of y with respect to θ is larger than y = fθ at the center-side viewing angle (hereinafter referred to as the center viewing angle), and the increase rate of y becomes smaller as θ increases at the peripheral-side viewing angle (hereinafter referred to as the peripheral viewing angle).

[0015] In addition, the optical system 11 has a θ-resolution characteristic showing the relationship between the half angle θ shown by the solid line in Fig. 3(b) and the optical resolution (the length of the image height y per unit angular field or the number of pixels of the subject image on the imaging device per unit angular field: hereinafter simply referred to as resolution). In the general equidistant projection shown by the dashed line in Fig. 3(b), it has the same resolution regardless of the angular field. On the other hand, the optical system 11 of this embodiment has a resolution characteristic in which the resolution varies depending on the angular field (central angular field and peripheral angular field). Specifically, it has a resolution characteristic in which the resolution at the central angular field is higher than y = fθ, and the resolution decreases as θ increases, becoming lower than y = fθ at the peripheral angular field. For this reason, the optical system 11 forms a subject image at the first resolution (low resolution) in the peripheral region on the imaging surface of the imaging device 12, and forms a subject image at the second resolution (high resolution) higher than the first resolution in the central region. For the first and second resolutions, for example, a resolution equal to or lower than a predetermined value may be set as the first resolution, and a resolution higher than the predetermined value may be set as the second resolution.

[0016] Note that Figs. 3(a) and (b) are examples of the θ-y projection characteristic and the θ-resolution characteristic of the optical system 11, and an optical system having other characteristics may be used as long as it has θ-y projection characteristics and θ-resolution characteristics similar to these.

[0017] Also, when the projection characteristic of the optical system 11 is y(θ) and the maximum half angle is θmax, it is more desirable that the optical system 11 be configured to satisfy the following conditional expression.

[0018]

Equation

[0019] f is the focal length of the optical system 11, and A is a predetermined constant. By setting the lower limit value to 1, the central resolution can be made higher than that of an orthographic projection type (y = f×sinθ) fisheye lens having the same maximum imaging height, and by setting the upper limit value to A, a good optical performance can be maintained while obtaining an angular field equivalent to that of a fisheye lens. The predetermined constant A may be determined in consideration of the balance between the region having the first resolution and the region having the second resolution, and it is preferably set to be between 1.9 and 1.4.

[0020] By configuring the optical system as described above, in the region having the second resolution, a high resolution can be obtained. On the other hand, in the region having the first resolution, the increase amount of the image height y per unit half field angle θ is reduced, and it becomes possible to image a wider field angle. Therefore, while imaging a wide field angle equivalent to that of a fish-eye lens, a high resolution can be obtained near the center field angle.

[0021] Furthermore, in the region having the second resolution, since the characteristics are close to the central projection method (y = f × tanθ), which is the projection characteristic of an optical system for normal imaging, the optical distortion is small and it is possible to display finely. As a result, in an in-vehicle camera monitoring system, a natural perspective of other vehicles and the like can be obtained visually, and deterioration of the image quality can be suppressed to obtain good visibility.

[0022] The imaging device 12 is a photoelectric conversion device in which a plurality of pixels are two-dimensionally arranged on its imaging surface, and is composed of a CMOS sensor, a CCD sensor, or the like. FIG. 4 shows an operation example of the imaging device 12. t is time. The imaging device 12 can simultaneously read out the pixel signals of the first pixel region (entire pixel region) 12a, which is the entire imaging surface, and the pixel signals of the second pixel region (partial pixel region) 12b, which is smaller than the first pixel region 12a, at different frame rates (hereinafter referred to as FR). Specifically, the pixel signals are read out from the first pixel region 12a at a period corresponding to the first FR (low FR), and the pixel signals are read out from the second pixel region 12b at a period corresponding to the second FR (high FR), which is higher than the first FR.

[0023] The image processing unit (image generation means) 13 generates a back monitor image 21 as a first image (full pixel image) from the pixel signals read from the first pixel region 12a at a first FR. Also, it generates a room mirror image 22 as a second image (partial pixel image) from the pixel signals read from the second pixel region 12b at a second FR. The image processing unit 13 performs image processing necessary for displaying the generated back monitor image 21 and room mirror image 22 and outputs them to the output unit 14. The image processing includes various correction processes such as developing processing for converting Bayer array data into RGB raster format, white balance adjustment, gain / offset adjustment, gamma processing, color matrix processing, unevenness correction, geometric correction, etc., resolution adjustment, color depth adjustment, frame rate adjustment, and compression processing.

[0024] The output unit 14 outputs the back monitor image 21 to the in-dash monitor 17 installed on the dashboard and displays it in response to the vehicle's transmission being set to reverse (back) or the user (driver) operating the monitor switching switch. When backing the vehicle, the driver can check for obstacles not only directly behind but also on the left and right rear while looking at the wide-angle (e.g., 180° full angle) back monitor image 21 displayed on the in-dash monitor 17. The in-dash monitor 17 can also display navigation information such as maps and TV images in addition to the first image 21.

[0025] Also, the output unit 14 outputs the room mirror image 22 to the electronic room mirror 16 provided at the front part of the vehicle ceiling and displays it. The driver can check for vehicles behind while looking at the room mirror image 22 displayed on the electronic room mirror 16.

[0026] The setting unit 15 is configured by a computer including a CPU or the like that performs processing according to a program, and sets a first pixel region 12a and a second pixel region 12b on the imaging surface of the imaging device 12. Here, the setting unit 15 sets the first and second pixel regions 12a and 12b so that the lowest resolution in the second pixel region 12b is higher than the lowest resolution in the first pixel region 12a. Specifically, as shown in FIG. 5, the setting unit 15 sets the entire pixel region A (corresponding to the first and second angle-of-view regions 11a and 11b in FIG. 2(a)) on the imaging surface of the imaging device 12, where the optical system 11 forms a subject image at the first and second resolutions, that is, regardless of the resolution, as the first pixel region 12a. At this time, the setting unit 15 sets the signal readout period from the first pixel region 12a to a period corresponding to the first FR as described above. FIG. 5 shows an example where the first pixel region 12a is a region of 3000 pixels horizontally and 2000 pixels vertically, and the first FR is 30 fps (frames per second).

[0027] Further, the setting unit 15 sets the second pixel region 12b within a pixel region B (corresponding to the second angle-of-view region 11b in FIG. 2(a)) on the imaging surface where the optical system 11 forms a subject image at the second resolution. In this embodiment, the second pixel region 12b is set as a pixel region within the first image region 12a where the resolution of the optical system 11 is higher by a predetermined value. At this time, the setting unit 15 sets the signal readout period from the second pixel region 12b to a period corresponding to the second FR as described above. FIG. 5 shows an example where the second pixel region 12b is a region of 2000 pixels horizontally and 400 pixels vertically, and the second FR is 60 fps.

[0028] Note that when the first pixel region 12a is fixed as the entire pixel region A, the setting unit 15 may set only the second pixel region 12b without setting the first pixel region 12a.

[0029] Furthermore, the setting unit 15 sets a first FR and a first display image size for a back monitor image generation unit 13a that generates a back monitor image 21 from the pixel signals read from the first pixel region 12a within the image processing unit 13. In FIG. 5, 2000 horizontal pixels × 1000 vertical pixels is illustrated as the first display image size. In this case, the back monitor image generation unit 13a performs geometric transformation and cropping processing on the image read from the first pixel region 12a and outputs it as the back monitor image. Thereby, the back monitor image generation unit 13a reads the pixel signals from the first pixel region 12a at the first FR, and generates and outputs the back monitor image 21 at the first FR and the first display image size.

[0030] Also, the setting unit 15 sets a second FR and a second display image size for a room mirror image generation unit 13b that generates a room mirror image 22 from the pixel signals read from the second pixel region 12b within the image processing unit 13. In FIG. 5, 2000 horizontal pixels × 400 vertical pixels is illustrated as the second display image size. Thereby, the room mirror image generation unit 13b reads the pixel signals from the second pixel region 12b at the second FR, and generates and outputs the room mirror image 22 at the second FR and the second display image size.

[0031] Thus, in this embodiment, the back monitor image 21 that captures a wide viewing angle is displayed on the in-dash monitor 17 at the first FR, and the high-resolution rearview mirror image 22 is displayed on the electronic rearview mirror 16 at a second FR higher than the first FR. For the display of the electronic rearview mirror 16, it is required to display an image that can be visually recognized at a high frame rate (e.g., 60 fps or higher) and high resolution. On the other hand, for the display of the back monitor image displayed on the in-dash monitor 17, it is required to capture and display a wide range of the rear of the vehicle. Therefore, according to this embodiment, the area with high resolution of the optical system 12 can be output at a high frame rate. For this reason, the electronic rearview mirror image can be displayed at high resolution and high frame rate, and the back monitor image with a wide viewing angle can be output at a relatively low frame rate. As a result, while suppressing an increase in the load of image processing, it is possible to simultaneously perform imaging with appropriate resolution and frame rate and imaging with an appropriate viewing angle.

[0032] Note that the setting unit 15 stores in advance data such as the positions and sizes (number of pixels) of the first and second pixel regions 12a and 12b set based on the characteristics of the optical system 11 shown in FIGS. 2(a) and 2(b) at the manufacturing stage of the in-vehicle camera. Then, when generating the back monitor image 21 and the rearview mirror image 22, the first and second pixel regions 12a and 12b, the first and second FRs, and further the first and second display image sizes are set for the image sensor 12 and the image processing unit 13 based on the data. Also, although the first pixel region read from the image sensor 12 has been described by taking the case of all the pixels of the image sensor 12 as an example, this is not the limit, and a predetermined region of the image sensor 12 may be used as the first pixel region.

[0033] For example, the setting unit 15 may set the 3000x1000 pixel region in the lower half of the imaging device 12 in FIG. 5 as the first pixel region. In this case, by displaying the background monitor image generated by the image processing unit 13 based on the first pixel region on the driver, the visibility near the ground behind the vehicle can be improved, and the image processing load of the image processing unit 13 and the transmission bandwidth between the imaging device 12 and the image processing unit 13 can be suppressed.

[0034] Further, as long as the range is such that the lowest resolution of the second pixel region is higher than the lowest resolution in the first pixel region, based on the user's instruction, the position and size of the first pixel region and the second pixel region 12b may be configured to be changed. For example, a menu image serving as a user interface is displayed on the in-dash monitor 17, and the user can select the position and size of the first pixel region 12a through the menu image, and the setting unit 15 may set the first pixel region 12a at the selected position and size. Similarly, a menu image serving as a user interface is displayed on the electronic rearview mirror 16, and the user can select the position and size of the second pixel region 12b through the menu image, and the setting unit 15 may set the second pixel region 12b at the selected position and size. [Example 2] Next, Example 2 will be described. In this example, as shown in FIG. 6(a), the in-vehicle cameras 10 fixed to the side portions of the vehicle body of the vehicle C image the sides (directly to the side, diagonally forward, and diagonally backward) of the vehicle C through a single optical system and imaging device, respectively. The optical system 11 of this example also has a maximum angle of view of 180°, enables imaging of the entire side with the first angles of view 11c, 11d, 11e, and enables imaging of the diagonally forward and diagonally backward with the second angles of view 11d, 11e, respectively.

[0035] By imaging at the second horizontal front viewing angle 11d by the in-vehicle camera 10, a side front image 23 showing the vicinity of the front wheels of the vehicle C that can be displayed on the in-dash monitor 17 (or the side monitor 18) as shown in FIG. 6(b) is generated. The driver who views the side front image 23 can confirm the presence or absence of obstacles or the like around the front wheels that are blind spots from the driver's seat. Further, by imaging at the second horizontal rear viewing angle 11e, a side mirror image 24 that can be displayed on the side monitor 18 provided on the dashboard instead of the side mirror is generated as shown in FIG. 6(c).

[0036] Furthermore, by imaging at the first viewing angles 11c, 11d, 11e of the entire side of the vehicle C, a peripheral view image 25 showing the surroundings of the vehicle C that can be displayed on the in-dash monitor 17 is generated as shown in FIG. 6(d). The peripheral view image 25 is generated by synthesizing the images generated by imaging at the first viewing angles 11c, 11d, 11e, the images generated by imaging the rear of the vehicle C as in the first embodiment, and the images generated by imaging the front of the vehicle C as in the third embodiment described later. The driver can confirm the presence or absence of obstacles around the vehicle C by viewing the peripheral view image 25 when starting the vehicle C or when starting off.

[0037] The optical system 11 of this embodiment has the θ-y projection characteristics shown by the solid line in FIG. 7(a). The optical system 11 has a maximum half viewing angle of 90°. Similar to FIG. 3(a), in equidistant projection (y = fθ), y increases in proportion to θ. On the other hand, the optical system 11 of this embodiment has projection characteristics in which y is lower than y = fθ between the center on the optical axis and the maximum half viewing angle, the increase rate of y with respect to θ at the center viewing angle is smaller than y = fθ, and the increase rate of y becomes larger as θ increases at the peripheral viewing angle.

[0038] In addition, the optical system 11 has a θ-resolution characteristic shown by a solid line in FIG. 7(b). Similar to FIG. 3(b), in equidistant projection, it has the same resolution regardless of the angle of view. On the other hand, the optical system 11 of this embodiment has a resolution characteristic in which the resolution is different between the central angle of view and the peripheral angle of view. Specifically, it has a resolution characteristic in which the resolution becomes higher as θ increases, such that the resolution at the central angle of view is lower than y = fθ and the resolution at the peripheral angle of view is higher than y = fθ. Therefore, the optical system 11 forms a subject image at a first resolution (low resolution) in the central region on the imaging surface of the imaging device 12, and forms a subject image at a second resolution (high resolution) higher than the first resolution in the peripheral region.

[0039] Note that FIGS. 7(a) and (b) are examples of the θ-y projection characteristic and the θ-resolution characteristic of the optical system 11, and an optical system having other characteristics may be used as long as it has θ-y projection characteristics and θ-resolution characteristics similar to these.

[0040] The setting unit 15' of this embodiment shown in FIG. 8 sets a first pixel region 12a and second pixel regions 12b and 12c on the imaging surface of the imaging device 12. Also in this embodiment, the setting unit 15' sets the first and second pixel regions 12a, 12b, and 12c such that the lowest resolution in the second pixel regions 12b and 12c is higher than the lowest resolution in the first pixel region 12a.

[0041] Specifically, the setting unit 15' sets the entire pixel region A (corresponding to the first angles of view 11c, 11d, and 11e in FIG. 6(a)) on the imaging surface of the imaging device 12 where the optical system 11 forms a subject image at the first resolution as the first pixel region 12a. At this time, the setting unit 15' sets the signal readout period from the first pixel region 12a to a period corresponding to the first FR. FIG. 8 shows, as an example, the case where the first pixel region 12a is a region of 3000 horizontal pixels × 2000 vertical pixels and the first FR is 30 fps.

[0042] Further, the setting unit 15' sets second pixel regions 12b and 12c within a pixel region B (corresponding to the second angular fields 11d and 11e in Fig. 6(a)) where the optical system 11 forms a subject image at the second resolution on the imaging surface. Also in this embodiment, the second pixel regions 12b and 12c are set within the first image region 12a. At this time, the setting unit 15' sets the signal readout period from the second pixel regions 12b and 12c to a period corresponding to the second FR. Fig. 8 shows, as an example, a case where the second pixel regions 12b and 12c are regions of 400 pixels horizontally and 600 pixels vertically, and the second FR is 60 fps.

[0043] Furthermore, the setting unit 15' sets the first FR and the first display image size for a peripheral view image generation unit 13c that generates a peripheral view image 25 using the pixel signals read from the first pixel region 12a within the image processing unit 13'. Fig. 8 exemplifies the first display image size as 2000 pixels horizontally and 1000 pixels vertically. Thereby, the peripheral view image generation unit 13c reads the pixel signals from the first pixel region 12a at the first FR, and synthesizes the peripheral view image based on the images from other cameras for constructing the peripheral view, and generates and outputs the peripheral view image 25 at the first FR and the first display image size.

[0044] Also, the setting unit 15' sets the second FR and the second display image size for a side mirror image generation unit 13d that generates a side mirror image 24 from the pixel signals read from the second pixel region 12c within the image processing unit 13'. Fig. 8 exemplifies the second display image size as 800 pixels horizontally and 600 pixels vertically. As can be seen from the θ-y projection characteristics shown in Fig. 7(a), since the subject image is compressed in the peripheral region on the imaging surface, the second display image size is set such that a side mirror image 24 stretched horizontally from the size of the second pixel region 12c is generated. Thereby, the side mirror image generation unit 13d reads the pixel signals from the pixel region 12c at the second FR, performs image processing such as geometric correction, and generates and outputs the side mirror image 24 at the second FR and the second display image size.

[0045] Thus, in this embodiment, the peripheral view image 25 synthesized based on a wide imaging angle is displayed on the in-dash monitor 17 at the first FR, and the side mirror image 24 with high resolution is displayed on the side monitor 18 at a second FR higher than the first FR.

[0046] For the display on the side monitor 18, it is required to display an image that can be visually recognized at a high frame rate (for example, 60 fps or more) and high resolution. On the other hand, for the peripheral view image displayed on the in-dash monitor 17, it is required to synthesize and display an image captured at a wide angle. Therefore, according to this embodiment, since the region with high resolution of the optical system 12 can be output at a high frame rate, the side mirror image can be displayed at high resolution and high frame rate. Also, the peripheral view image based on the image with a wide angle can be output and displayed at a relatively low frame rate. Thus, while suppressing an increase in the load of image processing, it is possible to simultaneously perform imaging with appropriate resolution and frame rate and imaging with an appropriate angle of view.

[0047] Note that in FIG. 8, illustration of the side front image generation unit that generates the side front image 23 shown in FIG. 6(b) is omitted. The setting unit 15' sets the signal readout period from the second pixel region 12b of the imaging surface to a period corresponding to the second FR. Also, the setting unit 15' sets the second FR and the second display image size for the side front image generation unit. Thereby, a high-resolution side front image 23 is generated from the pixel signals read out from the second pixel region at the second FR and displayed on the in-dash monitor 17 or the side monitor 18.

[0048] Also in this embodiment, as long as the range is such that the lowest resolution of the second pixel region is higher than the lowest resolution in the first pixel region with respect to the setting unit 15', the positions and sizes of the first pixel region and the second pixel regions 12b and 12c may be selectable based on an instruction from the user.

[0049] Also, an image generated from pixel signals from the first and second pixel regions 12a, 12b, and 12c may be output for sensing to detect the presence of obstacles or other objects near the vehicle. [Embodiment 3] Next, Embodiment 3 will be described. As shown in FIG. 9, an in-vehicle camera 10 fixed to the front part of the vehicle body of the vehicle C images the front (directly in front and left and right in front) of the vehicle C through a single optical system and an image sensor, respectively. The optical system 11 of this embodiment has a maximum viewing angle of 180°, and can image directly in front at the first viewing angle 11f, and can image the front left side and the front right side at the second viewing angles 11g and 11h, respectively. In FIG. 9, an object OBJ such as a person is approaching the vehicle C from the front left side.

[0050] By imaging at the first viewing angle 11f and the second viewing angles 11g and 11h by the in-vehicle camera 10, a front view image 26 showing the entire front as shown in FIG. 10(a) is generated. The driver who views the front view image 26 displayed on the dash monitor 17 can confirm an object existing directly in front and an object OBJ approaching from the front left side that is difficult to notice from the driver's seat. The optical system 11 of this embodiment also has the θ-y projection characteristics and θ-resolution characteristics shown in FIGS. 7(a) and 7(b) in Embodiment 2.

[0051] The setting unit 15″ of this embodiment shown in FIG. 10(a) sets the first pixel region 12a and the second pixel regions 12b and 12c on the imaging surface of the image sensor 12. Specifically, the setting unit 15″ sets, as the first pixel region 12a, a pixel region on the imaging surface of the image sensor 12 where the optical system 11 forms a subject image at the first resolution (corresponding to the first viewing angle 11f in FIG. 9). At this time, the setting unit 15″ sets the signal readout period from the first pixel region 12a to a period corresponding to the first FR. In FIG. 10(a), a case where the first pixel region 12a is a region of 2000 pixels horizontally and 1200 pixels vertically and the first FR is set to 30 fps is shown as an example.

[0052] Also, the setting unit 15″ sets the left and right pixel regions (corresponding to the second angle-of-view 11g, 11h in FIG. 9) in which the optical system 11 forms a subject image at the second resolution on the imaging surface as second pixel regions 12b and 12c, respectively. In the present embodiment, the second pixel regions 12b and 12c are set so as to be adjacent to the first image region 12a. At this time, the setting unit 15″ sets the signal readout period from the second pixel regions 12b and 12c to a period corresponding to the second FR. In FIG. 10(a), the case where the second pixel regions 12b and 12c are regions of 500 pixels in width and 1200 pixels in height, respectively, and the second FR is set to 60 fps is shown as an example.

[0053] Furthermore, the setting unit 15″ sets the first FR and the first display image size for the MF view image generation unit 13e that generates the main front (MF) view image 25 from the pixel signals read from the first pixel region 12a in the image processing unit 13″. Here, the first display image size is, for example, 2000 pixels in width and 1200 pixels in height. Thereby, the MF view image generation unit 13e reads the pixel signals from the first pixel region 12a at the first FR and generates the MF view image 25 at the first FR and the first display image size.

[0054] Also, the setting unit 15″ sets the second FR and the second display image size for the SF image generation unit 13f that generates the left and right side front (SF) view images 24 from the pixel signals read from the second pixel regions 12b and 12c in the image processing unit 13″. Here, the second display image size is, for example, 500 pixels in width and 1200 pixels in height. Thereby, the SF view image generation unit 13f reads the pixel signals from the left and right second pixel regions 12b at the second FR and generates the left and right SF view images 24 at the second FR and the second display image size.

[0055] Then, the image composition unit 13g in the image processing unit 13″ connects (composes) the MF view image from the MF view image generation unit 13e and the left and right SF view images from the SF image generation unit 13f into one and outputs it to the in-dash monitor 17 for display. As a result, on the in-dash monitor 17, a front view image 26 is displayed in which the resolution and FR in the central region are not high, but the resolution and FR in the left and right peripheral regions are high, making it easy to confirm the approach of the object OBJ from the front left and right.

[0056] Note that FIG. 10(b) shows the subject image on the imaging device 12 when a general fisheye lens (y = fθ) is used as the optical system. When a fisheye lens is used, from the θ-y projection characteristic shown by the broken line in FIG. 7(a), the subject image (image of the object OBJ) on the peripheral side is formed more peripherally on the imaging device than in FIG. 10(a). That is, the second pixel region 12b′ corresponding to the second pixel region 12b in FIG. 10(a) becomes narrower peripherally. Also, from the θ-resolution characteristic shown by the broken line in FIG. 7(b), the resolution of the left and right peripheral viewing angles becomes the same low resolution as the central viewing angle. As a result, it becomes more difficult to confirm the object OBJ compared to FIG. 10(a).

[0057] In this embodiment, by combining the high-resolution region formed by the optical system 11 and the pixel signal readout region at high FR in the imaging device 12, it is possible to make it easier to confirm the object OBJ in the peripheral region.

[0058] Note that also in this embodiment, the images generated from the pixel signals of the first and second pixel regions 12a, 12b, 12c may be output for sensing to detect the presence of obstacles or other objects near the vehicle.

[0059] In each of the above embodiments, the case where a second pixel region for reading out pixel signals at a high FR (second FR) is set in a region where the optical system forms a subject image at a high resolution (second resolution) on the imaging device has been described. In contrast, a second pixel region for reading out pixel signals at a high FR may be set in a region where the optical system forms a subject image at a low resolution (first resolution) on the imaging device. In this case, although the resolution of the image generated from the pixel signals from the second pixel region is not high, since it is an image with a high FR, it is effective in sensing an object near the vehicle and the like. The above embodiments include the following configurations.

[0060] (Configuration 1) Setting means for setting at least the second pixel region among a first pixel region capable of signal readout at a first frame rate and a second pixel region capable of signal readout at a second frame rate faster than the first frame rate on a single imaging device that captures a subject image formed by a single optical system. The setting means sets the second pixel region such that the lowest resolution in one of the first and second pixel regions is higher than the lowest resolution in the other pixel region when the optical system has a characteristic that the resolution of the subject image on the imaging device per unit angular field of view varies depending on the angular field of view. An imaging setting device characterized by this. (Configuration 2) The setting means sets the second pixel region such that the lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region when the optical system has the characteristic. The imaging setting device according to Configuration 1, characterized by this. (Configuration 3) The setting means sets the second pixel region on the central side of the imaging device when the optical system has a characteristic that the resolution at the central-side angular field of view is higher than the resolution at the peripheral-side angular field of view. The imaging setting device according to Configuration 2, characterized by this. (Configuration 4) When the setting means has a characteristic that the resolution of the optical system at the peripheral angle of view is higher than the resolution at the central angle of view, the second pixel region is set on the peripheral side of the imaging element. The imaging setting device according to Configuration 2. (Configuration 5) The setting means sets, as the second pixel region, a pixel region corresponding to an angle of view on the imaging element where the resolution is higher than a predetermined value. The imaging setting device according to Configuration 2. (Configuration 6) The setting means sets the second pixel region within the first pixel region. The imaging setting device according to any one of Configurations 1 to 5. (Configuration 7) The setting means sets the second pixel region to be adjacent to the first pixel region. The imaging setting device according to any one of Configurations 1 to 5. (Configuration 8) The setting means sets the second pixel region with a position and size respectively selected by the user. The imaging setting device according to any one of Configurations 1 to 7. (Configuration 9) The optical system has a characteristic that the resolution on one of the central and peripheral angles of view is higher and the resolution on the other is lower compared to equidistant projection. The imaging setting device according to any one of Configurations 1 to 8. (Configuration 10) An imaging setting device according to any one of Configurations 1 to 9, the optical system, the imaging element, and image generation means for generating a first image at the first frame rate using the signal read from the first pixel region and generating a second image at the second frame rate using the signal read from the second pixel region. An imaging device characterized by this. (Configuration 11) An imaging device mounted on a vehicle for imaging the rear of the vehicle, wherein the lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region, Output the first image as a rear monitor image, The imaging device according to configuration 10, characterized in that the second image is output as a room mirror image. (Configuration 12) An imaging device mounted on a vehicle for imaging the side of the vehicle, The lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region, Output the first image for generating a peripheral view image, The imaging device according to configuration 10, characterized in that the second image is output as a side mirror image or an image near the front wheels of the vehicle. (Configuration 13) An imaging device mounted on a vehicle for imaging the front of the vehicle, The lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region, The imaging device according to configuration 10, characterized in that a front view image including a true front image as the first image and front left and right images as the second image is output. (Configuration 14) An imaging device mounted on a vehicle, The imaging device according to configuration 10, characterized in that at least one of the first and second images is output for sensing an object near the vehicle. (Configuration 15) An imaging device according to any one of configurations 10 to 14, A vehicle, characterized by having a vehicle body on which the imaging device is fixed.

[0061] Each of the embodiments described above is merely a representative example, and various modifications and changes can be made to each embodiment when implementing the present invention.

Description of Signs

[0062] 10 In-vehicle camera 11 Optical system 12 Image sensor 15 Setting unit

Claims

1. Setting means for setting at least the second pixel region among a first pixel region capable of signal readout at a first frame rate and a second pixel region capable of signal readout at a second frame rate faster than the first frame rate on a single imaging element that images a subject image formed by a single optical system; Image generation means for generating a first image at the first frame rate using the signal read from the first pixel region and generating a second image at the second frame rate using the signal read from the second pixel region, and having, When the optical system has a characteristic that the resolution, which is the number of pixels of the subject image on the imaging element per unit angular field of view, varies depending on the angular field of view, the setting means sets the second pixel region so that the lowest resolution in one of the first and second pixel regions is higher than the lowest resolution in the other pixel region; The image generation means generates the first image and the second image simultaneously. An imaging device characterized by this.

2. The imaging device according to claim 1, wherein the setting means sets the second pixel region so that the lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region when the optical system has the characteristic.

3. The imaging device according to claim 2, wherein the setting means sets the second pixel region on the central side of the imaging element when the optical system has a characteristic that the resolution at the central angular field of view is higher than the resolution at the peripheral angular field of view.

4. The imaging device according to claim 2, wherein the setting means sets the second pixel region on the peripheral side of the imaging element when the optical system has a characteristic that the resolution at the peripheral angular field of view is higher than the resolution at the central angular field of view.

5. The imaging device according to claim 2, wherein the setting means sets a pixel region corresponding to an angle of view with a resolution higher than a predetermined value on the imaging element as the second pixel region.

6. The imaging device according to claim 1, wherein the setting means sets the second pixel region within the first pixel region.

7. The imaging device according to claim 1, wherein the setting means sets the second pixel region so as to be adjacent to the first pixel region.

8. The imaging device according to claim 1, wherein the setting means sets the second pixel region having a position and size respectively selected by a user.

9. The imaging device according to claim 1, wherein the optical system has a characteristic that the resolution on one of the center side and the peripheral side of the angle of view is higher and the resolution on the other side is lower than that of equidistant projection.

10. An imaging device mounted on a vehicle for imaging the rear of the vehicle, wherein the lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region, the first image is output as a back monitor image, and the second image is output as a rearview mirror image. The imaging device according to claim 1.

11. An imaging device mounted on a vehicle for imaging the side of the vehicle, wherein the lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region, the first image is output for generating a peripheral view image, and the second image is output as a side mirror image or an image near the front wheel of the vehicle. The imaging device according to claim 1.

12. An imaging device mounted on a vehicle and imaging the front of the vehicle, wherein the lowest resolution in the second pixel region is higher than the lowest resolution in the first pixel region, The imaging device according to claim 1, wherein a front view image including a true front image as the first image and a front left and right image as the second image is output.

13. An imaging device mounted on a vehicle, The imaging device according to claim 1, wherein at least one of the first and second images is output for sensing an object near the vehicle.

14. The imaging device according to claim 1, A moving body having a vehicle body on which the imaging device is fixed.

15. A setting step of setting at least the second pixel region among a first pixel region capable of signal readout at a first frame rate and a second pixel region capable of signal readout at a second frame rate faster than the first frame rate on a single image sensor that images a subject image formed by a single optical system, An image generation step of generating a first image at the first frame rate using the signal read from the first pixel region and generating a second image at the second frame rate using the signal read from the second pixel region, and In the setting step, when the optical system has characteristics in which the resolution, which is the number of pixels of the subject image on the image sensor per unit angle of view, varies depending on the angle of view, the second pixel region is set so that the lowest resolution in one of the first and second pixel regions is higher than the lowest resolution in the other pixel region, The imaging method, wherein the image generation step generates the first image and the second image simultaneously.

16. A program, characterized in that it causes a computer to execute processing according to the imaging method described in claim 15.

Images