Crew monitoring device

Abstract

To provide an occupant monitoring apparatus which can properly monitor an occupant.SOLUTION: In an occupant monitoring apparatus 1, an optical system 13 includes an imaging lens and a magnification optical system. The imaging lens forms an image on a camera module 10. The magnification optical system makes a field angle in a vertical direction according to the imaging lens unity magnification in a region including eyes of an occupant in at least a monitoring object region and magnifies the field angle in a horizontal direction intersecting the vertical direction at a predetermined magnification. The camera module 10 captures the image formed by the imaging lens at the field angle magnified in the horizontal direction by the magnification optical system. A pixel compensation unit 22 compensates for a pixel along the horizontal direction with respect to a compression image captured by the camera module 10. A monitoring unit 231 monitors the occupant by detecting the eyes of the occupant on the basis of the image whose pixel is compensated by the pixel compensation unit 22.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an occupant monitoring device.

Background Art

[0002] Conventionally, as an occupant monitoring device, for example, Patent Document 1 describes an occupant monitoring device including an imaging device that images a plurality of occupants sitting side by side in the vehicle width direction, and a plurality of light projecting devices provided corresponding to a plurality of seating positions and projecting light toward each seated occupant. Since this occupant monitoring device projects light toward each seated occupant from the plurality of light projecting devices provided corresponding to the seating positions of the seat one-to-one, an image in which the occupant is well imaged can be obtained even when a rapid change such as the presence or absence of sunlight occurs.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, since the occupant monitoring device described in Patent Document 1 above images a plurality of occupants sitting side by side in the vehicle width direction, for example, the imaging range by the imaging device becomes wide. For this reason, when monitoring the driver's eyes, for example, an image sensor having high resolution is required in the imaging device, but there is a problem that the high-resolution image sensor becomes large.

[0005] Therefore, the present invention has been made in view of the above, and an object thereof is to provide an occupant monitoring device that can appropriately monitor an occupant.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the occupant monitoring device according to the present invention comprises: an optical system provided in a vehicle that forms an image of a monitoring target area including the occupant; an imaging unit that captures the image formed by the optical system; an image processing unit that processes the image captured by the imaging unit; and a monitoring unit that detects the occupant's eyes and monitors the occupant based on the image processed by the image processing unit, wherein the optical system comprises an imaging lens that forms an image on the imaging unit, and the imaging lens that captures an image in at least the area including the occupant's eyes in the monitoring target area. The imaging unit includes a magnifying optical system that sets the vertical field of view corresponding to the image lens to 1x magnification and magnifies the horizontal field of view intersecting the vertical direction by a predetermined magnification, wherein the imaging unit captures an image formed by the imaging lens at the horizontal field of view magnified by the magnifying optical system, the image processing unit interpolates pixels along the horizontal direction in the image captured by the imaging unit, and the monitoring unit detects the eyes of the occupant and monitors the occupant based on the image in which the pixels have been interpolated by the image processing unit. [Effects of the Invention]

[0007] The occupant monitoring device according to the present invention can properly monitor occupants because it uses a small imaging unit to capture images of a wide imaging area with high resolution. In particular, the occupant monitoring device is effective when monitoring the degree of eye opening of the driver, as high-resolution images are required in such cases. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a conceptual diagram showing an example of the configuration of a crew monitoring device according to an embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the configuration of a crew monitoring device according to an embodiment. [Figure 3] Figure 3 shows the horizontal field of view according to the embodiment. [Figure 4] Figure 4 shows the vertical field of view according to the embodiment. [Figure 5] Figure 5 shows the uncompressed image in the imaging range for the comparative example. [Figure 6] FIG. 6 is a diagram showing a compressed image in an enlarged imaging range according to an embodiment. [Figure 7] FIG. 7 is a diagram showing the relationship between an imaging range according to a comparative example and an enlarged imaging range. [Figure 8] FIG. 8 is a diagram showing an example of pixel completion according to an embodiment. [Figure 9] FIG. 9 is a diagram showing an image of an eye before pixel completion according to an embodiment. [Figure 10] FIG. 10 is a diagram showing an image of an eye after pixel completion according to an embodiment. [Figure 11] FIG. 11 is a diagram showing an image of an eye according to a comparative example. [Figure 12] FIG. 12 is a diagram showing an example of the operation of an occupant monitoring device according to an embodiment. [Figure 13] FIG. 13 is a diagram showing a configuration example of an optical system according to a first modification example of an embodiment. [Figure 14] FIG. 14 is a diagram showing an imaging region according to a first modification example of an embodiment. [Figure 15] FIG. 15 is a diagram showing a configuration example of an optical system according to a second modification example of an embodiment. [Figure 16] FIG. 16 is a diagram showing a configuration example of an optical system according to a third modification example of an embodiment. [Figure 17] FIG. 17 is a diagram showing a configuration example of an optical system according to a fourth modification example of an embodiment. [Figure 18] FIG. 18 is a diagram showing an imaging region according to a fourth modification example of an embodiment. [Figure 19] FIG. 19 is a diagram showing a configuration example of an optical system according to a fifth modification example of an embodiment. [Figure 20] FIG. 20 is a diagram showing a configuration example of an optical system according to a sixth modification example of an embodiment. [Figure 21] FIG. 21 is a diagram showing a configuration example of an optical system according to a seventh modification example of an embodiment. [Figure 22] FIG. 22 is a diagram showing an imaging region according to a seventh modification example of an embodiment.

Best Mode for Carrying Out the Invention

[0009] The mode (embodiment) for carrying out the present invention will be described in detail while referring to the drawings. The present invention is not limited by the content described in the following embodiments. Further, the constituent elements described below include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the configurations described below can be combined as appropriate. Also, various omissions, substitutions, or changes in the configuration can be made without departing from the gist of the present invention.

[0010] 〔Embodiment〕 The occupant monitoring device 1 according to the embodiment will be described while referring to the drawings. The occupant monitoring device 1 is provided in a vehicle and monitors the occupants who have boarded the vehicle. The occupant monitoring device 1, for example, detects the eyes of a driver D as an occupant driving the vehicle and monitors the eye opening degree of the driver D. Hereinafter, the occupant monitoring device 1 will be described in detail.

[0011] As shown in FIGS. 1 and 2, the occupant monitoring device 1 includes a camera module 10 as an imaging unit and a control box 20 as an image processing unit. The camera module 10 and the control box 20 are connected to be communicable with each other.

[0012] Here, in the following description, the direction along gravity is referred to as the vertical direction Z, the direction intersecting (orthogonal) to the vertical direction Z is the horizontal direction X, and the direction intersecting the vertical direction Z and the horizontal direction X is the axial direction Y. The horizontal direction X is the direction along the vehicle width of the vehicle and is also the direction intersecting (orthogonal) to the straight-ahead direction in which the vehicle travels straight.

[0013] The camera module 10 is used to image the occupants of a vehicle. The camera module 10 is configured to image a monitored area E that includes the occupants of the vehicle. Here, the monitored area E includes at least the driver D seated in the driver's seat and the passenger seated in the front passenger seat. The monitored area E may also include the passenger seated in the rear seat in addition to the driver D and the passenger seated in the front passenger seat. One camera module 10 is installed inside the vehicle, for example, on the front side of the roof. The camera module 10 comprises a light source 11, a power supply 12, an optical system 13, and an image sensor 14.

[0014] The light source 11 is a device that emits light, and is, for example, an LED (Light-Emitting Diode) that emits near-infrared light. The light source 11 is connected to a power supply 12, and when power is supplied from the power supply 12, it emits light towards the occupants. For example, when power is supplied from the power supply 12, the light source 11 emits light towards the driver D, the passenger in the front seat, and the passenger in the rear seat.

[0015] The power supply 12 supplies power to the light source 11. The power supply 12 is connected to the control box 20 and the light source 11, and supplies power to the light source 11 based on the control signal output from the control box 20.

[0016] The optical system 13 forms an image on the image sensor 14. The optical system 13 is composed of, for example, an anamorphic lens 131 as a magnifying optical system and an imaging lens 132, as shown in Figures 3 and 4.

[0017] The anamorphic lens 131 magnifies the field of view in the horizontal direction X by a predetermined magnification. The anamorphic lens 131 consists of a single dome-shaped lens. The anamorphic lens 131 is positioned alongside the imaging lens 132 along the axial direction Y, and is located on the opposite side of the imaging lens 132 from the image sensor 14. In other words, the camera module 10 is arranged in the order of anamorphic lens 131, imaging lens 132, and image sensor 14 along the axial direction Y.

[0018] The anamorphic lens 131 sets the field of view θ1 in the monitored area E, including the occupant's eyes, to the same angle of view θ1 as the image lens 132 in the vertical Z direction, and the field of view θ2 in the horizontal X direction, which is magnified by a predetermined magnification (for example, 2 times), as the image lens 132. In other words, the anamorphic lens 131 sets the field of view θ1 in the vertical Z direction to the same angle of view θ1 as the image lens 132, and the field of view θ2 in the horizontal X direction to the same angle of view θ2 as the image lens 132, which is magnified (2 times). To put it another way, the anamorphic lens 131 sets the field of view θ1 in the vertical Z direction to be the same as the field of view of the image lens 132, and the field of view θ2 in the horizontal X direction to be twice the field of view of the image lens 132. Furthermore, the anamorphic lens 131 forms an uncompressed image along the vertical Z direction and a compressed image along the horizontal X direction.

[0019] The imaging lens 132 forms an image on the camera module 10 and is located between the anamorphic lens 131 and the image sensor 14. The imaging lens 132 is composed of, for example, two convex lenses, and includes a first imaging lens 132a and a second imaging lens 132b. The first imaging lens 132a and the second imaging lens 132b are arranged side by side along the axial direction Y, with the first imaging lens 132a located on the anamorphic lens 131 side and the second imaging lens 132b located on the image sensor 14 side. The first imaging lens 132a and the second imaging lens 132b form an image on the image sensor 14 with a field of view θ1 that is equal to the vertical Z direction and a field of view θ2 that is magnified in the horizontal X direction, using the anamorphic lens 131. In other words, the first imaging lens 132a and the second imaging lens 132b, using an anamorphic lens 131, make the field of view θ1 in the vertical direction Z equal to the field of view of the imaging lens 132, and the field of view θ2 in the horizontal direction X twice the field of view of the imaging lens 132, so that an image of the monitored area E, including the driver D, the passenger in the front seat, and the passenger in the rear seat, is formed on the image sensor 14.

[0020] The image sensor 14 captures an image formed by the imaging lens 132. The image sensor 14 is, for example, rectangular in shape and has 800 dots × 1280 dots of pixels. The image sensor 14 has a length along the horizontal direction X that is longer than the length along the vertical direction Z, with 1280 dots of pixels along the horizontal direction X and 800 dots of pixels along the vertical direction Z. For example, as shown in Figure 5, when the image sensor 14 captures an uncompressed, 1:1 image Q1 with a field of view corresponding to the resolution of the image sensor 14, without expanding the field of view along the horizontal direction X using the anamorphic lens 131. On the other hand, as shown in Figures 6 and 7, when the image sensor 14 expands the field of view along the horizontal direction X using the anamorphic lens 131, and the number of pixels per field of view of the image sensor 14 is small, and the entire image sensor E captures a compressed image Q2 with compression along the horizontal direction X. The image sensor 14 captures an image based on the control signal output from the control box 20 and outputs the captured compressed image Q2 to the control box 20.

[0021] The control box 20 processes the compressed image Q2 captured by the camera module 10. The control box 20 is composed of an electronic circuit mainly consisting of a well-known microcomputer including a CPU, ROM, RAM and an interface, and comprises an image clipping unit 21, a pixel interpolation unit 22 and a control unit 23. The image clipping unit 21, the pixel interpolation unit 22 and the control unit 23 are connected to each other so as to be able to communicate with one another.

[0022] The image clipping unit 21 cuts out a portion of the compressed image Q2 captured by the camera module 10. For example, the image clipping unit 21 cuts out a portion of the compressed image Q2 (for example, the portion including the driver D's eyes) based on a control signal output from the control unit 23. The image clipping unit 21 outputs the cut-out portion of the compressed image Q2 to the pixel interpolation unit 22.

[0023] The pixel interpolation unit 22 interpolates pixels. The pixel interpolation unit 22 interpolates pixels along the horizontal direction X in part or all of the compressed image Q2 output from the image clipping unit 21. The pixel interpolation unit 22 interpolates pixels using a well-known pixel interpolation method, such as nearest neighbor interpolation. Specifically, as shown in Figure 8, the pixel interpolation unit 22 interpolates pixels when the size of the compressed image Q2 is N H ×N V In this case, the size of the output image is (N H ×2-1)×N V The image is resized to the specified format. The pixel interpolation unit 22 then generates image Q3 by inserting blank pixels every other column along the horizontal direction X. That is, the pixel interpolation unit 22 generates image Q3 by inserting actual pixels P1 in odd-numbered columns and blank pixels in even-numbered columns along the horizontal direction X. The pixel interpolation unit 22 then interpolates the blank pixels based on the adjacent pixels before resizing. Specifically, if the pixel data before resizing is B[a,b] and the pixel data after resizing is D[a,b], then the pixel interpolation unit 22 generates image Q3 by inserting blank pixels every other column along the horizontal direction X, based on D[1,2n-1]=B[1,n]. The pixel interpolation unit 22 then interpolates pixels P2 in the blank pixels according to the following equation (1) to generate image Q4. Here, the image of eye G3 in Figure 11 related to the comparative example is an image with guaranteed resolution, and eye G3 is clearly displayed. As shown in Figure 10, the interpolated image Q4 clearly displays eye G2, and there is almost no difference from eye G3 in the comparative example image. As shown in Figure 9, the compressed image Q2 before interpolation shows eye G1 as slightly blurry. The pixel interpolation unit 22 outputs image Q4, in which pixels P2 have been interpolated in the blank pixels, to the control unit 23. D[1,2n]=(B[1,n]+B[1,n+1]) / 2 ···(1)

[0024] The control unit 23 performs various controls. For example, the control unit 23 outputs a control signal to the power supply 12 to control the lighting of the light source 11. The control unit 23 outputs a control signal to the image clipping unit 21 to control the portion of the compressed image Q2 that is clipped. The control unit 23 outputs a control signal to the image sensor 14 to control the imaging process. The control unit 23 is configured to include a monitoring unit 231.

[0025] The monitoring unit 231 monitors the occupants, and for example, it detects the driver D's eyes based on the image Q4 output from the pixel interpolation unit 22 and monitors the degree to which the driver D's eyes are open. In addition to the degree to which the driver D's eyes are open, the monitoring unit 231 may also monitor the driver D's posture, the passenger in the front seat, and the passengers in the rear seats.

[0026] Next, an example of the operation of the occupant monitoring device 1 will be described with reference to Figure 12. The occupant monitoring device 1 forms an image of the monitored area E, including the occupants (step S1). The occupant monitoring device 1 uses, for example, an anamorphic lens 131 to set the field of view θ1 in the vertical direction Z to be the same as the field of view of the imaging lens 132, and the field of view θ2 in the horizontal direction X to twice the field of view of the imaging lens 132, to form an image of the monitored area E, including the driver D, the passenger in the front seat, and the passenger in the rear seat, on the image sensor 14. Next, the occupant monitoring device 1 captures the image with the image sensor 14 (step S2). Next, the occupant monitoring device 1 interpolates pixels in the image captured by the image sensor 14 (step S3). The occupant monitoring device 1 uses, for example, a pixel interpolation unit 22 to interpolate pixels along the horizontal direction X in the compressed image Q2 and generate the interpolated image Q4. Next, the occupant monitoring device 1 detects the occupant's eyes based on the augmented image Q4 and monitors the occupant (step S4). For example, the monitoring unit 231 of the occupant monitoring device 1 detects the driver's eyes based on the augmented image Q4 and monitors the degree of eye opening of the driver D.

[0027] As described above, the occupant monitoring device 1 according to the embodiment comprises an optical system 13, a camera module 10, a pixel interpolation unit 22, and a monitoring unit 231. The optical system 13 is installed in the vehicle and forms an image of the area to be monitored E, including the occupants. The camera module 10 captures the image formed by the optical system 13. The pixel interpolation unit 22 processes the image captured by the camera module 10. The monitoring unit 231 detects the eyes of the occupants based on the image processed by the pixel interpolation unit 22 and monitors the occupants. Here, the optical system 13 includes an imaging lens 132 and an anamorphic lens 131. The imaging lens 132 forms an image on the camera module 10. The anamorphic lens 131 sets the field of view in the vertical direction Z to 1x magnification in at least the area including the eyes of the occupants in the area to be monitored E, and magnifies the field of view in the horizontal direction X intersecting the vertical direction Z by a predetermined magnification. The camera module 10 captures an image formed by an imaging lens 132 with a field of view θ2 expanded horizontally in the X direction by an anamorphic lens 131. The pixel interpolation unit 22 interpolates pixels along the horizontal X direction in the compressed image Q2 captured by the camera module 10. The monitoring unit 231 detects the occupant's eyes based on the image Q4 with pixels interpolated by the pixel interpolation unit 22 and monitors the occupant. With this configuration, the occupant monitoring device 1 can capture a wide imaging area E1 with high resolution using a small image sensor 14, thus enabling proper monitoring of the occupant. In particular, the occupant monitoring device 1 is effective when monitoring the degree of eye opening of the driver D, as a high-resolution image Q4 is required. Furthermore, by using a small image sensor 14, the occupant monitoring device 1 can suppress cost increases.

[0028] [Variation] Next, a modified example of the embodiment will be described. In the modified example, components equivalent to those in the embodiment will be denoted by the same reference numerals, and their detailed descriptions will be omitted.

[0029] Figure 13 shows an example of the configuration of an optical system 13A according to a first modified embodiment. Optical system 13A differs from optical system 13 according to the embodiment in that it divides the area to be monitored E into two areas, a first imaging area E3a and a second imaging area E3b. Optical system 13A is composed of a prism 133 as a magnifying optical system and an imaging lens 132.

[0030] The prism 133 is an optical component that refracts light and is composed of a first magnifying optical section 133a as a first magnifying optical system and a second magnifying optical section 133b as a second magnifying optical system.

[0031] The first magnifying optical unit 133a magnifies the field of view in the vertical direction Z at 1:1 ratio and magnifies the field of view in the horizontal direction X at a predetermined magnification. The second magnifying optical unit 133b is different from the first magnifying optical unit 133a, is formed in the same shape as the first magnifying optical unit 133a, and is provided alongside the first magnifying optical unit 133a along the horizontal direction X. The second magnifying optical unit 133b magnifies the field of view in the vertical direction Z at 1:1 ratio and magnifies the field of view in the horizontal direction X at a predetermined magnification.

[0032] The imaging lens 132 is located between the prism 133 and the image sensor 14. The imaging lens 132, through the first magnifying optical unit 133a, forms an image on the image sensor 14 of the first imaging area E3a, which is half of the monitored area E, with a field of view that is equal to the vertical Z and magnified in the horizontal X direction. The imaging lens 132, through the second magnifying optical unit 133b, also forms an image on the image sensor 14 of the second imaging area E3b, which is the remaining half of the monitored area E, with a field of view that is equal to the vertical Z and magnified in the horizontal X direction.

[0033] As shown in Figure 14, the image sensor 14 captures an image formed by the imaging lens 132 via the prism 133. For example, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E3a within the monitored area E with a field of view expanded horizontally in the X direction by the first magnifying optical unit 133a, and a second image formed by the imaging lens 132 of a second imaging area E3b within the monitored area E that is different from the first imaging area E3a, with a field of view expanded horizontally in the X direction by the second magnifying optical unit 133b, as a single compressed image Q2.

[0034] The pixel interpolation unit 22 interpolates pixels along the horizontal direction X as appropriate in a single compressed image Q2 captured by the image sensor 14. In the example shown in Figure 14, there is a section (seam) between the first imaging area E3a and the second imaging area E3b that is different from each other, but this seam is stored as an area that is not used for monitoring during calibration.

[0035] As described above, in the crew monitoring device 1, the prism 133 includes a first magnifying optical unit 133a and a second magnifying optical unit 133b. The first magnifying optical unit 133a magnifies the field of view in the vertical direction Z to 1x magnification and magnifies the field of view in the horizontal direction X by a predetermined magnification. The second magnifying optical unit 133b is a different optical system from the first magnifying optical unit 133a, and magnifies the field of view in the vertical direction Z to 1x magnification and magnifies the field of view in the horizontal direction X by a predetermined magnification. The camera module 10 captures a first image, formed by imaging the first region of the monitored area E with the imaging lens 132 using the field of view magnified in the horizontal direction X by the first magnifying optical unit 133a, and a second image, formed by imaging the second region of the monitored area E, which is different from the first region, with the imaging lens 132 using the field of view magnified in the horizontal direction X by the second magnifying optical unit 133b, as a single compressed image Q2. The pixel interpolation unit 22 interpolates pixels along the horizontal direction X as appropriate in a single compressed image Q2 captured by the camera module 10. In this way, the crew monitoring device 1 may be divided into multiple regions for imaging.

[0036] Figure 15 shows an example of the configuration of an optical system 13B according to a second modification of the embodiment. The optical system 13B differs from the optical system 13A according to the first modification in that it uses a convex lens to divide the area to be monitored E into two areas, a first imaging area E4a and a second imaging area E4b. The optical system 13B is composed of a first magnifying lens 134a as a first magnifying optical system, a second magnifying lens 134b as a second magnifying optical system, and an imaging lens 132.

[0037] The first magnifying lens 134a is formed in the shape of a convex lens, and magnifies the field of view in the vertical direction Z at 1x magnification and the field of view in the horizontal direction X at a predetermined magnification. The second magnifying lens 134b is different from the first magnifying lens 134a, and is formed in the same convex lens shape as the first magnifying lens 134a, and is provided alongside the first magnifying lens 134a along the horizontal direction X. The second magnifying lens 134b magnifies the field of view in the vertical direction Z at 1x magnification and magnifies the field of view in the horizontal direction X at a predetermined magnification.

[0038] The imaging lens 132 is positioned between the first magnifying lens 134a, the second magnifying lens 134b, and the image sensor 14. The imaging lens 132 uses the first magnifying lens 134a to image the first imaging area E4a, which is half of the monitored area E, onto the image sensor 14 with a field of view that is equal to the vertical Z and magnified in the horizontal X. The imaging lens 132 also uses the second magnifying lens 134b to image the second imaging area E4b, which is the remaining half of the monitored area E, onto the image sensor 14 with a field of view that is equal to the vertical Z and magnified in the horizontal X.

[0039] The image sensor 14 captures an image formed by the imaging lens 132 via the first magnifying lens 134a and the second magnifying lens 134b. For example, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E4a within the monitored area E with a field of view expanded horizontally in the X direction by the first magnifying lens 134a, and a second image formed by the imaging lens 132 of a second imaging area E4b within the monitored area E that is different from the first imaging area E4a, with a field of view expanded horizontally in the X direction by the second magnifying lens 134b, as a single compressed image Q2.

[0040] The pixel interpolation unit 22 interpolates pixels along the horizontal X direction as appropriate in a single compressed image Q2 captured by the image sensor 14.

[0041] Figure 16 shows an example of the configuration of an optical system 13C according to a third modified embodiment. The optical system 13C differs from the optical system 13B, etc., according to modified embodiment 2, in that it uses mirrors and convex lenses to divide the area to be monitored E into two areas, a first imaging area E5a and a second imaging area E5b. The optical system 13C is composed of first to fourth mirrors 135a to 135d, a first magnifying lens 135e as a first magnifying optical system, a second magnifying lens 135f as a second magnifying optical system, and an imaging lens 132.

[0042] The first magnifying lens 135e is formed in the shape of a convex lens, and magnifies the field of view in the vertical direction Z at 1x magnification and the field of view in the horizontal direction X at a predetermined magnification. The second magnifying lens 135f is different from the first magnifying lens 135e, and is formed in the same convex lens shape as the first magnifying lens 135e, and is provided side by side with a gap between it and the first magnifying lens 135e along the horizontal direction X. The second magnifying lens 135f magnifies the field of view in the vertical direction Z at 1x magnification and magnifies the field of view in the horizontal direction X at a predetermined magnification.

[0043] The first mirror 135a is positioned opposite the first magnifying lens 135e and reflects the light emitted from the first magnifying lens 135e toward the second mirror 135b. The second mirror 135b is positioned opposite the first mirror 135a and the imaging lens 132 and reflects the light reflected from the first mirror 135a toward the imaging lens 132.

[0044] The third mirror 135c is positioned opposite the second magnifying lens 135f and reflects the light emitted from the second magnifying lens 135f toward the fourth mirror 135d. The fourth mirror 135d is positioned opposite the third mirror 135c and the imaging lens 132 and reflects the light reflected from the third mirror 135c toward the imaging lens 132.

[0045] The imaging lens 132 is positioned between the first magnifying lens 135e, the second magnifying lens 135f, and the image sensor 14. The imaging lens 132 uses the first magnifying lens 135e to image the first imaging area E5a, which is half of the monitored area E, onto the image sensor 14 with a field of view that is equal to the vertical Z and magnified in the horizontal X direction. The imaging lens 132 also uses the second magnifying lens 135f to image the second imaging area E5b, which is the remaining half of the monitored area E, onto the image sensor 14 with a field of view that is equal to the vertical Z and magnified in the horizontal X direction.

[0046] The image sensor 14 captures an image formed by the imaging lens 132 via the first magnifying lens 135e and the second magnifying lens 135f. For example, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E5a within the monitored area E with a field of view expanded horizontally in the X direction by the first magnifying lens 135e, and a second image formed by the imaging lens 132 of a second imaging area E5b within the monitored area E that is different from the first imaging area E5a within the monitored area E, with a field of view expanded horizontally in the X direction by the second magnifying lens 135f, as a single compressed image Q2.

[0047] The pixel interpolation unit 22 interpolates pixels along the horizontal X direction as appropriate in a single compressed image Q2 captured by the image sensor 14.

[0048] Figure 17 shows an example of the configuration of the optical system 13D according to the fourth modified embodiment. The optical system 13D differs from the optical system 13C, etc., according to the third modified embodiment in that it divides the area to be monitored E into two areas, a first imaging area E6a and a second imaging area E6b, and does not enlarge the field of view of the first imaging area E6a. The optical system 13D is composed of an anamorphic lens 131A, a prism 136 and an imaging lens 132.

[0049] The anamorphic lens 131A magnifies the field of view in the horizontal X direction for the second imaging area E6b by a predetermined magnification, while maintaining the field of view in the horizontal X direction for the first imaging area E6a at 1:1 magnification without magnification.

[0050] The prism 136 is composed of a 1:1 optical section 136a and a magnifying optical section 136b.

[0051] The 1:1 magnification optical unit 136a magnifies the field of view in the vertical direction Z and the field of view in the horizontal direction X. The magnification optical unit 136b is different from the 1:1 magnification optical unit 136a and is provided alongside the 1:1 magnification optical unit 136a along the horizontal direction X. The magnification optical unit 136b magnifies the field of view in the vertical direction Z and magnifies the field of view in the horizontal direction X by a predetermined magnification.

[0052] The imaging lens 132 is positioned between the anamorphic lens 131A and the image sensor 14. The imaging lens 132, using the anamorphic lens 131A and the 1:1 magnification optical unit 136a, forms an image on the image sensor 14 of the first imaging area E6a, which includes the driver D's eyes, within the monitored area E, with a field of view that is equal to the vertical Z and equal to the horizontal X. The imaging lens 132, using the anamorphic lens 131A and the magnification optical unit 136b, also forms an image on the image sensor 14 of the second imaging area E6b, which does not include the driver D's eyes, within the monitored area E, with a field of view that is equal to the vertical Z and magnified in the horizontal X.

[0053] The image sensor 14 captures an image formed by the imaging lens 132 via the prism 136 and the anamorphic lens 131A. For example, as shown in Figure 18, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E6a of the monitored area E at a 1:1 magnification angle in the horizontal X direction using the 1:1 magnification optical unit 136a and the anamorphic lens 131A, and a second image formed by the imaging lens 132 of a second imaging area E6b different from the first imaging area E6a of the monitored area E at a magnified angle in the horizontal X direction using the magnification optical unit 136b and the anamorphic lens 131A, as a single compressed image Q2.

[0054] The pixel interpolation unit 22 does not interpolate pixels along the horizontal direction X for the first image corresponding to the first image of a single image captured by the image sensor 14, and appropriately interpolates pixels along the horizontal direction X for the second image corresponding to the second image of a single image captured by the image sensor 14.

[0055] As described above, in the occupant monitoring device 1, the camera module 10 captures a first image formed by the imaging lens 132 with a field of view that is equal to the horizontal X without including the magnifying optical system, for a first region of the monitoring area E, and a second image formed by the imaging lens 132 with a field of view that is magnified horizontally in the X direction by the magnifying optical system, for a second region of the monitoring area E different from the first region, as a single compressed image Q2. The pixel interpolation unit 22 does not interpolate pixels along the horizontal X direction for the first image corresponding to the first image of the single compressed image Q2 captured by the camera module 10, and interpolates pixels along the horizontal X direction for the second image corresponding to the second image of the single compressed image Q2 captured by the camera module 10. With this configuration, the occupant monitoring device 1 does not magnify the first imaging area E6a, which includes the driver D's eyes, with equal magnification, so that the degree of eye opening of the driver D can be properly detected. Furthermore, the occupant monitoring device 1 can capture a wide area with high resolution in the second imaging region E6b, which does not include the driver D's eyes, thus enabling proper monitoring of the occupant.

[0056] Figure 19 shows an example of the configuration of an optical system 13E according to a fifth modification of the embodiment. The optical system 13E differs from the optical system 13C, etc., according to the third modification, in that it divides the area to be monitored E into three areas, namely a first imaging area E7a, a second imaging area E7b, and a third imaging area E7c, using mirrors and convex lenses. The optical system 13E is composed of first to fourth mirrors 137a to 137d, a first magnifying lens 137e as a first magnifying optical system, a second magnifying lens 137f as a second magnifying optical system, and an imaging lens 132.

[0057] The first magnifying lens 137e is formed in the shape of a convex lens, and magnifies the field of view in the vertical direction Z at 1x magnification and the field of view in the horizontal direction X at a predetermined magnification. The second magnifying lens 137f is different from the first magnifying lens 137e, and is formed in the same convex lens shape as the first magnifying lens 137e, and is provided side by side with a gap between it and the first magnifying lens 137e along the horizontal direction X. The second magnifying lens 137f magnifies the field of view in the vertical direction Z at 1x magnification and magnifies the field of view in the horizontal direction X at a predetermined magnification.

[0058] The first mirror 137a is positioned opposite the first magnifying lens 137e and reflects the light emitted from the first magnifying lens 137e toward the second mirror 137b. The second mirror 137b is positioned opposite the first mirror 137a and the imaging lens 132 and reflects the light reflected from the first mirror 137a toward the imaging lens 132.

[0059] The third mirror 137c is positioned opposite the second magnifying lens 137f and reflects the light emitted from the second magnifying lens 137f toward the fourth mirror 137d. The fourth mirror 137d is positioned opposite the third mirror 137c and the imaging lens 132 and reflects the light reflected from the third mirror 137c toward the imaging lens 132.

[0060] The imaging lens 132 is positioned between the first magnifying lens 137e, the second magnifying lens 137f, and the image sensor 14. The imaging lens 132 uses the first magnifying lens 137e to project an image of the first imaging area E7a of the monitored area E onto the image sensor 14 with a field of view that is equal in the vertical Z direction and magnified in the horizontal X direction. The imaging lens 132 also uses the second imaging area E7b of the monitored area E onto the image sensor 14 with a field of view that is equal in the vertical Z direction and magnified in the horizontal X direction. Furthermore, the imaging lens 132 uses the second magnifying lens 137f to project an image of the third imaging area E7c of the monitored area E onto the image sensor 14 with a field of view that is equal in the vertical Z direction and magnified in the horizontal X direction.

[0061] The image sensor 14 captures images formed by the imaging lens 132 via the first magnifying lens 137e and the second magnifying lens 137f, and images formed by the imaging lens 132 without using the first magnifying lens 137e and the second magnifying lens 137f. For example, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E7a of the monitored area E with a field of view expanded horizontally in the X direction by the first magnifying lens 137e, a second image formed by the imaging lens 132 of the second imaging area E7b of the monitored area E with a field of view equal to the horizontal X direction, and a third image formed by the imaging lens 132 of the third imaging area E7c of the monitored area E with a field of view expanded horizontally in the X direction by the second magnifying lens 137f, as a single compressed image Q2.

[0062] The pixel interpolation unit 22 interpolates pixels along the horizontal direction X as appropriate in the portions corresponding to the first and third images of the compressed image Q2 captured by the image sensor 14, but does not interpolate pixels in the portion corresponding to the second image.

[0063] Figure 20 shows an example of the configuration of the optical system 13F according to the sixth modified embodiment. The optical system 13F differs from the optical system 13C, etc., according to the third modified embodiment, in that it uses a magnifying mirror that magnifies as it moves away from the center. The optical system 13F is composed of first and second magnifying mirrors 138a and 138c, first and second mirrors 138b and 138d, a first magnifying lens 138e as a first magnifying optical system, a second magnifying lens 138f as a second magnifying optical system, and an imaging lens 132.

[0064] The first magnifying lens 138e is formed in the shape of a convex lens, and magnifies the field of view in the vertical direction Z at 1x magnification, while magnifying the field of view in the horizontal direction X at a predetermined magnification. The second magnifying lens 138f is different from the first magnifying lens 138e, and is formed in the same convex lens shape as the first magnifying lens 138e, and is provided side by side with a gap between them along the horizontal direction X. The second magnifying lens 138f magnifies the field of view in the vertical direction Z at 1x magnification, while magnifying the field of view in the horizontal direction X at a predetermined magnification.

[0065] The first magnifying glass 138a magnifies the field of view as it moves away from the center, and is positioned opposite the first magnifying lens 138e, reflecting the light emitted from the first magnifying lens 138e toward the second mirror 138b. The second mirror 138b is positioned opposite the first magnifying glass 138a and the imaging lens 132, and reflects the light reflected from the first magnifying glass 138a toward the imaging lens 132.

[0066] The second magnifying glass 138c magnifies the field of view as it moves away from the center, and is positioned opposite the second magnifying lens 138f, reflecting the light emitted from the second magnifying lens 138f toward the second mirror 138d. The second mirror 138d is positioned opposite the second magnifying glass 138c and the imaging lens 132, and reflects the light reflected from the second mirror 138d toward the imaging lens 132.

[0067] The imaging lens 132 is positioned between the first magnifying lens 138e and the second magnifying lens 138f and the image sensor 14. The imaging lens 132 uses the first magnifying mirror 138a and the first magnifying lens 138e to image the first imaging area E8a, which is half of the monitored area E, onto the image sensor 14 with a field of view that is equal to the vertical Z and magnified in the horizontal X direction. The imaging lens 132 also uses the second magnifying mirror 138c and the second magnifying lens 138f to image the second imaging area E8b, which is the remaining half of the monitored area E, onto the image sensor 14 with a field of view that is equal to the vertical Z and magnified in the horizontal X direction.

[0068] The image sensor 14 captures an image formed by the imaging lens 132 via the first magnifying lens 138e and the second magnifying lens 138f, etc. For example, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E8a of the monitored area E with a field of view expanded horizontally in the X direction by the first magnifying mirror 138a and the first magnifying lens 138e, and a second image formed by the imaging lens 132 of a second imaging area E8b of the monitored area E that is different from the first imaging area E8a, with a field of view expanded horizontally in the X direction by the second magnifying mirror 138c and the second magnifying lens 138f, as a single compressed image Q2.

[0069] The pixel interpolation unit 22 interpolates pixels along the horizontal X direction as appropriate in a single compressed image Q2 captured by the image sensor 14.

[0070] Figure 21 shows an example of the configuration of the optical system 13G according to the seventh modification of the embodiment. The optical system 13G differs from the optical system 13 according to the embodiment in that, as shown in Figure 22, it expands the field of view in the vertical direction Z in the second imaging area E9b which does not include the eyes of the driver D. The optical system 13G is composed of an anamorphic lens 131B, a prism 139, and an imaging lens 132.

[0071] The anamorphic lens 131B magnifies the field of view in the vertical Z direction by a predetermined magnification for the second imaging area E9b, which does not include the driver D's eyes, while maintaining the field of view in the horizontal X direction and vertical Z direction at 1:1 magnification for the first imaging area E9a, which includes the driver D's eyes.

[0072] The prism 139 is composed of a 1x optical section 139a and a magnifying optical section 139b.

[0073] The 1:1 magnification optical unit 139a magnifies the field of view in the horizontal direction X and the field of view in the vertical direction Z. The magnification optical unit 139b is different from the 1:1 magnification optical unit 139a, is formed in a different shape from the 1:1 magnification optical unit 139a, and is provided alongside the 1:1 magnification optical unit 139b along the vertical direction Z. The magnification optical unit 139b magnifies the field of view in the horizontal direction X and magnifies the field of view in the vertical direction Z by a predetermined magnification.

[0074] The imaging lens 132 is positioned between the anamorphic lens 131B and the image sensor 14. The imaging lens 132, using the anamorphic lens 131B and the 1:1 magnification optical unit 139a, forms an image on the image sensor 14 of the first imaging area E9a, which includes the driver D's eyes, within the monitored area E, with a field of view that is equal to the horizontal X and equal to the vertical Z. The imaging lens 132, using the anamorphic lens 131B and the magnification optical unit 139b, forms an image on the image sensor 14 of the second imaging area E9b, which includes the steering wheel portion of the monitored area E and does not include the driver D's eyes, with a field of view that is equal to the horizontal X and magnified in the vertical Z.

[0075] The image sensor 14 captures an image formed by the imaging lens 132 via the prism 139 and the anamorphic lens 131B. For example, the image sensor 14 captures a first image formed by the imaging lens 132 of the first imaging area E9a within the monitored area E with a field of view of equal magnification in the horizontal X and vertical Z directions using the 1:1 magnification optical unit 139a and the anamorphic lens 131B, and a second image formed by the imaging lens 132 of a second imaging area E9b within the monitored area E that is different from the first imaging area E9a, with a field of view expanded in the vertical Z direction using the magnification optical unit 139b and the anamorphic lens 131B, as a single compressed image.

[0076] The pixel interpolation unit 22 does not interpolate pixels along the vertical direction Z for the first image corresponding to the first image of a single image captured by the image sensor 14, nor does it interpolate pixels along the vertical direction Z for the second image corresponding to the second image of a single image captured by the image sensor 14.

[0077] The monitoring unit 231 detects the degree of eye opening of driver D based on the first image corresponding to the first imaging area E9a, which is captured at the same magnification angle of view. The monitoring unit 231 also detects hands-on (driver D's intention to drive) based on the second image corresponding to the second imaging area E9b, which is captured at an enlarged angle of view. The pixel interpolation unit 22 may interpolate pixels along the vertical direction Z in the second image corresponding to the second image of a single image captured by the image sensor 14, in which case the accuracy of hands-on detection can be improved. [Explanation of Symbols]

[0078] 1. Crew monitoring device E. Monitoring Area 13 Optical system 10. Camera module (imaging unit) 20 Control box (image processing unit) 231 Monitoring Department 132 Imaging lens 131 Anamorphic lens (magnifying optical system) 133a First Magnifying Optical Section (First Magnifying Optical System) 133b Second Magnifying Optical Section (Second Magnifying Optical System) θ1, θ2 field of view Z vertical direction X Horizontal

Claims

1. An optical system installed in the vehicle that forms an image of the area to be monitored, including the occupants, An imaging unit that captures the image formed by the optical system, An image processing unit that processes the image captured by the imaging unit, The system includes a monitoring unit that detects the eyes of the occupant and monitors the occupant based on the image processed by the image processing unit, The optical system includes an imaging lens that forms an image on the imaging unit, and a magnifying optical system that magnifies the vertical field of view corresponding to the imaging lens at equal magnitude in at least the area including the occupant's eyes in the area to be monitored, and magnifies the horizontal field of view intersecting the vertical direction by a predetermined magnification. The imaging unit captures an image formed by the imaging lens at the angle of view expanded horizontally by the magnifying optical system. The image processing unit interpolates pixels along the horizontal direction in the image captured by the imaging unit. The crew monitoring device is characterized in that the monitoring unit detects the crew member's eyes based on the image obtained by interpolating the pixels by the image processing unit and monitors the crew member.

2. The magnifying optical system includes a first magnifying optical system that magnifies the vertical field of view at the same magnification and the horizontal field of view at a predetermined magnification, The system includes a second magnification optical system, which is different from the first magnification optical system, and which magnifies the vertical field of view at the same magnification and the horizontal field of view at a predetermined magnification, The imaging unit captures a first image by imaging a first region of the monitored area with the imaging lens at the angle of view expanded horizontally by the first magnifying optical system, and a second image by imaging a second region of the monitored area, different from the first region, with the imaging lens at the angle of view expanded horizontally by the second magnifying optical system, as a single image. The crew monitoring device according to claim 1, wherein the image processing unit interpolates pixels along the horizontal direction in the single image captured by the imaging unit.

3. The imaging unit captures a first image formed by the imaging lens on a first region of the monitored area at the same horizontal angle of view without using the magnifying optical system, and a second image formed by the imaging lens on a second region of the monitored area different from the first region at the horizontal angle of view magnified via the magnifying optical system, as a single image. The crew monitoring device according to claim 1, wherein the image processing unit does not interpolate pixels along the horizontal direction for the first image corresponding to the first image of the one image captured by the imaging unit, and interpolates pixels along the horizontal direction for the second image corresponding to the second image of the one image captured by the imaging unit.

Images