Vehicle monitoring device

Abstract

To provide a vehicle monitoring device that can improve vehicle functions by using cameras while reducing the number of cameras to be installed.SOLUTION: A vehicle monitoring device is equipped in a vehicle 1 and captures images of a monitored object within its capture range using omnidirectional cameras 60a, 60b, the omnidirectional cameras 60a, 60b are attached to a support member 70 supported on a corner portion 3C of a cab 3 and positioned outward from the side surface of the cab 3 and within the range from the upper end to the lower end of a windshield 33, and the capture range of the omnidirectional cameras 60a, 60b includes at least the interior of the cab 3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle monitoring device. [Background technology]

[0002] As automotive technology advances, technologies are becoming more widespread in vehicles such as vans, trucks, and buses that use cameras, radar, and other devices to recognize objects inside and outside the vehicle and operate the devices equipped in the vehicle based on that information.

[0003] For example, Patent Document 1 discloses a device that installs a driver monitor camera on the steering column inside the vehicle cabin to monitor the driving state of the driver, and also installs a forward monitoring sensor such as a millimeter wave radar on the vehicle to monitor whether the driver's line of sight obtained by the driver monitor camera is appropriate for the road conditions ahead obtained by the forward monitoring sensor, that is, whether the driver is not looking away from the road while driving. In this case, a driver monitor camera that monitors the inside of the vehicle and a forward monitoring sensor that monitors the outside of the vehicle are required.

[0004] Focusing particularly on camera technology, for example, technologies have been developed that use cameras mounted on the instrument panel inside the vehicle to capture images of the driver and detect sudden changes in the driver's physical condition or distracted driving, or capture images of the area in front of the vehicle to detect the risk of a collision with a vehicle ahead. Furthermore, the range of uses is expanding to include digital mirrors that capture images of the area behind the vehicle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2010-257293 A Summary of the Invention [Problem to be solved by the invention]

[0006] By the way, as the number of vehicle devices that recognize the inside and outside of the vehicle with cameras and control based on the image information increases, convenience and safety can be improved, but the number of cameras also increases, leading to increased costs and more complicated image processing. In addition, when cameras are installed inside the vehicle, installation space is required, and the installation of a mounting cover for the camera may narrow the driver's field of view.

[0007] The present invention was devised with an eye on such problems, and aims to provide a vehicle monitoring device that can improve vehicle functions by utilizing cameras while reducing the number of cameras that need to be installed. [Means for solving the problem]

[0008] The present invention has been made to solve at least part of the above problems, and can be realized in the following aspects or application examples. (1) The vehicle monitoring device according to this application example is a vehicle monitoring device that captures images of a monitoring target within a capture range using an omnidirectional camera mounted on a vehicle, the omnidirectional camera being attached to a front corner of the cab, being outboard of the side of the cab, and being positioned within a range from the top to the bottom of the windshield, and the capture range of the omnidirectional camera including at least the interior of the cab.

[0009] According to this application example, since the omnidirectional camera is disposed outside the cab, it can capture the surroundings of the vehicle from almost the entire circumference of the vehicle, and since the capturing range of the omnidirectional camera includes the interior of the cab, it is possible to capture the interior of the cab, for example, the driver's condition, without providing a separate camera inside the cab. Therefore, it is possible to realize improvement of the vehicle function using the cameras while suppressing the number of cameras installed.

[0010] (2) In this application example, the omnidirectional cameras are installed at the left and right front corners of the cab, respectively, and of the respective omnidirectional cameras, it is preferable that the omnidirectional camera on the driver's side is positioned rearward of the front end face of the cab and outward of the side face of the cab, and the omnidirectional camera on the passenger's side is positioned forward of the front end face of the cab and outward of the side face of the cab or on an extension of the side face of the cab. With this configuration, the two cameras can work together to capture the situation on the driver's side, while the omnidirectional camera on the passenger side can also capture the space directly in front of the cab.

[0011] (3) In this application example, it is preferable that the vehicle includes a windshield wiper device, and the shooting range of the omnidirectional camera includes at least a movable range of the wiper device. With this configuration, even if rain or snow adheres to the windshield, the interior of the cab can be photographed accurately with the omnidirectional camera by operating the windshield wiper device.

[0012] (4) In this application example, it is preferable that a control device is provided that controls the in-vehicle device based on information about the monitored object photographed by the omnidirectional camera, and that the control device controls the in-vehicle device equipped on the vehicle based on information about the monitored object photographed by the omnidirectional camera. With this configuration, various in-vehicle devices mounted on the vehicle can be controlled based on information about the monitored object captured by the omnidirectional camera.

[0013] (5) In this application example, it is preferable that the omnidirectional cameras are provided on both the driver's side and the passenger's side of the cab, the monitored object includes the driver inside the cab, and image information of the driver captured by the omnidirectional cameras on the driver's side and the passenger's side is used as control input information for a driver monitoring device. With this configuration, the driver monitoring device can function without installing a camera inside the cab to capture the face and posture of the driver.

[0014] (6) In this application example, when the driver is a subject of monitoring, it is preferable that image information regarding the driver's face captured by the omnidirectional camera on the driver's side and image information regarding changes in the driver's posture captured by the omnidirectional camera on the passenger's side are used as control input information for the driver monitoring device. With this configuration, the omnidirectional camera on the driver's side, which is closer to the driver, captures image information related to the driver's face, such as eyelid movements and mouth, and uses this information as control input information for the driver monitoring device, making it possible to quickly grasp the driver's drowsiness or fatigue. Also, the omnidirectional camera on the passenger side captures image information related to changes in the driver's posture and uses this information as control input information for the driver monitoring device, making it possible to quickly grasp if the driver falls into a situation where he or she cannot drive safely for some reason.

[0015] (7) In this application example, it is preferable that the omnidirectional cameras are provided on the driver's side and the passenger's side of the cab, the monitored object includes the rear of the vehicle side, and image information of the rear of the vehicle side captured by the omnidirectional cameras on the driver's side and the passenger's side is used as a display image on a vehicle rear display monitor. With this configuration, the omnidirectional camera can be used not only to take pictures of the inside of the cab, but also as a camera for displaying images on the vehicle rear display monitor.

[0016] (8) In this application example, it is preferable that the omnidirectional cameras are provided on the driver's side and the passenger's side of the cab, the monitored objects include a preceding vehicle in front of the vehicle, and image information of the preceding vehicle captured by the omnidirectional cameras on the driver's side and the passenger's side is used as control input information for a preceding vehicle following control. With this configuration, the omnidirectional camera can be used not only to take pictures of the inside of the cab, but also to obtain control input information for the preceding vehicle following control.

[0017] (9) In this application example, it is further preferable that the omnidirectional cameras are provided on the driver's side and passenger's side of the cab, the monitored objects include a preceding vehicle in front of the vehicle, and image information of the preceding vehicle captured by the omnidirectional cameras on the driver's side and passenger's side is used as control input information for preceding vehicle collision prevention control. With this configuration, the omnidirectional camera can be used not only to take pictures of the inside of the cab, but also to obtain control input information for the preceding vehicle collision prevention control.

[0018] (10) In this application example, it is further preferable that the omnidirectional cameras are provided on the driver's side and passenger's side of the cab, the monitored objects include the lanes of the road on which the vehicle is traveling, and image information of the lanes captured by the omnidirectional cameras on the driver's side and passenger's side is used as control input information for lane keeping control. With this configuration, the omnidirectional camera can be used not only to take pictures of the inside of the cab, but also to obtain control input information for lane keeping control.

[0019] (11) In this application example, it is further preferable that the omnidirectional camera is provided on the passenger side of the cab, the monitored object includes the vehicle exterior area on the passenger side of the vehicle while it is moving, and image information of the area on the left side of the vehicle captured by the omnidirectional camera is used as control input information for left-turn collision prevention control. With this configuration, a single omnidirectional camera can be used not only to take pictures of the inside of the cab, but also to provide control input information for prevention of collisions (when turning left) in the driver's blind spot (passenger side).

[0020] (12) In this application example, it is preferable that the omnidirectional camera is attached to a support member supported at a front corner portion of the cab, and the support member is supported at a connection portion between a front panel and a front pillar at the front corner portion of the cab. With this configuration, for example in the case of a small to medium-sized truck, it is possible to photograph the driver and others inside the cab without increasing the size of the support member.

[0021] (13) In this application example, it is also preferable that the omnidirectional camera is attached to a support member supported at a front corner portion of the cab, and the support member is supported at a connection portion between a roof panel and a front pillar at the front corner portion of the cab. With this configuration, for example in the case of a large truck, it is possible to capture an image of the driver or the like inside the cab without increasing the size of the support member.

[0022] (14) In this application example, it is also preferable that the omnidirectional camera is attached to a support member supported at a front corner portion of the cab, and the support member is supported at a middle portion of a front pillar at the front corner portion of the cab. With this configuration, for example in cases where the front window is large and extends over a wide area above and below the driver, such as in a large bus, it is possible to photograph the driver and others inside the cab without enlarging the size of the support member.

[0023] (15) In this application example, it is also preferable that the vehicle does not have a bonnet in front of the cab. With this configuration, for example, when applied to vehicles such as trucks and buses in which side mirrors are conventionally attached to the front corners of the cab, the omnidirectional camera can be installed without significant structural modifications. Effect of the Invention

[0024] According to the present invention, it is possible to improve vehicle functions using cameras while suppressing the number of installed cameras. [Brief description of the drawings]

[0025] [Figure 1] 1 is a perspective view of a vehicle according to an embodiment, seen from the left front. [Diagram 2] FIG. 1 is a perspective view showing an omnidirectional camera according to an embodiment. [Diagram 3] 3 is a side view for explaining a shooting area of ​​the omnidirectional camera shown in FIG. 2. [Figure 4] 2 is a perspective view showing an omnidirectional camera disposed in a corner portion on the right side (driver's side) of the cab of the vehicle shown in FIG. [Diagram 5] 2 is a perspective view showing an omnidirectional camera disposed in a corner portion on the left side (passenger seat side) of the cab of the vehicle shown in FIG. 1. [Figure 6] FIG. 2 is a side view of the right side (driver's side) of the cab of the vehicle shown in FIG. [Figure 7] FIG. 2 is a plan view of the front part of the cab of the vehicle shown in FIG. [Figure 8] FIG. 2 is a front view of the front part of the cab of the vehicle shown in FIG. [Figure 9] 2 is a schematic plan view showing a shooting area of ​​an omnidirectional camera of the vehicle shown in FIG. 1. [Figure 10] FIG. 13 is a schematic plan view showing a shooting area of ​​an omnidirectional camera of a vehicle according to a modified example. [Figure 11] FIG. 2 is a control block diagram illustrating a processing system for detecting information from the omnidirectional camera according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The embodiments of the present invention will be described with reference to the drawings. The following embodiments are merely examples, and are not intended to exclude the application of various modifications and techniques not specified in the embodiments. The configurations of the following embodiments can be modified in various ways without departing from the spirit of the embodiments. In addition, they can be selected or combined as necessary.

[0027] In the following description, the forward direction of the vehicle is defined as the front, and the opposite direction (the backward direction of the vehicle) is defined as the rear, with left and right defined based on the state in which the vehicle faces forward. The front-to-rear direction is also referred to as the vehicle length direction, and the left-to-right direction is also referred to as the vehicle width direction. Furthermore, the direction perpendicular to both the vehicle length direction and the vehicle width direction is referred to as the up-down direction. The vehicle is assumed to be on a horizontal road surface and in a position in which the up-down direction coincides with the vertical direction (the downward direction coincides with the direction of gravity). In this position, the vertical upward direction is defined as the height direction.

[0028] [1. Configuration] [1.1. Vehicle configuration] First, a vehicle equipped with a vehicle monitoring device 6 according to this embodiment will be described. As shown in FIG. 1, the vehicle 1 is a vehicle that does not have a bonnet in front of the cab, that is, a so-called non-bonnet vehicle, and a medium-sized or small-sized truck is exemplified here. The vehicle 1 includes a frame 2, a cab 3 mounted on the front of the frame 2, and a cargo box 4 mounted on the frame 2 behind the cab 3. Wheels 5 are attached to the lower part of the frame 2 via a suspension mechanism (not shown).

[0029] The cab 3 is provided at its front with a front panel 31 and front pillars 32, 32 provided on the left and right sides above the front panel 31, and a windshield 33 is provided between the left and right front pillars 32, 32. In addition, the cab 3 is provided with doors 34 on both the left and right sides, and each door 34 is provided with a door glass 35. Inside the cab 3, a driver's seat 36a is provided on the right side of the vehicle, and a passenger seat 36b is provided on the left side of the vehicle. In the cab 3 of such a vehicle 1, an omnidirectional camera 60 provided in the monitoring device 6 is disposed in a front corner portion 3C where the front pillars 32, 32 are provided. In this embodiment, an omnidirectional camera 60 is disposed in each of the left and right front corner portions 3C.

[0030] [1.2. Omnidirectional camera] Here, the omnidirectional camera 60 will be described. The omnidirectional camera 60 is a celestial sphere camera with a 360-degree shooting range. As shown in Figs. 2 and 3, the omnidirectional camera 60 of this embodiment is equipped with two lenses 62a and 62b on one side (the right side of the vehicle, the driver's seat 36a side) and the other side (the left side of the vehicle, the passenger seat 36b side) of a case 61, and is configured to capture one side of the screen (180 degrees + d, shown by angles α1 and α2 in Fig. 3) with one lens, synthesize two images, and shoot 360 degrees. Although not shown in Figs. 2 and 3, the omnidirectional camera 60 is equipped with an image sensor 63 (see Fig. 11) that captures the subject captured by each lens 62a and 62b and converts it into image data of an analog electric signal, and an image processing circuit 64 (see Fig. 11) that digitalizes the image data converted by the image sensor 63.

[0031] Such an omnidirectional camera 60 is attached to the cab 3 via a support member 70, as shown in Fig. 1 and Fig. 4 to Fig. 8. In this embodiment, the support member 70 includes a bracket 71 attached to a connection portion 37 between the front panel 31 and the front pillars 32, 32 at each corner portion 3C of the cab 3, an arm portion 72 having a base portion attached to the bracket 71, and a mounting base 73 provided at the tip of the arm portion 72. In the following description, the omnidirectional camera 60 on the driver's seat side is indicated by reference symbol 60a, and the omnidirectional camera 60 on the passenger's seat side is indicated by reference symbol 60b.

[0032] The arm portion 72 includes a base end portion 72a extending substantially horizontally from a horizontal mounting seat surface 71a formed horizontally on the bracket 71, and a tip end portion 72c extending substantially vertically upward from the base end portion 72a via a bent portion 72b. The planar position of the omnidirectional camera 60 attached to the mounting base 73 with respect to the cab 3 is determined by the shape, length, etc. of the base end portion 72a. In addition, the height position of the omnidirectional camera 60 attached to the mounting base 73 with respect to the cab 3 is determined by the length, etc. of the tip end portion 72c.

[0033] The arm portion 72 of the support member 70 on the driver's seat 36a side has a base end portion 72a extending outward from the side surface of the cab 3 (the outer panel of the door 34) and rearward from the front surface of the cab 3 (the front panel 31). As shown in FIG. 7, the omnidirectional camera 60 is located outward from the side surface of the cab 3 and rearward from the front surface of the cab 3. The reason why the planar position of the omnidirectional camera 60a on the driver's seat 36a side is set outward from the side surface of the cab 3 in this way is to include the rear of the vehicle in the shooting range. The reason why the planar position of the omnidirectional camera 60a on the driver's seat 36a side is set rearward from the front surface of the cab 3 is to shoot the facial expression of the driver 8 seated in the driver's seat 36a from a closer position through the door glass 35.

[0034] The arm portion 72 of the support member 70 on the passenger seat 36b side has a base end portion 72a extending outward from the side surface of the cab 3 (the outer panel of the door 34) and forward from the front surface of the cab 3 (the front panel 31), and the omnidirectional camera 60 is located outward from the side surface of the cab 3 and forward from the front surface of the cab 3, as shown in FIG. 7. The reason why the planar position of the omnidirectional camera 60b on the passenger seat 36b side is set outward from the side surface of the cab 3 in this way is to include the rear of the vehicle in the shooting range. The reason why the planar position of the omnidirectional camera 60b on the passenger seat 36b side is set forward from the front surface of the cab 3 is to include the area just before the cab 3 in the shooting range and to shoot the facial expression of the driver 8 seated in the driver's seat 36a through the windshield 33.

[0035] As shown in Fig. 6, the tip end portions 72c of the arm portions 72 of the support members 70 on the driver's seat 36a side and the passenger seat 36b side extend to a predetermined height. This height is a height at which it is easy to capture the facial expression of the driver 8 seated in the driver's seat 36a. In this embodiment, the height is set in this manner in order to capture the situation of the driver 8 with both the omnidirectional camera 60a on the driver's seat 36a side and the omnidirectional camera 60b on the passenger seat 36b side. However, for example, when the situation of the driver 8 is configured to be captured only by the omnidirectional camera 60a on the driver's seat 36a side, the height position of the omnidirectional camera 60b on the passenger seat 36b side can be set from a different perspective.

[0036] As shown in Figs. 7 and 8, the omnidirectional camera 60b on the passenger seat 36b side captures an image of the driver 8 in the cab 3 through the windshield 33. In rainy weather or snowy conditions, raindrops or snow may accumulate on the windshield 33. In response to this, the capture range of the omnidirectional camera 60b on the passenger seat 36b side includes at least the movable range 37 of a wiper device (not shown) for the windshield 33. Therefore, even if rain or snow is attached to the windshield 33, it is possible to capture an image of the driver 8 in the cab 3 by operating the wiper device. In Fig. 7, the capture range of the omnidirectional camera 60b on the passenger seat 36b side is shown with a background pattern. In Fig. 8, the portion of the windshield 33 other than the movable range 37 of the wiper device is shown with a background pattern.

[0037] 9 is a plan view showing the shooting range of the omnidirectional camera 60a on the driver's seat 36a side and the omnidirectional camera 60b on the passenger's seat 36b side. In this embodiment, one lens 62a of each of the omnidirectional cameras 60a, 60b is oriented toward the front of the vehicle, and the other lens 62b is oriented toward the rear of the vehicle. However, the orientation of the lenses 62a, 62b is not limited to this.

[0038] 9, the forward-facing lens 62a of the omnidirectional camera 60a on the driver's seat 36a side has an imaging range of α11, and the rearward-facing lens 62b of the omnidirectional camera 60a on the driver's seat 36a side has an imaging range of α12. The forward-facing lens 62a of the omnidirectional camera 60b on the passenger seat 36b side has an imaging range of α21, and the rearward-facing lens 62b of the omnidirectional camera 60b on the passenger seat 36b side has an imaging range of α22.

[0039] In FIG. 9, the area that can be photographed by at least one lens is shown in gray. Among the areas that can be photographed, areas that can be photographed by multiple lenses are shown in a darker gray shade. The number attached to the shade indicates the number of lenses that can be photographed, and the shade is shown in three stages: an area that can be photographed by only one lens (attached number 1), an area that can be photographed by two lenses, an area that can be photographed by three lenses (attached number 2), and an area that can be photographed by three lenses (attached number 3). The shaded area indicates the range that can be stereo-measured by the front lens 62a of the omnidirectional camera 60a on the driver's seat 36a side and the front lens 62a of the omnidirectional camera 60a on the passenger seat 36b side.

[0040] The area behind the rear wall of the cab 3 is assumed to be an area that cannot be photographed, and the door glass 35 of the cab 3 is assumed to cover the entire side. However, the area that exists behind two pieces of door glass 35 is assumed to be an area that cannot be photographed. In addition, the shadow of the pillar 32 is assumed to be an area that cannot be photographed, but the presence of occupants such as the driver's seat 36a, the passenger seat 36b, and the driver 8 is ignored.

[0041] As shown in FIG. 9, there are small areas immediately to the sides of each of the omnidirectional cameras 60a, 60b that cannot be photographed due to the shadow of the case 61, and there are areas behind the vehicle that cannot be photographed due to the shadow of the vehicle, but the omnidirectional cameras 60a, 60b can capture almost the entire area around the vehicle.

[0042] [1.3. Camera information processing system configuration] 11, each of the omnidirectional cameras 60a and 60b includes an image sensor 63 that captures an image of a subject captured by a lens 62a or 62b and converts the image data into image data of an analog electric signal, and an image processing circuit 64 that A / D converts the image data converted by the image sensor 63 and then converts it into a digital electric signal. Note that the image data here is image information in the form of an electric signal, and includes image data in the form of an analog electric signal and image data in the form of a digital electric signal.

[0043] The image data of the digital signal processed by the image processing circuit 64 is spherical data, and since the distortion is greater at the periphery, an image data processing device 80 is connected to correct this and obtain planar data in each direction. The image data processing device 80 converts the spherical data into planar data in each direction. At this time, since the side image includes a seam between the two images, a stitching process can be used to combine the two images to obtain a side planar image.

[0044] In this way, planar data in each direction can be obtained by the conversion processing of the image data processing device 80, so that planar data of the desired monitoring target can be obtained. Since the vehicle is equipped with some kind of control device that controls the on-board devices installed in the vehicle according to the status of the monitoring target obtained from the images taken by the omnidirectional cameras 60a, 60b, these control devices can be controlled based on the information of the monitoring target processed by the image processing circuit 64.

[0045] For example, if the vehicle 1 is equipped with a driver monitoring device 81 that monitors the driver 8's inattentive or drowsy driving status based on the driver's face and posture and notifies the driver, the image data (image data of driver images A and B) output from the image data processing device 80 can be used as control input information for the driver monitoring device 81.

[0046] Specifically, image data of a driver image A obtained by processing image data from the rear lens 62b of the omnidirectional camera 60a on the driver's seat 36a side by the image data processing device 80, and image data of a driver image B obtained by processing image data from the rear lens 62b of the omnidirectional camera 60b on the passenger seat 36b side by the image data processing device 80 are each used as control input information for the driver monitoring device 81.

[0047] In this case, the face of the driver 8 is set as the subject of image capture by the omnidirectional camera 60a on the driver's seat 36a side, and the change in the driver's posture is set as the subject of image capture by the omnidirectional camera 60b on the passenger seat 36b side. As a result, the omnidirectional camera 60a on the driver's seat 36a side, which is closer to the driver 8, uses image information (image data) related to the face, such as the movement of the driver's eyelids (see the two-dot chain line in FIG. 8) and the area around the mouth, as control input information for the driver monitoring device 81, so that it is possible to quickly and accurately grasp the occurrence of the driver 8's inattentiveness, drowsiness, fatigue, etc.

[0048] In addition, image information (image data) regarding changes in posture of the upper body, etc. of the driver 8 is collected by the omnidirectional camera 60b on the passenger seat 36b side and used as control input information for the driver monitoring device 81, making it possible to quickly detect changes in posture due to the driver 8 falling asleep, etc.

[0049] Based on the driver images A and B, it is judged whether the driver 8 is looking away or falling asleep, and if it is judged that the driver 8 is looking away or falling asleep, the driver 8 is notified, thereby preventing the driver from looking away or falling asleep while driving.

[0050] In addition, if the vehicle 1 is equipped with an abnormality control device 82 that performs abnormality control such as stopping the vehicle 1 on the roadside or reporting the driver 8's abnormality to the outside if it determines that the driver has a sudden illness or other health abnormality, it can determine the abnormality of the driver 8 based on the driver images A and B and perform driver abnormality control.

[0051] Furthermore, if the vehicle 1 is equipped with a following running control device 83 that performs following running control to follow the preceding vehicle based on image information in front of the vehicle, the image data (image data of the forward images A and B) output from the image data processing device 80 can be used as control input information for the vehicle following running control. That is, the position of the preceding vehicle can be determined by using, as stereo camera data (image data of the stereo camera), the image data of the forward image A obtained by processing the image data of the forward lens 62a of the omnidirectional camera 60a by the image data processing device 80 and the image data of the forward image B obtained by processing the image data of the forward lens 62a of the omnidirectional camera 60b by the image data processing device 80.

[0052] Furthermore, if the vehicle 1 is equipped with a collision prevention control device 84 that performs collision prevention control to prevent a collision with a preceding vehicle based on image information in front of the vehicle, the image data (image data of the front images A and B) output from the image data processing device 80 can be used as control input information for the preceding vehicle collision prevention control. In other words, the image data of the front images A and B obtained by processing the image data of each of the front lenses 62a of the omnidirectional cameras 60a and 60b by the image data processing device 80 can be used as stereo camera data to determine the position of the preceding vehicle and perform the preceding vehicle collision prevention control.

[0053] Furthermore, if the vehicle 1 is equipped with a lane keeping control device 85 that performs lane keeping control to keep the vehicle in the lane it is traveling in based on image information of a white line ahead of the vehicle, image information of the white line obtained from image data (data images of front images A and B) output from the image data processing device 80 can be used as control input information for the lane keeping control. In other words, the image data of the front images A and B obtained by processing image data of each of the front lenses 62a of the omnidirectional cameras 60a and 60b by the image data processing device 80 can be used as stereo camera data to determine the position of the white line ahead of the vehicle, and lane keeping control can be performed.

[0054] Furthermore, if the vehicle 1 is equipped with an entrapment prevention control device 86 that issues a warning based on image information on the sides of the vehicle when turning (particularly when turning left when the side of the vehicle on the passenger side is in the driver's blind spot), each piece of image information (side image information) on the sides of the vehicle obtained from image data (image data of side images A and B) output from the image data processing device 80 can be used as control input information for entrapment prevention control. In other words, the presence or absence of an entrapment target can be determined using image data of a right side image A or a left side image B obtained by processing image data of the front lens 62a and the rear lens 62b of either of the omnidirectional cameras 60a and 60b by the image data processing device 80, and entrapment prevention control can be implemented.

[0055] Furthermore, when the vehicle 1 is equipped with a side display device 87 that displays the situation on the sides of the vehicle on a side monitor based on side image information, each piece of side image information obtained from the image data (image data of side images A and B) output from the image data processing device 80 can be used as control input information for side image display control. That is, the right side image can be displayed based on the image data of the right side image A, and the left side image can be displayed on the side monitor based on the image data of the left side image B.

[0056] In addition, when the vehicle 1 is equipped with a rear display device 88 that displays the situation behind the vehicle on a rear monitor based on image information behind the vehicle, each piece of rear image information obtained from image data (data of rear images A and B) output from the image data processing device 80 can be used as control input information for rear image display control. That is, the image data of the right rear image A obtained by processing image data of the rear lens 62b of the omnidirectional camera 60a with the image data processing device 80 and the image data of the left rear image B obtained by processing image data of the rear lens 62b of the omnidirectional camera 60b with the image data processing device 80 can be synthesized to display the situation behind the vehicle on the rear monitor. In addition, it is also possible to provide two rear monitors for the right rear and the left rear, and display the right rear image on the right rear monitor based on image data of the right rear image A, and display the left rear image on the left rear monitor based on image data of the left rear image B.

[0057] [2. Actions and Effects] Since the vehicle monitoring device 6 according to this embodiment is configured as described above, it is possible to obtain the following actions and effects.

[0058] (1) In the present monitoring device 6, the omnidirectional cameras 60a, 60b are attached to a support member 70 supported by a front corner portion 3C of the cab 3, and are disposed outside the side surface of the cab 3 and within a range from the upper end to the lower end of the windshield 33, and the shooting range of the omnidirectional cameras 60a, 60b includes at least the interior of the cab 3. Since the omnidirectional cameras 60a, 60b are disposed outside the cab 3, the surrounding conditions of the vehicle 1 can be photographed from almost the entire circumference of the vehicle 1, and the shooting range of the omnidirectional cameras 60a, 60b includes the interior of the cab, so the situation of the driver 8 inside the cab can be photographed without providing a separate camera inside the cab 3. Therefore, it is possible to realize improvement of vehicle functions using cameras while suppressing the number of cameras installed.

[0059] (2) In this embodiment, the omnidirectional camera 60a on the driver's seat 36a side is disposed rearward of the front end face of the cab 3 and outward of the side face of the cab 3, and the omnidirectional camera 60b on the passenger's seat 36b side is disposed forward of the front end face of the cab 3 and outward of the side face of the cab 3, so that the two omnidirectional cameras 60a, 60b can jointly capture the situation on the driver's seat 36a side, mainly using the omnidirectional camera 60a on the driver's seat 36a side, and the omnidirectional camera 60b on the passenger's seat 36b side can also capture the space immediately in front of the cab 3. Therefore, the situation of the driver's seat 36a can also be grasped.

[0060] (3) In this embodiment, the imaging range of the omnidirectional camera 60 includes at least the movable range of the windshield wiper device 37. Therefore, even if there are raindrops or snow on the windshield, the interior of the cab 3 and the situation of the driver 8 can be accurately captured by the omnidirectional camera 60 by moving the windshield wiper device 37.

[0061] (4) In this embodiment, information about the monitored object captured by the omnidirectional cameras 60a, 60b is used to operate the control device, thereby making it possible to control various on-board devices equipped in the vehicle, making the system highly versatile.

[0062] (5) In this embodiment, the objects monitored by the omnidirectional cameras 60a, 60b include the driver 8 inside the cab 3, and image data of the driver 8 captured by the omnidirectional cameras 60a, 60b is used as control input information for the driver monitoring device 81 that detects the driver 8's looking away or falling asleep and notifies the driver 8. Therefore, the driver monitoring device 81 can function without installing a camera inside the cab 3 that captures an image of the driver.

[0063] (6) In this embodiment, when the driver 8 in the cab 3 is the subject of monitoring by the omnidirectional cameras 60a, 60b, image information on the face of the driver 8 captured by the omnidirectional camera 60a on the driver's seat 36a side and image information on a change in the driver's posture captured by the omnidirectional camera 60b on the passenger seat 36b side are used as control input information for the driver monitoring device 81, so that the drowsiness or fatigue state of the driver 8 can be grasped early by the omnidirectional camera 60a on the driver's seat 36a side, which is closer to the driver 8. In addition, image information on a change in the driver's posture by the omnidirectional camera 60b on the passenger seat 36b side is used as control input information for the driver monitoring device, so that a change in the driver's posture due to drowsiness, etc., can be grasped early.

[0064] (7) In this embodiment, the objects monitored by the omnidirectional cameras 60a, 60b include the rear sides of the vehicle, and image information of the rear sides of the vehicle captured by the omnidirectional cameras 60a, 60b on the driver's seat 36a side and the passenger seat 36b side is used as a display image on the vehicle rear display monitor. Therefore, in addition to capturing images inside the cab 3, the omnidirectional cameras can also be used as cameras for displaying images on the vehicle rear display monitor.

[0065] (8) In this embodiment, the objects monitored by the omnidirectional cameras 60a, 60b include the preceding vehicle in front of the vehicle, and image information of the preceding vehicle captured by the omnidirectional cameras 60a, 60b on the driver's seat 36a side and the passenger seat 36b side is used as control input information for the control to follow the preceding vehicle. Therefore, the omnidirectional cameras 60a, 60b can be used not only to capture images of the interior of the cab 3, but also to provide control input information for the control to follow the preceding vehicle.

[0066] (9) In this embodiment, the objects monitored by the omnidirectional cameras 60a, 60b include the preceding vehicle in front of the vehicle, and image information of the preceding vehicle captured by the omnidirectional cameras 60a, 60b on the driver's seat 36a side and the passenger seat 36b side is used as control input information for the preceding vehicle collision prevention control. Therefore, the omnidirectional cameras 60a, 60b can be used not only to capture images of the interior of the cab 3, but also to provide control input information for the preceding vehicle collision prevention control.

[0067] (10) In this embodiment, the objects monitored by the omnidirectional cameras 60a, 60b include the lanes of the road on which the vehicle is traveling, and image information of the lanes captured by the omnidirectional cameras 60a, 60b on the driver's seat 36a side and the passenger seat 36b side is used as control input information for lane keeping control. Therefore, the omnidirectional cameras 60a, 60b can be used not only to capture images of the interior of the cab 3, but also to provide control input information for the lane keeping control.

[0068] (11) In this embodiment, the object monitored by the omnidirectional camera 60b on the passenger seat 36b side includes the vehicle exterior area on the passenger seat 36b side of the vehicle while it is moving, and image information of the area on the left side of the vehicle captured by the omnidirectional camera 60b is used as control input information for left-turn collision prevention control. Therefore, one omnidirectional camera 60b can be used not only to capture images of the inside of the cab 3, but also as control input information for prevention control of collisions (left-turn collisions) on the blind spot side (passenger seat side) for the driver.

[0069] (12) In the present embodiment, the support member 70 is supported at the connection portion 37 between the front panel 31 and the front pillar 32 at the corner portion 3C of the cab 3. Therefore, in small and medium-sized trucks, it is possible to photograph the driver 8 and others inside the cab 3 without increasing the size of the support member 70.

[0070] Furthermore, in this embodiment, the objects monitored by the omnidirectional cameras 60a, 60b include the driver 8 inside the cab 3, and the image information of the driver 8 captured by the omnidirectional cameras 60a, 60b is also used as information for determining whether there is an abnormality in the driver 8. Therefore, control in the event of an abnormality in the driver can be implemented without adding any additional cameras or sensors. In this embodiment, the vehicle is equipped with a side monitor that displays the situation on the sides of the vehicle and a rear monitor that displays the situation behind the vehicle, but a front monitor that displays the situation in front of the vehicle may also be provided. By providing a front monitor, the situation in the position directly below the front of the vehicle, which is a blind spot from the driver's seating position, can be grasped.

[0071] [3.Other] The configuration of the above embodiment is merely an example, and can be modified appropriately without departing from the spirit and scope of the present invention. For example, in the above embodiment, the omnidirectional cameras 60a, 60b are installed at the left and right corners 3C of the cab 3, respectively, but they may be installed at only one of them. In this case, the range of use of the information from the omnidirectional camera 60 is limited, but some usable control is possible, and a certain degree of effect can be obtained.

[0072] In the above embodiment, the omnidirectional camera 60a on the driver's seat 36a side is disposed rearward of the front end surface of the cab 3 and outward of the side surface of the cab 3. However, as shown in FIG. 10, the omnidirectional camera 60a on the driver's seat 36a side may be disposed outward of the side surface of the cab 3 and on an extension of the front end surface of the cab 3. Although it is somewhat far from the driver's seat 36a, the shooting range can be set to just before the front end surface of the cab 3. In this case, it is preferable that the shooting range of the omnidirectional camera 60a on the driver's seat 36a side includes at least the movable range of the wiper device 37 for the windshield 33. In this way, when raindrops or snow are attached to the windshield, the wiper device 37 can be operated to shoot the driver 8 in the cab 3.

[0073] In addition, in the above embodiment, the omnidirectional camera 60b on the passenger seat 36b side is disposed forward of the front end face of the cab 3 and outward of the side face of the cab 3, but as shown in Fig. 10, for example, the omnidirectional camera 60b on the passenger seat 36b side may be disposed forward of the front end face of the cab 3 and on an extension of the side face of the cab 3. Although the shooting range in the rear becomes narrower, it becomes possible to get closer to the driver's seat 36a and to shoot the situation of the driver 8 in the driver's seat 36a more clearly.

[0074] Note that Fig. 10 corresponds to Fig. 9, and since the arrangement of the omnidirectional cameras 60a, 60b is different, the shooting ranges of the omnidirectional cameras 60a, 60b are also different. In the case of Fig. 10, in addition to the three stages of an area that can be shot with only one lens (attached number 1), an area that can be shot with two lenses, an area that can be shot with three lenses (attached number 2), and an area that can be shot with three lenses (attached number 3), an area that can be shot with four lenses (attached number 4) is also generated.

[0075] Furthermore, in the above embodiment, an example of application to a small or medium-sized truck that does not have a bonnet in front of the cab 3 has been described, but the present invention can also be applied to trucks, buses, passenger cars, etc. that have a bonnet in front of the cab 3. In the above embodiment, the support member 70 is supported at the connection portion between the front panel 31 and the front pillar 32 at the corner portion 3C of the cab 3, but the present invention is not limited to this.

[0076] For example, in the case of a vehicle such as a large truck in which the face of the driver 8 is located considerably higher than the upper edge of the front panel 31, it is preferable that the support member 70 is supported at a connection between the roof panel and the front pillar 32 at a corner portion 3C of the cab 3. This makes it possible to photograph the driver 8 and others inside the cab 3 without increasing the size of the support member 70.

[0077] In addition, for example, in the case of a large bus in which a large windshield is provided over a wide area above and below the driver 8, it is preferable that the support member 70 be supported at the middle part of the front pillar 32 at the corner part of the cab 3. This makes it possible to photograph the driver 8 and others inside the cab 3 without increasing the size of the support member 70.

[0078] In the above embodiment, the omnidirectional camera 60 is attached to the mounting base 73 provided at the tip of the arm portion 72 of the support member 70, but the omnidirectional camera 30 may be attached to each corner portion 3C of the cab 3 (for example, the connection portion 37 between the front panel 31 and the front pillar 32). In this way, it is possible to apply the present invention to a case where the mirror for checking the rear of the vehicle is a door mirror.

[0079] In the above embodiment, the omnidirectional cameras 60a and 60b are configured to be able to capture 360 ​​degrees using two lenses, but the capture range per lens is not limited to this. For example, three lenses capable of capturing 120 degrees may be used.

[0080] Furthermore, in the above embodiment, the imaging range of the omnidirectional camera 60 is set to 360 degrees, but it may be less than 360 degrees as long as it is possible to image the monitoring target.

[0081] [4. Notes] Regarding the above embodiment, the following supplementary notes are further disclosed.

[0082] (Appendix 1) A vehicle monitoring device that captures an image of a monitoring target within a capture range by an omnidirectional camera mounted on a vehicle, The omnidirectional camera is attached to a front corner of the cab, is outward from a side surface of the cab, and is disposed within a range from an upper end to a lower end of a windshield; The imaging range of the omnidirectional camera includes at least the interior of the cab. A vehicle monitoring device comprising:

[0083] (Appendix 2) The omnidirectional cameras are installed at the left and right front corners of the cab, Of the omnidirectional cameras, the driver's seat side omnidirectional camera is disposed rearward of the front end surface of the cab and outward of the side surface of the cab, and the passenger seat side omnidirectional camera is disposed forward of the front end surface of the cab and outward of the side surface of the cab or on an extension line of the side surface of the cab. 2. A vehicle monitoring device as described in claim 1.

[0084] (Appendix 3) the vehicle is equipped with a windscreen wiper device, The imaging range of the omnidirectional camera includes at least the movable range of the wiper device. 3. A vehicle monitoring device as described in claim 2.

[0085] (Appendix 4) a control device that controls an in-vehicle device installed in the vehicle based on image information of the monitoring target captured by the omnidirectional camera; The control device controls the in-vehicle device based on image information of the monitoring target captured by the omnidirectional camera. 3. A vehicle monitoring device according to claim 1 or 2,

[0086] (Appendix 5) The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes the driver in the cab, and image information of the driver captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for a driver monitoring device. 4. A vehicle monitoring device according to any one of claims 1 to 3.

[0087] (Appendix 6) When the driver is a subject of monitoring, image information on the face of the driver captured by the omnidirectional camera on the driver's seat side and image information on changes in the driver's posture captured by the omnidirectional camera on the passenger seat side are used as control input information for the driver monitoring device. 6. A vehicle monitoring device as described in claim 5.

[0088] (Appendix 7) The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes the rear side of the vehicle, and image information of the rear side of the vehicle captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as a display image on a vehicle rear display monitor. 7. A vehicle monitoring device according to any one of claims 1 to 6.

[0089] (Appendix 8) The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes a preceding vehicle ahead of the vehicle itself, and image information of the preceding vehicle captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for preceding vehicle following control. 8. A vehicle monitoring device according to any one of claims 1 to 7.

[0090] (Appendix 9) The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes a preceding vehicle ahead of the vehicle itself, and image information of the preceding vehicle captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for preceding vehicle collision prevention control. 9. A vehicle monitoring device according to any one of claims 1 to 8.

[0091] (Appendix 10) The omnidirectional cameras are provided on the driver's seat side and the passenger seat side of the cab, The monitored object includes the lane of the road on which the vehicle is traveling, and image information of the lane captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for lane keeping control. 10. A vehicle monitoring device according to any one of claims 1 to 9.

[0092] (Appendix 11) The omnidirectional camera is provided on the passenger side of the cab, The monitored object includes an external area of ​​the vehicle on the passenger side of the vehicle while the vehicle is moving, and image information of the area on the left side of the vehicle captured by the omnidirectional camera is used as control input information for left-turn collision prevention control. 11. A vehicle monitoring device according to any one of claims 1 to 10.

[0093] (Appendix 12) The support member is supported at a connection portion between a front panel and a front pillar at a front corner portion of the cab. 12. A vehicle monitoring device according to any one of claims 1 to 11.

[0094] (Appendix 13) The support member is supported at a connection portion between a roof panel and a front pillar at a front corner portion of the cab. 12. A vehicle monitoring device according to any one of claims 1 to 11.

[0095] (Appendix 14) The support member is supported by a middle portion of a front pillar at a front corner portion of the cab. 12. A vehicle monitoring device according to any one of claims 1 to 11.

[0096] (Appendix 15) The vehicle does not have a bonnet in front of the cab. 15. A vehicle monitoring device according to any one of claims 1 to 14. [Explanation of symbols]

[0097] 1 vehicle (non-bonnet vehicle) 2 Frames 3 Cab 3C Cab 3 front corner 4 packing boxes 5 wheels 6 Monitoring equipment 8 Driver 31 Front Panel 32 Front pillar 33 Windshield 34 Doors 35 Door Glass 36a Driver's seat 36b passenger seat 37 Range of movement of wiper device 60 Omnidirectional Camera 60a Omnidirectional camera on the driver's seat 36a side 60b All-around camera on the passenger seat 36b side 61 cases 62 Lens 62a Front lens 62b Rear lens 63 Image sensor 64 Image processing circuit 70 Support member 71 Bracket 71a Mounting surface of bracket 71 72 Arm section 72a: Base end portion of arm portion 72 72b Bending portion of arm portion 72 72c: the tip end portion of the arm portion 72 73 Mounting base 80 Image data processing device 81 Driver monitoring device 82 Emergency control device 83 Adaptive Cruise Control Device 84 Collision prevention control device 85 Lane Keeping Control Device 86 Entrapment prevention control device 87 Side display device 88 Rear display device α11 Shooting range of the forward-facing lens 62a of the omnidirectional camera 60a α12 Shooting range of rear-facing lens 62b of omnidirectional camera 60a α21 Shooting range of the forward-facing lens 62a of the omnidirectional camera 60b α22 Shooting range of rear-facing lens 62b of omnidirectional camera 60b

Claims

1. A vehicle monitoring device that captures an image of a monitoring target within a capture range by an omnidirectional camera mounted on a vehicle, The omnidirectional camera is attached to a front corner of the cab, is outward from a side surface of the cab, and is disposed within a range from an upper end to a lower end of a windshield; The imaging range of the omnidirectional camera includes at least the interior of the cab. A vehicle monitoring device comprising:

2. The omnidirectional cameras are installed at the left and right front corners of the cab, Of the omnidirectional cameras, the omnidirectional camera on the driver's seat side is disposed rearward of the front end surface of the cab and outward of the side surface of the cab, and the omnidirectional camera on the passenger's seat side is disposed forward of the front end surface of the cab and outward of the side surface of the cab or on an extension line of the side surface of the cab.

2. A vehicle monitoring device according to claim 1.

3. the vehicle is equipped with a windscreen wiper device, The imaging range of the omnidirectional camera includes at least the movable range of the wiper device.

2. A vehicle monitoring device according to claim 1.

4. a control device that controls an in-vehicle device installed in the vehicle based on image information of the monitoring target captured by the omnidirectional camera; The control device controls the in-vehicle device based on image information of the monitoring target captured by the omnidirectional camera.

2. The vehicle monitoring device according to claim 1 .

5. The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes the driver in the cab, and image information of the driver captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for a driver monitoring device.

2. A vehicle monitoring device according to claim 1.

6. When the driver is a target of monitoring, image information on the face of the driver captured by the omnidirectional camera on the driver's seat side and image information on changes in the driver's posture captured by the omnidirectional camera on the passenger seat side are used as control input information for the driver monitoring device.

6. A vehicle monitoring device according to claim 5.

7. The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes the rear side of the vehicle, and image information of the rear side of the vehicle captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as a display image on a vehicle rear display monitor.

2. A vehicle monitoring device according to claim 1.

8. The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes a preceding vehicle ahead of the vehicle itself, and image information of the preceding vehicle captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for preceding vehicle following control.

2. A vehicle monitoring device according to claim 1.

9. The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitoring target includes a preceding vehicle ahead of the vehicle itself, and image information of the preceding vehicle captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for preceding vehicle collision prevention control.

2. A vehicle monitoring device according to claim 1.

10. The omnidirectional cameras are provided on the driver's seat side and the passenger's seat side of the cab, The monitored object includes the lane of the road on which the vehicle is traveling, and image information of the lane captured by the omnidirectional cameras on the driver's seat side and the passenger seat side is used as control input information for lane keeping control.

2. A vehicle monitoring device according to claim 1.

11. The omnidirectional camera is provided on the passenger side of the cab, The monitoring target includes an external area of ​​the vehicle on the passenger side of the vehicle while the vehicle is moving, and image information of the area on the left side of the vehicle captured by the omnidirectional camera is used as control input information for left-turn collision prevention control.

2. A vehicle monitoring device according to claim 1.

12. The omnidirectional camera is attached to a support member supported on a front corner portion of the cab, The support member is supported at a connection portion between a front panel and a front pillar at a front corner portion of the cab.

2. A vehicle monitoring device according to claim 1.

13. The omnidirectional camera is attached to a support member supported on a front corner portion of the cab, The support member is supported at a connection portion between a roof panel and a front pillar at a front corner portion of the cab.

2. A vehicle monitoring device according to claim 1.

14. The omnidirectional camera is attached to a support member supported on a front corner portion of the cab, The support member is supported by a middle portion of a front pillar at a front corner portion of the cab.

2. A vehicle monitoring device according to claim 1.

15. The vehicle does not have a bonnet in front of the cab.

2. A vehicle monitoring device according to claim 1.

Images