Peripheral monitoring device

Abstract

To properly control turn-on / off of a lateral lamp.SOLUTION: A periphery monitoring apparatus comprises: first imaging devices 11, 12 which can image right and left lateral regions of a vehicle; second imaging devices 13, 14 which can image front and rear regions of the vehicle; lateral lamps 18, 19 which can illuminate the imaging range of the first imaging devices; and a control unit 10 which can control turn-on / off of the lateral lamps. The control unit acquires the lateral illuminance of the imaging range of the first imaging devices in the current cycle or the relevant value thereof as the first lateral illuminance relevant value, estimates the lateral illuminance as the second lateral illuminance when assuming that a vehicle arrives at a prescribed point included in a travel direction photographed image captured in the current cycle by the second imaging devices that image the travel direction region of the vehicle, stores the second lateral illuminance in a storage unit in association with the current cycle, and controls turn-on / off of the lateral lamps on the basis of the second lateral illuminance associated with the past cycle specified on the basis of the first lateral illuminance relevant value and the current vehicle speed when the vehicle speed is equal to or greater than a vehicle speed threshold.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a peripheral monitoring device that controls the lighting and extinguishing of a side lamp that irradiates the imaging range of a side imaging device that images the side of a vehicle.

Background Art

[0002] Conventionally, a peripheral monitoring device (hereinafter also referred to as a "conventional device") that controls the lighting and extinguishing of a lamp capable of irradiating the imaging range of an imaging device that images the periphery of a vehicle is known. The conventional device controls the lighting and extinguishing of the lamp based on the illuminance of the imaging range (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] An imaging image captured by an imaging device is, for example, image-processed and displayed on a display inside the vehicle. According to the conventional device, the display image displayed on the display may repeatedly turn on and off in a short cycle, thereby reducing the visibility of the display image. That is, when the vehicle travels in a "section where a bright region and a dark region are repeatedly alternated at relatively short intervals along the traveling direction", the illuminance around the vehicle repeatedly changes between a high state and a low state in a short cycle. Therefore, the lighting and extinguishing of the lamp are repeated in a short cycle. As a result, the display image may turn on and off, and its visibility may decrease. Note that the above section is, for example, a section where a plurality of tunnels are continuously present at relatively short intervals.

[0005] The present invention has been made to address the above-described problems. That is, one of the objects of the present invention is to provide a peripheral monitoring device capable of appropriately controlling the lighting and extinguishing of a side lamp capable of irradiating the imaging range of a side imaging device.

[0006] The peripheral monitoring device according to the present invention (hereinafter referred to as "the device of the present invention") is mounted on a vehicle, a first imaging device including a left-side imaging device (11) capable of imaging the left-side area of the vehicle and a right-side imaging device (12) capable of imaging the right-side area of the vehicle, a second imaging device including a front imaging device (13) capable of imaging the front area of the vehicle and a rear imaging device (14) capable of imaging the rear area of the vehicle, side lamps (18, 19) capable of independently irradiating the imaging ranges of the first imaging devices (11, 12), and a control unit (10) capable of controlling the lighting and extinguishing of the side lamps (18, 19). It is provided with. The first imaging devices (11, 12) and the second imaging devices (13, 14) execute imaging processing every time a predetermined operation cycle elapses. The control unit (10) acquires, as a first side illuminance related value, the side illuminance or its related value that is the illuminance of the imaging range of the first imaging devices (11, 12) in the current cycle, Based on a traveling direction imaging image (F) that is an imaging image captured by a second imaging device that images the area on the traveling direction side of the vehicle among the second imaging devices (13, 14) in the current cycle, when it is assumed that the vehicle has reached a predetermined point (D1) included in the traveling direction imaging image (F), the side illuminance is estimated as a second side illuminance, and the estimated second side illuminance is stored in a storage unit in association with the current cycle, when the vehicle is traveling at a vehicle speed equal to or higher than a predetermined vehicle speed threshold, a second side illuminance associated with a certain past cycle specified based on the current vehicle speed at a point that the vehicle will pass through in the near future and that is closer to the vehicle than the predetermined point (D1), i.e., at point (D2) is acquired from the storage unit, and the lighting and extinguishing of the side lamps (18, 19) are controlled based on the first side illuminance related value and the acquired second side illuminance. It is configured as described above.

[0007] The device of the present invention controls the turning on and off of the side lamp based on a first side illuminance related value and a second side illuminance associated with a certain past period (hereinafter also referred to as "reference second side illuminance"). Here, the second side illuminance is the side illuminance when it is assumed that the vehicle has reached a predetermined point included in the forward imaging image acquired in the current period. In other words, it is the side illuminance at the point where the vehicle will pass in the future. The device of the present invention associates the second side illuminance with the current period and stores it in the storage unit. When controlling the turning on and off of the side lamp in the current period, the second side illuminance associated with a certain past period specified based on the current vehicle speed is acquired from the storage unit. Here, the second side illuminance estimated in a certain past period is, of course, the side illuminance when it is assumed that the vehicle has reached a point in front of the above-mentioned predetermined point (a point located on the opposite side of the traveling direction with respect to the predetermined point). However, it is desirable that this point in front is "a point slightly ahead of the vehicle position in the current period in the traveling direction". By setting a certain past period as such a period, the reference second side illuminance becomes the side illuminance at the point where the vehicle will pass in the near future.

[0008] According to this configuration, it is possible to control the turning on and off of the side lamp not only based on the side illuminance or its related value (first side illuminance related value) at the "current" vehicle position, but also based on the side illuminance (reference second side illuminance) at the point where the vehicle will pass in the "near future". For this reason, for example, even if it is determined that the side of the current vehicle position is relatively bright according to the first side illuminance related value when the side lamp is in the on state, when it is determined that the side of the vehicle passing point in the near future is relatively dark according to the reference second side illuminance, it is possible to maintain the on state without turning off the side lamp. Thereby, the possibility that the turning on and off of the side lamp is repeated in a short cycle can be suppressed. Therefore, for example, when the imaging image captured by the imaging device is displayed on the in-vehicle display, the frequency of the display image flickering can be reduced, and the decrease in visibility caused by the flickering of the display image can be suppressed.

[0009] In the above description, for the purpose of facilitating the understanding of the invention, reference signs used in the embodiments are attached in parentheses to the constituent elements of the invention corresponding to the embodiments. However, each constituent element of the invention is not limited to the embodiments defined by the above reference signs.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0011] (Configuration) Hereinafter, a peripheral monitoring device (hereinafter also referred to as "this implementation device") according to an embodiment of the present invention will be described with reference to the drawings. This implementation device is mounted on a vehicle. As shown in FIG. 1, this implementation device includes a peripheral monitoring ECU 10. The peripheral monitoring ECU 10 mainly includes a microcomputer. ECU is an abbreviation for Electronic Control Unit. The microcomputer includes a CPU, a ROM, a RAM, an interface (I / F), etc., and the CPU realizes various functions by executing instructions (programs, routines) stored in the ROM. Hereinafter, the vehicle on which this implementation device is mounted is referred to as "the host vehicle".

[0012] The peripheral monitoring ECU 10 is connected to the left side camera 11, the right side camera 12, the front camera 13, the rear camera 14, the PVM switch 15, the shift position sensor (shift P sensor) 16, the vehicle speed sensor 17, the left side lamp 18, the right side lamp 19, and the display 20. Hereinafter, the peripheral monitoring ECU 10 may also be simply referred to as "ECU 10".

[0013] The left side camera 11 is installed on the lower surface of the left side mirror (not shown) of the host vehicle, and images the left side area of the host vehicle (in other words, images a subject located in the left side area). The left side camera 11 is capable of imaging in the visible light region and the infrared region, and images in the visible light region when the imaging range is relatively bright (typically, during the day), and images in the infrared region when the imaging range is relatively dark (typically, at night).

[0014] The left side camera 11 has an automatic exposure function. The automatic exposure function is a well-known function that executes control (exposure control) to set the exposure time to an appropriate exposure time and adjust the gain level to an appropriate gain based on the luminance of the subject, etc. Hereinafter, information including the appropriate exposure time and the appropriate gain is referred to as "exposure information". The left side camera 11 uses the automatic exposure function to image the subject, and outputs the left image data obtained by imaging to the ECU 10. The left side camera 11 corresponds to an example of the "left side imaging device" and the "first imaging device".

[0015] A left side lamp 18 is installed near the left side mirror. The left side lamp 18 is a lamp that emits infrared rays and can irradiate the imaging range of the left side camera 11. The ECU 10 can control the lighting and extinguishing of the left side lamp 18 (described later).

[0016] The right-side camera 12 is installed on the lower surface of the right-side mirror (not shown) of the host vehicle, and images the right-side area of the host vehicle (in other words, images a subject located in the right-side area). Since the right-side camera 12 has substantially the same configuration as the left-side camera 11, a detailed description thereof will be omitted. The right-side camera 12 images a subject using an automatic exposure function, and outputs the obtained right image data to the ECU 10. The right-side camera 12 corresponds to an example of a "right-side imaging device" and a "first imaging device".

[0017] A right-side lamp 19 is installed near the right-side mirror. The right-side lamp 19 is a lamp that emits infrared rays and can irradiate the imaging range of the right-side camera 12. The ECU 10 can control the lighting and extinguishing of the right-side lamp 19 (described later).

[0018] The front camera 13 is installed at the center of the front end of the host vehicle, and images the front area of the host vehicle (in other words, images a subject located in the front area). The front camera 13 images in the visible light region. That is, the headlamp (not shown) of the host vehicle can irradiate the imaging range of the front camera 13, and during the period when the headlamp is in the lit state, imaging in the visible light region is possible by the headlamp irradiating the imaging range.

[0019] The front camera 13 has an automatic exposure function. The front camera 13 images a subject using the automatic exposure function, and outputs the obtained front image data to the ECU 10. The front camera 13 corresponds to an example of a "front imaging device" and a "second imaging device".

[0020] The rear camera 14 is installed at the center of the rear end of the host vehicle and images the rear area of the host vehicle (in other words, images a subject located in the rear area). The rear camera 14 images in the visible light region. That is, the tail lamp (not shown) of the host vehicle can irradiate the imaging range of the rear camera 14, and during the period when the tail lamp is in the lit state, the tail lamp irradiates the imaging range, enabling imaging in the visible light region. In addition to the tail lamp, the back lamp (a lamp that lights up when the shift lever is in the R range) and / or the brake lamp may be configured to irradiate the imaging range of the rear camera 14.

[0021] The rear camera 14 has an automatic exposure function. The rear camera 14 uses the automatic exposure function to image a subject and outputs the obtained rear image data to the ECU 10. The rear camera 14 corresponds to an example of the "rear imaging device" and the "second imaging device".

[0022] The ECU 10 acquires the illuminance of the periphery (typically, the front area) of the host vehicle by, for example, an illuminance sensor, and controls the turning on and off of the headlamp based on the illuminance. The timing of turning on and off the tail lamp is synchronized with the timing of turning on and off the headlamp. Note that the turning on and off of the headlamp may be manually performed by the driver of the host vehicle. In this case, the ECU 10 controls only the turning on and off of the tail lamp based on the illuminance of the illuminance sensor.

[0023] The cameras 11 to 14 are configured to perform imaging processing every time a predetermined operation cycle elapses during the period when the ignition switch is in the on state, and output left image data, right image data, front image data, and rear image data to the ECU 10.

[0024] The PVM switch 15 is provided near a steering wheel (not shown) and can be pressed by the driver when displaying a PVM image (described later) on the display 20. When the PVM switch 15 is pressed while an off signal is being generated, an on signal is generated and continues to be generated until it is pressed again. When the PVM switch 15 is pressed while an on signal is being generated, an off signal is generated and continues to be generated until it is pressed again. The ECU 10 acquires the signal generated by the PVM switch 15 and detects whether the PVM switch 15 is in the on or off state based on the signal.

[0025] The shift P sensor 16 generates a signal corresponding to the shift position (D, N, R, or P) of a shift lever (not shown). The ECU 10 acquires the signal generated by the shift P sensor 16 and detects the shift position based on the signal (that is, identifies the traveling direction (including the stopped state) of the host vehicle).

[0026] The vehicle speed sensor 17 generates a signal corresponding to the traveling speed (vehicle speed) of the host vehicle. The ECU 10 acquires the signal generated by the vehicle speed sensor 17 and calculates the vehicle speed based on the signal.

[0027] The display 20 is provided at a position visible to the passengers of the host vehicle. In the present embodiment, a display used in a navigation system is used as the display 20. Note that a display other than the touch panel of the navigation system may be used as the display 20.

[0028] (Details of operation) Next, the details of the operation of the ECU 10 will be described. The ECU 10 generates a PVM image based on the left image data, right image data, front image data, and rear image data acquired from the left side camera 11, right side camera 12, front camera 13, and rear camera 14. Here, PVM (Panoramic View Monitor) is a well-known function that displays the surrounding area of the host vehicle on a display screen (in this embodiment, the display 20) when the host vehicle is traveling at a relatively low speed. The PVM image is an image generated for PVM and includes, for example, an overhead view image. The ECU 10 generates the overhead view image by synthesizing the above four image data and a planar image of the host vehicle pre-stored in the ROM of the ECU 10. The PVM image generation process is performed when there is a PVM image display request described later. Note that the PVM image may include side view images in addition to the overhead view image. The side view images are images including a left side image generated based on the left image data and a right side image generated based on the right image data, and can be used for safety confirmation of the side of the vehicle and the like.

[0029] The ECU 10 determines the presence or absence of a PVM image display request based on the vehicle speed and the state of the PVM switch 15. Specifically, when the vehicle speed is equal to or lower than a predetermined PVM vehicle speed threshold and the PVM switch 15 is in the ON state, the ECU 10 determines that there is a PVM image display request. In this case, the ECU 10 generates a PVM image and displays the generated PVM image on the display 20. On the other hand, when the vehicle speed exceeds the PVM vehicle speed threshold and / or the PVM switch 15 is in the OFF state, the ECU 10 determines that there is no PVM image display request. In this case, the ECU 10 does not generate a PVM image, and thus does not perform the display process on the display 20.

[0030] When it is determined that there is a PVM image display request, the ECU 10 controls the lighting and extinguishing of the left side lamp 18 and the right side lamp 19 (hereinafter, these may be collectively referred to simply as "side lamps 18, 19"). This will be specifically described below.

[0031] First, a method for controlling the turning on and off of the left side lamp 18 will be described. The ECU 10 generates a left captured image from the left image data output from the left side camera 11 in the current cycle, and calculates the average luminance of the left captured image based on the luminance values of the respective pixels constituting the left captured image. In addition, the ECU 10 acquires the exposure information of the left side camera 11 in the current cycle, and determines a luminance threshold based on the exposure information. The luminance threshold is a threshold for determining whether to set the side lamps 18 and 19 to the off state or the on state, and is a variable that changes according to the exposure information. When the average luminance of the left captured image (hereinafter referred to as "left average luminance") is equal to or lower than the luminance threshold, the ECU 10 determines that the left side of the current vehicle position is relatively dark, and sets the left side lamp 18 to the on state regardless of the current vehicle speed.

[0032] On the other hand, when the left average luminance exceeds the luminance threshold, the ECU 10 determines that the left side of the current vehicle position is relatively bright, but does not immediately set the left side lamp 18 to the off state, and switches the control method for turning on and off the left side lamp 18 based on whether the current vehicle speed acquired from the vehicle speed sensor 17 is equal to or lower than a predetermined vehicle speed threshold. That is, when the current vehicle speed is equal to or lower than the vehicle speed threshold, the ECU 10 determines that the left side of the vehicle will remain relatively bright for a while because the vehicle speed is low, and sets the left side lamp 18 to the off state. This vehicle speed threshold is lower than the PVM vehicle speed threshold. Note that the left average luminance corresponds to an example of the "first lateral illuminance related value".

[0033] In contrast, when the current vehicle speed exceeds the vehicle speed threshold, the ECU 10 controls the turning on and off of the left side lamp 18 based on the imaging data captured by the camera that images the area on the traveling direction side of the host vehicle in a certain past cycle (described later).

[0034] For example, when the host vehicle is moving forward, the ECU 10 generates a forward captured image from the forward image data output from the forward camera 13 in the current cycle. The forward captured image includes an area that the host vehicle will pass through in the future. The ECU 10 estimates the illuminance of the imaging range of the left side camera 11 when it is assumed that the host vehicle has reached a predetermined point D1 included in the forward captured image. Then, it is determined whether or not the illuminance is equal to or less than a predetermined side illuminance threshold value (described later), and the process of associating the determination result with the estimated illuminance and storing it in the RAM in the current cycle is repeated. Hereinafter, the illuminance of the imaging range of the left side camera 11 is also referred to as "left side illuminance". The ECU 10 acquires the left side illuminance and the determination result associated with a certain past cycle specified based on the current vehicle speed from the RAM, and controls the turning on and off of the left side lamp 18 based on the determination result. Note that the point D1 is a point separated from the front end of the host vehicle by a predetermined distance in the traveling direction (in this example, forward), and this distance can be determined by the optical characteristics of the camera on the traveling direction side (in this example, the forward camera 13).

[0035] The left side illuminance when it is assumed that the host vehicle has reached the point D1 corresponds to an example of the "second side illuminance". Also, the forward captured image in the above example corresponds to an example of the "traveling direction captured image".

[0036] First, a method for estimating the left side illuminance will be described with reference to FIG. 2. The image F in FIG. 2 is an example of a forward captured image generated from the forward image data in the current cycle. As shown in FIG. 2, there is a road Ro that curves to the right in front of the host vehicle. White lines 30 and 31 extend along the road shape on the road surface of the road Ro. Street lights 33 and 34 are installed on the left side of the road Ro with a guardrail 32 therebetween, and street trees 35 are provided. Since the image F is an image captured at night, the street lights 33 and 34 are lit. Also, the headlamp of the host vehicle is lit.

[0037] Generally, the luminance value I of the road surface of the road Ro is expressed as in the following formula (1).

Equation

[0038] The illuminance L h of the headlamp is pre-acquired from the design value of the headlamp and stored in the ROM of the ECU 10. Also, the reflectance R is pre-acquired by experiments and stored in the ROM of the ECU 10. In addition, the luminance value I and the illuminance L k of the road surface can be calculated based on the image F (front captured image). Specifically, the luminance value I can be calculated based on the luminance values of each pixel constituting the road surface area included in the image F. Note that the road surface area can be identified by a well-known method by analyzing the image F. The illuminance L k is specified by using "a pixel group having a luminance value of 0.99 or more in the area above the horizon HL in the image F" as a light source, and can be calculated by converting the luminance value of each pixel constituting the light source into illuminance by a well-known method. In this example, the street lamps 33 and 34 are specified as the light sources. Note that the luminance value used in this embodiment is scaled to a value between 0 and 1. When the values thus obtained or calculated are substituted into Equation (1), the value of the term including the ambient illuminance L e (the first term on the right side: L e R / π) is obtained. This value is used for calculating the left lateral illuminance (described later).

[0039] Rl and Rr in Fig. 2 represent a pair of route prediction lines of the host vehicle. The route prediction lines are lines that predict the route of the host vehicle assuming that the host vehicle travels while maintaining the current vehicle speed and steering angle. Rl indicates the route prediction line on the left side of the host vehicle, and Rr indicates the route prediction line on the right side of the host vehicle. The left route prediction line Rl and the right route prediction line Rr can be calculated by a well-known method based on the current position, vehicle speed, steering angle, etc. of the host vehicle. In this example, since the current steering angle is 0, the left and right route prediction lines Rl and Rr extend linearly forward of the host vehicle.

[0040] Here, a straight line extending vertically through the intersection point Pl between the horizon HL and the left route prediction line Rl is defined as the straight line Ll, and a straight line extending vertically through the intersection point Pr between the horizon HL and the right route prediction line Rr is defined as the straight line Lr. Then, the region A surrounded by the left and right route prediction lines Rl and Rr, the horizon HL, and the outer contour of the image F in the image F is divided into a left region Al, a right region Ar, and an upper region Au. The left region Al is the left region of the region A and is located on the left side with respect to the left route prediction line Rl and the straight line Ll. The right region Ar is the right region of the region A and is located on the right side with respect to the right route prediction line Rr and the straight line Lr. The upper region Au is the central region of the region A (in other words, the upper region in the image F) and is located between the left region Al and the right region Ar.

[0041] When assuming that the host vehicle reaches the point D1 from the current position, the light sources that irradiate the imaging range of the left side camera 11 are only the light sources existing in the left region Al, and the light sources that irradiate the imaging range of the right side camera 12 in this case are only the light sources existing in the right region Ar. This will be described in more detail below.

[0042] That is, when the road Ro curves to the right as in this example, when the host vehicle reaches point D1, the light sources existing in the left region Al will be located on the left side of the host vehicle, so they irradiate the imaging range of the left side camera 11. On the other hand, when the host vehicle reaches point D1 in this case, the light sources existing in the right region Ar will be located slightly in front of, to the right of, or on the right side of the host vehicle, so they irradiate the imaging range of the right side camera 11. Here, whether the light sources existing in the right region Ar are located slightly in front of, to the right of, or on the right side of the host vehicle depends on the road shape (for example, the curvature of the curve and the number of lanes) and the relative position of the host vehicle within the road. For example, when the curvature of the curve of the road Ro is relatively large as in this example, the light sources on the left side of the road Ro (that is, the street lights 34) may be allocated to the right region Ar. In such a case, when the host vehicle reaches point D1, the light source will be located slightly in front of and to the right of the host vehicle. The light sources in such a positional relationship irradiate the front and the right side of the host vehicle at point D1, but do not irradiate the left side of the host vehicle. This is because the light rays from the light sources are blocked by the vehicle body of the host vehicle and do not reach the left side of the host vehicle. On the other hand, when the curvature of the curve of the road is relatively small, even if the road curves to the right, the light sources on the right side of the road may be allocated to the right region Ar. In such a case, when the host vehicle reaches point D1, the light source will be located on the right side of the host vehicle. The light sources in such a positional relationship irradiate the right side and the right side of the host vehicle at point D1, but do not irradiate the left side of the host vehicle because the light rays from the light sources are blocked by the vehicle body.

[0043] Note that when the road curves to the left, the same concept as when the road Ro curves to the right can be applied.

[0044] On the other hand, when the road extends straight, when the host vehicle reaches point D1, the light sources existing in the left region Al will be located on the left side of the host vehicle, so they irradiate the imaging range of the left side camera 11, and the light sources existing in the right region Ar will be located on the right side of the host vehicle, so they irradiate the imaging range of the right side camera 12.

[0045] From the above, the predicted "predicted luminance value I of the road surface on the left side of the host vehicle at point D1 at the current time" el " is expressed by the following formula (2) using the term including the ambient illuminance L e calculated from the above formula (1). [Number] Here, L kAl represents the illuminance of the light source (in this example, street lamp 33) having a direction existing in the left region Al.

[0046] ECU10 estimates the left-side illuminance (the illuminance in the imaging range of the left side camera 11) at point D1 by multiplying the predicted luminance value I el by π / R. The above is the explanation of the method for estimating the left-side illuminance. ECU10 determines whether or not the left-side illuminance estimated in this way is equal to or less than a predetermined side illuminance threshold value, and stores the determination result in its RAM in association with the current cycle together with the left-side illuminance. ECU10 executes the above processing every time a predetermined calculation cycle elapses during the period when the ignition switch is in the ON state.

[0047] ECU10 acquires the left-side illuminance and the determination result at point D2 (a point closer to the host vehicle than point D1) where the host vehicle will pass in the near future (in this embodiment, after a predetermined time t has elapsed) from its RAM. Specifically, ECU10 assumes that the host vehicle maintains the current vehicle speed, calculates the position of point D2 by multiplying the current vehicle speed by the time t, and identifies the past cycle in which the left-side illuminance at the position was calculated. Then, ECU10 acquires the left-side illuminance and the determination result associated with the identified past cycle from its RAM. Hereinafter, the "past cycle in which the left-side illuminance at point D2 was calculated" is also referred to as the "reference cycle". Also, the left-side illuminance associated with the reference cycle is also referred to as the "reference left-side illuminance".

[0048] When the determination result indicates that the reference left-side illuminance ≤ the side illuminance threshold, the ECU 10 sets the left side lamp 18 to the lit state. In other words, even when the left average luminance exceeds the luminance threshold, if the current vehicle speed exceeds the vehicle speed threshold and the reference left-side illuminance is less than or equal to the side illuminance threshold, the ECU 10 determines that although the left side of the current vehicle position is relatively bright, it will become relatively dark after the elapse of time t (in the near future), and does not turn off the left side lamp 18 but sets it to the lit state.

[0049] On the other hand, when the determination result indicates that the reference left-side illuminance > the side illuminance threshold, the ECU 10 determines that the relatively bright state on the left side of the vehicle will continue for some time, and sets the left side lamp 18 to the off state.

[0050] Next, the control method for turning on and off the right side lamp 19 will be described. This control method is substantially the same as the control method for turning on and off the left side lamp 18. That is, the ECU 10 generates a right captured image from the right image data output from the right side camera 12 in the current cycle, and calculates the average luminance (right average luminance) of the image. When the right average luminance is less than or equal to the luminance threshold (a threshold determined based on the exposure information of the right side camera 12 in the current cycle), the ECU 10 determines that the right side of the current vehicle position is relatively dark, and sets the right side lamp 19 to the lit state.

[0051] On the other hand, when the current vehicle speed is less than or equal to the vehicle speed threshold when the right average luminance exceeds the luminance threshold, the ECU 10 determines that the relatively bright state on the right side of the vehicle will continue for some time because the vehicle speed is low, and sets the right side lamp 19 to the off state. Note that the right average luminance corresponds to an example of the "first side illuminance related value".

[0052] Here, taking the case where the host vehicle is moving forward as an example again, as described above, assuming that the host vehicle has reached point D1 from the current position, the light sources that irradiate the imaging range of the right side camera 12 are only the light sources existing in the right region Ar (see Figure 2). Therefore, the luminance prediction value I of the road surface on the right side of the host vehicle at point D1 predicted at the current time er" is the ambient illuminance L calculated from the above formula (1). e Using the terms including this, it is expressed as the following formula (3). [Number] Here, L kAr represents the illuminance of a light source having a direction existing in the right region Ar (in this example, the street lamp 34).

[0053] The ECU 10 estimates the value obtained by multiplying the luminance prediction value I er by π / R as the right lateral illuminance at point D1 (the illuminance in the imaging range of the right side camera 12). Then, it determines whether or not the illuminance is less than or equal to the lateral illuminance threshold value, and stores the determination result in association with the current cycle together with the right lateral illuminance in its RAM. The ECU 10 executes the above processing every time a predetermined calculation cycle elapses during the period when the ignition switch is in the ON state. The right lateral illuminance in the case where it is assumed that the host vehicle has reached point D1 corresponds to an example of the "second lateral illuminance".

[0054] The above-described reference cycle is also a past cycle in which the right lateral illuminance at point D2 was calculated. Therefore, the ECU 10 acquires the right lateral illuminance (reference right lateral illuminance) and the determination result associated with the reference cycle from its RAM. When the determination result indicates that the reference right lateral illuminance ≤ the lateral illuminance threshold value, the ECU 10 sets the right side lamp 19 to the lit state. In other words, even when the right average luminance exceeds the luminance threshold value, when the current vehicle speed exceeds the vehicle speed threshold value and the reference right lateral illuminance is less than or equal to the lateral illuminance threshold value, the ECU 10 determines that although the right side of the current vehicle position is relatively bright, it will become relatively dark after the elapse of time t (in the near future), and does not turn off the right side lamp 19 but sets it to the lit state.

[0055] On the other hand, when the determination result indicates that the reference right lateral illuminance > the lateral illuminance threshold value, the ECU 10 determines that the relatively bright state on the right side of the vehicle will continue for a while, and sets the right side lamp 19 to the off state.

[0056] On the other hand, when the host vehicle is reversing, the ECU 10, based on the rear captured image generated from the rear image data captured by the rear camera 14, estimates the left and right lateral illuminances when it is assumed that the host vehicle has reached a predetermined point included in the rear captured image by the same method as above, and stores in its RAM, in association with the current cycle, the determination result indicating the magnitude relationship with the lateral illuminance threshold together with the left and right lateral illuminances. Then, based on the left and right average luminances and the left and right reference lateral illuminances (strictly speaking, the determination results), the lighting and extinguishing of the side lamps 18 and 19 are controlled.

[0057] In other words, when the current vehicle speed is equal to or lower than the vehicle speed threshold, the ECU 10 sets the side lamps 18 and 19 to the lit state when the "luminance condition that the left and right average luminances in the current cycle are equal to or lower than the corresponding luminance thresholds" is satisfied. On the other hand, when the current vehicle speed exceeds the vehicle speed threshold, the ECU 10 sets the side lamps 18 and 19 to the lit state when at least one of the "luminance condition" and the "reference lateral illuminance condition that the left and right lateral illuminances (reference lateral illuminances) in the reference cycle are equal to or lower than the lateral illuminance threshold" is satisfied. That is to say, it can also be said that when the vehicle speed > vehicle speed threshold is established, the ECU 10 controls the lighting and extinguishing of the side lamps 18 and 19 based on the inclusive logical sum of the luminance condition and the reference lateral illuminance condition.

[0058] (Specific operation) Next, the specific operation of the CPU of the ECU 10 will be described. Hereinafter, for convenience of explanation, the lighting and extinguishing control of the left side lamp 18 will be described. The lighting and extinguishing control of the right side lamp 19 can be described by replacing "left" with "right" in the following description. The CPU is configured to repeatedly execute in parallel, for each elapse of a predetermined time during the period when the ignition switch is in the on state, the routines shown in the flowcharts of FIGS. 3 to 5.

[0059] When the specified timing arrives, the CPU starts processing from step 300 in FIG. 3 and proceeds to step 310, where it initializes the value of the first left lighting flag (described later). Subsequently, the CPU proceeds to step 320 and determines whether there is a PVM image display request. If there is no such request, the CPU determines "No" at step 320 and proceeds to step 395 to temporarily end this routine. On the other hand, if there is such a request, the CPU determines "Yes" at step 320 and sequentially executes the processes of the following steps 330 to 350. Step 330: The CPU generates a left captured image from the left image data. In addition, the CPU acquires the exposure information of the left side camera 11. Step 340: The CPU calculates the left average luminance based on the left captured image generated in step 330. Step 350: The CPU determines the luminance threshold based on the exposure information of the left side camera 11 acquired in step 330.

[0060] Thereafter, the CPU proceeds to step 360 and determines whether the left average luminance calculated in step 340 is less than or equal to the luminance threshold determined in step 350. If the condition left average luminance ≦ luminance threshold holds, the CPU determines "Yes" at step 360 and proceeds to step 370, where it sets the value of the first left lighting flag to "1". Thereafter, the CPU proceeds to step 395 to temporarily end this routine. On the other hand, if the condition left average luminance ≦ luminance threshold does not hold, the CPU determines "No" at step 360 and proceeds to step 395 to temporarily end this routine. In this case, the value of the first left lighting flag remains at the initial value (i.e., "0").

[0061] Note that the determination process in step 360 is equivalent to the process of determining whether the above-described luminance condition holds. That is, the value of the first left lighting flag being "1" means that the luminance condition holds, and the value of the first left lighting flag being "0" means that the luminance condition does not hold (or there is no PVM image display request).

[0062] Also, at a predetermined timing, the CPU starts processing from step 400 in FIG. 4 and sequentially executes the processing of the following steps 410 to 430. Step 410: The CPU initializes the value of the second left lighting flag (described later). Step 420: The CPU identifies the traveling direction of the host vehicle based on the shift position acquired from the shift P sensor 16, and generates a traveling direction captured image from the captured data captured by the camera on the traveling direction side. In addition, the CPU acquires the exposure information of the camera on the traveling direction side. Step 430: The CPU uses the traveling direction captured image generated in step 420 to calculate the luminance prediction value I el (see the above formulas (1) and (2)), and estimates the left side illuminance at point D1 based on the luminance prediction value I el .

[0063] Thereafter, the CPU proceeds to step 440 and determines whether the left side illuminance estimated in step 430 is less than or equal to the side illuminance threshold value. If the left side illuminance ≦ side illuminance threshold value holds, the CPU determines "Yes" in step 440, proceeds to step 450, and sets the value of the second left lighting flag to "1". Subsequently, the CPU proceeds to step 460 and stores (records) the value of the second left lighting flag (i.e., "1") in the RAM of the ECU 10 in association with the current cycle. Thereafter, the CPU proceeds to step 495 and temporarily ends this routine.

[0064] On the other hand, if the left side illuminance ≦ side illuminance threshold value does not hold, the CPU determines "No" in step 440, proceeds to step 460, and stores (records) the value of the second left lighting flag (i.e., the initial value "0") in the RAM of the ECU 10 in association with the current cycle. Thereafter, the CPU proceeds to step 495 and temporarily ends this routine.

[0065] Furthermore, at a predetermined timing, the CPU starts processing from step 500 in FIG. 5 and proceeds to step 510 to determine whether there is a PVM image display request. If there is no such request, the CPU determines "No" at step 510 and proceeds to step 595 to temporarily end this routine. On the other hand, if there is such a request, the CPU determines "Yes" at step 510 and proceeds to step 520.

[0066] In step 520, the CPU determines whether the value of the first left lighting flag is "1". If the value of the first left lighting flag is "1", the CPU determines "Yes" at step 520 (i.e., determines that the left side of the current vehicle position is relatively dark), and proceeds to step 530 to set the left side lamp 18 to the lit state.

[0067] On the other hand, if the value of the first left lighting flag is "0", the CPU determines "No" at step 520 and proceeds to step 540. In step 540, the CPU determines whether the current vehicle speed is less than or equal to the vehicle speed threshold. If the vehicle speed ≤ vehicle speed threshold holds, the CPU determines "Yes" at step 540 (i.e., determines that since the vehicle speed is low, the left side of the vehicle will remain relatively bright for a while), and proceeds to step 550 to set the left side lamp 18 to the extinguished state.

[0068] On the other hand, if the vehicle speed ≤ vehicle speed threshold does not hold, the CPU determines "No" at step 540 and proceeds to step 560. In step 560, the CPU specifies a reference period based on the current vehicle speed. Subsequently, the CPU proceeds to step 570, reads the value of the second left lighting flag associated with the reference period from the RAM of the ECU 10, and determines whether the value is "1". If the value of the second left lighting flag is "1", the CPU determines "Yes" at step 570 (i.e., determines that although the left side of the current vehicle position is relatively bright, it will become relatively dark after time t (in the near future)), and proceeds to step 580 to set the left side lamp 18 to the lit state.

[0069] On the other hand, when the value of the second left lighting flag is "0", the CPU determines "No" in step 570 (that is, it determines that the relatively bright state continues on the left side of the vehicle for a while), proceeds to step 590, and sets the left side lamp 18 to the extinguished state.

[0070] When any of the processes in step 530, step 550, step 580, or step 590 is completed, the CPU proceeds to step 595 and temporarily ends this routine.

[0071] Note that the determination process in step 570 is equivalent to the process of determining whether the above-described reference side illumination condition is satisfied. That is, the value of the second left lighting flag being "1" means that the reference side illumination condition is satisfied, and the value of the second left lighting flag being "0" means that the reference side illumination condition is not satisfied. In other words, the CPU controls the lighting and extinguishing of the side lamps 18 based on the inclusive logical sum of the values of the first left lighting flag and the second left lighting flag.

[0072] As described above, according to the present implementation device, not only the left and right average luminances, which are related values of the side illuminance at the "current" vehicle position, but also the left and right reference side illuminances, which are the side illuminances at point D2 where the host vehicle will pass in the "near future", are used to control the lighting and extinguishing of the side lamps 18 and 19. Therefore, compared with the configuration that controls the lighting and extinguishing of the side lamps based only on the left and right average luminances, the possibility that the lighting and extinguishing of the side lamps 18 and 19 are repeated in a short cycle can be suppressed. Accordingly, the frequency of the flickering of the display 20 can be reduced, and the decrease in visibility caused by the flickering of the display 20 can be suppressed.

[0073] In particular, when calculating the lateral illuminance at point D1 included in the forward-direction imaging image, the present implementation device is configured to exclude light sources among the light sources included in the forward-direction imaging image whose light rays may be blocked by the vehicle body. For this reason, compared with a configuration in which the lateral illuminance is calculated based on all the light sources included in the forward-direction imaging image, it is possible to estimate the accurate lateral illuminance. Therefore, the blinking of the side lamps 18 and 19 can be more appropriately controlled, and as a result, the decrease in the visibility of the display 20 can be further suppressed.

[0074] As described above, the peripheral monitoring device according to the present embodiment has been described. However, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the object of the present invention.

[0075] For example, the side lamps 18 and 19 may be lamps that emit visible light. This is because the side lamps 18 and 19 are installed near (typically, below) the left and right side mirrors, so even if the side lamps 18 and 19 blink, the vehicle occupants do not directly perceive the blinking.

[0076] Also, when setting the value of the first left lighting flag, instead of the left and right average luminances, the left and right illuminances (i.e., the left and right lateral illuminances) that can be calculated based on the left and right average luminances may be used.

Description of Reference Numerals

[0077] 10: Peripheral monitoring ECU, 11: Left side camera, 12: Right side camera, 13: Front camera, 14: Rear camera, 15: PVM switch, 16: Shift P sensor, 17: Vehicle speed sensor, 18: Left side lamp, 19: Right side lamp, 20: Display

Claims

【Claim 1】 mounted on a vehicle, a first imaging device including a left-side imaging device capable of imaging the left-side area of the vehicle and a right-side imaging device capable of imaging the right-side area of the vehicle; a second imaging device including a front imaging device capable of imaging the front area of the vehicle and a rear imaging device capable of imaging the rear area of the vehicle; side lamps capable of independently irradiating the imaging ranges of the first imaging device; a control unit capable of controlling the lighting and extinguishing of the side lamps; In a peripheral monitoring device provided with: the first imaging device and the second imaging device execute imaging processing every time a predetermined operation cycle elapses, the control unit obtains, as a first side illumination related value, the side illumination or its related value that is the illumination of the imaging range of the first imaging device in the current cycle; Based on a forward imaging image that is an imaging image captured by a second imaging device that images an area on the traveling direction side of the vehicle among the second imaging devices in the current cycle, when it is assumed that the vehicle has reached a predetermined point included in the forward imaging image, the side illumination is estimated as a second side illumination, and the estimated second side illumination is stored in a storage unit in association with the current cycle; when the vehicle is traveling at a vehicle speed equal to or higher than a predetermined vehicle speed threshold, obtains, from the storage unit, a second side illumination at a point that is associated with a certain past cycle specified based on the current vehicle speed and that the vehicle will pass in the near future and is closer to the vehicle than the predetermined point; controls the lighting and extinguishing of the side lamps based on the first side illumination related value and the obtained second side illumination; configured as a peripheral monitoring device.

Images