Environment recognition device

Abstract

The purpose of the present invention is to increase the timing accuracy of alerts and of control with respect to bumps, and to reduce false alarms and erroneous control. An environment recognition unit 102 has at least a bump characteristic generation part 202, a travel region identification part 203, a bump recognition part 204, and a result output part 205. Moreover, the bump recognition part 204 includes a candidate region extraction part 241 and a visible region determination part 242. Among bump candidate regions present in a travel path region, the bump recognition part 204 recognizes, as a bump, a bump candidate region that meets the standard that at least one of the length with respect to a width direction of the travel path region, the length in a depth direction of the bump candidate region, and the complexity of a contour line of the bump candidate region has a prescribed determination threshold value.

Description

environment recognition device

[0001] The present invention relates to a driving environment detection technology for detecting surrounding objects in a driving environment based on camera images.

[0002] Patent Literature 1 discloses a vehicle control device that can provide information corresponding to the vehicle's position to an occupant or a system even when a Global Navigation Satellite System (GNSS) is unavailable, and that detects an area where an object exists (object area) based on image data or three-dimensional data. As a method for detecting this object area, a method is disclosed in which, to detect road steps such as speed bumps, a planar road portion in the direction of travel is detected and road surface irregularities are detected from the depth and height information. It also describes that road surface irregularities can be detected accurately by detecting maximum and minimum values ​​of depth and height information, tracking corresponding points over time as the vehicle moves, and determining the positions of steps based on points where tracking is successful multiple times. Instead of detecting maximum and minimum values ​​of the road surface, there is also a method in which depth and height data is used as input for machine learning to learn which positions are steps. This method is said to be effective in improving detection accuracy, as it makes it possible to detect steps with complex shapes or multiple consecutive steps (Patent Document 1, paragraphs 0007 and 0027, etc.).

[0003] Patent Literature 2 also discloses an object recognition device capable of highly accurately recognizing objects captured across overlapping and non-overlapping fields of view in multiple images captured by a stereo camera. Regarding the object recognition device, the reliability calculation unit includes a first reliability calculation unit that calculates an object identification score (identification result) based on three-dimensional information and a second reliability calculation unit that calculates an object identification score (identification result) based on texture information. It also describes how the device may generate distance images or parallax images by superimposing the acquired three-dimensional information on a two-dimensional image, and perform identification processing based on the generated distance images or parallax images. It also describes how the object redetection process detects objects by analyzing the texture information of the area. A convolutional neural network is used as the detection method. The convolutional neural network outputs the image positions of the object in the upper left and lower right corners. The output of the convolutional neural network can identify the area of ​​a vehicle captured across overlapping and non-overlapping fields of view. The device also discloses how the area of ​​the object area, which is the processing area, is compared with a predetermined threshold to determine the object to be recognized. It is claimed that this method enables highly accurate recognition of objects captured across overlapping and non-overlapping areas of the visual field (Patent Document 2, paragraphs 0015, 0016, 0024 to 0028, etc.).

[0004] JP 2023-12649 A JP 2022-64388 A

[0005] The vehicle control device disclosed in Patent Document 1 describes that, to detect road bumps such as speed bumps, the device detects a flat portion of the road in the direction of travel, detects maximum and minimum values ​​of depth and height information, tracks corresponding points over time as the vehicle moves, and sets the points where tracking is successful multiple times as the location of the bump, thereby accurately detecting the bump. However, in an actual driving environment, the mere presence of an uneven road surface is not equivalent to the presence of a bump that requires a warning or various controls. Generally, a bump is a convex protrusion on the road surface that extends across the entire width of the road and cannot be avoided by, for example, steering control. However, the vehicle control device disclosed in Patent Document 1 cannot detect only such bumps.

[0006] The object recognition device disclosed in Patent Document 2 describes acquiring texture information for an area captured across an overlapping area and a non-overlapping area of ​​the field of view from images captured by two cameras. However, in an actual driving environment, other vehicles besides the vehicle itself are also traveling, and it is not always possible to capture an image of the entire road surface where a bump exists. Blind spots on the road surface caused by other vehicles exist, but Patent Document 2 does not take this into consideration. As a result, it may not be possible to correctly detect a bump that is installed as a convex protrusion across the entire width of the road. As a result, it is not possible to properly issue a warning or control the detected bump area.

[0007] Therefore, the present invention aims to detect bumps in the driving environment, while simultaneously suppressing false detection and extending the detection distance, improving the accuracy of the timing of warnings and control for detected bump areas, and reducing false alarms and erroneous control.

[0008] One aspect of the present disclosure is an environmental recognition device that recognizes bumps installed on a road surface based on an image captured by a camera mounted on a vehicle, comprising: a driving area identification unit that identifies a driving road area in the image in which the driving road on which the vehicle is driving is imaged; a bump feature generation unit that generates a feature map that shows areas including features of the bump from the image together with a degree of likelihood; and a bump recognition unit that recognizes the bump based on bump candidate areas in the feature map whose degree of likelihood is equal to or greater than a threshold, wherein the bump recognition unit recognizes as the bump, among the bump candidate areas present on the driving road area, bump candidate areas in which one or more of the length in the width direction of the driving road area, the length in the depth direction of the candidate area, or the complexity of the contour line of the bump candidate area meets a criterion having a predetermined judgment threshold.

[0009] According to the above-described aspect of the present disclosure, it is possible to provide an environment recognition device that detects bumps in a driving environment, while simultaneously suppressing false detections and extending the detection distance, improving the accuracy of the timing of warnings and control for detected bump areas, and reducing false alarms and erroneous control.

[0010] Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0011] 1 is a diagram illustrating a first embodiment of an in-vehicle system including an environment recognition device according to the present disclosure. FIG. 2 is a diagram illustrating the internal configuration of a camera unit and an environment recognition unit according to the first embodiment. FIG. 3 is a diagram illustrating the configuration of a candidate area extraction unit. FIG. 4 is a diagram illustrating the configuration of a viewable area determination unit. FIG. 5 is a diagram illustrating the internal configuration of a camera unit and an environment recognition unit according to a second embodiment. FIG. 6 is a diagram illustrating the processing flow of a bump re-recognition unit. FIG. 7 is a diagram illustrating an example of an output determination table inside a result output unit. FIG. 8 is a diagram illustrating the internal configuration of a camera unit and an environment recognition unit according to a third embodiment. FIG. 9 is a diagram illustrating an example of a feature map. FIG. 10 is a diagram illustrating a fourth embodiment of an in-vehicle system including an environment recognition device according to the present disclosure. FIG. 11 is a diagram illustrating the internal configuration of a camera unit and an environment recognition unit according to the fourth embodiment.

[0012] Hereinafter, an embodiment of an environment recognition device according to the present disclosure will be described with reference to the drawings.

[0013] 1 shows an embodiment of an in-vehicle system including an environment recognition unit 102 according to the present disclosure. In Fig. 1, a camera unit 101 is installed so as to include a field of view in the traveling direction of a vehicle 100. An image of the front of the vehicle captured by this camera unit 101 is input to an environment recognition unit (environment recognition device) 102.

[0014] The environment recognition unit 102 can be configured, for example, by one or more microcontrollers equipped with a central processing unit (CPU), memories such as ROM and RAM, a timer, and input / output units. The environment recognition unit 102 recognizes bumps installed on the road surface based on images of the area ahead of the vehicle captured by the camera unit 101. The environment recognition unit 102 then calculates the distance to the bump, the height of the bump, the relative speed, a detection reliability index, and the like, and transmits the calculation results to the control unit 103.

[0015] The control unit 103 determines the risk of an impact or collision based on the distance to the bump, the height of the bump, the relative speed, and a detection reliability index, and outputs commands according to the risk, such as displaying information on a display 104, turning on a warning light, sounding a warning from a speaker 105, reducing or suppressing acceleration with an accelerator 106, decelerating with a brake 107, or changing the shock absorption characteristics of a suspension 108. Like the environment recognition unit 102, the control unit 103 can be configured, for example, by one or more microcontrollers equipped with a central processing unit (CPU), memories such as ROM and RAM, a timer, and input / output units.

[0016] In this embodiment, the bump is described as a speed bump, but is not limited to this. The bump in this embodiment may be any uneven surface that causes vibration of a predetermined magnitude or greater to a vehicle when the vehicle passes over it, and may include, in addition to speed bumps, one or more of a crosswalk, a train track or railroad crossing, and a joint in a bridge or viaduct.

[0017] The speed bumps 902 (see Figure 9) that the environment recognition unit 102 recognizes have a length that extends across the entire width of the road surface 901, including the vehicle's driving lane and, if there are any adjacent lanes that are traveling in the same direction as the vehicle's driving lane, also the adjacent lanes; speed bumps in the lane of oncoming vehicles are not included in the recognition targets.

[0018] Next, a method in which the environment recognition unit 102 detects a bump based on an image captured by the camera unit 101 will be described with reference to FIG.

[0019] 2 is a block diagram showing the internal configuration of the camera unit 101 and the environment recognition unit 102. Note that, although the camera unit 101 is described here as a stereo camera 201, the camera unit 101 may be configured as a combination of one or more cameras, such as a monocular camera, and a LiDAR, a combination of three or more cameras, or a combination of one or more cameras and a millimeter wave sensor.

[0020] The environment recognition unit 102 has at least a bump feature generation unit 202, a driving area identification unit 203, a bump recognition unit 204, and a result output unit 205. The bump recognition unit 204 also includes a candidate area extraction unit 241 and a visible area determination unit 242.

[0021] The stereo camera 201 consists of two cameras. Each camera is equipped with a CMOS (Complementary Metal Oxide Semiconductor) sensor and a DSP (Digital Signal Processor). The CMOS sensor is an imaging device with a grid of photodiodes that exhibit a photoelectric effect. It releases the shutter at any timing and transmits the accumulated charge to the DSP while performing analog-to-digital (AD) conversion. The DSP converts the transmitted information into an image while performing image quality corrections such as contrast correction, gamma correction, and edge correction. Distance information (distance image) and invalid areas are calculated from these images as parallax information. Note that an invalid area here refers to an area where it is difficult to accurately obtain stable parallax information, such as an area with little texture or an area with high similarity. The distance information and invalid area are collectively referred to as parallax information. The details of the method for calculating the parallax information are omitted here because there are several well-known methods for stereo image processing. The parallax information and at least one of the images are sent to the bump feature generation unit 202 and the driving area identification unit 203, respectively.

[0022] The bump feature generation unit 202 generates a feature map that represents regions containing bump features from images captured by the camera unit 101, along with the degree of likelihood. The bump feature generation unit 202 receives as input disparity information and grayscale images acquired from the stereo camera 201, and outputs a feature map in which bump-likeliness is expressed as a likelihood ranging from 0.0 to 1.0. This feature map can be realized using a convolutional neural network (hereinafter referred to as CNN) or the like. This can be realized by preparing multiple sets of training data in advance, each set including disparity information with a resolution of width W1 × height H1 acquired by the stereo camera 201, grayscale images with a resolution of width W2 × height H2 also acquired by the stereo camera 201, and annotation images with a resolution of width W3 × height H3, and then learning a CNN network structure through supervised learning based on this training data. By learning the network structure of the CNN in advance, when the parallax information and grayscale images obtained from the stereo camera 201 while driving are input to the CNN, a feature map of the likelihood of bumps can be obtained.

[0023] In this case, the resolution widths W1, W2, and W3 and the heights H1, H2, and H3 may be the same or different resolutions. Using the same resolution has the effect of simplifying the network structure and improving learning efficiency. On the other hand, when the baseline of the stereo camera 201 is horizontal, i.e., when the two cameras are installed horizontally spaced apart, the effective resolution of the disparity information is lower in the horizontal direction than in the vertical direction. Therefore, by halving the resolution of only the horizontal widths of the disparity information and the output feature map, i.e., W1 = W3 = W2 ÷ 2 and H1 = H2 = H3, it is possible to reduce the computational cost while suppressing a decrease in the accuracy of the output feature map.

[0024] An example of a feature map is shown in Fig. 9. Fig. 9(a1) shows the input camera image, and Fig. 9(a2) shows the road surface unevenness at that time. Fig. 9(b) shows the input disparity information, and Fig. 9(c) shows the concept of the output likelihood feature map. Here, it is assumed that the grayscale image acquired from the stereo camera 201 shows, as shown in Fig. 9(a1), the nearest road surface 901, a speed bump 902 installed so as to overlap the position of a pedestrian crossing a little further ahead, and a preceding vehicle 903 further ahead.

[0025] According to the road surface unevenness shown in FIG. 9( a2 ), it can be seen that the road surface 901 other than the speed bump 902 is almost flat. In this situation, the disparity information acquired by the stereo camera 201 is as shown in FIG. 9( b ), and a point cloud 904 can be acquired beyond the point cloud 905 of the flat road surface. These point clouds can be calculated based on the general principle of stereo vision using the texture of the asphalt and road markings. The bump feature generation unit 202 performs learning using as input the grayscale image ( FIG. 9( a1 )), disparity information ( FIG. 9( b )), and a so-called annotation image in which a bump region is identified by, for example, filling in the region corresponding to the speed bump 902 in FIG. 9( a1 ). As a result, during inference, a feature map such as that shown in FIG. 9( c ) can be output. The feature map shown in Figure 9(c) shows that the likelihood is 0 outside region 908 corresponding to the speed bump 902, and is high inside region 908, with even higher likelihood in regions 906 and 907 which react strongly to the bump. Here, the feature map (Figure 9(c)) is expressed as the same size as the input grayscale image (Figure 9(a1)), and shows how likelihood values ​​are output for each corresponding pixel.

[0026] The driving area identification unit 203 identifies a driving road area in which the driving road on which the vehicle 100 is traveling is captured and an object area in which an object existing on the driving road is captured in the captured image captured by the camera unit 101. The driving area identification unit 203 inputs the disparity information and grayscale image acquired from the stereo camera 201, and outputs an area map of the area segmentation result in which the object type to which each pixel belongs is calculated. This area map can be realized using semantic segmentation (SS) or the like. Specifically, the disparity information acquired by the stereo camera 201, the grayscale image also acquired by the stereo camera 201, and annotation images in which each pixel on the grayscale image is labeled with the type of area to which each pixel belongs are prepared in advance, and multiple sets of these are prepared as one set of training data, and a CNN network structure is trained by supervised learning based on this training data. An annotation image is an image in which each pixel is labeled, for example, as road asphalt area (=1), paint area (=2) such as road markings like white lines and speed limits, vehicle area (=3) such as cars and trucks, two-wheeled vehicle area (=4) such as bicycles and motorcycles, pedestrian area (=5), animal area (=6) such as cows and sheep, etc. By learning the CNN network structure in advance, disparity information and grayscale images obtained from the stereo camera 201 while driving can be input to the CNN to obtain a region map as an inference result of semantic segmentation.

[0027] The bump recognition unit 204 receives the feature map output from the bump feature generation unit 202 and the area map output from the driving area identification unit 203, determines whether or not a bump is present, and outputs the bump recognition result and its accompanying information to the result output unit 205. The bump recognition result and accompanying information include information such as whether or not a bump is recognized as present, and if so, the distance from the vehicle to the bump, the height from the road surface to the apex of the bump, the width of the bump, and the confidence level at the time of recognition.

[0028] The result output unit 205 receives the output result of the bump recognition unit 204 as input, and generates output information for the control unit 103 based on the time-series bump recognition results and accompanying information. Generally, there is a non-zero probability of erroneous recognition when recognizing the outside world using image recognition, so the result output unit 205 may determine whether or not to output final recognition based on multiple recognition results.

[0029] Specifically, the output result of the bump recognition unit 204 and additional information may be input and used for the calculation as follows. For example, it is assumed that the following information is obtained: A recognition flag F takes a value of 1 if a bump is recognized during N time periods from time t0, t-1, ..., t-(N-1), and takes a value of 0 if not. ｔ０ , F ｔ－１ , ..., F ｔ－（Ｎ－１） , the distance to the recognized bump D ｔ０ , D ｔ－１ , ..., D ｔ－（Ｎ－１） , the height of the recognized bump H ｔ０ , H ｔ－１ , ..., H ｔ－（Ｎ－１） In this case, (Condition A) ΣFx≧Thr1, (Condition B) D ｔ０ ≧D ｔ－１ ≧・・・≧D ｔ－（Ｎ－１） , (condition C) Var(H ｘ ) Thr2≦Thr3, the bump recognition result and the accompanying information may be generated as output information to the control unit 103.

[0030] That is, (Condition A) means that detection must be performed at least Thr1 times within N time periods. If detection is performed at least Thr1 times while the distance to the target changes as the vehicle travels, it is possible to eliminate false positives that may occur accidentally. (Condition B) means that the distance to the bump becomes shorter over time and the bump approaches monotonically, which is possible to eliminate false positives caused by light or shadow that may occur accidentally. (Condition C) means that the variance of the height of the observed bump is equal to or less than a certain threshold Thr2 and that the bump is observed stably. Since the physical shape of the bump does not change over a short period of time, observing the bump multiple times while the distance to the target changes as the vehicle travels and verifying that its shape and height do not change is possible to eliminate false positives that may occur accidentally due to noise, etc. Note that (Condition B) and (Condition C) may use data only for the time period during which the bump is recognized.

[0031] Note that the detailed values ​​of the detection conditions are design matters, and the result output determination may be performed under conditions other than those described above. Also, while the above-mentioned (Condition B) sets the determination condition to be only that the vehicle is monotonically approaching the bump, the determination condition may also be that the vehicle's traveling distance or movement distance is calculated using a method generally known as the dead reckoning method, and the traveling distance or movement distance is compared with the amount of change in the distance to the bump, and the difference is within a certain value and is approximately the same.

[0032] The bump recognition unit 204 also includes a candidate area extraction unit 241 and a viewable area determination unit 242 .

[0033] The candidate area extraction unit 241 will be described using Fig. 3. The candidate area extraction unit 241 receives the bump features (hereinafter also referred to as feature maps) generated by the bump feature generation unit 202 as input, and outputs bump candidate areas. Of the bump candidate areas present on the road area, the candidate area extraction unit 241 extracts areas whose length in the width direction of the road area is equal to or greater than a determination threshold as bump candidate areas.

[0034] The candidate area extraction unit 241 includes a high likelihood area extraction unit 301 , a width direction feature calculation unit 302 , a depth direction feature calculation unit 303 , a contour shape feature calculation unit 304 , and a bump candidate area estimation unit 305 .

[0035] The high-likelihood area extraction unit 301 determines a bump candidate area based on the feature map generated by the bump feature generation unit 202, and further calculates the relative position of the bump candidate area with respect to the vehicle. As described above, the feature map is output so that the likelihood is high inside the area 908 corresponding to the bump area, and areas 906 and 907 have even higher likelihood in areas where the probability of a bump is even higher. Therefore, by comparing the likelihood of each point with a threshold value ThrC and extracting points with high likelihood, it is possible to extract a high-likelihood area with a high probability of being a bump. For example, among the high-likelihood areas with a high probability of being a bump, the area with the largest area is extracted, and the coordinates of the center of gravity of the extracted area are taken to determine the bump candidate area. In this way, it is possible to calculate the position of the bump candidate area, such as how far forward the bump candidate area is from the vehicle and how far it is offset to the left or right from the center of the vehicle. Furthermore, after extracting the area with the largest area from these bump candidate areas, the area with the next largest area may be calculated as the second bump candidate area by excluding that area and performing the same process again, and its position relative to the vehicle may be calculated.In this way, the high-likelihood area extraction unit 301 may calculate one or more bump candidate areas and output them as bump candidate areas together with their area information or ranking information indicating which area is the largest.

[0036] The width direction characteristic calculation unit 302 calculates the width direction characteristic FW, which is the length in the direction perpendicular to the vehicle's path, i.e., the width direction length of the bump, for the bump candidate area calculated by the high likelihood area extraction unit 301 described above.

[0037] The depth direction feature calculation unit 303 calculates a depth direction feature FD, which is the length in the direction of the vehicle's travel path, i.e., the depth direction length of the bump, for the bump candidate area calculated by the high likelihood area extraction unit 301 described above.

[0038] The contour shape feature calculation unit 304 calculates a contour shape feature FC, which is a numerical value that indexes the complexity of the contour of the bump candidate region calculated by the high-likelihood region extraction unit 301. Specifically, as the straightness of the sides in the width direction of the bump, the contour of each of the top and bottom sides of the bump candidate region may be approximated as a straight line using the Hough transform, and the variance of the points that make up the contour with respect to the approximated straight line may be calculated. If the top and bottom sides of the bump candidate region are straight lines, such as in the case of a perfect trapezoid, the variance of this straightness will be 0.

[0039] The bump candidate area estimation unit 305 receives outputs from the width direction feature calculation unit 302, depth direction feature calculation unit 303, and contour shape feature calculation unit 304, and determines whether or not each area extracted by the high likelihood area extraction unit 301 is to be a bump candidate area depending on whether or not each calculated feature satisfies a criterion having a predetermined judgment threshold. If the area is determined to be a bump candidate area, the unit 305 calculates a bump certainty factor that approaches 1 if the bump likelihood is high and approaches 0 if not.

[0040] Specifically, if the region is a bump, the width direction feature FW is expected to be at least equal to or greater than the vehicle width, so a threshold value ThrW1 equivalent to the vehicle width and a threshold value ThrW2 equivalent to the road width are used. If the width direction feature FW is smaller than threshold ThrW1, the region is determined not to be a bump candidate region, and if the width direction feature FW is equal to or greater than threshold ThrW1, the region is determined not to be a bump candidate region. If the width direction feature FW is equal to or greater than threshold ThrW1, the region is determined not to be a bump candidate region. If the width direction feature FD is equal to or greater than threshold ThrD1 and threshold ThrD2, which are the depth directions of standard bumps, the region is determined not to be a bump candidate region if it is not within that range. If the region is within that range, the region is determined not to be a bump candidate region. If the contour shape feature FC is equal to or greater than threshold ThrC, the region is determined not to be a bump candidate region. If the contour shape feature FC is equal to or greater than threshold ThrC, the region is determined not to be a bump candidate region. In this way, bump candidate regions can be extracted based on the feature amounts of the width direction feature calculation unit 302, the depth direction feature calculation unit 303, and the contour shape feature calculation unit 304.

[0041] In the above example, a bump candidate region is extracted based on the feature values ​​of the width direction feature calculation unit 302, the depth direction feature calculation unit 303, and the contour shape feature calculation unit 304. However, a bump candidate region may be extracted based on at least one of the feature values.

[0042] The bump certainty factor may be designed to decrease as the FW approaches the respective determination thresholds. Specifically, the certainty factor ConfW of the width direction characteristic can be calculated as ConfW = 0 (FW < ThrW1), ConfW = 1 (FW > ThrW2), ConfW = - (FW - ThrW2) ÷ (ThrW1 - ThrW2) ÷ 2 + 1 (FW ≧ ThrW1, FW ≦ ThrW2), etc., so that the certainty factor is low to about 0.5 when the width direction characteristic FW is approximately equal to the vehicle width, and approaches 1 as the FW becomes wider. Other certainty factors can be designed in a similar manner.

[0043] The visible area determination unit 242 will be described using Figure 4. The visible area determination unit 242 receives as input bump candidate areas, bump features (feature map), and information on road areas (area map), recognizes bumps based on the overlap rate of the bump candidate areas with the visible area, and outputs the recognition result and its accompanying information. The visible area determination unit 242 has a tentative bump candidate estimation unit 401, a distance / TTC calculation unit 402, a road width estimation unit 403, an obstructed area estimation unit 404, a bump visible area estimation unit 405, and a visible area overlap rate determination unit 406.

[0044] The tentative bump candidate estimation unit 401 performs subsequent processing on tentative bump candidates, starting with the most reliable candidate region among the bump candidate regions extracted by the candidate region extraction unit 241, and can terminate the processing when there is no more room for calculation resources or processing time, or when all bump candidate regions have been processed. This has the effect of preventing the processing time from significantly exceeding the limit, even when many bump candidate regions exist in the image.

[0045] The distance / TTC calculation unit 402 calculates the distance from the vehicle to the tentative bump candidate output from the tentative bump candidate estimation unit 401. There are several known techniques for calculating this distance. For example, when the vehicle is traveling stably, the mounting height and visual axis of the camera are known from the design values, and the camera angle of view and distortion information are also known. Therefore, it is easy to calculate, using the principle of triangulation, which point on the road surface plane on which the vehicle is traveling in real space corresponds to the coordinates observed on the image, and the distance can be calculated. Alternatively, when a stereo camera is used, the TTC can be calculated using the principle of triangulation from the coordinates observed on the left and right cameras and the relative positional relationship between the left and right cameras. Based on the distance calculated in this way and the vehicle speed, the TTC (Time To Collision) to the bump can be calculated. The coordinates observed on the camera at this time may be the center of the temporary bump candidate area, or the distance may be calculated from the center of the front side (closer to the vehicle) and the center of the back side (farther from the vehicle) of the temporary bump candidate, and then the average of these distances may be calculated. This has the effect of preventing this and making it possible to calculate coordinates closer to the true center, since the back side is reduced and the front side is enlarged on the image due to perspective transformation, so if the center of the candidate area on the image is simply taken, the calculation will end up being closer to the center.

[0046] The road width estimation unit 403 estimates the width Lane of the road based on the area map of the area division result, which calculates the object type to which each pixel belongs, identified by the road area identification unit 203. For example, the unit may calculate the distance between the left and right edges of the road at 10 points 11 m, 12 m, ..., 20 m ahead and output the most frequent value, or may extract left and right road edge boundary lines based on the boundaries between the road area and off-road area from the area map of the area division result, and use the distance between them as the width of the road.

[0047] The occluded area estimation unit 404 estimates the width of the object area of ​​a vehicle or obstacle that obscures the road surface (blind spot area) at the distance where the tentative bump candidate is located, based on the area map of the area segmentation result identified by the driving area identification unit 203, which calculates the object type to which each pixel belongs. Note that if the vehicle or obstacle is located farther away than the tentative bump candidate is located, the road surface width obscured by them will be zero. On the other hand, if the vehicle or obstacle is located closer than the tentative bump candidate is located, the road surface width obscured by them will be wider than the width of the vehicle or obstacle itself. Specifically, when a vehicle or obstacle with a width Width is located at a distance Dist1 from the camera, the obscured road surface width Mask (blind spot area) at a distance Dist2 where the tentative bump candidate is located is expressed as Width × Dist2 ÷ Dist1 (where Dist1 < Dist2).

[0048] The bump visible area estimation unit 405 estimates the visible area as an area obtained by excluding blind spot areas blocked by object areas from the road area. The bump visible area estimation unit 405 calculates the width of the unblocked road surface at the distance where the tentative bump candidate is located. Specifically, the bump visible area estimation unit 405 calculates the width of the unblocked road surface at the distance where the tentative bump candidate is located by subtracting the blocked road surface width Mask from the road width Lane (=Lane - Mask).

[0049] The viewable area overlap rate determination unit 406 determines to what extent the bump candidate area output from the candidate area extraction unit 241 matches the width of the bump visible at that distance estimated by the bump viewable area estimation unit 405. Specifically, the width of the bump candidate area is divided by the width over which the bump is estimated to be visible to calculate an overlap rate in the range of 0 to 1. If this overlap rate is greater than a threshold ThrO, it is determined that the candidate area sufficiently overlaps with the viewable area, and is output as a recognized bump.

[0050] In this way, the bump recognition unit 204, which has the candidate area extraction unit 241 and the viewable area determination unit 242, recognizes bumps. The bump recognition unit 204 recognizes bumps based on the viewable area excluding blind spot areas that are blocked by object areas within the road area.

[0051] The result output unit 205 receives the bump recognition result recognized by the bump recognition unit 204 and its associated information, as well as one or more pieces of information on the speed of the vehicle, the driving environment, and the lighting environment, and outputs how control should be performed. A specific example will be described with reference to Fig. 7. Fig. 7 shows an example of an output determination table stored inside the result output unit 205 (not shown).

[0052] For example, No. 1 is a scenario in which the vehicle is traveling on a paved road in a city on a clear day, and the parameters are set to output signals at standard output determination distances (30 m for warning, 20 m for pre-emptive control, and 10 m for deceleration control). No. 2 is a scenario in which the vehicle is traveling on a paved road in a city on a clear night. In this case, the vehicle's headlights cast shadows on the bumps, making them easier to detect than during the day and reducing the risk of false detection. In this case, since the driver has difficulty seeing the bumps, warnings and pre-emptive control are performed in the same way as during the day, but the output parameters are set to perform deceleration control earlier. Furthermore, in an environment where false detections are likely to occur, such as in rainy weather at night, the output parameters are set to not perform deceleration control in order to prevent false detections. Also, in No. 3, the output parameters are set to not perform deceleration control in order to prevent false detections. On unpaved roads like No. 4, vehicle body sway occurs due to road surface irregularities, making it difficult to accurately estimate the road surface, so an output parameter is set to shorten the output determination distance in order to repeatedly detect and verify warnings, preliminary control, and actual control. Also, when traveling at low speeds like No. 5, the risk is low even without control for bumps, so an output parameter may be set to only issue a warning.

[0053] In this way, the environment recognition unit of this embodiment 1 can detect bumps in the driving environment and output information for appropriate control even in situations where the detection performance of the camera is reduced.

[0054] Second Embodiment Hereinafter, a second embodiment of the environment recognition unit according to the present disclosure will be described with reference to FIG.

[0055] 5 shows one embodiment of an in-vehicle system including an environment recognition unit 102 according to the present disclosure. The environment recognition unit 102 of this embodiment differs from the environment recognition unit 102 of the first embodiment in that it additionally includes a bump re-recognition unit 501. Other aspects of the environment recognition unit 102 of this embodiment are similar to those of the environment recognition unit 102 of the first embodiment, and therefore similar parts are designated by the same reference numerals and description thereof will be omitted.

[0056] In the environment recognition unit 102 of the first embodiment, the bump recognition unit 204 outputs the bump recognition result and its accompanying information, but the bump recognition unit 204 of the present embodiment differs in that, even if a bump is not recognized, it outputs the result of non-recognition and the accompanying information for each candidate area output by the candidate area extraction unit 241. The bump re-recognition unit 501 has the function of receiving the result of non-recognition and the accompanying information, changing various thresholds according to the control type, performing bump determination again, and outputting the result.

[0057] The bump re-recognition unit 501 changes the determination threshold value (re-determination setting unit) based on the control type indicating the control state of the vehicle 100 and the distance to the bump candidate area of ​​the vehicle 100. Then, if there is a bump candidate area that was not recognized as a bump until the previous processing cycle, the bump re-recognition unit 501 re-recognizes the bump based on the changed determination threshold value.

[0058] The processing flow of the bump re-recognition unit 501 is shown in Fig. 6. First, in S101, a control type including one or more of warning, preliminary control, and deceleration control is set. This setting may be made using the control type included in the recognition request sent from the control unit 103, or may be made using the control type for which the output determination distance is set in the output determination table shown in Fig. 7 described above.

[0059] Next, in S102, it is determined whether the parallax accuracy is above a certain level for the distance of the candidate area output by the candidate area extraction unit 241. This parallax accuracy can be calculated using one or more of the stereo camera's baseline length, the resolution of each camera, exposure time, aging and temperature changes, and calibration status. For example, when a point corresponding to point P on the left camera image is captured as point Q on the right camera image, the coordinate difference between points P and Q changes depending on the distance. The distance that changes when this coordinate difference shifts by one unit is the parallax resolution. Here, one unit refers to the search width when performing a correspondence search. If searching using integer coordinates, it is one pixel. If performing a subpixel search, it is a subpixel such as a quarter pixel. This can be calculated from the camera installation status, baseline length, and resolution of each camera. Furthermore, in dark lighting environments, an attempt is made to capture a brighter image of the subject by extending the exposure time. However, extending the exposure time in an in-vehicle camera increases the likelihood of motion blur, which reduces parallax accuracy. In addition, distortion of the stereo camera housing due to aging and temperature changes may occur, resulting in a decrease in parallax accuracy. Furthermore, because calibration is performed using large-scale equipment when the stereo camera is shipped from the factory, the calibration state is good and highly accurate. However, if the stereo camera is replaced or reinstalled at a dealer or other facility for some reason, the calibration state may become poor if calibration is performed using simple equipment, which may result in a decrease in parallax accuracy. Therefore, the parallax accuracy can be calculated by multiplying the parallax resolution by a coefficient of 1 or more depending on the exposure time, aging, temperature changes, and changes in the calibration state. This coefficient may be recorded, for example, as a table during design. If the parallax accuracy at the distance of the candidate area calculated in this way is lower than the height of the bump to be detected (for example, if the parallax accuracy is 10 cm and the bump to be detected is 5 cm), the bump cannot be detected correctly. In such a case, S102 determines that the parallax accuracy is not above a certain level (= No).Conversely, if the calculated parallax accuracy value at the distance of the candidate area is smaller than the height of the bump to be detected, it is determined in S102 that the parallax accuracy is equal to or greater than a certain level (=Yes), and the process proceeds to S103.

[0060] In S103, a determination is made based on the control type and TTC (Time To Collision). If the control type set in S101 is deceleration control and the TTC calculated from the vehicle speed and the distance to the candidate area output by the candidate area extraction unit 241 is equal to or less than a preset TTC threshold value for deceleration control (= Yes), the process proceeds to S104; if not (= No), the process proceeds to S105.

[0061] In S105, a determination is made based on the control type and TTC, as in S103. If the control type set in S101 is preliminary control and the TTC calculated from the vehicle speed and the distance to the candidate area output by the candidate area extraction unit 241 is equal to or less than the preset TTC threshold for preliminary control (= Yes), the process proceeds to S106; otherwise (= No), the process proceeds to S107.

[0062] In S107, a determination is made based on the control type and TTC, as in S103. If the control type set in S101 is warning, and the TTC calculated from the vehicle speed and the distance to the candidate area output by the candidate area extraction unit 241 is equal to or less than the preset TTC threshold for warning (=Yes), the process proceeds to S108. If not (=No), the result of the bump re-recognition is output as non-recognition. In S104, the determination threshold is set to a determination threshold A that makes recognition easier than the bump recognition unit 204, and the process proceeds to S109. In other words, the determination threshold is set to a determination threshold A that is lower than the determination threshold in the bump recognition unit 204, and the process proceeds to S109. In S106, the determination threshold is set to a determination threshold B that makes recognition easier than the determination threshold A, and the process proceeds to S109. In other words, the determination threshold is set to a determination threshold B that makes recognition easier than the determination threshold in the bump recognition unit 204, and the process proceeds to S109. In S108, a judgment threshold C is set that is easier to recognize than the judgment threshold B, and the process proceeds to S109. In other words, the judgment threshold C is set to a value lower than the judgment threshold in the bump recognition unit 204, and the process proceeds to S109. In S109, bump shape determination is performed using the set judgment threshold, but this processing is the same as the processing in the bump recognition unit 204, so details will be omitted.

[0063] By doing this, the environment recognition unit 102 of this embodiment 2 waits for output when the detection accuracy of road surface unevenness is low at a distance, and is able to make a judgment once the resolution of the stereo camera becomes sufficient for bump detection, which has the effect of suppressing false detection.

[0064] In the above embodiment, three determination processes S103, S105, and S107 are provided, and when the TTC threshold value of each determination process is equal to or less than the threshold value, the determination threshold value is set to a value that makes it easier to recognize the signal. However, the present invention is not limited to this. For example, only the processes of S103 and S104, or only the processes of S103 to S106 may be performed.

[0065] Third Embodiment Hereinafter, a third embodiment of the environment recognition unit 102 according to the present disclosure will be described with reference to FIG.

[0066] 8 shows one embodiment of an in-vehicle system including an environment recognition unit 102 according to the present disclosure. The environment recognition unit 102 of this embodiment differs from the environment recognition unit 102 of the second embodiment in that it additionally includes a surrounding vehicle observation unit 801 and a map processing unit 802. Other aspects of the environment recognition unit 102 of this embodiment are similar to those of the environment recognition unit 102 of the previous embodiment, and therefore similar parts are designated by the same reference numerals and description thereof will be omitted.

[0067] Since bumps are installed to the left and right ends of the road, if there is another vehicle (surrounding vehicle) traveling ahead in the same direction as the host vehicle in an adjacent lane, that other vehicle will often decelerate just before the bump. The environment recognition unit 102 of this embodiment 3 is configured to detect the deceleration of another vehicle traveling ahead in an adjacent lane, and by changing the threshold for bump recognition, it is possible to make it easier to detect bumps even when not many bump features can be detected due to obstructions, etc.

[0068] In the environment recognition unit 102 of the first embodiment, the bump recognition section 204 outputs the bump recognition result and its accompanying information, but the bump recognition section 204 of the third embodiment differs in that it can estimate the presence of a bump by observing the behavior of other vehicles, even if the bump itself is not sufficiently observed by the stereo camera, making it easier to recognize the bump.

[0069] The surrounding vehicle observation unit 801 observes the behavior of preceding vehicles traveling in the same lane as the host vehicle or in adjacent lanes. Based on the area map of the area division results output by the traveling area identification unit 203, the surrounding vehicle observation unit 801 extracts vehicle areas such as passenger cars and trucks, and painted areas such as white lines, and tracks the movement of each vehicle area using the time-series extraction results. In addition, the surrounding vehicle observation unit 801 calculates the relative positions of these areas with respect to the host vehicle from the camera installation information and parallax information corresponding to the extracted vehicle areas.

[0070] The map processing unit 802 calculates the behavior of other vehicles on a map based on the other vehicle information observed by the surrounding vehicle observation unit 801 and the vehicle speed and steering angle of the vehicle. Specifically, the map calculates the traveling trajectory and traveling behavior, such as the vehicle speed changes, relative to the lane estimated from painted white lines and other markings. The map processing unit 802 arranges and tracks bump position candidates estimated from the bump features on the map.

[0071] In addition to the information described in the first or second embodiment, the bump recognition unit 204 also uses as input the travel path and travel behavior of other vehicles calculated by the map processing unit 802. The bump recognition unit 204 tracks the behavior of other vehicles on the map obtained by the surrounding vehicle observation unit 801, and when a preset condition is met, performs processing to lower the determination threshold so that the vehicle is more likely to be recognized as a bump. Specifically, when the travel path of the other vehicle is traveling along a lane and the vehicle is decelerating, there is a possibility that the vehicle is not changing lanes but that a bump is present ahead. Therefore, the width direction feature output by the width direction feature calculation unit 302 is used to lower the threshold used by the bump candidate area estimation unit 305 to estimate a bump candidate area, making it easier to recognize the bump. The bump recognition unit 204 performs processing to lower the determination threshold so that the vehicle is more likely to be recognized as a bump when one or more conditions are met, for example, when a vehicle in an adjacent lane has applied the brakes, the inter-vehicle distance has shortened, or the tail lamp illuminance has changed.

[0072] In this way, the environment recognition unit of this embodiment 3 has the advantage that even if the bump itself is not sufficiently observed by the stereo camera, it can estimate the presence of the bump by observing the behavior of other vehicles, making it easier to recognize the bump.

[0073] Fourth Embodiment Hereinafter, a fourth embodiment of the environment recognition unit 102 according to the present disclosure will be described with reference to FIGS.

[0074] Fig. 10 is a diagram illustrating a fourth embodiment of an in-vehicle system including an environment recognition unit according to the present disclosure, and Fig. 11 is a diagram illustrating the configuration of the environment recognition unit in the fourth embodiment. The environment recognition unit 102 in the fourth embodiment differs from the environment recognition unit 102 in the first or second embodiment described above in that it additionally includes an environment recording device 1001 and a communication device 1002. Other aspects of the environment recognition unit 102 in the fourth embodiment are similar to those of the environment recognition unit 102 in the first or second embodiment described above, and therefore similar parts are denoted by the same reference numerals and description thereof will be omitted.

[0075] The environment recognition unit 102 in the first embodiment operates independently of the vehicle itself, recognizing the ever-changing environment around the vehicle at any time based on external information obtained by a camera mounted on the vehicle. However, the environment recognition unit 102 in the fourth embodiment differs in that it keeps records and communicates with the outside world as needed.

[0076] The environment recording device 1001 can record the recognition results of the environment recognition unit 102 and the vehicle position output by a locator device (not shown) as a set, and can also read out the recognition results and recorded positions that exist near the input vehicle position from the content recorded using the vehicle position as a query.

[0077] The communication device 1002 may communicate with one or more of other vehicles, road facilities, or map servers, and upload the contents of the environment recording device 1001 recorded by the vehicle itself. The communication device 1002 corresponds to a communication unit in the claims. Conversely, the communication device 1002 may communicate with one or more of other vehicles, road facilities, or map servers, and receive recognition results such as bump positions and heights observed by devices other than the vehicle itself. Based on the received information, the environment recognition unit 102 may adjust parameters or override the recognition results so that the environment recognition unit 102 can more easily recognize the bumps.

[0078] Next, a method for the environment recognition unit 102 to detect a bump based on an image captured by the camera unit 101 will be described.

[0079] 11 is a block diagram showing the internal configuration of the camera unit 101 and the environment recognition unit 102. The environment recognition unit 102 of this embodiment differs from the environment recognition unit 102 of the first to third embodiments described above in that it additionally includes a self-position estimation unit 1101, such as a locator device, an environment recording unit 1102, and a communication device 1103.

[0080] The self-position estimation unit 1101 has a function of estimating the position of the vehicle by receiving, for example, latitude, longitude, and altitude obtained by a GNSS receiver, and outputs the result to the environment recording unit 1102. Alternatively, the self-position estimation unit 1101 may have a function of recognizing the environment around the vehicle using a camera or LiDAR, and estimating the position of the vehicle by comparing the environment with an internal map (not shown).

[0081] The environment recording unit 1102 receives as input the distance to the bump, height, and recognition reliability calculated by the bump recognition unit 204 through the processing described in the first embodiment, as well as the self-position of the self-position estimation unit 1101, and records these as a set of bump information on a hard disk (HDD), solid state disk (SSD), memory, etc. Furthermore, the environment recording unit 1102 reads out the recorded information, and when the self-position of the self-position estimation unit 1101 is input, it outputs to the bump recognition unit 204 one or more of the position, height, and recognition reliability of a nearby bump within a threshold value of the input self-position.

[0082] The bump recognition unit 204 has the position and height of the bump obtained from the environment recording unit 1102, and if the recognition reliability is greater than a predetermined threshold, the corresponding position, height, and recognition reliability of the bump may be replaced with the information obtained from the environment recording unit 1102. In this way, instead of performing detection several times from a distance to improve the detection reliability index, it becomes possible to detect bumps from a greater distance with higher detection reliability.

[0083] The communication device 1103 may communicate the contents of the environment recording unit 1102 with one or more of other vehicles, road facilities, and map servers to upload or download bump information indicating the position and height of bumps. This makes it possible to detect bumps that the vehicle has not detected before, such as bumps that exist on roads the vehicle is traveling on for the first time, from a greater distance with higher detection reliability, and to take action against the bumps from a greater distance.

[0084] According to the first to fourth embodiments of the present invention described above, the camera unit 101 installed in the vehicle 100 is used to observe the surroundings, and it is determined from the three-dimensional information obtained by the camera unit 101 whether or not a collision will occur even if the vehicle 100 proceeds. Even if it is determined that a collision will not occur, it is possible to analyze from the obtained three-dimensional road surface information whether or not there are bumps or other irregularities ahead of the vehicle 100 that will cause large vibrations to the vehicle 100, and based on the analysis results, it is possible to notify the driver and control the suspension and brakes to suppress vibrations of the vehicle 100. That is, the shape of the road surface ahead of the vehicle 100 is measured using the camera unit 101, and based on the measurement results, it is possible to alert the driver before vibrations are transmitted to the vehicle 100 for safer driving and control the vehicle 100 for more comfortable driving. For example, if it is predicted that vehicle 100 will enter a bump area at a speed higher than the appropriate vehicle speed range, the driver may be alerted by sound or display to slow down vehicle 100, or control may be performed to suppress acceleration or decelerate vehicle 100 just before the bump area, or semi-active control may be implemented to soften the suspension of vehicle 100 just before the bump, thereby suppressing the impact when passing over the bump, thereby mitigating the impact on the vehicle body.

[0085] Furthermore, according to the first to fourth embodiments of the present invention, when using, for example, disparity information obtained from a stereo camera to detect road bumps (e.g., bumps, deceleration bumps, cracks, potholes, etc.), false detections and degradation of detection performance due to the fundamental characteristics of the stereo camera can be suppressed. That is, because the resolution of disparity information obtained from a stereo camera is lower at a distance than at a nearby location, false detections may not be eliminated when attempting to confirm a detection at a distance because low-resolution disparity information is used. Furthermore, if the detection confirmation is always waited for until the disparity information becomes high-resolution and the detection is confirmed only when the vehicle is close, the detection distance may be shortened. According to the first to fourth embodiments of the present invention, it is possible to simultaneously suppress false detections and extend the detection distance, thereby improving the accuracy of the timing of warnings and control for detected bump areas and reducing false alarms and erroneous control.

[0086] Although the embodiments of the environment recognition unit according to the present disclosure have been described in detail above using the drawings, the specific configuration is not limited to this embodiment, and design changes and the like that do not deviate from the gist of the present disclosure are also included in the present disclosure. For example, the above-described embodiments have been described in detail to clearly explain the present disclosure, and are not necessarily limited to those that include all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0087] 100...vehicle, 101...camera unit (camera), 102...environment recognition unit (environment recognition device), 103...control unit, 201...stereo camera, 202...bump feature generation unit, 203...traveling area identification unit, 204...bump recognition unit, 205...result output unit, 241...candidate area extraction unit, 242...visible area determination unit, 301...high likelihood area extraction unit, 302...width direction feature calculation unit, 303...depth direction feature calculation unit, 304...contour shape feature calculation unit, 305...bump candidate area estimation unit, 401...bump tentative candidate estimation unit, 402...distance / TTC calculation unit, 403...road width estimation unit, 404...occlusion area estimation unit, 405...bump visible area estimation unit, 406...overlap rate determination unit for visible area, 501...bump re-recognition unit, 801...surrounding vehicle observation unit, 802...map processing unit, 901...road surface, 902...speed bump, 903...preceding vehicle, 904...bump area point cloud, 905...flat road surface point cloud, 906...area, 907...area strongly responding to bump, 908...area corresponding to bump area, 1001...environment recording device, 1002...communication device, 1101...self-position estimation unit, 1102...environment recording unit

Claims

1. An environmental recognition device that recognizes a Bump installed on a road surface based on an image captured by a camera mounted on a vehicle, the environmental recognition device comprising: a traveling region specifying unit that specifies a traveling road region in which a traveling road on which the vehicle travels is imaged in the image; a Bump feature generating unit that generates a feature map representing a region including the feature of the Bump from the image together with the degree of certainty; a Bump recognizing unit that recognizes the Bump based on a Bump candidate region in which the degree of certainty in the feature map is equal to or greater than a threshold value, wherein the Bump recognizing unit recognizes, as the Bump, a Bump candidate region among the Bump candidate regions existing on the traveling road region that conforms to a criterion having any one or more of a length in the width direction of the traveling road region, a length in the depth direction of the Bump candidate region, or the complexity of the contour line of the Bump candidate region being equal to or greater than a predetermined determination threshold value. An environmental recognition device characterized by this.

2. The traveling region specifying unit specifies an object region in which an object existing on the traveling road is imaged in the image, wherein the Bump recognizing unit recognizes the Bump based on a visible region excluding a dead angle region shielded by the object region from the traveling road region. The environmental recognition device according to claim 1, characterized by this.

3. The determination threshold value is changed based on a control type indicating a control state of the vehicle and a distance to the Bump candidate region of the vehicle, and when there is a Bump candidate region that was not recognized as the Bump by the previous processing cycle, the Bump is re-recognized based on the changed determination threshold value. The environmental recognition device according to claim 1, characterized by having a Bump re-recognizing unit.

4. It has a surrounding vehicle observing unit that observes the behavior of other vehicles in adjacent lanes, wherein the Bump recognizing unit lowers the determination threshold value so that it is more likely to be recognized as the Bump when any one or more of the conditions that another vehicle in the adjacent lane applies brakes, the inter-vehicle distance is shortened, or the tail lamp illuminance changes are satisfied. The environmental recognition device according to claim 1, characterized by this.

5. It has a map processing unit that arranges and tracks the position candidates of the Bump estimated from the feature map on a map. The Bump recognition unit tracks the behavior of other vehicles on the map obtained by the surrounding vehicle observation unit, and when the behavior conforms to a preset condition, lowers the determination threshold so as to be more likely to recognize it as the Bump. The environmental recognition device according to claim 4, characterized in that.

6. It has a communication unit capable of communicating with one or more of other vehicles different from the vehicle, road facilities, or a server, and an environmental recording unit that records Bump information including at least the position of the Bump. The communication unit transmits and receives the Bump information recorded in the environmental recording unit to and from one or more of the other vehicles, road facilities, or the server. The environmental recognition device according to claim 1, characterized in that.

7. It has a self-position estimation unit that estimates the position of the vehicle. The Bump recognition unit recognizes the Bump based on the self-position of the vehicle estimated by the self-position estimation unit and the Bump information received from one or more of the other vehicles, road facilities, or a map server via the communication unit. The environmental recognition device according to claim 6, characterized in that.

Images