Display control device, display control method and program

Abstract

To provide a display control device, a method, and a program, which can appropriately call attention.SOLUTION: A display control device includes: an image data acquisition section 121 that acquires image data of an image obtained by capturing the outside of an own vehicle; an object detection section 122 that detects an object included in the image; a distance measurement data acquisition section 125 that acquires distance measurement data on distance from the own vehicle to the object; a risk degree setting section 127 that sets a degree of risk to the object according to the size of the object when the distance is a threshold or less; a highlighting processing section 128 that performs highlighting processing according to the degree of risk on the image data; and an output section 129 that outputs a highlighted image obtained by performing the highlighting processing.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a display control device, a display control method, and a program. [Background technology]

[0002] Patent Document 1 discloses a surroundings monitoring device having a distance measuring detector that detects the distance from a vehicle to an object. The surroundings monitoring device of Patent Document 1 has a camera that captures an image of an object from an imaging unit provided in the vehicle. The surroundings monitoring device displays the color of the object and a color according to the height of the object in a superimposed manner.

[0003] Patent Document 2 discloses a vehicle surroundings visual recognition device that includes an obstacle detection means for detecting the distance between the vehicle and an obstacle, and an imaging unit for imaging the scenery outside the vehicle. When an obstacle is detected, the device of Patent Document 2 displays a figure on the captured image that allows the user to recognize the direction in which the obstacle is located. This device changes the display color or size of the figure depending on the distance to the obstacle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-97843 A [Patent Document 2] JP 2006-311222 A Summary of the Invention [Problem to be solved by the invention]

[0005] In such a device, it is desirable to appropriately warn the driver. For example, when multiple obstacles are included, it is preferable to warn the driver about objects with a higher degree of danger. However, when multiple objects are present around the vehicle, changing the display color according to the distance or height may cause the display to become cluttered. For example, if objects with a lower degree of danger are emphasized, it may not be possible to effectively warn the driver about objects with a higher degree of danger.

[0006] In view of the above problems, the present invention aims to provide a display control device, a display control method, and a program that can effectively alert a user by appropriately setting a risk level. [Means for solving the problem]

[0007] The display control device of this embodiment includes an image data acquisition unit that acquires image data of an image taken outside the vehicle, an object detection unit that detects objects contained in the image, a ranging data acquisition unit that acquires ranging data regarding the distance from the vehicle to the object, a risk level setting unit that sets a risk level of the object according to the size of the object when the distance falls below a threshold value, an emphasis display processing unit that performs emphasis display processing on the image data according to the risk level, and an output unit that outputs a highlight display image after the emphasis display processing.

[0008] The display control method of this embodiment includes the steps of acquiring image data of an image taken outside the vehicle, detecting an object contained in the image, acquiring ranging data regarding the distance from the vehicle to the object, and when the distance is equal to or less than a threshold, setting a danger level for the object according to its size, performing highlighting processing on the image data according to the danger level, and outputting a highlight image after the highlighting processing.

[0009] The program of this embodiment causes a computer to execute the steps of acquiring image data of an image taken outside the vehicle, detecting an object contained in the image, acquiring ranging data regarding the distance from the vehicle to the object, setting a danger level for the object based on the size of the object when the distance is below a threshold, performing highlighting processing on the image data based on the danger level, and outputting a highlight image after the highlighting processing. Effect of the Invention

[0010] According to the present invention, it is possible to provide a display control device, a display control method, and a program that can effectively alert a user by appropriately setting a risk level. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing a vehicle according to an embodiment; [Diagram 2] FIG. 1 illustrates a display control device according to an embodiment. [Diagram 3] A color bar shows a display color according to the degree of danger. [Figure 4] 4 is a flowchart showing a display control method according to the first embodiment. [Diagram 5] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 8] 10 is a flowchart showing a display control method according to a second embodiment. [Figure 9] FIG. 11 is a schematic diagram for explaining display control according to the second embodiment. [Figure 10] FIG. 11 is a schematic diagram for explaining another display control method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. In addition, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0013] <Embodiment 1> 1 is a diagram showing a vehicle 1 according to a first embodiment. The vehicle 1 includes a sensor unit 2 and a display control device 100. In the following description, in order to distinguish the vehicle 1 from surrounding vehicles, the vehicle 1 may be referred to as the host vehicle and the surrounding vehicles may be referred to as surrounding vehicles.

[0014] The sensor unit 2 has at least one imaging device and captures an image of the outside of the vehicle. The sensor unit 2 outputs image data captured by the imaging device to the display control device 100. The sensor unit 2 has a distance measurement sensor that detects an obstacle. The sensor unit 2 outputs distance measurement data measured by the distance measurement sensor to the display control device 100.

[0015] The display control device 100 may be installed at any position of the vehicle 1. The display control device 100 may be connected to a CAN (Controller Area Network). The display control device 100 performs image processing to detect an object (obstacle) in an image captured by the sensor unit 2. The display control device 100 performs highlighting to call the user's attention. For example, the display control device 100 applies a highlight color such as red to an object included in an image. This makes it easier for a user such as a driver to recognize the presence of the object and to avoid contact. The display control device 100 will be described later.

[0016] In the following, the term "video" also means "video data showing a video" as an object of information processing. Similarly, in the following, the term "image" also means "image data showing an image" as an object of information processing. In addition, images and videos may be moving images or successive still images.

[0017] 2 is a block diagram showing a configuration of a display control device 100 according to the first embodiment and a display control system 10 having the display control device 100. The display control system 10 has a sensor unit 2, a display unit 30, an ECU (Electronic Control Unit) 40, and the display control device 100. The display control device 100 is connected to each of the sensor unit 2, the display unit 30, and the ECU 40 so as to be able to communicate with each other.

[0018] The display unit 30 is, for example, a display. The display unit 30 displays an image under the control of the display control device 100. The display unit 30 may include a speaker. In this case, the display unit 30 may output sound from the speaker. In addition, when the display control device 100 is mounted on the vehicle 1, the display unit 30 may be provided inside the vehicle 1 at a position where the driver of the vehicle can see while driving. In this case, the display unit 30 may be realized by an interface unit 108 described later. The display unit 30 may be a monitor of a car navigation system. In addition, the display unit 30 does not need to be an in-vehicle device permanently installed in the vehicle 1. The display unit 30 may be configured by, for example, a display of an information terminal such as a smartphone or tablet terminal owned by the user. Furthermore, a part of the display control process may be performed by a smartphone having a display unit.

[0019] The sensor unit 2 includes a front camera 21F and a rear camera 21R. When the vehicle 1 moves forward, the front camera 21F captures a front image of the vehicle 1. When the vehicle 1 moves backward, the rear camera 21R captures a rear image of the vehicle 1. The front camera 21F and the rear camera 21R generate shooting data at, for example, 30 frames per second (30 fps) and supply the generated shooting data to the display control device 100 every 1 / 30th of a second. The shooting data may be generated using, for example, a format such as H.264 or H.265. Alternatively, the sensor unit 2 may be an all-around camera that captures an image of the entire periphery of the vehicle 1.

[0020] The sensor unit 2 has a distance measurement sensor 22 for detecting the distance to an object in the vicinity of the vehicle 1. The distance measurement sensor 22 detects the distance from the vehicle to the object. The distance measurement sensor 22 is, for example, a LIDAR (Laser Imaging Detection and Ranging). Alternatively, the distance measurement sensor 22 may be an infrared camera, a stereo camera, a millimeter wave radar, or the like. Furthermore, the distance measurement sensor 22 may be configured by combining these sensors. Furthermore, the distance measurement sensor 22 is not limited to a single physical device. For example, the distance measurement sensor 22 may be configured by LIDARs arranged at multiple locations on the vehicle 1.

[0021] The distance measurement sensor 22 supplies distance measurement data relating to the distance from the vehicle to an object to the display control device 100. Furthermore, the distance measurement sensor 22 may detect the direction of an obstacle as well as the distance to the obstacle. That is, the distance measurement data may correspond to the distance to the object and the direction of the object. Here, the direction of the object is a direction based on the vehicle. Furthermore, the forward / backward / left / right directions of the object may be indicated by azimuth angles, and the up / down directions may be indicated by elevation / depression angles.

[0022] The ECU 40 is included in the vehicle 1 and is a part of the configuration that controls the vehicle 1. The display control device 100 and the ECU 40 are connected to each other so as to be able to communicate with each other via an in-vehicle network such as a CAN.

[0023] The display control device 100 includes a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF: Interface). The display control device 100 includes a driving information acquisition unit 110, an image data acquisition unit 121, an object detection unit 122, a distance measurement data acquisition unit 125, a risk setting unit 127, an emphasis display processing unit 128, and an output unit 129. The driving information acquisition unit 110 includes a traveling direction acquisition unit 111 and a speed information acquisition unit 113.

[0024] The control unit 102 is a processor such as a CPU (Central Processing Unit). The control unit 102 has a function as an arithmetic device that performs control processing and arithmetic processing. The storage unit 104 is a storage device such as a memory or a hard disk. The storage unit 104 is a read only memory (ROM) or a random access memory (RAM). The storage unit 104 has a function for storing a control program, an arithmetic program, etc., executed by the control unit 102 or as a function of the control unit 102. The storage unit 104 also has a function for temporarily storing processing data, etc.

[0025] The communication unit 106 performs processing required to communicate with the sensor unit 2 and the display unit 30. The communication unit 106 may also perform processing required to communicate with a CAN (not shown). The communication unit 106 may include a communication port. The interface unit 108 is, for example, a user interface (UI). The interface unit 108 has an input device such as a keyboard, a touch panel, or a mouse, and an output device such as a speaker.

[0026] Therefore, the display control device 100 has a function as a computer. In addition, the display control device 100 realizes components such as the traveling information acquisition unit 110, the image data acquisition unit 121, the object detection unit 122, the distance measurement data acquisition unit 125, the risk setting unit 127, the highlighting processing unit 128, and the output unit 129 by the processor of the control unit 102 executing the program stored in the storage unit 104. In addition, each component of the display control device 100 is not limited to being realized by software by a program, but may be realized by any combination of hardware and software. In addition, each component of the display control device 100 may be realized by using an integrated circuit programmable by a user, such as a field-programmable gate array (FPGA), a microcomputer, or a system on a chip (SOC). In this case, the integrated circuit may be used to realize a program composed of each of the above components. This is also true in other embodiments described later.

[0027] The travel information acquisition unit 110 acquires travel information related to the travel of the vehicle 1. The travel information acquisition unit 110 acquires various information from the ECU 40. The travel information acquisition unit 110 includes a traveling direction acquisition unit 111 and a speed information acquisition unit 113.

[0028] The traveling direction acquisition unit 111 acquires information indicating the traveling direction of the vehicle 1. For example, the traveling direction acquisition unit 111 detects whether the vehicle 1 is moving forward or backward based on the shift position of the vehicle 1. When the shift position of the vehicle 1 is in the reverse (R) position, the traveling direction acquisition unit 111 detects that the vehicle 1 is moving backward. Alternatively, when the shift lever of the vehicle 1 is in a position such as drive (D), the traveling direction acquisition unit 111 detects that the vehicle 1 is moving forward.

[0029] Furthermore, the traveling direction acquisition unit 111 may acquire steering angle information as the traveling direction. The traveling direction acquisition unit 111 acquires a signal from a CAN or the like to acquire steering angle information indicating the steering angle of the wheels of the vehicle 1. Note that the steering angle information includes information indicating the steering direction, such as right or left, in addition to information indicating the steering angle.

[0030] The speed information acquisition unit 113 acquires speed information indicating the speed of the vehicle 1. For example, the speed information acquisition unit 113 detects the vehicle speed based on a vehicle speed pulse from the ECU 40. The speed information acquisition unit 113 acquires speed information indicating a vehicle speed of, for example, 40 kilometers per hour. Alternatively, the speed information may be information in stages such as low speed, medium speed, high speed, etc.

[0031] The image data acquisition unit 121 acquires image data captured by the front camera 21F and the rear camera 21R. The image data acquisition unit 121 may switch the image to be acquired depending on the traveling direction acquired by the traveling direction acquisition unit 111. For example, when the vehicle 1 is moving forward, the image data acquisition unit 121 acquires a front image from the front camera 21F. When the vehicle 1 is moving backward, the image data acquisition unit 121 acquires a rear image from the rear camera 21R. Of course, the image data acquisition unit 121 may acquire both a front image and a rear image. In this case, the display control device 100 may switch the image to be processed depending on the traveling direction.

[0032] The object detection unit 122 detects objects included in an image. The object detection unit 122 recognizes objects in an image by performing image analysis on image data. For example, the object detection unit 122 can perform object recognition by comparing a preregistered pattern with an image. Furthermore, when multiple objects are included in an image, the object detection unit 122 detects each of the objects. Furthermore, the object detection unit 122 may identify the position coordinates and size of the object in the image.

[0033] Here, the object is a surrounding vehicle, a wall, a wheel stopper, an overpass, a bicycle, etc. The object may also include a person such as a pedestrian. Note that a publicly known method can be used for the object detection process in the object detection unit 122, and therefore a detailed description thereof will be omitted.

[0034] The distance measurement data acquisition unit 125 acquires distance measurement data relating to the distance to an object. If an image includes images of multiple objects, the distance measurement data acquisition unit 125 obtains the distance to each object. In other words, the distance from the vehicle to each object is calculated based on the distance measurement data from the distance measurement sensor 22.

[0035] The risk level setting unit 127 sets a risk level for an object. When an image includes images of multiple objects, the risk level setting unit 127 sets a risk level for each object. That is, a value indicating a risk level is assigned to each object.

[0036] Specifically, the risk setting unit 127 sets the risk depending on the possibility of the vehicle coming into contact with the object and the impact of the contact. The risk setting unit 127 sets the risk depending on the distance to the object. The risk setting unit 127 expresses the risk value on a scale of, for example, 10, with 10 being the highest risk and 0 being the lowest risk. The risk value is set to be higher for closer objects and lower for farther objects. Furthermore, for objects with an extremely low possibility of contact, the risk setting unit 127 may set the risk to 0, and for objects that are significantly off the path of the vehicle 1, the risk setting unit 127 may set the risk to 0.

[0037] Alternatively, the risk setting unit 127 sets the risk level according to the size of the object. The risk setting unit 127 sets a higher risk level for larger objects and a lower risk level for smaller objects. For example, the risk setting unit 127 can calculate the size of an object according to the size of the object image in the image and the distance to the object. The processing in the risk setting unit 127 will be described later. Here, the size of an object can be one or both of the size in the horizontal direction (left-right direction) and the size in the vertical direction (up-down direction).

[0038] The highlighting processing unit 128 performs highlighting processing to highlight objects in the image according to the degree of danger. For example, the highlighting processing unit 128 superimposes an emphasis color such as red on objects in the image that are highly dangerous. The highlighting processing unit 128 superimposes a non-emphasis color such as blue on objects in the image that are low in danger. The highlighting processing unit 128 does not perform highlighting processing on objects that are not dangerous, that is, objects that are unlikely to come into contact with the vehicle. The highlighting processing unit 128 superimposes a color that is emphasized more as the object becomes more dangerous on the objects in the image in order to display a warning to the user.

[0039] The danger level can be represented by a color bar of color distribution as shown in FIG. 3. The higher the danger level, the redder the color, that is, the longer the wavelength, and the lower the danger level, the bluer the color, that is, the shorter the wavelength. The highlighting processing unit 128 superimposes a color according to the danger level on an object in the image. For example, in the image, red is superimposed on an object with a high danger level, and blue is superimposed on an object with a low danger level. For an object with an intermediate danger level, green is superimposed on the object in the image. Then, the highlighting processing unit 128 generates a superimposed image in which a color according to the danger level is superimposed on the object in the image. The output unit 129 outputs the image data so that the superimposed image is displayed on the display unit 30 as a highlighting image.

[0040] In this way, by changing the overlay color according to the degree of danger, it is possible to appropriately alert the user. In other words, the higher the degree of danger, the more emphasized the display is, so that the user can easily recognize objects with high danger. By overlaying an emphasis color on an object in an image, a warning display can be performed. The user can drive while paying more attention to objects with high danger than objects with low danger. It is possible to appropriately alert the user to pay attention to objects. This contributes to safer driving.

[0041] The highlighting is not limited to the color distribution as shown in FIG. 3. For example, the danger level may be indicated by shading. For example, the same color may be applied to the image of the object, and only the shading may be changed according to the danger level. The highlighting processing unit 128 may perform the highlighting process by changing the color tone, shading, brightness, transparency, etc. of the object image. The highlighting processing unit 128 may also perform the highlighting process by applying a contour line to the object. In this case, the contour line is made thicker for objects with a higher danger level. In this way, the highlighting processing unit 128 performs the highlighting process such that the higher the danger level, the more emphasized the object is.

[0042] An example of the risk level setting process and highlighting process will be described with reference to Figs. 4 to 7. Fig. 4 is a flowchart showing the display control process according to this embodiment. Figs. 5 to 7 are diagrams for explaining the risk level when backing up and the highlighting process. Figs. 5 to 7 show an example of the vehicle 1 backing up into the parking space 300. Figs. 5 to 7 show, from the top, a superimposed image, a color scheme according to the risk level, a camera image, and a top view. As the vehicle 1 backs up at a low speed, the state changes in the order of Figs. 5, 6, and 7.

[0043] A rear wall 301 is disposed at the rear of the parking space 300, and side walls 302 are disposed on the left and right. In other words, the parking space 300 is surrounded on three sides by the rear wall 301 and the side walls 302. The vehicle retreats into the parking space 300 partitioned by the rear wall 301 and the side walls 302. Wheel stops 304 are disposed on the road surface of the parking space 300. The rear camera 21R captures an image of the parking space 300. The distance measurement sensor 22 detects the distance to the rear wall 301 and the side walls 302 as the distance to the object.

[0044] First, the image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R (S401). The object detection unit 122 detects an object included in the image (S402). For example, the object detection unit 122 detects the rear wall 301, the side wall 302, and the wheel stopper 304 included in the image. Distance measurement data measured by the distance measurement sensor 22 is acquired (S403). As a result, the distance from the vehicle 1 to the rear wall 301, the side wall 302, and the wheel stopper 304 are calculated. The risk level setting unit 127 detects the size of the object based on the distance measurement data and the image data (S404). As a result, the distance and size of each object are detected.

[0045] Next, the risk setting unit 127 judges whether the distance from the vehicle to the object is equal to or less than the threshold distance (S405). For example, the risk setting unit 127 compares the distance to the closest object with the threshold distance. When multiple objects are included in the image, the risk setting unit 127 judges whether the distance to the closest object is equal to or less than the threshold distance. Here, the distance to the rear wall 301 is compared with the threshold distance.

[0046] If the distance to the object is not equal to or less than the threshold distance (NO in S405), the risk setting unit 127 sets a risk level according to the distance (S406). In other words, the closer the distance to the object, the higher the risk level set by the risk setting unit 127 for the object.

[0047] If the distance to the object is equal to or less than the threshold distance (YES in S405), the risk setting unit 127 sets a risk level according to the size of the object (S407). That is, the larger the object, the higher the risk level set by the risk setting unit 127.

[0048] Then, the highlighting processing unit 128 performs highlighting processing according to the degree of danger (S408). That is, the highlighting processing unit 128 generates a superimposed image by superimposing a red color on an object in the image that has a high degree of danger. The output unit 129 outputs the superimposed image that has been subjected to the highlighting processing to the display unit 30 (S409). The processing is repeated while the vehicle 1 is moving backward. As a result, the superimposed image shown in Figs. 5 to 7 is displayed.

[0049] For example, in Figs. 5 and 6, the distance to the rear wall 301 is farther than the threshold distance. Therefore, as in S406, the risk setting unit 127 sets the risk level according to the distance to the object. As shown in Fig. 5, when the distance to the rear wall 301 is far, the risk level is low. Then, as the vehicle 1 moves backward, the vehicle 1 approaches the rear wall 301 and the side wall 302. Therefore, the risk level gradually increases. In the state shown in Fig. 6, the vehicle 1 is closer to the rear wall 301 and the side wall 302 than in the state shown in Fig. 5, so the risk level of the rear wall 301 and the side wall 302 is higher. Therefore, as the vehicle 1 moves backward from the position shown in Fig. 5 to the position shown in Fig. 6, the color of the superimposed image changes.

[0050] On the other hand, in Fig. 7, the distance to the rear wall 301 is equal to or shorter than the threshold distance. Therefore, as in S407, the risk level setting unit 127 sets the risk level according to the size of the object. Since the rear wall 301 and the side wall 302 are larger than a predetermined size, the highest risk level is set. In the superimposed image in Fig. 7, the color red, which indicates the highest risk level, is superimposed on the rear wall 301 and the side wall 302.

[0051] In this way, when the distance to the rear wall 301 becomes equal to or less than the threshold distance, the risk setting unit 127 sets the risk according to the size of the object. In this way, the user can easily recognize large objects with high risk. For example, if the risk changes only according to the distance, the same risk is set regardless of the size of the object. Therefore, it is difficult to draw attention to a large object that is far from the vehicle 1. In contrast, according to this embodiment, when an object is closer than the threshold distance, the risk setting unit 127 sets the risk according to the size of the object. If the vehicle 1 comes into contact with a large object, the impact is large. In this embodiment, the risk of a large object can be set high, so that attention can be drawn to a large object that is far from the vehicle 1.

[0052] In this way, when the vehicle 1 approaches the object to within the threshold distance, the risk setting unit 127 sets the risk according to the size of the object. Therefore, the user can appropriately recognize large objects with high risk. The user, the driver, can decelerate and park appropriately. This can effectively assist driving. Also, when the vehicle 1 is away from the object by more than the threshold distance, the risk setting unit 127 sets the risk according to the distance to the object. Therefore, the risk changes according to the approach to the object, so the user can recognize the distance to the object. In this way, the risk can be appropriately set according to the situation, so that the user can be effectively alerted.

[0053] In step S407, if the size of an object closer than the threshold distance is smaller than a predetermined size, the risk setting unit 127 may set the risk level according to the distance, as in step S405. That is, as the distance becomes closer, the risk setting unit 127 increases the risk level of a small-sized object. In other words, if the size of an object closer than the threshold distance is smaller than a predetermined size, the risk level may be set to a medium level. Furthermore, the size of an object may be divided into three or more stages, and the risk level may be set in multiple stages.

[0054] Furthermore, the risk level setting unit 127 may exclude objects smaller than a predetermined size from comparison with the threshold distance in S405. In other words, the risk level setting unit 127 compares the distance to the object closest to the vehicle 1 among objects equal to or larger than a certain size with the threshold distance. Specifically, the wheel stopper 304 is an object smaller than the predetermined size, and is therefore not subject to comparison in step S404.

[0055] Furthermore, the risk level may be set to 0 for objects that are unlikely or have no possibility of contacting the vehicle 1. For example, the risk level setting unit 127 calculates height information indicating the height of the object as the size of the object. If the height of the object is less than the minimum ground clearance of the vehicle 1, the possibility of contacting the vehicle 1 is low. For example, the height of the wheel stopper 304 is lower than the minimum ground clearance of the vehicle 1, and therefore the risk level is low. Therefore, the wheel stopper 304 may be excluded from the comparison process in step S404.

[0056] The threshold distance may be determined according to the size of the vehicle 1. For example, a 4-ton truck has a long overall length of about 8 m. Therefore, the threshold distance from the object to the vehicle is set to about 4 m. Also, for example, a compact or ordinary passenger car has a short overall length of about 4.70 m or less. Therefore, the threshold distance from the object to the vehicle can be set to about 2 m. Alternatively, if the depth of the parking space 300 is about 10% longer than the overall length of the vehicle, the threshold distance may be set to less than the overall length of the vehicle. The threshold distance may also be determined according to the width of the vehicle 1.

[0057] <Embodiment 2> In the second embodiment, the risk setting unit 127 performs risk setting processing in accordance with the driving information. More specifically, the setting processing performed by the risk setting unit 127 is switched depending on the traveling direction. For example, when the vehicle 1 is moving forward, the risk setting unit 127 performs a first setting processing. When the vehicle 1 is moving backward, the risk setting unit 127 performs a second setting processing different from the first setting processing. Note that the configurations of the vehicle 1 and the display control system 10 are similar to those in Figs. 1 and 2, etc., and therefore description thereof will be omitted.

[0058] 8 is a flowchart showing the display control process in this embodiment. First, the driving information acquisition unit 110 acquires driving information from the ECU 40 (S801). That is, the traveling direction acquisition unit 111 acquires the traveling direction. Furthermore, the speed information acquisition unit 113 may acquire speed information.

[0059] The image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R (S802). The object detection unit 122 detects an object included in the image (S803). Distance measurement data measured by the distance measurement sensor 22 is acquired (S804). The risk level setting unit 127 detects the size of the object based on the distance measurement data and image data (S805). The processes of steps S802 to S805 are basically similar to the processes of steps S401 to S404 in the first embodiment, and therefore a description thereof will be omitted. However, step S802 differs from step S401 in that the image data acquisition unit 121 acquires image data of a forward image captured by the front camera 21F and a rear image captured by the rear camera 21R.

[0060] Next, it is determined whether the vehicle 1 is moving forward (S806). If the vehicle 1 is moving forward (YES in S806), the risk setting unit 127 sets the risk level by a first setting process (S807). The first setting process will be described later.

[0061] If the vehicle 1 is not moving forward (NO in S806), the risk setting unit 127 sets the risk level by a second setting process (S808). Note that the second setting process is different from the first setting process. If the vehicle 1 is moving backward, the risk setting unit 127 sets the risk level by a second setting process different from the first setting process. For the second setting process, for example, the method shown in the first embodiment can be used. That is, in the second setting process, the risk setting unit 127 sets the risk level according to the size and distance of the object.

[0062] Then, the highlighting processing unit 128 performs highlighting processing according to the degree of danger (S809). That is, the highlighting processing unit 128 generates a superimposed image by superimposing a red color on an object in the image that has a high degree of danger. The output unit 129 outputs the superimposed image that has been subjected to the highlighting processing to the display unit 30 (S810).

[0063] An example of the first setting process will be described below with reference to Fig. 9. Fig. 9 shows a camera image of a road 310 ahead of a vehicle and a superimposed image in which the camera image has been subjected to highlighting processing. A viaduct 311 is provided ahead of the vehicle. That is, the vehicle 1 is traveling on the road 310 that passes under the viaduct 311.

[0064] Here, it is assumed that the vehicle height of the vehicle 1 is such that the vehicle 1 cannot pass under the viaduct 311. For example, it is assumed that the viaduct 311 is 1.9 m above the ground, and the vehicle height is higher than 1.9 m.

[0065] The object detection unit 122 detects the viaduct 311 ahead in the traveling direction as an object. The risk setting unit 127 detects the ground height of the viaduct 311. In other words, it detects the clearance height from the road to the viaduct 311. The risk setting unit 127 compares the height of the vehicle 1 with the ground height of the viaduct 311. Then, the risk setting unit 127 sets the risk level according to the comparison result.

[0066] If the vehicle 1 is at a height where it cannot pass under the viaduct 311, the danger level setting unit 127 sets a high danger level for the viaduct 311. The emphasis display processing unit 128 performs emphasis display according to the danger level. The emphasis display processing unit 128 superimposes a red emphasis color corresponding to a high danger level on the viaduct 311 in the image as a warning display 312. Since there is a high possibility that the vehicle 1 will come into contact with the viaduct 311, the display unit 30 can display the warning display 312 to the user. Although the warning display 312 in the drawing is superimposed only on the viaduct 311 at the top in the traveling direction, it may be superimposed on the entire surface of the viaduct 311.

[0067] In general, the vehicle speed during forward travel is faster than the vehicle speed during reverse travel. For example, the vehicle 1 travels at a relatively slow speed during reverse travel, for example, for parking in a garage. When the vehicle is reversed for parking, the vehicle speed is slow, so the user is likely to notice the height restriction. In contrast, the vehicle speed is fast during forward travel, so the user may overlook the height restriction. Therefore, during forward travel, the risk level setting unit 127 detects the height of the object and compares the vehicle height with the height of the object. In this way, by switching the risk level setting process according to the traveling direction, the risk level can be appropriately set. The user, who is the driver, can be more appropriately alerted.

[0068] In the above description, an example of passing through the road 310 under the viaduct 311 has been described, but the processing of this embodiment can be used for other than the viaduct 311. For example, parking lots such as multi-storey parking lots, indoor parking lots, and mechanical parking lots may have height restrictions. When entering such parking lots with height restrictions, the processing of this embodiment can be applied. The risk setting unit 127 sets a risk level for the object according to the clearance limit height of the viaduct 311 and the height inside the building. The risk setting unit 127 compares the height of the object with the vehicle height of the vehicle 1 to determine whether the vehicle 1 can pass under the object. The risk setting unit 127 sets a risk level according to the determination result.

[0069] A margin may be provided for the comparison result between the vehicle height of the vehicle 1 and the clearance height of the object. For example, in the above description, a warning is displayed when the vehicle height of the vehicle 1 is higher than the clearance height limit of the viaduct 311, but a warning may be displayed if the vehicle height of the vehicle 1 is approximately the same as the clearance height limit of the viaduct 311. The risk setting unit 127 may increase the risk when the difference between the vehicle height of the vehicle 1 and the ground height of the object is equal to or less than a predetermined value. Specifically, it becomes possible to provide a height margin in consideration of steps in the road surface, etc.

[0070] In the first setting process, the danger level may be set according to the distance to the object. For example, weighting may be performed so that the closer the distance, the higher the danger level. Furthermore, if the distance to the object is farther than a threshold distance, the danger level may be set according to the distance as in the first embodiment. Then, the height may be compared at the timing when the distance to the object becomes equal to or less than the threshold distance. In this way, the user can be more appropriately warned. For example, when the distance is far, the accuracy of height detection is low.

[0071] Also, the risk level setting unit 127 may set the risk level based on the speed information. For example, the risk level setting unit 127 may adjust the timing of displaying a warning according to the speed information and the distance to the object. When the vehicle 1 is traveling at high speed, the time from object detection to arrival at the viaduct 311 is short. Therefore, it is preferable to display a warning to the user early. The faster the speed, the earlier the timing of displaying the highlight color is preferably set. It becomes possible to display a warning at an appropriate timing.

[0072] The risk setting unit 127 may measure at least one of the height of the object from the road surface and the left-right position of the object. The risk setting unit 127 determines whether the object is likely to collide with the vehicle 1 based on the measurement result. The risk setting unit 127 sets a risk level for the object depending on the determination result. This process will be described with reference to FIG. 10. Here, an example of parking forward or backward into the parking space 320 will be described.

[0073] Parking space 320 is provided with wheel stoppers 324. Furthermore, nearby vehicle 321 is parked to the side of parking space 320. Also, nearby vehicle 322 is parked at the back of parking space 320. The distances to nearby vehicles 321, 322 are closer to vehicle 1 than the threshold distance. Their sizes in the left-right direction (horizontal direction) are approximately the same as vehicle 1. Furthermore, there is a pedestrian 325 at the far side of parking space 320. The distance to pedestrian 325 is farther than the threshold distance.

[0074] Based on the distance measurement data, the risk level setting unit 127 measures the left-right positions of the surrounding vehicles 321, 322. Based on the measurement results of the left-right positions, the risk level setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with the surrounding vehicles 321, 322. The risk level setting unit 127 sets the risk levels of the surrounding vehicles 321, 322 according to the determination results.

[0075] For example, since nearby vehicle 321 is located immediately next to parking space 320, risk setting unit 127 determines that there is a possibility of collision. Since nearby vehicle 322 is located ahead of parking space 320 in the traveling direction, risk setting unit 127 determines that there is a possibility of collision. Therefore, risk setting unit 127 sets a high risk level for nearby vehicles 321, 322. Note that "ahead in the traveling direction" refers to the rear of vehicle 1 when vehicle 1 is moving backward, and the front of vehicle 1 when vehicle 1 is moving forward.

[0076] In the superimposed image, warning indications 331 and 332 are superimposed on the surrounding vehicles 321 and 322, respectively. In other words, a highlighting color such as red is superimposed on the surrounding vehicles 321 and 322 as the warning indications 331 and 332.

[0077] In the left-right direction (horizontal direction), the pedestrian 325 is located away from the front of the vehicle 1 in the traveling direction. The risk level setting unit 127 measures the position of the pedestrian 325 in the left-right direction based on the distance measurement data. The risk level setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with the pedestrian 325 based on the measurement result of the position in the left-right direction. The risk level setting unit 127 sets the risk level of the pedestrian 325 in accordance with the determination result. The risk level setting unit 127 determines that the possibility of colliding with the pedestrian 325 is extremely low. A low risk level is set for the pedestrian 325. The risk level setting unit 127 sets the risk level of the pedestrian 325 to 0, for example. Therefore, no warning is displayed for the pedestrian 325 in the superimposed image.

[0078] In this way, the risk setting unit 127 may set the risk level according to the left-right position and size (width) of the object. Also, it may be determined whether there is a possibility of collision with the vehicle 1 according to the steering angle, the width, and the left-right (lateral) position of the object. Then, when the possibility of contact with the vehicle 1 is low, the risk setting unit 127 may set the risk level low. Also, when there is no possibility of contact, the risk setting unit 127 may set the risk level to 0. Note that, when determining whether or not there will be a collision with the vehicle 1 based on the left-right position, steering angle information of the vehicle 1 may be taken into consideration. That is, the risk setting unit 127 may determine whether or not there is an object ahead in the traveling direction according to the steering angle, for example, ahead to the left or right.

[0079] The risk setting unit 127 measures the height of the wheel stopper 324 based on the distance measurement data. The risk setting unit 127 determines whether or not the object will collide with the wheel stopper 324 based on the height measurement result. The risk setting unit 127 sets a risk level for the object depending on the determination result. The wheel stopper 324 is lower than the minimum ground clearance of the vehicle 1. Therefore, the risk setting unit 127 determines that there is no possibility of collision with the wheel stopper 324, and sets the risk level to 0. Therefore, no warning is displayed for the wheel stopper 324 in the superimposed image.

[0080] In this way, the danger level setting unit 127 can set the danger level more appropriately. Therefore, it becomes possible to effectively alert the user. Furthermore, the display control device 100 may output a voice message or a warning sound for alerting the user from a speaker based on the danger level. In other words, the output unit 129 may generate voice data according to the danger level.

[0081] Also, the program for executing the above display control method can be stored using various types of non-transitory computer readable media and supplied to the computer. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (random access memory)). Also, the program may be supplied to the computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path. [Explanation of symbols]

[0082] 1 vehicle 2 Sensor unit 21F Front Camera 21R Rear camera 22 Distance sensor 30 Display section 40 ECU 100 Display control device 110 Driving information acquisition unit 111 Traveling direction acquisition unit 113 Speed ​​information acquisition section 121 Image data acquisition unit 122 Object detection unit 125 Distance measurement data acquisition unit 127 Risk Level Setting Section 128 Highlighting Processing Unit 129 Output section 301 Rear wall 302 Side wall 304 Wheel chock 310 Road 311 viaduct 312 Warning display 320 Parking spaces 321, 322 surrounding vehicles 324 Wheel chock 325 Pedestrians 331, 332 Warning display

Claims

1. an image data acquisition unit that acquires image data of an image captured outside the vehicle; an object detection unit that detects an object included in the image; a distance measurement data acquisition unit that acquires distance measurement data relating to a distance from the host vehicle to the object; a danger level setting unit that sets a danger level for the object in accordance with a size of the object when the distance becomes equal to or less than a threshold; an emphasis display processing unit that performs emphasis display processing on image data according to the degree of risk; an output unit that outputs a highlighted image that has been subjected to a highlighting process; Equipped with The threshold distance is set according to the size of the host vehicle. Display control device.

2. The display control device according to claim 1 , wherein the risk level setting unit sets a risk level for the object according to the distance to the object when the distance is greater than a threshold value.

3. The risk level setting unit is measuring at least one of a height of the object from a road surface and a lateral position of the object; determining whether the object will collide with the host vehicle based on the measurement result; The danger level of the object is set according to the judgment result. The display control device according to claim 1 or 2.

4. acquiring image data of an image captured outside the vehicle; detecting an object in the image; acquiring distance measurement data relating to a distance from the host vehicle to the object; When the distance is equal to or less than a threshold, setting a danger level for the object according to a size of the object; performing a highlighting process on the image data according to the degree of risk; outputting a highlighted image on which the highlighting process has been performed; Equipped with The threshold distance is set according to the size of the host vehicle. Display control method.

5. acquiring image data of an image captured outside the vehicle; detecting an object in the image; acquiring distance measurement data relating to a distance from the host vehicle to the object; When the distance is equal to or less than a threshold, setting a danger level for the object according to a size of the object; performing a highlighting process on the image data according to the degree of risk; outputting a highlighted image on which the highlighting process has been performed; A program for causing a computer to execute the following: The threshold distance is set according to the size of the host vehicle. program.

Images