Display control device, display control method and program

Abstract

To provide a display control device, method, and program that enable appropriate attention calling.SOLUTION: A display control device according to an embodiment includes: an image data acquisition unit 121 that acquires image data on an image captured by imaging an exterior of an own vehicle; an object detection unit 122 that detects an object included in the image; a ranging data acquisition unit 125 that acquires ranging data concerning a distance from the own vehicle to the object; a risk setting unit 127 that when a depth of a parking space is approximately 10 percent larger than a total length of the own vehicle, sets a distance of a threshold value to be less than the total length of the own vehicle, and when the distance is less than or equal to the threshold value, sets a risk for the object in accordance with the size of the object; a highlighted display processing unit 128 that performs highlighted display processing in accordance with the risk for the image data; and an output unit 129 that outputs a highlighted image having undergone the highlighted display processing.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a display control device, a display control method, and a program. [Background technology]

[0002] Patent Document 1 describes a surroundings monitoring device having a distance measurement detector that detects the distance from a vehicle to an object. The surroundings monitoring device of Patent Document 1 captures an image of an object from an imaging unit provided in a vehicle. The surroundings monitoring device has a camera that superimposes the color of the object and a color corresponding to the height of the object. It is displayed.

[0003] Patent Document 2 describes an obstacle detection unit that detects the distance between the vehicle and an obstacle, and a Patent Document 2 discloses a vehicle surroundings visual confirmation device that includes an imaging unit that captures images of the scenery. When an obstacle is detected, a graphic indicating the direction of the obstacle is displayed on the captured image. This device changes the display color or size of the figure depending on the distance to the obstacle. It is being done. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-97843 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-311222 Summary of the Invention [Problem to be solved by the invention]

[0005] In such a device, it is desirable to appropriately warn the driver. When multiple obstacles are involved, it is preferable to draw attention to the more dangerous object. However, if multiple objects exist around the vehicle, they may be displayed according to distance or height. Changing the display color may result in cluttered display. For example, if an object with low risk is displayed in a different color, If it is emphasized, it will not be possible to effectively draw attention to highly dangerous objects. It ends up like this.

[0006] In view of the above-mentioned problems, the present invention provides a method for effectively providing a user with a warning by appropriately setting a risk level. The present invention aims to provide a display control device, a display control method, and a program that can draw attention. The target. [Means for solving the problem]

[0007] The display control device according to this embodiment acquires image data of an image captured outside the vehicle. an image data acquisition unit that acquires an image of the vehicle; an object detection unit that detects an object included in the image; a distance measurement data acquisition unit that acquires distance measurement data relating to the distance to the object from the If the threshold value is reached or less, a risk level is set for the object according to the size of the object. a degree setting unit, and a highlighting processing unit that performs highlighting processing on image data according to the degree of risk. and an output unit that outputs a highlighted image that has been subjected to the highlighting process, When the depth of the parking space is about 10% longer than the overall length of the vehicle, The period shall be less than 1 year.

[0008] The display control method according to the present embodiment acquires image data of an image of the outside of the vehicle. a step of detecting an object included in the image; and a step of detecting the object from the host vehicle. acquiring distance measurement data relating to a distance to the body; and when the distance is equal to or less than a threshold value, setting a danger level for the object in accordance with the size of the object; a step of performing a highlighting process on the image data according to the highlighting process; and outputting a display image of the parking space, wherein the threshold distance is set based on the depth of the parking space. If the distance is approximately 10% longer than the overall length of the vehicle, it will be considered to be less than the overall length of the vehicle.

[0009] The program according to this embodiment acquires image data of an image taken of the outside of the vehicle. a step of detecting an object included in the image; and a step of detecting the object from the host vehicle. a step of acquiring distance measurement data relating to a distance to the parking space; When the distance is about 10% longer than the overall length of the vehicle, the threshold distance is set to be less than the overall length of the vehicle. When the threshold value is reached or less, a risk level is set for the object according to the size of the object. a step of performing a highlighting process on the image data according to the risk level; and outputting a highlighted image that has undergone the highlighting process. do. [Effects of the Invention]

[0010] According to the present invention, by appropriately setting the risk level, it is possible to effectively warn the user. It is possible to provide a display control device, a display control method, and a program that can perform the above-mentioned operations. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a vehicle according to an embodiment. [Figure 2] 1 is a diagram illustrating a display control device according to an embodiment; [Figure 3] This is a color bar that indicates the display color according to the degree of danger. [Figure 4]3 is a flowchart showing a display control method according to the first embodiment. [Figure 5] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 8] 10 is a flowchart showing a display control method according to a second embodiment. [Figure 9] FIG. 10 is a schematic diagram for explaining display control according to the second embodiment. [Figure 10] FIG. 10 is a schematic diagram for explaining another display control method. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below through embodiments of the invention. The present invention is not limited to the following embodiments. Furthermore, all of the configurations described in the embodiments do not solve the problems. It is not necessarily essential to solve the problem. The drawings and figures have been omitted or simplified as appropriate. The same reference numerals are used, and redundant explanations are omitted as necessary.

[0013] <First Embodiment> FIG. 1 is a diagram illustrating a vehicle 1 according to a first embodiment. The vehicle 1 includes a sensor unit 2 and In the following description, the vehicle 1 and its surrounding vehicles are In order to distinguish between the two, vehicle 1 may be referred to as the subject vehicle and the surrounding vehicles as the surrounding vehicles. do.

[0014] The sensor unit 2 has at least one imaging device and captures an image of the outside of the vehicle. The sensor unit 2 outputs image data captured by the imaging device to the display control device 100. The sensor unit 2 has a distance measuring sensor that detects obstacles. 2 outputs distance measurement data measured by the distance measurement sensor to the display control device 100.

[0015] The display control device 100 can be installed at any position in the vehicle 1. The display control device 100 The display control device 100 can be connected to a CAN (Controller Area Network). In the image captured by unit 2, image processing is performed to detect objects (obstacles). The display control device 100 performs highlighting to draw the user's attention. For example, The display control device 100 applies an emphasis color such as red to the objects included in the image. This makes it easier for drivers and other users to recognize the presence of objects and avoid contact. The display control device 100 will be described later.

[0016] In the following, the term "image" refers to an image that represents an image as a processing target in information processing. Similarly, hereinafter, the term "image" refers to the object of information processing. It also means "image data showing an image." Note that images and videos may be moving images, It may also be a still image.

[0017] FIG. 2 shows the display control device 100 according to the first embodiment and a display device having the display control device 100. 1 is a block diagram showing a configuration of a display control system 10. The display control system 10 is A unit 2, a display unit 30, an ECU (Electronic Control Unit) 40, and a display control device The display control device 100 includes a sensor unit 2, a display unit 30, an ECU 40, and a are communicatively connected to each of the above.

[0018] The display unit 30 is, for example, a display. The display unit 30 may also include a speaker. The display unit 30 may output a sound from a speaker. When the display unit 30 is installed, the display unit 30 is installed inside the vehicle 1 at a position where the driver of the vehicle can see the display unit 30 while driving. In this case, the display unit 30 can be controlled by the interface unit 108 described later. The display unit 30 may be a monitor of a car navigation system. The display unit 30 does not need to be an in-vehicle device permanently installed in the vehicle 1. For example, the display of a user's smartphone, tablet, or other information terminal. Furthermore, a part of the display control process may be performed by a smartphone having a display unit. This may be done by a phone or the like.

[0019] The sensor unit 2 includes a front camera 21F and a rear camera 21R. When the vehicle 1 is moving forward, the front camera 21F captures an image of the front of the vehicle 1. In this case, the rear camera 21R captures an image behind the vehicle 1. The camera 21R generates, for example, 30 frames per second (30 fps) of photographic data and The shadow data is supplied to the display control device 100 every 1 / 30 seconds. It may be generated using a format such as H.264 or H.265. The unit 2 may be a 360° camera that captures images of the entire periphery of the vehicle 1.

[0020] The sensor unit 2 is a distance measurement sensor for detecting the distance to an object in the vicinity of the vehicle 1. The distance measurement sensor 22 detects the distance from the vehicle to an object. The sensor 22 is, for example, a LIDAR (Laser Imaging Detection and Ranging) sensor. The distance measuring sensor 22 may be an infrared camera, a stereo camera, a millimeter wave radar, or the like. Furthermore, the distance measurement sensor 22 may be configured by combining these sensors. Furthermore, the distance measurement sensor 22 is not limited to a single physical device. The distance measurement sensor 22 may be configured by LIDARs arranged at several locations.

[0021] The distance measurement sensor 22 transmits distance measurement data relating to the distance from the vehicle to the object to the display control device 100. Furthermore, the distance sensor 22 detects the distance to the obstacle as well as the direction of the obstacle. In other words, the distance measurement data may be a data in which the distance to an object is associated with the direction of the object. Here, the direction of the object is the direction based on the vehicle itself. The right direction may be indicated by an azimuth angle, and the up and down directions may be indicated by an elevation angle.

[0022] The ECU 40 is a part of the configuration of the vehicle 1 that controls the vehicle 1. The display control device 100 and the ECU 40 are connected to each other via an in-vehicle network such as a CAN. and connect so that communication is possible.

[0023] The display control device 100 includes a control unit 102, a storage unit 104, a communication unit 106, and an interface. The display control device 100 is provided with a display unit 108 (IF: Interface). unit 110, image data acquisition unit 121, object detection unit 122, and distance measurement data acquisition unit 125 The device includes a risk level setting unit 127, a highlighting processing unit 128, and an output unit 129. The line information acquisition unit 110 includes a traveling direction acquisition unit 111 and a speed information acquisition unit 113 .

[0024] The control unit 102 is, for example, a processor such as a CPU (Central Processing Unit). The control unit 102 functions as a calculation device that performs control processing, calculation processing, etc. 104 is a storage device such as a memory or a hard disk. For example, it is a ROM (Read Only Memory) or a RAM (Random Access Memory). The memory unit 104 stores control information executed by the control unit 102 or as a function of the control unit 102. The memory unit 104 has a function for storing programs and calculation programs. has a function for temporarily storing processing data and the like.

[0025] The communication unit 106 performs the processing required to communicate with the sensor unit 2 and the display unit 30. The communication unit 106 also performs processing necessary for communication with a CAN (not shown). The communication unit 106 may include a communication port. The interface unit 108 may include, for example, The interface unit 108 is a user interface (UI). It has an input device such as a keyboard, touch panel, or mouse, and an output device such as a speaker.

[0026] Therefore, the display control device 100 has the function of a computer. The control device 100 executes a program stored in the storage unit 104 by the processor of the control unit 102. By executing this, the travel information acquisition unit 110, the image data acquisition unit 121, the object detection unit 1 22, distance measurement data acquisition unit 125, risk level setting unit 127, highlighting processing unit 128, output unit 1 29. Each component of the display control device 100 is realized by a program. It is not limited to being realized by software using a system, but also includes hardware and software. It may be realized by a combination of any of the above. The components are, for example, FPGA (field-programmable gate array), microcomputer, or SOC (SOC). The present invention may be implemented using a user-programmable integrated circuit, such as a programmable system on a chip. In this case, the integrated circuit is used to execute a program that is composed of the above components. This also applies to other embodiments described later.

[0027] The travel information acquisition unit 110 acquires travel information relating to the travel of the vehicle 1. The travel information acquisition unit 110 acquires various information from the ECU 40. The device includes a speed information acquisition unit 111 and a speed information acquisition unit 113.

[0028] The traveling direction acquisition unit 111 acquires information indicating the traveling direction of the vehicle 1. For example, The acquisition unit 111 detects whether the vehicle 1 is moving forward or backward based on the shift position of the vehicle 1. If the shift position of vehicle 1 is in the reverse (R) position, vehicle 1 will Or, the shift lever of vehicle 1 is in a position such as drive (D). In this case, it is detected that the vehicle 1 is moving forward.

[0029] Furthermore, the traveling direction acquisition unit 111 may acquire steering angle information as the traveling direction. The direction acquisition unit 111 acquires a signal from the CAN or the like to indicate the steering angle of the wheels of the vehicle 1. The steering angle information includes not only the information indicating the steering angle but also the information indicating whether the vehicle is turned right or left. The information also includes information indicating the steering direction.

[0030] The speed information acquisition unit 113 acquires speed information indicating the speed of the vehicle 1. For example, the speed information The acquisition unit 113 detects the vehicle speed based on the vehicle speed pulse from the ECU 40. The acquiring unit 113 acquires speed information indicating a vehicle speed of, for example, 40 kilometers per hour. Alternatively, the speed information may be information in stages such as low speed, medium speed, high speed, etc.

[0031] The image data acquisition unit 121 acquires image data captured by the front camera 21F and the rear camera 21R. In addition, the image data is acquired according to the traveling direction acquired by the traveling direction acquisition unit 111. The data acquisition unit 121 may switch the image to be acquired. For example, when the vehicle 1 is moving forward, When the vehicle 1 is backing up, the image data acquisition unit 121 acquires a front image from the front camera 21F. In this case, the image data acquisition unit 121 acquires a rear image from the rear camera 21R. The image data acquisition unit 121 may acquire both a forward image and a rearward image. The display control device 100 can simply switch the image to be processed depending on the direction of travel.

[0032] The object detection unit 122 detects an object included in an image. For example, the object detection unit 122 performs image analysis on the image data to recognize objects in the image. can perform object recognition by comparing images with pre-registered patterns. Furthermore, when multiple objects are included in an image, the object detection unit 122 detects each object. Furthermore, the object detection unit 122 identifies the position coordinates and size of the object in the image. That's fine.

[0033] Here, the objects are surrounding vehicles, walls, wheel stops, overpasses, bicycles, etc. The object detection unit 122 may also include people such as pedestrians. Since a known method can be used for the process, a detailed description thereof will be omitted.

[0034] The distance measurement data acquisition unit 125 acquires distance measurement data relating to the distance to an object. When the image contains images of multiple objects, the distance measurement data acquisition unit 125 acquires the distance to each object. That is, based on the distance data from the distance sensor 22, the distance from the vehicle to each object is calculated. Calculate the distance to.

[0035] The danger level setting unit 127 sets a danger level for an object. When the object is included in the object, the danger level setting unit 127 sets a danger level for each object. Each is assigned a value indicating the degree of danger.

[0036] Specifically, the risk setting unit 127 determines the possibility that the vehicle will come into contact with an object, and the risk of collision when the vehicle comes into contact with an object. The danger level setting unit 127 sets the danger level according to the distance to the object. The risk setting unit 127 expresses the value of the risk on a scale of 1 to 10, with 10 being The highest risk level is 0, and the lowest risk level is 0. The closer the object, the higher the risk level. The more distant the object, the lower the danger level. For example, the risk setting unit 127 may set the risk level to 0. For objects that are significantly out of the range, the danger level may be set to 0.

[0037] Alternatively, the risk level setting unit 127 sets the risk level according to the size of the object. The determining unit 127 determines whether the larger the object, the higher the danger level, and whether the smaller the object, the lower the danger level. For example, the risk level setting unit 127 may set the risk level based on the size of the object image in the image and the The size of the object can be calculated according to the distance to the body. The process of this will be described later. Here, the size of the object is determined by the size in the horizontal direction (left and right direction). This can be either the size or the vertical size (up and down direction), or both.

[0038] The highlighting processing unit 128 performs highlighting processing to highlight an object in the image according to the degree of danger. For example, the highlighting processing unit 128 highlights an object with a high degree of danger in red or the like. The highlighting processor 128 superimposes colors on the objects in the image. The highlighting processor 128 superimposes non-highlighted colors on objects in the image. Therefore, highlighting processing is not performed for objects that are unlikely to come into contact with the vehicle. The unit 128 displays a warning to the user by highlighting the object in a color that is more prominent as the object becomes more dangerous. is superimposed on the object in the image.

[0039] The risk level can be expressed by a color bar with a color distribution as shown in Figure 3. Indeed, the reddish color, or the longer wavelength side, and the lower the risk, the bluer, or the shorter wavelength side. The highlighting processor 128 superimposes a color corresponding to the degree of danger on an object in the image. In the image, red is superimposed on objects that are highly dangerous, and blue is superimposed on objects that are less dangerous. For objects with a medium risk level, green is superimposed on the object in the image. The display processor 128 generates a superimposed image in which a color corresponding to the danger level is superimposed on an object in the image. The output unit 129 outputs the image so that the superimposed image is displayed on the display unit 30 as a highlighted image. Output the data.

[0040] By changing the overlay color according to the level of danger, users can be appropriately alerted. In other words, the higher the risk level, the more emphasized it is displayed, so the user can By superimposing a highlight color on the object in the image, it is possible to easily recognize the object with high visibility. Users can be more likely to pay attention to high-risk objects than low-risk objects. The user can be appropriately warned to pay attention to objects. This contributes to safer driving.

[0041] The highlighting is not limited to the color distribution shown in Figure 3. For example, the degree of danger can be displayed in shades of gray. For example, the image of the object may be given the same color and only the shading may be used depending on the degree of danger. The highlighting processing unit 128 may change the color tone, shading, brightness, transparency, etc. of the object image. The highlighting processing unit 128 may perform highlighting processing by adding a A highlighting process may be performed by adding a contour line. In this case, the more dangerous an object is, the more the circle is drawn. In this way, the highlighting processor 128 makes the outline thicker as the risk level increases. Performs highlighting processing.

[0042] An example of the risk level setting process and highlighting process will be described with reference to FIGS. 4 to 7. 5 to 7 are flowcharts showing the display control process according to the present embodiment. 5 to 7 are diagrams illustrating the degree of danger in a parking space and the process of highlighting the danger. 5 to 7 show an example of parking in reverse at Pace 300. The color scheme according to the risk level, the camera image, and the top view are shown. As the vehicle moves backward, the state changes in the order shown in Figure 5, Figure 6, and Figure 7.

[0043] A rear wall 301 is disposed at the rear of the parking space 300, and side walls 302 are disposed on the left and right sides. That is, the parking space 300 is surrounded on three sides by a rear wall 301 and a side wall 302. Then, the vehicle backs up into a parking space 300 that is partitioned by a rear wall 301 and a side wall 302. Wheel stops 304 are provided on the road surface of the parking space 300. 1R captures an image of the parking space 300. The distance measurement sensor 22 captures the image of the rear wall 301 and the side wall 30 The distance to 2 is detected as the distance to the object.

[0044] First, the image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R. The object detection unit 122 detects an object included in the image (S402). For example, the object detection unit 122 detects the rear wall 301, the side wall 302, and the wheel stopper included in the image. 304 are detected. The distance measurement data measured by the distance measurement sensor 22 is acquired (S40 3) This reduces the distance from the vehicle 1 to the rear wall 301, the side wall 302, and the wheel stopper 304. The risk level setting unit 127 calculates the distance based on the distance measurement data and the image data. Then, the size of the object is detected (S404). As a result, the distance and size of each object are detected. will be done.

[0045] Next, the risk level setting unit 127 determines whether the distance from the vehicle to the object is equal to or less than a threshold distance. The risk level setting unit 127, for example, compares the distance to the closest object with a threshold distance. When multiple objects are included in an image, the risk setting unit 127 compares the risk up to the nearest object. Here, it is determined whether the distance to the rear wall 301 is equal to or less than the threshold distance. The distance is compared with a threshold distance.

[0046] If the distance to the object is not equal to or less than the threshold distance (NO in S405), the risk level setting unit 127 The degree of danger is set according to the distance (S406). The danger level setting unit 127 increases the danger level of the object.

[0047] If the distance to the object is equal to or less than the threshold distance (YES in S405), the risk setting unit 127 The degree of danger is set according to the size of the object (S407). The risk setting unit 127 sets the risk to be high.

[0048] Then, the highlighting processing unit 128 performs highlighting processing according to the degree of risk (S408). That is, the highlighting processor 128 superimposes red on objects in the image if the object is highly dangerous. The output unit 129 outputs the superimposed image that has been subjected to the highlighting process to the display unit 30. The process is repeated while the vehicle 1 is moving backward. The superimposed image shown in is displayed.

[0049] For example, in FIGS. 5 and 6, the distance to the rear wall 301 is greater than the threshold distance. Therefore, as in S406, the risk level setting unit 127 sets the risk level according to the distance to the object. As shown in FIG. 5, when the distance to the rear wall 301 is long, the risk is low. As the vehicle 1 moves backward, the vehicle 1 is pushed against the rear wall 301 and the side wall 302. Therefore, the danger level gradually increases. In the situation shown in Figure 6, Since the vehicle 1 is closer to the rear wall 301 and the side wall 302 than in the state shown in FIG. 5 to 6. As the vehicles retreat, the color of the overlaid image changes.

[0050] On the other hand, in FIG. 7, the distance to the rear wall 301 is equal to or less than the threshold distance. As in 07, the danger level setting unit 127 sets the danger level according to the size of the object. The wall 301 and the side wall 302 are larger than the predetermined size, so the highest risk level is set. In the superimposed image of 7, the red color indicating the highest level of danger is superimposed on the rear wall 301 and the side wall 302. are.

[0051] In this way, when the distance to the rear wall 301 becomes equal to or less than the threshold distance, the risk level setting unit 127 The danger level is set according to the size of the object. By doing this, the danger level can be reduced by setting the level of danger for large objects. For example, the degree of danger can be determined only according to the distance. If the risk changes, the same risk level is set regardless of the size of the object. It is difficult to draw attention to large objects at a short distance. According to the embodiment, when an object is closer than a threshold distance, a danger is detected depending on the size of the object. The danger level setting unit 127 sets the danger level. If the vehicle 1 comes into contact with a large object, the impact will be large. In this embodiment, the danger level of a large object can be set high, so that the vehicle 1 It can alert you to large objects that are far away.

[0052] In this way, when the vehicle 1 approaches the object within the threshold distance, the risk level setting unit 127 The danger level is set according to the size of the body. Therefore, if the user moves a large object that is dangerous, The user, the driver, can slow down and park appropriately. This allows for effective driving assistance. If the object is at a distance equal to or greater than the threshold distance from the object, the danger level setting unit 127 sets the danger level according to the distance to the object. Therefore, the danger level changes depending on how close the user is to the object. This allows the system to recognize the distance and set the appropriate danger level according to the situation. This allows the user to be effectively alerted.

[0053] In step S407, if the size of an object closer than the threshold distance is larger than the predetermined size, If the distance is smaller than the predetermined distance, the risk level setting unit 127 sets the risk level according to the distance, as in step S405. In other words, as the distance becomes closer, the risk level setting unit 127 sets the risk level to a smaller size object. In other words, if the size of an object closer than the threshold distance is larger than a predetermined size, the risk of the object being detected is increased. If the object is small, it may be classified as a medium level of danger. Therefore, the risk level may be set in multiple stages.

[0054] In addition, the risk level setting unit 127 sets the risk level for objects smaller than a predetermined size in S405. In other words, the risk level setting unit 127 may exclude a certain Of the objects of a certain size or larger, the distance to the object closest to vehicle 1 is compared with the threshold distance. Specifically, the wheel stopper 304 is an object smaller than the predetermined size, so step S4 It is not subject to comparison with 04.

[0055] Furthermore, for objects that are unlikely or impossible to come into contact with vehicle 1, For example, the risk level setting unit 127 may set the risk level to 0. Calculate height information indicating the height of the body. If the height of the object is less than the minimum ground clearance of the vehicle 1, There is a small possibility of contact with the vehicle 1. For example, the height of the wheel stopper 304 is set to be equal to the lowest position of the vehicle 1. Since the height is lower than the height above the ground, the risk is small. It may be excluded from the comparison process at 404.

[0056] The threshold distance may be determined according to the size of the vehicle 1. For example, a 4-ton truck The total length of the vehicle is about 8 m. Therefore, the threshold distance from the object to the vehicle is about 4 m. In addition, for example, the overall length of a compact or ordinary passenger car is short, at approximately 4.70 m or less. Therefore, the threshold distance from the object to the vehicle can be about 2 m. If the depth of the base 300 is about 10% longer than the overall length of the vehicle, the threshold distance is less than the overall length of the vehicle. The threshold distance may also be determined according to the width of the vehicle 1.

[0057] <Embodiment 2> In the second embodiment, the risk level setting unit 127 performs risk level setting processing according to the driving information. More specifically, the setting process performed by the risk level setting unit 127 is switched depending on the traveling direction. For example, while the vehicle 1 is moving forward, the risk level setting unit 127 performs a first setting process. When the vehicle 1 is moving backward, the risk level setting unit 127 performs a second setting process that is different from the first setting process. The vehicle 1 and the display control system 10 have the same configuration as those shown in FIGS. Therefore, the explanation will be omitted.

[0058] 8 is a flowchart showing the display control process according to the present embodiment. The information acquisition unit 110 acquires driving information from the ECU 40 (S801). The direction acquisition unit 111 acquires the direction of travel. Furthermore, the speed information acquisition unit 113 acquires speed information. You may do so.

[0059] The image data acquisition unit 121 acquires image data of the image captured by the rear camera 21R. (S802). The object detection unit 122 detects an object included in the image (S803). The distance measurement data measured by the sensor 22 is acquired (S804). The size of the object is detected based on the data and image data (S805). The processing of steps S802 to S805 is basically the same as the processing of steps S401 to S404 in the first embodiment. However, in step S802, the front camera 21F The image data acquisition unit 121 acquires image data of the front image and the rear image of the rear camera 21R. This differs from step S401 in that

[0060] Next, it is determined whether the vehicle 1 is moving forward (S806). 806), the risk level setting unit 127 sets the risk level by the first setting process (S 807). The first setting process will be described later.

[0061] If the vehicle 1 is not moving forward (NO in S806), the risk level setting unit 127 performs the second setting process. The risk level is set by the second setting process (S808). When the vehicle 1 is moving backward, the risk level setting unit 127 performs the first setting process. The risk level is set by a different second setting process. The second setting process is, for example, In other words, in the second setting process, the risk setting unit 1 27 sets the danger level according to the size and distance of the object.

[0062] Then, the highlighting processing unit 128 performs highlighting processing according to the degree of risk (S809). That is, the highlighting processor 128 superimposes red on objects in the image if the object is highly dangerous. The output unit 129 outputs the superimposed image that has been subjected to the highlighting process to the display unit 30. Output (S810).

[0063] An example of the first setting process will be described below with reference to FIG. 9. In FIG. 9, A camera image of the road 310 and a superimposed image in which the camera image has been subjected to highlighting processing are shown. In front of the vehicle, an overpass 311 is provided. The vehicle is traveling on road 310 passing under bridge 311.

[0064] Here, if the height of vehicle 1 is too high to pass under viaduct 311, For example, if the viaduct 311 is 1.9 m above the ground and the vehicle height is higher than 1.9 m, Let's say that it has become like this.

[0065] The object detection unit 122 detects the viaduct 311 ahead in the traveling direction as an object. The setting unit 127 detects the height above ground of the viaduct 311. The risk setting unit 127 detects the clearance height of the vehicle 1 and the viaduct 311. Then, the risk level setting unit 127 sets the risk level according to the comparison result. do.

[0066] If the vehicle 1 is at a height where it cannot pass under the viaduct 311, the risk level setting unit 1 27 sets a high danger level for the viaduct 311. The highlighting processor 128 The highlighting processing unit 128 highlights the message in red, which corresponds to a high level of danger, on the warning display. The vehicle 1 is superimposed on the viaduct 311 in the image as a mark 312. This allows the display unit 30 to display a warning 312 to the user. The warning sign 312 is superimposed only on the viaduct 311 above the road ahead. It may be superimposed on the entire surface.

[0067] Generally, the vehicle speed when moving forward is faster than when moving backward. For example, when moving backward, Therefore, vehicle 1 moves at a relatively slow speed. When reversing for parking, the vehicle speed slows down. Therefore, the user can easily notice the height limit. On the other hand, when moving forward, the vehicle speed increases. Therefore, there is a risk that the user may overlook the height limit. The danger level setting unit 127 detects the height of the object and compares the vehicle height with the height of the object. By switching the risk level setting process according to the direction of travel, the risk level can be set appropriately. This makes it possible to more appropriately alert the user, who is the driver, to the situation.

[0068] In the above explanation, an example of passing through the road 310 under the viaduct 311 was explained. The process of the embodiment can be used for other than the viaduct 311. For example, Indoor parking lots, mechanical parking lots, and other parking lots may have height restrictions. When entering a parking lot with restrictions, the processing of this embodiment can be applied. The danger level setting unit 127 assigns a danger level to the object according to the height limit under the beam of 311 and the height inside the building. The risk level setting unit 127 compares the height of the object with the height of the vehicle 1, and determines whether the vehicle 1 is in danger of the object. The risk setting unit 127 determines whether or not the object can pass under the risk. Set.

[0069] It is acceptable to add a margin to the comparison result between the vehicle height of vehicle 1 and the clearance height of the object. For example, in the above description, if the vehicle height of the vehicle 1 is higher than the clearance limit of the viaduct 311, However, if the vehicle height of Vehicle 1 is about the same as the clearance limit of Viaduct 311, For example, a warning may be displayed. The risk level setting unit 127 determines the risk level based on the vehicle height of the vehicle 1, the ground height of the object, If the difference is less than a predetermined value, the risk level may be increased. Taking this into consideration, it is possible to provide a height margin.

[0070] In the first setting process, the risk level may be set according to the distance to the object. The closer the object is to the object, the higher the risk of danger may be weighted. If the distance is greater than the threshold distance, the risk level is set according to the distance in the same way as in the first embodiment. Then, when the distance to the object becomes equal to or less than the threshold distance, the height is compared. In this way, it is possible to more appropriately alert the user. For example, when the distance is long, the accuracy of height detection is low.

[0071] Furthermore, the risk level setting unit 127 may set the risk level based on speed information. For example, The timing at which the danger level setting unit 127 displays a warning is determined according to the speed information and the distance to the object. When the vehicle 1 is traveling at high speed, the distance from the object detection point to the point where the vehicle 1 arrives at the viaduct 311 may be adjusted. Therefore, it is preferable to display a warning to the user early. It is preferable that the faster the speed, the earlier the timing at which the highlight color is displayed. This makes it possible to display a warning at the appropriate timing.

[0072] The risk level setting unit 127 determines the risk level based on the height of the object from the road surface and the position of the object in the left-right direction. The risk setting unit 127 may measure at least one of the following: The risk setting unit 127 determines whether there is a possibility of collision between the two vehicles. The degree of danger is set for the body. This process will be explained using Figure 10. An example of parking forward or backward into the parking space 320 will be described.

[0073] Parking space 320 is provided with a wheel stopper 324. A nearby vehicle 321 is parked to the side of parking space 20. The surrounding vehicle 322 is parked. The surrounding vehicles 321 and 322 are located at a distance equal to or less than the threshold distance from the vehicle 1. The size in the left-right direction (horizontal direction) is about the same as that of vehicle 1. Furthermore, there is a pedestrian 325 at the far end of the parking space 320. The distance at is greater than the threshold distance.

[0074] The risk level setting unit 127 determines the left and right directions of the surrounding vehicles 321 and 322 based on the distance measurement data. The risk level setting unit 127 determines the risk level of the vehicle 1 based on the measurement result of the position in the left and right direction. The risk setting unit 127 determines whether there is a possibility that the vehicle will collide with the surrounding vehicles 321 and 322. The degree of danger of the surrounding vehicles 321 and 322 is set according to the judgment result.

[0075] For example, the nearby vehicle 321 is located right next to the parking space 320, so the risk level setting unit 1 27 determines that there is a possibility of a collision. Since the vehicle is located ahead in the row direction, the risk setting unit 127 determines that there is a possibility of a collision. Therefore, the risk level setting unit 127 sets a high risk level for the nearby vehicles 321 and 322. The front of the vehicle in the direction of travel is behind vehicle 1 when reversing, and behind vehicle 1 when moving forward. It will be in front.

[0076] In the superimposed image, the surrounding vehicles 321 and 322 are displayed with warning signs 331 and 332, respectively. That is, the surrounding vehicles 3 are superimposed with a highlight color such as red as a warning display 331, 332. 21, 322 are superimposed.

[0077] In the left-right direction (horizontal direction), the pedestrian 325 is positioned away from the front of the vehicle 1 in the traveling direction. The danger level setting unit 127 determines the danger level in the left and right directions of the pedestrian 325 based on the distance measurement data. The risk level setting unit 127 measures the position of the vehicle based on the measurement result of the position in the left and right direction. The risk setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with a pedestrian 325. Depending on the result, the danger level of the pedestrian 325 is set. The possibility of a collision is judged to be extremely low. A low risk level is set for pedestrians 325. The risk level setting unit 127 sets the risk level of the pedestrian 325 to 0, for example. In the statue, pedestrian 325 is not given any warning signs.

[0078] In this way, the risk level setting unit 12 determines the risk level according to the left-right position and size (width) of the object. 7 may set the danger level. Also, the degree of danger may be set according to the steering angle, the width of the object, and the left-right (lateral) position. It may be possible to determine whether there is a possibility of collision with the vehicle 1 by using the above information. If the possibility of contact is low, the risk setting unit 127 may set the risk to be low. If there is no risk, the risk setting unit 127 may set the risk to 0. When determining whether or not a collision with the vehicle 1 will occur based on the vehicle speed, the steering angle information of the vehicle 1 may be taken into consideration. That is, the risk setting unit 127 determines the risk level in the forward direction of travel according to the steering angle, for example, in the forward left or It may also be determined whether or not there is an object in front of the vehicle on the right.

[0079] The risk level setting unit 127 measures the height of the wheel stopper 324 based on the distance measurement data. The risk setting unit 127 determines whether or not the vehicle will collide with the wheel stopper 324 based on the height measurement result. The danger level setting unit 127 sets a danger level for the object according to the judgment result. 24 is lower than the minimum ground clearance of the vehicle 1. Therefore, the risk level setting unit 127 324, and sets the risk level to 0. In this case, the wheel chock 324 does not have a warning mark.

[0080] In this way, the risk level setting unit 127 can set the risk level more appropriately. Therefore, it is possible to effectively alert the user. The display control device 100 outputs a voice message for warning from a speaker based on the degree of danger. In other words, the output unit 129 may output a sound according to the degree of danger. Voice data can be generated.

[0081] In addition, the program for executing the above display control method can be used for various types of non-transient Stored using a non-transitory computer readable medium The non-transitory computer-readable medium can be provided to a computer. This includes any type of tangible storage medium. Examples of data-readable media include magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read-Only Read-Only Only Memory), CD-R, CD-R / W, semiconductor memory (e.g., mask ROM, PR OM (Programmable ROM), EPROM (Erasable PROM), Flash ROM, RAM (random access memory). Programs also contain various types of temporary components. Provided to a computer by a transitory computer readable medium Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transient computer-readable media may be transmitted over wired communication paths such as electric wires and optical fibers, or wirelessly. The program can be supplied to the computer via a wired communication channel. [Explanation of symbols]

[0082] 1 vehicle 2 Sensor Unit 21F Front Camera 21R rear camera 22 Distance measurement sensor 30 Display section 40 ECU 100 Display control device 110 Driving information acquisition unit 111 Traveling direction acquisition unit 113 Speed ​​information acquisition section 121 Image data acquisition unit 122 Object detection unit 125 Distance measurement data acquisition unit 127 Risk Level Setting Section 128 Highlighting Processing Unit 129 Output Section 301 Rear wall 302 Side wall 304 Wheel chock 310 Road 311 viaduct 312 Warning display 320 parking spaces 321, 322 surrounding vehicles 324 Wheel chock 325 Pedestrians 331, 332 Warning display

Claims

1. an image data acquisition unit that acquires image data of an image captured outside the vehicle; an object detection unit that detects an object included in the image; a distance measurement data acquisition unit that acquires distance measurement data relating to the distance from the host vehicle to the object; When the distance becomes equal to or less than a threshold, a danger level is assigned to the object according to the size of the object. a risk level setting unit for setting the risk level; a highlighting processing unit that performs highlighting processing on image data according to the risk level; an output unit that outputs a highlighted image that has been subjected to the highlighting process; Equipped with The threshold distance is set to a value when the depth of the parking space is approximately 10% longer than the overall length of the vehicle when the vehicle is parked in the parking space. The length is less than the total length of the vehicle. Display control device.

2. When the distance is greater than a threshold value, the risk level setting unit sets a risk level according to the distance to the object. The display control device according to claim 1 , wherein a degree of danger is set for the object by detecting the object.

3. The risk level setting unit At least one of the height of the object from the road surface and the left-right position of the object is measured. death, determining whether the object will collide with the host vehicle based on the measurement results; The danger level of the object is set according to the judgment result. The display control device according to claim 1 or 2.

4. acquiring image data of an image captured outside the vehicle; detecting an object contained in the image; acquiring distance measurement data relating to a distance from the host vehicle to the object; When the distance becomes equal to or less than a threshold, a danger level is assigned to the object according to the size of the object. setting steps; performing a highlighting process on the image data according to the risk level; outputting a highlighted image that has undergone the highlighting process; Equipped with The threshold distance is set to a value when the depth of the parking space is approximately 10% longer than the overall length of the vehicle when the vehicle is parked in the parking space. The length is less than the total length of the vehicle. Display control method.

5. acquiring image data of an image captured outside the vehicle; detecting an object contained in the image; acquiring distance measurement data relating to a distance from the host vehicle to the object; a step of setting a threshold distance to be less than the overall length of the vehicle when the depth of the parking space is approximately 10% longer than the overall length of the vehicle when the vehicle is parked in the parking space, and setting a risk level for the object according to the size of the object when the distance is equal to or less than the threshold distance; a step of performing a highlighting process on the image data according to the risk level; outputting a highlighted image that has undergone the highlighting process; A program that causes a computer to execute the following.

Images