Display control device, display control method, and program
The display control device and method enhance driver awareness by setting risk levels and highlighting dangerous objects, addressing the complexity of existing systems that fail to prioritize high-risk obstacles.
Patent Information
- Application Number
- JP2024158308
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2041-03-02
AI Technical Summary
Existing display systems fail to effectively alert drivers to the most dangerous obstacles among multiple objects by using color indications that can become complicated, making it difficult to draw attention to high-risk objects.
A display control device and method that acquires image data, detects objects, measures distances, sets risk levels based on object size or proximity to a threshold, and applies highlighting processing to emphasize dangerous objects with colors or other visual cues.
Effectively alerts users to high-risk objects by appropriately setting risk levels and highlighting them, ensuring safer driving by drawing attention to potentially hazardous situations.
Smart Images

Figure 2024169468000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a display control device, a display control method, and a program. [Background technology]
[0002] Patent Document 1 describes a surroundings monitoring device having a distance measuring detector that detects the distance from a vehicle to an object. The surroundings monitoring device of Patent Document 1 captures an image of an object from an imaging unit provided in a vehicle. The surroundings monitoring device has a camera that superimposes the color of the object and a color according to the height of the object. It is displayed.
[0003] Patent Document 2 describes an obstacle detection unit that detects the distance between the vehicle and an obstacle, and a vehicle exterior direction The device disclosed in Patent Document 2 is a vehicle surroundings visual confirmation device that includes an imaging unit that captures an image of the scenery. When an obstacle is detected, a figure that allows the direction of the obstacle to be recognized is displayed on the captured image. This device changes the color or size of the figure depending on the distance to the obstacle. It is possible. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2016-97843 A [Patent Document 2] JP 2006-311222 A Summary of the Invention [Problem to be solved by the invention]
[0005] In such a device, it is desirable to appropriately alert the driver. For example, When multiple obstacles are involved, it is preferable to draw attention to the more dangerous object. However, if multiple objects are present around the vehicle, they may be displayed according to distance or height. Changing the display color may result in cluttering the display. For example, If it is emphasized, it will not be possible to effectively draw attention to high-risk objects. It ends up like this.
[0006] In view of the above problems, the present invention provides a method for effectively providing a user with a warning by appropriately setting a risk level. The present invention aims to provide a display control device, a display control method, and a program that can draw attention. The target. [Means for solving the problem]
[0007] The display control device according to the present embodiment acquires image data of an image captured outside the vehicle. an image data acquisition unit for acquiring an image of the vehicle; an object detection unit for detecting an object included in the image; a distance measurement data acquisition unit for acquiring distance measurement data relating to a distance to the object from the If the threshold value is reached, a risk level is set for the object according to the size of the object. a degree setting unit, and an emphasis display processing unit that performs emphasis display processing on image data according to the degree of risk. and an output unit that outputs a highlighted image that has been subjected to the highlighting process, When the depth of the parking space is about 10% longer than the overall length of the vehicle, The period shall be less than 18 months.
[0008] The display control method according to the present embodiment includes acquiring image data of an image captured outside the vehicle. a step of detecting an object included in the image; and a step of detecting the object from the host vehicle. obtaining distance measurement data relating to a distance to the body; and when the distance is equal to or smaller than a threshold value, setting a danger level for the object in accordance with a size of the object in the case where the object is detected; performing a highlighting process on the image data according to the highlighting process; and outputting a display image of the parking space. If the distance is approximately 10% larger than the overall length of the vehicle, it will be considered to be less than the overall length of the vehicle.
[0009] The program according to the present embodiment acquires image data of an image captured outside the vehicle. a step of detecting an object included in the image; and a step of detecting the object from the host vehicle. and acquiring distance measurement data relating to a distance to the vehicle. When the distance is about 10% longer than the overall length of the vehicle, the threshold distance is set to be less than the overall length of the vehicle. When the threshold value is reached, a danger level is set for the object according to the size of the object. a step of performing a highlighting process on the image data according to the degree of risk; outputting a highlighted image on which the highlighting process has been performed; do. Effect of the Invention
[0010] According to the present invention, by appropriately setting the risk level, it is possible to effectively warn the user. It is possible to provide a display control device, a display control method, and a program that can perform the display control. [Brief description of the drawings]
[0011] [Figure 1] 1 is a diagram showing a vehicle according to an embodiment; [Diagram 2] FIG. 1 illustrates a display control device according to an embodiment. [Diagram 3] The color bar shows the display color according to the degree of danger. [Figure 4]4 is a flowchart showing a display control method according to the first embodiment. [Diagram 5] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. [Figure 8] 10 is a flowchart showing a display control method according to a second embodiment. [Figure 9] FIG. 11 is a schematic diagram for explaining display control according to the second embodiment. [Figure 10] FIG. 11 is a schematic diagram for explaining another display control method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] The present invention will be described below through the embodiments of the invention. The present invention is not limited to the following embodiments. Furthermore, all of the configurations described in the embodiments do not solve the problems. It is not necessarily essential to solve the problem. For the sake of clarity, the following description and In the drawings, the same elements are omitted or simplified as appropriate. The same reference numerals are used, and duplicated explanations are omitted as necessary.
[0013] <Embodiment 1> FIG. 1 is a diagram illustrating a vehicle 1 according to a first embodiment. The vehicle 1 includes a sensor unit 2 and In the following description, the vehicle 1 and the surrounding vehicles are In order to distinguish between the two, vehicle 1 may be referred to as the subject vehicle and the surrounding vehicles as the surrounding vehicles. do.
[0014] The sensor unit 2 has at least one imaging device and captures an image of the outside of the vehicle. The sensor unit 2 outputs image data captured by the imaging device to the display control device 100. The sensor unit 2 has a distance measuring sensor that detects an obstacle. 2 outputs distance measurement data measured by the distance measurement sensor to the display control device 100.
[0015] The display control device 100 may be installed at any position in the vehicle 1. The display control device 100 can be connected to a sensor unit. Image processing is performed to detect objects (obstacles) in the images captured by unit 2. The display control device 100 performs highlighting to draw the user's attention. For example, The display control device 100 applies an emphasis color such as red to objects included in the image. This makes it easier for drivers and other users to recognize the presence of objects and avoid contact. The display control device 100 will be described later.
[0016] In the following, the term "image" refers to an image that shows an image as a processing target in information processing. Similarly, hereinafter, the term "image" refers to a processing target in information processing. It also means "image data showing an image." Note that images and videos may be moving images, The image may be a still image.
[0017] FIG. 2 is a diagram illustrating a display control device 100 according to the first embodiment and a display control device having the display control device 100. 1 is a block diagram showing a configuration of a display control system 10. The display control system 10 includes a sensor unit A unit 2, a display unit 30, an ECU (Electronic Control Unit) 40, and a display control device The display control device 100 includes a sensor unit 2, a display unit 30, and an ECU 40. The communication device is communicatively connected to each of the above.
[0018] The display unit 30 is, for example, a display. The display unit 30 may also include a speaker. The display unit 30 may output a sound from a speaker. When mounted, the display unit 30 is installed inside the vehicle 1 at a position that can be seen by the driver of the vehicle while driving. In this case, the display unit 30 can be controlled by an interface unit 108, which will be described later. The display unit 30 may be a monitor of a car navigation system. The display unit 30 does not need to be an in-vehicle device permanently installed in the vehicle 1. For example, the display of a user's smartphone, tablet, or other information terminal Furthermore, a part of the display control process may be implemented by a smart phone having a display unit. This may be done by a mobile phone or the like.
[0019] The sensor unit 2 includes a front camera 21F and a rear camera 21R. When the vehicle 1 is moving forward, the front camera 21F captures an image of the front of the vehicle 1. In this case, the rear camera 21R captures an image behind the vehicle 1. The camera 21R generates shooting data at, for example, 30 frames per second (30 fps) and The shadow data is supplied to the display control device 100 every 1 / 30th of a second. The photographed data may be, for example, It may be generated using a format such as H.264 or H.265. The unit 2 may be a 360° camera that captures images of the entire surroundings of the vehicle 1.
[0020] The sensor unit 2 is a distance measuring sensor for detecting the distance to an object in the vicinity of the vehicle 1. The distance measuring sensor 22 detects the distance from the vehicle to an object. The sensor 22 is, for example, a LIDAR (Laser Imaging Detection and Ranging) sensor. The distance measuring sensor 22 may be an infrared camera, a stereo camera, a millimeter wave radar, or the like. Furthermore, the distance measuring sensor 22 may be configured by combining these sensors. In addition, the distance measuring sensor 22 is not limited to a single physical device. The distance measurement sensor 22 may be configured with LIDARs arranged at several locations.
[0021] The distance measurement sensor 22 transmits distance measurement data relating to the distance from the vehicle to an object to the display control device 100. Furthermore, the distance measuring sensor 22 detects the distance to the obstacle as well as the direction of the obstacle. In other words, the distance measurement data may correspond to the distance to an object and the direction of the object. Here, the direction of the object is a direction based on the vehicle itself. The right direction may be indicated by an azimuth angle, and the up and down directions may be indicated by an elevation angle.
[0022] The ECU 40 is included in the vehicle 1 and is a part of the configuration that controls the vehicle 1. The display control device 100 and the ECU 40 are connected to each other via an in-vehicle network such as a CAN. and connect so that communication is possible.
[0023] The display control device 100 includes a control unit 102, a storage unit 104, a communication unit 106, and an interface. The display control device 100 includes a driving information acquisition unit 108 (IF: Interface). unit 110, image data acquisition unit 121, object detection unit 122, and distance measurement data acquisition unit 125 The device includes a risk level setting unit 127, an emphasis display processing unit 128, and an output unit 129. The line information acquisition unit 110 includes a travel direction acquisition unit 111 and a speed information acquisition unit 113 .
[0024] The control unit 102 is, for example, a processor such as a CPU (Central Processing Unit). The control unit 102 functions as a calculation device that performs control processing, calculation processing, and the like. 104 is a storage device such as a memory or a hard disk. For example, ROM (Read Only Memory) or RAM (Random Access Memory). The memory unit 104 stores control information executed by the control unit 102 or as a function of the control unit 102. The memory unit 104 has a function for storing programs and calculation programs. has a function for temporarily storing processing data, etc.
[0025] The communication unit 106 performs the processes required for communication with the sensor unit 2 and the display unit 30. The communication unit 106 also performs processing required for communication with a CAN (not shown). The communication unit 106 may include a communication port. The interface unit 108 may be, for example, The interface unit 108 is a user interface (UI). The computer has an input device such as a keyboard, a touch panel, or a mouse, and an output device such as a speaker.
[0026] Therefore, the display control device 100 has a function as a computer. The control device 100 executes a program stored in the storage unit 104 by a processor in the control unit 102. By executing the above, the driving information acquisition unit 110, the image data acquisition unit 121, and the object detection unit 1 22, distance measurement data acquisition unit 125, risk level setting unit 127, highlighting processing unit 128, output unit 1 29. Each component of the display control device 100 is realized by a program. The implementation is not limited to software by the system, but includes hardware and software. The display control device 100 may be realized by any combination of the above. The components are, for example, FPGA (field-programmable gate array), microcomputer, or SOC (SOC). The present invention may be implemented using a user programmable integrated circuit such as a programmable system on a chip. In this case, the integrated circuit is used to execute a program composed of the above components. This also applies to other embodiments described later.
[0027] The travel information acquisition unit 110 acquires travel information related to the travel of the vehicle 1. The driving information acquisition unit 110 acquires various information from the ECU 40. The information processing unit 110 includes a speed information acquisition unit 111 and a speed information acquisition unit 113.
[0028] The traveling direction acquisition unit 111 acquires information indicating the traveling direction of the vehicle 1. For example, The acquisition unit 111 detects whether the vehicle 1 is moving forward or backward based on the shift position of the vehicle 1. If the shift position of vehicle 1 is in the reverse (R) position, vehicle 1 is moving backwards. Or, the shift lever of vehicle 1 is in a position such as drive (D). In this case, it is detected that the vehicle 1 is moving forward.
[0029] Furthermore, the traveling direction acquisition unit 111 may acquire steering angle information as the traveling direction. The direction acquisition unit 111 acquires a signal from the CAN or the like to indicate the steering angle of the wheels of the vehicle 1. The steering angle information includes information indicating whether the vehicle is turned right or left in addition to the steering angle. The information also includes information indicating the steering direction.
[0030] The speed information acquisition unit 113 acquires speed information indicating the speed of the vehicle 1. For example, The acquisition unit 113 detects the vehicle speed based on the vehicle speed pulse from the ECU 40. The acquiring unit 113 acquires speed information indicating a vehicle speed of, for example, 40 kilometers per hour. Alternatively, the speed information may be information in stages such as low speed, medium speed, high speed, and the like.
[0031] The image data acquisition unit 121 acquires image data captured by the front camera 21F and the rear camera 21R. In addition, the image data is acquired according to the traveling direction acquired by the traveling direction acquisition unit 111. The data acquisition unit 121 may switch the image to be acquired. For example, when the vehicle 1 is moving forward, The image data acquisition unit 121 acquires a forward image from the forward camera 21F. In this case, the image data acquisition unit 121 acquires a rear image from the rear camera 21R. The image data acquisition unit 121 may acquire both a forward image and a rearward image. The display control device 100 may simply switch the image to be processed depending on the traveling direction.
[0032] The object detection unit 122 detects an object included in an image. For example, the object detection unit 122 performs image analysis on the image data to recognize objects in the image. can perform object recognition by comparing an image with pre-registered patterns. In addition, when multiple objects are included in an image, the object detection unit 122 detects each object. Furthermore, the object detection unit 122 identifies the position coordinates and size of the object in the image. This is also fine.
[0033] Here, the objects are surrounding vehicles, walls, wheel stops, viaducts, bicycles, etc. The object detection unit 122 may include people such as pedestrians. Since a known method can be used for the theory, detailed description thereof will be omitted.
[0034] The distance measurement data acquisition unit 125 acquires distance measurement data relating to the distance to an object. When the image includes images of multiple objects, the distance measurement data acquisition unit 125 measures the distance to each object. That is, the distance from the vehicle to each object is calculated based on the distance data from the distance measuring sensor 22. Calculate the distance to.
[0035] The danger level setting unit 127 sets a danger level for the object. If the object is included in the object, the danger level setting unit 127 sets a danger level for each object. Each is assigned a value indicating the degree of danger.
[0036] Specifically, the risk setting unit 127 determines the possibility that the vehicle will come into contact with an object, and when the vehicle comes into contact with the object, The danger level setting unit 127 sets the danger level according to the distance to the object. The danger level setting unit 127 expresses the danger level value on a 10-point scale, with 10 being The highest risk level is 0, and the lowest risk level is 1. The closer the object is, the higher the risk level is. The more distant the object, the lower the danger. Also, for objects with an extremely low possibility of contact, For the body, the risk level setting unit 127 may set the risk level to 0. For objects that deviate significantly from the threshold, the danger level may be set to 0.
[0037] Alternatively, the risk level setting unit 127 sets the risk level according to the size of the object. The determining unit 127 determines the danger level to be higher for larger objects and the danger level to be lower for smaller objects. For example, the risk level setting unit 127 reduces the risk level based on the size of the object image in the image and the The size of the object can be calculated based on the distance to the body. The process of determining the size of an object will be described later. This can be either the size or the vertical size (up and down direction), or both.
[0038] The highlighting processor 128 performs highlighting processing for highlighting an object in an image according to the degree of danger. For example, the highlighting processing unit 128 highlights objects with a high degree of danger using red or other highlighting colors. The highlighting processor 128 superimposes colors onto objects in the image. Objects with low risk are highlighted in blue, etc. The highlighting processor 128 superimposes non-highlighted colors on objects in the image. Therefore, the highlighting process is not performed for objects that are unlikely to come into contact with the vehicle. The unit 128 displays a warning to the user by selecting a color that is emphasized as the object becomes more dangerous. is superimposed on the object in the image.
[0039] The degree of danger can be expressed by a color bar with a color distribution as shown in Figure 3. Indeed, the red side is the longer wavelength side, and the lower the danger, the blue side is the shorter wavelength side. The highlighting processor 128 superimposes a color corresponding to the degree of danger on the object in the image. In the image, red is superimposed on objects that are highly dangerous, and blue is superimposed on objects that are less dangerous. For objects with a medium risk, green is superimposed on the object in the image. The display processing unit 128 generates a superimposed image in which a color corresponding to the danger level is superimposed on an object in the image. The output unit 129 outputs the image so that the superimposed image is displayed on the display unit 30 as a highlighted image. Output the data.
[0040] By changing the overlay color according to the level of danger, the system can appropriately alert users. In other words, the higher the risk, the more emphasized it is displayed, so that users can easily Objects with high visibility can be easily recognized. By superimposing an emphasis color on the object in the image, the warning The user is more likely to pay attention to high-risk objects than low-risk objects. The user can be appropriately alerted to the presence of an object when driving. This contributes to safer driving.
[0041] The highlighting is not limited to the color distribution shown in Figure 3. For example, the degree of danger can be represented by a light and dark shade. For example, the image of the object may be colored in the same color and the degree of danger may be indicated by only a shading. The highlighting processor 128 may change the color tone, shading, brightness, transparency, etc. of the object image. The highlighting processing unit 128 may perform highlighting processing by adding a A highlighting process may be performed by adding a contour line. In this case, the more dangerous the object, the more likely it is to be highlighted. In this way, the highlighting processor 128 makes the display more emphasized as the degree of danger increases. Performs highlighting processing.
[0042] An example of the risk level setting process and the highlighting process will be described with reference to FIGS. 4 to 7. FIG. 5 to 7 are flowcharts showing a display control process according to the present embodiment. 5 to 7 are diagrams for explaining the degree of danger in a parking space and the highlighting process thereof. 5 to 7 show an example of parking in reverse at Pace 300. The color scheme according to the danger level, the camera image, and the top view are shown. And, vehicle 1 is moving at a low speed. As the vehicle retreats, the state changes in the order of Figure 5, Figure 6, and Figure 7.
[0043] A rear wall 301 is disposed at the rear of the parking space 300, and side walls 302 are disposed on the left and right sides. That is, the parking space 300 is surrounded on three sides by a rear wall 301 and a side wall 302. Then, the vehicle retreats into a parking space 300 that is partitioned by a rear wall 301 and a side wall 302. The parking space 300 has a wheel stopper 304 on the road surface. 1R captures an image of a parking space 300. The distance measuring sensor 22 captures images of a rear wall 301 and a side wall 30 The distance to 2 is detected as the distance to the object.
[0044] First, the image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R. The object detection unit 122 detects an object included in the image (S402). For example, the object detection unit 122 detects the rear wall 301, the side wall 302, and the wheel stopper included in the image. 304 are detected. The distance measurement data measured by the distance measurement sensor 22 is acquired (S40 3) As a result, the distance from the vehicle 1 to the rear wall 301, the side wall 302, and the wheel stopper 304 is The risk level setting unit 127 calculates the distance from the target object to the target object based on the distance measurement data and the image data. Then, the size of the object is detected (S404). This allows the distance and size of each object to be detected. will be done.
[0045] Next, the risk level setting unit 127 determines whether the distance from the vehicle to the object is equal to or less than a threshold distance. The risk level setting unit 127, for example, compares the distance to the closest object with a threshold distance. When multiple objects are included in an image, the risk setting unit 127 compares the nearest object. Here, it is determined whether the distance to the rear wall 301 is equal to or less than the threshold distance. The distance is compared to a threshold distance.
[0046] If the distance to the object is not equal to or less than the threshold distance (NO in S405), the risk level setting unit 127 The degree of danger is set according to the distance (S406). In other words, the closer the object is, the greater the danger. The danger level setting unit 127 increases the danger level of the object.
[0047] If the distance to the object is equal to or shorter than the threshold distance (YES in S405), the risk setting unit 127 The degree of danger is set according to the size of the object (S407). The risk setting unit 127 sets the risk level to high.
[0048] Then, the highlighting processing unit 128 performs highlighting processing according to the degree of danger (S408). In other words, the highlighting processor 128 superimposes red on objects in the image that are highly dangerous. The output unit 129 outputs the superimposed image that has been subjected to the highlighting process to the display unit 30. The process is repeated while the vehicle 1 is moving backward. The superimposed image shown in FIG.
[0049] For example, in FIG. 5 and FIG. 6, the distance to the rear wall 301 is greater than the threshold distance. Therefore, as in S406, the risk level setting unit 127 sets the risk level according to the distance to the object. As shown in FIG. 5, when the distance to the rear wall 301 is long, the risk is low. As the vehicle 1 moves backward, the vehicle 1 is struck by the rear wall 301 and the side wall 302. The danger level gradually increases. In the state shown in Figure 6, Since the vehicle 1 is closer to the rear wall 301 and the side wall 302 than in the state shown in FIG. 5 to 6. As the two vehicles retreat, the color of the overlay image changes.
[0050] On the other hand, in FIG. 7, the distance to the rear wall 301 is equal to or less than the threshold distance. As in FIG. 07, the danger level setting unit 127 sets the danger level according to the size of the object. Since the wall 301 and the side wall 302 are larger than the predetermined size, the highest danger level is set. In the overlapping image of FIG. 7, the red color, which indicates the highest level of danger, is overlapped on the rear wall 301 and the side wall 302. is.
[0051] In this way, when the distance to the rear wall 301 becomes equal to or less than the threshold distance, the risk level setting unit 127 The danger level is set according to the size of the object. This makes it possible to detect large objects that pose a high danger. For example, the degree of danger is determined only according to the distance. If the object changes, the same danger level is set regardless of the size of the object. It is difficult to draw attention to large objects at a short distance. According to the embodiment, when an object is closer than a threshold distance, a danger is detected according to the size of the object. The danger level setting unit 127 sets the danger level. If the vehicle 1 comes into contact with a large object, the impact will be large. In this embodiment, the danger level of a large object can be set high, so that the vehicle 1 It can alert you to large objects that are far away.
[0052] In this way, when the vehicle 1 approaches the object within a threshold distance, the danger level setting unit 127 The danger level is set according to the size of the body. Therefore, when the user moves a large object that is dangerous, The user, the driver, can decelerate and park appropriately. This can effectively assist driving. If the object is located at a distance equal to or greater than the threshold distance, the danger level setting unit 127 sets the danger level according to the distance to the object. Therefore, the degree of danger changes depending on how close the user is to the object. It is possible to recognize the distance. By doing this, the danger level can be set appropriately according to the situation. This makes it possible to effectively alert the user.
[0053] In step S407, if the size of an object closer than the threshold distance is larger than a predetermined size, If the risk level is smaller than the threshold value, the risk level setting unit 127 sets the risk level according to the distance in the same manner as in step S405. In other words, as the distance becomes closer, the risk level setting unit 127 sets the risk level to a smaller size object. In other words, the risk of the size of an object closer than the threshold distance being larger than a predetermined size is increased. If the object is small, it may be classified as a medium level of danger. The degree of danger may be set in multiple stages.
[0054] In addition, the risk level setting unit 127 sets the risk level for objects smaller than a predetermined size in S405. In other words, the risk level setting unit 127 may exclude a certain Among objects of a certain size or larger, the distance to the object closest to vehicle 1 is compared with a threshold distance. Specifically, the wheel stopper 304 is an object smaller than the predetermined size, so step S4 It is not subject to comparison with 04.
[0055] Furthermore, for objects that are unlikely or have no possibility of coming into contact with vehicle 1, For example, the risk level setting unit 127 may set the risk level to 0. Calculate height information indicating the height of the body. If the height of the object is less than the minimum ground clearance of the vehicle 1, The possibility of contacting the vehicle 1 is small. For example, the height of the wheel stopper 304 is set to be equal to or smaller than the lowest point of the vehicle 1. Since the height is lower than the height above the ground, the risk is small. Therefore, for the wheel stopper 304, It may be excluded from the comparison process at 404.
[0056] The threshold distance may be determined according to the size of the vehicle 1. For example, a 4-ton truck, The total length of the vehicle is about 8 m. Therefore, the threshold distance from the object to the vehicle is about 4 m. For example, the overall length of a small or ordinary passenger car is short, at approximately 4.70 m or less. Therefore, the threshold distance from the object to the vehicle can be about 2m. If the depth of the base 300 is about 10% larger than the overall length of the vehicle, the threshold distance is less than the overall length of the vehicle. The threshold distance may also be determined according to the width of the vehicle 1.
[0057] <Embodiment 2> In the second embodiment, the risk setting unit 127 performs risk setting processing according to the driving information. More specifically, the setting process performed by the risk level setting unit 127 is switched depending on the traveling direction. For example, while the vehicle 1 is moving forward, the risk level setting unit 127 performs a first setting process. When the vehicle 1 is moving backward, the risk level setting unit 127 performs a second setting process different from the first setting process. The vehicle 1 and the display control system 10 are configured similarly to those shown in FIG. 1 and FIG. 2. Therefore, the description will be omitted.
[0058] FIG. 8 is a flowchart showing the display control process according to the present embodiment. The information acquisition unit 110 acquires driving information from the ECU 40 (S801). The direction acquisition unit 111 acquires the traveling direction. Furthermore, the speed information acquisition unit 113 acquires the speed information. You may do so.
[0059] The image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R. (S802). The object detection unit 122 detects an object included in the image (S803). The distance measurement data measured by the sensor 22 is acquired (S804). The size of the object is detected based on the data and image data (S805). The process of steps S802 to S805 is basically the same as the process of steps S401 to S404 in the first embodiment. However, in step S802, the front camera 21F The image data acquisition unit 121 acquires image data of the front image and the rear image of the rear camera 21R. This differs from step S401 in that
[0060] Next, it is determined whether the vehicle 1 is moving forward (S806). If the determination is YES in step S806, the risk level setting unit 127 sets the risk level by the first setting process (S 807). The first setting process will be described later.
[0061] When the vehicle 1 is not moving forward (NO in S806), the risk level setting unit 127 performs the second setting process. The risk level is set by the second setting process (S808). When the vehicle 1 is moving backward, the risk level setting unit 127 performs the first setting process. The risk level is set by a different second setting process. The second setting process is, for example, In other words, in the second setting process, the risk setting unit 1 27 sets the danger level according to the size and distance of the object.
[0062] Then, the highlighting processing unit 128 performs highlighting processing according to the degree of danger (S809). In other words, the highlighting processor 128 superimposes red on objects in the image that are highly dangerous. The output unit 129 outputs the superimposed image that has been subjected to the highlighting process to the display unit 30. Output (S810).
[0063] An example of the first setting process will be described below with reference to FIG. A camera image of a road 310 and a superimposed image in which the camera image has been subjected to highlighting processing are shown. In front of the vehicle, an elevated bridge 311 is provided. The vehicle is traveling on road 310 which passes under bridge 311.
[0064] Here, if the height of vehicle 1 is too high to pass under viaduct 311, For example, if the viaduct 311 is 1.9m above the ground and the vehicle height is higher than 1.9m, Let's say that it has become like this.
[0065] The object detection unit 122 detects the viaduct 311 ahead in the traveling direction as an object. The setting unit 127 detects the height of the viaduct 311 above ground. The risk level setting unit 127 detects the clearance height of the vehicle 1 and the viaduct 311. The risk level setting unit 127 then sets the risk level according to the comparison result. do.
[0066] When the vehicle 1 is at a height where the vehicle 1 cannot pass under the viaduct 311, the risk level setting unit 1 27 sets a high danger level for the viaduct 311. The highlighting processor 128 performs the following in accordance with the danger level: The highlighting processing unit 128 displays the red highlight color corresponding to the high danger level as a warning. The vehicle 1 is superimposed on the viaduct 311 in the image as a mark 312. Since the display unit 30 has high reliability, the display unit 30 can display a warning 312 to the user. The warning sign 312 is superimposed only on the viaduct 311 above the road ahead. It may be superimposed on the entire surface.
[0067] In general, the forward speed of a vehicle is faster than the reverse speed of a vehicle. For example, when reversing, Therefore, vehicle 1 moves at a relatively slow speed. When reversing for parking, the vehicle speed slows down. Therefore, the user can easily notice the height restriction. On the other hand, when moving forward, the vehicle speed increases. Therefore, the user may overlook the height limit. The danger level setting unit 127 detects the height of the object and compares the vehicle height with the height of the object. By switching the risk level setting process according to the direction of travel, the risk level can be set appropriately. It is possible to more appropriately alert the user, who is the driver, to the risk of collision.
[0068] In the above description, an example of passing through the road 310 under the viaduct 311 has been described. The process of the embodiment can be used for other than the viaduct 311. For example, a multi-storey car park, Indoor parking lots, mechanical parking lots, and other parking lots may have height restrictions. When entering a parking lot with parking restrictions, the process of this embodiment can be applied. The danger level setting unit 127 assigns a danger level to the object according to the clearance limit height of the 311 and the height inside the building. The risk level setting unit 127 compares the height of the object with the height of the vehicle 1 and determines whether the vehicle 1 is in danger of the object. The risk level setting unit 127 determines whether or not the vehicle can pass under the obstacle. Set.
[0069] A margin may be added to the comparison result between the vehicle height of vehicle 1 and the clearance height of the object. For example, in the above description, when the vehicle height of the vehicle 1 is higher than the clearance limit of the viaduct 311, A warning was displayed, but if the vehicle height of vehicle 1 was about the same as the clearance limit of viaduct 311, For example, a warning may be displayed. If the difference is less than a predetermined value, the risk level may be increased. Taking this into consideration, it is possible to provide a height margin.
[0070] In the first setting process, the degree of danger may be set according to the distance to the object. For example, The closer the object is, the higher the risk of danger may be weighted. If the distance is greater than the threshold distance, the risk level is set according to the distance in the same manner as in the first embodiment. Then, when the distance to the object becomes equal to or smaller than the threshold distance, the height is compared. In this way, it is possible to more appropriately alert the user. For example, when the distance is long, the accuracy of height detection is low.
[0071] In addition, the risk level setting unit 127 may set the risk level based on the speed information. For example, The timing at which the danger level setting unit 127 displays a warning depending on the speed information and the distance to the object. When the vehicle 1 is traveling at high speed, the vehicle 1 may be adjusted from the object detection to the arrival at the viaduct 311. Therefore, it is preferable to display a warning to the user early. It is preferable that the faster the speed is, the earlier the timing of displaying the highlight color is. This makes it possible to display a warning at an appropriate timing.
[0072] The risk level setting unit 127 determines the risk level based on one of the height of the object from the road surface and the position of the object in the left-right direction. The risk level setting unit 127 may measure at least one of the following: The risk setting unit 127 determines whether there is a possibility of collision between the vehicle 1 and the object 1. The degree of danger is set for the body. This process will be explained using FIG. 10. An example of parking forward or backward into the parking space 320 will be described.
[0073] Parking space 320 is provided with a wheel stopper 324. A nearby vehicle 321 is parked to the side of the parking space 20. The surrounding vehicle 322 is parked. The surrounding vehicles 321 and 322 are located at a distance equal to or smaller than the threshold distance to the vehicle 1. The horizontal size is about the same as that of vehicle 1. Furthermore, there is a pedestrian 325 at the far end of the parking space 320. The distance at is greater than the threshold distance.
[0074] The risk level setting unit 127 determines the left and right directions of the surrounding vehicles 321 and 322 based on the distance measurement data. The risk level setting unit 127 determines the risk of the vehicle 1 based on the measurement result of the position in the left and right direction. The risk level setting unit 127 determines whether there is a possibility that the vehicle 100 will collide with the surrounding vehicles 321 and 322. The degree of danger of the surrounding vehicles 321, 322 is set according to the judgment result.
[0075] For example, the surrounding vehicle 321 is located right next to the parking space 320, so the risk level setting unit 1 The surrounding vehicle 322 determines that there is a possibility of a collision. Since the vehicle is ahead in the row direction, the risk setting unit 127 determines that there is a possibility of collision. Therefore, the risk level setting unit 127 sets a high risk level for the surrounding vehicles 321 and 322. The forward direction of travel is the rear of vehicle 1 when reversing, and the front of vehicle 1 when moving forward. It will be in front.
[0076] In the superimposed image, the surrounding vehicles 321 and 322 are displayed with warning signs 331 and 332, respectively. In other words, the surrounding vehicles 331, 332 are displayed in a red or other highlighting color as warnings. 21, 322 are superimposed.
[0077] In the left-right direction (horizontal direction), the pedestrian 325 is positioned away from the front of the vehicle 1 in the traveling direction. The danger level setting unit 127 determines the danger level in the left and right directions of the pedestrian 325 based on the distance measurement data. The risk level setting unit 127 determines the risk of the vehicle based on the measurement result of the position in the left and right direction. The risk level setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with a pedestrian 325. Depending on the result, the danger level of the pedestrian 325 is set. The possibility of a collision is judged to be extremely low. A low risk level is set for pedestrians 325. The risk level setting unit 127 sets the risk level of the pedestrian 325 to 0, for example. In the statue, pedestrian 325 is not given any warning signs.
[0078] In this way, the risk level setting unit 12 determines the risk level according to the left-right position and size (width) of the object. The degree of danger may be set by the driver 7. Also, the degree of danger may be set according to the steering angle, the width of the object, and the left-right (lateral) position. Then, it may be possible to determine whether there is a possibility of collision with the vehicle 1. If the possibility of contact is low, the risk setting unit 127 may set the risk to be low. If there is no risk, the risk setting unit 127 may set the risk to 0. When determining whether or not a collision will occur with the vehicle 1 based on the vehicle speed, the steering angle information of the vehicle 1 may be taken into consideration. That is, the risk setting unit 127 determines the risk level in the forward direction in the traveling direction according to the steering angle, for example, in the left forward direction or It may be determined whether or not there is an object in the right front.
[0079] The risk level setting unit 127 measures the height of the wheel stopper 324 based on the distance measurement data. The risk setting unit 127 determines whether or not the vehicle will collide with the wheel stopper 324 based on the height measurement result. The danger level setting unit 127 sets a danger level for the object according to the judgment result. 24 is lower than the minimum ground clearance of the vehicle 1. Therefore, the risk level setting unit 127 324, and sets the risk level to 0. In this case, the wheel stopper 324 has no warning marking.
[0080] In this way, the risk level setting unit 127 can set the risk level more appropriately. Therefore, it is possible to effectively alert the user. The display control device 100 outputs a voice message for warning from a speaker based on the degree of danger. In other words, the output unit 129 may output a sound according to the degree of danger. Voice data may be generated.
[0081] In addition, a program for executing the above display control method can be used for various types of non-transient Stored using a non-transitory computer readable medium The non-transitory computer-readable medium can be provided to a computer. This includes any type of tangible storage medium. Examples of data-readable media include magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-R, CD-R / W, semiconductor memory (e.g., mask ROM, PR OM (Programmable ROM), EPROM (Erasable PROM), Flash ROM, RAM (random access memory). Programs also contain various types of temporary Provided to a computer by a transitory computer readable medium Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. A transitory computer-readable medium may be transmitted over a wired communication path, such as an electric wire or an optical fiber, or wirelessly. The program can be supplied to the computer via a wired communication line. [Explanation of symbols]
[0082] 1 vehicle 2 Sensor unit 21F Front Camera 21R Rear camera 22 Distance sensor 30 Display section 40 ECU 100 Display control device 110 Driving information acquisition unit 111 Traveling direction acquisition unit 113 Speed information acquisition section 121 Image data acquisition unit 122 Object detection unit 125 Distance measurement data acquisition unit 127 Risk Level Setting Section 128 Highlighting Processing Unit 129 Output section 301 Rear wall 302 Side wall 304 Wheel chock 310 Road 311 viaduct 312 Warning display 320 Parking spaces 321, 322 surrounding vehicles 324 Wheel chock 325 Pedestrians 331, 332 Warning display
Claims
1. an image data acquisition unit that acquires image data of an image captured outside the vehicle; an object detection unit that detects an object included in the image; a distance measurement data acquisition unit that acquires distance measurement data relating to a distance from the host vehicle to the object; When the distance becomes equal to or less than a threshold, a danger level is assigned to the object according to a size of the object. A risk level setting unit for setting a risk level; an emphasis display processing unit that performs emphasis display processing on image data according to the degree of risk; an output unit that outputs a highlighted image that has been subjected to a highlighting process; Equipped with The threshold distance is set to be less than the overall length of the vehicle when the depth of the parking space is approximately 10% larger than the overall length of the vehicle. Display control device.
2. When the distance is greater than a threshold, the risk level setting unit sets a risk level according to the distance to the object. The display control device according to claim 1 , further comprising: a display control unit for controlling a display of the object;
3. The risk level setting unit is At least one of the height of the object from the road surface and the left-right position of the object is measured. death, determining whether the object will collide with the host vehicle based on the measurement result; The danger level of the object is set according to the judgment result. The display control device according to claim 1 or 2.
4. acquiring image data of an image captured outside the vehicle; detecting an object in the image; acquiring distance measurement data relating to a distance from the host vehicle to the object; When the distance becomes equal to or less than a threshold, a danger level is assigned to the object according to a size of the object. A step of setting performing a highlighting process on the image data according to the degree of risk; outputting a highlighted image on which the highlighting process has been performed; Equipped with The threshold distance is set to be less than the overall length of the vehicle when the depth of the parking space is approximately 10% larger than the overall length of the vehicle. Display control method.
5. acquiring image data of an image captured outside the vehicle; detecting an object in the image; acquiring distance measurement data relating to a distance from the host vehicle to the object; When the depth of the parking space is about 10% longer than the overall length of the vehicle, the threshold distance is set to setting the distance to less than a total length of the host vehicle, and when the distance becomes equal to or less than the threshold, setting a danger level for the object according to a size of the object; performing a highlighting process on the image data according to the degree of risk; outputting a highlighted image on which the highlighting process has been performed; A program that causes a computer to execute the following.
Citation Information
Patent Citations
Vehicle surrounding monitoring device
JP2007049219A
Vehicular transmission display device
JP2016088187A
Driving support device and driving support system
JP2019075150A
Side safety assistant device and method for large vehicle
US20180165527A1
Vehicle peripheral visual recognition device
JP2006311222A
