In-vehicle display device and display control method

Abstract

In displaying images obtained from in-vehicle cameras, etc., dangerous areas will be appropriately indicated. [Solution] The in-vehicle display device 1 according to this disclosure comprises an acquisition unit 20, a direction of travel acquisition unit 10, and an image generation unit 80. The acquisition unit 20 acquires information about the surroundings of the vehicle 999. The direction of travel acquisition unit 10 detects the direction of travel of the vehicle 999. The image generation unit 80 generates a first image relating to dangerous areas around the vehicle 999 from the image included in the surrounding information acquired by the acquisition unit 20. The image generation unit 80 superimposes an image relating to the field of view of the first image onto the first image. The image generation unit 80 varies the superposition position of the image relating to the field of view of the first image according to the direction of travel of the vehicle 999.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an in-vehicle display device and a display control method. BACKGROUND ART

[0002] Patent Document 1 discloses an in-vehicle surveillance camera device that cuts out and displays an image of a hazardous location from an image captured by a wide-angle lens. PRIOR ART DOCUMENTS PATENT DOCUMENTS

[0003] PATENT DOCUMENT 1 Patent No. 6785953 PATENT DOCUMENT 2 Japanese Patent Laid-Open No. 2006-178667 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] However, the conventional in-vehicle surveillance camera device may not be able to appropriately display hazards to the driver.

[0005] The present disclosure has been made to solve such a problem, and one of its objectives is to appropriately display a hazardous location in the display of an image obtained by an in-vehicle camera or the like. MEANS FOR SOLVING THE PROBLEMS

[0006] The in-vehicle display device according to this disclosure comprises an acquisition unit, a direction of travel acquisition unit, and an image generation unit. The acquisition unit acquires information about the surroundings of the vehicle. The direction of travel acquisition unit detects the direction of travel of the vehicle. The image generation unit generates a first image relating to dangerous areas around the vehicle from the image included in the surrounding information acquired by the acquisition unit. The image generation unit superimposes an image relating to the field of view of the first image onto the first image. The image generation unit varies the superimposition position of the image relating to the field of view of the first image according to the direction of travel of the vehicle. [Effects of the Invention]

[0007] According to this disclosure, hazardous areas can be appropriately displayed in images obtained from in-vehicle cameras, etc. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the configuration of an in-vehicle display device in an embodiment. [Figure 2] Figure 2 shows an example of the imaging range of a camera installed at the rear of the vehicle. [Figure 3] Figure 3 shows an example of the imaging range of a camera installed on the front of the vehicle. [Figure 4] Figure 4 is a diagram illustrating the recognition of driving scenes and anticipated hazards by the anticipated hazard detection unit. [Figure 5] Figure 5 shows an example of a display in which images related to the field of view are superimposed. [Figure 6] Figure 6 shows an example of the relationship between the imaging range of the camera mounted on the vehicle and the location where hazardous areas are detected. [Figure 7] Figure 7 shows an example of the area captured by a rear-mounted camera on a vehicle. [Figure 8] Figure 8 shows an example of the position in which the image generation unit superimposes a radar chart on "Image C" according to the direction of travel. [Figure 9]FIG. 9 is a diagram illustrating an example of a position where the image generation unit superimposes a radar chart on "Image C" in accordance with the traveling direction. [Figure 10] FIG. 10 is a diagram illustrating an example of a position where the image generation unit superimposes a radar chart on "Image C" in accordance with the traveling direction. [Figure 11] FIG. 11 is a diagram illustrating an example of a position where the image generation unit superimposes a radar chart on "Image C" in accordance with the traveling direction. [Figure 12] FIG. 12 is a diagram illustrating an example of superimposing an upwardly convex radar chart when the traveling direction is forward by the image generation unit. [Figure 13] FIG. 13 is a diagram illustrating an example of superimposing a downwardly convex radar chart when the traveling direction is rearward by the image generation unit. [Figure 14] FIG. 14 is a diagram illustrating an example of superimposing a radar chart by the image generation unit in accordance with the traveling direction and the detected position of a hazardous location. [Figure 15] FIG. 15 is a diagram illustrating an example of superimposing a radar chart by the image generation unit in accordance with the traveling direction and the detected position of a hazardous location. [Figure 16] FIG. 16 is a diagram illustrating an example of a data table defining the shape and color of marks according to priority levels. [Figure 17] FIG. 17 is a diagram illustrating an example of a data table defining the shape and color of marks for each time or distance to contact. [Figure 18] FIG. 18 is a flowchart illustrating an example of the operation of the in-vehicle display device according to the embodiment. [Figure 19] FIG. 19 is a flowchart illustrating an example of the flow of processing for the image generation unit to superimpose an image related to the field of view. [Figure 20] FIG. 20 is a flowchart illustrating an example of the operation of the in-vehicle display device according to Modification 1 of the embodiment. [Mode for Carrying Out the Invention]

[0009] Hereinafter, embodiments will be described in detail with appropriate reference to the drawings. However, unnecessarily detailed descriptions may be omitted in some cases. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

[0010] It should be noted that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the appended claims.

[0011] (Embodiment) Hereinafter, the embodiment will be described with reference to Figures 1 to 14. The present embodiment relates to an in-vehicle display device, and particularly to an in-vehicle display device that monitors the surroundings of a vehicle with a camera and notifies the driver of a hazardous location with a risk of collision.

[0012] (Configuration of In-Vehicle Display Device) First, the configuration of the in-vehicle display device 1 according to the present disclosure will be described with reference to Figure 1. Figure 1 is a diagram illustrating an example of the configuration of the in-vehicle display device 1 in the embodiment.

[0013] In the present disclosure, "front" means the side in the forward traveling direction of the vehicle (front side), "rear" means the side in the reverse traveling direction of the vehicle opposite to "front" (rear side), "left" means the left side as viewed in the forward traveling direction of the vehicle, and "right" means the right side as viewed in the forward traveling direction of the vehicle. Note that the "front" of the vehicle may include a portion directly facing the front end (front portion) of the vehicle, and the left front and right front on its left and right sides. Similarly, the "rear" of the vehicle may include a portion directly facing the rear end (rear portion) of the vehicle, and the left rear and right rear on its left and right sides. Similarly, the "left" of the vehicle may include a portion directly facing the end on the left side surface of the vehicle (left side surface portion), and the left front and left rear on its front and rear sides. Similarly, the "right" of the vehicle may include a portion directly facing the end on the right side surface of the vehicle (right side surface portion), and the right front and right rear on its front and rear sides.

[0014] The in-vehicle display device 1 is an information processing device (computer) mounted in a vehicle such as an automobile, and controls the display (screen display) on the display device 5 (display) by executing a display control method. This computer may be, for example, an ECU (Electronic Control Unit) or OBU (On Board Unit) located inside the vehicle. Alternatively, the in-vehicle display device 1 may be an external device (computer) installed near the dashboard of the vehicle, or it may also function as a car navigation system. This computer (in-vehicle display device 1) has, for example, at least one processor and at least one memory. Various types of processors, such as a CPU (Central Processing Unit), can be used as the at least one processor. Various types of memory, such as RAM (Random Access Memory) and ROM (Read Only Memory), can be used as the at least one memory. This computer may perform various processes by loading a program stored in ROM into RAM and executing it, or it may have a dedicated circuit (hardware) configured to perform some or all of the processes.

[0015] As shown in Figure 1, the vehicle-mounted display device 1 includes a direction of travel acquisition unit 10, an acquisition unit 20, a detection rule setting unit 30, a hazardous area detection unit 40, a hypothetical hazardous area detection unit 50, a contact risk determination unit 60, a priority setting unit 70, an image generation unit 80, and a distortion correction unit 90. The vehicle-mounted display device 1 is also connected to the shift lever 2, millimeter-wave radar 3, camera 4, and display device 5.

[0016] The shift lever 2, connected to the in-vehicle display device 1, is a device installed in a vehicle such as an automobile that accepts input from the vehicle's driver. By moving the shift lever 2, the driver can determine the direction of travel of their vehicle 999.

[0017] Camera 4 is mounted on the rear and front of the vehicle 999, as shown in Figures 2 and 3. Figure 2 is a diagram showing an example of the imaging range of camera 4 located on the rear of the vehicle 999. Figure 3 is a diagram showing an example of the imaging range of camera 4 located on the front of the vehicle 999. Camera 4 is equipped with a wide-angle lens that has a wide imaging range. As a result, it is possible to acquire images of an imaging range with a wide field of view centered on the mounting position of camera 4, as shown in the imaging range (detection range 310) in Figures 2 and 3. The image captured by camera 4 is an ambient image taken around the vehicle 999. Camera 4 captures images of the rear and front of the vehicle 999 and outputs an ambient image (hereinafter referred to as "image A"). The imaging range of camera 4 may include part or all of the left and right sides of the vehicle 999. Alternatively, camera 4 may be further mounted on the left and right sides of the vehicle 999 to capture images of the left and right sides of the vehicle 999.

[0018] Furthermore, as shown in Figure 1, a millimeter-wave radar 3 is connected to the in-vehicle display device 1. Similar to the camera 4, the millimeter-wave radar 3 is mounted, for example, on the rear and front of the vehicle 999 to acquire information about the rear and front of the vehicle 999. The detection range of the millimeter-wave radar 3 may include part or all of the left and right sides of the vehicle 999. Alternatively, the millimeter-wave radar 3 may be further mounted on the left and right sides of the vehicle 999 to acquire information about the left and right sides of the vehicle 999. The millimeter-wave radar 3 emits radio waves toward the surroundings of the vehicle 999 and receives reflected waves that are reflected by objects (hazardous areas) in the surroundings. Based on the reflected waves, the millimeter-wave radar 3 outputs the coordinates of the reflection points as sensing information regarding hazardous areas. Note that sonar or the like may be used instead of the millimeter-wave radar 3.

[0019] Furthermore, as shown in Figure 1, a display device 5 is connected to the in-vehicle display device 1. Although Figure 1 illustrates four display devices 5, the number of display devices 5 may be one, two, three, or five or more. The display devices 5 are positioned inside the vehicle 999 so that they are visible to the occupants. For example, the display device 5 may be an instrument panel positioned so that it is visible to the driver operating the vehicle 999. As the display device 5, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display can be used as appropriate. At least one of the display devices 5 may be configured integrally with the in-vehicle display device 1. In addition, a mobile device of the occupants, such as a smartphone, may be used as the display device 5.

[0020] The in-vehicle display device 1 includes a distortion correction unit 90. The distortion correction unit 90 receives "image A" from the camera 4 and obtains an image (hereinafter referred to as "image B") from which the distortion caused by the wide-angle lens has been removed. Any known technique can be used for removing the distortion caused by the wide-angle lens through image processing. For example, a conventional technique such as the one described in Patent Document 2, which corrects the distortion in the width direction of the image by shifting the left-right portion of the image vertically based on a specific position in the image, may be used. The method for removing the distortion of "image A" is not particularly limited here.

[0021] The in-vehicle display device 1 includes a direction acquisition unit 10 as a means for acquiring the direction of travel of the vehicle 999.

[0022] The direction of travel acquisition unit 10 according to the embodiment of this disclosure detects the direction of travel of the vehicle 999 from the shift state. For example, the direction of travel acquisition unit 10 acquires the direction of travel of the vehicle 999 by acquiring the position of the shift lever 2 included in the vehicle signal. For example, if the position detection of the shift lever 2 detects that it is in the D (drive) position, the direction of travel acquisition unit 10 detects that the direction of travel of the vehicle 999 is "forward". Also, if the position detection of the shift lever 2 detects that it is in the R (reverse) position, the direction of travel acquisition unit 10 acquires that the direction of travel of the vehicle 999 is "reverse". The method of acquiring the direction of travel is not particularly limited. For example, the direction of travel of the vehicle may be acquired by acquiring the rotation direction of the tires. Alternatively, the direction of travel may be acquired from changes in GPS location information.

[0023] The in-vehicle display device 1 includes, as means for acquiring information on hazardous locations, an acquisition unit 20, a detection rule setting unit 30, a hazardous location detection unit 40, a hypothetical hazardous location detection unit 50, and a contact risk determination unit 60.

[0024] The acquisition unit 20 acquires ambient information, such as images captured by the camera 4 ("Image A") and sensor information from the millimeter-wave radar 3. Ambient information refers to images and sensor information showing road conditions and obstacles around the vehicle 999. Alternatively, there may be a server located outside the vehicle 999 to which information from the millimeter-wave radar 3 and camera 4 is uploaded, and the acquisition unit 20 may acquire ambient information by receiving information from that server. In other words, the acquisition unit 20 may acquire ambient information using the camera 4 etc. mounted on the in-vehicle display device 1, or it may acquire ambient information from an external camera 4 etc. that is mounted on the vehicle 999 together with the in-vehicle display device 1 or located outside the vehicle 999.

[0025] The detection rule setting unit 30 sets the detection rules for the subsequent hazard detection unit 40 based on the direction of travel detected by the direction of travel acquisition unit 10. Specifically, if the direction of travel of the vehicle 999 is forward, the detection rule setting unit 30 sets the detection rules to detect only hazards in front of the vehicle 999. Also, as shown in Figure 2, if the direction of travel of the vehicle 999 is backward, the detection rule setting unit 30 sets the detection rules to detect only hazards behind the vehicle 999. Furthermore, since it is easier for the driver to check the situation in front compared to the situation behind, the detection rule setting unit 30 may set the detection range for hazards in front to be shorter (looser) than in the case of backward (reverse). Here, shortening the detection range means, for example, shortening the range of distance to be detected, and may also be expressed as shortening the detection distance, or narrowing the detection range in the front and rear directions of the vehicle 999. For example, as shown in Figures 2 and 3, the detection rule setting unit 30 may set the detection range 310 of dangerous areas to 5 [m] when the vehicle 999 is moving backward (reverse), and the detection range 310a of dangerous areas to 3 [m] when the vehicle 999 is moving forward (forward).

[0026] The hazard detection unit 40 identifies a set of coordinates of reflection points as a hazardous area based on the coordinates of the reflection points included in the sensing information received (acquired) by the acquisition unit 20 from the millimeter-wave radar 3 at a preset period. Furthermore, the hazard detection unit 40 detects the location, path, and speed of the hazardous area from "image A" captured by the camera 4, and also detects the hazardous area based on the surrounding information acquired by the acquisition unit 20 in accordance with the detection rules set by the detection rule setting unit 30. For example, the hazard detection unit 40 detects a hazardous area relative to the vehicle 999 based on the location, path, and speed of the hazardous area in the direction of travel of the vehicle 999 and the position, path, and speed of the vehicle 999, in accordance with the detection rules set by the detection rule setting unit 30.

[0027] The assumed hazard detection unit 50 analyzes "Image B" to determine the driving scene and detects whether the assumed hazard locations expected in the driving scene are included in the hazard locations detected by the hazard detection unit 40. First, the assumed hazard detection unit 50 determines the driving scene by analyzing "Image B" from the acquisition unit 20. For example, if it recognizes a parking space enclosed by white lines in the image, it determines that the driving scene is "parking".

[0028] The assumed hazard detection unit 50 detects whether the assumed hazard locations, which are assumed to be present in the driving scene determined by image analysis, are included among the hazard locations detected by the hazard location detection unit 40. If the assumed hazard location detection unit 50 does not detect the assumed hazard locations among the hazard locations, it communicates to the priority setting unit 70 (described later) to increase the display priority of locations that are likely to contain assumed hazard locations based on the analysis of "Image B". Figure 4 is a diagram illustrating the recognition of the driving scene and assumed hazard locations by the assumed hazard detection unit 50. For example, as shown in Figure 4, the assumed hazard detection unit 50 detects the white lines 402 of the parking lot from "Image B". These white lines 402 are an example of a characteristic object in a "parking" driving scene. If the assumed hazard detection unit 50 determines that the driving scene is "parking" based on the detection of the white lines 402, it detects that the assumed hazard location is the wheel stop 403. The assumed hazard detection unit 50 determines whether the wheel stop 403 is included in the hazard locations detected by the hazard detection unit 40. If the wheel stop 403 is not included in the hazard locations, it communicates to the subsequent priority setting unit 70 to increase the display priority of "locations where wheel stops are likely to be present". The driving scene and the location of assumed hazard locations are determined by referring to a database of characteristic objects and the arrangement relationships of assumed hazard locations for each driving scene, which is stored in a memory unit installed in the vehicle, for example. This memory unit may be located inside the in-vehicle display device 1, or it may be located in another in-vehicle device that is communicably connected to the in-vehicle display device 1 via an in-vehicle network such as Ethernet®. For example, a location where wheel stops are likely to be present may be determined, based on the data, to be 20 cm from the white line of the parking space. The driving scene may also be determined from the detection results of radar sensors, map data, steering sensors, etc.

[0029] Furthermore, even if a driving scene contains anticipated hazardous areas, if the anticipated hazard detection unit 50 does not detect any anticipated hazardous areas from among the hazardous areas detected by the hazardous area detection unit 40, the subsequent image generation unit 80 may create an image by synthesizing a virtual hazardous area in a location where the anticipated hazardous area is likely to exist. For example, if an anticipated hazardous area could not be detected due to a malfunction of the radar or other equipment, but considering the driving scene, there is a high probability that a specific anticipated hazardous area exists, the 3D data of an object indicating the specific anticipated hazardous area stored in the memory unit of the vehicle 999 may be read and synthesized into "Image B". This means that, for example, even if an anticipated hazardous area that should exist could not be detected due to a malfunction of the camera 4 or millimeter-wave radar 3, the driver can be appropriately shown the hazardous area by displaying an image of a location where the anticipated hazardous area is likely to exist.

[0030] The contact risk determination unit 60 determines, for each hazardous area detected by the hazardous area detection unit 40, the time until the vehicle 999 makes contact with the hazardous area and the distance between the hazardous area and the vehicle 999. In this embodiment, the contact risk is defined as information indicating the possibility of contact between the hazardous area and the vehicle 999, based on the "time until contact" and the "distance between the hazardous area and the vehicle." Here, the hazardous area that is the target of contact risk determination among the detected hazardous areas is referred to as the first hazardous area. Contact risk determination means calculating (detecting) the contact risk for the target first hazardous area based on the "time until contact" and the "distance between the hazardous area and the vehicle." This first hazardous area for contact risk determination is an example of a hazardous area to be displayed, and is an example of a hazardous area to be extracted as "Image C" described later.

[0031] The method for detecting the "time until contact" in the contact risk determination unit 60 will be explained. First, the case where the first hazardous area is a dynamic hazardous area will be explained. The first hazardous area here includes pedestrians, animals, tricycles, bicycles, motorcycles, other vehicles, etc. Based on the position, path, and speed of the first hazardous area and the position, path, and speed of the vehicle 999, the contact risk determination unit 60 determines the intersection point (hereinafter referred to as "intersection point X") where the path of the vehicle 999 and the path of the first hazardous area intersect. Next, the contact risk determination unit 60 determines the arrival time of the first hazardous area to intersection point X (hereinafter referred to as "arrival time t1") and the arrival time of the vehicle 999 to intersection point X (hereinafter referred to as "arrival time t2"). The contact risk determination unit 60 considers the smaller of arrival time t1 and arrival time t2 as the "time until contact". Furthermore, the contact risk determination unit 60 determines the distance between the first hazardous area and the vehicle 999, based on the location of the first hazardous area and the location of the vehicle 999, and defines this distance as the "distance between the hazardous area and the vehicle 999."

[0032] Next, we will explain the case where the first hazardous location is not a dynamic hazardous location. In this case, the first hazardous location may include static hazardous locations such as guardrails and utility poles, as well as hazardous road conditions present on the road surface, such as puddles, snow, ice, and potholes. If the contact risk determination unit 60 is unable to obtain the speed of the first hazardous location, or if the speed is below a predetermined threshold (for example, zero or near it), that is, if the first hazardous location is a static hazardous location or a hazardous road condition, the contact risk determination unit 60 determines whether the first hazardous location is on the path of the vehicle 999. If the first hazardous location is on the path of the vehicle 999, the location of the first hazardous location is defined as intersection X. Next, the contact risk determination unit 60 determines the "time until the vehicle 999 makes contact" with intersection X. Furthermore, the contact risk determination unit 60 determines the distance between the first hazardous area and the vehicle 999, based on the location of the first hazardous area and the location of the vehicle 999, and defines this distance as the "distance between the hazardous area and the vehicle 999."

[0033] Furthermore, the in-vehicle display device 1 includes a priority setting unit 70 and an image generation unit 80 as means for displaying information.

[0034] The priority setting unit 70 sets priorities based on the results of the hazardous location detection unit 40, the assumed hazardous location detection unit 50, and the contact risk determination unit 60. In other words, the priority setting unit 70 sets priorities according to the contact risk detection result, the assumed hazardous location detection result, the hazardous road condition detection result, and the direction of travel detection result of the vehicle 999. Priority is a parameter used in the image generation unit 80, described later, to determine the display mark for hazardous locations and the order of the display switching sequence. For example, priorities can be set within the range of "1" to "10," with "1" assigned to the hazardous location with the lowest priority and "10" assigned to the hazardous location with the highest priority. Priorities may also be determined as "high" to "low" rather than numerically.

[0035] If the assumed hazard detection unit 50 detects a driving scene in which an assumed hazard exists, but the hazard detection unit 40 detects no assumed hazard, the priority setting unit 70 increases the display priority of the portion of "Image B" that is presumed to contain an assumed hazard. For example, if the assumed hazard detection unit 50 determines the driving scene to be "parking," but the hazard detection unit 40 detects no assumed hazard, such as a "wheel stop," the priority setting unit 70 increases the display priority of the image (hereinafter referred to as "Image D") extracted from "Image B" in the same way as "Image C" described later. In other words, "Image C" is an image extracted from the image captured by camera 4 that was determined to contain a hazard. Also, "Image D" is an image that was not determined to contain a hazard or assumed hazard from the image captured by camera 4, but was extracted from the image as an assumed hazard based on the driving scene.

[0036] Furthermore, the priority setting unit 70 sets the priority for image generation or display for each hazardous area detected by the hazardous area detection unit 40, according to the time until contact with the first hazardous area or the distance to the first hazardous area calculated by the contact risk determination unit 60. The priority setting unit 70 sets a higher priority the shorter the time until contact with the hazardous area or the shorter the distance to the hazardous area.

[0037] The image generation unit 80 generates "Image C" (the first image) by extracting an image of a dangerous area detected by the dangerous area detection unit 40 from "Image B" based on the surrounding information of the direction of travel of the vehicle 999 detected by the direction of travel acquisition unit 10. "Image C" is an image extracted from "Image B" within a predetermined range such that the dangerous area is located in the horizontal center of "Image C". The predetermined range is, for example, a pre-set pixel range. The predetermined range may be a fixed value regardless of the distance to the dangerous area. Alternatively, the image may be cropped so that the entire dangerous area fits within the image. Furthermore, the higher the priority, the larger the predetermined range may be, i.e., the larger the image to be cropped. The same applies to the cropping of "Image D".

[0038] Next, the image generation unit 80 superimposes an image showing the field of view (an image related to the field of view) onto the displayed "Image C" or "Image D" so that the driver can understand the direction of the hazardous area. In the following explanation, the superimposition of the image related to the field of view onto "Image C" will be mainly illustrated, but the same image related to the field of view can also be superimposed onto "Image D". Figure 5 is a diagram showing an example of a display with the image related to the field of view superimposed. Figure 5 illustrates "Image C" with the image related to the field of view superimposed. In this case, "Image C" is an image that includes the hazardous area 200. As shown in Figure 5, a radar chart 210 showing the field of view range and the direction of the center of the field of view of "Image C" is superimposed on "Image C". The radar chart 210 is an example of an image related to the field of view of "Image C". As shown in Figure 5, the radar chart 210 has a semicircle centered on the vehicle 999. The radar chart 210 displays the field of view range 211 and the direction of the center of the field of view 212 of "Image C". In Figure 5, the shaded area indicates the field of view 211, and the solid line within the shaded area indicates the direction of the center of the field of view 212. Here, the radar chart 210 displays both the field of view 211 and the direction of the center of the field of view 212 of "Image C," but it is also possible to display only one of them.

[0039] Let's explain the radar chart 210 in more detail. Let's compare Figure 5 and Figure 6. Figure 6 is a diagram showing an example of the relationship between the imaging range of the camera 4 mounted on the vehicle 999 and the detection position of the hazardous area. In Figure 6, the dotted semicircle is the imaging range (detection range 310) of the camera 4. The shaded area is the display range 311, also displayed as "Image C". The line drawn in the display range 311 of the shaded area is a direction line 312 that represents the direction of the hazardous area 200, starting from the camera 4 in front of the vehicle 999. The direction of the hazardous area 200 indicates the direction of the center of the field of view of "Image C". At this time, the imaging range (detection range 310) in Figure 6 corresponds to the semicircle of the radar chart 210 in Figure 5. Similarly, the display range 311 in Figure 6 corresponds to the field of view range 211 in Figure 5. Furthermore, the direction line 312 in Figure 6 corresponds to the direction of the center of the field of view 212 in Figure 5.

[0040] Furthermore, the image generation unit 80 superimposes the radar chart 210 image onto "Image C" at a different position depending on the direction of travel of the vehicle 999. That is, it superimposes the radar chart 210 at a position corresponding to the direction of travel. The superimposition position of the radar chart 210 will be explained using Figures 6 to 9. Figure 6 is a diagram showing an example of the relationship between the range captured by the camera 4 mounted at the front of the vehicle and the detection position of a dangerous area. Figure 7 is a diagram showing an example of the range captured by the camera 4 at the rear of the vehicle. Figures 8 and 9 are diagrams showing examples of the positions at which the image generation unit 80 superimposes the radar chart 210 onto "Image C" according to the direction of travel. Figures 8 and 9 each illustrate "Image C" in a state where an image relating to the field of view is superimposed. As shown in Figure 6, when the direction of travel acquisition unit 10 determines that the direction of travel is "forward" and a hazardous area 200 is detected by the camera 4 mounted in front of the vehicle 999, the image generation unit 80 superimposes the radar chart 210 above the vertical center virtual line B of "Image C," as shown in Figure 8. In the case of Figure 8, it is superimposed above the hazardous area 200. This allows the driver to intuitively understand that the displayed hazardous area is located in front of their vehicle. Also, as shown in Figure 7, when the direction of travel acquisition unit 10 determines that the direction of travel is "rear" and the image of the hazardous area 200 is detected by the camera 4 mounted behind the vehicle 999, the image generation unit 80 superimposes the radar chart 210 below the vertical center virtual line B of "Image C," as shown in Figure 9. In the case of Figure 9, it is superimposed below the hazardous area 200. This allows the driver to intuitively understand that the displayed hazardous area is located behind the vehicle. In this way, the superimposed position of the radar chart 210 changes depending on the direction of travel (the direction in which the hazardous area is detected), allowing the driver to more intuitively grasp the location of the hazardous area. Furthermore, as shown in Figures 8 and 9, by placing the radar chart 210 near the hazardous area 200, the driver can intuitively understand the relationship between the hazardous area 200 and the radar chart 210. The same applies to the other examples below.

[0041] Furthermore, when the direction of travel is "forward," the radar chart 210 may be superimposed below the central virtual line B, and when the direction of travel is "rearward," the radar chart 210 may be superimposed above the central virtual line B.

[0042] The image generation unit 80 may superimpose the radar chart 210 to the side of "Image C" depending on the direction of travel. Figures 10 and 11 show examples of the position in which the image generation unit 80 superimposes the radar chart 210 on "Image C" depending on the direction of travel. Figures 10 and 11 each illustrate "Image C" with an image related to the field of view superimposed. If the direction of travel is forward, the radar chart may be superimposed to the left of the horizontal center virtual line A of the image, as shown in Figure 10, and if the direction of travel is backward, the radar chart may be superimposed to the right of the horizontal center virtual line A of the image, as shown in Figure 11. Alternatively, if the direction of travel is forward, the radar chart may be superimposed to the right of the horizontal center virtual line A of the image, and if the direction of travel is backward, the radar chart may be superimposed to the left of the horizontal center virtual line A of the image. By varying the superimposition position depending on the direction of travel, the driver can more intuitively grasp the location of dangerous areas.

[0043] Furthermore, the image generation unit 80 may change the orientation of the radar chart 210 according to the direction of travel, as shown in Figures 12 and 13. Figure 12 shows an example in which the image generation unit 80 superimposes an upwardly convex radar chart 210 when the direction of travel is forward. Figure 13 shows an example in which the image generation unit 80 superimposes a downwardly convex radar chart 210 when the direction of travel is backward. Figures 12 and 13 each illustrate "Image C" in a state where images relating to the field of view are superimposed. When the direction of travel is forward, the image generation unit 80 may superimpose an upwardly convex radar chart 210 relative to the vertical direction of the image, as shown in Figure 12. Also, when the direction of travel is backward, as shown in Figure 13, the image generation unit 80 may superimpose a downwardly convex radar chart 210 relative to the vertical direction of the image. Furthermore, while Figures 12 and 13 show how the superimposed position and the orientation of the radar chart 210 change according to the direction of travel, the configuration is not limited to this; the superimposed position may remain fixed while only the orientation of the radar chart 210 changes. This allows the driver to more intuitively grasp the location of hazardous areas relative to their own vehicle 999.

[0044] Furthermore, the radar chart 210 may be convex to the right when the direction of travel is forward, and convex to the left when the direction of travel is backward. Also, the radar chart 210 may be convex to the left when the direction of travel is forward, and convex to the right when the direction of travel is backward.

[0045] Furthermore, the image generation unit 80 may change the superimposition position of the radar chart 210 according to the direction of travel and the location of the detected hazard, as shown in Figures 14 and 15. Specifically, if the direction of travel is forward, the radar chart 210 is superimposed on the upper side; if the direction of travel is backward, the radar chart 210 is superimposed on the lower side; and if the hazard is to the left of the vehicle 999, the radar chart 210 is superimposed on the left side of "Image B"; and if the hazard is to the right of the vehicle 999, the radar chart 210 is superimposed on the right side of "Image B". Here, "the hazard is to the left of the vehicle 999" refers to "the left side within the detection range set according to the direction of travel". That is, if the direction of travel is forward, the hazard is to the left front of the vehicle; if the direction of travel is backward, the hazard is to the left rear of the vehicle. Here, "the hazard is to the right of the vehicle 999" refers to "the right side within the detection range set according to the direction of travel". In other words, when the direction of travel is forward, a dangerous location on the right side refers to the front right as seen from the vehicle's perspective, and when the direction of travel is backward, a dangerous location on the right side refers to the rear right as seen from the vehicle 999. Figures 14 and 15 show examples of how the image generation unit 80 superimposes the radar chart 210 according to the direction of travel and the location of the detected dangerous area. Figures 14 and 15 each illustrate "Image C" in a state where images related to the field of view are superimposed. In this case, if the direction of travel is forward and the detected dangerous area is located to the front left of the vehicle 999, the radar chart 210 is superimposed above the vertical center virtual line B of the image and to the left of the horizontal center virtual line A of the image, as shown in Figure 14. If the direction of travel is backward and the detected dangerous area is located to the rear right of the vehicle 999, the radar chart 210 is superimposed below the vertical center virtual line B of "Image C" and to the right of the horizontal center virtual line A of the image, as shown in Figure 15. This allows drivers to more intuitively understand the location of hazards relative to their own vehicle, the 999.

[0046] Furthermore, if the detected hazardous area is directly in front of the vehicle 999, the radar chart 210 may be superimposed near the center. The area near the center is, for example, the position where it overlaps with the second block from the right when "Image C" is divided horizontally into three sections.

[0047] Furthermore, the superimposition position and orientation (direction of convexity) of the radar chart 210 relative to "Image C" may be determined such that the field of view 211 and the direction of the center of the field of view 212 shown by the radar chart 210 are directed toward "Image C". For example, in the example of Figure 14, an upwardly convex radar chart 210 may be superimposed on the lower right side of "Image C". For example, in the example of Figure 15, a downwardly convex radar chart 210 may be superimposed on the upper left side of "Image C". Even with this configuration, the driver can more intuitively grasp the location of hazardous areas relative to their vehicle 999.

[0048] Furthermore, the field of view information superimposed on "Image C" does not necessarily have to be in the form of a semicircular radar chart 210. For example, the detection range and the direction and location of dangerous areas may be indicated in text. Alternatively, a spherical field of view diagram, such as that of a fisheye lens, may be used.

[0049] Furthermore, the image generation unit 80 generates an image with a mark 220 superimposed on it, whose shape and color can be switched as shown in Figure 5, according to the priority set by the priority setting unit 70, or the time until contact with the hazardous area detected by the contact risk determination unit 60, or the distance to the hazardous area, and outputs this image to the display device 5. The output image may be an image representing the screen display itself with the mark 220 superimposed on "Image C", or it may be an image of "Image C" and the mark 220 superimposed on "Image C", each constituting a part of the screen display. In other words, the in-vehicle display device 1 may output the image displayed on the screen, or it may output display information used for the screen display. This display information used for the screen display includes, for example, "Image C" and the image of the mark 220 superimposed on "Image C". This display information may also include layout information that defines the arrangement of these images. The mark 220 highlights the hazardous areas in the image to draw the attention of the occupants. This allows the occupants to recognize the urgency of each hazardous area.

[0050] While Figure 5 illustrates an example of a mark 220 superimposed on an image, the marks superimposed on the image are not limited to this. The shape and color of the mark 220 superimposed on the image may be defined according to priority. Figure 16 shows an example of a data table in which the shape and color of the marks are defined according to priority. As shown in Figure 16, for example, if the priority is set between "1" and "10", then for a dangerous location with a priority of "10" (highest priority), the mark will be a triple triangle and the color will be set to yellow to indicate high priority. For a dangerous location with a priority of "5" (medium priority), the mark will be a double triangle and the color will be set to green. On the other hand, for a dangerous location with a priority of "1" (lowest priority), the mark will be a triangle and the color will be set to blue to indicate low priority. The color of the marks may be a gradient between blue, green, and yellow according to priority.

[0051] Furthermore, the shape and color of the mark 220 superimposed on the image may be defined according to the time or distance to contact. Figure 17 is an example of a data table that defines the shape and color of the mark 220 for each time or distance to contact. As shown in Figure 17, if the time to contact is less than 3 seconds or the distance to contact is less than 3 m, the image generation unit 80 sets a red mark 220 that is a rectangle surrounding the hazardous area to indicate high priority. If the time to contact is 3 seconds or more but less than 10 seconds or the distance to contact is 3 m or more but less than 6 m, the image generation unit 80 sets an orange mark 220 that is a bracket covering the hazardous area from above and below to indicate medium priority. If the time to contact is 10 seconds or more or the distance to contact is 6 m or more, the yellow mark 220 is on one side of the bracket to indicate low priority. In this way, by displaying the mark 220 and the radar chart 210 in "Image C," the driver can be simultaneously presented with the direction and priority of dangerous areas. Note that the shape and color of the mark 220 are merely examples and are not limited to these.

[0052] Furthermore, the superimposed marks may flash. In this case, the flashing speed may change depending on the priority, time to contact, and distance.

[0053] (Display control-related processing in the vehicle display control device 1) Next, we will explain an example of the flow of processing related to the display control of the in-vehicle display device 1 (referred to as the display control processing). Figure 18 is a flowchart showing an example of the operation of the in-vehicle display device 1 according to the embodiment.

[0054] As shown in Figure 18, the in-vehicle display device 1 starts its operation when power is turned on. First, the direction of travel acquisition unit 10 detects the direction of travel of the vehicle 999 from the shift state (step S111).

[0055] Next, the acquisition unit 20 acquires the image captured by the camera 4 ("Image A") and surrounding information such as the area above the sensor of the millimeter-wave radar 3 (step S112).

[0056] Next, the distortion correction unit 90 generates an image (referred to as "image B") from which lens distortion has been removed from the image acquired by the acquisition unit 20 from the camera 4 (step S113).

[0057] Next, the detection rule setting unit 30 sets the detection rule based on the direction of travel of the vehicle 999 acquired by the direction of travel acquisition unit 10 in step S111 (step S114).

[0058] Next, the hazard detection unit 40 determines the detection range according to the detection rules set in step S114, and detects the presence and location of hazardous areas from "image B" or other captured images taken by camera 4 within the detection range through image recognition processing (step S115).

[0059] Next, the assumed hazard detection unit 50 detects whether there are assumed hazard locations within the detected hazard locations. The assumed hazard detection unit 50 infers the current driving scene from "Image B" or other captured images taken by Camera 4. The assumed hazard detection unit 50 detects whether there are hazard locations (assumed hazard locations) that are highly likely to exist in the inferred current driving scene within the hazard locations detected in step S115 (step S116). The detection results are used in the subsequent priority setting unit 70.

[0060] Next, the contact risk determination unit 60 identifies the hazardous area to be used for calculating the contact risk from the hazardous areas detected in step S115 as the first hazardous area, calculates the time until the vehicle 999 makes contact with the first hazardous area, and calculates the distance between the first hazardous area and the vehicle 999 (step S117).

[0061] Next, the priority setting unit 70 determines the priority of image generation and display order based on the direction of travel, the presence or absence of anticipated hazardous areas, and the risk of contact. The priority setting unit 70 sets the priority based on the detection results of the contact risk determination unit 60 in step S117, the anticipated hazardous area detection unit 50 in step S116, and the direction of travel acquisition unit 10 in step S111 (step S118).

[0062] Next, the image generation unit 80 generates images corresponding to each dangerous area, where the extracted dangerous area is at the horizontal center of the image and a certain range is cropped regardless of distance. Specifically, based on the priority set in step S118, for each of the four dangerous areas with the highest priority from the first dangerous areas, a predetermined pixel range is cropped from "image B" so that the dangerous area is at the horizontal center of the image, and "image C" is generated (step S119). Note that the number of images to be generated is not limited to four and is merely an example. For example, "image C" may be generated for the single dangerous area with the highest priority.

[0063] Next, the image generation unit 80 superimposes a semicircular radar chart 210 centered on the vehicle 999 onto "Image C" as information about the field of view of "Image C" (step S120). A detailed flowchart of S120 is shown in Figure 19. Figure 19 is a flowchart showing an example of the process flow in which the image generation unit 80 superimposes images related to the field of view. First, for each generated "Image C", a radar chart 210 is created that shows the range of the extracted "Image C" in "Image B" and the direction of the dangerous area (step S1201). After step S1201, it is determined whether the direction of travel detected by the direction of travel acquisition unit 10 is forward (step S1202). If the direction of travel is forward (YES in step S1202), it is determined whether the dangerous area in "Image C" is located to the left of the vehicle 999 (step S1203a). If the hazardous area is to the left of the vehicle 999 (YES in step S1203a), the image generation unit 80 superimposes the radar chart 210 onto the upper left side of the generated "image C" (step S1204a). The process then proceeds to step S121.

[0064] If the hazardous area is not to the left of the vehicle 999, that is, to the right of the vehicle 999 (NO in step S1203a), the image generation unit 80 superimposes the radar chart 210 onto the upper right side of the generated image (step S1204b). The process then proceeds to step S121.

[0065] If the direction of travel is not forward, i.e., if the direction of travel is backward (NO in step S1202), it is determined whether the hazardous area in the image is located to the left of the vehicle 999 (step S1203b). If the hazardous area is to the left of the vehicle 999 (YES in step S1203b), the image generation unit 80 superimposes the radar chart 210 on the lower left side of the generated image (step S1204c). The process then proceeds to step S121.

[0066] If the hazardous area is not to the left of the vehicle 999, that is, to the right of the vehicle 999 (NO in step S1203b), the image generation unit 80 superimposes the radar chart 210 onto the upper right side of the generated image (step S1204d). The process then proceeds to step S121.

[0067] Next, as shown in Figure 18, the image generation unit 80 superimposes marks 220, defined in the tables shown in Figures 16 and 17, onto "Image C" so that their shape and color change according to the time and distance to contact. The marks 220 are superimposed on the pixel range corresponding to the hazardous areas in "Image C" (step S121). Furthermore, the image generation unit 80 may also indicate the presence of other hazardous areas not shown in "Image C" using arrows 230, such as left-side hazardous area information and right-side hazardous area information, as shown in Figures 5 and 8 to 15. In addition, a number indicating the number of hazardous areas may be displayed within the arrows 230.

[0068] Next, the image generation unit 80 outputs "Image C" with the mark superimposed on it to the external display device 5 (step S122). At this time, the images may be output in such a way that the order in which multiple "Image C" images are displayed to the external display device 5 is determined by the priority set in step S118. These multiple "Image C" images can be displayed sequentially each time the arrow 230 is operated. Note that the display order of the images may be determined not by priority, but by the distance to the hazardous area, etc.

[0069] The process from step S111 to step S122 is a periodic loop process. After the process up to step S122 is completed, the system waits until "image A" from camera 4 is input, and then the process from step S111 to step S122 is restarted.

[0070] (Variation 1) In the example described above, the in-vehicle display device 1 has a detection rule setting unit 30, and the control of the hazardous area detection unit 40 has been explained. On the other hand, the in-vehicle display device 1 may also have a judgment rule setting unit. The judgment rule setting unit sets judgment rules, which are used as criteria for determining a first hazardous area that is the target of calculating contact risk from among the hazardous areas present around the vehicle 999, according to the direction of travel of the vehicle 999. The contact risk determination unit 60 also determines the first hazardous area that is the target of calculating contact risk from among the hazardous areas present around the vehicle 999 according to the judgment rules. In other words, the contact risk determination unit 60 narrows down the hazardous areas to be displayed, i.e., the hazardous areas to be extracted as "image C", from among the hazardous areas present around the vehicle 999 according to the judgment rules.

[0071] The control flow of the in-vehicle display device 1 having a judgment rule setting unit will be explained below with reference to Figure 20. Figure 20 is a flowchart showing an example of the operation of the in-vehicle display device 1 according to Modification 1 of the Embodiment. Steps that perform the same operation may be omitted from the explanation.

[0072] Steps S211 to S213 are identical to steps S111 to S113 in Figure 18, respectively. Then, in step S214, the hazard detection unit 40 detects hazardous areas around the vehicle from the information of the acquisition unit 20, similar to step S115 in Figure 18.

[0073] In step S215, the assumed hazard area detection unit detects the assumed hazard area in the same manner as in step S116 in Figure 18. The judgment rule setting unit sets the distance at which the subsequent contact risk determination unit 60 determines the contact risk, based on the direction of travel of the vehicle 999 obtained in step S211 (step S216).

[0074] For example, when the preceding hazard detection unit 40 has detected hazards up to a distance of 5 [m] for determining contact risk, and the direction of travel is forward, the judgment rule setting unit sets the distance for determining contact risk to 3 [m], which is shorter than when the direction of travel is backward, because the driver can easily check the situation ahead themselves. When the direction of travel of the vehicle 999 is backward, the judgment rule setting unit sets the distance for determining contact risk to 5 [m], which is longer than when the direction of travel is forward. Here, increasing the judgment distance means, for example, increasing the range of distances at which contact risk is determined, and this can also be expressed as making the judgment range longer (stricter), or widening the judgment range in the front and rear directions of the vehicle 999.

[0075] The in-vehicle display device 1 may have a determination rule setting unit instead of a detection rule setting unit 30, or it may have both a detection rule setting unit 30 and a determination rule setting unit. The determination distance defined in the determination rule is, for example, equal to the detection range defined in the detection rule, but may be any other length within that detection range.

[0076] The subsequent steps S217 to S222 are identical to steps S117 to S122 in Figure 18, respectively.

[0077] (Modification 2) Furthermore, the in-vehicle display device 1 according to this embodiment may have a road condition detection unit that detects dangerous road conditions (snow, puddles, ice, snow accumulation) for the vehicle 999 from among dangerous locations based on surrounding information such as images captured by the camera 4. If the road condition detection unit detects a dangerous road condition among the dangerous locations detected by the dangerous location detection unit 40, it transmits this information to the priority setting unit 70 to increase the priority of the dangerous road condition.

[0078] The road condition detection unit detects dangerous road conditions by analyzing "Image A". If the road condition detection unit detects a dangerous area, the priority setting unit 70 may increase the display priority of the dangerous area detected by the road condition detection unit. This makes it possible to inform the driver of dangerous road surface conditions that are flat and difficult for the driver to see.

[0079] (Effects and Benefits) Next, the effects and advantages of the in-vehicle display device 1 of this disclosure will be explained.

[0080] As described above, the in-vehicle display device 1 of the technology 1 disclosed herein comprises an acquisition unit 20, a direction of travel acquisition unit 10, and an image generation unit 80. The acquisition unit 20 acquires information about the surroundings of the vehicle 999. The direction of travel acquisition unit 10 acquires the direction of travel of the vehicle 999. The image generation unit 80 generates an "image C" relating to dangerous areas around the vehicle 999 from the images included in the surrounding information acquired by the acquisition unit 20. The image generation unit 80 superimposes an image relating to the field of view of "image C" onto "image C". The image generation unit 80 varies the superposition position of the image relating to the field of view of "image C" onto "image C" according to the direction of travel of the vehicle 999.

[0081] According to this, the superposition position of images related to the field of view differs depending on the direction of travel. Therefore, the driver can intuitively recognize the direction of dangerous areas based on the superposition position.

[0082] Furthermore, the in-vehicle display device 1 of Technology 2 relating to this disclosure is the in-vehicle display device 1 described in Technology 1. In addition, in the in-vehicle display device 1 of Technology 2, if the direction of travel is acquired by the direction of travel acquisition unit 10, the image generation unit 80 superimposes the image relating to the field of view of "Image C" above the vertical center of "Image C". Also, if the direction of travel is detected by the direction of travel acquisition unit 10, the image generation unit 80 superimposes the image relating to the field of view of "Image C" below the vertical center of "Image C".

[0083] According to this, when the direction of travel is forward, an image related to the field of view is superimposed on the top of "Image C," and when the direction of travel is backward, an image related to the field of view is superimposed on the bottom of "Image C." Thus, the superimposition position differs in the vertical direction of the display. Therefore, the driver can intuitively recognize the direction of a hazardous area based on the superimposition position on either the upper or lower side, which corresponds to the direction of travel (forward or backward).

[0084] Furthermore, the in-vehicle display device 1 of Technology 3 relating to this disclosure is the in-vehicle display device 1 described in Technology 1 or Technology 2. In addition, in the in-vehicle display device 1 of Technology 3, if a hazardous area is detected to the left of the vehicle 999, the image generation unit 80 superimposes an image relating to the field of view to the left of the horizontal center of "Image C". Also, if a hazardous area is detected to the right of the vehicle 999, the image generation unit 80 superimposes an image relating to the field of view to the right of the horizontal center of "Image C".

[0085] According to this, the superimposed position of the image related to the field of view differs in the vertical and horizontal directions depending on the direction of travel and the direction in which the hazard is detected. Therefore, the superimposed position on either the upper or lower side corresponding to the direction of travel (forward or reverse) shifts to either the left or right side depending on the direction in which the hazard is detected, allowing the driver to intuitively recognize the direction of the hazard more effectively.

[0086] Furthermore, the in-vehicle display device 1 of Technology 4 in this disclosure is the same as the in-vehicle display device 1 described in Technology 1. In addition, in the in-vehicle display device 1 of Technology 4, if the direction of travel is detected by the direction of travel acquisition unit 10, the image generation unit 80 superimposes the image relating to the field of view of "Image C" to the left of the horizontal center of "Image C". Also, if the direction of travel is detected by the direction of travel acquisition unit 10, the image generation unit 80 superimposes the image relating to the field of view of "Image C" to the right of the horizontal center of "Image C".

[0087] According to this, the superposition position of the images related to the field of view differs in the left-right direction depending on the direction of travel. Therefore, the driver can intuitively recognize the direction of the hazardous area by the superposition position on either the left or right side corresponding to the forward and reverse directions of travel. In addition, in the in-vehicle display device 1 of Technology 4, the image generation unit 80 may, in the same manner as in Technology 3, display the superposition position in the left-right direction corresponding to the direction of travel shifted to either the up or down side according to the detection direction of the hazardous area.

[0088] Furthermore, the in-vehicle display device 1 of Technology 5 in this disclosure is the in-vehicle display device 1 described in Technology 1. In addition, in the in-vehicle display device 1 of Technology 5, if the direction of travel is detected by the direction of travel acquisition unit 10, the image generation unit 80 superimposes the image relating to the field of view of "Image C" to the right of the horizontal center of "Image C". Also, if the direction of travel is detected by the direction of travel acquisition unit 10, the image generation unit 80 superimposes the image relating to the field of view of "Image C" to the left of the horizontal center of "Image C".

[0089] According to this, the superposition position of the image related to the field of view differs in the left-right direction depending on the direction of travel. Therefore, the driver can intuitively recognize the direction of the hazardous area by the superposition position on either the right or left side corresponding to the forward and reverse directions of travel. In addition, in the in-vehicle display device 1 of Technology 5, the image generation unit 80 may, in the same manner as in Technology 3, display the superposition position in the left-right direction corresponding to the direction of travel shifted to either the up or down side according to the detection direction of the hazardous area.

[0090] Furthermore, the in-vehicle display device 1 of Technology 6 relating to this disclosure is the in-vehicle display device 1 described in any one of the technologies from Technology 1 to Technology 5. In addition, in the in-vehicle display device 1 of Technology 6, the image relating to the field of view of "Image C" is a semicircular image showing a semicircular region centered on the vehicle 999. The image generation unit 80 superimposes the semicircular image, which is convex, onto the display side corresponding to the direction of travel acquired by the direction of travel acquisition unit 10. For example, if the direction of travel acquisition unit 10 acquires that the direction of travel is forward, the image generation unit 80 superimposes the semicircular image, which is convex upward. If the direction of travel acquisition unit 10 acquires that the direction of travel is rearward, the image generation unit 80 superimposes the semicircular image, which is convex downward.

[0091] According to this, the format of the field of view differs depending on the direction of travel. Therefore, the driver can intuitively recognize the direction of dangerous areas based on the differences in the format of the field of view.

[0092] Furthermore, the in-vehicle display device 1 of Technology 7 relating to this disclosure is the in-vehicle display device 1 described in any one of the technologies from Technology 1 to Technology 6. The in-vehicle display device 1 of Technology 7 further comprises a determination rule setting unit and a contact risk determination unit 60. The determination rule setting unit sets determination rules for determining a first hazardous location to be used for calculating contact risk from among the hazardous locations present around the vehicle 999, according to the direction of travel of the vehicle 999. The contact risk determination unit 60 determines the first hazardous location for calculating contact risk according to the determination rules and detects the contact risk, including the time until the determined first hazardous location and the vehicle 999 come into contact.

[0093] According to this, the in-vehicle display device 1 determines the risk of contact with surrounding hazardous areas and sets the range for determining the risk of contact according to the direction of travel. Since the ease with which a driver can check the situation differs between the front and the rear, setting judgment rules according to the direction of travel allows for the appropriate determination of hazardous areas.

[0094] Furthermore, the in-vehicle display device 1 of Technology 8 in this disclosure is the in-vehicle display device 1 described in Technology 7. In the in-vehicle display device 1 of Technology 8, the judgment rule setting unit sets the judgment rules such that when the direction of travel is determined to be forward by the direction of travel acquisition unit 10, the judgment range for determining a first dangerous area is shorter than when the direction of travel is determined to be rearward by the direction of travel acquisition unit 10.

[0095] According to this, the in-vehicle display device 1 shortens the contact risk detection range for dangerous areas in front and lengthens the contact risk detection range for dangerous areas behind. Since the ease with which a driver can check the situation differs between the front and the rear, setting judgment rules according to the direction of travel allows for appropriate detection of dangerous areas.

[0096] Furthermore, the in-vehicle display device 1 of Technology 9 relating to this disclosure is the in-vehicle display device 1 described in any one of the technologies from Technology 1 to Technology 8. The in-vehicle display device 1 of Technology 9 further comprises a detection rule setting unit 30 and a hazardous location detection unit 40. The detection rule setting unit 30 sets a detection rule for detecting hazardous locations based on surrounding information according to the direction of travel of the vehicle 999. The hazardous location detection unit 40 detects hazardous locations based on surrounding information acquired by the acquisition unit 20 in accordance with the set detection rule.

[0097] According to this, the in-vehicle display device 1 detects dangerous areas from surrounding information and sets the detection range for dangerous areas according to the direction of travel. Since the ease with which a driver can check the situation differs between the front and the rear, setting detection rules according to the direction of travel allows for the detection of appropriate dangerous areas.

[0098] Furthermore, the in-vehicle display device 1 of Technology 10 relating to this disclosure is the in-vehicle display device 1 described in Technology 9. In the in-vehicle display device 1 of Technology 10, the detection rule setting unit 30 sets the detection rules such that, when the direction of travel is determined to be forward by the direction of travel acquisition unit 10, the detection range to be detected as a dangerous area is shorter than when the direction of travel is determined to be forward by the direction of travel acquisition unit 10.

[0099] According to this, the in-vehicle display device 1 shortens the detection range for hazardous areas based on the surrounding information in front and lengthens the detection range for the surrounding information behind. Since the ease with which the driver can check the situation differs between the front and the rear, by setting detection rules according to the direction of travel, it is possible to detect appropriate hazardous areas.

[0100] Furthermore, the in-vehicle display device 1 of Technology 11 relating to this disclosure is the in-vehicle display device 1 described in any one of the technologies from Technology 1 to Technology 10. The in-vehicle display device 1 of Technology 11 further comprises a hypothetical hazard location detection unit 50. The hypothetical hazard location detection unit 50 recognizes a driving scene based on the surrounding information acquired by the acquisition unit 20 and detects whether the hypothetical hazard locations expected in the recognized driving scene are included in the detected hazard locations.

[0101] According to this, it is possible to display appropriate hazardous areas depending on the driving situation.

[0102] Furthermore, the in-vehicle display device 1 of Technology 12 in this disclosure is the in-vehicle display device 1 described in Technology 11. The in-vehicle display device 1 of Technology 12 further comprises a priority setting unit 70. The priority setting unit 70 sets the priority for generating or displaying images of multiple hazardous locations in the direction of travel. If a hypothetical hazardous location in a driving scene recognized by the hypothetical hazardous location detection unit 50 is not detected among the hazardous locations, the priority setting unit 70 sets the priority of the hypothetical hazardous location higher than that of the other hazardous locations.

[0103] According to this, even if a potential hazardous area in a driving scenario cannot be properly detected due to a malfunction of the millimeter-wave radar 3, the presence of the hazardous area can still be notified to the driver.

[0104] Furthermore, the in-vehicle display device 1 of Technology 13 in this disclosure is the in-vehicle display device 1 described in Technology 12. In the in-vehicle display device 1 of Technology 13, the assumed hazard location detection unit 50 detects whether a wheel stop, which is an assumed hazard location for parking, is included in the hazard locations when the driving scene is parking. If no wheel stop is detected among the hazard locations, the priority setting unit 70 sets the priority of the location where a wheel stop is assumed to be located higher than that of other hazard locations.

[0105] According to this, even if the wheel stop is not properly detected in a parking scene due to a malfunction of the millimeter-wave radar 3, the presence of the wheel stop can still be notified to the driver.

[0106] Furthermore, the in-vehicle display device 1 of Technology 14 relating to this disclosure is the in-vehicle display device 1 described in any one of the technologies from Technology 1 to Technology 13. The in-vehicle display device 1 of Technology 14 further comprises a road condition detection unit and a priority setting unit 70. The road condition detection unit detects a road condition that is dangerous to the vehicle 999 from among the dangerous locations detected from "image B" which shows the surrounding conditions of the vehicle 999 included in the surrounding information. The priority setting unit 70 sets the priority for image generation or display of multiple dangerous locations in the direction of travel. If the road condition detection unit detects a dangerous road condition from among the dangerous locations in the direction of travel of the vehicle 999, the priority setting unit 70 sets the priority of the dangerous road condition higher than that of the other dangerous locations.

[0107] According to this, dangerous road conditions present on road surfaces that are nearly flat and difficult for drivers to see can be appropriately displayed to the driver.

[0108] Furthermore, the in-vehicle display device 1 of Technology 16 relating to this disclosure is the in-vehicle display device 1 described in any one of the technologies from Technology 1 to Technology 15. The in-vehicle display device 1 of Technology 16 further comprises a contact risk determination unit 60, a hypothetical hazardous location detection unit 50, a road condition detection unit, and a priority setting unit 70. The contact risk determination unit 60 detects the contact risk, including the time until the vehicle 999 makes contact with a target first hazardous location among the hazardous locations in the direction of travel of the vehicle 999. The hypothetical hazardous location detection unit 50 recognizes the driving scene based on the surrounding information acquired by the acquisition unit 20 and detects whether the hypothetical hazardous locations expected in the recognized driving scene are included in the hazardous locations. The road condition detection unit detects road conditions that are dangerous to the vehicle 999 from among the hazardous locations detected from the image showing the surrounding conditions of the vehicle 999 included in the surrounding information. The priority setting unit 70 sets the priority for image generation or display of multiple hazardous locations in the direction of travel. The priority setting unit 70 sets priorities according to the contact risk detection result by the contact risk determination unit 60, the assumed hazardous location detection result by the assumed hazardous location detection unit 50, the hazardous road condition detection result by the road condition detection unit, and the acquisition result of the direction of travel of the vehicle 999 by the direction of travel acquisition unit 10.

[0109] According to this, dangerous locations and objects while driving can be appropriately displayed to the driver.

[0110] Furthermore, the display control method of Technology 17 relating to this disclosure is a method executed by a computer (in-vehicle display device 1) mounted on the vehicle 999. The display control method of Technology 17 acquires information about the surroundings of the vehicle 999, acquires the direction of travel of the vehicle 999, generates an "image C" relating to dangerous areas around the vehicle 999 from the images included in the acquired surrounding information, superimposes an image relating to the field of view of "image C" onto "image C", and varies the superposition position of the image relating to the field of view of "image C" onto "image C" according to the direction of travel of the vehicle 999.

[0111] According to this, the superposition position of images related to the field of view differs depending on the direction of travel. Therefore, the driver can intuitively recognize the direction of dangerous areas based on the superposition position.

[0112] (Other variations) The embodiments described above are in all respects merely illustrative and should not be interpreted restrictively. Furthermore, any variations or modifications that fall within the equivalent scope of the claims are all within the scope of this disclosure.

[0113] For example, the detection rule setting unit and the judgment rule setting unit may set detection rules or judgment rules based on information obtained from environmental sensors such as temperature sensors and humidity sensors. This information obtained from environmental sensors may also be included in the surrounding information of the vehicle 999. For example, based on the information obtained from environmental sensors, if the weather is rainy, the detection range or judgment range may be expanded in addition to setting the detection range or judgment range according to the direction of travel. Furthermore, information about the environment, including weather, may be obtained from the car navigation system.

[0114] The program executed by the in-vehicle display device 1 of the above-described embodiment may be provided as a file in an installable or executable format, recorded on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0115] Furthermore, the program executed by the in-vehicle display device 1 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program executed by the in-vehicle display device 1 of the above-described embodiment may be provided or distributed via a network such as the Internet.

[0116] Furthermore, the program executed by the in-vehicle display device 1 of the above-described embodiment may be pre-installed and provided in ROM or the like. [Explanation of symbols]

[0117] 1 Automotive display device 2 Shift lever 3 mm wave radar 4 cameras 5 Display device 10 Traveling direction acquisition section 20 Acquisition Department 30 Detection Rule Setting Section 40 Hazardous Area Detection Unit 50. Unit for detecting anticipated hazardous areas 60 Contact Risk Determination Unit 70 Priority setting section 80 Image generation unit 90 Distortion Correction Section 999 My vehicle

Claims

1. An acquisition unit that acquires information about the surroundings of the vehicle, The aforementioned direction acquisition unit acquires the direction of travel of the vehicle, The system includes an image generation unit that generates a first image relating to dangerous areas around the vehicle from the image included in the surrounding information acquired by the acquisition unit, The image generation unit, The image relating to the field of view of the first image is superimposed onto the first image. Depending on the direction of travel of the vehicle, the superposition position of the images related to the field of view of the first image is changed. Automotive display device.

2. The image generation unit, If the direction of travel is determined to be forward by the direction of travel acquisition unit, the image relating to the field of view of the first image is superimposed above the vertical center of the first image. If the direction of travel is determined to be rearward by the direction of travel acquisition unit, the image relating to the field of view of the first image is superimposed below the vertical center of the first image. The in-vehicle display device according to claim 1.

3. The image generation unit, If the aforementioned hazardous area is detected to the left of the vehicle, the image relating to the field of view of the first image is superimposed to the left of the horizontal center of the first image. If the aforementioned hazardous area is detected to the right of the vehicle, the image relating to the field of view of the first image is superimposed to the right of the horizontal center of the first image. The in-vehicle display device according to claim 2.

4. The image generation unit, If the direction of travel is determined to be forward by the direction of travel acquisition unit, the image relating to the field of view of the first image is superimposed to the left of the horizontal center of the first image. If the direction of travel is determined to be rearward by the direction of travel acquisition unit, the image relating to the field of view of the first image is superimposed to the right of the horizontal center of the first image. The in-vehicle display device according to claim 1.

5. The image generation unit, If the direction of travel is determined to be forward by the direction of travel acquisition unit, the image relating to the field of view of the first image is superimposed to the right of the horizontal center of the first image. If the direction of travel is determined to be backward by the direction of travel acquisition unit, the image relating to the field of view of the first image is superimposed to the left of the horizontal center of the first image. The in-vehicle display device according to claim 1.

6. The image relating to the field of view of the first image is a semicircular image showing a semicircular region centered on the vehicle itself. The image generation unit, If the direction of travel is determined to be forward by the direction of travel acquisition unit, the upward-convex semicircular image is superimposed. If the direction of travel is determined to be the rear by the direction of travel acquisition unit, the downward-convex semicircular image is superimposed. The in-vehicle display device according to claim 1.

7. A determination rule setting unit sets determination rules for determining a first hazardous area, which is the target for calculating the risk of contact, from among the hazardous areas present around the vehicle, according to the direction of travel of the vehicle. The system further includes a contact risk determination unit that determines the first hazardous area according to the established determination rules and detects the contact risk, including the time until the determined first hazardous area comes into contact with the vehicle. An in-vehicle display device according to any one of claims 1 to 6.

8. The judgment rule setting unit sets the judgment rule such that, when the direction of travel is determined to be forward by the direction of travel acquisition unit, the judgment range to be determined as the first dangerous area is shorter than when the direction of travel is determined to be backward by the direction of travel acquisition unit. The in-vehicle display device according to claim 7.

9. A detection rule setting unit sets detection rules for detecting dangerous locations according to the direction of travel of the vehicle, The system further comprises a hazardous location detection unit that detects the hazardous location based on the surrounding information acquired by the acquisition unit in accordance with the set detection rules. An in-vehicle display device according to any one of claims 1 to 6.

10. The detection rule setting unit sets the detection rule such that, when the direction of travel is determined to be forward by the direction of travel acquisition unit, the detection range for detecting the hazardous area is shorter than when the direction of travel is determined to be backward by the direction of travel acquisition unit. The in-vehicle display device according to claim 9.

11. The system further includes a potential hazard detection unit that recognizes a driving scene based on the surrounding information acquired by the acquisition unit and detects whether a potential hazard location anticipated in the recognized driving scene is included in the hazard location. An in-vehicle display device according to any one of claims 1 to 6.

12. The system further includes a priority setting unit that sets the priority for generating or displaying images of multiple hazardous locations in the aforementioned direction of travel, If the assumed hazardous location in the driving scene recognized by the assumed hazardous location detection unit is not detected among the hazardous locations, the priority setting unit sets the priority of the assumed hazardous location higher than that of the other hazardous locations. The in-vehicle display device according to claim 11.

13. The aforementioned assumed hazard location detection unit detects whether the assumed hazard location for parking, namely the wheel stop, is included in the hazard location if the driving scene is parking. If no wheel stop is detected among the aforementioned hazardous locations, the priority setting unit sets the priority of the location where the wheel stop is presumed to be located higher than that of the other aforementioned hazardous locations. The in-vehicle display device according to claim 12.

14. A road condition detection unit detects dangerous road conditions for the vehicle from among the dangerous locations detected from the image showing the surrounding conditions of the vehicle included in the surrounding information, The system further includes a priority setting unit for setting the priority of generating or displaying images of multiple hazardous locations in the direction of travel, If the road condition detection unit detects a dangerous road condition in the direction of travel of the vehicle, the priority setting unit sets the priority of the dangerous road condition higher than that of other dangerous locations. An in-vehicle display device according to any one of claims 1 to 6.

15. A contact risk determination unit that detects the contact risk, including the time until the vehicle comes into contact with the target first hazardous area among the hazardous areas in the direction of travel of the vehicle, Based on the surrounding information acquired by the acquisition unit, the assumed hazard detection unit recognizes a driving scene and detects whether assumed hazard locations are included in the recognized driving scene. A road condition detection unit detects dangerous road conditions for the vehicle from among the dangerous locations detected from the image showing the surrounding conditions of the vehicle included in the surrounding information, The system further includes a priority setting unit for setting the priority of generating or displaying images of multiple hazardous locations in the direction of travel, The priority setting unit sets the priority according to the contact risk detection result by the contact risk determination unit, the assumed hazardous location detection result by the assumed hazardous location detection unit, the hazardous road condition detection result by the road condition detection unit, and the vehicle's direction of travel acquisition result by the direction of travel acquisition unit. An in-vehicle display device according to any one of claims 1 to 6.

16. The computer installed in the vehicle, Acquire information about the surroundings of your vehicle, The direction of travel of the vehicle is obtained, From the images included in the acquired surrounding information, a first image is generated relating to dangerous areas around the vehicle. The image relating to the field of view of the first image is superimposed onto the first image. Depending on the direction of travel of the vehicle, the superposition position of the images in relation to the field of view of the first image is changed. Display control method.

Images