Display device, mobile object, display method, and program
The display device addresses the challenge of large size and cost by using a narrow projection field to present route guidance information efficiently, ensuring effective visibility for drivers.
Patent Information
- Application Number
- JP2020205061
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-01-24
- Filing Date
- 2020-12-10
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-12-10
AI Technical Summary
Existing display devices require a wide projection angle of view to display route guidance information, which increases the size and cost of the device.
A display device that generates and projects a presentation image with a narrow projection field of view, using a configuration that includes an image generation unit and a projection unit to superimpose a virtual image on the foreground, with figures arranged in a vertical direction and intervals smaller than the maximum projection angle, allowing for route guidance information to be presented efficiently.
Enables the presentation of route guidance information with a narrow projection angle of view, reducing the size and cost of the display device while maintaining effective visibility for the driver.
Smart Images

Figure 0007585754000001 
Figure 0007585754000002 
Figure 0007585754000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a display device, a mobile object, a display method, and a program. [Background technology]
[0002] 2. Description of the Related Art Display devices such as head-up displays are known that project a projected image including various types of information onto the front windshield or the like of a vehicle, thereby superimposing and displaying the various types of information on the foreground of the vehicle so that the driver of the vehicle can visually recognize the information.
[0003] Furthermore, as one type of display information in such a display device, a technology has been disclosed in which an image showing the planned route the vehicle will travel (planned travel route) is displayed as route guidance information to guide the vehicle's route, and the display form is changed based on the remaining distance to the location to be notified (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]
[0004] However, the technique of Patent Document 1 requires a wide projection angle of view to display an image, which may increase the size and cost of the display device.
[0005] An object of the present invention is to make it possible to present route guidance information even with a narrow projection angle of view. [Means for solving the problem]
[0006] A display device according to one aspect of the present invention is a display device that displays a projected image, and includes an image generating unit that generates a presentation image that presents route guidance information, and a projection unit that is provided on a moving body and projects the projected image including the presentation image, and the presentation image presents a planned movement direction of the moving body. a virtual image of the presentation image is displayed superimposed on a foreground of the moving body, the virtual image of the presentation image presents the intended movement direction with a plurality of figures arranged in a direction along a vertical direction in the foreground, the distance between each of the plurality of figures is equal to or less than a maximum projection angle of the projection unit in the direction along the vertical direction, and the display distance between the plurality of figures is smaller than an overlap distance indicating a length of an overlap range in the traveling direction of the moving body. . Effect of the Invention
[0007] According to the present invention, route guidance information can be presented even with a narrow projection angle of view. [Brief description of the drawings]
[0008] [Figure 1] 1 is a diagram showing a configuration example in which a display device according to an embodiment is mounted on a vehicle; [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a display unit. [Diagram 3] 1 is a block diagram showing an example of the configuration of an in-vehicle system in which a display device is mounted in a vehicle. [Figure 4] FIG. 2 is a block diagram showing a configuration example of a detection device. [Diagram 5] 2 is a block diagram showing an example of a functional configuration of a recognition device; [Figure 6] 11A-11C are diagrams illustrating example relative positions of obstacles with respect to a vehicle or display components of the vehicle. [Figure 7] FIG. 2 is a block diagram of an example of a hardware configuration of a control unit according to the embodiment. [Figure 8] FIG. 4 is a block diagram of an example of a functional configuration of a control unit according to the embodiment. [Figure 9] 10 is a flowchart illustrating an example of processing by a control unit according to the embodiment. [Figure 10] 13A and 13B are diagrams illustrating an example of a display in the foreground when the remaining distance is not equal to or less than the presentation threshold. [Figure 11] 13A and 13B are diagrams illustrating an example of a display in the foreground when the remaining distance is equal to or less than a presentation threshold. [Figure 12] 1A and 1B are diagrams showing the relationship between the size of a projected image and a presented image, where (a) is a diagram according to a comparative example, and (b) is a diagram according to an embodiment. [Figure 13] 5A and 5B are diagrams illustrating an example of the relationship between the size of a projected image and the projection angle of view. [Figure 14] 13A and 13B are diagrams illustrating an example of the relationship between the display interval of arrow graphics and the overlapping range. [Figure 15] 11A and 11B are diagrams illustrating examples of display positions of arrow graphics. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and duplicated descriptions may be omitted.
[0010] In the embodiment, a presentation image presenting a planned moving direction of a moving object is generated as route guidance information, and a projected image including the presentation image is projected and displayed by a projection unit provided on the moving object. By projecting a small image presenting the planned moving direction of the moving object instead of a large image presenting the planned moving route of the moving object, route guidance information can be presented even with a narrow projection angle of view.
[0011] <Terminology in the embodiment> (Mobile) This refers to a movable object. For example, a mobile body includes a land mobile body that can move on land, such as an automobile, a vehicle, a train, a steam train, or a forklift, an aerial mobile body that can move in the air, such as an airplane, a balloon, or a drone, and a water mobile body that can move on water, such as a ship, a vessel, a steamship, or a boat.
[0012] (foreground) The term "foreground" refers to the scenery ahead of a moving body as seen by the driver of the moving body. For example, the scenery ahead of the vehicle as seen by the driver through the front windshield corresponds to the foreground.
[0013] (Planned route) The planned travel route refers to a route along which a moving object is scheduled to travel in the near future among routes along which a moving object is scheduled to travel from a departure point to a destination. In the embodiment, since a case where the moving object is a vehicle is taken as an example, the planned travel route will be described as a planned driving route of the vehicle.
[0014] (Planned direction of travel) The planned direction of travel of a moving object is the direction in which the moving object is scheduled to move along the planned travel route of the moving object. For example, if there is an intersection on the planned travel route, the direction in which the moving object should proceed at the intersection corresponds to the planned travel direction. In the embodiment, the moving object is a vehicle, and therefore the planned travel direction is described as the planned driving direction of the vehicle.
[0015] (Route guidance information) This refers to information that guides a moving object on a route from a departure point to a destination. Route guidance information includes information on the planned route (planned driving route), the planned direction of travel (planned driving direction), and information on points where route guidance information is presented.
[0016] (Projected image) This refers to an image that is projected by a projection unit and is visible to the driver of a mobile vehicle.
[0017] (Presented image) This refers to an image that presents route guidance information. The image is projected by a projection unit and is visually recognized by a driver of a moving object. The presentation image according to the embodiment can present the planned moving direction of the moving object (the planned driving direction of the vehicle).
[0018] (Driver) A person who drives or operates a mobile vehicle.
[0019] Hereinafter, an embodiment will be described taking a head-up display (HUD) mounted on a vehicle as an example of a display device.
[0020] <Configuration example> FIG. 1 is a diagram showing an example of a configuration in which a display device 1 according to an embodiment is mounted on a vehicle 8. As shown in FIG.
[0021] The display device 1 is embedded inside a dashboard provided in the host vehicle 8, and projects a projected image from an exit window 3 provided on the upper surface of the display device 1 toward the front windshield 71. The projected image is displayed as a virtual image I in the positive Y direction from the front windshield 71, and is viewed by the driver V of the host vehicle 8. By viewing the virtual image I, the driver V can obtain information useful for driving with minimal eye movement while keeping his or her gaze on other vehicles or the road surface in the positive Y direction of the host vehicle 8.
[0022] The display device 1 may be installed on the dashboard, the ceiling, the sun visor, etc. as long as it can project an image onto the front windshield 71.
[0023] As shown in FIG. 1, the display device 1 includes a projection unit 10 and a control unit 20 as main components. The projection unit 10 projects a projected image toward a front windshield 71. Known projection methods of the projected image by the projection unit 10 include a laser scanning method and a panel method. The laser scanning method is a method in which a laser beam emitted from a laser light source is scanned by a two-dimensional scanning device to form an intermediate image (a real image projected on a screen, which will be described later). The panel method is a method in which an intermediate image is formed by an imaging device such as a liquid crystal panel, a DMD (Digital Micro-mirror Device) panel, or a vacuum fluorescent display (VFD; Vacuum Fluorescent Display).
[0024] Unlike the panel type, which forms an image by illuminating the entire screen and then partially blocking it, the laser scanning type can assign light emission / non-emission to each pixel, so it is advantageous in that it can generally form a high-contrast image. Since high contrast improves visibility, it has been revealed that vehicle occupants can view information with fewer attention resources than a panel type HUD.
[0025] In particular, with the panel method, light that cannot be completely blocked is projected even in areas where there is no information, and a display frame is projected in the area where the HUD can display an image (this phenomenon is called a postcard). With the laser scanning method, this phenomenon does not occur, and only the content can be projected. In particular, in AR (Augmented Reality), the reality is improved when the generated image is superimposed on the real landscape.
[0026] AR is a term that translates to "augmented reality," and refers to a technology that virtually expands the world in front of you by overlaying images of non-existent objects onto real scenery. However, even if it is a panel-type HUD, it is acceptable as long as it can display information in a way that requires less attention resources (less fatigue).
[0027] Fig. 2 is a diagram showing an example of the configuration of the projection unit 10. The projection unit 10 mainly includes a light source unit 101, an optical deflector 102, a mirror 103, a screen 104, and a concave mirror 105. Note that Fig. 2 shows only the main components, and the projection unit 10 may include components other than those shown in the figure, or may not include some of the components shown in the figure.
[0028] The light source unit 101 includes three laser light sources (hereinafter referred to as LDs: laser diodes) corresponding to Red (R), Green (G), and Blue (B), a coupling lens, an aperture, a combining element, a lens, etc., and combines the laser beams emitted from the three LDs and guides them toward the reflecting surface of the optical deflector 102. The laser beams guided to the reflecting surface of the optical deflector 102 are two-dimensionally deflected by the optical deflector 102.
[0029] The optical deflector 102 may be a single minute mirror that oscillates about two orthogonal axes, or two minute mirrors that oscillate or rotate about one axis. The optical deflector 102 may be a MEMS (Micro Electro Mechanical Systems) mirror manufactured by a semiconductor process or the like. The optical deflector 102 may be driven by an actuator using the deformation force of a piezoelectric element as a driving force. However, the optical deflector 102 may also be a galvanometer mirror, a polygon mirror, or the like.
[0030] The laser beam two-dimensionally deflected by the optical deflector 102 is incident on the mirror 103 and is reflected by the mirror 103 to draw a two-dimensional image (intermediate image) on the surface (surface to be scanned) of the screen 104. A concave mirror or the like can be used as the mirror 103, but a convex mirror or a plane mirror may also be used. Deflecting the direction of the laser beam with the optical deflector 102 and the mirror 103 allows the projection unit 10 to be made smaller and the arrangement of the components to be flexibly changed.
[0031] As the screen 104, it is preferable to use a microlens array or a micromirror array that has a function of diverging a laser beam at a desired divergence angle, but a diffusion plate that diffuses the laser beam, a transmission plate or a reflection plate with a smooth surface, etc. may also be used. In general, the components from the light source unit 101 to the screen 104 are called a HUD device. However, other components may also be included.
[0032] The laser beam emitted from the screen 104 is reflected by the concave mirror 105 and projected onto the front windshield 71. The concave mirror 105 has a function similar to that of a lens, and has a function of forming an image at a predetermined focal length. For this reason, if the concave mirror 105 is a lens, the image on the screen 104 corresponding to the object is formed at a position of a distance R2 determined by the focal length of the concave mirror 105. Therefore, when viewed from a passenger such as the driver V of the vehicle 8, the virtual image I is displayed at a position of a distance R1+R2 from the front windshield 71. If the distance from the driver V of the vehicle 8 to the front windshield 71 is R3, in FIG. 2, the virtual image I is displayed (formed) at a position of a distance R (=R1+R2+R3) from the viewpoint E of the driver V of the vehicle 8.
[0033] At least a part of the light beam toward the front windshield 71 is reflected toward the viewpoint E of the driver V of the vehicle 8. As a result, the driver V of the vehicle 8 can view the virtual image I, which is an enlarged version of the intermediate image on the screen 104, through the front windshield 71. In other words, as viewed from the driver V of the vehicle 8, the virtual image I, which is an enlarged version of the intermediate image, is displayed through the front windshield 71.
[0034] Generally, the front windshield 71 is not flat but slightly curved. Therefore, the imaging position of the virtual image I is determined not only by the focal length of the concave mirror 105 but also by the curved surface of the front windshield 71, but the distance R is determined substantially by the distance R1+R2 as described above. To form the virtual image I at a distance so as to reduce the line of sight movement, the distance R1 or R2 is increased. One method for increasing the distance R1 is to fold the optical path with a mirror, and one method for increasing the distance R2 is to adjust the focal length of the concave mirror 105.
[0035] In addition, because the influence of the front windshield 71 causes optical distortion in which the horizontal line of the intermediate image has a convex shape upward or downward, it is preferable that at least one of the mirror 103 and the concave mirror 105 is designed and positioned so as to correct the distortion. Alternatively, it is preferable that the projected image is corrected taking the distortion into consideration.
[0036] Also, a combiner may be disposed as a transmissive reflecting member closer to viewpoint E than front windshield 71. Even if light from concave mirror 105 is irradiated onto the combiner, information can be presented as virtual image I in the same way as when light from concave mirror 105 is irradiated onto front windshield 71.
[0037] <Configuration Example of In-Vehicle System 2 in Which Display Device 1 is Mounted> Next, an in-vehicle system 2 in which the display device 1 is mounted on the vehicle 8 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the in-vehicle system 2 in which the display device 1 is mounted on the vehicle 8.
[0038] The in-vehicle system 2 includes a car navigation system 11 that communicates via an in-vehicle network NW such as a CAN (Controller Area Network) bus, an engine ECU (Electronic Control Unit) 12, a display device 1, a brake ECU 13, a steering ECU 14, a recognition device 15, and a detection device 16.
[0039] The car navigation system 11 includes a global navigation satellite system (GNSS) such as a global positioning system (GPS), detects the current location of the vehicle 8, and displays the location of the vehicle 8 on an electronic map.
[0040] In addition, the system accepts input of a departure point and a destination, searches for route guidance information from the departure point to the destination and displays it on an electronic map, and guides the occupants of the vehicle 8 on the direction of travel before changing course by voice, text (displayed on a display), animation, or the like. The car navigation system 11 may communicate with a server via a mobile phone network, etc. In this case, the server can transmit the electronic map to the vehicle 8 and perform route searches.
[0041] The engine ECU 12 determines the ideal fuel injection amount, advances / retards the ignition timing, controls the valve mechanism, etc., based on information from each sensor and the situation of the vehicle 8. It also determines the need for gear changes by referring to a map in which gear change lines are determined for the relationship between the current vehicle speed and accelerator opening. The engine ECU 12 combines these to control acceleration and deceleration when following a preceding vehicle. Note that an electric motor may be used as the power source together with or without the engine.
[0042] The brake ECU 13 controls the braking force for each wheel of the vehicle 8 even without the driver V of the vehicle 8 operating the brake pedal, for example, ABS (Antilock Braking System) control, braking control when following a preceding vehicle, automatic braking based on the TTC (Time To Collision) with an object, and control to maintain a stopped state when starting on a slope.
[0043] The steering ECU 14 detects the direction and amount of steering of the steering wheel by the driver V of the vehicle 8, and performs power steering control to add steering torque in the steering direction. Also, the steering ECU 14 steers the vehicle 8 in a direction to avoid deviation from the driving lane, in a direction to keep the vehicle in the center of the driving lane, or in a direction to avoid approaching an obstacle, even without steering wheel operation by the driver V of the vehicle 8.
[0044] The detection device 16 includes various sensors, and detects objects around the vehicle 8, and detects operation information of the vehicle 8 by the driver V, etc.
[0045] The recognition device 15 recognizes what the object detected by the detection device 16 is, the relative position (direction and distance) of the object with respect to the vehicle 8, and the relative position (direction and distance) of the object with respect to the display component showing the vehicle 8. Information such as the vehicle speed, and the identification result and relative position of the object are input to the display device 1.
[0046] <Configuration Example of Detection Device 16> Next, the configuration of the detection device 16 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of the detection device 16. The detection device 16 includes a vehicle speed sensor 161 for detecting the vehicle speed displayed by the display device 1, a vehicle information sensor 162 for acquiring vehicle information displayed by the display device 1, a radar sensor 163 and a surrounding camera 164 for detecting an object, an occupant status information sensor 165 for acquiring occupant status information which is information about an occupant, a VICS receiving device 166 (registered trademark: VICS: Vehicle Information and Communication System Center) for receiving traffic congestion information and the like from the outside, and an external communication device 167 for connecting to the Internet and the like.
[0047] It should be noted that the sensors provided in the detection device 16 do not need to be present all together in the detection device 16, but may be mounted on the vehicle.
[0048] The vehicle speed sensor 161 detects a magnet that rotates with the shaft of the drive train using a sensor unit fixed to the vehicle body, and generates a pulse wave proportional to the rotation speed. The vehicle speed can be detected based on the number of pulses per unit time.
[0049] The vehicle information sensor 162 includes one or more sensors that detect vehicle information other than the vehicle speed sensor 161. For example, these include a fuel gauge sensor, a shift lever position sensor, an odometer, a trip meter, a blinker sensor, and a water temperature sensor. These may have a general configuration and may be capable of acquiring each vehicle information. The fuel gauge sensor detects the current remaining fuel.
[0050] The shift lever position sensor detects the position of the shift lever operated by the driver V of the vehicle 8. The odometer accumulates the distance traveled by the vehicle 8 to provide the total distance traveled. The trip meter provides the distance traveled in the section from the time the driver V of the vehicle 8 initialized it to the present.
[0051] The turn signal sensor detects the direction of the turn signal operated by the driver V of the vehicle 8. The water temperature sensor detects the temperature of the engine coolant. These are merely examples of information that can be obtained from the vehicle, and other information that can be obtained from the vehicle 8 can also be vehicle information. For example, in an electric vehicle or hybrid vehicle, the remaining battery charge, the amount of regenerated power, the amount of power consumed, etc. can also be obtained.
[0052] The surrounding camera 164 is an imaging device that captures images of the surroundings of the vehicle. The mounting locations are preferably multiple locations so that the rear of the vehicle can be imaged from the side. For example, the surrounding camera 164 is mounted on the roof or bumper of the vehicle 8 at the left rear corner, right rear corner, rear, etc. An imaging device mounted at the rear is called a back monitor, but the surrounding camera 164 at the rear is not limited to a back monitor. It may also be mounted on a side mirror, a pillar, a side portion of the roof, or a door. The surrounding camera 164 may include an imaging device that captures images in front. For example, it may be mounted on the back of the rear-view mirror or nearby thereto.
[0053] The surrounding camera 164 may be a monocular camera or a stereo camera. In the case of a monocular camera or a stereo camera capable of acquiring distance information, the radar sensor 163 may not be required. However, by providing the radar sensor 163 in addition to the surrounding camera 164 capable of acquiring distance information, the distance information of the surrounding camera 164 and the distance information of the radar sensor 163 are fused (integrated) to complement each other's disadvantages and highly accurate distance information can be acquired. In addition to the radar sensor 163 and the surrounding camera 164, a sonic sensor (ultrasonic sensor) or the like may be provided.
[0054] The radar sensor 163 transmits radar to the surroundings of the vehicle 8, such as in front, to the sides, and behind the vehicle 8, and receives the radar reflected back from objects. It may be placed anywhere that allows it to detect obstacles around the vehicle 8. The radar sensor 163 detects the distance to an object based on the time from transmission to reception, and also uses a Time Of Flight (TOF) method that detects the direction of an object based on the direction of radar irradiation.
[0055] LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) is known as a TOF radar sensor. There is also the FMCW (Frequency Modulation Continuous Wave) method, which generates a mixed wave of the received wave and the transmitted wave while continuously increasing the frequency of the transmitted wave, and converts the beat frequency of the mixed wave generated by the slight difference in frequency into distance. In the FMCW method, the direction of an object is estimated by detecting the phase shift of the received wave with multiple receiving antennas.
[0056] The occupant status information sensor 165 is a sensor that detects occupant status information directly or indirectly detected from an occupant of the vehicle 8. A representative example is a face camera. The face camera captures an image of the face of an occupant of the vehicle 8 and performs face authentication to specify or identify the occupant of the vehicle 8. It also becomes possible to detect the face direction and line of sight direction from the face image.
[0057] In addition, the occupant status information sensor 165 may be, for example, an electrocardiogram sensor, a heart rate sensor, a blood pressure sensor, a body temperature sensor, a pulse sensor, a respiration sensor, a sweat sensor, a blink sensor, a pupil sensor, an electroencephalogram sensor, or a myoelectric potential sensor. As the occupant status information sensor 165, for example, a wristwatch-type wearable terminal (smart watch) worn by the occupant of the vehicle may be used.
[0058] The VICS receiver 166 receives radio waves distributed by the VICS. The VICS is a system that transmits road traffic information such as congestion and traffic regulations to an in-vehicle device in real time using FM multiplex broadcasting and beacons. The external communication device 167 connects to the Internet via networks such as 3G, 4G, 5G, LTE, and wireless LAN to receive various types of information. For example, it can receive weather information such as rain, snow, and fog.
[0059] It may also receive news, music, videos, etc. The external communication device 167 may also obtain, for example, traffic light status information and the time until the traffic light changes. In this way, the VICS receiving device 166 and the external communication device 167 may perform road-to-vehicle communication. In addition, the external communication device 167 may obtain information detected by another vehicle 6 through vehicle-to-vehicle communication.
[0060] In addition, the advanced driving system (ADAS) may not only display information and warn, but also control the vehicle 8. In this case, the ECU cooperates with the engine ECU 12, the brake ECU 13, and the steering ECU 14 to perform various driving assistance based on distance information of an object detected by at least one of the radar sensor 163 and the surrounding camera 164. For example, acceleration / deceleration control when following a preceding vehicle, automatic braking, avoidance of deviation from a driving lane, lane keeping driving, and steering to avoid an object are performed. For such control, the ECU recognizes road paint such as white lines from images captured by the surrounding camera 164.
[0061] In acceleration / deceleration control during following, the ECU controls power and braking to maintain a target distance according to the vehicle speed. In automatic braking, depending on the TTC, it issues a warning, displays a message encouraging the driver to press the brake pedal, and retracts the seat belt and applies collision avoidance braking if there is a high possibility of a collision. In avoiding deviation from the driving lane, the driving assistance ECU 36 recognizes white lines (traffic lane markings) from the captured image and applies steering torque in the opposite direction to the direction of deviation from the driving lane.
[0062] In lane keeping, the center of the lane is set as the target line, and a steering torque proportional to the deviation from the target line is applied in the opposite direction to the deviation. In object avoidance steering, if it is determined that a collision cannot be avoided by braking, a driving line to avoid the object is determined, and a steering torque is applied to travel along this driving line.
[0063] In addition, for example, when changing lanes, if the radar sensor 163 or the surrounding camera 164 detects a vehicle traveling alongside in an area (blind spot area) that is not reflected in the door mirror in the adjacent lane, a warning is given to the occupants. This type of support is called a blind spot monitor.
[0064] <Example of functional configuration of recognition device 15> Next, the functional configuration of the recognition device 15 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the functional configuration of the recognition device 15. The recognition device 15 includes an object determination unit 151 and a relative position determination unit 152.
[0065] The object determination unit 151 determines the classification of the object captured by analyzing the surrounding image data mainly captured by the surrounding camera 164. In the embodiment, for example, the object determination unit 151 determines whether the object is another vehicle, a pedestrian, a motorcycle, or the like.
[0066] In addition, when the surrounding camera 164 is a stereo camera, each pixel or pixel block of the surrounding image data contains distance information. Such surrounding image data is called a distance image. Also, the object determination unit 151 can determine the object not only from the surrounding image data but also from the radar distance measurement information. For example, when the density of the LIDAR point cloud is sufficiently high, the shape of the object can be obtained, and the object classification can be determined by analyzing the shape.
[0067] One of the methods by which the object determination unit 151 determines an object is an image recognition method using machine learning. Machine learning is a technology for making a computer acquire a learning ability like a human, and refers to a technology in which a computer autonomously generates an algorithm required for determining data identification and the like from learning data that is previously loaded, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods. Any learning method for machine learning is acceptable. Machine learning techniques include perceptron, deep learning, support vector machine, logistic regression, naive Bayes, decision tree, random forest, and the like.
[0068] The relative position determination unit 152 determines the relative position (direction and distance) of an object with respect to the host vehicle 8 or the display component 62 representing the host vehicle 8, as shown in Fig. 6. Fig. 6 is a diagram illustrating an example of the relative position of an object with respect to the host vehicle 8 or the display component 62 representing the host vehicle 8. As shown in Fig. 6, it is assumed that the display component 62 representing the host vehicle 8 (center of virtual image I) is displayed at a position Q in the vehicle width direction and P in the vehicle length direction from the center of the host vehicle 8.
[0069] It is assumed that the other vehicle 6 has been detected as having coordinates (A, B) with the center of the vehicle 8 as the origin based on the surrounding image data or radar ranging information. The distance from the center of the vehicle 8 to the other vehicle 6 is represented as L1, and the direction as θ1. If the surrounding image data is a range image, (A, B) can be obtained directly, but if the radar ranging information is used, the distance and direction can be resolved into (A, B). The relative position determination unit 152 converts the coordinates (A, B) into coordinates (C, D) based on the display component 62 indicating the vehicle 8.
[0070] The display component 62 representing the vehicle 8 is presented as a virtual image I, and is therefore located at a fixed position in front of the vehicle 8. As described above, if the distance from the center of the vehicle to the display component 62 representing the vehicle 8 in the vehicle width direction is Q and the distance in the vehicle length direction is P, then the coordinates (C, D) are determined as follows: (C, D) = (A + Q, B + P)
[0071] The relative position determination unit 152 performs similar processing for each radar sensor or surrounding camera arranged on the vehicle, and determines the relative position of the object with respect to the display component 62 showing the host vehicle 8. By determining the relative position, the distance and direction can also be obtained. The distance from the display component 62 showing the host vehicle 8 to the other vehicle 6 is represented by L2, and the direction is represented by θ2. In the example of FIG. 6, the distance L2=√(C 2 +D 2 ), and direction θ2=arctan(D / C). Note that the reference for directions θ1 and θ2 (which direction is set to 0 degrees) may be set appropriately. In FIG. 6, the 9 o'clock direction is set to 0 degrees.
[0072] The recognition device 15 outputs to the control unit 20 classification information indicating the classification of the object determined by the object determination unit 151, distance information from the host vehicle 8 to the object acquired by the relative position determination unit 152, and position information indicating the relative position of the object with respect to the host vehicle 8.
[0073] <Hardware configuration example of control unit 20> Next, the hardware configuration of the control unit 20 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of the hardware configuration of the control unit 20. As shown in Fig. 7, the control unit 20 includes a field-programmable gate array (FPGA) 201, a central processing unit (CPU) 202, a read only memory (ROM) 203, a random access memory (RAM) 204, an interface (I / F) 205, a bus line 206, an LD driver 207, and a MEMS controller 208. The FPGA 201, the CPU 202, the ROM 203, the RAM 204, and the I / F 205 are connected to each other via the bus line 206.
[0074] The CPU 202 controls each function of the control unit 20. The ROM 203 stores a program 203p that the CPU 202 executes to control each function of the control unit 20. The program 203p is loaded into the RAM 204, which is used as a work area for the CPU 202 to execute the program 203p. The RAM 204 also includes an image memory 209.
[0075] The image memory 209 is used to generate an image to be projected as the virtual image I. The I / F 205 is an interface for communicating with the recognition device 15 and the detection device 16, and is connected to a CAN (Controller Area Network) bus or Ethernet (registered trademark) of the vehicle 9, or the like.
[0076] The FPGA 201 controls the LD driver 207 based on the image created by the CPU 202. The LD driver 207 drives the LD of the light source unit 101 of the projection unit 10 to control the light emission of the LD according to the image. The FPGA 201 operates the optical deflector 102 of the projection unit 10 via the MEMS controller 208 so that the laser beam is deflected in a direction according to the pixel position of the image.
[0077] <Example of functional configuration of control unit 20> Next, the functional configuration of the control unit 20 will be described with reference to Fig. 8. Fig. 8 is a block diagram showing an example of the functional configuration of the control unit 20.
[0078] As shown in Fig. 8, the control unit 20 includes a remaining distance acquisition unit 21, a determination unit 22, a direction determination unit 23, an image generation unit 24, and an output unit 25. Among these, the functions of the remaining distance acquisition unit 21, the determination unit 22, the direction determination unit 23, and the image generation unit 24 are realized by the CPU 202 in Fig. 7 executing a predetermined program, etc. Also, the function of the output unit 25 is realized by the I / F 205 in Fig. 7, etc.
[0079] The remaining distance acquisition unit 21 acquires information on the remaining distance until the host vehicle 8 reaches the point where the presentation image is presented, based on the route guidance information input from the car navigation system 11, and outputs the acquired result to the determination unit 22.
[0080] The determination unit 22 determines whether the remaining distance is equal to or less than a predetermined presentation threshold, and outputs the determination result to the direction determination unit 23.
[0081] When the remaining distance becomes equal to or less than the presentation threshold, the direction determination unit 23 determines a planned driving direction of the vehicle 8 on the planned driving route based on the route guidance information input from the car navigation system 11, and outputs the determination result to the image generation unit 24. For example, when there is an intersection on the planned driving route of the vehicle 8, the direction in which the vehicle 8 should proceed at the intersection is determined.
[0082] The image generation unit 24 generates a presentation image for presenting the planned driving direction in the planned driving route of the vehicle 8, according to the planned driving direction determined by the direction determination unit 23. Then, image data of the generated presentation image is output to the projection unit 10 via the output unit 25.
[0083] The projection unit 10 (see FIG. 1) projects a projected image including a presentation image onto the front windshield 71 based on the input presentation image data, and causes the driver V of the vehicle 8 to visually recognize a virtual image of the presentation image.
[0084] The image generating unit 24 can generate not only the presentation image but also other images such as a remaining distance image that displays the remaining distance information acquired by the remaining distance acquiring unit 21, and a vehicle speed image that displays the vehicle speed information of the vehicle 8 detected by the vehicle speed sensor 161 in Fig. 4. Then, the projected image data in which the presentation image, remaining distance image, vehicle speed image, etc. are arranged in one image can be output to the projection unit 10 via the output unit 25.
[0085] <Example of processing by the control unit 20> Next, the flow of processing by the control unit 20 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of processing by the control unit 20.
[0086] First, in step S91, the remaining distance acquisition unit 21 and the direction determination unit 23 input route guidance information from the car navigation system 11.
[0087] Next, in step S92, the remaining distance acquisition unit 21 acquires information on the remaining distance until the host vehicle 8 reaches the point where the presentation image is presented, based on the route guidance information.
[0088] Next, in step S93, the determination unit 22 determines whether or not the remaining distance is equal to or less than the presentation threshold value.
[0089] If it is determined in step S93 that the remaining distance is not equal to or less than the presentation threshold value (step S93, No), the process from step S91 onwards is repeated again.
[0090] On the other hand, if it is determined in step S93 that the remaining distance is equal to or less than the presentation threshold (Yes in step S93), then in step S94, the direction determination unit 23 determines the planned traveling direction of the host vehicle 8 based on the route guidance information.
[0091] Next, in step S95, the image generating unit 24 generates a presentation image according to the planned traveling direction determined by the direction determining unit 23.
[0092] Next, in step S96, the output unit 25 outputs the presentation image data to the projection unit 10.
[0093] In this way, the control unit 20 can generate a presentation image and output it to the projection unit 10. Note that during steps S91 to S94, the image generation unit 24 can generate the above-mentioned remaining distance image, vehicle speed image, etc., and output projected image data including these to the projection unit 10 via the output unit 25. In this case, a projected image that does not include a presentation image but includes a remaining distance image, vehicle speed image, etc. is displayed superimposed on the foreground of the host vehicle 8.
[0094] (Various display examples) Next, various display examples by the display device 1 will be described. In the following, a case where an image is superimposed and displayed on the foreground of the vehicle 8 when a destination is set using the car navigation system 11 in Fig. 3 will be described as an example. Also, in a general car navigation system, the planned driving route of the vehicle is guided at each intersection or branch point where the road branches. Therefore, a case where a presentation image is displayed at each intersection or branch point corresponding to this and the planned driving route is guided will be described as an example.
[0095] First, FIG. 10 is a diagram showing an example of a display in the foreground 300 of the vehicle 8 when the remaining distance is not equal to or less than the presentation threshold value.
[0096] The foreground 300 includes a leading vehicle 301 and a road 302. In addition, a vehicle speed image 303 and a remaining distance image 304 are displayed superimposed on the foreground 300.
[0097] The vehicle speed image 303 is an image that displays the current vehicle speed of the host vehicle 8. In the example of Fig. 10, 70 (km / h) is displayed as the vehicle speed.
[0098] The remaining distance image 304 is an image that displays the remaining distance until the vehicle 8 reaches the point where the presentation image is presented. In the example of Fig. 10, "Tsutsumi IC" is displayed as the point where the presentation image is presented, and 200 mm, which is the remaining distance to "Tsutsumi IC", is displayed.
[0099] The remaining distance image 304 is displayed when the remaining distance is equal to or less than a predetermined display threshold (e.g., 500 m) and is displayed until the remaining distance is equal to or less than a presentation threshold (e.g., 100 m). In FIG. 10, the remaining distance is not equal to or less than the presentation threshold, so the presentation image is not yet displayed.
[0100] Next, FIG. 11 is a diagram showing an example of a display in the foreground 300a of the host vehicle 8 when the remaining distance is equal to or less than the presentation threshold value.
[0101] In foreground 300a, a cross road 305 appears that intersects with road 302. In addition, because the remaining distance is equal to or less than the presentation threshold, remaining distance image 304 in FIG. 10 is no longer displayed in foreground 300a, and presentation image 306 is newly displayed.
[0102] The presentation image 306 is an image that presents the planned traveling direction of the vehicle 8 so as to guide (introduce) the vehicle to an intersecting road 305. As shown in Fig. 11, the presentation image 306 includes two arrow figures 306a and 306b.
[0103] The arrow graphics 306a and 306b have a directionality so as to point to the intersecting road 305. The direction pointed by the arrow graphics 306a and 306b corresponds to the planned traveling direction of the vehicle 8 (the planned moving direction of the moving object).
[0104] Each of the arrow figures 306a and 306b is a figure with an image size smaller than the size corresponding to the projection angle of view of the projection unit 10 in the Z direction, and is displayed intermittently from the positive Z direction to the negative Z direction.
[0105] Here, the Z direction indicated by an arrow in Fig. 11 corresponds to the vertical direction in the foreground 300a, and the X direction corresponds to the horizontal direction in the foreground 300a. This also applies to the X and Z directions shown in the following Figs. 12 to 15. The relationship between the size of the projected image and the projection angle of view will be described later with reference to Fig. 13.
[0106] The driver V of the vehicle 8 can easily recognize the planned driving direction on the planned driving route by visually checking the presentation image 306 superimposed on the foreground 300a.
[0107] The arrow graphics 306a and 306b are an example of "plural graphics". In the example of Fig. 11, two arrows are shown, but the number of arrows may be increased. Furthermore, the graphics are not limited to arrows, and may be graphics of any shape as long as they indicate a direction.
[0108] 11, an area 307 indicated by a dashed line indicates an area where the projected image is displayed. The projected image is displayed within the area of area 307 in the foreground 300a. The presented image 306 is an image included in the projected image, and is therefore displayed within this area 307. Note that area 307 is illustrated to explain the area where the projected image is displayed, and is not displayed in the foreground 300a of the host vehicle 8.
[0109] Next, Fig. 12 is a diagram for explaining the relationship between the size of the projected image and the presented image. (a) is a diagram showing the relationship between the size of the projected image and the presented image in a comparative example, and (b) is a diagram showing the relationship between the size of the projected image and the presented image in the embodiment. Fig. 12 corresponds to a view of the superimposed display of the projected image on the foreground in Figs. 10 and 11 as viewed from above the vehicle 8 (looking down from the sky).
[0110] 12(a), in order to present a planned driving route of the vehicle 8 traveling on a road 302, a presentation image 306' of a thick arrow extending along the route is displayed. In this case, since it is necessary to extend the thick arrow along the planned driving route, a projected image including the presentation image 306' is required to have a size capable of presenting an overlapping range 308'. The overlapping range here is a range determined by the projection angle of view and depression angle (downward angle) of the projection unit 10, and refers to the range in which the foreground and the projected image are overlapped.
[0111] 12(b), the presentation image 306 according to the embodiment does not need to show the entire route in order to present the planned driving direction. Therefore, it is possible to generate a projected image with a size that can present an overlapping range 308 that is smaller than the overlapping range 308' in FIG. 12(a).
[0112] Next, Fig. 13 is a diagram illustrating an example of the relationship between the size of the projected image and the projection angle of the projection unit 10. In Fig. 13, virtual image I' shows a virtual image according to a comparative example. The virtual image I' is a virtual image of the projected image that is visually recognized by the driver V when the projected image is projected at the projection angle of view φ'. On the other hand, virtual image I shows a virtual image according to the embodiment. The virtual image I is a virtual image of the projected image that is visually recognized by the driver V when the projected image is projected at the projection angle of view φ.
[0113] The size h in the Z direction of the virtual image I of the projected image including the presentation image 306 according to the embodiment is smaller than the size h' in the Z direction of the virtual image I' of the projected image including the presentation image 306' according to the comparative example. Therefore, it is possible to present the presentation image 306 at a narrower projection angle of view φ compared to the projection angle of view φ' and present route guidance information.
[0114] Here, the projection angle of view φ corresponds to the maximum projection angle of view of the projection unit 10 in the Z direction along the vertical direction.
[0115] Next, Fig. 14 is a diagram for explaining an example of the relationship between the display interval of the arrow figures 306a and 306b in the presentation image 306 and the overlapping range. In Fig. 14, the display interval d indicates the display interval of the arrow figures 306a and 306b, and the overlapping distance e indicates the length (distance) of the overlapping range in the driving direction of the vehicle 8 (direction perpendicular to both the X direction and the Y direction). As shown in Fig. 14, it is preferable that the display interval d is smaller than the overlapping distance e. In addition, if the display interval d is set to be equal to or smaller than the maximum projection angle of the projection unit 10 in the Z direction, multiple figures such as the arrow figures 306a and 306b can be displayed in the foreground, and the planned driving direction can be presented more clearly with multiple figures, which is preferable.
[0116] <Effects of the display device 1> As described above, in the embodiment, a presentation image presenting the planned traveling direction of the vehicle 8 is generated as route guidance information, and a projected image including the presentation image is projected and displayed by the projection unit 10 provided in the vehicle 8. Since a large image for presenting the planned traveling route of the vehicle 8 is not projected, but a small image for presenting the planned traveling direction of the vehicle 8 is projected, it is possible to present the presentation image 306 at a narrow projection angle of view φ (see FIG. 13) and present the route guidance information. This makes it possible to avoid an increase in size and cost of the display device 1.
[0117] Moreover, the presentation image 306 according to the embodiment includes arrow figures 306a and 306b, which are multiple figures with an image size smaller than the size corresponding to the projection angle of the projection unit 10. The arrow figures 306a and 306b are displayed intermittently from the positive Z direction to the negative Z direction. In this way, the area of the presentation image 306 superimposed on the road in the foreground can be made smaller than when a presentation image of a thick arrow extending along the planned driving route is superimposed on the road in the foreground. This can eliminate visual bother (clutter) in the foreground.
[0118] Here, the planned traveling direction can also be presented by the display positions of the arrow graphics 306a and 306b in the presentation image 306. FIG.
[0119] It is simplest to display the arrow figures 306a and 306b on the central axis 309 of the driver V, but it is also possible to determine a reference position 310 of the intersection and display the arrow figures 306a and 306b so that they are positioned on a straight line connecting the reference position 310 and the driver V.
[0120] Such a display state will be described by replacing it with a case where the foreground 300a of the vehicle 8 is viewed. The virtual image of the presentation image 306 is a plurality of figures arranged in a direction along the vertical direction in the foreground 300a, and presents the planned driving direction. The plurality of figures present the planned driving direction at a position closer to the intersection in the vertical direction in the foreground, and the closer to the intersection in the horizontal direction in the foreground, the more the figures present the planned driving direction.
[0121] More specifically, with reference to FIG. 11, of the arrow figures 306a and 306b constituting the presentation image 306, the arrow figure 306a presents the intended driving direction at a position closer to the intersecting road 305 in the vertical direction (Z direction) in the foreground 300a, and presents the intended driving direction at a position closer to the intersecting road 305 in the horizontal direction (X direction) in the foreground 300a, compared to the arrow figure 306b.
[0122] In this manner, the driver V can recognize the intended traveling direction not only from the directions of the arrow graphics 306a and 306b but also from the display positions of the arrow graphics 306a and 306b.
[0123] Note that the remaining distance image 304 in the above-described embodiment presents information on the point where the presentation image is presented and information on the remaining distance until the vehicle 8 reaches this point, and therefore can also be said to be a presentation image that presents route guidance information. However, since the remaining distance image 304 is text information and does not require a wide projection angle of view for display, the impact on the projection angle of view is small.
[0124] <Other suitable examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.
[0125] In the above-described embodiment, a part or all of the functions of the recognition device 15 may be provided in the control unit 20. Also, a part of the functions of the control unit 20 may be provided in other components such as the recognition device 15, the car navigation system 11, the detection device 16, etc.
[0126] The embodiment also includes a display method. For example, the display method is a display method by a display device that displays a projected image, and includes a step of generating a presentation image that presents route guidance information, and a step of projecting the projected image including the presentation image by a projection unit provided on a moving object, and the presentation image presents a planned moving direction of the moving object. By such a display method, it is possible to obtain the same effect as the above-mentioned display device.
[0127] The embodiment also includes a program. For example, the program is a program that runs on a display device that displays a projected image, and causes a computer to execute a process of generating a presentation image that presents route guidance information, and projecting the projected image including the presentation image by a projection unit provided on a moving object, and the presentation image presents a planned moving direction of the moving object. Such a program can provide the same effect as the display device described above.
[0128] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above. [Explanation of symbols]
[0129] 1 Display device 2. In-vehicle systems 8. Vehicle (an example of a moving object) 10 Projection part 11. Car navigation system 15 Recognition device 16 Detection device 164 Surrounding Camera 20 Control section 21 Remaining distance acquisition section 22 Judgment section 23 Direction Determination Unit 24 Image Generation Unit 25 Output section 300, 300a foreground 301 Leading vehicle 302 Road 303 Vehicle speed image 304 Remaining distance image 305 Crossroads 306 Presentation Image 306a, 306b Arrow shapes (examples of multiple shapes) 307 areas 308 Overlap Range d Display interval e Overlap distance h Size of the virtual image in the Z direction I Virtual Image V. Driver X: along the horizontal direction in the foreground Z: along the vertical direction in the foreground φ Projection angle of view [Prior art documents] [Patent documents]
[0130] [Patent Document 1] JP 2019-207155 A
Claims
1. A display device that displays a projected image, an image generating unit that generates a presentation image that presents route guidance information; a projection unit that is provided on a moving object and projects the projected image including the presentation image; the presentation image presents a planned movement direction of the moving object, A virtual image of the presented image is superimposed on the foreground of the moving object, the virtual image of the presentation image presents the intended travel direction at the intersection in the foreground with a plurality of figures arranged in a vertical direction in the foreground, an interval between each of the plurality of figures is equal to or smaller than a maximum projection angle of the projection unit in a direction along the vertical direction; The display interval of the plurality of figures is smaller than an overlap distance indicating the length of the overlap range in the running direction of the moving object. Display device.
2. The graphic is an image of an arrow graphic having a direction indicating the intended direction of movement. The display device according to claim 1 .
3. The plurality of figures are arranged so that the closer the figure presents the intended movement direction in a position along the vertical direction in the foreground and to the intersection, the closer the figure presents the intended movement direction in a position along the horizontal direction in the foreground and to the intersection. The display device according to claim 1 .
4. A moving object comprising the display device according to any one of claims 1 to 3.
5. A display method for displaying a projected image by a display device, the display device comprising: generating a presentation image presenting route guidance information; projecting the projected image including the presented image by a projection unit provided on a moving object; the presentation image presents a planned movement direction of the moving object, A virtual image of the presented image is superimposed on the foreground of the moving object, the virtual image of the presentation image presents the intended movement direction with a plurality of figures arranged in a vertical direction in the foreground; an interval between each of the plurality of figures is equal to or smaller than a maximum projection angle of the projection unit in a direction along the vertical direction; The display interval of the plurality of figures is smaller than an overlap distance indicating the length of the overlap range in the running direction of the moving object. Display method.
6. A program that operates on a display device that displays a projected image, generating a presentation image presenting route guidance information; The projected image including the presented image is projected by a projection unit provided on the moving body. The process is executed by a computer, the presentation image presents a planned movement direction of the moving object, A virtual image of the presented image is superimposed on the foreground of the moving object, the virtual image of the presentation image presents the intended movement direction with a plurality of figures arranged in a vertical direction in the foreground; an interval between each of the plurality of figures is equal to or smaller than a maximum projection angle of the projection unit in a direction along the vertical direction; The display interval of the plurality of figures is smaller than an overlap distance indicating the length of the overlap range in the running direction of the moving object. program.
Citation Information
Patent Citations
Vehicular display apparatus and vehicular display method
JP2017021019A
On-vehicle system
JP2019119318A
Image control device, display device, vehicle, image control method, and program
JP2019174456A
Display control device, and display control program
JP2019207155A
Vehicular display device
WO2017056211A1
