Information processing device, information processing method, program, and projection device

Abstract

An information processing device according to one embodiment of the present technology comprises an acquisition unit and a projection control unit. The acquisition unit acquires surrounding environment information about a surrounding environment of a vehicle. The projection control unit controls, on the basis of the surrounding environment information, a display of a projection pattern projected from this projection unit mounted on the vehicle onto the road surface around the vehicle. Accordingly, it is possible to enhance safety during driving.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, an information processing method, a program, and a projection apparatus applicable to display control of a projection pattern projected from a vehicle onto a road surface.

Background Art

[0002] Patent Document 1 describes a predicted travel trajectory display device that displays a predicted travel trajectory of a vehicle on the ground surface. In this device, a predicted travel trajectory of the vehicle is calculated from the steering angle of the steering wheel of the vehicle and the forward / backward information indicating the forward / backward movement of the vehicle. Then, the irradiation angle of a laser emitter mounted on the vehicle is controlled, and the predicted travel trajectory is drawn on the ground surface (paragraphs

[0021]

[0022]

[0023] of the specification of Patent Document 1, FIG. 7, etc.).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Thus, by presenting the traveling direction of the vehicle, it is possible to prompt attention inside and outside the vehicle. In an actual traffic environment, in addition to the traveling direction of the vehicle, it is necessary to direct attention to various objects, and a technology capable of enhancing the safety during driving is required.

[0005] In view of the above circumstances, an object of the present technology is to provide an information processing apparatus, an information processing method, a program, and a projection apparatus capable of enhancing the safety during driving.

Means for Solving the Problems

[0006] To achieve the above object, an information processing apparatus according to one aspect of the present technology includes an acquisition unit and a projection control unit. The acquisition unit acquires surrounding environment information related to the environment around the vehicle. The projection control unit controls the display of the projection pattern projected onto the road surface around the vehicle from the projection unit mounted on the vehicle, based on the surrounding environment information.

[0007] In this information processing system, a projection pattern is projected onto the surrounding road surface from a projection unit installed in the vehicle. The display of this projection pattern is controlled based on surrounding environment information related to the vehicle's surroundings. This makes it possible to present information such as the vehicle's location to both the inside and outside of the vehicle via the projection pattern, thereby enhancing driving safety.

[0008] An information processing method relating to one embodiment of this technology is an information processing method performed by a computer system, and includes acquiring ambient environmental information relating to the environment around a vehicle. Based on the surrounding environment information, the display of the projection pattern projected onto the road surface around the vehicle from the projection unit mounted on the vehicle is controlled.

[0009] A program relating to one form of this technology causes a computer system to perform the following steps. A step to acquire surrounding environment information regarding the environment around the vehicle. A step of controlling the display of a projection pattern projected onto the road surface around the vehicle from a projection unit mounted on the vehicle, based on the surrounding environment information.

[0010] A projection device according to one embodiment of this technology comprises a projection unit, an acquisition unit, and a projection control unit. The projection unit is mounted on a vehicle and projects a projection pattern onto the road surface surrounding the vehicle. The acquisition unit acquires surrounding environment information relating to the environment around the vehicle. The projection control unit controls the display of the projection pattern projected from the projection unit based on the surrounding environment information. [Brief explanation of the drawing]

[0011] [Figure 1] It is a schematic diagram showing the appearance of a vehicle equipped with a projection device according to the first embodiment of the present technology. [Figure 2] It is a schematic diagram showing an example of a projection pattern. [Figure 3] It is a schematic diagram showing an example of a scene where a projection pattern is projected. [Figure 4] It is a block diagram showing a configuration example of a projection device according to the first embodiment. [Figure 5] It is a block diagram showing a configuration example of a surrounding environment recognition unit. [Figure 6] It is a flowchart showing a basic operation example of a projection device. [Figure 7] It is a schematic diagram showing the state in front of a vehicle when changing lanes. [Figure 8] It is a schematic diagram showing an example of a front line projected when changing lanes. [Figure 9] It is a schematic diagram showing an example of a lane change in a merging lane. [Figure 10] It is a block diagram showing a configuration example of a projection device according to the second embodiment. [Figure 11] It is a block diagram showing a configuration example of a surrounding environment recognition unit. [Figure 12] It is a flowchart showing a basic operation example of a projection device. [Figure 13] It is a schematic diagram showing an example of a rear pattern according to a collision risk. [Figure 14] It is a block diagram showing a configuration example of a projection device according to the third embodiment. [Figure 15] It is a block diagram showing a configuration example of a surrounding environment recognition unit. [Figure 16] It is a flowchart showing a basic operation example of a projection device. [Figure 17] It is a schematic diagram showing an example of a scene where a vehicle width line is projected. [Figure 18] It is a schematic diagram showing an example of a vehicle width line according to an approach distance. [Figure 19]It is a schematic diagram showing an example of the display of the vehicle width line projected on an obstacle. [Figure 20] It is a schematic diagram showing an example of the vehicle width line projected when an obstacle is detected.

Embodiments for Carrying out the Invention

[0012] Hereinafter, embodiments according to the present technology will be described while referring to the drawings.

[0013] <The First Embodiment> FIG. 1 is a schematic diagram showing the appearance of a vehicle equipped with a projection device according to the first embodiment of the present technology. FIG. 1A is a perspective view showing a configuration example of the vehicle 1, and FIG. 1B is a top view of the vehicle 1 from above. The vehicle 1 is equipped with a projection device 100 that projects an image onto the road surface around the vehicle 1.

[0014] The projection device 100 has a plurality of projection units 10. The projection unit 10 is an element (projector) that projects an image by irradiating light onto the road surface. Hereinafter, the image projected from the projection unit 10 onto the road surface around the vehicle 1 will be referred to as a projection pattern. In the projection device 100, the projection patterns projected from each projection unit 10 can be controlled individually. The specific configuration of the projection unit 10 will be described later.

[0015] In FIGS. 1A and 1B, eight projection units 10a to 10h provided on the vehicle 1 are schematically shown. In the example shown in FIG. 1, the projection units 10a to 10d and the projection units 10e to 10h are arranged on the left and right sides of the bottom of the vehicle 1 so as to be symmetric with each other. Projection units 10a and 10e are located at the front bottom of the vehicle 1 (for example, below the front bumper) and project a projection pattern forward of the vehicle 1. Projection units 10b and 10f are located at the front bottom of the front door, and projection units 10c and 10g are located at the rear bottom of the rear door. Projection units 10b and 10f and projection units 10c and 10g project a projection pattern to the side of the vehicle 1. Projection units 10d and 10h are located at the rear bottom of the vehicle 1 (for example, below the rear bumper) and project a projection pattern backward of the vehicle 1. Note that in Figure 1B, for convenience, each projection unit 10 (projector) is shown protruding from the vehicle body in the top view of the vehicle 10. In the actual configuration, the projection units 10 are housed in the lower part of the vehicle body and installed so that they are not visible from above. This makes it possible to implement the projection device 100 without compromising the appearance of the vehicle 10. Alternatively, as shown in Figure 1B, each projection unit 10 may be installed so as to protrude from the vehicle body. This makes it possible, for example, to expand the range over which the projection pattern can be projected.

[0016] The arrangement and number of projection units 10 are not limited. For example, the projection unit 10 (projection units 10a and 10e in Figure 1) that projects a pattern onto the front of the vehicle 10 may be installed on the front of the vehicle body. Specifically, the projection unit 10 may be provided around the headlights (above, below, left, and right of the headlights, etc.) or around the fog lamps (above, below, left, and right of the fog lamps, etc.). Alternatively, the projection unit 10 may be provided at the location of the front grille or in the central part of the entire front of the vehicle body. Furthermore, projection patterns for the left front and right front may be projected from a single projection unit 10. Furthermore, projection units 10 (projection units 10b, 10c, 10f, and 10g in Figure 1) that project patterns onto the sides of the vehicle 10 may be provided at the bottom of the side mirror or at the lower part of the B-pillar separating the front and rear doors (in the center of the vehicle in the front-to-rear direction). Furthermore, projection units 10 (projection units 10d and 10h in Figure 1) that project patterns to the rear of the vehicle 10 may be installed on the rear surface of the vehicle body. Specifically, projection units 10 may be provided around the brake lights (above, below, left, and right of the brake lights, etc.), around the license plate (above, below, left, and right of the license plate, etc.), or in the central part of the rear surface of the vehicle body. Also, projection patterns for the left rear and right rear may be projected from a single projection unit 10. In addition, the projection unit 10 may be appropriately positioned at a location where a desired projection pattern can be projected.

[0017] Figure 2 is a schematic diagram showing an example of a projection pattern. Figure 3 is a schematic diagram showing an example of a scene on which the projection pattern is projected. In this embodiment, multiple projection modes are selected depending on the operating conditions of the vehicle 1, and a projection pattern 2 corresponding to the projection mode is projected. Figures 2A to 2D schematically illustrate the projection patterns 2 projected in each projection mode: normal driving mode, low-speed driving mode, reverse mode, and parking mode.

[0018] Of the multiple projection modes, all modes other than the parking mode are modes in which vehicle 1 is actually driven. In these modes in which vehicle 1 is driven (Figures 2A to 2C), a line pattern 3 extending in a linear fashion is used as the projection pattern 2. Therefore, projection pattern 2 includes a linear pattern (line pattern 3). Line pattern 3 is, for example, a continuous strip-like pattern. Alternatively, line pattern 3 may be constructed by arranging small patterns in a linear fashion at regular intervals.

[0019] In this embodiment, the line pattern 3 includes a first linear pattern projected forward in the direction of travel 4 of the vehicle 1, and a second linear pattern projected backward in the direction of travel 4. Figures 2A to 2C schematically illustrate arrows indicating the direction of travel 4 of vehicle 1. The size of the arrows represents the speed of vehicle 1. The pattern projected forward relative to this direction of travel 4 is the first pattern, and the pattern projected backward is the second pattern. In this embodiment, a third pattern (the central line 3b, described later) is used, which is projected onto the road surface around the central part of the vehicle 1 (the road surface extending from below to the side of the vehicle 1).

[0020] As will be described later, the projection device 100 calculates a predicted trajectory 5, which is the trajectory that vehicle 1 is expected to pass, and a trajectory 6, which is the trajectory that vehicle 1 actually passed. These predicted trajectory 5 and trajectory 6 are represented using a first pattern and a second pattern. Specifically, the first pattern is generated as a line pattern 3 representing the predicted trajectory 5 of vehicle 1, and the second pattern is generated as a line pattern 3 representing the trajectory of vehicle 1.

[0021] The following sections will explain the normal driving mode, low-speed driving mode, and reverse mode, which use line pattern 3, as well as the parking mode, which uses a pattern other than line pattern 3.

[0022] Figure 2A shows an example of projection pattern 2, which is projected in normal driving mode. The upper and lower diagrams of Figure 2A are schematic views of vehicle 1 from the side and above. Here, the "normal driving mode" is the projection mode selected when, for example, vehicle 1 is driving forward normally without slowing down. The normal driving mode is selected when vehicle 1 is driving at a speed faster than a typical slow speed (for example, less than 10 km / h). Therefore, the normal driving mode is used when driving forward in the driving lane, rather than when slowing down for operations such as stopping, turning right, turning left, or parking.

[0023] In normal driving mode, three types of line patterns 3 are projected as projection pattern 2: a front line 3a, a center line 3b, and a rear line 3c. Each line 3a to 3c is composed of a pair of line patterns 3 projected onto the left and right sides of the vehicle 1. The forward line 3a is a line pattern 3 projected onto the road surface in front of the vehicle 1. The left and right forward lines 3a are projected from projection sections 10a and 10e, for example, as shown in Figure 1. The center line 3b is a line pattern 3 projected onto the road surface to the side from below the vehicle 1. The left center line 3b is projected from, for example, projection sections 10b and 10c, and the right center line 3b is projected from, for example, projection sections 10f and 10g. The rear line 3c is a line pattern 3 projected onto the road surface behind the vehicle 1. The left and right rear lines 3c are projected from, for example, projection sections 10d and 10h. Furthermore, the correspondence between each line 3a to 3c and the projection unit 10 is not limited to the example described above; for example, a configuration in which one line pattern 3 is projected using two projection units 10 is also possible.

[0024] As described above, the line pattern 3 shown in Figure 2A is projected while vehicle 1 is moving forward. Therefore, in normal driving mode, the forward line 3a becomes a linear first pattern projected forward in the direction of travel of the vehicle 1, and is generated as a line pattern 3 representing the predicted trajectory 5 of the vehicle 1. Specifically, the line shape of the forward line 3a is set to represent the predicted trajectory 5 of the front tires of the vehicle 1. Furthermore, in normal driving mode, the rear line 3c becomes a second linear pattern projected behind the direction of travel of vehicle 1, and is generated as line pattern 3 representing the vehicle's trajectory 6. Specifically, the line shape of the rear line 3c is set to represent the trajectory 6 of the vehicle's rear tires. In addition, in normal driving mode, the center line 3b (third pattern) is used as a linear light that illuminates both sides of the center of vehicle 1.

[0025] Figure 3A schematically illustrates a scenario in which the normal driving mode is applied. In this scenario, a forward line 3a representing the predicted trajectory 5, a rear line 3c representing the passing trajectory 6, and a center line 3b representing linear illumination are projected onto the road surface around a vehicle 1 traveling at a relatively high speed in the driving lane. In this way, line patterns 3 representing the predicted trajectory 5 and the passing trajectory 6 are projected around vehicle 1. This makes it possible to explicitly communicate the direction of travel and driving history of vehicle 1 to pedestrians outside the vehicle and drivers of other vehicles, and also makes it possible to effectively present vehicle 1 while it is in motion.

[0026] Figure 2B shows an example of projection pattern 2 projected in low-speed driving mode. The upper and lower diagrams of Figure 2B are schematic views of vehicle 1 from the side and above. Here, the low-speed driving mode is a projection mode selected, for example, when vehicle 1 is moving forward at a slow speed. Therefore, the low-speed driving mode is used when performing driving that requires slow speed, such as stopping, turning right, turning left, or parking. In low-speed driving mode, as with the normal driving mode described above, three types of line patterns 3 are projected as projection pattern 2: a front line 3a, a center line 3b, and a rear line 3c.

[0027] The line pattern 3 shown in Figure 2B is projected while vehicle 1 is moving forward, similar to Figure 2A. Therefore, in low-speed driving mode, the forward line 3a becomes a linear first pattern projected forward in the direction of travel of the vehicle 1, and is generated as a line pattern 3 representing the predicted trajectory 5 of the vehicle 1. Specifically, the line shape of the forward line 3a is set to represent the predicted trajectory 5 of the front tires of the vehicle 1. Furthermore, in low-speed driving mode, the rear line 3c becomes a second linear pattern projected behind the direction of travel of vehicle 1, and is generated as line pattern 3 representing the vehicle 1's trajectory 6. Specifically, the line shape of the rear line 3c is set to represent the trajectory 6 of the vehicle 1's rear tires.

[0028] Furthermore, in low-speed driving mode, the center line 3b (third pattern) is generated as line pattern 3 representing the predicted trajectory 5 of vehicle 1. Specifically, the center line 3b is set so that its line shape represents the predicted trajectory 5 of the rear tires of vehicle 1. This makes it possible to clearly indicate to pedestrians and others the trajectory that the rear tires are expected to follow when vehicle 1 moves forward slowly. This makes it possible to reduce the risk of being caught in the vehicle's blind spots when turning left or right.

[0029] Figure 3B schematically illustrates an example of a scenario in which the low-speed driving mode is applied, showing vehicle 1a being driven in a parking lot. Vehicle 1a is moving forward from the parking space on the right side of the figure to exit. In this way, when a stationary vehicle 1a moves forward, it is driving at a slow speed, and therefore the low-speed driving mode is used. Here, a forward line 3a representing the predicted trajectory 5 of the front tires and a center line 3b representing the predicted trajectory 5 of the rear tires are projected onto the road surface around vehicle 1a. Note that in Figure 3B, the rear line 3c representing the trajectory 6 of the rear tires of vehicle 1a is omitted from the illustration. In this way, line patterns 3 representing the predicted trajectories 5 of the front and rear tires are projected around the vehicle 1a. This helps to alert pedestrians and effectively avoid accidents such as being caught in the vehicle or colliding with it. Furthermore, the front line 3a may be set to represent the width of vehicle 1. For example, the spacing between the right and left lines may be set to the maximum width of vehicle 1. This makes it possible to encourage steering and other actions that take the vehicle width into consideration.

[0030] Figure 2C shows an example of projection pattern 2 projected in reverse mode. Here, "reverse mode" refers to the projection mode selected when vehicle 1 is moving in reverse (backing up). Reverse mode is used, for example, when parking, such as when reversing to park. In reverse mode, as with the other driving modes described above, three types of line patterns 3 are projected as projection patterns 2: a front line 3a, a center line 3b, and a rear line 3c.

[0031] The line pattern 3 shown in Figure 2C is projected when vehicle 1 is moving in reverse. Therefore, in reverse mode, the forward line 3a becomes a second linear pattern projected behind the direction of travel of the vehicle 1. In this case, the forward line 3a is generated as a line pattern 3 representing the trajectory 6 of the vehicle 1. Specifically, the line shape of the forward line 3a is set to represent the trajectory 6 of the vehicle 1's front tires. In reverse mode, the rear line 3c becomes a first linear pattern projected forward in the direction of travel of vehicle 1. In this case, the rear line 3c is generated as a line pattern 3 representing the predicted trajectory 5 of vehicle 1. Specifically, the line shape of the rear line 3c is set to represent the predicted trajectory 5 of the rear tires of vehicle 1.

[0032] In reverse mode, the center line 3b (third pattern) is generated as line pattern 3 representing the predicted trajectory 5 of vehicle 1. Specifically, the center line 3b is set so that its line shape represents the predicted trajectory 5 of the front tires of vehicle 1. This makes it possible to clearly indicate to pedestrians and others the trajectory that the front tires are expected to follow when vehicle 1 is reversing. This helps to reduce the risk of being caught in obstacles when parking.

[0033] Figure 3B schematically illustrates an example of a scenario in which reverse mode is applied, showing vehicle 1b being driven in a parking lot. Vehicle 1b is moving in reverse to park in the parking space on the left side of the figure. Here, a rear line 3c representing the predicted trajectory 5 of the rear tires and a center line 3b representing the predicted trajectory 5 of the front tires are projected onto the road surface around vehicle 1b. Note that in Figure 3B, the front line 3a representing the trajectory 6 of the front tires of vehicle 1b is omitted from the illustration. In this way, line patterns 3 representing the predicted trajectories 5 of the front and rear tires are projected around the vehicle 1b, making it possible to sufficiently avoid accidents such as getting caught or making contact during reverse operation. Furthermore, similar to the forward movement, the rear line 3c may be set to represent the width of vehicle 1. This makes it possible to perform reverse operations, etc., while checking the vehicle width.

[0034] Figure 2D shows an example of projection pattern 2 projected in parking mode. Here, the parking mode is the projection mode selected when the shift position of vehicle 1 is in parking ("P"), that is, when vehicle 1 is stationary. In parking mode, the line pattern 3 described above is not displayed, and a lighting pattern (hereinafter referred to as parking pattern 7) that surrounds the entire circumference of vehicle 1 is projected as projection pattern 2. Here, parking pattern 7 uses front lighting 7a, side lighting 7b, and rear lighting 7c. Each of the lights 7a to 7c is a gradient pattern in which the color becomes lighter as it moves away from vehicle 1. The design of parking pattern 7 is not limited.

[0035] Thus, projection pattern 2 includes a stopping pattern 7 that is different from line pattern 3. In this embodiment, stopping pattern 7 is an example of another pattern. By using stopping pattern 7, it is possible to communicate to pedestrians and others outside the vehicle that vehicle 1 is stopped with its shift position set to parking, that is, that vehicle 1 is not moving. This allows pedestrians and other vehicles to pass around vehicle 1 with peace of mind. It should be noted that the projection modes are not limited to those described above, and other modes may be set. For example, a Welcome light mode may be set that displays a predetermined lighting pattern when the driver unlocks vehicle 1, opens a door, or starts the engine.

[0036] Figure 4 is a block diagram showing an example configuration of the projection device 100 according to the first embodiment. The projection device 100 includes the projection unit 10 described above, a vehicle information sensor unit 11, a driver monitoring camera 12, a surrounding environment sensor unit 13, a storage unit 15, and a controller 20.

[0037] The projection unit 10 is an element that projects a projection pattern 2 by irradiating it with light, and is configured to change the shape, color, etc., of the projection pattern 2. For the projection unit 10, for example, a projector that emits laser light as the illumination light can be used. By using laser light, it is possible to display a high-brightness projection pattern 2 over a long distance. In addition to the laser light source, an LED light source or a lamp light source may also be used.

[0038] The method of modulating the irradiated light is not limited; for example, light modulation elements using transmissive liquid crystal panels or MEMS (Micro Electro Mechanical Systems) can be used. Furthermore, by combining this with phase modulation elements using reflective liquid crystal panels, it becomes possible to project light in a way that concentrates it within a predetermined range. This makes it possible to significantly improve the brightness of projection pattern 2. Furthermore, the specific configuration of the projection unit 10 is not limited, and for example, a projection light capable of modulating the irradiated light or a laser light source may be used.

[0039] The vehicle information sensor unit 11 has sensors that detect information regarding the status of each part of the vehicle 1. Specifically, the vehicle is equipped with a steering angle sensor to detect the steering angle of the steering wheel, a speed sensor to detect the vehicle's speed, and an acceleration sensor to detect the acceleration acting on the vehicle. Furthermore, the vehicle information sensor unit 11 includes an accelerator opening sensor for detecting the degree of accelerator opening and a brake opening sensor for detecting the degree of brake opening (braking force). Furthermore, the vehicle information sensor unit 11 includes an accelerator pedal pressure sensor that detects the pressure applied to the accelerator pedal (accelerator operating force) and a brake pedal pressure sensor that detects the pressure applied to the brake pedal (brake operating force). These pressure sensors may be sensors that detect the total pressure value applied to the pedals, or sensors that detect the pressure distribution.

[0040] Furthermore, the vehicle information sensor unit 11 includes a shift position sensor that detects the position of the shift lever (shift position), and a side brake sensor that detects whether the side brake is on or off. Furthermore, an ignition sensor for detecting the on / off state of the ignition switch, a turn signal sensor for detecting the on / off state of the turn signals, a hazard sensor for detecting the on / off state of the hazard lights, and a light sensor for detecting the on / off state of the headlights and the switching between high beam and low beam (passing) may also be provided. In addition, any sensor that detects information related to vehicle 1 may be used as the vehicle information sensor unit 11.

[0041] The driver monitoring camera 12 is a camera that photographs the driver of vehicle 1, and is installed inside vehicle 1 so as to be able to photograph the driver from the front. For example, a digital camera equipped with an image sensor such as a CMOS or CCD can be used as the driver monitoring camera 12. In this embodiment, as will be described later, the image captured by the driver monitoring camera 12 is used as data for detecting the driver's line of sight.

[0042] The surrounding environment sensor unit 13 has sensors that detect the state of the surrounding environment of the vehicle 1. In this embodiment, the surrounding environment sensor unit 13 is provided with an object detection sensor for detecting objects in the vicinity of the vehicle 1. As an object detection sensor, for example, a camera that photographs the area around vehicle 1 can be used. For example, a front camera, rear camera, left camera, and right camera installed on the front, rear, left, and right sides of vehicle 1 can be used. These cameras installed on the front, rear, left, and right sides capture the scenery including vehicles and other objects traveling around vehicle 1. Alternatively, a panoramic camera capable of photographing the entire circumference of vehicle 1 using a fisheye lens or the like may be used. Furthermore, radar sensors, ultrasonic sensors, LiDAR sensors, etc., that detect objects in the front, rear, left, and right directions may be used as object detection sensors. These sensors detect the positions of objects around the vehicle 1. Furthermore, the type of object detection sensor is not limited; for example, a camera and other distance measuring sensors may be used in combination.

[0043] The storage unit 15 is a non-volatile storage device. For example, the storage unit 15 may be a recording medium using solid elements such as an SSD (Solid State Drive) or a magnetic recording medium such as an HDD (Hard Disk Drive). Furthermore, the type of recording medium used as the storage unit 15 is not limited; for example, any recording medium that records data non-temporarily may be used.

[0044] The storage unit 15 stores a control program for controlling the overall operation of the projection device 100. This control program corresponds to the program in this embodiment. The storage unit 15 also functions as a computer-readable recording medium on which the program is stored. Furthermore, the memory unit 15 stores data specifying the shape, color, etc., of the projection pattern 2. The types of data stored in the memory unit 15 are not limited, and any data necessary for the operation of the projection device 100 may be stored.

[0045] The controller 20 controls the operation of each block of the projection device 100. The controller 20 has the necessary hardware configuration for a computer, such as a CPU and memory (RAM, ROM). Various processes are executed by the CPU loading the control program stored in the memory unit 15 into the RAM and executing it. The controller 20 functions as an information processing device according to this embodiment.

[0046] As the controller 20, devices such as FPGAs (Field Programmable Gate Arrays) or other PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits) may be used. Alternatively, a processor such as a GPU (Graphics Processing Unit) may be used as the controller 20.

[0047] In this embodiment, the CPU of the controller 20 executes the program according to this embodiment, thereby realizing the following functional blocks: a vehicle information acquisition unit 21, a trajectory calculation unit 22, a projected image determination unit 23, a video data generation unit 24, a gaze detection unit 25, and a surrounding environment recognition unit 26. These functional blocks then execute the information processing method according to this embodiment. Dedicated hardware such as ICs (integrated circuits) may be used as appropriate to realize each functional block. Furthermore, these functional blocks may be realized by other computers or the like that can communicate with the controller 20.

[0048] The vehicle information acquisition unit 21 acquires information (vehicle information) related to the vehicle 1 detected by each sensor of the vehicle information sensor unit 11. In this embodiment, the vehicle information acquisition unit 21 acquires speed-related information regarding the speed of vehicle 1. The speed-related information includes the speed of vehicle 1, information representing acceleration and deceleration that change with speed, and information regarding operations (accelerator operation and brake operation) that change these physical quantities. Speed-related information, including speed information representing the speed of vehicle 1 and acceleration information representing the acceleration of vehicle 1, is acquired. For example, the detection results of the speed sensor and acceleration sensor are read as speed information and acceleration information.

[0049] In addition, speed-related information, such as accelerator and brake information, is acquired. Of this, accelerator information includes the accelerator opening degree and the pressure applied to the accelerator pedal (accelerator operating force). Brake information includes the brake opening degree and the pressure applied to the brake pedal (brake operating force). For example, the detection results of the accelerator opening degree sensor and accelerator pedal pressure sensor are read as the accelerator opening degree and operating force. Similarly, the detection results of the brake opening degree sensor and brake pedal pressure sensor are read as the brake opening degree and operating force.

[0050] Furthermore, the vehicle information acquisition unit 21 acquires driving status information related to the driving state of vehicle 1. This driving status information includes shift lever information and parking brake information. Of these, the shift lever information includes information indicating the state of the shift position of vehicle 1. The parking brake information includes information indicating the state of the parking brake of vehicle 1. For example, the detection results of the shift position sensor and the parking brake sensor are read as shift lever information and parking brake information.

[0051] Furthermore, the vehicle information acquisition unit 21 acquires steering angle information, which indicates the steering angle of the steering wheel. For example, the detection result of the steering angle sensor is read as steering angle information. Furthermore, the vehicle information acquisition unit 21 acquires turn signal information (direction indicator information) related to the operation of the turn signals. The turn signal information indicates whether the right and left turn signals are on or off. In addition, the detection results of each sensor constituting the vehicle information sensor unit 11 are acquired as appropriate.

[0052] The trajectory calculation unit 22 calculates the predicted trajectory 5 and the actual trajectory 6 of the vehicle 1 (see Figures 2 and 3). In this embodiment, the trajectory calculation unit 22 estimates a predicted trajectory 5 that is expected to be followed by the vehicle 1, based on the steering angle information, speed information, and acceleration information of the vehicle 1. Here, for example, the trajectory of the front tires (or rear tires) predicted when the vehicle 1 proceeds with the current steering angle, speed, and acceleration is estimated. At this time, corrections may be appropriately performed according to centrifugal force, tire grip force, etc. Instead of the predicted tire trajectory 5, for example, the trajectory of the center of vehicle 1 may be estimated. The method for estimating the predicted trajectory 5 is not limited; for example, technologies such as trajectory prediction processing used in autonomous driving can be applied.

[0053] Furthermore, in this embodiment, the trajectory 6 of the vehicle 1 is calculated by recording the behavior of the vehicle 1. Here, the trajectory 6 of the front tires (or rear tires) is calculated from the recorded steering angle, speed, and acceleration of the vehicle 1. For this process, techniques such as dead reckoning can be used. Alternatively, GPS positioning or WiFi positioning may be used. For example, it is also possible to record the predicted trajectory 5 and use it as the trajectory 6. The method for estimating the trajectory 6 is not limited; for example, any process capable of reproducing the trajectory of vehicle 1 may be used. In this embodiment, the trajectory calculation unit 22 functions as a predicted trajectory calculation unit and a trajectory calculation unit.

[0054] The projection image determination unit 23 determines the display content and display parameters of projection pattern 2 and outputs data for projection pattern 2. This process controls the display of projection pattern 2. In this disclosure, the process of controlling the display of projection pattern 2 is, for example, the process of setting the display parameters of projection pattern 2. Typically, parameters such as the color, width, length, and blinking of line pattern 3 (see Figures 2A to 2C) are set. In the projection device 100, the projection image determination unit 23 controls the display of the projection pattern 2 projected onto the road surface around the vehicle 1 from the projection unit 10 mounted on the vehicle 1, based on the surrounding environment information. The surrounding environment information is information acquired by the surrounding environment recognition unit 26, which will be described later, based on the output of the surrounding environment sensor unit 13, and represents information such as the condition of the surrounding roads and other vehicles.

[0055] As described above, the first and second patterns of projection pattern 2 (here, the front line 3a and the rear line 3c) include a pair of lines projected onto the left and right sides of the vehicle. In this embodiment, the projection image determination unit 23 controls the display of the target line projected onto the target lane side of the first pattern (forward line 3a) that is subject to the lane change of the vehicle 1, based on surrounding environment information.

[0056] Here, lane change refers to a driving operation that involves crossing a lane boundary to move from the lane you are currently in to an adjacent lane. Lane changes include changing lanes while driving in multiple lanes, merging into a merging lane, or moving onto the shoulder. The lane to which such a driving operation leads is the target lane. In the projection image determination unit 23, the display parameters of the target line projected on the side of the forward line 3a closest to the target lane are controlled using information about vehicles and other things traveling around the vehicle 1 (surrounding environment information). The details regarding the surrounding environment information and the display control of the target line will be explained in more detail later.

[0057] Furthermore, the projection image determination unit 23 controls the shape of the line pattern 3 to represent the trajectory of the vehicle 1 (predicted trajectory 5 and passing trajectory 6) calculated by the trajectory calculation unit 22. Specifically, the projection image determination unit 23 generates a first pattern representing the predicted trajectory 5. That is, the shape of the forward line 3a is set to a shape that follows the predicted trajectory 5. Furthermore, the projection image determination unit 23 generates a second pattern representing the trajectory 6. That is, the shape of the rear line 3c is set to a shape that follows the trajectory 6. Furthermore, the spacing between the left and right lines in the front line 3a (or rear line 3c) may be set to represent the width of the vehicle. In this embodiment, the projection image determination unit 23 corresponds to the projection control unit.

[0058] The video data generation unit 24 generates video data to be output to each projection unit 10 based on the projection pattern 2 data output from the projection image determination unit 23. For example, a frame image is generated that represents the shape of projection pattern 2 as viewed from above vehicle 1. This frame image is then processed to correct distortions and brightness biases associated with the projection, according to the projection angle of each projection section 10. This series of frame images, with the distortions corrected, becomes video data. In addition, any image processing or other actions may be performed to properly project projection pattern 2.

[0059] The gaze detection unit 25 detects the driver's gaze direction based on the driver's image captured by the driver monitoring camera 12. The method for detecting the gaze direction is not limited; for example, the pupil method, which detects the gaze from the direction of the driver's pupil (the black part of the eye), can be used. In addition, any other method capable of detecting the gaze direction using the driver's image may be used. Alternatively, instead of the driver's line of sight, the direction of the driver's face may be detected. In this case, the angle at which the driver rotates their head from side to side may be detected. The gaze detection unit 25 outputs gaze information representing the driver's gaze direction, or face orientation information representing the driver's face orientation, to the projection image determination unit 23.

[0060] Figure 5 is a block diagram showing an example configuration of the surrounding environment recognition unit 26. The surrounding environment recognition unit 26 performs recognition processing regarding the surrounding environment of the vehicle 1 based on the output of the surrounding environment sensor unit 13. It then detects objects (pedestrians, other vehicles, curbs, etc.) present around the vehicle 1 and calculates various types of information about the surrounding environment (surrounding environment information). In other words, the surrounding environment recognition unit 26 calculates and acquires surrounding environment information regarding the environment around the vehicle. In this embodiment, the surrounding environment recognition unit 26 corresponds to the acquisition unit. As shown in Figure 5, the surrounding environment recognition unit 26 includes an object detection unit 30, a relative velocity calculation unit 31, a lane detection unit 32, and a spatial recognition unit 33.

[0061] The object detection unit 30 detects objects around the vehicle 1 based on the detection results from the surrounding environment sensor unit 13. The object detection unit 30 may be a learning device that performs image recognition processing using, for example, machine learning. For example, in a configuration where a camera is provided as the surrounding environment sensor unit 13, the image captured by the camera is input to the object detection unit 30. In this case, objects in the input image are detected and their attributes are identified using reference information generated in advance through learning. The reference information is, for example, dictionary information that stores the correspondence between the type and characteristics of objects. Furthermore, if a DNN (Deep Neural Network) or the like is used as the learning device, recognition model information or the like is used as reference information. In this process, objects around vehicle 1 are detected, and the attributes of objects such as cars (e.g., types of vehicles such as passenger cars, trucks, buses, etc.), motorcycles, bicycles, and people are identified. Each detected object is assigned, for example, a serial number ID. Furthermore, the position of each detected object in the image is determined. At this time, the distance (relative distance) from the position in the image to the object may also be calculated. The object detection unit 30 outputs ID information, attribute information, distance information, and position information for each object.

[0062] The relative velocity calculation unit 31 calculates the relative velocity of the object with respect to the vehicle 1 based on the information about the object output from the object detection unit 30. For example, for each object, the change in distance per unit time is calculated, and the relative velocity to the vehicle (vehicle 1) is calculated from the change in distance. The relative velocity calculation unit 31 outputs ID information, attribute information, distance information, position information, and velocity information for each object.

[0063] The lane detection unit 32 detects the lane (lane markings) on the road in which the vehicle 1 is traveling. For example, from images of the road captured by cameras installed on the front, rear, left, and right sides of the vehicle 1, processing is performed to recognize white lines (boundaries separating road lanes) and curbs using edge detection processing, etc. The area between these white lines and curbs is detected as the driving lane in which the vehicle 1 is traveling. In addition, adjacent lanes adjacent to the driving lane 40, or the shoulder area, etc., may also be detected. The lane detection unit 32 outputs information indicating the lane area.

[0064] The spatial recognition unit 33 recognizes the conditions of the space surrounding the vehicle 1 based on the detection results of the object detection unit 30 and the lane detection unit 32. Specifically, adjacent lanes are identified from the lane information output from the lane detection unit 32. Furthermore, information such as the position and size of other vehicles traveling in the adjacent lane is extracted from the output of the object detection unit 30. The location and size of the available space are then calculated using this extracted information. Here, "available space" refers to, for example, a space where no other vehicles are present. Furthermore, using information on the location and size of available spaces, it is determined whether there is space for vehicle 1 to change lanes (i.e., movable space in the target lane that vehicle 1 can enter). This determination result is output as information on whether or not movable space is available. Furthermore, the system determines the presence or absence of objects (typically other vehicles) in the lane in which the vehicle is traveling, as well as in adjacent lanes and the lanes next to those, and calculates lane condition information based on the information of the objects in each lane. The spatial recognition unit 33 outputs information on available space (location and size), information indicating the presence or absence of movable space, and lane status information.

[0065] In this embodiment, as shown in Figure 5, the outputs of the relative velocity calculation unit 31 and the spatial recognition unit 33 become the final outputs of the surrounding environment recognition unit 26. Therefore, the surrounding environment recognition unit 26 outputs information regarding the object's ID information, attribute information, distance information (relative distance), position information, and velocity information (relative velocity). Furthermore, it outputs information on available space, information indicating movable space, and lane status information. All of this information corresponds to surrounding environment information.

[0066] Furthermore, if a radar sensor, ultrasonic sensor, LiDAR sensor, or the like other than a camera is used as the surrounding environment sensor unit 13, the object detection unit 30 detects the position and distance of the object, and an ID is assigned to the detected object. In this case, the attribute information of the object is not output. Then, the relative velocity calculation unit 31 calculates the relative velocity of the detected object. In this case, the output of the surrounding environment recognition unit 26 (relative velocity calculation unit 31) will be the object's ID information, distance information, position information, and velocity information. If a radar sensor is used, it is possible to detect not only the object but also the relative velocity from its output.

[0067] Figure 6 is a flowchart illustrating a basic example of the operation of the projection device 100. The process shown in Figure 6 is, for example, a loop process that is repeatedly executed during the operation of the projection device 100. Here, it is assumed that vehicle 1 is moving forward and that the normal driving mode (or low-speed driving mode) is selected as the projection mode for projection pattern 2.

[0068] First, speed-related information is acquired by the vehicle information acquisition unit 21 (step 101). Specifically, speed-related information, including speed information, acceleration information, accelerator information (accelerator opening and operating force), and brake information (brake opening and operating force), is read from sensors installed in various parts of vehicle 1. At this time, driving status information (shift lever information, handbrake information), as well as steering angle information, are also read.

[0069] Next, the trajectory calculation unit 22 calculates the trajectory of vehicle 1 (predicted trajectory 5 and trajectory 6) (step 102). Here, as shown in Figures 2A and 2B, the projection mode used during forward driving (normal driving mode, low-speed driving mode) is selected, and the predicted trajectory 5 of the front tires and the trajectory 6 of the rear tires of vehicle 1 are calculated.

[0070] Next, the surrounding environment recognition unit 26 performs various processes to recognize the surrounding environment and calculates surrounding environment information (step 103). In this embodiment, traffic information is calculated as surrounding environment information, representing the traffic conditions around vehicle 1, including the target lane that vehicle 1 is subject to lane change. Traffic information includes information representing available space in the target lane. This includes the aforementioned available space information and information indicating movable space. Traffic information also includes information on the placement of other vehicles traveling around Vehicle 1. This includes, for example, distance information, location information, and lane status information for objects identified as cars based on attribute information. Traffic information also includes information on the relative speed of other vehicles to Vehicle 1. This includes, for example, speed information for objects identified as cars. If attribute information is not detected, only the location information and speed information of the objects will be detected.

[0071] Figure 7 is a schematic diagram showing the view in front of vehicle 1 when changing lanes. In this diagram, the driving lane 40a in which vehicle 1 is traveling merges with the adjacent lane 40b to the right of driving lane 40a. Therefore, in front of vehicle 1, driving lane 40a gradually narrows, and the driver needs to change lanes (merge) into the adjacent lane 40b to the right before driving lane 40a disappears.

[0072] In the surrounding environment recognition unit 26, for example, the object detection unit 30 detects other vehicles 50 traveling in the adjacent lane 40. Here, a truck traveling to the right front of vehicle 1 is detected as another vehicle 50. In addition, other vehicles 50 (not shown) to the sides and rear of vehicle 1 can also be detected. Position information, distance information, etc., for each other vehicle 50 are then calculated. Furthermore, the relative speed calculation unit 31 calculates the relative speed (speed information) with respect to vehicle 1 from the amount of movement of other vehicles 50, etc.

[0073] Furthermore, the lane detection unit 32 detects the boundary lines 51 that divide the lanes (solid lines on the left and dotted lines on the right), and calculates information representing the area of ​​the driving lane 40a. Furthermore, the spatial recognition unit 33 detects the adjacent lane 40b that is adjacent to the area of ​​the driving lane 40a, and calculates the position and size of the empty space 52 (the shaded area in the figure) based on the position information of other vehicles 50 in the adjacent lane 40b. The spatial recognition unit 33 also determines whether there is a sufficiently large empty space 52 in a position where vehicle 1 can move. In the example shown in Figure 7, for example, it is determined that there is no space for vehicle 1 to enter (no space to move) because it is close to another vehicle 50 to the right front. Furthermore, the spatial recognition unit 33 calculates lane status information that shows information (such as spatial arrangement) of other vehicles 50 in the adjacent lane 40b.

[0074] Returning to Figure 6, the projection image determination unit 23 detects a preliminary action for changing lanes (step 104). A preliminary action for changing lanes is, for example, an action performed by the driver immediately before changing lanes, such as moving the eyes or head to visually check the side to which the lane change will occur (the right side in Figure 7). In this embodiment, the gaze detection unit 25 shown in Figure 4 constantly monitors the driver's gaze direction or face orientation. Based on the detection results of the gaze detection unit 25, the projection image determination unit 23 tracks the driver's gaze movement and detects whether or not the driver is checking the left or right side (preparatory action for changing lanes) through the side mirror or by direct visual inspection. Furthermore, if a preliminary action is detected, information indicating the direction of confirmation (left side and left side) as visually confirmed by the driver is calculated simultaneously.

[0075] Next, it is determined whether or not a preliminary action for a lane change has been detected (step 105). For example, if no preliminary action is detected (No. in step 105), the projection image determination unit 23 performs normal display processing for projection pattern 2 (step 106). Normal display processing is the display processing used, for example, when no lane changes are performed. In this process, the left and right lines that make up the front line 3a and rear line 3c are set to the same display parameters (color, width, length, blinking).

[0076] If a preliminary operation is detected (Yes in step 105), the projection image determination unit 23 performs lane change display processing for projection pattern 2 (step 107). The lane change display process is a display process used when a lane change occurs. Specifically, the display parameters of the target line projected onto the target lane side of the forward line 3a (first pattern) that is subject to the lane change are set according to the traffic condition information (surrounding environment information) described above. In the example shown in Figure 7, the adjacent lane 40b to the right of the driving lane 40a becomes the target lane 41. Also, among the forward lines 3a, the rightmost line closest to the target lane 41 becomes the target line 53. Thus, in this embodiment, it is determined whether or not to perform display control of the target line 53 based on the change in the line of sight direction.

[0077] The method for detecting preparatory actions is not limited. For example, a turn signal operation performed at a speed above a certain level may be detected as a preparatory action. Also, both a change in gaze direction and a turn signal operation may be detected as preparatory actions. For example, if a turn signal operation is not detected within a certain time after a change in gaze direction is detected, the lane change display process may be switched to the normal display process. This avoids situations where the forward lane 3a is unnecessarily changed.

[0078] I will now explain in detail the process for displaying lane changes. In this embodiment, the lane change risk associated with vehicle 1 changing lanes is estimated based on traffic condition information. The lane change risk is a parameter that indicates the possibility of contact or collision with other vehicles 50, for example, when vehicle 1 changes lanes.

[0079] For example, if there is an available space 52 (movable space) where vehicle 1 can change lanes, the risk of changing lanes is set low. On the other hand, the smaller the movable space, or the further away the movable space is, the higher the risk of changing lanes is set. Also, if there is no movable space, the risk is set higher than when there is a movable space. For example, if another vehicle 50 is detected in an adjacent lane 40, the closer the vehicle is to the other vehicle 50, the higher the risk of change is set. Also, the greater the relative speed to the other vehicle 50 behind, the higher the risk of change is set. Conversely, the greater the relative speed to the other vehicle 50 in front, the lower the risk of change is set. Furthermore, if there is another vehicle to the side, the risk is set higher compared to when there are other vehicles 50 only behind or in front. As a risk of change, for example, the sum of values ​​calculated for each item can be used. Alternatively, a cost map may be generated from the arrangement and relative speed of other vehicles 50, and the cost value in the movable space may be estimated as a change risk. Furthermore, information such as turn signals from other vehicles 50 and information on predicted movement may also be used. In addition to the conditions of the target lane 41, the risk of change may also be estimated based on the conditions of adjacent lanes, etc., located behind the target lane 41 from the perspective of vehicle 1. Furthermore, the method for estimating the risk of change is not limited, and any method that can estimate the possibility of contact or collision with other vehicles 50 may be used.

[0080] The projection image determination unit 23 performs a process to control the color or blinking of the target line 53 based on the change risk. Specifically, the color corresponding to the change risk is set as the color of the target line 53. Alternatively, if the change risk exceeds a certain value, an instruction to blink the target line 53 is output. In this case, the blinking speed, etc., may be set according to the change risk. This makes it possible to fully support lane-changing driving operations and significantly improve the safety of lane changes.

[0081] Figure 8 is a schematic diagram showing an example of a forward line 3a projected when a lane change is performed. The forward line 3a shown in Figures 8A to 8C is projected when a preliminary action for a lane change is detected. In this case, the line projected to the right side (upper side in the figure) of vehicle 1 among the forward lines 3a is the target line 53. Figure 8A shows the forward line 3a that is projected when a lane change is safe. In this case, the color of the target line 53 is set to, for example, green. The line on the opposite side of the target line 53 (the line to the left of vehicle 1) is set to, for example, the color used in normal display processing (white, etc.). This is also the case in Figures 8B and 8C. Figure 8B shows the forward line 3a that is projected when attention is required for lane changes. In this case, the color of the target line 53 is set to, for example, orange. Figure 8C shows the forward line 3a that is projected when a lane change is dangerous. In this case, the color of the target line 53 is set to, for example, red.

[0082] The safety level of a lane change (safe, caution, dangerous, etc.) is determined, for example, by performing threshold processing on the change risk as described above. For example, if the change risk is between 0% and 30% of its maximum value, the safety level is determined to be "safe"; if the change risk is between 30% and 70%, the safety level is determined to be "caution"; and if the change risk is between 70% and 100%, the safety level is determined to be "dangerous". Of course, the range for determining the change risk can be set arbitrarily.

[0083] In this embodiment, the risk of lane changes is determined by dividing it into multiple levels, and the target line 53 is set to a different color for each determined level. This makes it possible for the driver to easily recognize the safety level of lane changes while keeping their eyes on the road ahead. In addition to controlling the color of the target line 53, blinking or other similar actions may also be controlled. In this case, if the value exceeds a certain level of risk for lane changes, the target line 53 will blink. This ensures that the driver is reliably informed of the safety level of the lane change. In addition, depending on the risk of change, processing such as changing the width or length of the target line 53 may be performed.

[0084] In addition to the target line 53 of the forward line 3a, the display of the center line 3b and the rear line 3c may also be controlled. Specifically, the display of the center line 3b and the rear line 3c projected on the same side as the target line 53, i.e., on the target lane 41 side of the lane change, is controlled. For example, when a preliminary action for a lane change is detected, the lines on the target lane 41 side of the center line 3b and rear line 3c are flashed in orange. This functions as a sign similar to a turn signal, making it possible to inform other vehicles 50 that vehicle 1 is changing lanes. In addition, control may be implemented to change the colors of the center line 3b and the rear line 3c according to the risk of change.

[0085] Returning to Figure 6, once the normal display process or lane change display process is complete, the video data generation unit 24 generates video data to be output to each projection unit 10 based on the data of the line pattern 3 (front line 3a and rear line 3c) (step 108). For example, a frame image representing the line pattern 3 is generated. By performing a correction process on this frame image according to the projection angle of the projection unit 10, etc., video data is generated. The generated video data is output to each projection unit 10. Then, each projection unit 10 provided on the vehicle 1 projects a corresponding line pattern 3 based on the video data (step 109). For example, the projection units 10a and 10e shown in Figure 1 project the front line 3a, the projection units 10b, 10c, 10f, and 10g project the center line 3b, and the projection units 10d and 10h project the rear line 3c.

[0086] Figure 9 is a schematic diagram illustrating an example of lane changes on a merging lane. Figures 9A to 9D sequentially illustrate the process of vehicle 1 changing lanes to merge onto the main road from an entry lane connected to the main road, divided into four stages. The dotted lines in the figures represent, for example, the area illuminated by the headlights of vehicle 1.

[0087] In Figure 9A, vehicle 1, traveling in the entry lane (driving lane 40a), is entering the merging section where the entry lane connects to the main road. The main road is a two-lane road, and in this case, the lane to the left of the main road (adjacent lane 40b) becomes the target lane 41 that will be subject to the lane change. At the stage when entering the merging section, no preliminary action for lane change has been detected, and the projection pattern 2 of vehicle 1 is the pattern set by the normal display process. For example, the front line 3a is set to white, the center line 3b to blue, and the rear line 3c to red.

[0088] In Figure 9B, a preliminary action for a lane change is detected, and the lane change display process is initiated. At this time, the adjacent lane 40b is set as the target lane 41 based on the direction the driver is looking and the direction of the turn signal. Furthermore, in the target lane 41, another vehicle 50a traveling to the right front of vehicle 1 and another vehicle 50b traveling to the right rear of vehicle 1 are detected. At this time, although there is space to move to the right of vehicle 1, the distance to other vehicles 50b is relatively close, so the risk of change is determined to be at a cautionary level. Therefore, the target line 53 is set to orange (see Figure 8B). On the other hand, the line opposite the target line 53 of the forward line 3a is set to the normal display color (white). Additionally, since the lane change display process has started, the center line 3b and rear line 3c will also change to orange.

[0089] In Figure 9C, vehicle 50b, which was previously to the right and rear of vehicle 1, is now traveling alongside vehicle 1. Therefore, there is a high probability of a collision if vehicle 1 changes lanes. For this reason, the risk of lane change is determined to be at a dangerous level. Accordingly, the target lane 53 is set to red (see Figure 8C). In this way, a red target line 53 is projected onto the lane being changed, allowing the driver to recognize that changing lanes is dangerous while keeping their gaze forward.

[0090] In Figure 9D, vehicle 50b, which was previously to the side, overtakes vehicle 1 and is now traveling to the right and in front of it. Vehicle 50c is also detected behind vehicle 50b. Here, since vehicle C is sufficiently far away and its relative speed is low, the risk of change is determined to be at a safe level. Therefore, the target line 53 is set to green (see Figure 8C). This allows the driver to confirm that they are in a position to change lanes without having to change their line of sight.

[0091] In Figure 9D, the portion of the target line 53 that is projected into the merging section (i.e., the portion up to the end of the driving lane 40a) is set to green, while the portion beyond that is set to another color (for example, orange). The portion projected into the merging section is the portion that vehicle 1 can travel on. The portion beyond the merging section is the portion that vehicle 1 cannot travel on. For example, in an image of the forward line 3a, the drivable area can be detected by detecting the position where the forward line 3a is distorted by an obstacle (such as a guardrail) (see Figure 19). Alternatively, the drivable area may be detected from surrounding map information and GPS positioning information. In Figure 9D, the line opposite to the target line 53 is color-coded using the same method.

[0092] In this embodiment, the target line 53 is color-coded into sections where vehicle 1 can travel and sections where vehicle 1 cannot. Then, display control according to the risk of change is applied to the sections where vehicle 1 can travel. This makes it possible to display the distance to the end of the merging section to the driver, and to support them in timing lane changes.

[0093] In the controller 20 according to this embodiment, a projection pattern 2 is projected onto the surrounding road surface from a projection unit 10 provided on the vehicle 1. The display of this projection pattern 2 is controlled based on surrounding environment information regarding the environment around the vehicle 1. This makes it possible to present, for example, the situation in which the vehicle 1 is located to the inside and outside of the vehicle 1 via the projection pattern 2, thereby improving safety during driving.

[0094] When driving a car, changing lanes or merging requires checking the road ahead, assessing the distance and relative speed to vehicles to the sides and diagonally behind, and deciding on the timing of the lane change. Depending on traffic conditions, this may need to be done instantaneously. For example, at night, it can be difficult to instantly judge relative speed. On the other hand, focusing on the sides or diagonally behind can lead to neglecting to check the road ahead, increasing the risk of traffic accidents and near misses. Methods to reduce these risks during lane changes include electronic mirrors and displaying warnings and relative speeds on the instrument panel inside the car, but both involve shifting the driver's gaze, which still carries the risk of neglecting to check for safety ahead.

[0095] In this embodiment, the display of the forward line 3a projected onto the side of the lane to be changed is controlled using information about the surrounding environment in which the vehicle 1 is traveling. Since the forward line 3a is projected directly in front of the driver, the driver can check for the presence, distance, relative speed, etc. of vehicles behind them while keeping their eyes fixed on the road ahead, making it possible to provide safe lane changes and merges to many people.

[0096] <Second Embodiment> A projection device according to a second embodiment of this technology will now be described. In the following description, parts that are similar to the configuration and operation of the projection device 100 described in the above embodiment will be omitted or simplified. Road surface display device (warning to following vehicles) By displaying the wheel trajectory on the road surface, this system solves the driver's challenges.

[0097] Figure 10 is a block diagram showing an example configuration of a projection device according to the second embodiment. In this embodiment, as a display control of projection pattern 2 using surrounding environment information, the display of the second pattern projected behind the vehicle in the direction of travel is controlled in stages according to the collision risk estimated from information about the vehicle following vehicle 1 (following vehicle information). As shown in Figure 10, the projection device 200 includes a projection unit 210, a vehicle information sensor unit 211, a surrounding environment sensor unit 213, a storage unit 215, and a controller 220. Of these, the projection unit 210, the vehicle information sensor unit 211, and the storage unit 215 are configured similarly to, for example, the projection unit 10, the vehicle information sensor unit 11, and the storage unit 15 shown in Figure 4.

[0098] The surrounding environment sensor unit 213 has sensors that detect the state of the surrounding environment of the vehicle 1. In this embodiment, the surrounding environment sensor unit 213 is provided with rear sensors (such as a rear camera, radar sensor, ultrasonic sensor, or LiDAR sensor installed at the rear of the vehicle 1) for detecting objects behind the vehicle 1. In addition, sensors similar to the surrounding environment sensor unit 213 shown in Figure 4 (such as a front camera or side cameras) may be provided.

[0099] The controller 220 controls the operation of each block of the projection device 200. In this embodiment, the CPU of the controller 220 executes the program according to this embodiment stored in the memory unit 215, thereby realizing the following functional blocks: the vehicle information acquisition unit 221, the trajectory calculation unit 222, the projection image determination unit 223, the video data generation unit 224, and the surrounding environment recognition unit 226. Of these, the vehicle information acquisition unit 221, the trajectory calculation unit 222, and the video data generation unit 224 are configured similarly to, for example, the vehicle information acquisition unit 21, the trajectory calculation unit 22, and the video data generation unit 24 shown in Figure 4.

[0100] The projection image determination unit 223 determines the display content and display parameters of projection pattern 2 and outputs data for projection pattern 2. In this embodiment, the projection image determination unit 223 estimates the risk of collision with the following vehicle based on the following vehicle information and discontinuously changes the second pattern according to the collision risk. Here, the following vehicle information refers to information about a vehicle traveling behind vehicle 1, and is the surrounding environment information calculated by the surrounding environment recognition unit 226, which will be described later. Collision risk is a parameter that represents the probability of a following vehicle colliding with vehicle 1, etc. In the projection image determination unit 223, for example, the display parameters of the second pattern are set to change in stages according to the level of collision risk.

[0101] Figure 11 is a block diagram showing an example configuration of the surrounding environment recognition unit 226. The surrounding environment recognition unit 226 performs recognition processing regarding the surrounding environment of vehicle 1 based on the output of the surrounding environment sensor unit 213. It then detects objects (pedestrians, other vehicles, curbs, etc.) present around vehicle 1 and calculates various information about the objects. The surrounding environment recognition unit 226 includes an object detection unit 230 and a relative speed calculation unit 231. The object detection unit 230 detects distance information, position information, etc., of objects around the vehicle 1. The relative speed calculation unit 231 detects the relative speed of each object with respect to the vehicle 1. The object detection unit 230 and the relative velocity calculation unit 231 are configured similarly to, for example, the object detection unit 30 and the relative velocity calculation unit 31 shown in Figure 4.

[0102] In this embodiment, the object detection unit 230 detects objects around the vehicle 1, particularly the vehicle following behind the vehicle 1, and calculates its distance and position information. The relative speed calculation unit 231 then calculates the relative speed of the following vehicle. In other words, the surrounding environment recognition unit 226 calculates information about the vehicle following vehicle 1 based on the detection results from the rear camera, etc. Therefore, the surrounding environment recognition unit 226 calculates and acquires the information about the vehicle following vehicle. The information about the vehicle following vehicle includes the ID information, attribute information, distance information (or position information representing relative position) of the vehicle following vehicle, and speed information representing relative speed. The vehicle following vehicle includes automobiles, motorcycles, bicycles, etc.

[0103] Figure 12 is a flowchart showing a basic example of the operation of the projection device 200. Figure 13 is a schematic diagram showing an example of a rearward pattern according to collision risk.

[0104] The process shown in Figure 13 is, for example, a loop process that is repeatedly executed while the projection device 200 is operating. This process may also be executed when a following vehicle is detected, when the relative speed of the following vehicle is high, or when the distance to the following vehicle is short. First, the vehicle information acquisition unit 221 acquires speed-related information (step 201), and then the trajectory calculation unit 222 calculates the trajectory of vehicle 1 (predicted trajectory 5 and passing trajectory 6) (step 202). Then, the surrounding environment recognition unit 226 detects a following vehicle and calculates information about the following vehicle (step 203).

[0105] Next, the projection image determination unit 223 estimates the collision risk (step 204). The collision risk is set to a higher value, for example, the closer the relative distance to the following vehicle, and the faster the relative speed to the following vehicle. As an example, the collision risk is calculated as relative speed / relative distance. Alternatively, the collision risk may be calculated from either the relative distance or the relative speed alone. Furthermore, the collision risk may be calculated by taking into account the deceleration amount of the vehicle 1 by referring to brake information, etc.

[0106] Next, it is determined whether the collision risk is above a first threshold (step 205). The first threshold is, for example, a threshold for detecting cases where the collision risk is relatively high. The first threshold is set to a value of, for example, 30% to 50% of the maximum value of the collision risk. In addition, the first threshold can be set as appropriate.

[0107] If the collision risk is less than the first threshold (No. in step 205), the projection image determination unit 223 determines to project the second pattern in normal display mode (step 206). Normal display refers to the display method during normal braking operation (normal display processing). Figure 13A schematically illustrates an example of the second pattern (rear line 3c) in normal display. Here, a rear line 3c with a relatively narrow width is projected. The rear line 3c is also set to a red color, for example, the same color as the brake light 35.

[0108] Returning to Figure 12, if the collision risk is greater than or equal to the first threshold (Yes in step 205), it is determined whether the collision risk is greater than or equal to the second threshold (step 207). The second threshold is, for example, a threshold for detecting a sufficiently high collision risk. For example, a value of 50% or more of the maximum value of the collision risk is set as the second threshold. In addition, a second threshold can be set as needed.

[0109] If the collision risk is less than the second threshold (No. in step 207), the projection image determination unit 223 determines that the width of the second pattern should be displayed with a relatively wide fixed width (step 208). This method can be described as a method of statically highlighting and displaying the second pattern. For example, the width of the second pattern may be set to the maximum possible width. Alternatively, the width of the second pattern may be set to approximately 90% or 80% of the maximum width. Figure 13B schematically illustrates an example of the second pattern (rear line 3c) displayed with a wider width. Here, the rear line 3c, which is set to a fixed width that is wider than the normal display, is projected. As a result, in addition to the brake light 35, the wider rear line 3c is projected, making it possible to emphasize to following vehicles that the risk of collision with vehicle 1 is increased.

[0110] Returning to Figure 12, if the collision risk is greater than or equal to the second threshold (Yes in step 207), the projection image determination unit 223 determines to project the second pattern with a flashing display (step 209). Flashing display is a method of displaying a second pattern by making it flash, and can be described as a method of dynamically emphasizing the second pattern. Figure 13C schematically illustrates an example of the second pattern (rear line 3c) in the flashing display. In the flashing display, for example, the width of the rear line 3c is set to be the same as or greater than the fixed width in Figure 13B. The rear line 3c is displayed to flash at least a portion of it. Furthermore, if a portion of the rear line 3c flashes, it is also possible to display it in a way that the flashing portion moves. In other words, the rear line 3c may be displayed as an animation. As a result, the flashing rear line 3c is projected in addition to the brake light 35, making it possible to sufficiently emphasize and communicate the risk of collision with vehicle 1. This makes it possible to effectively draw the attention of following vehicles, etc.

[0111] Returning to Figure 12, in steps 206, 208, and 209, After the display of the two patterns is set, the video data generation unit 224 generates video data to be output to each projection unit 210 based on the set data (step 210). Then, each projection unit 210 provided on the vehicle 1 projects the corresponding line pattern 3 based on the video data (step 211).

[0112] Thus, in this embodiment, the projection image determination unit 223 increases the width of the second pattern if the collision risk is greater than or equal to the first threshold but less than the second threshold which is greater than the first threshold, and flashes the second pattern if the collision risk is greater than or equal to the second threshold. This makes it possible to clearly communicate to following vehicles the possibility of a collision with vehicle 1 in front.

[0113] When calculating collision risk, factors such as the degree of braking may be used. For example, the stronger the braking force, the greater the deceleration of vehicle 1, and therefore the higher the collision risk. This makes it possible to adequately avoid collisions with following vehicles. The example shown in Figure 12 illustrates discontinuous display control of the second pattern in response to collision risk. Instead of collision risk, it is also possible to discontinuously control the second pattern in response to, for example, the degree of braking operation in vehicle 1. Specifically, the projection image determination unit 223 estimates the degree of braking operation performed by the driver. Then, a process is executed to gradually change the display of the rear pattern according to the degree of braking operation. This process is, for example, the process shown in Figure 12, but using the degree of braking operation of vehicle 1 instead of collision risk.

[0114] Furthermore, in the process described with reference to Figure 12, etc., a pattern (rear line) that serves as a warning display for following vehicles is switched and displayed. In addition to this, it is also possible to emit a warning sound towards following vehicles using, for example, a directional speaker. In this case, since an audible warning sound is conveyed along with warnings such as flashing displays, it becomes possible to effectively communicate braking actions and collision risks.

[0115] Accidents have been reported where following vehicles collide with stopped cars at traffic lights or at the end of traffic jams. To prevent such accidents, actions such as "pulling the brakes and steering to make the brake lights flash" and "turning on the hazard lights" are recommended as self-defense measures. One way to automate these actions is to monitor following vehicles and automatically flash the hazard lights if there is a risk of a rear-end collision. However, such actions may not always be sufficiently effective, and there is a need for effective warning methods to make drivers of following vehicles aware that their vehicle is stopped.

[0116] In this embodiment, a rear line 3c (second pattern) displaying the wheel trajectory, etc., is projected behind the moving vehicle 1. Depending on the risk of collision with a following vehicle, this second pattern can be switched to a display with a thicker line width (see Figure 13B) or a flashing display (see Figure 13C). In this way, it is possible to use not only static displays but also dynamic displays to alert the driver of the following vehicle to the road ahead. This makes it possible to sufficiently avoid collisions with following vehicles.

[0117] <Third Embodiment> In this embodiment, a pair of line patterns 3 representing the width of vehicle 1 are projected as a first pattern (typically a forward line 3a) projected in front of the vehicle 1 in the direction of travel. Thus, the first pattern functions as an extension of the vehicle width. The display of the line patterns 3 corresponding to the vehicle width is controlled based on the positional relationship between vehicle 1 and surrounding objects. By displaying an extension of the vehicle's width on the road surface in this way, it becomes possible to clearly indicate the distance to surrounding objects, thereby assisting the driver in recognizing the vehicle's width.

[0118] Figure 14 is a block diagram showing an example configuration of a projection device according to the third embodiment. As shown in Figure 14, the projection device 300 includes a projection unit 310, a vehicle information sensor unit 311, a surrounding environment sensor unit 313, a storage unit 315, and a controller 320. Of these, the projection unit 310, the vehicle information sensor unit 311, the surrounding environment sensor unit 313, and the storage unit 315 are configured similarly to, for example, the projection unit 10, the vehicle information sensor unit 11, the surrounding environment sensor unit 13, and the storage unit 15 shown in Figure 4.

[0119] The controller 320 controls the operation of each block of the projection device 300. In this embodiment, the CPU of the controller 320 executes the program according to this embodiment stored in the memory unit 315, thereby realizing the following functional blocks: the vehicle information acquisition unit 321, the trajectory calculation unit 322, the projection image determination unit 323, the video data generation unit 324, and the surrounding environment recognition unit 326. Of these, the vehicle information acquisition unit 321, the trajectory calculation unit 322, and the video data generation unit 324 are configured similarly to, for example, the vehicle information acquisition unit 21, the trajectory calculation unit 22, and the video data generation unit 24 shown in Figure 4.

[0120] The projection image determination unit 323 determines the display content and display parameters of projection pattern 2 and outputs data for projection pattern 2. In this embodiment, the projection image determination unit 323 generates a pair of lines representing the width of the vehicle 1 as the forward line 3a (first pattern). The vehicle width of vehicle 1 refers, for example, to the maximum width of the vehicle body. Alternatively, the width including side mirrors, etc., may be used as the vehicle width. For example, if the width of each line constituting the front line 3a is sufficiently narrow (e.g., less than 10 cm), the distance between the centers of each line (center width) is set to the width of vehicle 1. If the width of each line is relatively wide (e.g., 10 cm or more), the distance between the outer edges of each line relative to vehicle 1 (outer width) is set to the width of vehicle 1. In addition, the spacing between lines may be set according to the shape of each line, etc., so as to be able to represent the width of vehicle 1. Furthermore, the forward line 3a is set to a shape that represents the predicted trajectory 5 of the vehicle 1, similar to the embodiment described above. In the following, the front line 3a representing the vehicle width will be referred to as the vehicle width line. Furthermore, the lines projected onto the right and left sides of vehicle 1 from the vehicle width line (front line 3a) will be referred to as the right line and left line, respectively.

[0121] Furthermore, the projection image determination unit 323 individually controls the display of a pair of lines based on positional relationship information representing the positional relationship between the vehicle 1 and objects surrounding the vehicle 1. Positional relationship information refers to information that can represent the positional relationship between vehicle 1 and objects in the surrounding environment (other vehicles, pedestrians, curbs, guardrails, road markings, etc.), and is acquired by the surrounding environment recognition unit 326. For example, the distance between an object and vehicle 1, the relative position of an object as seen from vehicle 1, and the relative orientation are all part of the positional relationship information. Alternatively, the positional information of vehicle 1 and the object may be used directly as positional relationship information. The projection image determination unit 323 detects, for example, the proximity of vehicle 1 to an object and the displacement of vehicle 1 relative to the object, based on the positional relationship between vehicle 1 and the object. Display parameters (typically color and blinking) for the right and left lines are set to represent these detection results. This will be explained in more detail later.

[0122] In this embodiment, in addition to the vehicle width line, a projection pattern 2 (hereinafter referred to as the boundary line pattern) representing the boundary line 51 that demarcates the lane in which the vehicle 1 travels is generated. This boundary line pattern is generated, for example, using the detection result of the white line detected by the lane detection unit 332, which will be described later. Furthermore, if, for example, a change in the attitude of the vehicle 1 can be detected, a process is performed to correct the boundary line pattern according to the change in the attitude of the vehicle 1.

[0123] Figure 15 is a block diagram showing an example configuration of the surrounding environment recognition unit 326. The surrounding environment recognition unit 326 detects objects (pedestrians, other vehicles, curbs, etc.) present around the vehicle 1 based on the output of the surrounding environment sensor unit 313, and calculates positional relationship information for each object as surrounding environment information. As shown in Figure 15, the surrounding environment recognition unit 326 includes a projection status determination unit 330, a lane detection unit 332, and a spatial recognition unit 333.

[0124] The projection status determination unit 330 detects the vehicle width line (projection pattern 2) from the forward image captured by the front camera while the vehicle width line is projected. In the process of detecting the vehicle width line, data (shape, color, width, length, blinking, etc.) of the vehicle width line (forward line 3a) output from the projection image determination unit 323 is used. This makes it possible to detect the vehicle width line projected onto the forward image with high accuracy. Similarly, the projection status determination unit 330 detects boundary line patterns from the forward image.

[0125] Furthermore, the projection status determination unit 330 uses how the detected vehicle width line is projected onto the road surface to determine the situation in front of vehicle 1. For example, if the vehicle width line is projected without distortion (such as refraction), it is determined that the road surface ahead is flat. Alternatively, if the vehicle width line is distorted, it is determined that there is an obstacle ahead. If an obstacle is detected, information is calculated indicating the distorted portion of the right and left lines that make up the vehicle width line. Specifically, the position of the curved part in each line (for example, 1 / 4 of the way from the tip) is calculated. This information represents the relative position of the obstacle to vehicle 1.

[0126] In this way, the projection status determination unit 330 detects the position of obstacles on the lines by detecting line distortion from images (forward images) of the lines (right line and left line) as positional ring-shaped information. By utilizing the line distortion in the images of the lines, it is possible to easily detect obstacles present in the path of the vehicle 1. In this embodiment, the projection status determination unit 330 corresponds to the second detection unit. The projection status determination unit 330 outputs the detection result of the forward projection pattern 2 (vehicle width line and boundary line pattern) and the relative position of the obstacle.

[0127] The lane detection unit 332 is configured similarly to the lane detection unit 32 shown in Figure 5, for example, and detects the lane (lane markings) on the road in which the vehicle 1 is traveling. More specifically, it performs a process to detect objects such as white lines and curbs on the road, and their positions are detected. The white lines and curbs are boundary lines 51 that represent the area in which the vehicle 1 should travel. Therefore, it can also be said that the lane detection unit 332 detects the relative position of the boundary lines 51 as positional relationship information of the objects.

[0128] The spatial recognition unit 333 recognizes the conditions of the space surrounding the vehicle 1 based on the detection results of the projection situation determination unit 330 and the lane detection unit 332. In this embodiment, the distance between the boundary line 51 and the vehicle width line is calculated based on the position information of the boundary line 51 of the driving lane 40 on which the vehicle 1 travels and the position information of the vehicle width line. For example, the distance between the right line and the boundary line 51 on the right side of the vehicle 1, and the distance between the left line and the boundary line 51 on the left side of the vehicle 1 are calculated. This process is performed based on a forward image taken in front of the vehicle 1. In this way, the spatial recognition unit 333 detects the distance between the boundary line 51 and the lines (right line and left line) from an image in which both the boundary line 51 and the lines are captured simultaneously. By using an image in which both the boundary line 51 and the lines are captured, it is possible to easily calculate the distance of each line. In this embodiment, the spatial recognition unit 333 corresponds to the first detection unit. The distance between the boundary line 51 and each line representing the vehicle width represents the distance of vehicle 1 to its lane, i.e., the distance between vehicle 1 and the boundary line 51. The spatial recognition unit 333 determines whether each of these distances is within a predetermined threshold.

[0129] Furthermore, the spatial recognition unit 333 calculates the amount of deviation of the boundary pattern from the actual boundary 51 (white line or curb) based on the positional information of the boundary line 51 of the driving lane and the boundary pattern. This amount of deviation is used in the projection image determination unit 323 to calibrate the position of the boundary pattern. The spatial recognition unit 333 outputs the distance between the boundary line 51 and the line, the determination result, and the amount of deviation of the boundary line pattern.

[0130] In this embodiment, as shown in Figure 15, the outputs of the projection status determination unit 330 and the spatial recognition unit 333 become the final output of the surrounding environment recognition unit 326. Therefore, the surrounding environment recognition unit 326 outputs projection status information (information indicating whether or not there is distortion in the vehicle width line, and location information of the distortion points (obstacles)). In addition, the distance between the boundary line 51 and the line, the determination result, and the amount of deviation of the boundary line pattern are output. The method for detecting the positions of objects around vehicle 1 is not limited. For example, SLAM (Simultaneous Localization and Mapping) technology, which self-generates a 3D map of the area around vehicle 1, may be used. Furthermore, an object detection unit, as shown in Figure 5, may be provided, and the position and distance of an object may be detected using a radar sensor, ultrasonic sensor, LiDAR sensor, etc.

[0131] Figure 16 is a flowchart showing a basic example of the operation of the projection device 300. The process shown in Figure 16 is a loop process that is repeatedly executed while the projection device 300 is operating. Typically, when the vehicle 1 is moving forward, the process is executed with the forward line 3a, which is the vehicle width line, as the control target. When the vehicle 1 is moving in reverse, the process may be executed with respect to the rear line 3c, which is generated as the vehicle width line. First, the vehicle information acquisition unit 321 acquires speed-related information (step 301), and then the trajectory calculation unit 222 calculates the trajectory of vehicle 1 (predicted trajectory 5 and passing trajectory 6) (step 302).

[0132] Next, the surrounding environment recognition unit 326 recognizes the positional relationship between the vehicle 1 and surrounding objects and performs a process to calculate positional relationship information (step 303). Specifically, the projection status determination unit 330 calculates positional relationship information between the boundary line 51 and the vehicle width line (distance between the boundary line 51 and the line and the determination result). Furthermore, if an obstacle exists on the vehicle width line, the spatial recognition unit 333 calculates positional relationship information with respect to that obstacle (information indicating whether or not there is distortion in the vehicle width line, and positional information of the distortion point (obstacle)).

[0133] Next, the projection image determination unit 323 sets the display parameters for the vehicle width line 8 (step 304). For example, if the distance between the boundary line 51 and the vehicle width lines (right line and left line) is determined to be smaller than a predetermined threshold, the settings for the color and flashing of the line in question are changed (see Figures 17 and 18). This makes it possible to warn the driver if vehicle 1 is approaching the edge of the lane (approaching the boundary line 51), etc. For example, if an obstacle is detected on the vehicle width lines (right line and left line), the color and flashing settings of the line where the obstacle was detected are changed so that the location of the detected obstacle is clear (see Figures 19 and 20). This makes it possible to warn the driver in advance of the presence of an obstacle that could potentially cause collision if vehicle 1 continues to move forward. If no proximity to the boundary line 51 or any obstacles are detected, the projection image determination unit 323 performs a normal display process (normal display process).

[0134] Next, the video data generation unit 324 generates video data to be output to each projection unit 310 based on the projected image data (step 305). Then, each projection unit 210 provided on the vehicle 1 projects the corresponding line pattern 3 based on the video data (step 306). The following sections will specifically explain the process when approaching the boundary line 51 and when an obstacle is detected.

[0135] Figure 17 is a schematic diagram showing an example of a scene where the vehicle width line is projected. Figure 17A schematically illustrates the view from the driver's perspective when vehicle 1 is traveling straight on a single-lane road (driving lane 40a). Here, a vehicle width line 8 (forward line 3a) representing the vehicle width is projected onto the front of vehicle 1 as projection pattern 2. Of these, the lines projected onto the right and left sides of vehicle 1 become the right line 8R and the left line 8L. Furthermore, a boundary pattern 9 representing the boundary line 51 of the driving lane 40a is projected in front of vehicle 1. Of these, boundary pattern 9a, projected on the right side in the diagram, is projected to overlap with the white line (center line) that represents the boundary with the oncoming lane. Boundary pattern 9b, projected on the left side in the diagram, is projected to overlap with the curb that marks the boundary with the sidewalk.

[0136] In this embodiment, the projection image determination unit 323 controls the color or blinking of each line based on the positional relationship between the boundary line 51 and the right line 8R and the left line 8L. Specifically, the distance between the boundary line 51 and each line representing the vehicle width (hereinafter referred to as the proximity distance) is used as an indicator of the positional relationship. Figure 17A also shows arrows representing the distance between the right line 8R and the right boundary line 51 (white line) (proximity distance on the right side), and the distance between the left line 8L and the left boundary line 51 (curb) (proximity distance on the left side). Note that the arrows representing the proximity distance may or may not be displayed as projection pattern 2.

[0137] As described above, the spatial recognition unit 333 determines whether the approach distance to the right and left sides is lower than a predetermined threshold. Then, the projection image determination unit 323 sets the color and whether or not the right line 8R and left line 8L blink according to the determination result regarding the approach distance. In Figure 17A, the proximity distances on the right and left sides are determined to be greater than a predetermined threshold. In this case, the colors of the right line 8R and the left line 8L are both set to colors used in normal display processing (e.g., white or blue). The colors of the boundary patterns 9a and 9b are also set to colors used in normal display processing (e.g., green).

[0138] In Figure 17B, vehicle 1 is moving further to the right than in the state shown in Figure 17A, and it is determined that the approach distance between the right line 8R and the right boundary line 51 is smaller than a predetermined threshold. In this case, the projection image determination unit 323 sets the color of the right line 8R to a different color from the normal display processing color (for example, orange). Also in the example shown in Figure 17B, the right boundary line pattern 9a is set to a different color from the normal display processing color (for example, red) along with the right line 8R. Alternatively, the right line 8R and the boundary line pattern 9a may be set to blink. Furthermore, the approach distance on the left side is determined to be greater than a predetermined threshold. Therefore, the color of the left line 8L and the left boundary pattern 9b does not change.

[0139] Thus, in this embodiment, if the distance between the boundary line 51 and the right line 8R or the left line 8L is less than a predetermined threshold, a process is executed to change the color of the line or to make the line blink. This makes it possible to clearly communicate to the driver that the vehicle width line 8 is approaching the oncoming lane or sidewalk. As a result, it becomes possible to intuitively recognize whether the driver's current driving position is appropriate, thereby enabling safer driving. Furthermore, it can encourage drivers to be mindful of the vehicle width, supporting them in developing the ability to maintain an appropriate vehicle width distance.

[0140] Figure 18 is a schematic diagram showing an example of a vehicle width line 8 that corresponds to the approach distance. Here, the approach distance is determined in three stages, and the color of the vehicle width line 8 (forward line 3a) is set to a different color at each stage. Such a determination process can be performed, for example, by thresholding using two thresholds. Figure 18A shows the projected vehicle width lines 8 when the approach distances on both the right and left sides are within a safe range. In this case, the colors of the right line 8R and the left line 8L are set to, for example, the normal display colors (white or blue). This is the same setting as, for example, Figure 17A. In Figure 18B, the approach distance on the right is determined to be within the range requiring caution, while the approach distance on the left is determined to be within the safe range. In this case, the color of the right line 8R of the vehicle width lines 8 is set to, for example, orange. The color of the left line 8L remains unchanged. In Figure 18C, the approach distance on the right is determined to be within a dangerous range, while the approach distance on the left is determined to be within a safe range. In this case, the color of the right line 8R of the vehicle width lines 8 is set to, for example, red. The color of the left line 8L remains unchanged.

[0141] In addition to controlling the color of the target line 53, flashing or other effects may also be controlled. In this case, for example, in areas requiring attention, the target line may be set to orange and flash, while in dangerous areas, the target line may be set to red and the frequency of flashing may be increased. This allows the driver to accurately recognize the safety level of their current driving position while keeping their eyes on the road ahead.

[0142] In addition to methods for determining proximity, the display of the vehicle width line 8 may also be controlled using the output of a proximity sensor (such as a radar sensor or other distance measuring sensor). For example, when the vehicle 1 approaches an object nearby (such as a guardrail or another vehicle), the color of the line on the side where the proximity is detected changes and it flashes. This makes it possible to prevent collisions not only in front of the vehicle 1, but also to the side or rear.

[0143] Figure 19 is a schematic diagram showing an example of the display of the vehicle width line 8 projected onto an obstacle. Figure 19 schematically illustrates a scene in which the vehicle width line 8 intersects with an obstacle 55 in front of vehicle 1. For example, in the vehicle width lines 8, the right line 8R intersects with a rectangular obstacle 55a. The right line 8R, projected along the ground surface, bends where the side of the obstacle 55a touches the ground surface and is projected onto the side of the obstacle 55a. In this case, the projection status determination unit 330 detects a point (refraction point P1) where the right line 8R is distorted, based on the original shape of the right line 8R. This refraction point P1 is used as positional information for the obstacle 55a. The portion in front of the refraction point P1 becomes the intersection point 56 where it intersects with the obstacle 55a.

[0144] For example, in the vehicle width line 8, the middle portion of the left line 8L intersects with an obstacle 55b placed on the ground surface with its cylindrical curved surface facing upwards. Here, only the intersection portion 56 where the left line 8L intersects with the curved surface of obstacle 55b is distorted, while the portions in front of and behind the intersection are projected correctly. In this case, the projection condition determination unit 330 detects a refraction point P2 from the front and a refraction point P3 from the rear of the left line 8L, based on the original shape of the left line 8L. These refraction points P2 and P3 are used as positional information for the obstacle 55b. The area between refraction points P2 and P3 becomes the intersection 56 where the obstacle 55b is located.

[0145] In this manner, when an intersection 56 is detected, the display of the vehicle width line 8 is controlled based on the assumption that the vehicle width line 8 is projected onto the obstacle 55. Specifically, the projection image determination unit 323 colors the right line 8R and the left line 8L into the intersection 56 with the obstacle 55 and the other parts, based on the position (refraction point) of the obstacle 55. For example, for the right line 8R, the color of the intersection 56 in front of the refraction point P1 is set to red, and the part behind the refraction point P1 is set to the normal display color. Similarly, for example, for the left line 8L, the color of the intersection 56 between the refraction points P2 and P3 is set to red, and the part in front of the refraction point P2 and the part behind the refraction point P3 are set to the normal display color. This makes it possible to clearly communicate to the driver the presence and location of obstacles 55 that vehicle 1 may come into contact with if it moves forward.

[0146] Figure 20 is a schematic diagram showing an example of a vehicle width line 8 projected when an obstacle 55 is detected. Here, the safety level for the obstacle 55 is determined in three stages, and the color of the vehicle width line 8 (forward line 3a) is set to a different color for each stage. The safety level for the obstacle 55 is set lower (more dangerous) the closer the distance to the obstacle 55 is, or the longer the length of the obstacle 55 (intersection 56). Also, the higher the speed of the vehicle 1, the lower the safety level is set. In addition, the method for determining the safety level for the obstacle 55 is not limited.

[0147] Figure 20A shows the vehicle width lines 8 projected when, for example, no obstacle 55 is detected and the area is safe. In this case, the colors of the right line 8R and the left line 8L are set to the normal display colors (white or blue) throughout. Even if an obstacle 55 is detected, the colors of the lines do not need to be changed if it is sufficiently far away. Figure 20B shows the vehicle width line 8 projected when the safety level of the obstacle 55 is at a level that requires attention. Here, the obstacle is detected at a relatively distant position from the vehicle 1. In this case, the color of the intersection 56 of the right line 8R and the left line 8L is set to, for example, orange. Alternatively, the intersection 56 is set to blink. The colors of the other parts of the line are set to, for example, the normal display color. Figure 20C shows the vehicle width line 8 projected when the safety level of the obstacle 55 is at a dangerous level. Here, an obstacle is detected in the vicinity of vehicle 1. In this case, the color of the intersection 56 is set to, for example, red. Alternatively, the flashing speed of the intersection 56 is set to a high value. The colors of areas other than the intersection 56 are set to, for example, the normal display colors. However, this is not limited to this; for example, the colors of areas other than the intersection 56 may be changed to draw attention to the presence of dangerous obstacles 55.

[0148] One of the fundamental skills of driving is judging the width of your vehicle. This is crucial in various situations, such as deciding whether to enter a narrow road, passing oncoming vehicles, maintaining your position within your lane, and maneuvering close to guardrails or walls. Auxiliary tools for accurately judging vehicle width include corner poles and fender mirrors, but these devices have become less common in recent years due to their detracting from the vehicle's appearance. While there are an increasing number of vehicles with wider forward visibility to make it easier to judge the vehicle's width, the ability to perceive the vehicle's width still largely depends on the driver's "familiarity" with it.

[0149] In this embodiment, a vehicle width line 8 representing the vehicle width is projected by projecting light. The display of the right line 8R and left line 8L of the vehicle width line 8 is controlled according to the positional relationship between the vehicle 1 and objects in the surrounding environment. This makes it possible to intuitively present information to the driver, such as whether the current driving position is close to a white line or whether it is about to come into contact with an obstacle. This enhances the driver's driving safety and supports the acquisition of a correct sense of vehicle width. Furthermore, the projection unit 310 for projecting projection pattern 2 can be configured to fit within the main body of the vehicle 1. Therefore, by using the projection device 300, it is possible to provide a vehicle width perception aid that does not affect the appearance. <Other Embodiments> This technology is not limited to the embodiments described above, and various other embodiments can be realized.

[0150] In the above embodiment, a pattern representing the vehicle's trajectory (passage trajectory or predicted trajectory) was primarily set as the line pattern. However, it is not limited to this; for example, a fixed pattern of straight lines extending along the longitudinal direction of the vehicle 1 can also be used. Alternatively, a pattern with closed ends on the left and right lines may be used. In addition, line patterns of any shape, size, and color may be used as the projection pattern.

[0151] In the above, a standalone controller was given as an example of one embodiment of the information processing device relating to this technology. However, the controller may be configured separately, and the information processing device relating to this technology may be realized by any computer connected to the controller via wired or wireless connection. For example, the information processing method relating to this technology may be executed by a cloud server. Alternatively, the information processing method relating to this technology may be executed by the controller and another computer working in conjunction.

[0152] In other words, the information processing method and program relating to this technology can be executed not only in a computer system composed of a single computer, but also in a computer system in which multiple computers operate in conjunction. In this disclosure, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure or not. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device in which multiple modules are housed in one enclosure, are both considered systems.

[0153] The execution of the information processing method and program related to this technology by a computer system includes both cases where, for example, the acquisition of surrounding environment information and the control of the display of projection patterns are performed by a single computer, and cases where each process is performed by different computers. Furthermore, the execution of each process by a predetermined computer includes having part or all of the process executed by another computer and obtaining the results.

[0154] In other words, the information processing method and program related to this technology can also be applied to cloud computing configurations in which a single function is shared and processed collaboratively by multiple devices via a network.

[0155] It is also possible to combine at least two of the feature features of the present technology described above. In other words, the various feature features described in each embodiment may be combined arbitrarily, regardless of the specific embodiment. Furthermore, the various effects described above are merely examples and not limiting, and other effects may also be exhibited.

[0156] In this disclosure, "same," "equal," "orthogonal," etc., are concepts that include "substantially the same," "substantially equal," "substantially orthogonal," etc. For example, states that fall within a predetermined range (e.g., a range of ±10%) based on "exactly the same," "exactly equal," "exactly orthogonal," etc.

[0157] Furthermore, this technology can also be configured as follows. (1) An acquisition unit that acquires surrounding environment information regarding the environment around the vehicle, A projection control unit controls the display of a projection pattern projected onto the road surface surrounding the vehicle from a projection unit mounted on the vehicle, based on the surrounding environment information. An information processing device equipped with the following. (2) An information processing device as described in (1), The projection pattern includes a first linear pattern projected forward in the direction of travel of the vehicle, and a second linear pattern projected backward in the direction of travel. Information processing device. (3)(2) Information processing device described above, The first pattern includes a pair of lines projected onto the left and right sides of the vehicle, The projection control unit controls the display of the target line projected onto the target lane side of the first pattern, which is subject to the vehicle's lane change, based on the surrounding environment information. Information processing device. (4)(3) The information processing device described above, The surrounding environment information includes traffic condition information representing the traffic conditions around the vehicle, including the target lane. The projection control unit estimates the risk of change associated with the vehicle changing lanes based on the traffic condition information, and controls the color or flashing of the target line based on the risk of change. Information processing device. (5)(4) Information processing device, The traffic information includes information representing available space in the target lane, information on the arrangement of other vehicles traveling around the vehicle, and information on the relative speed of the other vehicles to the vehicle. Information processing device. An information processing device as described in (6), (4), or (5), The projection control unit determines the change risk by dividing it into multiple levels, and sets the target line to a different color for each determined level. Information processing device. (7) An information processing device described in any one of (3) to (6), The projection control unit colors the target line into portions on which the vehicle can travel and portions on which the vehicle cannot travel. Information processing device. (8) An information processing device described in any one of (3) to (7), further, The vehicle is equipped with a gaze detection unit that detects the direction of the driver's gaze, The projection control unit determines whether or not to perform display control of the target line based on the change in the line of sight direction. Information processing device. An information processing device described in any one of (9)(2) to (8), The surrounding environment information includes information about following vehicles, The projection control unit estimates the risk of collision with the following vehicle based on the following vehicle information and discontinuously changes the second pattern according to the collision risk. Information processing device. (10)(9) Information processing device, The following vehicle information includes at least one of the relative distance and relative speed between the vehicle and the following vehicle. Information processing device. An information processing device as described in (11), (9), or (10), The projection control unit increases the width of the second pattern if the collision risk is greater than or equal to a first threshold but less than a second threshold greater than the first threshold, and flashes the second pattern if the collision risk is greater than or equal to the second threshold. Information processing device. An information processing device described in any one of (12)(2) to (11), The surrounding environment information includes positional relationship information that represents the positional relationship between the vehicle and objects around the vehicle. The projection control unit generates a pair of lines representing the vehicle width as the first pattern, and individually controls the display of the pair of lines based on the positional relationship information. Information processing device. (13)(12) Information processing device, The aforementioned object is a boundary line that represents the area in which the vehicle should travel, The projection control unit controls the color or blinking of the line based on the positional relationship between the boundary line and the line. Information processing device. (14)(13) Information processing device, The acquisition unit includes a first detection unit that detects the distance between the boundary line and the line from an image in which the boundary line and the line are simultaneously captured, as positional relationship information. The projection control unit changes the color of the line or makes the line blink if the distance between the boundary line and the line is less than a predetermined threshold. Information processing device. An information processing device described in any one of (15)(12) to (14), The acquisition unit includes a second detection unit that detects the position of an obstacle on the line by detecting distortion of the line from an image of the line taken as positional relationship information. The projection control unit colors the line into the portion that intersects with the obstacle and the other portion, based on the position of the obstacle. Information processing device. An information processing device described in any one of (16)(2) to (15), further, The system includes a predicted trajectory calculation unit that calculates the predicted trajectory that the aforementioned vehicle is expected to pass through. The projection control unit generates the first pattern representing the predicted trajectory. Information processing device. An information processing device described in any one of (17)(2) to (16), further, The system includes a trajectory calculation unit that calculates the trajectory of the vehicle, The projection control unit generates the second pattern representing the trajectory. Information processing device. (18) Obtain surrounding environmental information regarding the environment around the vehicle, Based on the surrounding environment information, the display of the projection pattern projected onto the road surface around the vehicle from the projection unit mounted on the vehicle is controlled. A method of information processing performed by a computer system. (19) A step of obtaining surrounding environment information regarding the environment around the vehicle, A step of controlling the display of a projection pattern projected onto the road surface surrounding the vehicle from a projection unit mounted on the vehicle, based on the surrounding environment information. A program that causes a computer system to execute something. (20) A projection unit mounted on a vehicle that projects a projection pattern onto the road surface surrounding the vehicle, An acquisition unit that acquires surrounding environment information regarding the environment around the vehicle, A projection control unit controls the display of the projection pattern projected from the projection unit based on the surrounding environment information. A projection device equipped with the following: [Explanation of symbols]

[0158] 1, 1a, 1b... Vehicles 2…Projection Pattern 3…Line Pattern 3a…Front line 3c... Rear line 3b... Center line 4…Direction of travel 5…Predicted trajectory 6…Passing trajectory 8L…Left line 8R…Right line 8... Vehicle width line 10, 10a~10h, 210, 310...Projection section 12… Driver surveillance camera 13, 213, 313... Surrounding environment sensor section 20, 220, 320… controllers 22, 222, 322...trajectory calculation section 23, 223, 323... Projection image determination unit 26, 226, 326... Surrounding environment recognition unit 41…Target lane 51...Boundary Line 53…Target line 55, 55a, 55b... Obstacles 100, 200, 300…projection device

Claims

1. An acquisition unit that acquires surrounding environment information regarding the environment around the vehicle, A projection control unit controls the display of a projection pattern projected onto the road surface surrounding the vehicle from a projection unit mounted on the vehicle, based on the surrounding environment information. It is equipped with, The projection pattern includes a first linear pattern projected forward in the direction of travel of the vehicle. The surrounding environment information includes positional relationship information that represents the positional relationship between the vehicle and objects around the vehicle. The projection control unit generates a pair of vehicle width lines representing the vehicle width as the first pattern, and individually controls the display of the pair of vehicle width lines based on the positional relationship information. The acquisition unit includes an obstacle detection unit that detects the position of an obstacle on the vehicle width line by detecting distortion of the vehicle width line from an image of the vehicle width line taken as positional relationship information. The projection control unit changes the display content of the vehicle width line at the intersection with the obstacle and other parts based on the position of the obstacle. Information processing device.

2. An information processing apparatus according to claim 1, The projection pattern includes a second linear pattern projected behind the direction of travel. Information processing device.

3. An information processing apparatus according to claim 1, The first pattern includes a pair of lines projected onto the left and right sides of the vehicle, The projection control unit generates a predetermined pattern as the pair of lines when the vehicle does not change lanes, and when the vehicle does change lanes, it controls the display of the target line projected onto the target lane side of the pair of lines that is subject to the lane change, based on the surrounding environment information, so that the display content is different from the predetermined pattern. Information processing device.

4. An information processing apparatus according to claim 3, The surrounding environment information includes traffic condition information representing the traffic conditions around the vehicle, including the target lane. The projection control unit estimates the change risk associated with the vehicle changing lanes based on the traffic condition information, sets the color of the target line to a color set in accordance with the change risk value, or flashes the target line if the change risk value exceeds a predetermined value. Information processing device.

5. An information processing apparatus according to claim 4, The traffic information includes information representing available space in the target lane, information on the arrangement of other vehicles traveling around the vehicle, and information on the relative speed of the other vehicles to the vehicle. Information processing device.

6. An information processing apparatus according to claim 4, The projection control unit determines the change risk by dividing it into multiple levels, and sets the target line to a different color for each determined level. Information processing device.

7. An information processing apparatus according to claim 3, The surrounding environment information includes obstacle information representing the location of obstacles around the vehicle, The projection control unit detects the range in which the vehicle can travel based on the obstacle information, and based on the detection result, colors the target line into parts in which the vehicle can travel and parts in which the vehicle cannot travel. Information processing device.

8. The information processing apparatus according to claim 3, further, The vehicle is equipped with a gaze detection unit that detects the direction of the driver's gaze, The projection control unit detects a preliminary operation for lane change based on the change in line of sight, and executes display control of the target line when the preliminary operation for lane change is detected. Information processing device.

9. An information processing apparatus according to claim 2, The surrounding environment information includes information about following vehicles, The projection control unit estimates the risk of collision with the following vehicle based on the following vehicle information, and discontinuously changes the second pattern according to the value of the collision risk, so that the second pattern becomes one of a plurality of different patterns set in accordance with the value of the collision risk. Information processing device.

10. An information processing apparatus according to claim 9, The following vehicle information includes at least one of the relative distance and relative speed between the vehicle and the following vehicle. Information processing device.

11. An information processing apparatus according to claim 9, The projection control unit increases the width of the second pattern if the collision risk is greater than or equal to a first threshold but less than a second threshold greater than the first threshold, and flashes the second pattern if the collision risk is greater than or equal to the second threshold. Information processing device.

12. An information processing apparatus according to claim 1, The aforementioned object is a boundary line that represents the area in which the vehicle should travel, The projection control unit controls the display of the vehicle width line according to the distance between the boundary line and the vehicle width line, so that it becomes one of a plurality of different patterns set corresponding to the distance between the boundary line and the vehicle width line. Information processing device.

13. An information processing apparatus according to claim 12, The acquisition unit includes a distance detection unit that detects the distance between the boundary line and the vehicle width line from an image of the boundary line and the vehicle width line taken simultaneously, as positional relationship information. The projection control unit changes the color of the vehicle width line or makes the vehicle width line blink if the distance between the boundary line and the vehicle width line is less than a predetermined threshold. Information processing device.

14. An information processing apparatus according to claim 1, The projection control unit colors the line into the portion that intersects with the obstacle and the other portion, based on the position of the obstacle. Information processing device.

15. The information processing apparatus according to claim 1, further, The system includes a predicted trajectory calculation unit that calculates the predicted trajectory that the aforementioned vehicle is expected to pass through. The projection control unit generates the first pattern representing the predicted trajectory. Information processing device.

16. The information processing apparatus according to claim 2, further, The system includes a trajectory calculation unit that calculates the trajectory of the vehicle, The projection control unit generates the second pattern representing the trajectory. Information processing device.

17. Steps include acquiring surrounding environment information regarding the environment around the vehicle, A step of controlling the display of a projection pattern projected onto the road surface surrounding the vehicle from a projection unit mounted on the vehicle, based on the surrounding environment information. An information processing method performed by a computer system, The projection pattern includes a first linear pattern projected forward in the direction of travel of the vehicle. The surrounding environment information includes positional relationship information that represents the positional relationship between the vehicle and objects around the vehicle. The step of controlling the display of the projection pattern involves generating a pair of vehicle width lines representing the vehicle width as the first pattern, and individually controlling the display of the pair of vehicle width lines based on the positional relationship information. The step of acquiring the surrounding environment information involves detecting the position of obstacles on the vehicle width line by detecting distortion of the vehicle width line from an image of the vehicle width line taken as positional relationship information. The step of controlling the display of the projection pattern involves changing the display content of the portion of the vehicle width line that intersects with the obstacle and the other portions based on the position of the obstacle. Information processing methods.

18. Steps include acquiring surrounding environment information regarding the environment around the vehicle, A step of controlling the display of a projection pattern projected onto the road surface surrounding the vehicle from a projection unit mounted on the vehicle, based on the surrounding environment information. A program that causes a computer system to execute, The projection pattern includes a first linear pattern projected forward in the direction of travel of the vehicle. The surrounding environment information includes positional relationship information that represents the positional relationship between the vehicle and objects around the vehicle. The step of controlling the display of the projection pattern involves generating a pair of vehicle width lines representing the vehicle width as the first pattern, and individually controlling the display of the pair of vehicle width lines based on the positional relationship information. The step of acquiring the surrounding environment information involves detecting the position of obstacles on the vehicle width line by detecting distortion of the vehicle width line from an image of the vehicle width line taken as positional relationship information. The step of controlling the display of the projection pattern involves changing the display content of the portion of the vehicle width line that intersects with the obstacle and the other portions based on the position of the obstacle. program.

19. A projection unit mounted on a vehicle, which projects a projection pattern onto the road surface surrounding the vehicle, An acquisition unit that acquires surrounding environment information regarding the environment around the vehicle, A projection control unit controls the display of the projection pattern projected from the projection unit based on the surrounding environment information. It is equipped with, The projection pattern includes a first linear pattern projected forward in the direction of travel of the vehicle. The surrounding environment information includes positional relationship information that represents the positional relationship between the vehicle and objects around the vehicle. The projection control unit generates a pair of vehicle width lines representing the vehicle width as the first pattern, and individually controls the display of the pair of vehicle width lines based on the positional relationship information. The acquisition unit includes an obstacle detection unit that detects the position of an obstacle on the vehicle width line by detecting distortion of the vehicle width line from an image of the vehicle width line taken as positional relationship information. The projection control unit changes the display content of the vehicle width line at the intersection with the obstacle and other parts based on the position of the obstacle. Projection device.

Images