Information projection device

Abstract

This invention provides an information projection device, relating to a technology capable of more appropriately displaying images facing the driver and images facing people around the vehicle. According to the invention, it is possible to contribute to the Sustainable Development Goal 3, "Good Health and Well-being." An information projection device mounted in a vehicle and capable of projecting images onto a road surface includes: an acquisition unit that acquires information about the vehicle; and a projection unit that projects images, wherein, when the vehicle is reversing, the acquisition unit acquires detection information about obstacles around the vehicle, and the image is projected based on the detected information.

Description

Technical Field

[0001] This invention relates to an information projection device, an information projection system, and an information projection method. Background Technology

[0002] As shown in Patent Documents 1 and 2, techniques for projecting information onto the road surface are known.

[0003] Patent Document 1 discloses a vehicle projection device that projects a path guidance image onto the road surface in front of the vehicle. This vehicle projection device includes a path search unit and a projection unit. The path search unit searches for the vehicle's current position and the path from the current position to a set destination. When the vehicle approaches a fork in the path to a point where the occupants can see the fork, the projection unit projects a path guidance image onto the road surface in front of the vehicle, based on the path information obtained by the path search unit, to guide the vehicle in the direction of the fork.

[0004] Patent Document 2 discloses a technology that can easily and understandably alert pedestrians and other moving objects, and as an example, it discloses a road surface projection method. This road surface projection method includes: a step of pre-storing image data of an image to be projected onto the road surface; a step of monitoring the area surrounding the vehicle; a step of identifying a moving object based on the monitoring results; a step of setting an identification area in the monitored surrounding area; and a step of reading the pre-stored image data and projecting the image onto the road surface near the moving object when the identified moving object is located within the identification area.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2012-247369

[0008] Patent Document 2: Japanese Patent Application Publication No. 2008-007079 Summary of the Invention

[0009] The technical problem that the invention aims to solve

[0010] The technical problem of the present invention is to provide a technology that can more appropriately display images facing the driver and images facing vehicles and people around the vehicle.

[0011] Technical means to solve the problem

[0012] According to an embodiment of the present invention, an information projection device mounted in a vehicle and capable of projecting images onto a road surface is provided, comprising: an acquisition unit that acquires information about the vehicle; and a projection unit that projects images, wherein, when the vehicle is reversing, the acquisition unit acquires detection information about obstacles around the vehicle, and projects images based on the detected information.

[0013] Invention Effects

[0014] According to the present invention, a technology is provided that can more appropriately display images facing the driver and images facing vehicles and people around the vehicle. Furthermore, technical problems, features, and effects other than those described above will become clear through the following detailed description of embodiments. Attached Figure Description

[0015] Figure 1 This is an example diagram showing an overview of a vehicle equipped with an information projection device.

[0016] Figure 2 This is a diagram illustrating an example of a structure installed in a vehicle.

[0017] Figure 3A This is an example of an image display showing information projected onto the road surface in front of a vehicle.

[0018] Figure 3B This is an example of an image display showing information projected onto the road surface behind a vehicle.

[0019] Figure 4A This is a diagram illustrating an example of the structure of a projection device.

[0020] Figure 4B This is a diagram illustrating an example of the structure of a projection device.

[0021] Figure 4C This is a diagram illustrating an example of the structure of a projection device.

[0022] Figure 5A This is a diagram used to illustrate an example of the configuration of optical components, etc.

[0023] Figure 5B This is a diagram used to illustrate an example of the configuration of optical components, etc.

[0024] Figure 5C This is a diagram used to illustrate an example of the configuration of optical components, etc.

[0025] Figure 6 This is an example diagram showing the display area of ​​the image seen by the driver.

[0026] Figure 7 This is a diagram illustrating an example of how image light is projected onto a display area.

[0027] Figure 8 This is a diagram illustrating one example of an embodiment of information projection.

[0028] Figure 9 This diagram illustrates the method for detecting obstacles in various regions.

[0029] Figure 10 It is a diagram used to illustrate how an image is projected according to the area where an object exists.

[0030] Figure 11A This is a diagram used to illustrate an example of how an image is projected accordingly based on a region.

[0031] Figure 11B This is a diagram used to illustrate an example of how an image is projected accordingly based on a region.

[0032] Figure 11C This is a diagram used to illustrate an example of how an image is projected accordingly based on a region.

[0033] Figure 11D This is a diagram used to illustrate an example of how an image is projected accordingly based on a region.

[0034] Figure 12A This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0035] Figure 12B This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0036] Figure 12C This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0037] Figure 13A This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0038] Figure 13B This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0039] Figure 13C This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0040] Figure 14A This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0041] Figure 14B This is a diagram used to illustrate a variation of how an image is projected accordingly based on a region.

[0042] Figure 15 This is a diagram illustrating an example of the process of projecting information.

[0043] Figure 16 This is a diagram used to illustrate an example of a table representing an object.

[0044] Figure 17 This is a diagram illustrating an example of an embodiment of projecting an image while a vehicle is reversing.

[0045] Figure 18 This is a diagram used to illustrate detection distance and risk level.

[0046] Figure 19 This is a diagram used to illustrate an example of how projection occurs according to an object.

[0047] Figure 20A This is a diagram illustrating an example of the positional relationship between a vehicle and a detected object.

[0048] Figure 20B This is a diagram illustrating an example of the positional relationship between a vehicle and a detected object.

[0049] Figure 20C This is a diagram illustrating an example of the positional relationship between a vehicle and a detected object.

[0050] Figure 21A This is a diagram illustrating an example of how an image is projected according to the position of an object.

[0051] Figure 21B This is a diagram illustrating an example of how an image is projected according to the position of an object.

[0052] Figure 21C This is a diagram illustrating an example of how an image is projected according to the position of an object.

[0053] Figure 21D This is a diagram illustrating an example of how an image is projected according to the position of an object.

[0054] Figure 21E This is a diagram illustrating an example of how an image is projected according to the position of an object.

[0055] Figure 21F This is a diagram illustrating an example of how an image is projected according to the position of an object.

[0056] Figure 22 This is a diagram used to illustrate a variation of the image projected when a vehicle is reversing.

[0057] Figure 23 This is a diagram used to illustrate an example of how the degree of danger is represented according to the object.

[0058] Figure 24 This is a diagram used to illustrate an example of how projection occurs according to an object.

[0059] Figure 25A This is a diagram used to illustrate an example of the direction and speed of movement of objects and vehicles.

[0060] Figure 25B This is a diagram used to illustrate an example of the direction and speed of movement of objects and vehicles.

[0061] Figure 25C This is a diagram used to illustrate an example of the direction and speed of movement of objects and vehicles.

[0062] Figure 25D This is a diagram used to illustrate an example of the direction and speed of movement of objects and vehicles. Detailed Implementation

[0063] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. These embodiments are examples for illustrating the present invention, and appropriate omissions and simplifications have been made to make the description clearer. The present invention can also be implemented in various other ways. Unless otherwise specified, each constituent element can be a single element or a plurality of elements.

[0064] To facilitate understanding of the invention, the positions, sizes, shapes, and extents of the constituent elements shown in the accompanying drawings do not always represent their actual positions, sizes, shapes, and extents. Therefore, the invention is not necessarily limited to the positions, sizes, shapes, and extents disclosed in the accompanying drawings.

[0065] As examples of various types of information, we sometimes use terms like "table," "list," and "queue" to describe them, but various types of information can also be expressed using data structures other than these. For example, we can refer to various types of information such as "XX table," "XX list," and "XX queue" as "XX information." When describing identification information, we use terms like "identification information," "identifier," "name," "ID," and "number," but these terms can be interchanged.

[0066] When multiple constituent elements have the same or identical function, different suffixes are sometimes added to the same reference numerals for description. Conversely, when it is not necessary to distinguish between these multiple constituent elements, the suffixes are sometimes omitted in the description.

[0067] In implementation methods, the term "processing performed by executing a program" is sometimes used. Here, the computer executes the program via a processor (e.g., CPU, GPU) and uses storage resources (e.g., memory) and interface devices (e.g., communication ports) to perform the processing specified by the program. Therefore, the entity performing the processing by executing a program can also be considered as a processor. Similarly, the entity performing the processing by executing a program can also be a controller, device, system, computer, or node that has a processor. The entity performing the processing by executing a program is simply a computing unit and can include dedicated circuitry for performing specific processing. Here, dedicated circuitry can be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0068] The program can be installed onto a computer from a program source. The program source can be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server distributes the program to other computers. Furthermore, in this embodiment, two or more programs can be implemented by one program, or one program can be implemented by two or more programs.

[0069] In this embodiment, an example of a technique for displaying information on the road surface using a projection device mounted on a vehicle will be described. In this embodiment, for the vehicle and the driver, the horizontal direction is the left-right direction, the lateral direction of the vehicle, or the width direction of the vehicle; the vertical direction is the up-down direction of the vehicle, or the longitudinal direction; and the perpendicular direction orthogonal to the lateral or longitudinal direction of the vehicle is the front-back direction of the vehicle or the direction of travel of the vehicle.

[0070] Figure 1 It is a diagram showing the outline of the vehicle. For example... Figure 1 As shown, vehicle 2 includes, for example, a projection device 11 (information projection device), an in-vehicle image display device 12, a vehicle navigation system 150, headlights (i.e., headlights 13), taillights (not shown), turn signals (direction indicators) 14, and a controller 100. Furthermore, vehicle 2 incorporates an in-vehicle system 300 including these structures. This in-vehicle system 300 is equipped with an in-vehicle network, and the controller 100 is capable of sending and receiving data or information with these structures, as well as those described later. The in-vehicle system 300 may include, for example, CAN (Controller Area Network), in-vehicle Ethernet, or LIN (Local Interconnect Network).

[0071] Furthermore, the vehicle-mounted system 300 can communicate with the outside of the vehicle via a communication device. As an example of communication with the outside of the vehicle, either direct or indirect communication methods can be used. In direct communication, frequency bands such as the 760MHz band and the 5.9GHz band, which are internationally used for ITS (Intelligent Transport System) communication, are used for direct communication between vehicles, between roads, and between people. Conversely, in indirect communication, mobile phone frequency bands other than 5.9GHz are used to communicate indirectly via a mobile operator's network. The vehicle-mounted system 300 can, for example, send and receive data or information with a server 24 connected to network 21 via access point 22 or relay base station 23 on network 21. Additionally, the vehicle-mounted system 300 can communicate with other vehicles, external devices 25, portable terminals 26 held by pedestrians, and infrastructure such as terminals installed on the road on which the vehicle travels. As an example of communication between roads, traffic congestion information and weather information are received using radio beacons or optical beacons.

[0072] As an example of data or information acquired by the vehicle system 300, vehicle information 4 includes, for example, speed information, gear information, steering wheel angle information, light information, external light information, distance information, infrared information, engine on / off information, camera image information, accelerometer / gyroscope information, GPS information, navigation information, vehicle-to-vehicle communication information, road-to-road communication information, pedestrian-to-vehicle communication information, LiDAR (Light Detection and Ranging), road surface condition information, raindrop correlation information, wiper correlation information, and other vehicle sensor information. Camera image information includes in-vehicle camera image information and external camera image information. GPS information includes current time information, latitude, and longitude information. This vehicle information can be obtained, for example, from sensors described later.

[0073] Additionally, vehicle information 4 includes information input by the driver. The driver can input information using an appropriate device. This device can be a device pre-installed in the vehicle. Alternatively, the device can be an external device that can connect to the vehicle system 300 via wired or wireless means, such as an input device, tablet, smartphone, AR (Augmented Reality) glasses or HMD (Head Mounted Display) wearable device, or personal computer.

[0074] The vehicle system 300 can use the acquired data or information to perform various controls such as driving control and display control.

[0075] The projection device 11 projects image light for displaying information. The specific structure of the projection device 11 will be described later. The driver, people around the vehicle 2, such as pedestrians walking near the vehicle, and drivers and passengers of other vehicles traveling near the vehicle can see the image projected by the projection device 11.

[0076] The in-vehicle image display device 12 generates image light for displaying information and projects the image light onto a designated display area 5 of the windshield 3. Thus, the in-vehicle image display device 12 allows the driver (driver's viewpoint) to see a virtual image corresponding to the displayed image, superimposed on the scenery. Furthermore, while this example projects image light onto the display area 5 of the windshield 3, the projection unit for projecting the image light could also be a projection component such as a combiner. The in-vehicle image display device 12 could, for example, be a known HUD (Head-Up Display) that includes a light source, a display panel for forming the displayed image, and a control unit.

[0077] Vehicle navigation device 150 is an electronic device referred to as a vehicle navigation system. Vehicle navigation device 150 is, for example, a device that uses map information and GPS location information to indicate the current location and route to the destination. Vehicle navigation device 150 can use traffic information such as VICS (Vehicle Information and Communication System, registered trademark) to indicate an efficient route to the destination.

[0078] In this example, a pair of headlights 13 are positioned at the front of the vehicle, one on the left and one on the right. The headlights 13 integrate the light fixtures that serve as light emitters. The turn signals 14 are devices used to indicate the direction to the surroundings when turning left or right, changing lanes, etc., and are also positioned at the front of the vehicle, one on the left and one on the right, similar to the headlights 13.

[0079] The controller 100 is an electronic control unit (ECU) installed within the vehicle 2, which may include, for example, a processing unit (e.g., a central processing unit), a storage unit, and input / output (I / O) units. The storage unit may be configured using a main storage unit and an auxiliary storage unit. The main storage unit is the working area of ​​the processing unit, where the processing unit stores data and performs data processing. The main storage unit may be, for example, RAM (Random Access Memory). The auxiliary storage unit is a non-volatile storage device that non-volatilely stores data. The auxiliary storage unit may be, for example, ROM (Read-Only Memory).

[0080] The controller 100 receives data or information via input / output devices and the vehicle network. Additionally, the controller 100 can control various devices connected to the vehicle network via the input / output devices and the vehicle network.

[0081] For example, vehicle information 4 can be input via an input / output device, or information obtained from server 24. Then, based on the obtained information, controller 100 can control the driving of headlight 13, the driving of turn signals 14, the operation of projection device 11, the operation of in-vehicle image display device 12, the operation of vehicle navigation device 150, etc.

[0082] The projection device 11 can connect to various sensors mounted on the vehicle 2, devices mounted on the vehicle 2 (such as a vehicle navigation system 150), and communication devices for communicating with the outside of the vehicle to acquire data or information. Then, the projection device 11 can use the acquired data or information to generate image light for displaying information and project the image light.

[0083] Furthermore, the projection device 11 can omit communication with the controller 100. Here, the projection device 11 can, for example, communicate via CAN, vehicle Ethernet, or other similar means to obtain data or information from a different structure than the controller 100. Similarly, the in-vehicle image display device 12 can also be configured in the same way.

[0084] Alternatively, the controller 100 may generate image data using the acquired data or information, and send the generated image data to the projection device 11. The projection device 11 then generates image light for displaying information based on the image data generated by the controller 100 and projects the image light. Alternatively, the image data may be generated by the image generation unit 527 (described later) instead of the controller 100, and the generated image data may be sent to the projection device 11. Here, the controller 100 and the projection device 11 may communicate via, for example, FPD-LinkIII or GMSL (Gigabit Multimedia Serial Link), and the projection device 11 may acquire image data from the controller 100. Furthermore, the projection device 11 processes the acquired image data, performing image processing such as distortion correction, color correction, brightness correction, contrast correction, and conversion (e.g., decoding). The image data may be stored in advance in the storage device of the controller or the projection device, or it may be processed in real-time without being stored in the storage device. If the data is stored in advance by the controller or the projection device's storage device, the stored image data can be gradually changed through system-side updates or user operations using wired or wireless methods. Similarly, as described above, the projection device 11 connects to various sensors installed in the vehicle, devices installed in the vehicle 2 (such as the vehicle navigation system 150), and communication devices for communicating with the outside of the vehicle, acquiring data or information through communication based on CAN, in-vehicle Ethernet, etc. Then, the projection device 11 can generate image light for displaying information based on the acquired information and project the image light. The in-vehicle image display device 12 can also be configured similarly.

[0085] Furthermore, the controller 100 can coordinately control the headlights 13 and the projection device 11. For example, the controller 100 can illuminate the headlights 13 and cause the projection device 11 to project an image light in front of the vehicle. Alternatively, the controller 100 can, for example, turn off the headlights 13 and cause the projection device 11 to generate an image of the information to be displayed and project the image light in front of the vehicle. In other words, the headlights 13 and the projection device 11 can operate in coordination via the controller 100.

[0086] On the other hand, the headlight 13 and the projection device 11 can also operate without the controller 100. For example, the headlight 13 can be connected to the projection device 11, and the operation of the headlight 13 can be controlled by the projection device 11 (more specifically, the control unit of the projection device 11). The projection device 11 can, for example, illuminate the headlight 13 and generate an image of the information to be displayed, projecting the image light in front of the vehicle. Alternatively, the projection device 11 can, for example, turn off the headlight 13 and generate an image of the information to be displayed, projecting the image light in front of the vehicle.

[0087] The projection device 11 can be mounted, for example, at the front of the vehicle body, and the image light from the projection device 11 is projected onto the road surface in front of the vehicle. The projection device 11 can also be mounted, for example, near the roof of the vehicle 2. Alternatively, the projection device 11 can be mounted, for example, on the side mirror, on the roof, or on the side or bottom of the vehicle 2. However, the invention is not limited to these locations.

[0088] One or more projection devices 11 can be mounted. For example, a pair of projection devices 11 can be mounted at the front end of the vehicle 2. Alternatively, the projection devices 11 can be integrated into the headlight 13. When the projection devices 11 are integrated into the headlight 13, the light source of the headlight 13 can also be used as the projection light source.

[0089] The projection device 11 can be mounted, for example, at the rear of the vehicle body, and the image light from the projection device 11 is projected onto the road surface behind the vehicle. Alternatively, a pair of projection devices 11 can be mounted at the rear end of the vehicle 2. Furthermore, the projection device 11 can be integrated into the taillight. When the projection device 11 is integrated into the taillight, the taillight's light source can also be used as the projection light source. The aforementioned taillight can also be a brake light or a reversing light; the following description of taillight can be replaced with brake light or reversing light.

[0090] Furthermore, the controller 100 can coordinately control the taillights and the projection device 11. For example, the controller 100 can illuminate the taillights and cause the projection device 11 to generate an image of the information to be displayed, projecting the image light behind the vehicle. Alternatively, the controller 100 can, for example, extinguish the taillights and cause the projection device 11 to generate an image of the information to be displayed, projecting the image light behind the vehicle. In other words, the taillights and the projection device 11 can operate in coordination via the controller 100.

[0091] On the other hand, the taillights and the projection device 11 can also operate without the controller 100. For example, the taillights can be connected to the projection device 11, and the operation of the taillights can be controlled by the projection device 11 (more specifically, the control unit of the projection device 11). The projection device 11 can, for example, illuminate the taillights and generate an image of the information to be displayed, projecting the image light behind the vehicle. Alternatively, the projection device 11 can, for example, turn off the taillights and generate an image of the information to be displayed, projecting the image light behind the vehicle.

[0092] Furthermore, the controller 100 can coordinately control the turn signal 14 and the projection device 11. For example, the controller 100 can illuminate the turn signal 14 and cause the projection device 11 to generate an image of the information to be displayed, projecting image light onto the front or rear side of the vehicle. Conversely, the controller 100 can deactivate the turn signal 14 and cause the projection device 11 to generate an image of the information to be displayed, projecting image light onto the front or rear side of the vehicle. That is, the turn signal 14 and the projection device 11 can operate in coordination via the controller 100. Additionally, the driver can control the illumination and deactivation of the turn signal 14. The projection device can, for example, be configured to project image light onto the road surface on one side of the vehicle.

[0093] Vehicle information, such as information 4, is obtained using cameras and various sensors. Figure 2 This illustrates examples of how the controller 100 connects to various devices. Additionally, the control units of the projection device 11, the in-vehicle image display device 12, the vehicle navigation system 150, etc., can connect to... Figure 2 The device connection shown obtains information directly without going through controller 100. Figure 2 Various devices can be appropriately deleted or added to other types of devices, or replaced with other types of devices.

[0094] Vehicle speed sensor 501 is used to detect the speed of vehicle 2 and generate speed information as the detection result. Gear position sensor 502 is used to detect the current gear and generate gear information as the detection result. Steering wheel angle sensor 503 is used to detect the current steering wheel angle and generate steering wheel angle information as the detection result.

[0095] The headlight sensor 504 detects, for example, the ON / OFF state of the headlight 13. Additionally, the headlight sensor 504 can detect the brightness of the headlight 13 when it is illuminated. The headlight sensor 504 is used to generate illumination information as a result of the detection. Furthermore, a high / low beam sensor can be provided in vehicle 2 to detect the high beam or low beam state of the headlight 13, and this high / low beam sensor is used to generate information indicating high beam or low beam. Additionally, a taillight sensor (not shown) can be provided in vehicle 2. The taillight sensor detects, for example, the ON / OFF state of the taillight. Additionally, the taillight sensor can detect the brightness of the taillight when it is illuminated. The taillight sensor is used to generate illumination information as a result of the detection. Similarly, sensors (not shown) that detect the ON / OFF state and brightness of the brake lights or reverse lights, hazard lights, and turn signals 14 can also be provided.

[0096] Illuminance sensor 505 and colorimetry sensor 506 are used to detect external light from vehicle 2 and generate external light information as the detection result. Additionally, colorimetry sensor 506 can detect the road surface color around vehicle 2 and generate projection surface color information as the detection result. Distance sensor 507 is used to detect the distance between vehicle 2 and external objects, or the distance between external objects, and generates distance information as the detection result. Infrared sensor 508 is used to detect the presence and distance of objects near the vehicle and generates infrared information as the detection result. Engine start sensor 509 is used to detect the engine's ON / OFF state and generates ON / OFF information as the detection result.

[0097] Vehicle operation switches 510 are various switches operated by the driver and others, used to generate ON / OFF operation information for these switches. Vehicle operation switches 510 include steering wheel buttons, switches on the instrument panel, door switches, switches on the armrests, switches on the center console, etc.

[0098] The communication unit 511 is a structure for communication, including, for example, a first communication unit 5111, a second communication unit 5112, a third communication unit 5113, an in-vehicle wireless communication unit 5114, and an external wireless communication unit 5115.

[0099] The first communication unit 5111 is, for example, a structure for performing FPD-Link III communication, including a communication line and a communication device for FPD-Link III communication.

[0100] The second communication unit 5112 incorporates a communication protocol (CAN communication protocol), including a communication line and communication device for CAN communication.

[0101] The third communication unit 5113 is a structure for in-vehicle Ethernet communication, incorporating an in-vehicle Ethernet communication protocol and including a communication cable and communication device for in-vehicle Ethernet communication. Additionally, the third communication unit 5113 may include a USB (Universal Serial Bus) port, through which the driver or others can appropriately connect devices (such as USB memory devices or devices with computer functions). The third communication unit 5113 can then perform in-vehicle Ethernet communication between the device connected to this port and the target structure.

[0102] The in-vehicle wireless communication unit 5114 is a structure that communicates with in-vehicle information equipment, and it incorporates a communication protocol, including a wireless device. The in-vehicle wireless communication unit 5114 can perform wireless communication such as Wi-Fi (registered trademark) and Bluetooth (registered trademark). Additionally, the in-vehicle wireless communication unit 5114 can perform short-range wireless communication such as NFC (Near Field Communication).

[0103] The external wireless communication unit 5115 is a structure for communicating with the outside of the vehicle 2, and it incorporates a communication protocol, including a wireless device. The external wireless communication unit 5115 can perform wireless communication such as LTE (Long Term Evolution), 5G, and Wi-Fi.

[0104] Furthermore, the structure of the communication unit 511 can be appropriately modified. For example, the communication unit 511 can be configured to perform LIN communication. Alternatively, the communication unit 511 can be configured to perform GMSL communication.

[0105] Accelerometer 512 and gyroscope 513 are used to detect the acceleration and angular velocity of vehicle 2, generating acceleration and gyroscope information representing the attitude and behavior of vehicle 2. Temperature sensor 514 is used to detect the temperature inside and outside the vehicle, the road surface, etc., and generate temperature information as the detection result.

[0106] The inter-vehicle communication wireless transceiver 515 generates inter-vehicle communication information through inter-vehicle communication between vehicle 2 and roads, signs, traffic lights, etc. The inter-vehicle communication wireless transceiver 516 generates inter-vehicle communication information through inter-vehicle communication between vehicle 2 and other surrounding vehicles. The terminal inter-vehicle communication wired / wireless communication unit 517 is a device that obtains information from a device connected to an LTE network (e.g., a WiFi device) via wired or wireless communication. The controller 100 or control unit can obtain information transmitted and received via the LTE network through the terminal inter-vehicle communication wired / wireless communication unit 517.

[0107] GPS receiver 518 generates GPS information obtained from GPS signals received from GPS satellites. For example, it can obtain the current time, latitude, and longitude. VICS receiver 519 generates VICS information obtained by receiving VICS signals. These VICS signals include congestion information, weather information, etc., provided by radio or optical beacons. GPS receiver 518 and VICS receiver 519 can also be configured as part of a navigation system.

[0108] The in-vehicle camera 520 and the external camera 521 are used to capture images inside and outside the vehicle, generating in-vehicle and external camera image information, respectively. Specifically, the in-vehicle camera 520 is, for example, a camera used in a Driver Monitoring System (DMS) to capture the driver's posture, eye position, and movements. In this case, by analyzing the captured images, it is possible to understand the driver's fatigue level, eye position, etc.

[0109] The driver's voice is input to the voice input device 522, which generates sound information. The driver can input operation commands via the voice input device 522 by uttering a sound. The voice output device 523 is, for example, a device that outputs the sound processed by the controller 100 or the control unit.

[0110] Humidity sensor 524 is used to detect humidity and generate humidity information. Humidity sensor 524 can detect the humidity outside the vehicle 2 and the road surface, generating humidity information for the exterior of the vehicle 2. Additionally, humidity sensor 524 can detect the humidity inside the vehicle 2, generating humidity information for the interior of the vehicle.

[0111] Rain sensor 525 is used to detect raindrops and generate rainfall information. Wiper switch 526 is used to detect the ON / OFF state of wiper 6 and generate ON / OFF information for wiper 6. Turn signal switch (not shown) is used to detect the driver's operation of turn signal 14 and generate turn signal information.

[0112] The image generation unit 527 can generate image information based on information obtained from various sensors, information from external device 25 or portable terminal 26, or information from the Internet, or it can generate image information based on information obtained from controller 100. Information outside the vehicle includes information from external device 25 or portable terminal 26 and information from the Internet. Alternatively, the image generation unit 527 may not be included; in this case, the image generation function can be performed by controller 100.

[0113] Raindrop sensor 528 detects the amount and size of raindrops. Raindrop sensor 528 can determine the rainfall state based on the phenomenon that the amount of light incident on the detector decreases due to the presence of raindrops. The amount of raindrops detected by raindrop sensor 528 can be used to generate related information such as the operating time of wiper 6. Furthermore, the operating state of wiper 6 can be changed in conjunction with the vehicle 2's travel speed and the detection results of raindrop sensor 528, and can also be used to correct related information such as the operating time of wiper 6.

[0114] The road surface sensor 529 detects information about the road surface condition. The road surface sensor 529 can directly measure the road surface wetness or detect friction between the sensor and the road surface. Alternatively, it can detect acceleration based on the road surface condition, and classify the road surface condition into dry, semi-wet, wet, slush, snow accumulation, compacted snow, and frozen states based on the acceleration waveform.

[0115] The controller 100 or processor can, for example, transmit and receive data or information wirelessly to obtain information required for driving. Additionally, the controller 100 or processor can obtain information required for autonomous driving. Furthermore, the controller 100 or processor can, for example, transmit and receive data or information wirelessly to perform data or information update processing. As an update process, the controller 100 or processor can, for example, update various data or information (map data, data for image processing, software, etc.). Such technology is sometimes referred to as OTA (Over-The-Air) technology.

[0116] Next, refer to Figure 3A and Figure 3B An example of how a projection device projects information will be illustrated. For example... Figure 3A As shown, an image light is projected onto the road surface ahead from the projection device. The figure shows projection area 14a projected from the projection device 11 via the window 13a on the right front of the vehicle 2, and projection area 14b projected from the projection device 11 via the window 13b on the left front of the vehicle. The projected images of each projection area (14a, 14b) are combined to project an image onto the road surface in front of the vehicle 2 (in this example, arrow 16 indicating that the vehicle 2 is traveling straight). In this example, the projection areas (14a, 14b) are divided left and right, but the projection areas can also be divided according to proximity to the vehicle and distance. Alternatively, only one of the projection devices 11 located on the left and right sides of the vehicle can be used to project the image light. In this example, the projection device 11 is integrated into the headlight 13, and the light source of the headlight 13 is used as the projection light source. However, as mentioned above, the configuration of the projection device 11 can be appropriately changed. The projection device 11 can also have a different light-emitting structure than the headlight 13, and the light emitted by this structure is used to project the image light.

[0117] like Figure 3BAs shown, projection device 11 projects image light onto the road surface behind the vehicle. The figure shows projection area 17a projected from the projection device 11 on the right rear side of the vehicle 2 via the window 15a on the right rear side of the vehicle 2, and projection area 17b projected from the projection device 11 on the left rear side of the vehicle 2 via the window 15b on the left rear side of the vehicle 2. The projected images of each projection area (17a, 17b) are combined to project an image onto the road surface behind the vehicle (in this example, arrow 18 indicating a reversing sign for the vehicle traveling straight). In this example, the projection areas (17a, 17b) are divided left and right, but the projection areas could also be divided according to proximity to the vehicle and distance. Alternatively, only one of the projection devices 11 positioned on the left and right sides of the vehicle can be used to project the image light. In this example, the projection device 11 is configured to use the taillight light source as the projection light source. However, as described above, the configuration of the projection device 11 can be appropriately changed. The projection device 11 may also have a light-emitting structure different from that of the taillight, and use the light emitted by the structure to project image light.

[0118] Reference Figure 4A The following describes a structural example of the projection device 11. The projection device 11, for example, uses data or information obtained via various sensors and a communication unit to generate an image of the information to be displayed and projects image light. Additionally, data or information can be input to the projection device 11 from the controller 100. For example, signal data, image data, etc., can be input to the projection device 11 from the controller 100. The projection device 11 can then process the input information. Alternatively, the control unit of the projection device 11 can be configured to control the projection device 11 instead of the control unit of the projection device 11. Furthermore, the control unit of the projection device 11 and the controller 100 can also perform processing separately; for example, the controller 100 can control the brightness of the headlights 13 and taillights, while the control unit of the projection device 11 can control the brightness of the projected image.

[0119] like Figure 4A As shown, the projection device 11 includes a projection optical system 701 and a light source device 702. Additionally, the projection device 11 includes a power supply 703, a cooling unit 704, an operation input unit 705, an image signal input unit 706, an audio signal input unit 707, an audio output unit 708, a communication unit 709, a non-volatile memory 710, a main memory 711, a storage unit 712, an adjustment unit 713, and a control unit 714. The control unit 714 can be configured as a processing device. The control unit 714 can control the operation of the projection device 11. On the other hand, in the absence of a control unit, the controller 100 can also control the projection device 11.

[0120] The projection optical system 701 is a structure for projecting light. The projection optical system 701 includes, for example, optical components such as lenses and / or mirrors.

[0121] The light source device 702 is a device capable of generating image light. The light source device can use, for example, a high-pressure mercury lamp, a xenon lamp, an LED light source, or a laser light source. Furthermore, the light source device may include optical elements for focusing and homogenizing light.

[0122] Power supply 703 supplies power to light sources, for example. Power supply 703 also supplies the power required by other components.

[0123] The cooling unit 704 cools various parts that are in a high-temperature state, such as the light source, power supply 703, or light source device 702, by means of air cooling or liquid cooling as needed.

[0124] The operation input unit 705 is, for example, an operation button or a light-receiving part of a remote control, which inputs operation signals from the user. By inputting operation signals from the user, the switch (operation switch) that activates the projection device 11 is switched ON / OFF.

[0125] The image signal input unit 706 is an interface device for acquiring image data from the outside. The audio signal input unit 707 is an interface device for acquiring audio data from the outside. The audio output unit 708, for example, can output audio based on the audio data input to the audio signal input unit 707. The audio output unit 708 can output operation tones and error warning tones.

[0126] The communication unit 709 is an interface device for communicating with external devices. For example, the communication unit 709 can connect to an external information processing device (such as the controller 100) to input and output various control signals. The communication unit 709 can connect to various sensors, communication devices installed in the vehicle 2, etc., to input and output various data or information.

[0127] The non-volatile memory 710 stores, for example, various data used in the projection function. The data stored in the non-volatile memory 710 also includes pre-prepared image data, video data, etc., for projecting images.

[0128] The memory 711 stores the image data to be projected, control parameters of various parts of the device, etc.

[0129] Storage unit 712 is a device for recording images, pictures, sounds, and various data. For example, images, pictures, sounds, and various data can be pre-recorded at the time of product shipment, or images, pictures, sounds, and various data obtained from external devices or servers via communication unit 709. Furthermore, control unit 714 can obtain updated data or information from external sources via communication unit 709 and update the recorded data or information to the new data or information. Moreover, users can select to update part or all of the recorded data or information to the new data or information. Images, pictures, and various data recorded in storage unit 712 can be output as projected images. Sound recorded in storage unit 712 can be output as sound from sound output unit 708.

[0130] The adjustment unit 713 is capable of adjusting the image light, and includes, for example, an image adjustment unit and a polarization adjustment unit. The image adjustment unit performs image processing on the image data input through the image signal input unit, the image data stored in the non-volatile memory 710, and the image data. This image processing includes, for example, image distortion correction, scaling processing (such as magnification, reduction, and distortion), brightness adjustment processing (changing image brightness), contrast adjustment processing (changing the image contrast curve, including adjusting the linear characteristics of brightness levels), color correction processing (changing image chromaticity), and Retinex processing (decomposing the image into light components (illumination light component, reflected light component, ambient light component) and changing the weights of each component). Furthermore, the image adjustment unit is implemented by the control unit 714 storing data in the memory 711 and performing image processing.

[0131] The polarization adjustment unit adjusts the degree of polarization of the image light to be projected. Here, the degree of polarization refers to the ratio of the polarized light component in the light, i.e., the P-polarized component to the S-polarized component (P-polarization and S-polarization are defined relative to the projection surface). For example, the projection device 11 may include a structure such as a polarization separation element and a polarization conversion element (described later) that can adjust the degree of polarization of the emitted light. The control unit 714 adjusts the degree of polarization of the image light by controlling this structure. Furthermore, the polarization control unit is implemented by the control unit 714 storing data (parameters used in control, etc.) in the memory 711 and controlling this structure. Alternatively, multiple projection devices 11 with different degrees of polarization can be provided and switched between them.

[0132] Figure 4BIn the example, the light source device 702a includes a light source 7021, a display element 7022, an optical element 7023, a polarization separation element 7024, and a polarization conversion element 7025. The light source 7021 generates light for image projection. The optical element 7023 is used to converge and homogenize light, etc. Additionally, descriptions similar to those described above are sometimes omitted. The polarization separation element 7024 is an element that separates incident light into S-polarized light and P-polarized light. For example, a polarizing beamsplitter is used as the polarization separation element 7024 to reflect or transmit light with a specific polarization state, thereby separating the light. The polarization conversion element 7025 is an element used to change the degree of polarization. For example, a wave plate is used as the polarization conversion element 7025 to delay the phase of light, thereby changing the polarization state. Here, for example, the polarization conversion element 7025 can be connected to an actuator (e.g., an electric motor) for controlling the angle of incidence during transmission. Figure 4A The adjustment unit 713 or polarization adjustment unit 722 adjusts the incident light to a polarization degree corresponding to the incident angle.

[0133] In addition, the projection device 11 includes a display element 7022 and a display element driving unit (not shown). The display element 7022 is a component that modulates transmitted or reflected light to generate an image, such as a transmissive LCD (Liquid Crystal Display) panel, a reflective LCD panel, or a DMD (Digital Micromirror Device) (registered trademark) panel. The display element driving unit sends a driving signal to the display element 7022, causing the display element 7022 to generate an image. In addition to structures such as DMDs and LCDs that can display multiple images according to control signals, the display element 7022 may also include structures such as a mask (light-blocking template) type that only displays a fixed image.

[0134] Figure 4C In this example, the light source device 702b does not have a light source that serves as a backlight for the display element. In this example, the display element itself emits light. Additionally, descriptions similar to those above may sometimes be omitted.

[0135] The light source device 702b includes a display light-emitting element 7026, an optical element 7023, a polarization separating element 7024, and a polarization conversion element 7025. The display light-emitting element 7026 is configured as a self-emissive display such as an LED array or an OLED (Organic Light Emitting Diode) display. The optical element 7023 is used in an optical system for focusing and homogenizing light. Furthermore, the optical element 7023, polarization separating element 7024, and polarization conversion element 7025... Figure 4B The content described in the text is the same.

[0136] Next, refer to Figures 5A-5C Examples of the configuration of optical components, etc., will be explained. Figure 5A This is an example using a reflective display element, in which images are projected using a DMD (Digital Multi-Density Array) method. The first optical element 901 is an optical element that converges the light generated by the light source 900, such as a collimator. The second optical element 902 is a reflector that projects the converged light onto the display element 907. In this example, the display element 907 is configured as a DMD-type panel, adjusting or controlling the light projection for each display pixel. The third optical element 903 is a structure for projecting image light, constructed using optical components such as lenses and reflectors. A reflective LCD panel can also be used as a reflective display element.

[0137] Figure 5B This is an example of a display light-emitting element 908 emitting light itself and projecting an image. The display light-emitting element 908 is configured as an LED array panel or an LED matrix panel, generating an image through a combination of the light-emitting positions and colors of the LEDs on the panel. The fourth optical element 904 is a structure used to project image light, constructed using optical components such as lenses and mirrors. In addition to the aforementioned panel, an OLED display can also be used as the display light-emitting element.

[0138] Figure 5C This is an example of displaying an image by allowing light to pass through the display element 909. The fifth optical element 905 is the same as the first optical element 901 described above. The display element 909 is, for example, a transmissive LCD panel, which generates an image. The sixth optical element 906 is a structure for projecting image light, constructed using optical components such as lenses and mirrors. Alternatively, the display element 909 may employ a structure in which a mask with a predetermined pattern is arranged on the light source side. In this case, a fixed image determined by the pattern formed on the mask can be displayed using light that is not blocked by the display element, i.e., the mask, and passes through the mask. Alternatively, the display element 909 may employ a structure in which a lens with a predetermined pattern is arranged on the light source side. In this case, the display element 909 can display a fixed image determined by the pattern formed in the lens. Furthermore, the pattern formed in the lens is formed on the lens surface or inside the lens, and the formed pattern can partially block the transmission of light or partially allow the light to pass through. The lens used in the display element 909 may be a microlens array. In this case, the patterns formed in each lens are different.

[0139] Explanation instead Figure 5CThe display element includes a modified version of the light-shielding plate. For example, the light-shielding plate has a window through which light from the light source 900 is transmitted. The shape of the window is not limited; for example, it can be a geometric shape such as a rectangle or a V-shape, or an icon, text, or number. Furthermore, the window can be a shape that can be changed according to the image to be projected; for example, a rectangular image can be changed to a V-shaped image based on acquired information and then continued to be projected. The sixth optical element 906 is a structure that projects light passing through the window of the light-shielding plate onto the road surface, and is constructed using optical components such as lenses, mirrors, and apertures. The length, size, and brightness of the image projected onto the road surface can be controlled using the sixth optical element 906. Furthermore, the length, size, and brightness of the image projected onto the road surface can also be controlled by changing the shape, size, transmittance, and opening duty of the window of the light-shielding plate. Additionally, the image projected onto the road surface can be made to flicker by repeatedly emitting and de-emitting light from the light source 900.

[0140] Furthermore, as another variation, multiple groups consisting of a light source 900, a first optical element 901, a light-shielding plate, and a sixth optical element 906 are provided. Then, by using the sixth optical element 906 to project overlapping light rays passing through the windows of the multiple light-shielding plates onto the road surface, a rectangular image can be projected onto the road surface as a straight line. Additionally, by using the sixth optical element 906 to project separately light rays passing through the windows of the multiple light-shielding plates onto the road surface, multiple rectangular images arranged in a straight line can be projected onto the road surface. Furthermore, by sequentially emitting light from the multiple light sources 900, rectangular images emitting light sequentially in a predetermined direction (e.g., the direction from which vehicles depart) can be projected onto the road surface.

[0141] in addition, Figure 5A , Figure 5C The light source shown can be the light source of the projection device 11, but it can also be the light source of the headlight 13 or the taillight. In this case, the light source of the projection device 11 can be omitted. On the other hand, Figure 5A , Figure 5C The light source shown can also be a structure that combines the light source of the projection device 11 and the light source of the headlight 13 or the taillight. Furthermore, the light source of the projection device 11 can be a single unit or multiple units. Similarly, the display element or light shield corresponding to the light source can also be one or more units.

[0142] Next, refer to Figure 6 This illustrates an example of the display area of ​​the image seen by the driver. Figure 6The diagram shows the high beam illumination area A1 of the headlight 13, the low beam illumination area A2 of the headlight 13, and the image area A3 for projecting road images from the projection device 11. The high beam illuminates the upper illumination area, providing a higher position of high beam. That is, it illuminates this area with bright high beam that illuminates distant objects. The lower illumination area illuminates the lower high beam, which overlaps with the low beam projection area. Therefore, low beam is sometimes formed by blocking a portion of the illumination area of ​​the headlight 13 that can project high beam. Furthermore, the image area A3 projected by the projection device 11 is included in the illumination area below the low beam illumination area A2 and the high beam illumination area A1. Therefore, the projection device 11 can also project images using both the low beam and high beam of the headlight 13, and the projection device 11 can be installed outside the headlight 13.

[0143] Reference Figure 7 This illustrates an example of the projection area around a vehicle. Figure 7 The diagram shows a road surface display area A11 in front of the vehicle, a road surface display area A12 behind the vehicle, and road surface display areas A13 to the left and right of the vehicle. The road surface display area A11 in front of the vehicle is a display area formed on the road surface in front of the vehicle, based on the vehicle's position. The projection device 11 can be configured, for example, to project image light onto this area to display information. The road surface display area A12 behind the vehicle is a display area formed on the road surface behind the vehicle, and the projection device 11 can be configured, for example, to project image light onto this area to display information. The road surface display areas to the left and right of the vehicle are display areas formed on the road surface to the left and / or right of the vehicle, based on the vehicle's position. The projection device 11 can be configured, for example, to project image light onto this area to display information. The road surface display areas to the left and right of the vehicle A13 can be configured such that vehicles and people (pedestrians, etc.) around the vehicle can view the displayed information.

[0144] In the information projection of this embodiment of the invention, the processor controls the projection of the image and the projection of the image. The function of the processor can be implemented by any of the vehicle controller 100, the control unit of the projection device 11, or an external device. For example, the projection device 11 and the projection of the image can be controlled by the controller 100, an external device, or a portable terminal. Alternatively, the control unit of the projection device 11 can control the projection of the image and the projection of the image based on vehicle information or external information, or the controller 100 and the projection device 11 can share the processing. As vehicle information 4, the acquisition unit can acquire reversing (backward) information, steering wheel angle information, speed information, gear information, accelerometer gyroscope information, road condition information, and information about the turn signals. The acquisition unit can be a sensor mounted in the vehicle, or it can have the function of inputting and outputting information from the vehicle or the outside.

[0145] Figure 8This is an example of information projection. For example... Figure 8 As shown, step S801 detects that the vehicle is reversing based on information from the vehicle. If it is determined that the vehicle is reversing based on information from the vehicle, proceed to step S802. For example, whether the vehicle is reversing is detected based on the ON / OFF information of the reverse gear from the gear position sensor 502.

[0146] If the vehicle is determined to be reversing in step S801, in step S802, information from the sensor is obtained based on a pre-set projection area. In this embodiment, the projection area can be divided into an area near the vehicle and an area far from the vehicle, or it can be divided into an area near the vehicle, an area far from the vehicle, and an "outside the projectable area (non-projectable area)". Alternatively, the user can arbitrarily choose the method of dividing the area as needed. In addition, the area near the vehicle can be called the first area, and the area far from the vehicle can be called the second area. When the "outside the projectable area" is set as an area, the area outside the projectable area (i.e., the non-projectable area) can be called the third area. The information detected by the sensor includes the presence information of the detected object, the presence direction of the detected object relative to the vehicle, the distance from the vehicle body to the detected object, the relative position or relative velocity vector between the vehicle body and the detected object, the type of detected object, and the surrounding environment. For example, when the detected object is an obstacle, it includes pedestrians, trees, walls, vehicles, and objects around the vehicle that may affect the display. The sensors used in this embodiment may include, for example, an external camera 521, a ranging sensor 507, an infrared sensor 508, an ultrasonic sensor, a millimeter-wave sensor, or a LiDAR (Light Detection and Ranging) sensor.

[0147] Step S803 determines whether a detection object exists based on the information detected by the sensor in step S802. In step S803, for example, if an object is detected (yes), the process proceeds to step S804; if no object is detected (no), the process proceeds to step S806.

[0148] Step S804 adds a marker (e.g., a flag) to the area where the detected object (e.g., an obstacle) was identified in step S803. Step S805 performs processing corresponding to the marker. For example, the type or information of the sensor to be used in image projection can be switched accordingly based on the marker.

[0149] In step S806, an image is projected based on the information obtained in step S803 or step S805. For example, if it is determined in step S803 that no detection object exists, normal projection is performed. Based on the information obtained in step S805, the image is projected after being reduced in size, the brightness of the image is changed and then projected, or no image is projected.

[0150] Step S807 is the step of determining whether image projection has ended. In step S807, based on the projection end condition including vehicle information 4, it is determined whether image projection should end. If it is determined in step S807 that image projection has ended (yes), image projection ends; otherwise, it proceeds to step S801. Alternatively, steps S801 to S806 can be repeated until it is determined that image projection has ended.

[0151] Figure 9 This is a table illustrating the methods for detecting obstacles in each region. In this embodiment, the region behind the vehicle in Example 1 is divided into a first region and a second region, and the region behind the vehicle in Example 2 is divided into a first region, a second region, and a third region. Specifically, for example, the first region is defined as a region where the distance from the rear of the vehicle in the longitudinal direction is less than 1 meter, and the straight-line distance from the left and right sides of the vehicle in the width direction is less than 2 meters. The first region is necessary to represent the relationship between the vehicle and the projected image. If the closest distance between the image and the vehicle exceeds 1 meter, it is difficult to recognize that the image is being projected from the vehicle. In addition, interference between the projected image and obstacles in this region may obscure part or all of the image, so there is a possibility that the information of the projected image cannot be correctly conveyed. The second region is defined as a region where, excluding the first region, the distance from the rear of the vehicle in the longitudinal direction is less than 3 meters, and the straight-line distance from the left and right sides of the vehicle in the width direction is less than 2 meters. The second region is associated with the projectable region, and for obstacles present in this region, the driver and those around can be aware of the presence of obstacles, their location, attributes, etc., through the image. The third zone is the area behind the vehicle other than the first and second zones. Furthermore, the extent of the zone is not limited to the distances described above. For example, it can be set according to the vehicle's height; a narrower zone can be set for vehicles with lower heights, and a wider zone for vehicles with higher heights. Alternatively, the zone size can be changed according to the vehicle's rear visibility or whether sensors such as rear cameras are installed. For example, a wider zone can be set for vehicles where it is difficult to identify obstacles behind them.

[0152] In the area behind the vehicle, the detection ranges of multiple detection methods overlap, enabling the simultaneous detection of obstacles using multiple methods. Examples 1 and 2 include one or both of an ultrasonic sensor and a millimeter-wave sensor, and one or both of an external camera and a LiDAR. Furthermore, the detection methods differ in the following ways: An ultrasonic sensor utilizes high-frequency sound waves inaudible to humans, detecting glossy or transparent objects regardless of color, thus enabling the detection of obstacles and the distance between the vehicle and the obstacle. A millimeter-wave sensor can instantaneously detect the position and speed of moving vehicles or pedestrians using millimeter waves, exhibiting stable detection performance even in adverse weather conditions such as snow and fog. A LiDAR illuminates the area with a laser, measuring the distance and shape of objects based on the reflected light. An external camera acquires images of the area behind the vehicle; by using these images for image recognition, obstacle identification can be performed. When multiple detection methods are available, it is preferable to switch the detection method used in the decision-making process according to the characteristics of each method.

[0153] In Example 1, an ultrasonic sensor or millimeter-wave sensor is used to detect obstacles in the first area, and an external camera or LiDAR is used to detect obstacles in the second area. Ultrasonic and millimeter-wave sensors emit detection waves isotropically, eliminating spatial delay and resulting in excellent obstacle detection performance. Conversely, LiDAR uses lasers to scan space, acquiring spatial information in a time-division manner, which may lead to slower obstacle detection. Furthermore, obstacle detection using an external camera requires image recognition, which evaluates the matching degree between the image and pre-trained objects. While it can identify objects with high matching degrees, it may fail to recognize objects with low matching degrees or misidentify them as different objects. The first area is located near the vehicle, and information about the presence of obstacles is prioritized over information about obstacle attributes. Therefore, information from the ultrasonic or millimeter-wave sensor is used for detection in this area. The second area is farther from the vehicle and requires a time delay to reach it, so it is preferable to apply information other than the presence or absence of obstacles. Therefore, information from the external camera or LiDAR is used for detection in this area.

[0154] In Example 2, the area behind the vehicle is divided into three regions: Region 1, Region 2, and Region 3. Obstacles in Region 1 are detected using ultrasonic or millimeter-wave sensors, obstacles in Region 2 are detected using an external camera or LiDAR, and obstacles in Region 3 are detected using LiDAR. In this example, in Region 3, the objects are far enough away from the vehicle that there is concern about the low pixel count of the recorded object parts and the reduced object detection accuracy when using an external camera. Therefore, no obstacle detection is performed in Region 3; only LiDAR is used for detection.

[0155] Figure 10 This is a table indicating how images are projected according to the area where an object is located. When an object (e.g., an obstacle) is present in the first area, specifically in the projection area of ​​the image projection unit within the first area, image display can be stopped. When image projection is obstructed by an obstacle, there is a possibility that part or all of the image on the road surface may be missing, preventing the intended image from being displayed. In this case, unexpected changes to the image may cause misunderstandings in the surrounding environment, so image display can be stopped. Alternatively, as a variation where an object is present in the first area, when an obstacle is present in the image area projected by the image projection unit within the first area, the projected images may differ based on the position information of the object detected by an ultrasonic sensor or millimeter-wave sensor in the projection area on the side shorter than the object detected behind the vehicle and in the projection area on the side longer than the object detected behind the vehicle. Specifically, image projection is stopped for the projection area shorter than the object detected behind the vehicle, or in the projection area where the detected object is located. The projected image is displayed for the projection area longer than the object detected behind the vehicle, or in the projection area where no detected object is located. By using multiple ultrasonic sensors or millimeter-wave sensors, or by detecting the reflection angle of the detection wave, it is possible to obtain not only the distance to the detected obstacle but also its position information. If image defects caused by a detected obstacle obstructing image projection only affect one side of the projected images on either side of the vehicle, or if image defects may only occur on one side, the image on the side unaffected by the detected obstacle can be projected only. Alternatively, if no obstacle-related defects are found in the image near the vehicle, the image can be projected only near the vehicle. Furthermore, if a detected obstacle causes the image on one side to stop, the image on the side currently being projected can be made to flicker.

[0156] When an object exists in the second region, specifically when an obstacle is located in the projection area of ​​the image projection unit within the second region, for example, based on the position information of the object detected by LiDAR or an external camera, an image projected onto the projection area shorter than the object detected behind the vehicle is displayed in accordance with the projection distance; or, an image projected onto the projection area where the object detected behind the vehicle is located is displayed in accordance with the projection distance. That is, for the image projection unit (image projection device) on the side shorter than the object detected behind the vehicle, the projection distance of its projected image is limited to the position of the detected object. As a variation of the second region, when an obstacle is detected within the image area projected by the image projection unit in the second region, the image projected onto the projection area shorter than the object detected behind the vehicle or the projection area where the object is located is flickered.

[0157] When an object exists in both the first and second regions, specifically when the object is located in the image projection area of ​​both regions, for example, the image projection of the image projection unit near the object in the first region (or the projection area where the object is located) is stopped, and the projected image of the image projection unit near the object in the second region (or the projection area where the object is located) is limited to the object detection position. Furthermore, when an object is detected simultaneously in both the first and second regions in the image projection area on the same side of the vehicle's projection device, projection is controlled based on the object in the first region. As a variation where the detected object exists in both the first and second regions, if multiple objects are detected, and at least one object is detected in the image projection area of ​​the first region and another in the image projection area of ​​the second region, the image projection of the image projection unit near the detected object in the first region (or the area of ​​the projected image where the object is located) is stopped, and the projected image of the image projection unit near the detected object in the second region (or the area of ​​the projected image where the object is located) flickers.

[0158] When the object exists in the third region, specifically when the obstacle is outside the projection area of ​​the image projection unit, for example, based on the object's position information detected by LiDAR, an image is projected onto the projection area of ​​the object detected near the rear of the vehicle. For example, if the object detected in the third region is near the projection area on the left side of the vehicle, a attention image (an image used to draw attention) is projected onto the projection area on the left side of the vehicle that is near the third region. Alternatively, the image projected onto the projection area near the third region where the object is located can be made to flicker.

[0159] Figures 11A-11D This is an example of projecting an image according to a region. For example... Figures 11A-11D As shown, the system includes a range sensor 507 for monitoring the rear and an external camera 521. Image recognition of the images acquired by the external camera 521 allows for estimation of the distance from the vehicle to an object. In this embodiment, the projection area is divided into a first area A12-1 and a second area A12-2. The range sensor 507 detects objects within the display range of the image projection area, and the external camera 521 estimates the distance to distant objects. Furthermore, obstacles will be used to describe the objects below.

[0160] Figure 11A This is the normal projection state when the vehicle is reversing. Rectangular images representing reversing are projected into the first area A12-1 and the second area A12-2 behind the vehicle. Specifically, the rectangular images consist of multiple images; when the vehicle is reversing, one rectangular image is projected into the first area A12-1, and two rectangular images are projected into the second area A12-2. The rectangular images can be hidden, partially displayed, or flashed depending on the location of obstacles. This normal projection occurs when no obstacles are detected behind the vehicle. In this example, the projected image is divided into three rectangles, but it can also be displayed as a single rectangle, with the length of the rectangle varying according to the location of obstacles, or it can be divided into more than three rectangles for display.

[0161] Figure 11B This example illustrates the positional relationship between the vehicle and the obstacle behind it when the vehicle is reversing, showing the vehicle's position relative to the obstacle within the projection area. For example... Figure 11B As shown, an obstacle 600 exists near the taillight (image projection unit) on the left side of the vehicle. In this embodiment, the obstacle 600 is located between the image projection area of ​​the first region A12-1 and the image projection area of ​​the second region A12-2. Because the obstacle 600 is detected in the first region A12-1, image projection is stopped. By stopping the image projection, it is possible to prevent the image projected onto the road surface from being damaged due to the image light being blocked by the obstacle 600.

[0162] Figure 11C Is Figure 11BThe image is projected in the following state. When the obstacle 600 is present in the image projection area of ​​the first region A12-1 or the second region A12-2 on the left rear side of the vehicle, the projection device on the right rear side of the vehicle, where the obstacle is not present, projects the image normally, while the projection device on the left rear side of the vehicle does not project an image. That is, depending on which projection device (left or right) is located at the rear of the vehicle, the image projection unit on the side of the first region A12-1 where the obstacle 600 is present does not project an image. On the other hand, the image projection unit on the side of the first region A12-1 where the obstacle 600 is not present can project an image.

[0163] Figure 11D yes Figure 11C A variation of this example. As shown in this example, because of the presence of obstacle 600, the projection device on one side (single side) stops projecting relative to the image that was originally to be projected, and the image is relatively smaller, which may reduce the effect of alerting surrounding vehicles and pedestrians. In such cases, the image of the projection device on the side that did not stop projecting can be made to flicker.

[0164] Figures 12A-12C This is a variation of projecting an image according to the region. Figures 12A-12C Explanation and Figures 11A-11D Different characteristics. Figure 12A This is the normal projection state when the vehicle is reversing, where rectangular images representing reversing are projected in the first area A12-1 and the second area A12-2 behind the vehicle.

[0165] Figure 12B This is an example of projecting an image when an obstacle 600 exists in the second region. Specifically, such as... Figure 12B As shown, when the vehicle is reversing, an obstacle 60 is present in the projection image area of ​​the second region on the left side of the vehicle. In this case, there is no obstacle 600 on the right rear projection device side of the vehicle, so normal image projection is performed. On the left rear projection device side of the vehicle, the image is projected at a distance appropriate to the obstacle 600. In this example, the projection image is shortened on the left rear projection device side of the vehicle. That is, when the obstacle 600 is not located in the first region A12-1 but in the image projection area of ​​the image projection unit in the second region A12-2, the image is projected at a distance appropriate to the location of the obstacle 600.

[0166] like Figure 12C As shown, the image on the left side of the vehicle can be flashed. That is, by flashing the image of the projection device on the side where the obstacle 600 exists, the direction of the obstacle's presence can be communicated to people or vehicles in the vicinity, as well as to the driver of the vehicle.

[0167] Figures 13A-13CThis is an example of projecting an image based on the area where multiple obstacles are present. Figures 13A-13C Explanation and Figures 11A-11D Different characteristics. Figure 13A This is the normal projection state when the vehicle is reversing. Rectangular images representing reversing are projected in the first area A12-1 and the second area A12-2 behind the vehicle.

[0168] Figure 13B This is an example of a situation where multiple obstacles are detected. In this example, obstacle 600 is located near the left side of the vehicle in the second area A12-2 and near the right taillight of the vehicle in the first area A12-1. Specifically, obstacle 600 is located near the right taillight of the vehicle in the image projection area of ​​the first area A12-1 and near the left side of the vehicle in the image projection area of ​​the second area A12-2. In this case, the projection content can be changed individually on the right and left sides behind the vehicle according to the position of the detected object. If obstacle 600 is detected in the image projection area of ​​the first area A12-1 on the right side behind the vehicle, the projection device on the right side behind the vehicle will not project an image to prevent the projected image from being obstructed by obstacle 600. If obstacle 600 is detected in the image projection area of ​​the second area A12-2 on the left side behind the vehicle, the projection device on the left side behind the vehicle will project an image according to the distance to obstacle 600. That is, the projection device on the left side behind the vehicle shortens the image and projects it relative to the normal image.

[0169] Figure 13C yes Figure 13B A variation of this example. As shown in this example, because of the presence of obstacle 600, the projection device on one side (single side) stops projecting relative to the image that was originally to be projected, and the image is relatively smaller, which may reduce the effect of alerting surrounding vehicles and pedestrians. In such cases, the image of the projection device on the side that did not stop projecting can be made to flicker.

[0170] Figures 14A-14B This is a variation of projecting an image according to the region. Figures 14A-14B Explanation and Figures 11A-11D Different characteristics. Figure 14A This is the normal projection state when the vehicle is reversing, where rectangular images representing reversing are projected in the first area A12-1 and the second area A12-2 behind the vehicle. Figure 14BThis refers to the case where the detected obstacle 600 exists outside the image projection area, i.e., in the third region. In this example, obstacle 600 exists on the left rear of the vehicle in the third region; that is, obstacle 600 is located outside the image projection area of ​​the projection device. In this situation, both the projection devices on the left rear and right rear of the vehicle can project normally, and simultaneously project a noticeable image near the outer periphery of the projection area close to obstacle 600. Since the detected obstacle 600 is located outside the image projection area, direct projection is not possible. However, by displaying a noticeable image in the image projection area closest to the obstacle's location, as shown in the figure, advance notification can be provided. Alternatively, the noticeable image of the projection device can be made to flash.

[0171] Figure 15 This is an example of projecting information when a vehicle is reversing. For example... Figure 15 As shown, step S901 determines whether the vehicle is reversing based on information from the vehicle. For example, if the reverse gear is detected to be ON based on information from the vehicle, it is determined that the vehicle is reversing; if the reverse gear is detected to be OFF, it is determined that the vehicle is not reversing. Furthermore, the reverse gear information can be obtained from the gear position sensor 502.

[0172] In step S902, after determining in step S901 that the vehicle has begun reversing, based on external information obtained by the sensors, it is determined whether there are any obstacles around the vehicle that may hinder driving. If it is determined in step S902 that there are obstacles in the area behind the vehicle, proceed to step S903.

[0173] In step S903, if it is determined in step S902 that an obstacle exists in the area behind the vehicle, an image displaying information about the obstacle is selected for projection. Information about the obstacle can be obtained from, for example, an external camera 521, a ranging sensor 507, or an infrared sensor 508. The obstacle is an object or a detected object. Objects include people (pedestrians), trees, walls, vehicles, and objects around the vehicle that may affect driving. For example, if an obstacle is detected within a predetermined distance behind the vehicle, attention projection, warning projection, etc., are selected / determined for display. Conversely, if no obstacle is detected within a predetermined distance behind the vehicle, normal projection is selected / determined. Next, in step S904, the image selected in step S903 is projected.

[0174] Step S905 is the step of determining whether to end image projection. In step S905, based on the projection end condition including vehicle information 4, it is determined whether to end image projection. For example, if the gear is changed to parking in step S905, it is determined that image projection should be ended, so image projection ends.

[0175] Alternatively, steps S901 to S904 can be repeated until the image projection is determined to be complete in step S905.

[0176] Figure 16 It is a table representing objects. For example... Figure 16 As shown, the object in Example 1 is a person. A person, as an object, possesses the property of movement; a moving object is one that can move at its own will. Furthermore, since the object is a person, it is capable of recognizing (understanding) the image projected by the vehicle, and therefore can use the image to notify of danger. Thus, the image recognition property is "capable," and the person is the object whose attention is drawn through the image display. For example, when a person is detected, in order to alert both the driver and the detected person, an image that the person can recognize is projected onto the person, and a notification is given to the driver using sound.

[0177] In Example 2, the object is a living being other than a person or an object. The property of a person as an object is that it is a stationary object, meaning it cannot move on its own. Conversely, the property of a living being other than a person as an object is that it is a moving object. Living beings other than people or objects cannot recognize the images projected by the vehicle, so they cannot be alerted to danger through the images; therefore, the image recognition property is "unable." In this case, the alert should not be given to the detected object, but rather to the driver. Therefore, when a living being other than a person or an object is detected, the driver should be alerted with sound.

[0178] In Example 3, the object is either uncertain or unidentified. For instance, when identifying objects from captured images, the presence of the object is detected, and it is identified as belonging to a type of learned (trained) object. The identification result is evaluated based on the matching degree between the detected object and the learned objects, and it is identified as belonging to a category with high confidence. If the captured image is unclear, or the object is not within a learned object category, the confidence of the image recognition decreases. If the confidence level is below a certain value, the detected object cannot be identified, thus there are cases where the detected object cannot be determined. In such cases where the confidence of image recognition is low and the object identification result is questionable, the detected object is considered "uncertain" and projected according to a human standard. In such cases, the object is an unidentified object. The movement attributes and image recognition attributes of the unidentified object are "uncertain," but for safety reasons, it is necessary to alert the driver and the surrounding area.

[0179] Figure 17This is an embodiment of projecting an image while the vehicle is reversing. In step S1001, it is detected whether the vehicle is reversing based on information about the vehicle from the vehicle. For example, if the reverse gear is detected to be ON, it is determined that the vehicle is reversing. If the vehicle is detected to be reversing, proceed to step S1002. In addition to the reverse gear, the backward movement itself can also be detected based on the rotation direction of the wheels or GPS, etc., regarding the vehicle reversing. In step S1002, an object (e.g., an obstacle) is detected using a sensor. If an obstacle is detected in the area behind the vehicle (yes), proceed to step S1003; if no obstacle is detected in the area behind the vehicle (no), proceed to step S1006.

[0180] Step S1003 is the step of identifying / detecting an object. In step S1003, it is determined whether the detected obstacle is a person or an object, etc. After identifying the detected object, proceed to step S1004. In step S1004, the relative distance between the reversing vehicle and the detected object, or the location of the detected object, is detected. After detecting the relative distance between the vehicle and the detected object, proceed to step S1005. In step S1005, the degree of danger is determined based on the relative distance and location of the vehicle and the detected object detected in step S1004.

[0181] If the hazard level determined in step S1005 is "low", proceed to step S1006. Here, a state of no hazard, such as no object being detected, is included in the "low" hazard level. If the hazard level determined in step S1005 is "medium", proceed to step S1007. If the hazard level determined in step S1005 is "high", proceed to step S1008. The hazard levels described here will be explained later.

[0182] Step S1006 performs normal projection. Step S1007 performs attention projection. Step S1008 performs warning projection. Step S1009 is a step to end image projection based on conditions. If image projection is not ended in step S1009, for example, if the reversing action continues and the detected object (such as a person) has passed the area (no object detected), it is necessary to re-detect whether there are obstacles in the area behind the vehicle that will continue to move, so the process returns to step S1002. Alternatively, when switching from reversing to forward action, image projection is not ended, or image projection related to the vehicle's forward movement is performed.

[0183] Figure 18This indicates the detection distance and the level of danger. In this embodiment, the area behind the vehicle includes a projectable area (or projected area) and an area outside the projected area (non-projectable area). The projectable area can be set, for example, to a distance of less than a predetermined distance (e.g., 3 meters) in the direction away from the rear of the vehicle in the longitudinal direction, and less than a predetermined distance (e.g., 2 meters) in the straight-line distance away from the left and right sides of the vehicle in the width direction. Within the projectable area, a attention distance located far from the vehicle and a warning distance located near the vehicle are set. Additionally, outside the projected area, there are attention distances, warning distances, and safe distances located further away from the vehicle than the attention distance. For example, in Example 1, the distance from the vehicle to an object (e.g., an obstacle such as a tree) is a safe distance. The detection location of an object located at the safe distance is outside the projected area of ​​the projected image. When an object is located at the safe distance and exists outside the projected area, or when no object is detected, the level of danger is "low," and normal projection is performed.

[0184] In Example 2, the distance from the vehicle to the object is the attention distance. The detection location of an object within the attention distance can be within or outside the projection area. When the object is within the attention distance of the projection area behind the vehicle, the hazard level is "medium," and attentional projection is performed. On the other hand, when the object is within the attention distance outside the projection area, i.e., outside the area behind the vehicle (e.g., on either side of the vehicle), although the hazard level is "medium," attentional projection may not be performed.

[0185] In Example 3, the distance from the vehicle to the object is the warning distance. The detection location of an object within the warning distance can be either within or outside the projection area. When the object is located within the warning distance of the projection area behind the vehicle, the danger level is "high," and the image projection is a warning projection. The safe distance, attention distance, and warning distance are preset distances.

[0186] Figure 19 This is an example of projection based on the detected object and the type of projection. Example 1: The detected object is a person. In the area behind the vehicle, the distance from the vehicle to the person is determined as the attention distance, and in this example, the hazard level is set to "medium." In this case, it is necessary to alert the detected person and the area around the vehicle (safe distance), so the projection is an attention projection. For example, the projection shows a typical image of the vehicle reversing, and an attention image is projected onto the area near the detected person. Thus, the detected person recognizes the attention image displayed near them, thereby prompting them to stop walking or take evasive action. Alternatively, a notification can be given to the driver using sound.

[0187] Next, in Example 2, the detected object is an object. In the area behind the vehicle, the distance from the vehicle to the object is determined as the attention distance, and the danger level is set to "medium". In this case, it is necessary to alert the area around the vehicle, so attention projection is performed. For example, the projection shows a typical image of the vehicle reversing. Alternatively, a notification can be given to the driver using sound.

[0188] Example 3 detects an uncertain object (unidentified object). In the area behind the vehicle, the distance from the vehicle to the unidentified object is determined as the attention distance, and the hazard level is set to "medium." In this case, there is a possibility that the detected object is a person, so it is necessary to alert the driver to the detected unidentified object and the area around the vehicle. Therefore, attention projection is performed. For example, a normal image of the vehicle reversing is projected, and an attention image is projected onto the area near the detected unidentified object. Thus, if the detected unidentified object is a person, the person can recognize the attention image displayed near them, thereby prompting them to stop walking or take evasive action. Alternatively, a notification can be given to the driver using sound.

[0189] Example 4: The detected object is a person. In the area behind the vehicle, the distance from the vehicle to the person is determined as the warning distance, and the danger level is set to "high." In this case, it is necessary to warn both the detected person and the driver who is reversing, so warning projection is used. For example, a flashing projection shows the normal image of the vehicle reversing, and a flashing attention image is projected onto the area near the detected person. By performing flashing projection as shown in this example, the effect of attracting the attention of the detected person is achieved. Furthermore, the flashing projection causes the image of the reversing camera to flash, which also has the effect of attracting the gaze of the driver who is reversing. In addition, in this example, the driver is also notified audibly, thus notifying the user of the danger even if they are unaware of it.

[0190] Example 5 detects an object, such as a traffic cone. In the area behind the vehicle, the distance from the vehicle to the object is determined as the warning distance, and the danger level is set to "high." In this case, a warning needs to be issued to the driver who is reversing, so a warning projection is performed. For example, a flashing projection shows a typical image of the vehicle reversing, and an audible notification is given to the driver.

[0191] Example 6 detects an uncertain object (unidentified object). In the area behind the vehicle, the distance from the vehicle to the unidentified object is determined as a warning distance, and the danger level is set to "high." In this case, there is a possibility that the detected unidentified object is a person, so it is necessary to warn both the detected unidentified object and the driver who is reversing. Therefore, a warning projection is performed. For example, a flashing projection shows a normal image of the vehicle reversing, and a flashing attention image is projected onto the vicinity of the detected unidentified object. In addition, an audible notification is also given to the driver. Furthermore, although not illustrated, in cases where no object is detected, or when the detected object is at a safe distance, a normal projection is performed to prevent unnecessary attention from being drawn.

[0192] Figures 20A-20C This is a diagram showing the positional relationship between the vehicle and the detected objects. The projection area 20 is the area (projectable area) from which information for notifying pedestrians, drivers, etc., can be projected; specifically, in this embodiment, it is the area behind the vehicle. Furthermore, as examples of detected objects, a person 210 and an obstacle 220 (trees, walls, traffic cones, etc.) are illustrated. L1, L2, and Ls are specified distances for performing projection judgments.

[0193] like Figure 20A As shown, the safety zones from 0 to L1, L2, and Ls are defined using straight-line distances from behind the vehicle. The area from 0 to L2 is the area where images can be projected (within the projection area), while areas longer than L2 or areas further away from the vehicle compared to L2 are areas where images cannot be projected (outside the projection area). Additionally, the distance from 0 to L1 can be set as the warning projection area, and the distance from L1 to L2 can be set as the attention projection area. The area from L2 to Ls is outside the projection area, therefore... Figure 20A Person 210 and obstacle 220 are located outside the vehicle's projection area 20.

[0194] like Figure 20B As shown, regarding the distances from the center of the circle behind the vehicle in the width direction to L1, L2, and Ls, a circle centered at the rear center of the vehicle is used to define the safety zone. This allows for consideration of the difference in hazard levels between directly behind and diagonally behind the vehicle. The area within radius L2 is the area where images can be projected (projection area), while the area with a radius greater than radius L2 (e.g., the area between radius L2 and radius Ls) is the area where images cannot be projected (outside the projection area). Furthermore, the area within radius L1 can be designated as the warning projection area, and the area from radius L1 to radius L2 can be designated as the attention projection area. The area from radius L2 to radius Ls is outside the projection area, therefore... Figure 20BPerson 210 and obstacle 220 are located outside the vehicle's projection area 20.

[0195] like Figure 20C As shown, for L1, L2, and Ls, a straight-line distance is used to define the portion equivalent to the vehicle width, while concentric circles centered on the vehicle edge are used to define the outer edge of the vehicle width. By combining straight-line and circular regions in this way, more precise region settings can be achieved. In this example, a straight-line distance is used to define the portion equivalent to the vehicle width, but it is also possible to use a straight-line distance to define the portion equivalent to the width of the projection area. Figure 20A , Figure 20B Similarly, Figure 20C People 210 and obstacles 220 are also present outside the vehicle's projection area 20.

[0196] Figures 21A-21F It is a diagram that projects an image based on the object. Figure 21A , Figure 21B and Figure 21C This is an example of a case where the detected object is a person. Additionally, Figure 21D , Figure 21E and Figure 21F This is an example of a case where the detected object is a physical object. Figure 21A This example illustrates a situation where a person 210 is detected by a sensor while the vehicle is reversing, and the detected person 210 is located outside the projection area 20. In this case, the projection device (or projection unit) detects that the vehicle is in reverse and performs a normal projection, such as projecting a rectangular image to the left or right behind the vehicle. Additionally, the driver of the vehicle can be notified via voice that "there is someone outside the projection area."

[0197] Figure 21B This example illustrates a situation where a person 210, detected by sensors while the vehicle is reversing, is located within the projection area 20 and within the warning distance. Alternatively, if the distance from the vehicle to the person 210 is greater than the warning distance within the projection area 20, the danger level is set to "medium." In this case, it is necessary to alert the detected person 210, so the projection device projects an image of attention. For example, a standard rectangular image is projected to the left and right of the rear of the vehicle, and an image of attention is projected to the vicinity of the detected person 210. The image of attention is projected in a direction that the detected person 210 can see, such as in front of the person's feet. Alternatively, the driver can be notified audibly that "there is someone in the projection area."

[0198] Figure 21CThis example illustrates a situation where a person 210, detected by sensors while the vehicle is reversing, is located within the projection area 20 and within the warning distance zone. Alternatively, the distance from the vehicle to the person 210 may be the warning zone, with the danger level set to "high." In this case, a warning is issued to the detected person 210, so the projection device projects a warning. For example, a typical rectangular image flashes to the left and right behind the vehicle, while a warning image flashes in the vicinity of the detected person 210. Alternatively, the driver may be notified audibly that "there is someone in the danger distance zone of the vehicle." Figure 21A , Figure 21B and Figure 21C In the middle, when the vehicle is reversing, as the distance from the vehicle to the person 210 becomes shorter, the projection device changes from normal projection to attention projection and warning projection.

[0199] Figure 21D This example illustrates an obstacle 220 detected by sensors while the vehicle is reversing, and the detected obstacle 220 is located outside the projection area 20. In this case, the projection device performs a normal projection, such as projecting a rectangular image to the left or right behind the vehicle. Alternatively, the driver can be notified via voice that "there is an object outside the projection area."

[0200] Figure 21E This example illustrates an obstacle 220 detected by sensors while the vehicle is reversing, located within the projection area 20 and within the attention distance range. Alternatively, if the distance from the vehicle to the obstacle 220 is greater than the warning distance within the projection area 20, the hazard level is set to "medium." In this case, the projection device performs attention projection. For example, it projects a typical rectangular image to the left and right behind the vehicle. Alternatively, it may also notify the driver audibly that "an object is in the projection area."

[0201] Figure 21F This example illustrates an area within the projection area 20 where an obstacle 220 detected by sensors is located within a warning distance when the vehicle is reversing. Alternatively, the distance from the vehicle to the obstacle 220 may be considered a warning area, with the danger level set to "high." In this case, the projection device projects a warning. For example, a typical rectangular image is projected, flashing left and right behind the vehicle. Alternatively, the length of the rectangular image projected on the side behind the vehicle where the obstacle 220 is present may be asymmetrical to the length of the rectangular image on the side where the obstacle 220 is not present; additionally, the brightness may be increased and the image may flash. These changes in image information can make the driver, who may not have noticed the obstacle 220 through images from a rear camera, aware of its presence. Furthermore, the driver may be notified audibly that "an object is in the danger distance area of ​​the vehicle." Figure 21D , Figure 21E and Figure 21F In the middle, when the vehicle is reversing, as the distance from the vehicle to the obstacle 220 becomes shorter, the projection device changes from normal projection to attention projection and warning projection.

[0202] Figure 22 This is an example of projecting an image when a vehicle is reversing. In step S1501, reversing is detected based on information from the vehicle. For example, if the reverse gear is detected to be ON, it is determined that the vehicle is reversing. If reversing is detected, proceed to step S1502.

[0203] In step S1502, the sensor detects obstacles in the area behind the vehicle. If an obstacle is detected in the area behind the vehicle (yes), proceed to step S1503; if no obstacle is detected in the area behind the vehicle (no), proceed to step S1508.

[0204] In step S1503, it is determined whether the detected object is an object. In step S1503, if the object is an object (yes), proceed to step S1504; if the object is not an object (no), proceed to step S1505.

[0205] In step S1504, the relative distance between the reversing vehicle and the detected object, or the position of the detected object, is detected. After obtaining the relative distance information or position information between the vehicle and the detected object, the process proceeds to step S1507.

[0206] In step S1505, the relative distance between the reversing vehicle and the detected object, or the position of the detected object, is detected, and the process proceeds to step S1506. Next, in step S1506, the movement vector of the detected object is detected, and the process proceeds to step S1507. Here, regarding the movement vector, for example, by analyzing information obtained from a camera or LiDAR between frames, the direction and speed of movement relative to the vehicle can be determined. For example, if the detected object (e.g., a person) is moving, the level of danger varies depending on whether it is moving towards or away from the vehicle. Furthermore, depending on the magnitude of the movement vector, whether the object will come into contact with the vehicle changes accordingly, thus changing the level of danger.

[0207] In step S1507, the degree of danger is determined based on the information detected in steps S1504, or steps S1505 and S1506. If the degree of danger determined in step S1507 is "low," proceed to step S1508; the case of no danger is also included in the "low" degree of danger. If the degree of danger determined in step S1507 is "medium," proceed to step S1509. If the degree of danger determined in step S1507 is "high," proceed to step S1510.

[0208] Step S1508 performs normal projection, and step S1509 performs attention projection. Step S1510 performs warning projection. Step S1511 is a step to determine whether to end image projection based on conditions. If image projection is not ended in step S1511, for example, if a person or other object has passed the area within a specified time and is no longer detected, it is necessary to re-detect whether there are obstacles in the projection area to be traveled next, so the process returns to step S1502.

[0209] Figure 23 This is a table indicating the level of danger when the detected object is a person or an unknown quantity. The level of danger when the detected object is an object is already indicated in... Figure 18 In the middle. For example Figure 23 As shown in Example 1, the distance from the vehicle to the detected object is a safe distance. The detection position of an object located at the safe distance is outside the image projection area, the hazard level is set to "low", and the projection device performs normal projection.

[0210] In Example 2, the distance from the vehicle to the detected object is the attention distance. The detection location of the object within the attention distance is within the projection area of ​​the image. Furthermore, in this example, the vehicle is not located on the extension of the relative velocity vector of the detected object relative to the vehicle. Under this condition, even if the vehicle and the detected object move separately, they will not come into contact. In such a case, the danger level is set to "low," and the projection device performs normal projection.

[0211] In Example 3, the distance from the vehicle to the detected object is the attention distance. The detection location of the object within the attention distance is within the projection area of ​​the image. Furthermore, in this example, the vehicle is located on the extension of the relative velocity vector of the detected object relative to the vehicle. Under this condition, the probability of contact between the vehicle and the detected object is high when they move separately. In this case, the danger level is set to "medium," and the projection device performs attention projection.

[0212] Figure 23In Example 4, the distance from the vehicle to the object is the warning distance. The detection location of the object within the warning distance is within the projection area of ​​the image. Furthermore, in this example, the vehicle is not located on the extension of the relative velocity vector of the detected object relative to the vehicle. Under this condition, even if the vehicle and the detected object move separately, they will not come into contact. In this case, the danger level is set to "medium," and attention projection is performed.

[0213] In Example 5, the distance from the vehicle to the object is the warning distance. The detection location of the object within the warning distance is within the projection area of ​​the image. Furthermore, in this example, the vehicle is located on the extension of the relative velocity vector of the detected object relative to the vehicle. Under this condition, the probability of contact between the vehicle and the detected object is high when they move separately. In this case, the danger level is set to "high," and the projection device projects a warning. The safe distance, attention distance, and warning distance are preset distances.

[0214] Figure 24 This is an example of how the projected image changes accordingly based on the detected object. Example 1: The detected object is a person. The distance from the vehicle to the detected person is determined as the attention distance. The detected person is moving towards the vehicle, and in this example, the danger level is set to "medium." In this case, it is necessary to draw attention to the detected person and the area around the vehicle (safe distance), so the projection device projects an attention image. For example, it projects a normal image showing the vehicle reversing, and also projects an attention image to the area near the detected person.

[0215] Example 2: The detected object is a physical object. The distance from the vehicle to the detected object is determined as the attention distance; in this example, the hazard level is set to "moderate." Attention projection is then performed. For example, a typical image of the vehicle reversing is projected. Alternatively, a notification can be given to the driver via audio.

[0216] Example 3: The detected object is an uncertain object (unidentified object). The distance from the vehicle to the detected unidentified object is determined as the attention distance. The detected unidentified object is moving towards the vehicle, and in this example, the danger level is set to "medium." In this case, there is a possibility that the detected object is a person, so it is necessary to alert the driver to the detected unidentified object and the area around the vehicle. The projection device projects attention. For example, it projects a typical image of the vehicle reversing and projects an attention image onto the area near the detected unidentified object. Thus, if the detected unidentified object is a person, the person can recognize the attention image displayed near them, thereby prompting them to stop walking or take evasive action. Alternatively, a notification can be given to the driver using sound.

[0217] Example 4: The detected object is a person. The distance from the vehicle to the detected person is determined as the warning distance. The detected person is moving towards the vehicle in the vicinity of the projection area, and in this example, the danger level is set to "high". In this case, a warning needs to be issued to both the detected person and the driver who is reversing, and the projection device performs a warning projection. For example, a flashing projection shows the normal image of the vehicle reversing, and a flashing attention image is projected onto the vicinity of the detected person. By performing the flashing projection as shown in this example, the effect of attracting the attention of the detected person is achieved. Furthermore, the flashing projection causes the image of the reversing camera to flash, which also has the effect of attracting the gaze of the driver who is reversing. In addition, in this example, the driver is also notified audibly, thus notifying the user of the danger even if they are unaware of it.

[0218] Example 5: The detected object is an obstacle, such as a traffic cone. The distance from the vehicle to the detected object is determined as the warning distance. The detected object is located in the vicinity of the vehicle within the projection area, and in this example, the danger level is set to "high." In this case, the projection device projects a warning signal. For example, a flashing projection may show a typical image of the vehicle reversing, and an audible notification may be given to the driver.

[0219] Example 6 detects an uncertain object (unidentified object). The distance from the vehicle to the detected unidentified object is determined as the warning distance. The unidentified object is located in the vicinity of the vehicle within the projection area and is moving towards the vehicle. In this example, the danger level is set to "high". In this case, it is necessary to warn both the detected unidentified object and the driver who is reversing, so the projection device performs a warning projection. For example, a flashing projection shows a normal image of the vehicle reversing, and a flashing attention image is projected onto the vicinity of the detected unidentified object. Additionally, an audible notification is given to the driver. Furthermore, although not illustrated, normal projection is used to prevent unnecessary attention from being drawn when no object is detected or when the detected object is at a safe distance.

[0220] Figures 25A-25D It is a graph that shows the direction and speed of movement of people and vehicles as examples of detected objects. Figures 25A-25D Let's use a person 210 as an example. With the vehicle's velocity vector va and the person's velocity vector vb, to account for the relative motion between the person 210 and the vehicle, the vehicle's velocity vector va is transformed into a velocity vector in the opposite direction and assigned to the person 210, thus considering only the motion of the person 210. The person's velocity vector -va and velocity vector vb are added together to obtain the vector (vb-va), which is the relative velocity vector between the vehicle and the person 210. If the vehicle continues to move while positioned on the extension of the relative velocity vector, a collision will occur.

[0221] Reference Figures 25A to 25D This illustrates how the dangerous situation dynamically changes in response to changes in the behavior of the vehicle and the detected person 210. Figure 25A In this scenario, the relative velocity vector between the vehicle and the pedestrian (210) is pointing towards the vehicle, making the risk of collision extremely high. Figure 25A The collision risk is highest in this state. Figures 25B to 25D In the current state, the danger is not high, but Figure 25C If the vehicle's reversing speed increases, there is a risk of collision. Additionally, Figure 25B and Figure 25D In the current state, the danger is not high, but caution is needed when the vehicle's reversing direction changes or when the person's movement direction changes. Figure 25A and Figure 25C In this case, depending on the distance between the vehicle and the detected person 210, the projection device changes from normal projection to attention projection or warning projection accordingly. Figure 25B and Figure 25D In the case where the vehicle's reversing direction changes to the direction in which the detected person 210 is located, the projection device changes from normal projection to attention projection or warning projection according to the distance between the vehicle and the detected person 210. That is, the projection device performs normal projection, attention projection, or warning projection accordingly based on the direction and speed of movement of the detected object and vehicle.

[0222] Furthermore, the present invention is not limited to the above embodiments, but includes various modifications and equivalent structures within the technical concept of the claimed technical solutions. For example, the present invention has been described in detail in the above embodiments for ease of understanding, but the present invention is not limited to having all the described structures. In addition, for example, other structures may be added, deleted, or substituted for a part of the structure of the embodiments.

[0223] Furthermore, by using the technology described above, necessary information such as images facing the driver and images of people around the vehicle can be appropriately displayed. This provides an information projection device that aids in safe driving, thereby preventing traffic accidents. Furthermore, it contributes to the United Nations Sustainable Development Goals (SDGs) "3. Good Health and Well-being".

[0224] Explanation of reference numerals in the attached figures

[0225] 2: Vehicles

[0226] 11: Projection device,

[0227] 13: Headlight,

[0228] 14: Turn signals

[0229] 100: Controller.

Claims

1. An information projection device mounted in a vehicle, capable of projecting images onto a road surface, characterized in that, include: The acquiring department obtains information about the vehicle; and The projection unit, and its projected image, Specifically, when the vehicle is reversing, the area behind the vehicle is divided into multiple zones. The acquisition unit obtains detection information about obstacles around the vehicle. The image is projected based on information about the area where the detected object is located.

2. The information projection device as described in claim 1, characterized in that: The area behind the vehicle has a first area and a second area. The first area is the area near the vehicle, and the second area is the area far from the vehicle. If the object exists in the projection area of ​​the projection part in the first region, the image is not projected.

3. The information projection device as described in claim 1, characterized in that: The area behind the vehicle has a first area and a second area. The first area is the area near the vehicle, and the second area is the area far from the vehicle. When the object exists in the projection area of ​​the projection unit in the second region, the image is projected in accordance with the distance from the vehicle to the object.

4. The information projection device as described in claim 3, characterized in that: In the projection area of ​​the second region, the images projected onto the side where the object is present are different from those projected onto the side where the object is not present.

5. The information projection device as described in claim 4, characterized in that: The image is projected to blink.

6. The information projection device as described in claim 2 or 3, characterized in that: The area behind the vehicle also has a third area. The third region is located outside the projection region of the projection unit. If the object is present in the third region, a attention image is projected onto the first region or the second region that is close to the object.

7. The information projection device as described in claim 1, characterized in that: The acquisition unit acquires multiple obstacle detection information around the vehicle. Based on information about areas with obstacles, information to be used is selected from multiple obstacle detection data, and the image is projected based on the selected information.

8. An information projection device mounted in a vehicle, capable of projecting images onto a road surface, characterized in that, include: The acquiring department obtains information about the vehicle; and The projection unit, and its projected image, When the vehicle is reversing, the projection unit performs normal projection, attention projection, or warning projection based on the position information of objects in the area behind the vehicle obtained by the acquisition unit.

9. The information projection device as described in claim 8, characterized in that: The area behind the vehicle has a projectile area and an area outside the projectile area. When the object is located outside the projection area and is not behind the vehicle, the projection unit performs the normal projection.

10. The information projection device as described in claim 8, characterized in that: The area behind the vehicle has a projectile area and an area outside the projectile area. A notice distance and a warning distance are set within the projectible area, with the notice distance being farther from the vehicle compared to the warning distance. When the object is present at the attention distance and is located behind the vehicle, the projection unit performs the attention projection.

11. The information projection device as described in claim 10, characterized in that: The projection unit projects the warning when the object is present at the warning distance and behind the vehicle.

12. The information projection device as described in claim 10, characterized in that: When the object is a person, the projection unit projects an attention image onto the area near the person.

13. The information projection device as described in claim 11, characterized in that: In the case that the object is a person, the projection unit flashes a attention image to the area near the person and sends an audio notification message to the driver of the vehicle.

14. The information projection device as described in claim 8, characterized in that: Depending on the direction and speed of movement of the object and the vehicle, the projection unit performs the normal projection, the attention projection, or the warning projection.

Images