Image projection device, storage medium, and image projection method

Abstract

The present disclosure relates to an image projection device, a storage medium, and an image projection method. The image projection device includes a control unit and a projection unit, and is mounted on a vehicle or worn by a driver of the vehicle. The control unit receives a selection designating a specific user, detects a current position, acquires an image showing at least a part of an appearance of the specific user, and projects the image on a driver's seat of the vehicle by the projection unit, the image being captured when the specific user has previously driven the specific user's vehicle at a position the same as or close to the current position.

Description

Technical Field

[0001] The present disclosure relates to an image projection device, a storage medium, and an image projection method. Background Art

[0002] It is a well-known technique to display a virtual space for providing entertainment on a head-mounted display worn by a passenger while an autonomous vehicle is in motion (for example, see Japanese Unexamined Patent Application Publication No. 2017-146963). Summary of the Invention

[0003] This technique enables passengers to enjoy playing games and reading. However, the passenger experience can be further improved.

[0004] The present disclosure provides an image projection device, a storage medium, and an image projection method, all of which provide a realistic and visual experience of a specific user's driving to the driver of a vehicle.

[0005] An image projection device according to a first aspect of the present disclosure is installed on a vehicle or worn by a driver of the vehicle. The image projection device includes a control unit and a projection unit. The control unit is configured to receive a selection specifying a specific user, detect a current position, acquire an image showing at least a part of the appearance of the specific user, and project the image on the driver's seat of the vehicle by the projection unit, the image being taken when the specific user has previously driven the specific user's vehicle at a position identical or close to the current position.

[0006] A storage medium according to a second aspect of the present disclosure stores a program that causes a calculator to perform the following operations, the computer being an image projection device installed on a vehicle or worn by the driver of the vehicle: receive a selection specifying a specific user; detect a current position; acquire an image showing at least a part of the appearance of the specific user, the image being taken when the specific user has previously driven the specific user's vehicle at a position identical or close to the current position; and project the image on the driver's seat of the vehicle.

[0007] An image projection method according to a third aspect of the present disclosure is performed by an image projection device installed on a vehicle or worn by the driver of the vehicle. The image projection method includes: receiving a selection specifying a specific user, detecting a current position, acquiring an image showing at least a part of the appearance of the specific user, and projecting the image on the driver's seat of the vehicle, the image being taken when the specific user has previously driven the specific user's vehicle at a position identical or close to the current position.

[0008] An image projection device, a storage medium, and an image projection method according to various aspects of the present disclosure can provide a realistic and visual experience of a specific user's driving to a driver of a vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, in which like reference numerals represent like elements, and in which:

[0010] Figure 1 is a schematic diagram illustrating an image projection device and a vehicle according to the present embodiment;

[0011] Figure 2 is a block diagram schematically showing a configuration of the image projection device;

[0012] Figure 3 is a block diagram schematically showing a configuration of the vehicle;

[0013] Figure 4 is a diagram showing a data structure of a close relative database;

[0014] Figure 5 is a diagram showing a display screen displayed on the image projection device;

[0015] Figure 6 is a diagram showing a projected image; and

[0016] Figure 7 is a flowchart showing operations of the image projection device. DETAILED DESCRIPTION

[0017] Figure 1 is a schematic diagram illustrating an image projection device 1 and a vehicle 2 according to the present embodiment. The image projection device 1 is mounted on the vehicle 2. The image projection device 1 can establish communication with the vehicle 2 via, for example, a mobile communication network or the Internet. The vehicle 2 is driven by a user U01.

[0018] The image projection device 1 is a device mounted at an arbitrary position inside or outside the vehicle 2. In Figure 1 , the image projection device 1 is attached to a windshield inside the vehicle 2, but it can also be attached to another position. The image projection device 1 projects an image into the real space. The image includes at least one of a still image and a moving image.

[0019] As an alternative example, the image projection device 1 can be a HUD (Head-Up Display) mounted on the vehicle 2. As another alternative example, the image projection device 1 can be augmented reality (AR) glasses worn and operated by the user U01. The image projection device 1 projects an image into the real space, for example, a virtual image.

[0020] As another alternative example, the image projection device 1 may be a head-mounted display worn and operated by the user U01. In this case, the image projection device 1 sets up a virtual space and projects an image onto the virtual space.

[0021] Examples of the vehicle 2 include gasoline vehicles, diesel vehicles, HVs, PHVs, EVs, FCVs, and any other type of vehicle. "HV" is an abbreviation for a hybrid vehicle. "PHV" is an abbreviation for a plug-in hybrid vehicle. "EV" is an abbreviation for an electric vehicle. "FCV" is an abbreviation for a fuel cell vehicle. In the present embodiment, the vehicle 2 is driven by the user U01. As an alternative example, the vehicle 2 may be automated to some extent. The degree of automation of the vehicle may be classified according to the SAE level, for example, classified into any level from level 1 to level 5. "SAE" is an abbreviation for the "Society of Automotive Engineers". The vehicle 2 may be a MaaS vehicle. "MaaS" is an abbreviation for "Mobility as a Service".

[0022] Hereinafter, an outline process executed by the image projection device 1 of the present embodiment will be described. The image projection device 1 is installed on the vehicle 2 or worn by the driver of the vehicle 2. The control unit 11 of the image projection device 1 accepts a selection specifying a specific user, detects the current location, acquires an image showing at least a part of the appearance of the specific user, and projects the image onto the seat of the driver of the vehicle 2, the image having been taken when the specific user previously drove their vehicle at the same or a nearby location as the current location. With this configuration, the driver of the vehicle 2 can see up close an image of the specific user driving. The image projection device 1 provides the driver with a real and visual experience of the specific user driving.

[0023] Reference will be made to Figure 2 describe in detail the internal structure of the image projection device 1.

[0024] The image projection device 1 includes a control unit 11, a communication unit 12, a storage unit 13, a sensor 14, a projection unit 15, and a display unit 16. The components of the image projection device 1 are communicably connected to each other via, for example, a dedicated line.

[0025] The control unit 11 includes, for example, at least one general-purpose processor, which includes a central processing unit (CPU) or a microprocessing unit (MPU). The control unit 11 may include at least one dedicated processor for specific processing. The control unit 11 may include at least one dedicated circuit instead of a processor. For example, the dedicated circuit may be a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The control unit 11 may include an electronic control unit (ECU). The control unit 11 transmits and receives any information via the communication unit 12.

[0026] The communication unit 12 may include a communication module corresponding to at least one wired or wireless local area network (LAN) standard for connecting to a network. The communication unit 12 may include a module corresponding to at least one mobile communication standard such as Long Term Evolution (LTE), 4th Generation (4G), or 5th Generation (5G). The communication unit 12 may include a communication module corresponding to at least one short-range communication standard or specification such as IrDA, or RFID. The communication unit 12 transmits and receives any information via the network.

[0027] The storage unit 13 may include, for example, a semiconductor memory, a magnetic memory, an optical memory, or a combination of at least two of them. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 13 may be used as, for example, a main storage device, an auxiliary storage device, or a cache storage device. The storage unit 13 may store information about the results analyzed or processed by the control unit 11. The storage unit 13 may store various information about the operation or control of the image projection device 1. The storage unit 13 may store system programs, application programs, embedded software, etc. The storage unit 13 may be provided outside the image projection device 1 and accessed from the image projection device 1. The storage unit 13 includes the next-of-kin database described later.

[0028] The sensor 14 may include a position information sensor. The position information sensor includes at least one GNSS receiver. "GNSS" is an abbreviation for Global Navigation Satellite System. GNSS includes, for example, at least one of GPS, QZSS, Beidou, GLONASS, and Galileo. "GPS" is an abbreviation for Global Positioning System. "QZSS" is an abbreviation for Quasi-Zenith Satellite System. QZSS satellites are called quasi-zenith satellites. "GLONASS" is an abbreviation for Global Navigation Satellite System. The sensor 14 measures the position of the image projection device 1. The measurement result generated by the sensor 14 is obtained by the control unit 11 as the position information of the image projection device 1. "Position information" is information that can specify the position of the vehicle 2 and includes, for example, coordinates. The position of the image projection device 1 substantially coincides with the position of the vehicle 2.

[0029] The sensor 14 may further include an image sensor. The image sensor includes a camera.

[0030] The sensor 14 also includes a sensor for detecting driving details of the vehicle 2 or an interface with the sensor. The sensors include, for example, sensors for detecting the speed, longitudinal acceleration, lateral acceleration, deceleration, accelerator operation amount, brake operation amount, steering angle, selected eco mode, odometer reading, engine specifications, etc. of the vehicle 2. The driving details can indicate driving operations such as acceleration, deceleration, steering, engine braking, eco mode selection, appropriate driving path, etc.

[0031] The display unit 16 can be an LCD monitor or an organic EL monitor. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescence. The display unit 16 is visually recognized by the driver of the vehicle 2.

[0032] The display unit 16 can be connected to the vehicle 2 as an external output device instead of being provided in the image projection device 1. Any scheme such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used as the connection.

[0033] In an embodiment where the image projection device 1 is a HUD, the display unit 16 can be a combiner.

[0034] Reference will be made to Figure 3 Describe the internal structure of the vehicle 2 in detail.

[0035] The vehicle 2 includes a control unit 21, a communication unit 22, and a storage unit 23. The components of the vehicle 2 are communicably connected to each other via, for example, dedicated lines.

[0036] The hardware configurations of the control unit 21, the communication unit 22, and the storage unit 23 can be the same as those of the control unit 11, the communication unit 12, and the storage unit 13 of the image projection device 1. Therefore, some explanations are omitted herein.

[0037] Hereinafter, the processing performed by the image projection device 1 of the present embodiment will be described in detail. For example, a situation where the user U01 experiences the driving of another person while driving the vehicle 2 will be described. Here, the other person may be a close relative of the user U01 who has passed away.

[0038] The user U01 sends a request to the image projection device 1 to be operated to provide an experience of driving performed by a specific user. When the request is received, the image projection device 1 refers to the storage unit 13.

[0039] As Figure 4As shown, the close relative database of the storage unit 13 stores the driving routes (corresponding to "driving routes") when a close relative drives any vehicle, images showing the appearance of the close relative when driving on the driving route (corresponding to "appearance images"), the operations of the vehicle when the close relative drives on the driving route (corresponding to "vehicle operations"), and information on whether the close relative has driven safely (corresponding to "safe driving?") in association with the close relative IDs. The information stored in the close relative database is collected by any device when the close relative drives. Close relative N01 and close relative N02 are stored in the close relative database as close relatives of user U01.

[0040] As Figure 5 shown, the control unit 11 displays on the display unit 16 a screen that requests selecting an image to be projected from the images showing close relatives N01 and N02. For better understanding, the case where user U01 selects close relative N01 will be described. The control unit 11 accepts the selection made by user U01 to specify close relative N01.

[0041] The control unit 11 obtains information indicating the current position from the sensor 14 to detect the current position. The current position is the position of the image projection device 1 or the vehicle 2. The control unit 11 determines the driving route of the vehicle 2 from the obtained information indicating the current position. The control unit 11 refers to the close relative database and determines whether close relative N01 has driven on a driving route including the driving route of the vehicle 2. The case where the driving route R01 includes the current driving route of the vehicle 2 will be described.

[0042] The control unit 11 obtains at least a part of the image V01 from the close relative database, which shows the appearance of close relative N01 when close relative N01 drove the vehicle driven by close relative NO1 at a position the same as or close to the current position. "A position close to (the current position)" may indicate that the two positions are within a predetermined distance. The vehicle of close relative N01 may be the vehicle 2 or another vehicle. The image V01 of close relative N01 may be an image showing the entire appearance of close relative N01 or an image showing a part of the appearance of close relative N01.

[0043] As Figure 6 shown, user U01 operates the steering wheel SW at the driver's seat DS of the vehicle 2 to drive the vehicle 2. The control unit 11 projects the image V01 of close relative N01 onto the driver's seat DS via the projection unit 15. The image V01 may be projected near user U01 or at a position where the image is at least partially superimposed on user U01. The image V01 may include a steering wheel image. User U01 can visually recognize the image V01 of close relative N01 nearby.

[0044] In addition, the control unit 11 refers to Figure 4The related - kin database shown in , and obtain information indicating the operation M01 of the vehicle driven by the related - kin N01 when the related - kin N01 drives his vehicle at a position the same as or close to the current position. As Figure 6 shown, the control unit 11 of the image projection device 1 projects an image of the vehicle 3 operating according to the operation M01 near the vehicle 2. As Figure 6 shown, the image of the vehicle 3 is projected in front of the vehicle 2. As an alternative example, the image of the vehicle 3 can be projected to any position of the vehicle 2, such as the side or the rear of the vehicle 2. The image of the vehicle 3 can be projected to a position where the image at least partially overlaps the vehicle 2.

[0045] As another alternative example, the control unit 11 can link the operation M01 of the vehicle driven by the related - kin N01 with the operation of the vehicle 2 driven by the user U01. In the autonomous driving mode, the vehicle 2 performs substantially the same operation as the vehicle driven by the related - kin N01.

[0046] The control unit 11 refers to the related - kin database and determines whether the operation M01 of the vehicle driven by the related - kin N01 is certified as safe driving. As Figure 4 shown, the operation M01 of the vehicle driven by the related - kin N01 is certified as safe driving. The control unit 11 calculates the matching rate (i.e., the synchronization rate) between the operation M01 of the vehicle driven by the related - kin N01 and the operation of the vehicle 2 driven by the user U01.

[0047] When it is determined that the operation M01 of the vehicle driven by the related - kin N01 is certified as safe driving and the matching rate between the operation M01 of the vehicle driven by the related - kin N01 and the operation of the vehicle 2 driven by the user U01 exceeds the reference value, the control unit 11 notifies the user U01, who is the driver, of the determination result. This notification can be made by sound or displayed on the screen by the image projection device 1 or the vehicle 2.

[0048] will refer to Figure 7 Describe the image projection method performed by the image projection device 1 of this embodiment.

[0049] In step S1, the control unit 11 of the image projection device 1 accepts a selection from the user U01 to specify a particular user.

[0050] In step S2, the control unit 11 detects the current position of the image projection device 1 or the vehicle 2 by using the sensor 14.

[0051] In step S3, the control unit 11 obtains at least a part of the image taken of the appearance of a particular user (e.g., the related - kin N01) when the particular user drives his vehicle at a position the same as or close to the current position from the storage unit 13.

[0052] In step S4, the control unit 11 projects the acquired image onto the driver's seat of the vehicle 2. The acquired image can be projected to a position where the image is at least partially superimposed on the user U01 of the vehicle 2. The control unit 11 can further acquire information indicating the operation of the vehicle of a specific user when the specific user drives his vehicle at the same or a position close to the current position. The control unit 11 can project the acquired image near the vehicle 2, or alternatively, project it to a position where the image is at least partially superimposed on the vehicle 2.

[0053] In step S5, when the operation of the vehicle of the specific user is authenticated as safe driving, the control unit 11 determines whether the matching rate between the operation of the vehicle of the specific user and the operation of the vehicle 2 of the user U01 exceeds a reference value.

[0054] When the determination in step S5 is "yes", the control unit 11 notifies the user U01 of the determination result in step S6.

[0055] When the determination in step S5 is "no", the control unit 11 ends the process.

[0056] According to the present embodiment described above, the control unit 11 accepts the selection of a specific user, detects the current position, acquires an image showing at least a part of the appearance of the specific user, and projects the image onto the driver's seat of the vehicle 2, which image was taken when the specific user previously drove his vehicle at the same or a position close to the current position. With this configuration, the driver of the vehicle 2 can see the driving image of the specific user up close. The image projection device 1 provides the driver with a real and visual experience of the driving of the specific user.

[0057] Furthermore, according to the present embodiment, the control unit 11 acquires information indicating the operation of the vehicle of the specific user when the specific user previously drove the vehicle at the same or a position close to the current position, and projects an image of the vehicle operating in response to this information near the driver's vehicle 2. As an alternative example, the control unit 11 can at least partially superimpose an image of the vehicle operating in response to the information indicating the operation on the driver's vehicle 2. The control unit 11 can partially superimpose at least a part of the image of the specific user on the driver. With this configuration, the image projection device 1 provides the driver with a real and visual experience of the driving of the specific user.

[0058] According to the present embodiment, the specific user can include a close relative of the driver. With this configuration, the image projection device 1 can give the driver the opportunity to experience the driving of a close relative who is close to the driver.

[0059] According to this embodiment, when it is determined that the operation of the vehicle of a specific user is authenticated as safe driving and the matching rate between the operation of the vehicle of the specific user and the operation of the vehicle 2 driven by the driver exceeds a reference value, the control unit 11 notifies the driver who is the driver of the determination result. With this configuration, the control unit 11 can encourage the driver to drive safely, thereby creating a safer traffic environment.

[0060] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can make various changes and modifications based on the present disclosure. Other changes or modifications can be made without departing from the scope of the present disclosure. For example, the functions included in each unit or each step can be rearranged without logical contradiction, and multiple units or steps can be combined into a single component or divided into multiple units or steps.

[0061] For example, in the above embodiment, a program that executes all or part of the functions or processes of the image projection device 1 can be recorded on a computer-readable recording medium. The computer-readable recording medium includes a non-temporary computer-readable recording medium and is, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program can be deployed or distributed by selling, transmitting, or renting a portable recording medium, such as a digital versatile disc (DVD) or a compact disc read-only memory (CD-ROM) on which the program is stored. The program can be stored in the memory of any server and transferred to another computer for deployment or distribution. The program can be provided as a program product. The present disclosure can also be implemented as a program executable by a processor.

[0062] The computer temporarily stores the program recorded in the portable recording medium or the program transmitted from the server, for example, in the main storage device. Then, the computer causes the processor to read the program stored in the main storage device and causes the processor to execute processing according to the read program. The computer can directly read the program from the portable recording medium and execute processing according to the program. Each time the program is transmitted from the server to the computer, the computer can sequentially execute processing according to the received program. The processing may not be to transfer the program from the server to the computer, but to be executed by a so-called ASP service that realizes the function only by instructing execution and obtaining results. "ASP" is an abbreviation for Application Service Provider. The program includes information for processing by an electronic computer, which corresponds to the program. For example, data that is not a direct command to the computer but has the characteristics of defining computer processing corresponds to "corresponding to the program".

Claims

1. An image projection device, which is installed on a vehicle or worn by a driver of the vehicle, the image projection device comprising: A control unit; And A projection unit, Wherein, the control unit is configured to: Receive a selection specifying a specific user; Detect the current position; Obtain an image showing at least a part of the appearance of the specific user, the image being taken when the specific user previously drove the specific user's vehicle at a position the same as or close to the current position; and Project the image onto the driver's seat of the vehicle through the projection unit.

2. The image projection device according to claim 1, wherein, The control unit is configured to: Obtain information indicating the operation of the specific user's vehicle when the specific user previously drove the specific user's vehicle at a position the same as or close to the current position; And Project an image of a vehicle that operates in response to the information indicating the operation near the driver's vehicle.

3. The image projection apparatus according to claim 1, wherein, The control unit is configured to: Obtain information indicating the operation of the specific user's vehicle when the specific user previously drove the specific user's vehicle at a position the same as or close to the current position; And At least partially superimpose an image of a vehicle that operates in response to the information indicating the operation on the driver's vehicle.

4. The image projection apparatus according to claim 1, wherein, The control unit is configured to: Obtain information indicating the operation of the specific user's vehicle when the specific user previously drove the specific user's vehicle at a position the same as or close to the current position; And Link the operation of the driver's vehicle with the operation of the specific user's vehicle.

5. The image projection apparatus according to any one of claims 1 to 4, wherein, The control unit is configured to at least partially superimpose the image showing at least a part of the appearance of the specific user on the driver.

6. The image projection device according to any one of claims 1 to 4, wherein, The specific user includes a close relative of the driver.

7. The image projection device according to any one of claims 2 to 4, wherein, The control unit is configured to notify the driver of the determination result when it is determined that the operation of the specific user's vehicle is certified as safe driving and the matching rate between the operation of the specific user's vehicle and the operation of the driver's vehicle exceeds a reference value.

8. A vehicle equipped with the image projection device according to any one of claims 1 to 7.

9. A storage medium storing a program that causes a computer to perform the following operations, the computer being an image projection device installed on a vehicle or worn by a driver of the vehicle: Receive a selection specifying a specific user; Detect the current position; Obtain an image showing at least a part of the appearance of the specific user, the image being taken when the specific user previously drove the specific user's vehicle at a position the same as or close to the current position; and Project the image onto the driver's seat of the vehicle.

10. The storage medium according to claim 9, wherein, The program causes the computer to: Obtain information indicating the operation of the specific user's vehicle when the specific user previously drove the specific user's vehicle at a position the same as or close to the current position; And Project an image of a vehicle that operates in response to the information indicating the operation near the driver's vehicle.

11. The storage medium according to claim 9, wherein, The program causes the computer to: Obtain information indicating the operation of the specific user's vehicle when the specific user has previously driven the specific user's vehicle at a location that is the same as or close to the current location; And At least partially superimpose an image of a vehicle operating in response to the information indicating the operation on the specific user's vehicle.

12. The storage medium according to claim 9, wherein, The program causes the computer to: Obtain information indicating the operation of the specific user's vehicle when the specific user has previously driven the specific user's vehicle at a location that is the same as or close to the current location; And Link the operation of the specific user's vehicle with the operation of the driver's vehicle.

13. The storage medium according to any one of claims 9 to 12, wherein, The program causes the computer to at least partially superimpose an image showing at least a part of the appearance of the specific user on the driver.

14. The storage medium according to any one of claims 9 to 12, wherein, The specific user includes a close relative of the driver.

15. An image projection method, which is performed by an image projection device installed on a vehicle or worn by a driver of the vehicle, the image projection method comprising: Receiving a selection specifying a specific user; Detecting the current location; Obtaining an image showing at least a part of the appearance of the specific user, the image being taken when the specific user has previously driven the specific user's vehicle at a location that is the same as or close to the current location; and Projecting the image onto the driver's seat of the vehicle.

16. The image projection method according to claim 15, further comprising: Obtaining information indicating the operation of the specific user's vehicle when the specific user has previously driven the specific user's vehicle at a location that is the same as or close to the current location; And Projecting an image of a vehicle operating in response to the information indicating the operation near the driver's vehicle.

17. The image projection method according to claim 15, further comprising: Obtaining information indicating the operation of the specific user's vehicle when the specific user has previously driven the specific user's vehicle at a location that is the same as or close to the current location; And At least partially superimposing an image of a vehicle operating in response to the information indicating the operation on the driver's vehicle.

18. The image projection method according to claim 15, further comprising: Obtaining information indicating the operation of the specific user's vehicle when the specific user has previously driven the specific user's vehicle at a location that is the same as or close to the current location; And Linking the operation of the driver's vehicle with the operation of the specific user's vehicle.

19. The image projection method according to any one of claims 15 to 18, further comprising: At least partially superimposing an image showing at least a part of the appearance of the specific user on the driver.

20. The image projection method according to any one of claims 15 to 18, wherein, The specific user includes a close relative of the driver.

Images