Information processing device

Abstract

To enable an occupant to grasp a positional relation between a vehicle and an object through an on-vehicle display.SOLUTION: An information processing device comprises a control part that when a sensing part senses an object existing in a running direction of a vehicle running on a road and when a positional relation between the vehicle and the object is in a predetermined state, displays an image of the object showing the object, on a position where the positional relation is reflected on a display part in the vehicle on which a vehicle image showing the vehicle is displayed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device. [Background technology]

[0002] Patent Document 1 discloses a technique that enables an occupant to clearly see an object that exists outside the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2020 / 031912 publication Summary of the Invention [Problem to be solved by the invention]

[0004] The technology of Patent Document 1 displays object-related information indicating the object on a HUD (Head-Up Display), but there is room for improvement in the content displayed on the in-vehicle display to allow occupants to recognize objects outside the vehicle.

[0005] Therefore, an object of the present disclosure is to provide an information processing device that allows an occupant to understand the positional relationship between the vehicle and an object through an in-vehicle display. [Means for solving the problem]

[0006] The information processing device according to claim 1 includes a control unit that, when a detection unit detects an object present in the traveling direction of a vehicle traveling on a road and the positional relationship between the vehicle and the object is in a predetermined state, displays an object image showing the object at a position that reflects the positional relationship on a display unit inside the vehicle that displays a vehicle image showing the vehicle. The detection unit is capable of detecting the distance and relative speed between the vehicle and the object, and the control unit determines that the vehicle is in the predetermined state when the relative collision time between the vehicle and the object, calculated based on the distance and the relative speed detected by the detection unit, is less than or equal to a predetermined value, and causes the display unit to display a road image showing the road based on the detection information detected by the detection unit, and if the actual position of the object is outside the road, causes the display unit to display the object image at a position on the road image that indicates the outside of the road, and causes the display unit to display the object image in a circular frame with an icon selected from a plurality of icons indicating the type of the object based on the detection information.

[0007] In the information processing device according to claim 1, when an object is detected by the detection unit and the positional relationship between the vehicle and the object is in a predetermined state, the control unit causes the display unit that displays the vehicle image to display the object image at a position that reflects the positional relationship. As a result, the information processing device causes the display unit to display the object image at a position that reflects the positional relationship between the vehicle and the object, allowing the occupant to grasp the positional relationship between the vehicle and the object through the in-vehicle display. The information processing device according to claim 2 is in claim 1, wherein the control unit controls updating of registration, deletion or addition of the object that is the target detected by the detection unit based on input from an occupant of the vehicle.

[0008] Other aspects of The information processing device before The control unit displays the vehicle image that reflects the actual state of the vehicle.

[0009] Other aspects of In the information processing device, the control unit displays a vehicle image that reflects the actual state of the vehicle, thereby allowing the occupant of the information processing device to understand the actual state of the vehicle by looking at the display unit.

[0010] Other aspects of The information processing device before When the brakes of the vehicle are applied, the control unit turns on brake lights in the vehicle image.

[0011] Other aspects of In the information processing device, the control unit turns on the brake lights in the vehicle image when the vehicle brakes are applied, thereby allowing the occupant of the information processing device to know that the vehicle brakes are applied by looking at the display unit.

[0012] Other aspects of The information processing device before When a turn signal of the vehicle is activated, the control unit blinks the turn signal lamp in the vehicle image.

[0013] Other aspects ofIn the information processing device, when the turn signal of the vehicle is activated, the control unit causes the turn signal lamp in the vehicle image to flash, thereby allowing the occupant of the information processing device to know that the turn signal of the vehicle is activated by looking at the display unit.

[0014] Other aspects of The information processing device before The control unit causes the display unit to display a road image showing the road, and causes the vehicle image and the object image to be displayed in positions on the road image that reflect the positional relationship.

[0015] Other aspects of In the information processing device, the control unit displays the vehicle image and the object image in a road image at a position that reflects the positional relationship between the vehicle and the object, thereby allowing the occupant of the information processing device to understand, through the in-vehicle display, the positional relationship between the vehicle and the object on the road on which the vehicle is traveling. [Effects of the Invention]

[0016] As described above, the information processing device according to the present disclosure allows the occupant to understand the positional relationship between the vehicle and an object through the in-vehicle display. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of a vehicle. [Figure 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of a vehicle. [Figure 3] 10 is a flowchart showing the flow of a control process. [Figure 4] FIG. 10 is a first explanatory diagram showing an example of a display displayed on a display unit while the vehicle is traveling. [Figure 5] FIG. 2 is a second explanatory diagram showing an example of a display displayed on the display unit while the vehicle is traveling. [Figure 6] FIG. 10 is a third explanatory diagram showing an example of a display displayed on the display unit while the vehicle is traveling. DETAILED DESCRIPTION OF THE INVENTION

[0018] A vehicle 10 according to this embodiment will be described. The vehicle 10 may be any of an engine vehicle, a hybrid vehicle, and an electric vehicle, but in this embodiment, as an example, the vehicle 10 is an engine vehicle.

[0019] Fig. 1 is a block diagram showing the hardware configuration of a vehicle 10. As shown in Fig. 1, the vehicle 10 includes an on-board device 20, an ECU (Electronic Control Unit) 30, and on-board equipment 40. The on-board device 20 is an example of an "information processing device."

[0020] The vehicle-mounted device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage unit 24, an in-vehicle communication I / F (Interface) 25, an input / output I / F 26, and a wireless communication I / F 27. The CPU 21, the ROM 22, the RAM 23, the storage unit 24, the in-vehicle communication I / F 25, the input / output I / F 26, and the wireless communication I / F 27 are connected to each other via an internal bus 28 so as to be able to communicate with each other.

[0021] The CPU 21 is a central processing unit that executes various programs and controls each part. That is, the CPU 21 reads programs from the ROM 22 or the storage unit 24 and executes the programs using the RAM 23 as a work area. The CPU 21 controls each of the above components and performs various arithmetic processing in accordance with the programs recorded in the ROM 22 or the storage unit 24.

[0022] The ROM 22 stores various programs and various data. The RAM 23 serves as a working area for temporarily storing programs or data.

[0023] The storage unit 24 is configured with a storage device such as an eMMC (embedded Multi Media Card) or a UFS (Universal Flash Storage), and stores various programs and various data. The storage unit 24 stores an information processing program 24A for causing the CPU 21 to execute control processing, which will be described later.

[0024] The in-vehicle communication I / F 25 is an interface for connecting to the ECU 30. This interface uses a communication standard based on the CAN protocol. The in-vehicle communication I / F 25 is connected to an external bus 29. Although not shown in the figure, a plurality of ECUs 30 are provided for each function of the vehicle 10. For example, the ECU 30 includes a turn signal ECU that controls the blinking or extinguishing of turn signal lamps that indicate the direction when changing lanes in response to the operation of a turn signal switch by the occupant (driver), and a brake ECU that controls the braking force applied to the vehicle 10 and the turning on or off of brake lamps in response to the operation of the brake pedal by the occupant.

[0025] The input / output I / F 26 is an interface for communicating with an in-vehicle device 40 mounted on the vehicle 10 .

[0026] The on-vehicle devices 40 are various devices mounted on the vehicle 10. The vehicle 10 includes, as examples of the on-vehicle devices 40, a detection unit 40A and a display unit 40B.

[0027] The detection unit 40A is a detection device capable of detecting the state of the vehicle 10, including the inside and outside of the vehicle. The detection unit 40A includes, for example, a camera, a vehicle speed sensor, a millimeter wave sensor, and a GPS (Global Positioning System) device.

[0028] The camera serving as the detection unit 40A is, for example, an imaging device that captures images using an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera is provided at the front of the vehicle 10 and captures images ahead of the vehicle. The images captured by the camera are used, for example, to recognize the distance between the vehicle and a preceding vehicle traveling ahead of the vehicle, the lane, and objects present ahead of the vehicle. The images captured by the camera are stored in the storage unit 24. The camera may be configured as an imaging device for other purposes, such as a drive recorder and an ADAS (Advanced Driver Assistance System). The camera may also be connected to the in-vehicle device 20 via the ECU 30 (for example, a camera ECU).

[0029] The vehicle speed sensor serving as the detection unit 40A is a sensor for detecting the vehicle speed of the vehicle 10, and is provided on, for example, a wheel.

[0030] The millimeter wave sensor serving as the detection unit 40A is a sensor for detecting objects present in front of the vehicle, i.e., in the traveling direction of the vehicle 10, and is provided at least at the front of the vehicle 10. The millimeter wave sensor is used to detect objects by transmitting a transmission wave ahead of the vehicle 10 and receiving a reflected wave from an object ahead. Here, the objects are, for example, objects that may pose a danger to the traveling of the vehicle 10, such as pedestrians, bicycles, walls, and oncoming vehicles. Note that the objects that serve as targets may be registered in the vehicle 10 in advance, or may be registered by input by the occupant. Furthermore, it is desirable that the objects that serve as targets can be updated, such as by deletion and addition, automatically by the in-vehicle device 20 or based on input by the occupant.

[0031] The GPS device serving as the detection unit 40A is a device that detects the current location of the vehicle 10. The GPS device includes an antenna (not shown) that receives signals from GPS satellites. The GPS device may be connected to the vehicle-mounted device 20 via a car navigation system that is connected to the ECU 30 (for example, a multimedia ECU).

[0032] The display unit 40B is an in-vehicle display for displaying an operation suggestion related to a function of the vehicle 10, an image related to an explanation of the function, etc. The display unit 40B is provided, for example, on a meter panel.

[0033] The wireless communication I / F 27 is a wireless communication module for communicating with the outside world, and uses communication standards such as 5G, LTE, and Wi-Fi (registered trademark).

[0034] Next, a description will be given of the functional configuration of the vehicle 10. Fig. 2 is a block diagram showing an example of the functional configuration of the vehicle 10.

[0035] 2, the CPU 21 of the vehicle-mounted device 20 has, as functional components, an acquisition unit 21A and a control unit 21B. Each functional component is realized by the CPU 21 reading and executing an information processing program 24A stored in the storage unit 24.

[0036] The acquisition unit 21A periodically acquires detection information detected by the detection unit 40A while the vehicle 10 is traveling. The detection information includes, for example, an image captured by a camera and detection results detected by a vehicle speed sensor, a millimeter wave sensor, and a GPS device. The acquisition unit 21A also periodically acquires vehicle information related to the vehicle 10 from the ECU 30 while the vehicle 10 is traveling. The vehicle information includes, for example, the operation state of a turn signal switch, the illumination state of a turn signal lamp, the operation state of a brake pedal, and the illumination state of a brake lamp.

[0037] The control unit 21B controls the display content of the display unit 40B based on the detection information and vehicle information acquired by the acquisition unit 21A. For example, while the vehicle 10 is traveling, the control unit 21B can display a vehicle image 50 showing the vehicle 10, a road image 52 showing the road on which the vehicle 10 is traveling, and an object image 54 showing an object on the display unit 40B based on the detection information and vehicle information (see FIGS. 4, 5, and 6).

[0038] 3 is a flowchart showing the flow of a control process in which the vehicle-mounted device 20 controls the display content of the display unit 40B. The control process is performed by the CPU 21 reading the information processing program 24A from the storage unit 24, expanding it into the RAM 23, and executing it. As an example, the control process shown in FIG. 3 is performed periodically while the vehicle 10 is traveling.

[0039] 3, the CPU 21 acquires the detection information detected by the detection unit 40A, and then the CPU 21 proceeds to step S11.

[0040] In step S11, the CPU 21 acquires vehicle information from the ECU 30. Then, the CPU 21 proceeds to step S12.

[0041] In step S12, the CPU 21 displays the vehicle image 50 and the road image 52 on the display unit 40B based on the detection information acquired in step S10 and the vehicle information acquired in step S11. Specifically, the CPU 21 displays the vehicle image 50 in a position on the road image 52 that reflects the actual position of the vehicle 10. For example, when the vehicle 10 is traveling on the left side of the road, the CPU 21 displays the vehicle image 50 at the left edge of the road image 52. Furthermore, the CPU 21 displays the vehicle image 50 that reflects the actual state of the vehicle 10. For example, when the brakes of the vehicle 10 are applied, the CPU 21 turns on the brake lights in the vehicle image 50. Then, the CPU 21 proceeds to step S13.

[0042] In step S13, the CPU 21 determines whether the display condition is met based on the detection information acquired in step S10. If the CPU 21 determines that the display condition is met (step S13: YES), the CPU 21 proceeds to step S14. On the other hand, if the CPU 21 does not determine that the display condition is met (step S13: NO), the CPU 21 terminates the control process. In this embodiment, the CPU 21 determines that the display condition is met when the detection unit 40A detects an object present in the traveling direction of the vehicle 10 and the positional relationship between the vehicle 10 and the object is in a predetermined state. In this embodiment, the positional relationship between the vehicle 10 and the object is in a predetermined state when the relative time to collision (TTC), which is calculated by dividing the relative speed between the vehicle 10 and the object, is equal to or less than a predetermined value. The CPU 21 calculates the TTC between the vehicle 10 and the object using the distance and relative speed between the vehicle 10 and the object as detection results of the millimeter wave sensor, which is the detection unit 40A.

[0043] In step S14, CPU 21 displays object image 54 on display unit 40B based on the detection information acquired in step S10. Specifically, CPU 21 displays object image 54 at a position that reflects the positional relationship between vehicle 10 and the object in road image 52. For example, if the actual position of the object is outside the road, CPU 21 displays object image 54 at a position that indicates outside the road in road image 52. Then, CPU 21 ends the control process.

[0044] Next, a display example displayed on the display unit 40B as a result of the control process shown in FIG. 3 being performed by the vehicle-mounted device 20 will be described.

[0045] Fig. 4 is a first explanatory diagram showing an example of a display displayed on the display unit 40B while the vehicle 10 is traveling. Specifically, Fig. 4 shows an example of a display on the display unit 40B when the vehicle 10 is traveling on a straight road.

[0046] 4 displays a vehicle image 50 showing a vehicle 10 traveling in the direction of arrow S, and a road image 52 showing the straight road along which the vehicle 10 is traveling. As an example, the vehicle image 50 is made up of a substantially rectangular body section 50A and substantially square lamp sections 50B and 50C. In the road image 52, the area between two straight lines is inside the road, and the area outside the two straight lines is outside the road.

[0047] Fig. 5 is a second explanatory diagram showing a display example displayed on display unit 40B while vehicle 10 is traveling. Specifically, Fig. 5 shows a display example on display unit 40B after a predetermined time has elapsed since the display example shown in Fig. 4 was displayed.

[0048] Here, when the turn signals of the vehicle 10 are activated, the CPU 21 causes the turn signal lamps to flash in the vehicle image 50. As an example, in this embodiment, when the frame of the lamp unit 50B or the lamp unit 50C of the vehicle image 50 is displayed in white, it indicates that the lamp is off, and when the frame is displayed in black, it indicates that the lamp is on. Therefore, the CPU 21 notifies the blinking of the turn signal lamps in the vehicle image 50 by changing the color of the frame of the lamp unit 50B or the lamp unit 50C to white or black every predetermined time. Note that the display unit 40B shown in FIG. 5 shows a case where the turn signal on the right side of the vehicle is activated, and the frame of the lamp unit 50C changes from black to white after the predetermined time has elapsed.

[0049] Fig. 6 is a third explanatory diagram showing a display example displayed on display unit 40B while vehicle 10 is traveling. Specifically, Fig. 6 shows a display example on display unit 40B after a predetermined time has elapsed since the display example shown in Fig. 5 was displayed.

[0050] Here, the CPU 21 changes the vehicle image 50 and the road image 52 in accordance with changes in the actual position of the vehicle 10. As a result, in Fig. 6, the position of the vehicle image 50 changes from Fig. 5 to the upper side in the figure as the vehicle 10 travels straight along a straight road. Furthermore, although not shown, when the vehicle 10 enters a curved road from a straight road, the CPU 21 changes the two straight lines in the road image 52 into two curved lines.

[0051] 6 displays an object image 54 in addition to the vehicle image 50 and the road image 52, as a result of the CPU 21 determining that the display conditions are met. The object image 54 is displayed on the forward side of the vehicle 10 in the traveling direction, outside the road to the right of the vehicle image 50 in the drawing, reflecting the actual positional relationship between the vehicle 10 and the object.

[0052] As an example, the object image 54 is configured with a circular frame 54A and an icon 54B displayed within the frame 54A and indicating the type of object. In this embodiment, a plurality of types of icons 54B are provided, and an icon 54B selected by the CPU 21 based on the acquired detection information is displayed within the frame 54A. In FIG. 6, an icon 54B indicating a pedestrian is displayed.

[0053] Here, when the brakes of the vehicle 10 are applied, the CPU 21 turns on the brake lights in the vehicle image 50. In this embodiment, the CPU 21 notifies the driver that the brake lights are on by changing the color of the frames of the lamp units 50B and 50C to black in the display unit 40B shown in FIG. 6, therefore, by displaying the frames of the lamp units 50B and 50C in black, it is indicated that the brakes of the vehicle 10 are applied.

[0054] As described above, in the vehicle-mounted device 20, when an object is detected by the detection unit 40A and the positional relationship between the vehicle 10 and the object is in a predetermined state, the CPU 21, as a function of the control unit 21B, causes the display unit 40B on which the vehicle image 50 is displayed to display the object image 54 at a position that reflects the positional relationship between the vehicle 10 and the object. As a result, the vehicle-mounted device 20 causes the display unit 40B to display the object image 54 at a position that reflects the positional relationship between the vehicle 10 and the object, allowing the occupant to grasp the positional relationship between the vehicle 10 and the object through the in-vehicle display.

[0055] Furthermore, in the vehicle-mounted device 20, the CPU 21, as a function of the control unit 21B, displays a vehicle image 50 that reflects the actual state of the vehicle 10. This allows the occupants of the vehicle 10 to understand the actual state of the vehicle 10 by looking at the display unit 40B.

[0056] Furthermore, in the vehicle-mounted device 20, the CPU 21, as a function of the control unit 21B, turns on the brake lights in the vehicle image 50 when the brakes of the vehicle 10 are applied. This allows the occupants of the vehicle-mounted device 20 to know that the brakes of the vehicle 10 are applied by looking at the display unit 40B.

[0057] Furthermore, in the vehicle-mounted device 20, the CPU 21, as a function of the control unit 21B, causes the turn signal lamps to flash in the vehicle image 50 when the turn signals of the vehicle 10 are activated. This allows the occupants of the vehicle-mounted device 20 to know that the turn signals of the vehicle 10 are activated by looking at the display unit 40B.

[0058] Furthermore, in the vehicle-mounted device 20, the CPU 21, as a function of the control unit 21B, displays the vehicle image 50 and the object image 54 at positions that reflect the positional relationship between the vehicle 10 and the object on the road image 52. This allows the vehicle-mounted device 20 to allow the occupant to understand, through the in-vehicle display, the positional relationship between the vehicle 10 and the object on the road on which the vehicle 10 is traveling.

[0059] (others) In the above embodiment, the in-vehicle device 20 is an example of an information processing device, but the present invention is not limited thereto. An external device such as a server not mounted on the vehicle 10 may also be an example of an information processing device, or a combination of the in-vehicle device 20 and the external device may also be an example of an information processing device. For example, when the combination of the in-vehicle device 20 and the external device is an example of an information processing device, at least some of the functional configurations of the CPU 21 of the in-vehicle device 20 shown in Fig. 2 may be performed by a CPU of the external device. In this case, the control process shown in Fig. 3 is executed by one processor of the CPU 21 of the in-vehicle device 20 or the CPU of the external device, or by a combination of multiple processors of the CPU 21 of the in-vehicle device 20 and the CPU of the external device.

[0060] In the above embodiment, the detection unit 40A includes a camera, a vehicle speed sensor, a millimeter wave sensor, and a GPS device, but the configuration of the detection unit 40A is not limited to this. For example, the detection unit 40A includes a millimeter wave sensor as a detection device that detects an object present in the traveling direction of the vehicle 10. However, instead of or in addition to the millimeter wave sensor, the detection unit 40A may include a LIDAR (Laser Imaging Detection and Ranging) to detect an object.

[0061] In the above embodiment, the display unit 40B is provided on the meter panel, but the location of the display unit 40B inside the vehicle is not limited to this. For example, the display unit 40B may be provided on the instrument panel or configured as a HUD that projects an image onto the display surface of the front windshield.

[0062] In the above embodiment, the display unit 40B is a display device provided in the vehicle 10, but this is not limiting. A portable terminal such as a smartphone of a passenger may be installed inside the vehicle and used as the display unit 40B.

[0063] In the above embodiment, the case where the positional relationship between the vehicle 10 and the object is in a predetermined state is defined as the case where the TTC between the vehicle 10 and the object is equal to or less than a predetermined value, but this is not limiting. For example, other cases, such as the case where the distance between the vehicle 10 and the object is equal to or less than a predetermined value, may also be defined as the case where the positional relationship between the vehicle 10 and the object is in a predetermined state.

[0064] In the above embodiment, a case where the brakes or blinkers of the vehicle 10 are activated is shown as an example of displaying the vehicle image 50 that reflects the actual state of the vehicle 10, but the actual state of the vehicle 10 reflected in the vehicle image 50 is not limited to this. For example, if a door, hatchback, or the like of the vehicle 10 is open, the CPU 21 may show the door, hatchback, or the like in the open state in the vehicle image 50, or if there is a scratch, dirt, or the like on the vehicle 10, the CPU 21 may express the scratch, dirt, or the like in the vehicle image 50.

[0065] In the above embodiment, the control processing executed by the CPU 21 after reading the software (program) may be executed by various processors other than a CPU. Examples of such processors include programmable logic devices (PLDs) such as field-programmable gate arrays (FPGAs), whose circuit configuration can be changed after fabrication, and application-specific integrated circuits (ASICs), which are dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing. The control processing may be executed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0066] In the above embodiment, the information processing program 24A is pre-stored (installed) in the storage unit 24, but the present invention is not limited to this. The information processing program 24A may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 24A may also be downloaded from an external device via a network. [Explanation of symbols]

[0067] 10 vehicles 20 In-vehicle device (information processing device) 21B Control section 40A detection unit 40B Display section 50 vehicle images 52 Road Images 54 Object images

Claims

1. a control unit that, when a detection unit detects an object present in the traveling direction of a vehicle traveling on a road and the positional relationship between the vehicle and the object is in a predetermined state, displays an object image of the object at a position that reflects the positional relationship on a display unit inside the vehicle that displays a vehicle image of the vehicle; the detection unit is capable of detecting a distance and a relative speed between the vehicle and the object, The control unit determining that the vehicle is in the predetermined state when a relative collision time between the vehicle and the object, which is calculated based on the distance and the relative speed detected by the detection unit, is equal to or less than a predetermined value; displaying a road image showing the road on the display unit based on the detection information detected by the detection unit, and when the actual position of the object is outside the road, displaying the object image at a position on the road image showing the outside of the road; displaying the object image in which an icon selected from a plurality of icons indicating the type of the object is displayed within a circular frame based on the detection information; Information processing device.

2. The control unit controls updating of registration, deletion, or addition of the target object detected by the detection unit based on input from an occupant of the vehicle. The information processing device according to claim 1 .

Images