Information processing apparatus and information processing program product

Abstract

The present application relates to an information processing device and an information processing program product. A control section of the information processing device performs control so that, if there are a predetermined number or more of vehicles, display is started from a surrounding vehicle that is close to the host vehicle, and if there is a monitoring object that has an influence on the action of the host vehicle, the monitoring object is preferentially displayed on a display.

Description

Technical Field

[0001] This invention relates to an information processing device and an information processing program product. Background Technology

[0002] Japanese Patent Application Publication No. 2022-66952 discloses a display device that displays the vehicle and other vehicles traveling in a driving lane together with the driving lane on an information display screen. When other vehicles are located in an area far ahead of the vehicle's direction of travel, outside the display range of the screen, a mark is displayed at a fixed position on the driving lane displayed on the screen. This mark indicates that the other vehicles are located in an area far ahead of the vehicle's direction of travel, outside the display range of the screen. Summary of the Invention

[0003] When the number of targets that can be displayed is limited, if targets are displayed sequentially from those closest to the vehicle, targets that affect the vehicle's operation, such as those controlled by advanced safety functions or those that serve as warnings, may not be displayed.

[0004] The present invention was made in view of the above facts, and its object is to provide an information processing device that can display objects that are monitored and affect the operation of the vehicle, even when the number of objects that can be displayed is limited.

[0005] The information processing apparatus according to the first method includes: a display unit that displays a predetermined number of target objects together with a simulated image of the vehicle based on detection information of target objects around the vehicle; and a control unit that controls the display so that if there are more than the number of target objects, the display starts with the target objects closest to the vehicle, and if there is a monitored object that affects the operation of the vehicle, the monitored object is displayed preferentially on the display unit.

[0006] According to the first method, if there are more than a predetermined number of targets, the targets closest to the vehicle are displayed first. If there are monitored objects that affect the operation of the vehicle, the monitored objects are displayed first. Therefore, even if the number of targets that can be displayed is limited, the monitored objects that affect the operation of the vehicle can be displayed reliably.

[0007] In the information processing apparatus of the second method, in the information processing apparatus of the first method, the control unit controls the display unit to prioritize displaying at least one of the control object and warning object of the advanced safety function as the monitoring object.

[0008] According to the second method, even when the number of targets that can be displayed is limited, it is possible to reliably display surveillance objects that affect advanced security functions.

[0009] In the information processing device involved in the third method, in the information processing device involved in the first or second method, the control unit controls the display for each lane based on priority.

[0010] According to the third method, even when the number of visible targets is limited, it is possible to reliably display monitored objects affecting the vehicle in each lane.

[0011] The information processing program product involved in the fifth method includes an information processing program that enables a computer to perform the following processing: based on the detection information of target objects around the vehicle, displaying a predetermined number of the target objects together with an image of the vehicle simulating the vehicle; and if there are more than the number of the target objects, displaying the target objects closest to the vehicle first, and if there are monitored objects that affect the operation of the vehicle, displaying the monitored objects preferentially on the display unit.

[0012] According to the fifth method, if there are more than a predetermined number of targets, the targets closest to the vehicle are displayed first. If there are monitored objects that affect the operation of the vehicle, the monitored objects are displayed first. Therefore, even if the number of displayed targets is limited, the monitored objects that affect the operation of the vehicle can be displayed reliably.

[0013] As explained above, according to the present invention, an information processing device is provided that can display objects that affect the operation of the vehicle, even when the number of objects that can be displayed is limited. Attached Figure Description

[0014] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which the same reference numerals denote the same elements, and wherein:

[0015] Figure 1 It is a block diagram representing the hardware configuration of a vehicle.

[0016] Figure 2 This diagram illustrates an example of display control based on the control unit.

[0017] Figure 3 This is an example of how, when the number of surrounding vehicles that can be displayed is limited, surrounding vehicles that are being monitored are not shown.

[0018] Figure 4This diagram illustrates an example where, with two vehicles being displayed but restricted, the surrounding vehicles surrounded by dashed lines (which are the subjects of surveillance) are prioritized for display.

[0019] Figure 5 This is a flowchart illustrating the process of a specific procedure performed by the instrument ECU involved in this embodiment.

[0020] Figure 6 This diagram illustrates an example of a vehicle surrounding a monitored object, surrounded by a dashed line, when the European overtaking suppression function is applied to the monitored object. Detailed Implementation

[0021] The vehicle 10 described below is an example of this embodiment.

[0022] Figure 1 This is a block diagram representing the hardware configuration of vehicle 10. For example... Figure 1 As shown, vehicle 10 includes an Electronic Control Unit (ECU) 20. Vehicle 10 is an example of a "vehicle" in this invention, and the ECU 20 is an example of an "information processing device" in this invention.

[0023] The instrument cluster ECU 20 comprises a central processing unit (CPU) 21, a read-only memory (ROM) 22, a random access memory (RAM) 23, a storage device 24, an in-vehicle communication interface (I / F) 25, an input / output I / F 26, and a wireless communication I / F 27. These components are communicatively connected to each other via an internal bus 28.

[0024] CPU 21 is the central processing unit, which executes various programs or controls various parts. That is, CPU 21 reads programs from ROM 22 or storage device 24 and uses RAM 23 as the working area to execute the programs. CPU 21 performs control and various arithmetic operations on the above-mentioned components according to the programs recorded in ROM 22 or storage device 24.

[0025] ROM22 stores various programs and data. RAM23 serves as a temporary storage area for programs or data.

[0026] Storage device 24 is composed of storage media such as embedded Multi Media Card (eMMC) or Universal Flash Storage (UFS), and stores various programs and data. Information processing program 24A is stored in storage device 24. Information processing program 24A is used to cause CPU 21 to perform specific processes described later (see reference). Figure 2 (The program).

[0027] The vehicle communication I / F25 is an interface for connecting to other ECUs 30, such as the Advanced Driver-Assistance Systems (ADAS) ECU, using a communication standard based on the Controller Area Network (CAN) protocol. The vehicle communication I / F25 connects to the external bus 29. Furthermore, although not illustrated, the ECU is not limited to ECU 30, but rather multiple ECUs are provided depending on the various functions within the vehicle 10.

[0028] Input / output I / F26 is an interface for communicating with onboard equipment 40 mounted in vehicle 10.

[0029] The vehicle-mounted equipment 40 refers to various devices mounted on the vehicle 10. As an example of the vehicle-mounted equipment 40, the vehicle 10 includes a sensor group 42 and a display 44.

[0030] Sensor group 42 comprises various sensors and imaging mechanisms used to detect the state of vehicle 10 and its surroundings. The detection results of sensor group 42 are output to advanced safety functions such as ADAS-ECU and used within these functions. Specifically, sensor group 42 includes, for example, various sensors such as 3D-LiDAR sensors, millimeter-wave sensors, infrared sensors, turn signal sensors, throttle position sensors, vehicle speed sensors, wheel angle sensors, angular velocity sensors, Global Positioning System (GPS) sensors, illuminance sensors, gyroscope sensors, steering angle sensors, and acceleration sensors, as well as multiple cameras that capture images of the surroundings of vehicle 10. Sensor group 42 outputs the detection results from each sensor and the images captured by each camera to designated electronic control units such as instrument cluster ECU 20 and ECU 30.

[0031] Display 44 is an instrument display provided on the dashboard located in front of the driver's seat of vehicle 10 and used to display suggestions related to the functions of vehicle 10 and images related to the explanation of those functions. Display 44 is an example of a "display unit" in this invention.

[0032] The I / F27 wireless communication module is used for communicating with external devices. This wireless communication module uses communication standards such as 5G, LTE, Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0033] Furthermore, the CPU 21 of the instrument ECU 20 has an acquisition unit 21A and a control unit 21B as functional configurations. Each functional configuration is implemented by the CPU 21 reading and executing the information processing program 24A stored in the storage device 24.

[0034] The acquisition unit 21A acquires various types of information available in the vehicle 10. Specifically, the acquisition unit 21A acquires detection information, including at least one of the detection results from the various sensors constituting the sensor group 42 and the images captured by each camera, as various types of information.

[0035] The control unit 21B performs display control related to the display on the display 44. For example, as part of this display control, the control unit 21B, based on the detection information acquired by the acquisition unit 21A, displays the surrounding environment of the vehicle 10 as viewed from a virtual viewpoint and a simulated vehicle image 10A of the vehicle 10 (see reference). Figure 2 The virtual viewpoint (e.g., the position of the vehicle image 10A) is displayed on the display 44. Here, the "virtual viewpoint" is a viewpoint set in a three-dimensional virtual space with the position of the vehicle image 10A as the origin. This virtual viewpoint is defined by specified viewpoint coordinates and viewpoint angle (or orientation) in the virtual space. For example, this virtual viewpoint is a viewpoint in the virtual space that overlooks the vehicle 10 and its surroundings from specific viewpoint coordinates located above the vehicle image 10A and from a specific viewpoint angle. Furthermore, as a surrounding condition display, an image 32 showing surrounding vehicles or an image 34 showing lane lines and other lines representing the surrounding vehicles of the vehicle 10 is displayed. The control unit 21B detects surrounding vehicles and lane lines and other lines based on detection information including detection results from various sensors and images captured by a camera, and displays the surrounding condition on the display 44. The control unit 21B is an example of a "control unit" in this invention.

[0036] The control unit 21B displays surrounding vehicles and other targets on the display 44 as a surrounding situation display. If the number of targets that can be displayed is limited, targets located close to the vehicle may sometimes be displayed. In this case, if targets close to the vehicle are displayed, monitored targets that affect the vehicle's operation, such as vehicles controlled by advanced safety functions like Adaptive Cruise Control (ACC), may not be displayed. For example, if... Figure 3As shown, assuming a real-world scenario with three lanes, three vehicles are traveling in the left lane, three vehicles (including the vehicle in front of it) are traveling in the middle lane, and two vehicles are traveling in the right lane. Furthermore, the display limit for each lane is two vehicles. In this example, Figure 3 Vehicles surrounded by dashed lines sometimes become monitoring targets based on controls such as ACC. If the display is prioritized based on distance, starting with the nearest surrounding vehicles, then vehicles that are being monitored will not be displayed. Figure 3 The surrounding vehicles are enclosed by dashed lines.

[0037] Therefore, in this embodiment, the control unit 21B controls the display so that if there are more than a predetermined number of surrounding vehicles, the display starts with the image 32 of the surrounding vehicles closest to the vehicle; and if there is a monitored object that affects the vehicle's movement, the monitored object is displayed on the display 44 first. Specifically, in Figure 3 In real-world scenarios, such as Figure 4 As shown, even if there are only two vehicles on display, the surrounding vehicles that are monitored and surrounded by dashed lines are displayed first, thus preventing the monitoring objects from not being displayed.

[0038] In addition, as an example of a monitored object that affects the operation of the vehicle, at least one of the control object and warning object to which advanced safety functions are applied.

[0039] Next, the specific processing performed by the instrument ECU20 constructed as described above will be explained. Figure 5 This is a flowchart illustrating the process of a specific process performed by the instrument ECU 20 according to this embodiment. The CPU 21 reads the information processing program 24A from the storage device 24, loads it into the RAM 23, and executes it to perform the specific process. As an example, the specific process is automatically executed repeatedly at predetermined time intervals (e.g., 100ms).

[0040] In step 100, CPU 21 acquires the detection information of sensor group 42, and then proceeds to step 102. That is, acquisition unit 21A acquires detection information including detection results based on the various sensors constituting sensor group 42 and at least one of the images captured by each camera.

[0041] In step 102, CPU 21 detects surrounding vehicles and then proceeds to step 104. That is, control unit 21B detects surrounding vehicles based on detection information including at least one of the detection results from various sensors and images captured by a camera.

[0042] In step 104, CPU 21 determines whether there are lanes with more than 3 surrounding vehicles. If the determination is negative, the process proceeds to step 106; if the determination is positive, the process proceeds to step 108. Additionally, as an example, the display shows a limit of 2 vehicles per lane.

[0043] In step 106, CPU 21 displays all surrounding vehicles on display 44 and ends a series of processes. That is, control unit 21B, based on the detection information acquired by acquisition unit 21A, such as... Figure 2 As shown, the surrounding conditions display representing the surrounding conditions of vehicle 10 when viewed from a virtual viewpoint and the vehicle image 10A of the simulated vehicle 10 are displayed on the display 44, and the surrounding vehicle images 32 corresponding to all surrounding vehicles detected as surrounding conditions are displayed on the display 44.

[0044] In step 108, CPU 21 determines whether a monitored vehicle exists. This determination checks for surrounding vehicles that could affect the vehicle's actions. For example, it checks whether a monitored vehicle controlled by ACC exists. If the determination is negative, the process proceeds to step 110; if the determination is positive, the process proceeds to step 112.

[0045] In step 110, the CPU 21 displays the number of surrounding vehicles (for example, two per lane) in order of proximity to the vehicle on the display 44 and then completes a series of processes. Specifically, the control unit 21B, based on the detection information acquired by the acquisition unit 21A, displays a surrounding condition display showing the surrounding conditions of the vehicle 10 from a virtual viewpoint and a simulated image 10A of the vehicle 10 on the display 44. Then, when displaying the surrounding conditions, the control unit 21B calculates priorities for each lane and displays images 32 of surrounding vehicles corresponding to the display limit number on the display 44. This display limit number of surrounding vehicles is determined from the surrounding vehicles detected as part of the surrounding conditions, in order of proximity to the vehicle in each lane.

[0046] On the other hand, in step 112, the CPU 21 prioritizes vehicles that are monitored, displays the display limit number of vehicles (for example, 2 vehicles per lane) on the display 44, and ends a series of processes. That is, the control unit 21B, based on the detection information acquired by the acquisition unit 21A, displays the surrounding conditions display representing the surrounding conditions of vehicle 10 from the virtual viewpoint and the simulated vehicle image 10A of the vehicle itself on the display 44. Then, when displaying the surrounding conditions, the control unit 21B calculates the priority for each lane and displays the surrounding vehicle images 32 corresponding to the display limit number of surrounding vehicles on the display 44. The display limit number of surrounding vehicles is determined from the surrounding vehicles detected as surrounding conditions, prioritizing monitored vehicles and in order of proximity to the vehicle in each lane. For example, in Figure 4 In the example, in the lanes to the left and right of this vehicle, images 32 of the surrounding vehicles corresponding to the two surrounding vehicles are displayed in order of proximity to this vehicle. In the lane of this vehicle, images 32 of the surrounding vehicles corresponding to the closest surrounding vehicles and the surrounding vehicles surrounded by dashed lines that are the objects of monitoring are displayed. In this way, if there are surrounding vehicles that are the objects of monitoring, even if the number that can be displayed is limited, the images 32 of the surrounding vehicles corresponding to the objects of monitoring are displayed first, thus preventing the surrounding vehicles that are the objects of monitoring from not being displayed.

[0047] Furthermore, in the above embodiments, the control object of ACC was cited as an example of a monitoring object that affects the operation of the vehicle, but it is not limited to this. Control objects of other advanced safety functions, and monitoring objects other than advanced safety functions, can also be applied. For example, as another example of an advanced safety function, the control object of the European overtaking suppression function (a function that suppresses overtaking from the lane side opposite to the overtaking lane) can be applied. For example, such as... Figure 6 As shown, the actual scenario assumes that there are 3 vehicles traveling in each of the 3 lanes, with the rightmost lane being the overtaking lane and the others being driving lanes. Furthermore, the display limit for each lane is 2 vehicles. In this example, in... Figure 6 When surrounding vehicles enclosed by dashed lines become monitoring targets for overtaking suppression control, if the display prioritizes nearby vehicles, the surrounding vehicles enclosed by dashed lines that are being monitored will not be displayed. However, as in this embodiment, by prioritizing the display of monitoring targets (…),… Figure 6 The display (distance + control status) prioritizes the display of surrounding vehicle images 32, which are the monitored objects and are surrounded by dashed lines, even if there are only 2 units for display. This prevents the monitored objects from being undisplayed.

[0048] Furthermore, in the above embodiment, priority is calculated for each lane and the display of surrounding vehicle images 32 is controlled according to the lane, but it is not limited to this. For example, priority can also be calculated for the entire monitoring range instead of for each lane to control the display of surrounding vehicle images 32.

[0049] Furthermore, in the above embodiments, examples of surrounding vehicles were described as targets around the vehicle, but this is not a limitation and other targets besides vehicles can also be applied.

[0050] Furthermore, the specific processing performed by the software (program) read and executed by the CPU 21 in the above embodiment can also be executed by various processors other than the CPU. Examples of processors in this case include field-programmable gate arrays (FPGAs) and programmable logic devices (PLDs) whose circuit configuration can be changed after manufacturing, as well as application-specific integrated circuits (ASICs) and other processors with circuit configurations specifically designed for executing specific processing, i.e., dedicated circuits. Furthermore, the specific processing can be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit composed of circuit elements such as semiconductor elements.

[0051] Furthermore, a specific process can also be a combination of software-based processing and hardware-based processing.

[0052] Furthermore, while the above embodiment describes the information processing program 24A being pre-stored (installed) in the storage device 24, it is not a limitation. The information processing program 24A may also be provided on a recording medium such as a compact disk read-only memory (CD-ROM), a digital versatile disk read-only memory (DVD-ROM), or a universal serial bus (USB) memory. Additionally, the information processing program 24A may be downloaded from an external device via a network. Furthermore, the technology of this invention can also be applied to programs and program products.

[0053] Moreover, the present invention is not limited to the above, and can of course be implemented in various modifications without departing from its spirit.

Claims

1. An information processing device, characterized in that, have: The display unit, based on the detection information of targets around the vehicle, displays a predetermined number of the targets along with a simulated image of the vehicle; and The control unit controls the display to prioritize displaying objects that, if there are more than the number of the target objects, start displaying the target objects closest to the vehicle; and if there are monitored objects that affect the operation of the vehicle, prioritize displaying the monitored objects on the display unit.

2. The information processing device according to claim 1, characterized in that, The control unit performs control such that at least one of the control objects and warning objects of the advanced safety functions is prioritized for display on the display unit as the monitoring object.

3. The information processing device according to claim 1, characterized in that, The control unit controls the display for each lane based on priority.

4. An information processing program product, comprising an information processing program, characterized in that, This information processing program is used to cause the computer to perform the following processes: Based on the detection information of targets around the vehicle, a predetermined number of the targets are displayed together with an image simulating the vehicle; and If there are more than the specified number of target objects, the display will start with the target object closest to the vehicle. If there is a monitored object that affects the operation of the vehicle, the monitored object will be displayed on the display unit first.

Images