Information processing apparatus

Abstract

An information processing apparatus includes a control section that detects a moving average of a host vehicle based on an input value with a travel lane as a reference, and uses the detected moving average for display control of an image of the host vehicle and an image representing the travel lane, the control section performing processing in which, when the host vehicle makes a lane change, past input values of the moving average before the lane change are coordinate-converted with a lane of a lane change target as a reference, and a moving average after the change is calculated using the converted input values.

Description

Technical Field

[0001] This invention relates to an information processing device. Background Technology

[0002] Japanese Patent 7048398 discloses a vehicle control device capable of providing reassurance to occupants through automated driving. In the technology disclosed in Japanese Patent 7048398, images showing the vehicle and images showing the surrounding environment of the vehicle are displayed on a display unit. Summary of the Invention

[0003] For objects displayed on the display unit, the lateral coordinates of the objects are adjusted using the vehicle's driving lane as a reference, and the vehicle's moving average is used. However, in the prior art, display control when the vehicle changes lanes is not taken into account. Therefore, if the moving average is reset when the vehicle changes lanes, it will cause the value to jump, resulting in incorrect markings or an uncomfortable display.

[0004] The purpose of this invention is to provide an information processing device that can reduce the discomfort of display control during lane changes.

[0005] The information processing device described in technical solution 1 includes a control unit. The control unit detects the moving average of the vehicle based on an input value with the driving lane as a reference, and uses the detected moving average for display control of the image of the vehicle and the image representing the driving lane. The control unit performs the following processing: when the vehicle changes lanes, it transforms the past input value of the moving average before the lane change with the target lane as a reference for coordinate transformation, and uses the transformed input value to calculate the moving average after the change.

[0006] The information processing device described in technical solution 1 is configured to calculate a moving average based on the input value after coordinate transformation when changing lanes. This reduces the discomfort associated with display and control during lane changes.

[0007] Regarding the information processing apparatus described in technical solution 2, in the information processing apparatus described in technical solution 1, the control unit performs the following processing: using the input value to determine whether a lane change has been performed; if a lane change has been determined, the vehicle's driving lane is changed to the changed lane; the vehicle's position is calculated using a moving average after coordinate transformation; the position of the changed driving lane relative to the vehicle's position is calculated; and the calculated position of the changed driving lane is used in the display control of the display unit to adjust the lateral position of the changed driving lane and display an image representing the driving lane. According to the information processing apparatus described in technical solution 2, the discomfort caused by lane switching during lane changes can be reduced.

[0008] Regarding the information processing device described in technical solution 3, in the information processing device described in technical solution 2, the control unit performs the following processing: using the relative distance between the position of the current vehicle and the detected surrounding vehicles, calculated based on the moving average of the coordinate transformation, to control the display of images representing the surrounding vehicles of the current vehicle. According to the information processing device described in technical solution 3, the discomfort caused by the display of surrounding vehicles during lane changes can be reduced.

[0009] According to the technology of the present invention, the discomfort of display control during lane changes can be reduced. Attached Figure Description

[0010] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which the same reference numerals denote the same elements, and wherein:

[0011] Figure 1 It is a block diagram representing the hardware structure of a vehicle.

[0012] Figure 2 This is a display example used to illustrate the surrounding conditions of this vehicle.

[0013] Figure 3 This is a graph that shows the detected input value being switched due to lane change.

[0014] Figure 4 This is an example of the input values ​​for the moving average filter and the calculation of the moving average during lane changes.

[0015] Figure 5 This is an example of the input values ​​of the moving average filter during lane change in this embodiment and the calculation using coordinate transformation in the moving average.

[0016] Figure 6It is a flowchart representing the process of a specific procedure performed by the instrument ECU. Detailed Implementation

[0017] The vehicle 10 described below is an example of this embodiment.

[0018] Figure 1 This is a block diagram representing the hardware structure of vehicle 10. For example... Figure 1 As shown, vehicle 10 includes an Electronic Control Unit (ECU) 20. Vehicle 10 is an example of a "vehicle" in this invention, and the ECU 20 is an example of an "information processing device" in this invention.

[0019] The instrument cluster ECU 20 comprises a central processing unit (CPU) 21, a read-only memory (ROM) 22, a random access memory (RAM) 23, a storage device 24, an in-vehicle communication interface (I / F) 25, an input / output I / F 26, and a wireless communication I / F 27. These components are communicatively connected to each other via an internal bus 28.

[0020] CPU 21 is the central processing unit, which executes various programs or controls various components. Specifically, CPU 21 reads programs from ROM 22 or storage device 24 and uses RAM 23 as its working area to execute the programs. CPU 21 performs control and various arithmetic operations on the aforementioned components according to the programs recorded in ROM 22 or storage device 24.

[0021] ROM22 stores various programs and data. RAM23 serves as a temporary storage area for programs or data.

[0022] Storage device 24 is composed of storage media such as embedded Multi Media Card (eMMC) or Universal Flash Storage (UFS), and stores various programs and data. Information processing program 24A is stored in storage device 24. Information processing program 24A is used to cause CPU 21 to perform specific processes described later (see reference). Figure 6 (The program).

[0023] The in-vehicle communication I / F25 is an interface for connecting with other ECUs 30, using a communication standard based on the Controller Area Network (CAN) protocol. The in-vehicle communication I / F25 connects to the external bus 29. Furthermore, although not shown in the diagram, the ECU is not limited to ECU 30, but rather multiple ECUs are provided depending on the various functions within the vehicle 10.

[0024] Input / output I / F26 is an interface for communicating with onboard equipment 40 installed in vehicle 10.

[0025] The vehicle-mounted equipment 40 refers to various devices mounted on the vehicle 10. As an example of the vehicle-mounted equipment 40, the vehicle 10 includes a sensor group 42 and a display 44.

[0026] Sensor group 42 comprises various sensors and camera mechanisms used to detect the state of vehicle 10 and its surroundings. Specifically, sensor group 42 includes, for example, various sensors such as 3D-LiDAR sensors, millimeter-wave sensors, infrared sensors, turn signal sensors, throttle position sensors, vehicle speed sensors, wheel angle sensors, angular velocity sensors, Global Positioning System (GPS) sensors, illuminance sensors, gyroscope sensors, steering wheel angle sensors, and acceleration sensors, as well as multiple cameras that capture images of the surroundings of vehicle 10. Sensor group 42 outputs the detection results from each sensor and the image captured by each camera to designated electronic control units such as instrument ECU 20 and ECU 30.

[0027] Display 44 is disposed on the dashboard located in front of the driver's seat of vehicle 10, and is an instrument display used to display suggestions for actions related to the functions of vehicle 10 and images related to explanations of those functions. Display 44 is an example of a "display unit" in this invention.

[0028] The I / F27 wireless communication module is used for communicating with external devices. This wireless communication module uses communication standards such as 5G, LTE, Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0029] Furthermore, the CPU 21 of the instrument ECU 20 has an acquisition unit 21A and a control unit 21B as its functional structure. Each functional structure is implemented by the CPU 21 reading and executing the information processing program 24A stored in the storage device 24.

[0030] The acquisition unit 21A acquires various information available in the vehicle 10. Specifically, as various information, the acquisition unit 21A acquires detection results detected by various sensors constituting the sensor group 42 and detection information such as video images captured by each camera. In this embodiment, the sensor group 42 detects the position of the vehicle 10 and the positions of surrounding vehicles identified within the vehicle. Furthermore, in this embodiment, the sensor group 42 detects the vehicle's driving lane and detects lane changes. The acquisition unit 21A acquires the input value based on the vehicle's driving lane, as described later, and outputs it to the control unit 21B.

[0031] The control unit 21B performs display control related to the display on the display 44. For example, as part of this display control, the control unit 21B displays a surrounding environment display 100 showing the surrounding conditions of the vehicle 10 from a virtual viewpoint on the display 44 (see reference 21A) based on the detection information acquired by the acquisition unit 21A. Figure 2 As part of the surrounding conditions display 100, the display 44 shows the vehicle marker 10A (representing the vehicle 10), the driving lane display 30A (based on the vehicle), and the surrounding markers 40A (representing surrounding vehicles identified in the vehicle). The vehicle marker 10A is an image representing the vehicle, the surrounding markers 40A are images representing surrounding vehicles, and the driving lane display 30A is an image representing the driving lane and lane markings.

[0032] Here, "virtual viewpoint" refers to a viewpoint set in a three-dimensional virtual space with the position of the vehicle landmark 10A as the origin. This virtual viewpoint is defined by specified viewpoint coordinates and viewpoint angle (orientation) in the virtual space. As an example, this virtual viewpoint is a viewpoint in the virtual space that overlooks the vehicle 10 and its surroundings from specific viewpoint coordinates located above the vehicle landmark 10A and from a specific viewpoint angle. The control unit 21B is an example of a "control unit" in this invention.

[0033] An example of the display on monitor 44 will be described. Figure 2 This is a display example used to illustrate the surrounding conditions of the vehicle. The surrounding conditions display is part of the display 44 and is configured so that the driver, sitting in the driver's seat, can visually recognize it through the opening in the steering wheel.

[0034] refer to Figure 2The display method of the driving lane display 30A in the virtual space is explained. The driving lane display 30A is displayed by dividing the lanes with white lines (L), showing the vehicle's driving lane 31A and adjacent lanes 31B and 31C. Here, in the driving lane display 30A displayed in the virtual space, the road width on the instrument panel is fixed, and the difference in road width is normalized to determine the vehicle's drawing position. Regarding normalization, for example, the positions of the lane lines on both sides are identified to calculate the width of the vehicle's lane. For example, it is assumed that the vehicle is driving on a road with a normalized lane width of a standard lane width (3.6m). Then, by calculating the relative distances between the lane lines on both sides and the vehicle, the relative position is calculated based on the center line (c) in the normalized lane width. Regarding the position of the vehicle (i.e., the display position of the vehicle's marker 10A) or the display position of the surrounding markers 40A, the relative position is calculated and displayed based on the relative distance with the center line (c) of the vehicle's driving lane as the reference. Furthermore, in the virtual space, the vehicle marker 10A is fixedly displayed near the center, and the lateral position of the lane display 30A is adjusted. The display position of the surrounding markers 40A is indirectly calculated based on the vehicle's relative position (the vehicle's center) calculated from the centerline (c) of the vehicle's lane. As described above, the lane display 30A is displayed in the virtual space with a normalized lane width. In this case, lane changes occur when the vehicle moves 1.8m to either the left or right based on the centerline (c) of its lane. The control of the display method will be explained below, assuming a normalized road width.

[0035] In this embodiment, the control unit 21B detects the moving average of the vehicle based on an input value referenced to the vehicle's driving lane, and uses this value for display control of the vehicle marker 10A and the driving lane display 30A. Here, the input value referenced to the vehicle's driving lane represents the difference in distance along the lateral axis from the centerline of the driving lane to the center of the vehicle, relative to the driving lane identified as the vehicle's driving lane. The input value is the distance difference to the centerline of the driving lane automatically detected by the sensor group 42. Hereinafter, the display control will be described using a moving average. Furthermore, the vehicle marker 10A is an example of the "image of the vehicle" of this invention, and the driving lane display 30A is an example of the "image representing the driving lane" of this invention. Also, the surrounding marker 40A is an example of the "image representing surrounding vehicles" of this invention.

[0036] Here, an example of using a moving average method in the display of the vehicle marker 10A and the driving lane display 30A will be described. In virtual space, the vehicle marker 10A is fixedly displayed near the center; instead, the driving lane 31A is displayed in a manner that moves laterally when viewed from the vehicle. Here, as part of the display control for the vehicle marker 10A and the driving lane display 30A, the control unit 21B adjusts the position of the lateral axis of the driving lane display 30A by identifying how much the driving lane 31A has moved relative to the vehicle based on the detected moving average of the vehicle. That is, by applying a moving average filter to the adjustment of the display position of the driving lane, display stabilization is achieved.

[0037] Here, we will explain the issue of calculating the moving average in the display control when a lane change occurs. Figure 3 This is a graph that shows the detected input value being switched due to lane change. Figure 4 This is an example of the input values ​​and moving average calculation for a moving average filter during lane changes. The moving average is calculated based on the input values ​​in 0.1-second increments. Figure 3 (A1) represents the lane change before the change, and (A2) represents the lane change after the change, indicating a lane change from lane L1 to lane L2. The direction of movement for the lane change is indicated by a large arrow (M). The arrow at the front of the vehicle marker 10A indicates the centerline of the driving lane used as the reference value. Figure 3 As shown, after a lane change, the centerline that serves as the reference for the moving average filter's input value switches from the centerline (c1) of lane L1 to the centerline (c2) of lane L2. Figure 4In the table, “Moving Average (Continued)” is the calculated value of a moving average of 5 times, and the number of times the input value was moved averaged is indicated in parentheses. In addition, with the center line of the reference object as the reference, the left direction of the lane is negative and the right direction of the lane is positive, and a movement of ±1.8 is a lane change. The unit of the input value is (m) (illustration omitted). The part enclosed by the thick frame is before and after the lane change. If the calculation is continued by moving average (continued) 0.5 seconds after the lane change, the moving average is calculated in the opposite direction because there is a deviation between the input value and moving average before the lane change and the input value after the lane change. The moving average 0.4 seconds before the lane change is “-1.6 (5)”, and the continued moving average 0.5 seconds after the lane change is “-0.98 (5)”. Thus, if the moving average is continued, the appropriate calculation result will not be obtained, so the moving average is usually reset. “Moving Average (Reset)” resets the calculation of the moving average when the lane changes and recalculates it based on the input value after the lane change. The moving average 0.4 seconds before the lane change is "-1.6 (5)", and the moving average 0.5 seconds after the lane change, immediately after the reset, is "1.7 (1)". However, in the case of a reset, the averaging effect of the moving average disappears, and the new value immediately after the reset directly becomes the value of the moving average, causing a jump in the display. Furthermore, the display positions of other surrounding vehicles are calculated based on the display of this vehicle and the driving lane, so the display of surrounding vehicles is also affected.

[0038] Therefore, in this embodiment, when the vehicle changes lanes, the control unit 21B performs coordinate transformation on the past input value of the moving average before the lane change, using the lane of the lane change target as a reference, and uses the transformed input value to calculate the moving average after the change. Figure 5 This is an example of the input value of the moving average filter during lane change in this embodiment and the calculation of coordinate transformation in the moving average. In order to calculate the moving average after lane change, as shown in the "Input Value Coordinate Transformation" column, the input value from 0 seconds to 0.4 seconds is replaced with the value of +3.6m of the lane reference of the lane change target. Therefore, without resetting after lane change, the display control with deviation suppression can continue by using the moving average based on the lane change target. Regarding the calculation of the moving average after coordinate transformation, at the 0.5-second time point, it is performed as follows: "2.1+2.0+1.9+1.8+1.7=1.9(5)". Figure 5 In the example, it is able to mitigate the deviation of the 0.2m moving average relative to “1.7(1)” at the time of reset.

[0039] In this way, the past input values ​​are transformed with the driving lane of the lane change target as the reference, and the calculated moving average value is replaced with the transformed input values, thereby suppressing the effect of display deviation in display control using moving average.

[0040] The control unit 21B uses input values ​​to determine whether the vehicle has changed lanes. If a lane change is determined, the control unit 21B changes the vehicle's lane (31A) in the lane display 30A to the new lane. Furthermore, the control unit 21B calculates the vehicle's position using a coordinate-transformed moving average and calculates the position of the new lane relative to the vehicle's position. The calculated new lane position is then used in the display control on the display 44 to adjust the lateral position of the new lane and display the lane display 30A. Additionally, the control unit 21B uses the relative distance between the vehicle's position (calculated based on the coordinate-transformed moving average) and detected surrounding vehicles to control the display of surrounding objects 40A. This lane change determination is performed separately from the automatic detection by the sensor group 42 for coordinate transformation and display control purposes.

[0041] Figure 6 This is a flowchart illustrating the process of a specific procedure executed by the instrument ECU 20. The CPU 21 reads the information processing program 24A from the storage device 24 and expands it into the RAM 23 for execution, thereby performing the specific procedure. As an example, the specific procedure is automatically executed repeatedly at predetermined time intervals starting from when the ambient situation display 100 is first displayed on the display 44.

[0042] In step S100, the acquisition unit 21A acquires the input value detected by the sensor group 42 based on the vehicle's driving lane.

[0043] In step S102, the control unit 21B detects the moving average of the vehicle based on the input value with reference to the vehicle's driving lane.

[0044] In step S104, the control unit 21B uses the input value to determine whether the vehicle has changed lanes. If a lane change is determined to have occurred, the process proceeds to step S106; if no lane change is determined to have occurred, the process returns to step S100 and repeats.

[0045] In step S106, the control unit 21B changes the vehicle's driving lane (31A) in the driving lane display 30A to the lane after the lane change.

[0046] In step S108, the control unit 21B uses a coordinate-transformed moving average to calculate the position of the vehicle and calculates the position of the driving lane relative to the position of the vehicle.

[0047] In step S110, in the display control of the display 44, the control unit 21B uses the calculated position of the changed driving lane to adjust the lateral position of the changed driving lane and displays the driving lane display 30A. After display, it returns to step S100 and repeats the moving average process. In addition, if there is a surrounding object 40A in the surrounding conditions display 100, in addition to adjusting and displaying the lateral position of the driving lane, the display position of the surrounding object 40A is calculated and displayed based on the relative distance between its position and the adjusted lateral position of the driving lane.

[0048] In summary, the instrument ECU20 of this embodiment can reduce the discomfort of display control when changing lanes.

[0049] Furthermore, the various processes performed by the CPU 11 in reading and executing the software (program) in the above embodiments can also be executed by various processors other than the CPU. Examples of processors in this case include field-programmable gate arrays (FPGAs) and programmable logic devices (PLDs) whose circuit structure can be changed after manufacturing, graphics processing units (GPUs), and application-specific integrated circuits (ASICs)—processors with dedicated circuit structures specifically designed for executing specific processes. Moreover, each of the above processes can be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit composed of circuit elements such as semiconductor elements.

[0050] Furthermore, in the above embodiments, the information processing program is described as being pre-stored (installed) in a computer-readable, non-transitory recording medium. For example, the information processing program is pre-stored in ROM 12 or storage device 14. However, it is not limited to this; the programs may also be provided in a non-transitory recording medium such as a Compact Disc Read Only Memory (CD-ROM), a Digital Versatile Disc Read Only Memory (DVD-ROM), or a Universal Serial Bus (USB) memory. Furthermore, the information processing program may also be configured to be downloaded from an external device via a network.

[0051] The processing flow described in the above embodiments is an example. Unnecessary steps can be deleted, new steps can be added, or the processing order can be replaced without departing from the main idea.

Claims

1. An information processing device, characterized in that, The system includes a control unit that detects the moving average of the vehicle based on an input value referenced to the driving lane, and uses the detected moving average for display control of images of the vehicle and images representing the driving lane. The control unit performs the following processing: when the vehicle changes lanes, it transforms the past input value of the moving average before the lane change with the target lane as the reference, and uses the transformed input value to calculate the moving average after the change.

2. The information processing device according to claim 1, characterized in that, The control unit performs the following processing: using the input value to determine whether the lane change has been performed; if the lane change has been performed, the vehicle's driving lane is changed to the lane after the lane change; the position of the vehicle is calculated using the moving average after the coordinate transformation; the position of the changed driving lane relative to the position of the vehicle is calculated; and the calculated position of the changed driving lane is used in the display control of the display unit to adjust the lateral position of the changed driving lane and display an image representing the driving lane.

3. The information processing device according to claim 2, characterized in that, The control unit performs the following processing: using the relative distance between the position of the vehicle and the detected surrounding vehicles, calculated based on the moving average of the coordinate transformation, to control the display of images representing the surrounding vehicles of the vehicle.

Images