Display control device

Abstract

The present application relates to a display control device. An instrument ECU is provided with: an acquisition section for acquiring a surrounding situation of a vehicle; and a control section which, when an image representing the surrounding situation of the vehicle is displayed on a display provided in the vicinity of a driver's seat of the vehicle, assumes the width of a lane on which the vehicle is traveling only in a case where a lane division line located on one side of the vehicle is recognized, and displays an image representing the vehicle on the basis of the position of the vehicle in the width direction of the lane determined with the lane division line as a reference.

Description

Technical Field

[0001] This invention relates to a display control device. Background Technology

[0002] Japanese Patent Application Publication No. 2022-066952 discloses the following technology: when other vehicles are present in a distant area in front of the vehicle's direction of travel that is outside the display range of the screen, a mark indicating the presence of other vehicles in a distant area in front of the vehicle's direction of travel that is outside the display range of the screen is displayed at a fixed position on the driving lane. Summary of the Invention

[0003] According to the technology described in Japanese Patent Application Publication No. 2022-066952, it is difficult to accurately calculate the vehicle's side position when only one side of the lane markings is identified. If display control is implemented to bring the image representing the vehicle closer to the center of the lane, then when the occupant intentionally moves the vehicle closer to the lane markings while changing lanes, the image representing the vehicle will always be displayed in the center of the lane, sometimes causing the displayed position to deviate from the actual position.

[0004] The object of the present invention is to provide a display control device that can display an image representing the vehicle with a movement close to actual movement, even when the recognition of the zoning line is only on one side.

[0005] The display control device according to the first method includes: an acquisition unit for acquiring the surrounding conditions of the vehicle; and a display control unit that, when displaying an image representing the surrounding conditions of the vehicle on a display unit provided around the driver's seat of the vehicle, if only a lane marking line located on one side of the vehicle is identified, assumes the width of the lane in which the vehicle is traveling, and displays an image representing the vehicle based on the position of the vehicle in the lane width direction determined with reference to the lane marking line.

[0006] According to the display control device involved in the first method, even when the recognition of the dividing line is only on one side, it is possible to display an image representing the vehicle with a movement close to actual movement.

[0007] In the second embodiment of the display control device, which is the same as the first embodiment, the display control unit assumes that the position where the marking line is offset by a predetermined width is the center of the lane in which the vehicle is traveling, and determines the position of the vehicle in the lane width direction based on the marking line.

[0008] According to the display control device involved in the second method, the center of the lane in which the vehicle is traveling (i.e., the width of the lane) can be assumed by offsetting the zoning lines.

[0009] The display control device according to the third method, in addition to the display control device according to the second method, includes a setting unit that sets a predetermined width including the center of the lane as a non-sensitive area that does not respond to the movement of the vehicle in the lane width direction. When the vehicle is located outside the non-sensitive area, the display control unit causes the position of the image representing the vehicle in the lane width direction to change.

[0010] According to the display control device involved in the third method, there is no need to display the slight movement of the vehicle in the lane width direction, thus reducing display annoyance.

[0011] In the fourth embodiment of the display control device, when the display control device is able to identify the zoning lines on both sides before identifying only the zoning line on one side, the display control unit assumes the lane width determined by the zoning lines on both sides as the width of the lane in which the vehicle is traveling, and determines the position of the vehicle in the lane width direction based on the zoning line on one side.

[0012] According to the display control device involved in the fourth method, the width of the lane in which the vehicle is traveling can be assumed by using the lane width determined by the lane lines on both sides identified before only the lane lines on one side are identified.

[0013] The display control device according to the fifth embodiment, in addition to the display control device according to the fourth embodiment, includes: a setting unit that sets a non-sensitive area that does not respond to the movement of the vehicle in the lane width direction based on the lane width determined by the lane marking lines on both sides; when the vehicle is outside the non-sensitive area, the display control unit causes the position of the image representing the vehicle in the lane width direction to change.

[0014] According to the display control device involved in the fifth method, there is no need to display the slight movement of the vehicle in the lane width direction, thus reducing display annoyance.

[0015] As explained above, according to the present invention, even when the identification of the dividing line is only on one side, the vehicle can be displayed with a movement close to the actual movement. Attached Figure Description

[0016] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which the same reference numerals denote the same elements, and wherein:

[0017] Figure 1 It is a block diagram representing the hardware structure of a vehicle.

[0018] Figure 2A It is a diagram used to illustrate the positional relationships of vehicles.

[0019] Figure 2B It means Figure 2A The image shown represents the positional relationship of the vehicles.

[0020] Figure 3 This is a diagram representing a state where the vehicle only recognizes the first zone marking on one side.

[0021] Figure 4A This diagram illustrates a method for determining the position of a vehicle when only one side of the first zone marking is identified.

[0022] Figure 4B It means through Figure 4A The method shown is used to determine the display image representing the vehicle's position.

[0023] Figure 5 This is a diagram illustrating an example of a state where a non-sensitive area is defined.

[0024] Figure 6 This is a flowchart illustrating an example of the display control processing performed by the instrument ECU. Detailed Implementation

[0025] The vehicle 10 described below is an example of this embodiment.

[0026] Figure 1 This is a block diagram representing the hardware structure of vehicle 10. For example... Figure 1 As shown, vehicle 10 includes an Electronic Control Unit (ECU) 20. Vehicle 10 is an example of a "vehicle" in this invention, and the ECU 20 is an example of a "display control device" in this invention.

[0027] The instrument cluster ECU 20 is configured to include a central processing unit (CPU) 21, a read-only memory (ROM) 22, a random access memory (RAM) 23, a storage device 24, an in-vehicle communication interface (I / F) 25, an input / output interface (I / F) 26, and a wireless communication interface (I / F) 27. These components are communicatively connected to each other via an internal bus 28.

[0028] CPU 21 is the central processing unit, which executes various programs or controls various parts. That is, CPU 21 reads programs from ROM 22 or storage device 24 and uses RAM 23 as its working area to execute the programs. CPU 21 performs control of the above-mentioned structures and various arithmetic operations according to the programs recorded in ROM 22 or storage device 24.

[0029] ROM22 stores various programs and data. RAM23 serves as a temporary storage area for programs or data.

[0030] The storage device 24 is composed of a storage medium such as an embedded Multi Media Card (eMMC) or Universal Flash Storage (UFS), and stores various programs and data. The storage device 24 stores an information processing program 24A. The information processing program 24A is a program used to cause the CPU 21 to execute the display control processing involved in this embodiment.

[0031] The in-vehicle communication I / F25 is an interface for connecting with other ECUs 30, using a communication standard based on the Controller Area Network (CAN) protocol. The in-vehicle communication I / F25 connects to the external bus 29. Furthermore, although not illustrated, the ECU is not limited to ECU 30; multiple ECUs may be present depending on the various functions within the vehicle 10.

[0032] The input / output I / F26 is an interface used to communicate with the on-board equipment 40 mounted on the vehicle 10.

[0033] The vehicle-mounted equipment 40 refers to various devices mounted on the vehicle 10. As an example of the vehicle-mounted equipment 40, the vehicle 10 includes a sensor group 42 and a display 44.

[0034] The sensor group 42 is configured to include various sensors and camera devices for detecting the state of the vehicle 10 and the surrounding conditions of the vehicle 10. Specifically, the sensor group 42 includes, for example, various sensors such as 3D-LiDAR sensors, millimeter-wave sensors, infrared sensors, turn signal sensors, throttle position sensors, vehicle speed sensors, wheel angle sensors, angular velocity sensors, Global Positioning System (GPS) sensors, illuminance sensors, gyroscope sensors, steering wheel angle sensors, and acceleration sensors, as well as multiple cameras for capturing images of the surroundings of the vehicle 10. The sensor group 42 outputs the detection results of each sensor and the image captured by each camera to the specified electronic control units such as the instrument cluster ECU 20 and ECU 30.

[0035] Display 44 is an instrument display screen disposed on the instrument panel located in front of the driver's seat of vehicle 10 and used to display operation suggestions related to the functions of vehicle 10 and images related to the explanation of the functions. Display 44 is an example of a "display unit" in this invention.

[0036] The I / F27 wireless communication module is a wireless communication module used for communicating with external devices. This module can use communication standards such as 5G, LTE, Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0037] Furthermore, the CPU 21 of the instrument ECU 20 has an acquisition unit 21A and a control unit 21B as its functional structure. Each functional structure is implemented by the CPU 21 reading and executing the information processing program 24A stored in the storage device 24.

[0038] The acquisition unit 21A acquires various types of information that can be obtained in the vehicle 10. Specifically, the acquisition unit 21A acquires detection information such as detection results from various sensors constituting the sensor group 42 and camera images from each camera as various types of information. The acquisition unit 21A also acquires peripheral information indicating the surrounding conditions of the vehicle 10.

[0039] The control unit 21B performs display control related to the display on the display 44. For example, as part of this display control, the control unit 21B displays, based on the detection information acquired by the acquisition unit 21A, a surrounding environment display showing the surrounding conditions of the vehicle 10 from a virtual viewpoint and a vehicle image showing the vehicle 10 (see reference). Figure 2B (etc.) are displayed on the display 44. Here, "virtual viewpoint" refers to a viewpoint set in a three-dimensional virtual space with the position of the vehicle image as the origin. This virtual viewpoint is defined by specified viewpoint coordinates and viewpoint angle (orientation) in the virtual space. As an example, this virtual viewpoint is a viewpoint that looks down on the vehicle 10 and its surroundings from a specific viewpoint coordinate located above the vehicle image in the virtual space at a specific viewpoint angle. The control unit 21B is an example of the "display control unit" and "setting unit" in this invention.

[0040] Figure 2A This is a diagram used to illustrate the positional relationships of vehicle 10. Figure 2A In this configuration, vehicle 10 travels in lane 102 defined by a first lane marking 101L and a second lane marking 101R located on either side of vehicle 10. The first lane marking 101L is located on the left side of vehicle 10, and the second lane marking 101R is located on the right side of vehicle 10. Unless otherwise specified, the first lane marking 101L and the second lane marking 101R are simply referred to as lane marking 101. Figure 2B It means Figure 2AThe diagram shown is a display image 60 illustrating the positional relationship of the vehicles 10. Additionally, for ease of explanation, the display image 60, including the vehicle image 61, is shown in a two-dimensional format. Figure 2B As shown, the control unit 21B will Figure 2A The display image 60, showing the positional relationship of the vehicle 10, is displayed on a designated display area of ​​the display 44. This designated display area is a portion of the display 44 and is designed so that the driver, seated in the driver's seat, can visually recognize it through the opening in the steering wheel. The display image 60 includes a vehicle image 61 showing the vehicle 10, a first slit image 62L showing the first slit line 101L, a second slit image 62R showing the second slit line 101R, and a lane image 63 showing the lane 102. The first slit image 62L is the slit image located to the left of the vehicle image 61, and the second slit image 62R is the slit image located to the right of the vehicle image 61. Unless otherwise specified, the first slit image 62L and the second slit image 62R are simply referred to as the slit image 62. Furthermore, an image 64 showing the vehicle 10's speed and an image 65 showing the vehicle 10's gear shift position are displayed above the lane image 63.

[0041] However, as Figure 3 As shown, when vehicle 10 only recognizes the zoning line 101 on one side, it is difficult to accurately calculate the position of the vehicle side of vehicle 10. Figure 3 This indicates that vehicle 10 only recognizes the first lane marking 101L on one side. In this case, if control is performed to bring vehicle image 61 closer to the center of lane image 63, when the occupant intentionally moves the vehicle closer to the lane marking while changing lanes, vehicle image 61 will always be displayed in the center of lane image 63, sometimes causing the displayed position to deviate from the actual position.

[0042] Therefore, when the control unit 21B of this embodiment displays an image 60 showing the surrounding conditions of the vehicle 10 on a display 44 provided around the driver's seat of the vehicle 10, it assumes the width of the lane 102 in which the vehicle 10 is traveling, and displays the vehicle image 61 based on the position of the vehicle 10 in the lane width direction, using the lane marking 101 as a reference, even if only the lane marking 101 on one side of the vehicle 10 is identified. Here, "assume" means that if the lane marking 101 on both sides is not identified, the width of the lane 102 cannot be determined, but when only the lane marking 101 on one side is identified, a temporary lane width (e.g., a normal lane width of 3.6m, etc.) is determined. In addition, the position of the vehicle 10 in the lane width direction refers to the direction that intersects (e.g., orthogonally) with the direction of travel of the vehicle 10 (indicated by an arrow), that is, the position in the left-right direction (lane width direction) when viewed from the vehicle 10. Accordingly, even when the lane marking 101 is identified only on one side, the vehicle image 61 can be displayed with a movement close to actual movement.

[0043] Figure 4A This diagram illustrates a method for determining the position of vehicle 10 when only one side of the first zone line 101L is identified. (See diagram for example.) Figure 4A As shown, the control unit 21B assumes that the position M1 at which the first lane marking 101L is offset by a predetermined width is the center of the lane 102 in which the vehicle 10 travels, and determines the position M2 of the vehicle 10 in the lane width direction based on the first lane marking 101L. Specifically, the typical lane width is, for example, 3.6m, so half of 3.6m, i.e., 1.8m, is used as the predetermined width for offsetting. That is, in Figure 4A In the example case, the center of lane 102 is assumed to be M1 = 1.8m. Furthermore, the lane width is not limited to 3.6m; various lane widths can be preset. Then, the position M2 of vehicle 10 in the lane width direction is determined based on the first lane marking 101L. This also applies even if vehicle 10 only recognizes the second lane marking 101R on one side. That is, the control unit 21B assumes the position where the second lane marking 101R is offset by a predetermined width as the center of lane 102 where vehicle 10 travels, and determines the position of vehicle 10 based on the second lane marking 101R. Accordingly, the center of lane 102 (i.e., the lane width) where vehicle 10 travels can be assumed by offsetting the lane marking 101.

[0044] Furthermore, if the control unit 21B can identify both the first zone lines 101L and the second zone lines 101R on both sides before identifying only the first zone line 101L on one side, the control unit 21B can assume the lane width determined by the first zone lines 101L and the second zone lines 101R on both sides as the width of the lane 102 in which the vehicle 10 travels, and determine the position of the vehicle 10 in the lane width direction based on the first zone line 101L. If the control unit 21B can identify both the first zone lines 101L and the second zone lines 101R on both sides before identifying only the first zone line 101L on one side, the control unit 21B stores the lane width determined by the first zone lines 101L and the second zone lines 101R on both sides in the storage device 24. In addition, "before identifying only the first zone line 101L on one side" preferably refers to the moment before identifying only the first zone line 101L on one side, but is not limited to the moment before. Accordingly, the width of the lane 102 in which the vehicle 10 travels can be assumed by using the lane width determined by the lane lines 101 on both sides identified before only the first lane line 101L on one side is identified.

[0045] Figure 4B It means through Figure 4A The method shown represents the position of vehicle 10 as displayed in image 60. Figure 4B As shown, even when the vehicle 10 only recognizes the first zone line 101L on one side, it is able to display the vehicle image 61 with a movement close to the actual movement.

[0046] Figure 5 This diagram illustrates an example of a state where a non-sensitive region R1 is defined. For example... Figure 5 As shown, the control unit 21B sets a predetermined width, including the center C1 of lane 102, as a non-sensitive region R1 that does not respond to movement of vehicle 10 in the lane width direction. The non-sensitive region R1 is a region within the recognition range of sensor group 42, where movement of vehicle 10 does not respond, and therefore, movement of vehicle 10 is not reflected in the display. The non-sensitive region R1 can be set such that sensor group 42 recognizes the vehicle but the display remains unchanged, or it can be set such that sensor group 42 cancels recognition and the display remains unchanged. In this case, if vehicle 10 is outside the non-sensitive region R1, the control unit 21B changes the position of vehicle image 61 representing vehicle 10 in the lane width direction. Furthermore, the position of vehicle 10 can be changed if at least a portion of vehicle 10 is outside the non-sensitive region R1, or it can be changed only if the entire vehicle 10 is outside the non-sensitive region R1. Therefore, it is unnecessary to display minute movements of vehicle 10 in the lane width direction, thus reducing display clutter.

[0047] Furthermore, if both first zone lines 101L and second zone lines 101R can be identified before only the first zone line 101L on one side is identified, a non-sensitive region that does not respond to movement of the vehicle 10 in the lane width direction can be set based on the lane width determined by the first zone lines 101L and second zone lines 101R on both sides. In this case, even if the vehicle 10 is outside the non-sensitive region R1, the control unit 21B will still change the position of the vehicle image 61 representing the vehicle 10. Accordingly, it is not necessary to display the slight movement of the vehicle 10 in the lane width direction, thus reducing display distractions.

[0048] Figure 6 This is a flowchart illustrating an example of the display control processing performed by the instrument ECU 20. The CPU 21 reads the information processing program 24A from the storage device 24 and loads it into the RAM 23 for execution, thereby performing the display control processing. As an example, the display control processing is executed automatically and repeatedly at predetermined time intervals.

[0049] exist Figure 6 In step S101, the CPU 21 acquires peripheral information representing the surrounding conditions of the vehicle 10. Specifically, as described above, a sensor group 42 capable of detecting the surrounding conditions is provided in the vehicle 10, and the information detected by these sensor groups 42 is acquired as peripheral information.

[0050] In step S102, the CPU21 determines whether the boundary lines 101 on both sides are identified based on the surrounding information obtained in step S101. If the boundary lines 101 on both sides are not identified, that is, only the boundary line 101 on one side is identified (in the case of negative determination), the process proceeds to step S103. If the boundary lines 101 on both sides are identified (in the case of positive determination), the process proceeds to step S105.

[0051] In step S103, CPU21 assumes the width of lane 102 in which vehicle 10 travels. Specifically, as an example, as described above... Figure 4A As shown, the position M1, where the first zone line 101L on one side is offset by a predetermined width, is assumed to be the center of the lane 102 in which the vehicle 10 travels. Furthermore, if the first zone lines 101L on both sides and the second zone lines 101R on both sides can be identified before only the first zone line 101L on one side is identified, the lane width determined by the first zone lines 101L and the second zone lines 101R on both sides can be assumed to be the width of the lane 102 in which the vehicle 10 travels.

[0052] In step S104, as an example, as described above... Figure 4BAs shown, CPU21 uses the first zone line 101L on one side identified in step S102 as a reference, displays vehicle image 61 based on the position M2 of vehicle 10 in the lane width direction, and returns to step S101 to repeat the processing.

[0053] On the other hand, in step S105, as an example, as described above... Figure 2B As shown, CPU21 uses the dividing lines 101 on both sides identified in step S102 as a reference, displays vehicle image 61 based on the position of vehicle 10 in the lane width direction, and returns to step S101 to repeat the processing.

[0054] As explained above, according to this embodiment, when the recognition of the lane markings is only on one side, when the vehicle is intentionally driven close to the lane markings when changing lanes, the vehicle can be displayed as if it were actually moving.

[0055] Furthermore, in the above embodiments, various processors other than the CPU can execute various processes performed by the software (program) read and executed by the CPU 21. Examples of processors in this case include programmable logic devices (PLDs) such as Field-Programmable Gate Arrays (FPGAs) whose circuit structure can be modified after manufacturing, and dedicated circuits such as Application-Specific Integrated Circuits (ASICs) that have circuit structures specifically designed for executing specific processes. Furthermore, various processes can be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit composed of combined semiconductor elements and other circuit elements.

[0056] Furthermore, while the above embodiment describes the information processing program 24A being pre-stored (installed) in the storage device 24, it is not a limitation. The information processing program 24A can be provided on recording media such as a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), or a universal serial bus (USB) memory. Additionally, the information processing program 24A can also be downloaded from an external device via a network. Moreover, the technology of this invention can also be applied to programs and program products.

Claims

1. A display control device, characterized in that, have: Acquisition unit, used to acquire information about the vehicle's surroundings; and The display control unit, when displaying an image representing the surrounding conditions of the vehicle on a display unit provided around the driver's seat of the vehicle, recognizes only the lane markings on one side of the vehicle, assumes the width of the lane in which the vehicle is traveling, and displays an image representing the vehicle based on the position of the vehicle in the lane width direction determined with reference to the lane markings.

2. The display control device according to claim 1, characterized in that, The display control unit assumes that the position where the zoning line is offset by a predetermined width is the center of the lane in which the vehicle is traveling, and determines the position of the vehicle in the lane width direction based on the zoning line.

3. The display control device according to claim 2, characterized in that, It also has: The setting unit defines a predetermined width, including the center of the lane, as a non-sensitive area that does not respond to movement of the vehicle in the lane width direction. When the vehicle is located outside the non-sensitive area, the display control unit changes the position of the image representing the vehicle in the lane width direction.

4. The display control device according to claim 1, characterized in that, If the lane markings on both sides can be identified before only the lane markings on one side are identified, the display control unit assumes the lane width determined by the lane markings on both sides as the width of the lane in which the vehicle is traveling, and determines the position of the vehicle in the lane width direction based on the lane markings on one side.

5. The display control device according to claim 4, characterized in that, It also has: The setting unit sets a non-sensitive area that does not respond to the movement of the vehicle in the lane width direction based on the lane width determined by the lane marking lines on both sides. When the vehicle is located outside the non-sensitive area, the display control unit changes the position of the image representing the vehicle in the lane width direction.

Images