Display control device, display device, display system, vehicle, display control method, and non-transitory storage medium

Abstract

The present application provides a display control device, a display device, a display system, a vehicle, a display control method, and a non-transitory storage medium that appropriately correct a positional deviation that occurs between a position at which a sign that alerts attention to an object is displayed in superposition and a position of the actual object. The display control device includes a position information acquisition unit that acquires position information of an object based on detection information of an object detection unit that detects an object in the vicinity of the vehicle; a speed information acquisition unit that acquires speed information of the vehicle; a approaching direction acquisition unit that acquires an approaching direction of the object approaching the vehicle; a shift unit that shifts the position acquired by the position information acquisition unit in the approaching direction based on the speed information acquired by the speed information acquisition unit; and a sign generation unit that generates a sign at a shifted position at which the shift unit has performed the shift, the sign being displayed in superposition in a third display region provided in the vehicle cabin and alerting attention to the object.

Description

Technical Field

[0001] This disclosure relates to a display control device, a display device, a display system, a vehicle, a display control method, and a non-transitory storage medium. Background Technology

[0002] There has long been a known display device, such as a tilting display, which projects an image onto the windshield of a vehicle and overlays a virtual image of an AR (Augmented Reality) image onto the foreground of the vehicle.

[0003] For example, Japanese Patent No. 6278222 discloses a technology that includes an object detection unit and a display control unit. The object detection unit detects objects that need to be alerted to the driver of the vehicle and calculates the distance from the object to the vehicle. The display control unit displays a warning sign that alerts the driver to the object detected by the object detection unit adjacent to the object at a predetermined distance from the object on the windshield when viewed from the driver's perspective. The display control unit changes the predetermined distance based on the distance from the object to the vehicle, thereby changing the display position of the warning sign. It also corrects the display position of the warning sign based on the time difference between detecting the object and displaying the warning sign, and further corrects the predetermined distance based on the speed of the vehicle when the object is detected.

[0004] According to the above structure, since the display position of the attention sign is corrected based on the time difference from the detection of the object to be noticed to the display of the attention sign, and the predetermined distance is further corrected based on the speed of the vehicle when the object to be noticed is detected, the attention sign can be displayed at an appropriate position regardless of the time difference to the display of the attention sign and the speed of the vehicle.

[0005] However, the technology disclosed in Japanese Patent No. 6278222 is a technology that corrects the display position by changing the distance between the attention sign and the attention sign based on the time difference from the detection of the object to be noticed to the display of the attention sign and the speed of the vehicle when the object to be noticed. Since there is still a positional deviation between the object to be noticed and the attention sign, there is room for improvement. Summary of the Invention

[0006] This disclosure appropriately corrects for positional deviations that occur between the position of an overlapping marker that draws attention to an object and the actual position of the object.

[0007] The display control device according to the first method includes: a position information acquisition unit that acquires position information related to the position of an object based on detection information from an object detection unit that detects objects around the vehicle; a speed information acquisition unit that acquires speed information related to the speed of the vehicle; an approach direction acquisition unit that acquires the approach direction in which the object approaches the vehicle; an offset unit that offsets the position information acquired by the position information acquisition unit to the approach direction acquired by the approach direction acquisition unit based on the speed information acquired by the speed information acquisition unit; and an identification mark generation unit that generates an identification mark based on the offset position information after the offset by the offset unit, the identification mark being overlaid and displayed in a display area provided in the passenger compartment to draw attention to the object.

[0008] According to the display control device of the first method, a marker is generated at a position after the position of the object acquired by the position information acquisition unit has been shifted based on speed information. This allows a marker that alerts the user to the object to be observed at the position obtained by shifting the position of the object acquired by the position information acquisition unit according to speed. Therefore, it is possible to correct for positional deviations between the overlaid marker and the actual object caused by the processing time from when the object is detected by the object detection unit to when the marker generation unit generates the marker.

[0009] Furthermore, by shifting the position of the target acquired by the position information acquisition unit in the approach direction of the target, the position of the target acquired by the position information acquisition unit can be shifted in a direction that would cause a positional deviation between the overlaid marker and the actual target. Therefore, the positional deviation between the position of the overlaid marker that draws attention to the target and the position of the actual target can be appropriately corrected. As a result, the position of the overlaid marker can be made closer to the position of the actual target.

[0010] The second method involves a display control device that, in the first method, sets the speed information to be related to the absolute speed of the vehicle when the approach direction obtained by the approach direction acquisition unit is the vehicle's travel direction.

[0011] According to the display control device of the second method, since the speed information is set to the absolute speed of the vehicle, the speed information is obtained by detecting the speed of the vehicle. Therefore, compared with the case where the speed information is set to the relative speed of the vehicle relative to the object, the speed information can be obtained more quickly. As a result, the time for the offset unit to shift the position of the object obtained by the position information acquisition unit can be shortened. In addition, the load on the CPU mounted on the display control device can also be reduced. Therefore, the positional deviation between the position of the overlapping display indicating attention to the object and the actual position of the object can be more appropriately corrected.

[0012] The third method involves a display control device that, in the first method, when the approach direction obtained by the approach direction acquisition unit is an inclined direction that is inclined relative to the vehicle's travel direction, sets the speed information to information related to the relative speed of the vehicle relative to the target.

[0013] According to the display control device involved in the third method, by setting the speed information to the relative speed of the vehicle relative to the target vehicle, the offset unit is appropriately set based on this relative speed to offset the position of the target acquired by the position information acquisition unit by an offset distance. Therefore, it is possible to more appropriately correct the positional deviation between the position of the overlapping display indicating attention to the target and the actual position of the target.

[0014] The fourth method involves a display control device in any one of the first to third methods, wherein the offset unit adjusts the offset distance by which the offset unit shifts the position information based on the processing time taken from the time the object detection unit detects the object until the mark generation unit generates the mark.

[0015] According to the display control device of the fourth method, the offset distance that causes the position of the object acquired by the position information acquisition unit to deviate is adjusted based on the processing time from the time the object detection unit detects the object to the time the mark generation unit generates the mark. This allows for setting a larger offset distance when the processing time is long and a smaller offset distance when the processing time is short. Therefore, the offset distance that causes the offset unit to deviate the position of the object acquired by the position information acquisition unit can be appropriately set according to the processing time. As a result, the position of the overlaid mark can be corrected to an appropriate position corresponding to the processing time.

[0016] The display control device involved in the fifth method is, in any one of the first to fourth methods, the position information acquisition unit acquires vehicle travel direction position information related to the position of the object in the vehicle's travel direction based on radar detection information, and acquires vehicle orthogonal direction position information related to the position of the object in an orthogonal direction orthogonal to the vehicle's travel direction based on camera images.

[0017] According to the display control device involved in the fifth method, the position information of an object in the vehicle's direction of travel is obtained based on radar detection information, and the position information of an object in the orthogonal direction to the vehicle's direction of travel is obtained based on camera images. Therefore, more accurate object position information can be obtained. As a result, it is possible to more appropriately correct for positional deviations between the position of an overlaid marker indicating attention to the object and the actual position of the object.

[0018] The sixth method involves a display control device in which, in any one of the first to fifth methods, the identifier generation unit changes the size of the identifier based on the position information obtained by the position information acquisition unit.

[0019] According to the display control device of the sixth method, the size of the sign is changed based on the position information obtained by the position information acquisition unit. This allows the sign to be set to a larger size when the vehicle is close to the object, and a smaller size when the vehicle is far from the object. Therefore, it is possible to overlay signs of appropriate sizes according to the distance to the object. As a result, occupants can recognize the distance to the object based on the size of the sign.

[0020] The display device involved in the seventh method includes: a display control device as described in any one of the first to sixth methods; an output unit that outputs an image that causes the mark to be superimposed on a display area provided in the carriage; and a display area that superimposes the image output by the output unit.

[0021] According to the display device involved in the seventh method, it is possible to appropriately correct the positional deviation between the position of the mark that reminds attention to the object in the overlapping display and the actual position of the object.

[0022] The display system according to the eighth method includes: a display device according to the seventh method; and a target detection unit that detects the targets around the vehicle.

[0023] According to the display system involved in the eighth method, it is possible to appropriately correct the positional deviation between the position of the marker that reminds attention to the object in the overlapping display and the actual position of the object.

[0024] The vehicle involved in the ninth method includes: the display system of the eighth method; and a windshield that constitutes the display area.

[0025] According to the vehicle involved in the ninth method, since the sign is displayed in the display area of ​​the windshield, the occupant can recognize the sign while keeping their eyes focused forward. Therefore, the occupant can recognize the sign without taking their eyes off the front of the vehicle while driving.

[0026] The display control method involved in the tenth method performs the following processing by a processor: obtaining position information related to the position of the object based on the detection information of the object detection unit that detects objects around the vehicle; obtaining speed information related to the speed of the vehicle; obtaining the approach direction of the object approaching the vehicle; shifting the position information to the approach direction based on the speed information; generating an identifier based on the position information after the shift, and displaying the identifier overlaid in a display area set in the passenger compartment to draw attention to the object.

[0027] According to the display control method involved in the tenth method, it is possible to appropriately correct the positional deviation between the position of the marker that reminds attention to the object in the overlapping display and the actual position of the object.

[0028] The eleventh method involves a non-temporary storage medium storing a program that causes a processor to perform the following processing: obtaining position information related to the position of an object based on detection information from an object detection unit that detects objects around the vehicle; obtaining speed information related to the speed of the vehicle; obtaining the approach direction of the object approaching the vehicle; shifting the position information in the approach direction based on the speed information; and generating an identifier based on the shifted position information, which is overlaid and displayed in a display area set in the passenger compartment to draw attention to the object.

[0029] According to the non-temporary storage medium involved in the eleventh method, it is possible to appropriately correct the positional deviation between the position of the overlapping display of the marker reminding attention to the object and the actual position of the object.

[0030] According to this disclosure, it is possible to appropriately correct for positional deviations between the position of an overlaid marker that draws attention to an object and the actual position of the object. Attached Figure Description

[0031] Embodiments of the present invention will be described in detail with reference to the following figures, wherein:

[0032] Figure 1 This is a top-view view of a vehicle equipped with the display system according to the first embodiment.

[0033] Figure 2 This is a schematic diagram illustrating the state of the front side of a vehicle as viewed from inside the passenger compartment of a vehicle equipped with the display system according to the first embodiment.

[0034] Figure 3 This is a block diagram illustrating the hardware structure of the display system according to the first embodiment.

[0035] Figure 4 This is a block diagram illustrating the functional structure of the display control device according to the first embodiment.

[0036] Figure 5 This is a top-view view of a vehicle equipped with the display system according to the first embodiment, and it is a view showing the approach direction of the object as the direction of the vehicle's travel.

[0037] Figure 6 This is a top-view view of a vehicle equipped with the display system according to the first embodiment, and a view showing the case where the approach direction of the object is tilted relative to the direction of travel of the vehicle.

[0038] Figure 7 This diagram illustrates a display example of the display area of ​​the display device according to the first embodiment, and also illustrates the case where the approach direction of the object is the vehicle's travel direction.

[0039] Figure 8 This diagram shows a display example of the display area of ​​the display device according to the first embodiment, and also shows the case where the approach direction of the object is tilted relative to the vehicle's direction of travel.

[0040] Figure 9 This diagram illustrates a display example of the display area of ​​the display device according to the first embodiment, and shows a case where the object is located at a relatively far position.

[0041] Figure 10 This is a flowchart illustrating the display processing flow of the display control device according to the first embodiment.

[0042] Figure 11 This diagram illustrates a display example of a comparative display device, and also shows the case where the approach direction of the object is the direction of vehicle travel.

[0043] Figure 12This diagram illustrates a display example of the display device used for comparison, and also shows the case where the approach direction of the object is tilted relative to the direction of vehicle travel.

[0044] Figure 13 This diagram illustrates a display example of the display device according to the second embodiment. Detailed Implementation

[0045] [First Implementation]

[0046] Hereinafter, the display system according to the first embodiment will be described with reference to the accompanying drawings. Furthermore, Figure 1 The arrow marker FR indicates the front side of the vehicle, and the arrow marker RH indicates the right side of the vehicle. Furthermore, in the first embodiment, an example of the vehicle 10 traveling forward at a speed V1 will be described.

[0047] [Structure of a vehicle equipped with a display system]

[0048] like Figure 1 As shown, the vehicle 10 includes a camera 12 and a radar 14, both serving as object detection units. The camera 12 is mounted on the front bumper of the vehicle 10 and captures images of the area in front of the vehicle. The camera 12 can be configured to have a viewing angle 12A.

[0049] Radar 14 is mounted on the front bumper of vehicle 10. Radar 14 detects the distance between vehicle 10 and object 90, and the direction of object 90 relative to vehicle 10, by receiving reflected waves from objects 90 surrounding vehicle 10 after the transmitted waves have contacted them. In other words, radar 14 detects the position information of object 90 relative to vehicle 10. Radar 14 has a smaller field of view 14A compared to the field of view 12A of camera 12.

[0050] As an object 90, it can be set to the vehicle 92 that is traveling in front of the vehicle (see reference). Figure 1 ), pedestrians 94 (refer to) Figure 6 Or bicycles, oncoming vehicles traveling in the opposite lane, vehicles that are parked, or other obstacles.

[0051] like Figure 2 As shown, an instrument panel 18 and a windshield (also known as a front windshield) 22 are provided on the front side of the passenger compartment of vehicle 10.

[0052] (Dashboard)

[0053] The dashboard 18 is arranged to extend in the width direction of the vehicle. A steering wheel 20 is provided on the right side of the dashboard 18. In the first embodiment, as an example, the vehicle 10 is configured as a right-hand drive vehicle with the steering wheel 20 provided on the right side of the vehicle.

[0054] The dashboard 18 is provided with a first display unit 28 as an output unit and a second display unit 30 as an output unit, wherein the first display unit 28 has a first display area G1 as a display area and the second display unit 30 has a second display area G2 as a display area.

[0055] The first display unit 28 is located on the right side of the instrument panel 18 and in front of the steering wheel 20. The first display unit 28 is configured, for example, as a speedometer that displays the vehicle speed, or an instrument display that displays directional indicators and warnings.

[0056] The second display unit 30 is positioned at the center of the instrument panel 18 in the vehicle width direction. The second display unit 30 is configured, for example, as a central display showing images output by the navigation system.

[0057] (windshield)

[0058] The windshield 22 is supported by the front pillar 24. The front pillar 24 is located at the front, right and left sides of the vehicle, and extends in a generally vertical direction.

[0059] A third display unit 32 is provided on the windshield 22, and the third display unit 32 has a third display area G3 as a display area. Furthermore, based on information output from the display control device 35 located inside the instrument panel 18, an image projected from the projection unit 51 (which is an output unit) is projected onto the third display unit 32 on the windshield 22. Thus, this image is superimposed as a virtual image on the occupant (driver) H (reference). Figure 1 (in front of)

[0060] The display device 34, which includes a display control device 35, a projection unit 51, and a third display unit 32, constitutes a bottom-view display device. The third display unit 32, on which images projected from the projection unit 51 are projected, constitutes the projection surface of the bottom-view display device.

[0061] like Figure 1 As shown, the projection unit 51 is configured to have a smaller viewing angle 51A compared to the viewing angle 12A. The viewing angle 51A of the projection unit 51 is configured such that the viewpoint E of the occupant H is virtually taken as the origin. The viewing angle 51A of the projection unit 51 refers to the angular range in which an image can be displayed in the third display area G3.

[0062] like Figure 3 As shown, the display device 34, camera 12, radar 14 and vehicle speed sensor 15 constitute the display system 16.

[0063] [Hardware Structure of the Display System]

[0064] like Figure 3 As shown, the display system 16 inputs the camera image captured by the camera 12, the detection information detected by the radar 14, and the measurement information measured by the vehicle speed sensor 15 into the display control device 35, and outputs the processed information processed by the display control device 35 to the first display unit 28, the second display unit 30, and the projection unit 51.

[0065] Camera 12 captures images of the area in front of the vehicle. The images captured by camera 12 are input into display control device 35. Radar 14 detects the position information of an object 90 in front of the vehicle. The detection information detected by radar 14 is input into display control device 35. Vehicle speed sensor 15 detects the speed V1 of vehicle 10 (reference). Figure 1 The speed of the vehicle 10, V1, measured by the vehicle speed sensor 15, is input to the display control device 35.

[0066] The display control device 35 is configured as an ECU (Electronic Control Unit) that implements various controls. The display control device 35 is configured to include a CPU (Central Processing Unit) 36, a ROM (Read-Only Memory) 38, a RAM (Random Access Memory) 40, a memory 42, a communication interface (communication I / F) 44, and an input / output interface (input / output I / F) 46. All components are connected together via a bus 48 in a manner enabling mutual communication.

[0067] CPU 36 is a central processing unit that executes various programs or controls various components. Specifically, as a processor, CPU 36 reads programs from ROM 38 (memory) or memory 42 (memory) and executes the programs using RAM 40 as its working area. Furthermore, CPU 36 performs control of the aforementioned structures and various arithmetic operations based on the programs recorded in ROM 38 or memory 42.

[0068] ROM 38 stores various programs and data. RAM 40 serves as a working area for temporary storage of programs or data. Storage 42 is a non-temporary recording medium, constructed of HDD (Hard Disk Drive) or SSD (Solid State Drive), that stores various programs, including the operating system, and various data. In the first embodiment, display programs, etc., for implementing the display processing described later are stored in ROM 38 or storage 42.

[0069] The input / output interface 46 is connected to a first display unit 28, a second display unit 30, a projection unit 51, a camera 12, a radar 14, and a vehicle speed sensor 15.

[0070] [Functional Structure of the Display Control Device]

[0071] The function of the display control device 35 in the first embodiment will be explained.

[0072] like Figure 4 As shown, the display control device 35 functionally includes a position information acquisition unit 54, a speed information acquisition unit 56, a proximity direction acquisition unit 58, a processing time estimation unit 60, an offset unit 62, and a marker generation unit 64. Additionally, Figure 4 The functional structures shown are implemented, for example, by having the CPU 36 execute programs stored in the ROM 38 or the memory 42.

[0073] (Location Information Acquisition Department)

[0074] The position information acquisition unit 54 acquires position information related to the position of objects 90 surrounding the vehicle 10. Specifically, the position information acquisition unit 54 detects the position information of objects 90 in the direction of travel of the vehicle 10 based on the detection information of the radar 14. Furthermore, the position information acquisition unit 54 detects the position information of objects 90 in the orthogonal direction to the direction of travel of the vehicle 10 based on the camera image of the camera 12. Alternatively, the position information of objects 90 in the direction of travel of the vehicle 10 and the position information of objects 90 in the orthogonal direction to the vehicle 10 can be detected based on at least one of the detection information of the radar 14 and the camera image of the camera 12.

[0075] Here, in a top-down view of vehicle 10, an xy coordinate system is envisioned, with the vehicle's direction of travel defined as the y-axis and the direction orthogonal to the vehicle's direction of travel defined as the x-axis. For example... Figure 5As shown, when the vehicle 92 is moving forward in front of the vehicle 10, the position information acquisition unit 54 detects the y-coordinate of the position P1 of the vehicle 92 in the direction of travel of the vehicle 10, which is position information of the vehicle's direction of travel, based on the detection information of the radar 14. That is, the position information acquisition unit 54 acquires the position P1(y1) of the vehicle 92 based on the detection information of the radar 14. Furthermore, in Figure 5 In the middle, vehicle 92 is traveling forward at a speed of V2.

[0076] like Figure 6 As shown, when pedestrian 94 is moving to the right of vehicle 10, diagonally in front of vehicle 10, the position information acquisition unit 54 detects the y-coordinate of the position P1 of pedestrian 94 in the direction of travel of vehicle 10, which is the position information of the vehicle's direction of travel, based on the detection information of radar 14, and detects the x-coordinate of the position P1 of pedestrian 94 in the orthogonal direction to the direction of travel of vehicle 10, which is the position information of the orthogonal direction of vehicle 10, based on the camera image of camera 12.

[0077] That is, the location information acquisition unit 54 acquires the position P1(x1, y1) of pedestrian 94 based on the detection information from radar 14 and the camera image from camera 12. Additionally, in Figure 6 In the middle, pedestrian 94 is moving to the right of the vehicle at a speed of V3.

[0078] (Speed ​​Information Acquisition Department)

[0079] The speed information acquisition unit 56 acquires speed information related to the speed of the vehicle 10. This speed information includes information on the absolute speed of the vehicle 10 and information on the relative speed of the vehicle 10 relative to the target 90.

[0080] Specifically, such as Figure 5 As shown, in front of vehicle 10, when vehicle 92 is moving forward, speed information acquisition unit 56 acquires speed V1, which is the absolute speed of vehicle 10, based on the measurement information of vehicle speed sensor 15.

[0081] like Figure 6 As shown, when pedestrian 94 is moving to the right of vehicle 10, diagonally in front of vehicle 10, speed information acquisition unit 56 calculates the shift of pedestrian 94's position information based on multiple camera images captured by camera 12, and obtains the relative speed of pedestrian 94 relative to vehicle 10 based on this shift. In other words, speed information acquisition unit 56 can obtain the relative speed of pedestrian 94 relative to vehicle 10 based on the change in the position of pedestrian 94 when the camera images of camera 12 are arranged in a time sequence.

[0082] Furthermore, when pedestrian 94 is moving to the right of vehicle 10, at the left diagonal front of vehicle 10, speed information acquisition unit 56 can also calculate the shift of pedestrian 94's position information based on multiple detection information detected by radar 14, and obtain the relative speed of pedestrian 94 relative to vehicle 10 based on this shift.

[0083] (Approaching the direction acquisition unit)

[0084] The approach direction acquisition unit 58 acquires the approach direction of the object 90 as it approaches the vehicle 10. Specifically, the approach direction acquisition unit 58 calculates the shift of the object 90's position information based on multiple camera images captured by the camera 12, and acquires the approach direction of the object 90 relative to the vehicle 10 based on this shift. Alternatively, the approach direction acquisition unit 58 can also calculate the shift of the object 90's position information based on multiple detection information detected by the radar 14, and acquire the approach direction of the object 90 relative to the vehicle 10 based on this shift.

[0085] like Figure 5 As shown, when a vehicle 92 is traveling forward in front of vehicle 10, the approach direction acquisition unit 58 acquires the approach direction of the vehicle 92 relative to vehicle 10 as the travel direction of vehicle 10 based on multiple detection information detected by radar 14. Furthermore, when a vehicle 92 is traveling forward in front of vehicle 10, the approach direction acquisition unit 58 can also acquire the approach direction of the vehicle 92 relative to vehicle 10 as the travel direction of vehicle 10 based on multiple camera images captured by camera 12.

[0086] Furthermore, setting the approach direction to the travel direction of vehicle 10 also includes situations where the vehicle 92 in front is traveling forward or backward at a position different from vehicle 10 in the vehicle width direction. That is, setting the approach direction to the travel direction of vehicle 10 also includes situations where the vehicle 92 in front is traveling forward or backward at the same position as vehicle 10 in the vehicle width direction, and situations where the vehicle 92 in front is traveling forward or backward at a position different from vehicle 10 in the vehicle width direction.

[0087] like Figure 6 As shown, when a pedestrian 94 is moving to the right of the vehicle 10 at a position diagonally to the left front of the vehicle 10, the approach direction acquisition unit 58 can acquire the approach direction of the pedestrian 94 relative to the vehicle 10 as the tilt direction relative to the vehicle 10's direction of travel based on multiple camera images captured by the camera 12. The tilt direction relative to the vehicle 10's direction of travel includes directions orthogonal to the vehicle 10's direction of travel.

[0088] Furthermore, when a pedestrian 94 is moving to the right of the vehicle at the left diagonal front of the vehicle 10, the approach direction acquisition unit 58 can also obtain the approach direction of the pedestrian 94 relative to the vehicle 10 as the tilt direction relative to the traveling direction of the vehicle 10 based on multiple detection information detected by the radar 14.

[0089] (Processing Time Estimation Department)

[0090] The processing time estimation unit 60 records the time from when the object 90 is captured by the camera 12 or detected by the radar 14 until the mark generation unit 64 generates a mark M1 to remind the user to pay attention to the object 90 (see reference). Figure 5 The processing time estimation unit 60 estimates the processing time up to this point. Specifically, the processing time estimation unit 60 can estimate the processing time based on the load of the CPU 36. For example, it can be set such that the greater the load on the CPU 36, the longer the processing time estimation unit 60 estimates the processing time.

[0091] Furthermore, it can be configured such that when the approach direction obtained by the approach direction acquisition unit 58 is the travel direction of the vehicle 10, the processing time estimation unit 60 estimates a shorter processing time compared to when the approach direction is an inclined direction that is inclined relative to the travel direction of the vehicle 10. It can also be configured such that the slower the speed information obtained by the speed information acquisition unit 56, the shorter the processing time estimation unit 60 estimates.

[0092] (Offset section)

[0093] The offset unit 62 offsets the position P1 of the target 90 obtained by the position information acquisition unit 54 towards the approach direction obtained by the approach direction acquisition unit 58 based on the velocity information obtained by the velocity information acquisition unit 56.

[0094] Specifically, the offset unit 62 sets the offset distance for offsetting based on the following calculation formula (1).

[0095] Offset distance [m] = velocity [km / h] × object coordinate coefficient…(1)

[0096] Object coordinate coefficient = 0.00005

[0097] Here, the object coordinate coefficient is taken into account from the time the camera 12 captures the object 90 or the radar 14 detects the object 90 until the marker generation unit 64 generates the marker M1 to remind attention to the object 90 (refer to...). Figure 5The coefficient is the processing time spent up to this point. The object coordinate coefficient can be set, for example, in the case that when the vehicle 10 approaches the object 90 at a speed of 60 km / h with a speed of V1, the offset distance is 3 to 5 m when the processing time is estimated to be 200 milliseconds.

[0098] Furthermore, the offset unit 62 can adjust the offset distance that causes the position P1 acquired by the position information acquisition unit 54 to deviate based on the processing time estimated by the processing time estimation unit 60. Specifically, when the processing time estimated by the processing time estimation unit 60 is short, the offset distance by which the offset unit 62 performs the offset can be shortened compared to when the processing time is long. That is, the object coordinate coefficient can be changed based on the processing time estimated by the processing time estimation unit 60.

[0099] To elaborate further, such as Figure 5 As shown, when a vehicle 92 is traveling forward in front of vehicle 10, the offset unit 62, based on the speed V1 obtained by the speed information acquisition unit 56 (which is the absolute speed of vehicle 10), offsets the position P1(y1) obtained by the position information acquisition unit 54 by an offset distance L1 towards the direction of travel of vehicle 10, which is the approach direction obtained by the approach direction acquisition unit 58. In this case, the offset position is set as the position P2(y2) as the offset position information.

[0100] like Figure 6 As shown, when a pedestrian 94 is moving to the right of the vehicle 10 at a position diagonally in front of the left, the offset unit 62, based on the relative velocity of the pedestrian 94 relative to the vehicle 10 obtained by the velocity information acquisition unit 56, offsets the position P1(x1, y1) obtained by the position information acquisition unit 54 by an offset distance L2 in an tilt direction relative to the travel direction of the vehicle 10 obtained by the approach direction acquisition unit 58. In this case, the offset position is set as the offset position P2(x2, y2) as the offset position information.

[0101] (Identification Generation Department)

[0102] The label generation unit 64 performs the offset position P2 based on the offset unit 62, and generates a label M1 to remind the object 90 to pay attention (see reference). Figure 5 ).

[0103] Specifically, such as Figure 5 As shown, when a vehicle 92 is traveling forward in front of the vehicle 10, the sign generation unit 64 generates a sign M1 at position P2(y2), which is the position information after the offset is performed by the offset unit 62, to remind the driver to pay attention to the vehicle 92 in front.

[0104] Moreover, such as Figure 3 As shown, the image information of the identifier M1 generated by the identifier generation unit 64 is output to the projection unit 51, and the image projected from the projection unit 51 is projected onto the third display unit 32 of the windshield 22.

[0105] Specifically, such as Figure 7 As shown, when the vehicle 92 in front of the vehicle 10 is moving forward, the logo M1 is displayed as a virtual image overlaid in the third display area G3 of the third display unit 32. That is, the logo M1 is displayed as a virtual image overlaid on the foreground of the windshield 22.

[0106] like Figure 6 As shown, when a pedestrian 94 is moving to the right of the vehicle at the left diagonal front of the vehicle 10, the sign generation unit 64 generates a sign M1 at the position P2 (x2, y2) of the offset position information after the offset is performed by the offset unit 62, to remind the pedestrian 94 to pay attention.

[0107] Moreover, such as Figure 8 As shown, when a pedestrian 94 is moving to the right of the vehicle 10 at the left diagonal front, the sign M1 is superimposed as a virtual image in the third display area G3 of the third display unit 32. That is, the sign M1 is superimposed as a virtual image on the foreground of the windshield 22.

[0108] The label generation unit 64 can also generate a label indicating the speed V1 of the vehicle 10, and as shown in the image. Figure 7 as well as Figure 8 As shown, the second indicator M2, which shows the speed V1 of the vehicle 10, is overlaid on the third display area G3 of the third display unit 32.

[0109] Furthermore, the identifier generation unit 64 can also change the size of the identifier M1 based on the position information obtained by the position information acquisition unit 54. Specifically, as follows: Figure 9 As shown, it can be set such that the farther the object 90 is from the vehicle 10, the smaller the size of the mark M1 will be in the mark generation unit 64.

[0110] [The flow of display processing implemented by the display control device]

[0111] based on Figure 10 The flowchart shown illustrates the display processing flow implemented by the display control device 35.

[0112] like Figure 10 As shown, when the display processing begins, the position information acquisition unit 54 acquires the position P1 of the target 90 based on the detection information of the radar 14 and the camera image of the camera 12 (step S101).

[0113] Next, the approach direction acquisition unit 58 acquires the approach direction of the target 90 approaching the vehicle 10 (step S102).

[0114] Next, the display control device 35 determines whether the approach direction obtained by the approach direction acquisition unit 58 is the travel direction of the vehicle 10 (step S103). If the approach direction obtained by the approach direction acquisition unit 58 is the travel direction of the vehicle 10 ("yes" in step S103), the speed information acquisition unit 56 acquires the speed V1, which is the absolute speed of the vehicle 10, based on the measurement information of the vehicle speed sensor 15 (step S104).

[0115] Next, the offset unit 62 offsets the position P1 obtained by the position information acquisition unit 54 towards the vehicle's travel direction, which is the approach direction, obtained by the approach direction acquisition unit 58, based on the speed V1, which is the absolute speed of the vehicle 10 obtained by the speed information acquisition unit 56 (step S105). In addition, the offset unit 62 can adjust the offset distance based on the processing time inferred by the processing time inference unit 60.

[0116] Next, the identifier generation unit 64 generates identifier M1 at position P2 after the offset unit 62 has performed the offset (step S106).

[0117] Next, the information of the identifier M1 generated by the identifier generation unit 64 is output to the projection unit 51, and the identifier M1 is superimposed as a virtual image in the third display area G3 of the third display unit 32 (step S107), and the display process ends.

[0118] On the other hand, in step S103, if the approach direction obtained by the approach direction acquisition unit 58 is an inclined direction that is tilted relative to the travel direction of the vehicle 10 (in step S103, it is "No"), the speed information acquisition unit 56 obtains the relative speed of the vehicle relative to the target 90 (step S108).

[0119] Next, the offset unit 62 offsets the position P1 obtained by the position information acquisition unit 54 towards the tilt direction obtained by the approach direction acquisition unit 58 based on the relative speed of the vehicle 10 obtained by the speed information acquisition unit 56 (step S109). In addition, the offset unit 62 can adjust the offset distance based on the processing time inferred by the processing time inference unit 60.

[0120] Next, the identifier generation unit 64 generates identifier M1 at position P2 after the offset unit 62 has performed the offset (step S110).

[0121] Next, the information of the identifier M1 generated by the identifier generation unit 64 is output to the projection unit 51, and the identifier M1 is superimposed as a virtual image in the third display area G3 of the third display unit 32 (step S111), and the display process ends.

[0122] [Effect of the first implementation method]

[0123] Next, the function and effects of the first embodiment will be explained.

[0124] The display control device 35 of the first embodiment includes: a position information acquisition unit 54, which acquires position information related to the position of the object 90 based on at least one of a camera image from a camera 12 that captures an object 90 in front of the vehicle 10 and detection information from a radar 14 that detects the object 90 in front of the vehicle 10; a speed information acquisition unit 56, which acquires speed information related to the speed of the vehicle 10; an approach direction acquisition unit 58, which acquires the approach direction of the object 90 approaching the vehicle 10; an offset unit 62, which offsets the position P1 acquired by the position information acquisition unit 54 towards the approach direction acquired by the approach direction acquisition unit based on the speed information acquired by the speed information acquisition unit 56; and a mark generation unit 64, which generates a mark M1, which is superimposed on the position P2 after the offset by the offset unit 62 in a third display area G3 provided in the passenger compartment to draw attention to the object 90 (see reference). Figure 4 ).

[0125] Additionally, from the moment the camera 12 captures the object 90 or the radar 14 detects the object 90, until the marker generation unit 64 generates a marker M1 to alert attention to the object 90 (see reference). Figure 5 Up to this point, processing time will be consumed by communication, drawing, etc. Therefore, a time lag will occur due to this processing time, resulting in... Figure 11 as well as Figure 12 As shown, there is a problem where a positional discrepancy exists between the position of the overlapping marker M0 that draws attention to the object 90 and the actual position of the object 90. This positional discrepancy becomes more pronounced the faster the vehicle 10 travels.

[0126] In the first embodiment, a marker M1 is generated at a position P2 after the position P1 of the target 90 acquired by the position information acquisition unit 54 has been shifted based on velocity information. This allows the marker M1, which draws attention to the target 90, to be overlaid and displayed at the position P2 after the position of the target 90 acquired by the position information acquisition unit 54 has been shifted according to velocity. Therefore, it is possible to generate a marker M1 (see reference 64) from the moment the target 90 is captured by the camera 12 or detected by the radar 14 until the marker generation unit 64 generates the marker M1 that draws attention to the target 90. Figure 5 The positional deviation between the overlapping marker M1 and the actual object 90, which is caused by the processing time up to this point, is corrected.

[0127] Furthermore, by shifting the position of the target 90 acquired by the position information acquisition unit 54 toward the approach direction of the target 90, the position P1 of the target 90 acquired by the position information acquisition unit 54 is shifted toward the direction that will produce a positional deviation between the overlapping display mark M1 and the actual target 90.

[0128] Therefore, it is possible to appropriately correct the positional deviation between the position of the overlapping marker M1 that draws attention to the object 90 and the actual position of the object 90. As a result, the position of the overlapping marker M1 can be made closer to the position of the actual object 90.

[0129] In the display control device 35 of the first embodiment, when the approach direction acquired by the approach direction acquisition unit 58 is the travel direction of the vehicle 10, the speed information is set as the speed V1, which is the absolute speed of the vehicle 10 (see reference). Figure 5 ).

[0130] By setting the speed information to the absolute speed of vehicle 10, speed information is obtained by detecting the speed of vehicle 10. Therefore, compared to setting the speed information to the relative speed of vehicle 10 with respect to object 90, speed information can be obtained more quickly. As a result, the time it takes for the offset unit 62 to offset the position P1 of object 90 obtained by position information acquisition unit 54 can be shortened. In addition, the load on the CPU mounted on display control device 35 can also be reduced.

[0131] Therefore, it is possible to more appropriately correct the positional deviation between the position of the marker M1 that is displayed in an overlapping manner to draw attention to the object 90 and the actual position of the object 90.

[0132] In the display control device 35 of the first embodiment, when the approach direction acquired by the approach direction acquisition unit 58 is an inclined direction that is tilted relative to the travel direction of the vehicle 10, the speed information is set to information related to the relative speed of the vehicle 10 relative to the target 90 (see reference). Figure 6 ).

[0133] By setting the speed information to the relative speed of the vehicle 10 relative to the target 90, the offset distance by which the offset unit 62 offsets the position of the target 90 acquired by the position information acquisition unit 54 can be appropriately set based on this relative speed. Therefore, the positional deviation between the position of the marker M1, which is displayed in an overlay to remind attention to the target 90, and the actual position of the target 90 can be corrected more appropriately.

[0134] In the display control device 35 of the first embodiment, the offset unit 62 generates an indicator M1 (see reference) to remind attention to the object 90 from the time the object 90 is captured by the camera 12 or detected by the radar 14. Figure 5 The processing time spent up to this point is used to adjust the offset distance of the position P1 of the target 90 acquired by the position information acquisition unit 54.

[0135] From the moment object 90 is captured by camera 12 or detected by radar 14, the marker generation unit 64 generates a marker M1 to alert attention to object 90 (see reference). Figure 5 The offset distance by which the position P1 of the target 90 acquired by the position information acquisition unit 54 is offset is adjusted based on the processing time so far. This means that the offset distance is set to be longer when the processing time is long and shorter when the processing time is short. Therefore, the offset distance by which the offset unit 62 offsets the position of the target 90 acquired by the position information acquisition unit 54 can be appropriately set according to the processing time. As a result, the position of the overlaid marker M1 can be corrected to an appropriate position corresponding to the processing time.

[0136] In the display control device 35 of the first embodiment, the position information acquisition unit 54 acquires vehicle travel direction position information related to the position of the object 90 in the travel direction of the vehicle 10 based on radar detection information, and acquires vehicle orthogonal direction position information related to the position of the object 90 in an orthogonal direction orthogonal to the travel direction of the vehicle 10 based on the camera image of the camera 12 (see reference). Figure 1 ).

[0137] The position information of the object 90 in the direction of travel of vehicle 10 is obtained based on the detection information of radar 14, and the position information of the object 90 in the orthogonal direction to the direction of travel of vehicle 10 is obtained based on the camera image of camera 12. Therefore, more accurate position information of object 90 can be obtained. As a result, it is possible to more appropriately correct the positional deviation between the position of the marker M1 that is overlaid to remind attention to object 90 and the actual position of object 90.

[0138] In the display control device 35 of the first embodiment, the identifier generation unit 64 changes the size of the identifier M1 based on the position P1 obtained by the position information acquisition unit 54 (see reference). Figure 9 ).

[0139] By changing the size of the marker M1 based on the position P1 obtained by the position information acquisition unit 54, the size of the marker M1 can be enlarged when the vehicle 10 is close to the object 90, and reduced when the vehicle 10 is far from the object 90. Therefore, the marker M1 of an appropriate size can be overlaid and displayed according to the distance to the object 90. As a result, the occupant H can identify the distance to the object 90 based on the size of the marker M1.

[0140] The display system 16 of the first embodiment includes: a display device 34; a camera 12 that captures images of an object 90 in front of the vehicle 10; and a radar 14 that detects the object 90 in front of the vehicle 10 (see reference). Figure 3 ).

[0141] The display system 16, equipped with display device 34, camera 12, and radar 14, can appropriately correct for positional deviations between the position of the overlaid marker that draws attention to the object and the actual position of the object. As a result, the position of the overlaid marker can be made closer to the position of the actual object.

[0142] In the vehicle 10 of the first embodiment, a third display area G3 (see reference) is provided on the windshield 22. Figure 2 ).

[0143] Since the symbol M1 is displayed in the third display area G3 of the windshield 22, the occupant H can recognize the symbol M1 while keeping their gaze forward. Therefore, the occupant H can recognize the symbol without taking their eyes off the road while driving.

[0144] [Second Implementation]

[0145] The display system of the second embodiment differs from the display system of the first embodiment in that it overlaps the display areas of the markers that remind attention to the object.

[0146] The structure of the display system according to the second embodiment will be described below. Furthermore, the same terms or symbols will be used to describe parts that are the same as or equivalent to those described in the first embodiment.

[0147] In the second embodiment, such as Figure 13 As shown, in the second display area G2 of the second display unit 30, the mark M1 that reminds attention to the object 90 is displayed in an overlapping manner.

[0148] The camera image F captured by camera 12 is displayed on the upper side of the second display area G2, and the current location 10A of vehicle 10 and the path (guide path) R to the destination are displayed on the map via the navigation system on the lower side of the second display area G2.

[0149] The information of the image of the identifier M1 generated by the identifier generation unit 64 is output to the second display unit 30, which serves as the output unit, and the identifier M1 is overlaid and displayed in the second display area G2.

[0150] Even with the structure described above, the same effect as the display system of the first embodiment can be achieved.

[0151] The display system of this disclosure has been described above based on the embodiments described above. However, the specific structure is not limited to these embodiments, and design changes are permitted as long as they do not depart from the spirit of the disclosure.

[0152] In the above embodiment, an example is shown where the camera 12 captures an image of the area in front of the vehicle and the radar 14 detects the position information of an object 90 in front of the vehicle. However, the camera and radar are not limited to this method; they can also capture or detect images of the area around the vehicle.

[0153] In the above embodiment, an example is shown where the position information acquisition unit 54 acquires the position information of the target 90 based on the detection information of the radar 14 and the camera image of the camera 12. However, the position information acquisition unit is not limited to this method; for example, the position information of the target may also be acquired based on the detection information of at least one of radar, camera, LiDAR (Laser Radar), and sonar.

[0154] In the above embodiment, an example is shown where the speed information acquisition unit 56 acquires the relative speed of the pedestrian 94 relative to the vehicle 10 based on multiple camera images captured by the camera 12 or multiple detection information detected by the radar 14. However, the speed information acquisition unit is not limited to this method; for example, it may acquire the relative speed of an object relative to the vehicle based on detection information from at least one of radar, camera, LiDAR, and sonar. Furthermore, the speed information acquisition unit may also acquire the relative speed based on information about the inter-vehicle distance.

[0155] In the above embodiment, an example is shown where the approach direction acquisition unit 58 acquires the approach direction of an object relative to the object 90 of the vehicle 10 based on multiple camera images captured by the camera 12 or multiple detection information detected by the radar 14. However, the approach direction acquisition unit 58 is not limited to this method; for example, the approach direction of the object relative to the vehicle may also be acquired based on detection information from at least one of radar, camera, LiDAR, and sonar.

[0156] The above embodiment shows an example in which the offset unit 62 adjusts the offset distance of the position P1 obtained by the position information acquisition unit 54 based on the processing time inferred by the processing time inference unit 60. However, the offset unit may also adjust the offset distance without considering the processing time inferred by the processing time inference unit.

[0157] In the first embodiment, an example is shown where the image of the logo M1 generated by the logo generation unit 64 is projected onto the third display unit 32 of the windshield 22. In the second embodiment, an example is shown where the image of the logo M1 generated by the logo generation unit 64 is displayed on the second display unit 30. However, the image of the logo generated by the logo generation unit can be displayed on the first display unit 28 or projected onto a synthesizer provided on the upper surface of the dashboard.

[0158] In the above embodiments, an example of applying the display system of this disclosure to a vehicle 10 traveling in front is shown. However, the display system of this disclosure can also be applied to vehicles traveling on curves and vehicles traveling around corners.

[0159] Although the processing implemented by the display control device 35 in the above embodiment has been described as software processing implemented by executing a program, it is not limited to this. For example, it may also be a process implemented by hardware. Alternatively, it may be a process that combines both software and hardware. Furthermore, in the case of software processing, the program may be stored in various non-transitory storage media such as CD-ROM (Compact Disc Read-Only Memory), DVD-ROM (Digital Versatile Disc Read-Only Memory), and USB (Universal Serial Bus) memory, and circulated therein, so that a processor such as CPU 36 can execute it. The program may also be provided by being downloaded from an external device via a network.

Claims

1. A display control device, comprising: The location information acquisition unit acquires location information related to the location of the object based on the detection information of the object detection unit that detects objects around the vehicle; The speed information acquisition unit acquires speed information related to the speed of the vehicle. An approach direction acquisition unit acquires the approach direction of the target approaching the vehicle; The offset unit, based on the velocity information obtained by the velocity information acquisition unit, offsets the position information obtained by the position information acquisition unit towards the approach direction obtained by the approach direction acquisition unit; The marking generation unit generates a marking based on the offset position information after the offset unit has performed an offset. The marking is overlaid and displayed in a display area set inside the carriage to draw attention to the object. It is determined whether the approach direction is the vehicle's direction of travel or an inclination direction that is tilted relative to the vehicle's direction of travel. The speed information is the absolute speed of the vehicle when the approach direction is the vehicle's direction of travel, and the relative speed between the target and the vehicle when the approach direction is the tilt direction. The offset unit adjusts the offset distance by which it shifts the position information based on the processing time taken from the time the object detection unit detects the object to the time the identifier generation unit generates the identifier. The processing time is set to be shorter when the approach direction is determined to be the vehicle's direction of travel, and longer when the approach direction is determined to be the tilt direction.

2. The display control device as described in claim 1, wherein, The location information acquisition unit acquires vehicle travel direction location information related to the position of the object in the vehicle's travel direction based on radar detection information, and acquires vehicle orthogonal direction location information related to the position of the object in an orthogonal direction orthogonal to the vehicle's travel direction based on camera images.

3. The display control device as described in claim 1, wherein, The identifier generation unit changes the size of the identifier based on the location information obtained by the location information acquisition unit.

4. A display device comprising: The display control device according to any one of claims 1 to 3; An output unit that outputs an image that overlays the logo onto a display area located inside the carriage. The display area overlays the image output by the output unit.

5. A display system, comprising: The display device according to claim 4; The object detection department detects the objects around the vehicle.

6. A vehicle, comprising: The display system according to claim 5; The windshield forms the display area.

7. A display control method, wherein, The following processing is performed by the processor: Position information related to the position of the object is obtained based on the detection information of the object detection unit that detects objects around the vehicle; Obtain speed information related to the speed of the vehicle; Obtain the approach direction of the target towards the vehicle; The position information is shifted towards the approach direction based on the speed information; An identifier is generated based on the offset position information, and the identifier is overlaid in a display area set in the carriage to draw attention to the object; It is determined whether the approach direction is the vehicle's direction of travel or an inclination direction that is tilted relative to the vehicle's direction of travel. The speed information is the absolute speed of the vehicle when the approach direction is the vehicle's direction of travel, and the relative speed between the target and the vehicle when the approach direction is the tilt direction. The offset unit adjusts the offset distance by which the position information is shifted based on the processing time taken from the time the object detection unit detects the object to the time the identifier generation unit generates the identifier. The processing time is set to be shorter when the approach direction is determined to be the vehicle's direction of travel, and longer when the approach direction is determined to be the tilt direction.

8. A non-transitory storage medium storing a program, wherein, The program causes the processor to perform the following processing: Position information related to the position of the object is obtained based on the detection information of the object detection unit that detects objects around the vehicle; Obtain speed information related to the speed of the vehicle; Obtain the approach direction of the target towards the vehicle; The position information is shifted towards the approach direction based on the speed information; An identifier is generated based on the offset position information, and the identifier is overlaid in a display area set in the carriage to draw attention to the object; It is determined whether the approach direction is the vehicle's direction of travel or an inclination direction that is tilted relative to the vehicle's direction of travel. The speed information is the absolute speed of the vehicle when the approach direction is the vehicle's direction of travel, and the relative speed between the target and the vehicle when the approach direction is the tilt direction. The offset unit adjusts the offset distance by which the position information is shifted based on the processing time taken from the time the object detection unit detects the object to the time the identifier generation unit generates the identifier. The processing time is set to be shorter when the approach direction is determined to be the vehicle's direction of travel, and longer when the approach direction is determined to be the tilt direction.

Images