Display control method and display control device

Abstract

The display control method and the display control device acquire viewpoint displacement information indicating a displacement of a viewpoint position of an occupant in a vehicle in a stationary system of the vehicle, and extract, on the basis of the viewpoint displacement information, at least one or more specific components having only a frequency equal to or less than a predetermined frequency from among at least one or more components constituting the displacement. A display position of a virtual image displayed in a display area around the see-through vehicle is corrected based on the extracted specific component.

Description

Technical Field

[0001] The invention relates to a display control method and a display control device. Background Art

[0002] Patent Document 1 discloses a head-up display device that displays a virtual image in a manner superimposed on an object in the scenery outside a vehicle. According to the head-up display device, the position of a viewer's viewpoint in the vertical direction is acquired, and the virtual image is corrected to suppress the virtual image from deviating from the object based on the amount of deviation of the acquired viewer's viewpoint in the vertical direction from a reference position, and a display image is generated based on the corrected virtual image.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-103888 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] According to the technology described in Patent Document 1, there is a problem that, when the position of the viewpoint of the occupant changes rapidly, a time delay in correcting the virtual image causes an unnatural feeling to the corrected virtual image.

[0008] The present invention has been made in view of the above problems and aims to provide a display control method and a display control device that can suppress the unnatural feeling of a virtual image corrected and displayed so as to overlap with a foreground seen by the occupant even when the position of the occupant's viewpoint changes quickly.

[0009] Solutions for solving problems

[0010] In order to solve the above-mentioned problem, a display control method and a display control device according to one embodiment of the present invention obtain viewpoint displacement information indicating the displacement of the viewpoint position of an occupant riding in a vehicle in a stationary system of the vehicle, extract at least one or more specific components having only frequencies below a predetermined frequency from at least one or more components constituting the displacement based on the viewpoint displacement information, and correct the display position of a virtual image displayed in a display area around a see-through vehicle based on the extracted specific components.

[0011] Effects of the Invention

[0012] According to the present invention, even when the position of the viewpoint of the occupant changes rapidly, it is possible to suppress a sense of unnaturalness with respect to a virtual image corrected and displayed so as to be superimposed on a foreground seen by the occupant. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing the structure of a display control device according to an embodiment of the present invention.

[0014] Figure 2 This is a flowchart showing the processing of the display control device according to one embodiment of the present invention.

[0015] Figure 3 This is a diagram showing an example of the relationship between the amplitude of a specific component and the correction amount of the display position.

[0016] Figure 4 This is a diagram showing an example of the deviation of the position of the virtual image caused by the displacement of the viewpoint position of the occupant. DETAILED DESCRIPTION

[0017] Next, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description, the same reference numerals are given to the same structures and repeated descriptions are omitted.

[0018] [Structure of display control device]

[0019] Reference Figure 1 The following describes a configuration example of a display control device according to the present embodiment. As an example, the display control device is mounted on a vehicle. Figure 1 As shown, the display control device includes a display unit 21 , a detection unit 30 , and a controller 100 .

[0020] The display unit 21 is connected to the controller 100, and has a display area 23 that allows viewing of the surroundings of the vehicle from the driver's seat or other seats of the vehicle. The AR (Augmented Reality) guidance (image) generated by the controller 100 is displayed in the display area 23. Therefore, the AR guidance is displayed in the display area 23 in a manner superimposed on the surrounding scenery of the vehicle.

[0021] The AR guidance presented by the display unit 21 shows, for example, a guidance route to the vehicle's destination, a road map, the vehicle's current position, and points of interest (POIs) that are scattered between the vehicle and the destination and are of interest to the user.

[0022] For example, the display unit 21 is a head-up display device. The display area 23 may be a predetermined area in the windshield WS (front windshield) of the vehicle, and the AR guidance is projected from a projector (not shown) onto the display area 23 to present the AR guidance to the user. As a result, the user visually recognizes the AR guidance superimposed on the surrounding scenery of the vehicle that can be seen through the windshield WS.

[0023] The detection unit 30 obtains viewpoint displacement information indicating the displacement of the viewpoint position of the passenger in the vehicle in the stationary system of the vehicle. More specifically, the detection unit 30 obtains an image obtained by photographing the head of the passenger through a camera (in-vehicle camera) installed in the vehicle compartment, and obtains the viewpoint position (eye position) and line of sight (the direction in which the eyes are facing) of the passenger based on the image. The detection unit 30 can also be used as a driver monitoring system (DMS: Driver Monitoring System) or an occupant monitoring system (OMS: Occupant Monitoring System).

[0024] The driver monitoring system is a system that monitors the driving status of passengers through the in-vehicle camera to prevent dangerous driving / accidents. For example, the driver monitoring system performs facial recognition, dangerous driving detection, distracted driving detection, fatigue driving detection, and age / gender / expression determination. The occupant monitoring system is also a system that monitors the status of passengers through the in-vehicle camera.

[0025] The viewpoint displacement information indicating the displacement of the viewpoint position of the occupant riding in the vehicle shows the amount of deviation of the viewpoint position of the occupant from the position (reference position) fixed in the stationary system of the vehicle. The viewpoint displacement information is composed of the following information: information on the displacement of the viewpoint position in the front-rear direction of the vehicle (the direction of travel when the vehicle is traveling straight); information on the displacement of the viewpoint position in the vertical direction of the vehicle (the height direction of the vehicle); and information on the displacement of the viewpoint position in the left-right direction orthogonal to the front-rear direction and the vertical direction.

[0026] The controller 100 is a general-purpose computer including a CPU (Central Processing Unit), a memory, a storage device, an input / output unit, etc. The controller 100 may be connected to a navigation device (not shown). For example, the navigation device executes route guidance for the vehicle.

[0027] A computer program for making the controller 100 function as a display control device is installed in the controller 100. By executing the computer program, the controller 100 functions as a plurality of information processing circuits (110, 150, 160, 170, 140) included in the display control device. In addition, the computer program may be stored in a recording medium that can be read and written by a computer.

[0028] In this embodiment, an example of realizing a plurality of information processing circuits (110, 150, 160, 170, 140) by software is shown. However, dedicated hardware for executing each information processing shown below can be prepared to constitute the information processing circuit (110, 150, 160, 170, 140). In addition, the plurality of information processing circuits (110, 150, 160, 170, 140) can also be constituted by independent hardware. Furthermore, the information processing circuit (110, 150, 160, 170, 140) can also be used as a navigation device or a control unit for controlling a vehicle.

[0029] like Figure 1 As shown, the controller 100 includes an image generating unit 110 , a spectrum converting unit 150 , an extracting unit 160 , a position correcting unit 170 , and an outputting unit 140 as a plurality of information processing circuits ( 110 , 150 , 160 , 170 , 140 ).

[0030] The image generation unit 110 generates, for example, an AR guide (virtual image) superimposed on the path of the vehicle scheduled to travel, which is visually recognized by the occupant in the display area 23 when viewed from the driver's seat. In addition, the image generation unit 110 sets the display position of the AR guide so that the viewpoint of the occupant sitting in the driver's seat is at a fixed position (reference position) in the stationary system of the vehicle. The display position is corrected by the position correction unit 170 described later based on the viewpoint displacement information.

[0031] The spectrum conversion unit 150 decomposes the displacement of the occupant's viewpoint position into at least one component based on the viewpoint displacement information. For example, the spectrum conversion unit 150 performs Fourier transform on the displacement of the occupant's viewpoint position to decompose it into components of each frequency. In addition, the spectrum conversion unit 150 can also calculate the phase of each component obtained by decomposition.

[0032] The extraction unit 160 extracts at least one specific component having only frequencies below a predetermined frequency from at least one or more components constituting the displacement of the viewpoint position of the occupant based on the viewpoint displacement information. For example, the extraction unit 160 may extract at least one specific component having only frequencies below a predetermined frequency from the components of each frequency obtained by the spectrum conversion unit 150. Alternatively, the extraction unit 160 may extract only specific components having only frequencies below a predetermined frequency from the displacement of the viewpoint position of the occupant using a low-pass filter.

[0033] The extraction of specific components by the extraction unit 160 may also be performed in each direction of the front-rear direction, the vertical direction, and the left-right direction of the vehicle. That is, the extraction unit 160 may extract specific components for the displacement of the viewpoint position along the front-rear direction of the vehicle. The extraction unit 160 may extract specific components for the displacement of the viewpoint position along the vertical direction of the vehicle. The extraction unit 160 may extract specific components for the displacement of the viewpoint position along the left-right direction of the vehicle.

[0034] The position correction unit 170 corrects the display position of the virtual image displayed in the display area 23 based on the specific component extracted by the extraction unit 160. The specific component has a frequency below a predetermined frequency and does not have a frequency greater than the predetermined frequency. Therefore, the correction of the display position by the position correction unit 170 is performed based on only the component having a frequency below the predetermined frequency.

[0035] Therefore, when the position of the viewpoint of the occupant changes at a frequency greater than the predetermined frequency due to the vibration of the vehicle, etc., the display position is not corrected by the position correction unit 170. On the other hand, when the position of the viewpoint of the occupant changes at a frequency less than the predetermined frequency due to the fatigue of the occupant, the change of the posture during driving, etc., the display position is corrected by the position correction unit 170. As a result, the unnatural feeling of the corrected virtual image caused by the time delay in the control of the correction display position is reduced.

[0036] Alternatively, the position correction unit 170 may set a larger correction amount as the amplitude of the specific component increases, and the position correction unit 170 may change the displayed position by only the correction amount. This prevents the position of the AR guide displayed in a manner superimposed on the scenery around the vehicle from being displayed at a position that is significantly deviated from the scenery around the vehicle.

[0037] Reference Figure 4 The point that the display position of the AR guide changes due to the displacement of the viewpoint position of the occupant will be described. Figure 4 The figure shows an example of the positional deviation of the virtual image caused by the displacement of the viewpoint of the passenger. As a result of light projected from a projector (not shown) being reflected at a reflection point DP on the windshield WS and reaching the viewpoint of the passenger, the passenger visually recognizes the AR guidance.

[0038] When the occupant's viewpoint is at position PS1, the occupant recognizes that the virtual image corresponding to the AR guidance is at position MG1. Here, when the occupant's viewpoint changes to position PS2, the occupant sees that the virtual image corresponding to the AR guidance is at position MG2, while the reflection point DP does not change. That is, when the occupant's viewpoint changes from position PS1 to position PS2, the position of the virtual image corresponding to the AR guidance changes from position MG1 to position MG2.

[0039] Since the position MG1 and the position MG2 are offset by a distance LG in the front-rear direction of the vehicle, the virtual image displayed in a manner superimposed on the surrounding scenery of the vehicle may feel unnatural depending on the size of the distance LG. Therefore, it is necessary to change the reflection point DP to suppress the position of the virtual image corresponding to the AR guidance from changing from the position MG1 to the position MG2.

[0040] However, the change in the position of the occupant's viewpoint can include both fast changes due to the vibration of the vehicle, etc., and slow changes due to the fatigue of the occupant, changes in posture during driving, etc. Therefore, when the display position is corrected based on the components of all frequencies included in the change in the position of the occupant's viewpoint, there may be a situation where the correction of the display position is delayed due to a time delay in correcting the virtual image, etc. In this case, an unnatural feeling is generated in the corrected virtual image. In order to suppress the unnatural feeling caused by such correction, the position correction unit 170 corrects the display position based on components having only frequencies below a specified frequency.

[0041] In addition, the position correction unit 170 may also correct the display position when the amplitude of a specific component is greater than a predetermined threshold value. Thus, when the display position of the AR guidance does not deviate significantly from the surrounding scenery of the vehicle, the display position is not corrected, thereby reducing the processing load associated with the correction of the display position.

[0042] In addition, the position correction unit 170 may also set the first threshold value as a prescribed threshold value when the specific component is a component along the vertical direction of the vehicle. Moreover, the position correction unit 170 may also set a second threshold value greater than the first threshold value as a prescribed threshold value when the specific component is a component along the left-right direction of the vehicle. The reason can be explained as follows. Regarding the position deviation of the virtual image caused by the component along the vertical direction, the change in the distance between the virtual image and the vehicle is the largest factor. This is because: the position deviation of the virtual image caused by the component along the vertical direction is likely to cause an unnatural feeling to the occupant, and the necessity of correction is high.

[0043] In addition, the position correction unit 170 may also set a third threshold value greater than the second threshold value as a predetermined threshold value when the specific component is a component along the front-back direction. The reason can be explained as follows. Regarding the position deviation of the virtual image caused by the component along the front-back direction, the factor of the change in the distance between the virtual image and the vehicle is the smallest. This is because: the position deviation of the virtual image caused by the component along the front-back direction is unlikely to cause an unnatural feeling to the occupant, and the necessity of correction is low.

[0044] Figure 3 The following shows how the correction amount is set by the position correction unit 170 . Figure 3 is a diagram showing an example of the relationship between the amplitude of a specific component and the correction amount of the display position. Figure 3 As shown, the position correction unit 170 may also correct the display position when the amplitude (displacement) of the specific component is larger than the predetermined threshold value TH. Furthermore, the larger the amplitude of the specific component, the larger the correction amount of the display position is set by the position correction unit 170. Thus, it is possible to suppress the unnatural feeling caused by the correction of the display position.

[0045] Furthermore, the position correction unit 170 may increase the correction amount of the display position continuously and smoothly in a manner corresponding to the increase in the amplitude of the specific component. Here, "smoothly increase" means that when the correction amount of the display position is regarded as a function of the amplitude of the specific component, the differential of the amplitude based on the function is continuous. In this way, it is possible to suppress the unnatural feeling caused by the correction of the display position.

[0046] The output unit 140 outputs the AR guide generated by the image generating unit 110 . The output AR guide is displayed in the display area 23 at the display position corrected by the position correcting unit 170 .

[0047] [Processing of the display control device]

[0048] Next, refer to Figure 2 The processing procedure of the display control device involved in this embodiment is described with reference to the flowchart. Figure 2 This is a flowchart showing the processing of the display control device according to one embodiment of the present invention.

[0049] In step S101 , the detection unit 30 acquires viewpoint displacement information indicating the displacement of the viewpoint position of an occupant riding in a vehicle in a stationary system of the vehicle.

[0050] In step S103 , the spectrum conversion unit 150 performs Fourier transformation on the displacement of the viewpoint position of the occupant to decompose it into components for each frequency.

[0051] In step S105 , the extraction unit 160 extracts at least one or more specific components having only frequencies equal to or lower than a predetermined frequency from among the components for each frequency obtained by the spectrum conversion unit 150 .

[0052] In step S107 , the position correction unit 170 determines whether the amplitude of the specific component is larger than a predetermined threshold value.

[0053] When it is determined that the amplitude of the specific component is larger than the predetermined threshold value (in the case of "Yes" in step S107), the position correction unit 170 calculates the correction amount of the display position in step S109. On the other hand, when it is determined that the amplitude of the specific component is less than the predetermined threshold value (in the case of "No" in step S107), the position correction unit 170 sets the correction amount of the display position to 0 in step S111.

[0054] In step S113 , the image generation unit 110 generates an AR guide (image). Thereafter, the position correction unit 170 corrects the display position of the virtual image displayed in the display area.

[0055] In step S115, the output unit 140 outputs the AR guide whose display position is corrected. The output AR guide is presented to the user via the display area 23. Thereafter, the processing of the display control device ends.

[0056] [Effects of Embodiment]

[0057] As described in detail above, the display control method and display control device according to the present embodiment acquire viewpoint displacement information indicating the displacement of the viewpoint position of the occupant riding in the vehicle in the stationary system of the vehicle, extract at least one or more specific components having only frequencies below a predetermined frequency from at least one or more components constituting the displacement based on the viewpoint displacement information, and correct the display position of the virtual image displayed in the display area around the see-through vehicle based on the extracted specific components.

[0058] Thus, even when the position of the viewpoint of the occupant changes rapidly, it is possible to suppress a sense of unnaturalness regarding the virtual image corrected and displayed so as to be superimposed on the foreground seen by the occupant.

[0059] In particular, when the position of the occupant's viewpoint changes at a frequency greater than a predetermined frequency due to vehicle vibration, etc., the display position is not corrected. On the other hand, when the position of the occupant's viewpoint changes at a frequency less than a predetermined frequency due to occupant fatigue, changes in driving posture, etc., the display position is corrected. As a result, the unnatural feeling of the corrected virtual image caused by the time delay in the control of the correction display position is reduced.

[0060] In addition, the display control method and display control device of the present embodiment can also correct the display position when the amplitude of the specific component is larger than the specified threshold value. Thus, when the display position of the AR guidance does not deviate significantly from the surrounding scenery of the vehicle, the display position is not corrected to reduce the processing load caused by the correction of the display position.

[0061] Furthermore, the display control method and the display control device involved in the present embodiment may also set the first threshold value as a prescribed threshold value when the specific component is a component along the vertical direction of the vehicle. Furthermore, a second threshold value greater than the first threshold value may also be set as a prescribed threshold value when the specific component is a component along the left-right direction orthogonal to the front-rear direction and the vertical direction of the vehicle. Regarding the position deviation of the virtual image caused by the component along the vertical direction, the factor of the change in the distance between the virtual image and the vehicle is the largest. The position deviation of the virtual image caused by the component along the vertical direction is likely to cause an unnatural feeling to the occupants, and the necessity of correction is high. Therefore, correction is limited to the case where the necessity of correction of the display position is high to reduce the processing load.

[0062] In addition, the display control method and display control device involved in this embodiment can also set a third threshold value greater than the second threshold value as a prescribed threshold value when the specific component is a component along the front-back direction. Regarding the position deviation of the virtual image caused by the component along the front-back direction, the factor of the change in the distance between the virtual image and the vehicle is minimal. The position deviation of the virtual image caused by the component along the front-back direction is unlikely to cause an unnatural feeling to the occupant, and the necessity for correction is low. Therefore, correction is limited to the case where the necessity for correction of the display position is high to reduce the processing load.

[0063] Furthermore, the display control method and display control device according to the present embodiment may set a larger correction amount as the amplitude of the specific component increases, and change the display position by the correction amount. This can suppress the unnatural feeling caused by the correction of the display position.

[0064] Each function described in the above embodiments can be implemented by one or more processing circuits, which include programmed processors, circuits, etc., and also include devices such as application-specific integrated circuits (ASICs), circuit components configured to perform the described functions, etc.

[0065] The content of the present invention has been described above according to the embodiments, but it is clear to those skilled in the art that the present invention is not limited to these descriptions and that various modifications and improvements can be made. It should not be understood that the discussion and drawings constituting a part of the present disclosure are used to limit the present invention. Those skilled in the art can clearly understand various alternative embodiments, examples, and application technologies based on the present disclosure.

[0066] The present invention naturally includes various embodiments not described here, etc. Therefore, the protection scope of the present invention is determined only by the invention specific matters according to the appropriate claims based on the above description.

[0067] Description of Reference Numerals

[0068] 21: display unit; 23: display area; 30: detection unit; 100: controller; 110: image generation unit; 140: output unit; 150: spectrum conversion unit; 160: extraction unit; 170: position correction unit.

Claims

1. A display control method for controlling a device having a display unit, a detection unit, and a controller, wherein: The display unit has a display area that sees through the surroundings of the vehicle. The detection unit acquires viewpoint displacement information indicating displacement of a viewpoint position of a passenger riding in the vehicle in a stationary system of the vehicle. The controller extracts, based on the viewpoint displacement information, at least one specific component having only a frequency equal to or lower than a predetermined frequency from among at least one or more components constituting the displacement. The controller corrects a display position of a virtual image displayed in the display area based on the specific component.

2. The display control method according to claim 1, characterized in that: The controller corrects the display position when the amplitude of the specific component is larger than a predetermined threshold value.

3. The display control method according to claim 2, characterized in that: the controller sets the first threshold value to the predetermined threshold value when the specific component is a component along the vertical direction of the vehicle, The controller sets a second threshold value greater than the first threshold value as the predetermined threshold value when the specific component is a component along a left-right direction orthogonal to the front-rear direction of the vehicle and the vertical direction.

4. The display control method according to claim 3, characterized in that: The controller sets a third threshold value greater than the second threshold value as the prescribed threshold value when the specific component is a component along the front-rear direction.

5. The display control method according to any one of claims 1 to 4, characterized in that: The controller sets a larger correction amount as the amplitude of the specific component increases, and the controller changes the display position relative to the set position by the correction amount.

6. A display control device comprising a display unit, a detection unit, and a controller, wherein: The display unit has a display area that sees through the surroundings of the vehicle. The detection unit acquires viewpoint displacement information indicating displacement of a viewpoint position of a passenger riding in the vehicle in a stationary system of the vehicle. The controller extracts, based on the viewpoint displacement information, at least one specific component having only a frequency equal to or lower than a predetermined frequency from among at least one or more components constituting the displacement. The controller corrects a display position of a virtual image displayed in the display area based on the specific component.

Images