Display control method and display control device

Abstract

This display control method and display control device acquire viewing point displacement information that indicates displacement of the position of a viewing point of an occupant riding in a vehicle, in a stationary system of the vehicle, and extract, on the basis of the viewing point displacement information, at least one specific component having only a frequency lower than or equal to a predetermined frequency from at least one or more components constituting the displacement. The display control method and display control device correct, on the basis of the extracted specific component, a display position of a virtual image to be displayed in a display area through which surroundings of the vehicle are seen.

Description

[Technical Field]

[0001] The present invention relates to a display control method and a display control device. [Background technology]

[0002] Patent Document 1 discloses a head-up display device that displays a virtual image so as to be superimposed on an object in a scene outside a vehicle. This head-up display device acquires the position of the viewer's viewpoint in the vertical direction, corrects the virtual image according to the amount of deviation of the acquired position of the viewer's viewpoint from a reference position in the vertical direction so as to prevent the virtual image from deviating from the object, and generates a display image based on the corrected virtual image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-103888 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 has a problem in that when the position of the occupant's viewpoint changes rapidly, the time delay when correcting the virtual image can cause discomfort to the corrected virtual image.

[0005] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a display control method and a display control device that can suppress the discomfort felt by a virtual image that is corrected and displayed so as to overlap with the foreground seen by the occupant, even when the position of the occupant's viewpoint changes rapidly. [Means for solving the problem]

[0006] In order to solve the above-described problems, a display control method and a display control device according to one aspect of the present invention acquire viewpoint displacement information indicating a displacement of the viewpoint position of an occupant riding in a vehicle in a stationary system of the vehicle, and extract, based on the viewpoint displacement information, at least one specific component having only frequencies equal to or lower than a predetermined frequency from at least one component constituting the displacement, and correct the display position of a virtual image displayed in a display area that transmits the surroundings of the vehicle based on the extracted specific component. [Effects of the Invention]

[0007] According to the present invention, even when the position of the occupant's viewpoint changes rapidly, it is possible to suppress the sense of incongruity felt by a virtual image that is corrected and displayed so as to overlap with the foreground seen by the occupant. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of a display control device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart showing the processing of a display control device according to one embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of the relationship between the amplitude of a specific component and the correction amount of the display position. [Figure 4] FIG. 4 is a diagram showing an example of a displacement of the position of a virtual image due to a displacement of the viewpoint position of an occupant. DETAILED DESCRIPTION OF THE INVENTION

[0009] Next, an embodiment of the present invention will be described in detail with reference to the drawings. In the description, the same components are designated by the same reference numerals and duplicated explanations will be omitted.

[0010] [Configuration of display control device] An example of the configuration of a display control device according to this embodiment will be described with reference to Fig. 1. The display control device is mounted on a vehicle, for example. As shown in Fig. 1, the display control device includes a display unit 21, a detection unit 30, and a controller 100.

[0011] The display unit 21 is connected to the controller 100 and has a display area 23 that allows the surroundings of the vehicle to be seen through so that it can be seen from the driver's seat or another seat in the vehicle. An AR (Augmented Reality) guide (image) generated by the controller 100 is displayed in the display area 23. Therefore, the AR guide is displayed in the display area 23 superimposed on the scenery around the vehicle.

[0012] The AR guide presented by the display unit 21 represents, for example, a guided route to the vehicle's destination, a road map, the vehicle's current position, and points of interest (POIs) that are points of interest to the user scattered between the vehicle and the destination.

[0013] For example, the display unit 21 is a head-up display device. The display area 23 may be a predetermined area on the windshield WS (front glass) of the vehicle, and an AR guide may be projected onto the display area 23 from a projector (not shown) to present the AR guide to the user. As a result, the user will visually recognize the AR guide superimposed on the scenery around the vehicle visible through the windshield WS.

[0014] The detection unit 30 acquires viewpoint displacement information indicating displacement of the viewpoint position of an occupant riding in the vehicle in a stationary system of the vehicle. More specifically, the detection unit 30 acquires an image of the occupant's head using a camera installed in the vehicle cabin (in-vehicle camera), and acquires the occupant's viewpoint position (eye position) and line of sight (direction in which the eyes are facing) based on the image. The detection unit 30 may also function as a driver monitoring system (DMS) or an occupant monitoring system (OMS).

[0015] A driver monitoring system uses an in-car camera to monitor the driving status of passengers, with the aim of preventing dangerous driving and accidents. For example, a driver monitoring system performs facial recognition, dangerous driving detection, distracted driving detection, drowsy driving detection, and age, gender, and facial expression determination. Similarly, an occupant monitoring system uses an in-car camera to monitor the status of passengers.

[0016] Viewpoint displacement information, which indicates the displacement of the viewpoint position of a vehicle occupant, indicates the amount of deviation of the viewpoint position of the occupant from a fixed position (reference position) in the vehicle's stationary system. The viewpoint displacement information consists of information on the displacement of the viewpoint position along the front-to-rear direction of the vehicle (the direction of travel when the vehicle is traveling straight), information on the displacement of the viewpoint position along the vertical direction of the vehicle (the direction of vehicle height), and information on the displacement of the viewpoint position along the left-to-right direction, which is perpendicular to the front-to-rear and vertical directions.

[0017] The controller 100 is a general-purpose computer including a CPU (Central Processing Unit), a memory, a storage device, an input / output unit, etc. The controller 100 may be connected to a navigation device (not shown). For example, the navigation device performs route guidance for the vehicle.

[0018] A computer program for functioning as a display control device is installed in the controller 100. By executing the computer program, the controller 100 functions as a plurality of information processing circuits (110, 150, 160, 170, 140) included in the display control device. The computer program may be stored in a computer-readable and writable recording medium.

[0019] In this embodiment, an example is shown in which multiple information processing circuits (110, 150, 160, 170, 140) are realized by software. However, it is also possible to configure the information processing circuits (110, 150, 160, 170, 140) by preparing dedicated hardware for executing each of the information processes described below. Also, the multiple information processing circuits (110, 150, 160, 170, 140) may be configured by individual hardware. Furthermore, the information processing circuits (110, 150, 160, 170, 140) may also be used as a navigation device or a control unit used to control the vehicle.

[0020] As shown in FIG. 1, the controller 100 includes an image generation unit 110, a spectrum conversion unit 150, an extraction unit 160, a position correction unit 170, and an output unit 140 as a plurality of information processing circuits (110, 150, 160, 170, 140).

[0021] For example, the image generation unit 110 generates an AR guide (virtual image) that overlaps with the route along which the vehicle is scheduled to travel, which is visible to the occupant in the display area 23 as viewed from the driver's seat. The image generation unit 110 sets the display position of the AR guide by assuming that the viewpoint of the occupant sitting in the driver's seat is at a fixed position (reference position) in the vehicle's stationary system. The display position is corrected by the position correction unit 170, which will be described later, based on viewpoint displacement information.

[0022] The spectrum transform unit 150 decomposes the displacement of the occupant's viewpoint position into at least one or more components based on the viewpoint displacement information. For example, the spectrum transform unit 150 performs a Fourier transform on the displacement of the occupant's viewpoint position to decompose it into components for each frequency. Alternatively, the spectrum transform unit 150 may calculate the phase of each component obtained by the decomposition.

[0023] The extraction unit 160 extracts, based on the viewpoint displacement information, at least one specific component having only a frequency equal to or lower than a predetermined frequency from at least one component constituting the displacement of the occupant's viewpoint position. For example, the extraction unit 160 may extract at least one specific component having only a frequency equal to or lower than a predetermined frequency from the components for each frequency obtained by the spectrum conversion unit 150. Alternatively, the extraction unit 160 may use a low-pass filter to extract, from the displacement of the occupant's viewpoint position, a specific component having only a frequency equal to or lower than a predetermined frequency.

[0024] The extraction unit 160 may extract a specific component for each of the longitudinal, vertical, and lateral directions of the vehicle. That is, the extraction unit 160 may extract a specific component for a displacement of the viewpoint position along the longitudinal direction of the vehicle. The extraction unit 160 may extract a specific component for a displacement of the viewpoint position along the vertical direction of the vehicle. The extraction unit 160 may extract a specific component for a displacement of the viewpoint position along the lateral direction of the vehicle.

[0025] The position corrector 170 corrects the display position of the virtual image displayed in the display area 23 based on the specific component extracted by the extractor 160. The specific component has a frequency equal to or less than a predetermined frequency, but does not have a frequency higher than the predetermined frequency. Therefore, the correction of the display position by the position corrector 170 is performed based on components having only frequencies equal to or less than the predetermined frequency.

[0026] Therefore, if the position of the occupant's viewpoint changes at a frequency greater than a predetermined frequency due to vehicle vibrations or the like, the display position is not corrected by the position corrector 170. On the other hand, if the position of the occupant's viewpoint changes at a frequency equal to or less than a predetermined frequency due to occupant fatigue, changes in posture while driving, or the like, the display position is corrected by the position corrector 170. As a result, the sense of incongruity caused by the virtual image after correction, which is due to a time delay during control to correct the display position, is reduced.

[0027] Furthermore, the position corrector 170 may set a larger correction amount as the amplitude of the specific component increases, and change the display position by the amount of the correction amount. This prevents the position of the AR guide superimposed on the scenery around the vehicle from being displayed at a position that is significantly different from the scenery around the vehicle.

[0028] The change in the display position of the AR guide due to a displacement of the occupant's viewpoint will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a shift in the position of a virtual image due to a displacement of the occupant's viewpoint. Light projected from a projector (not shown) is reflected, for example, by a reflection point DP on the windshield WS and reaches the occupant's viewpoint, causing the occupant to view the AR guide.

[0029] When the occupant's viewpoint is at position PS1, the occupant perceives the virtual image corresponding to the AR guide as being at position MG1. If the occupant's viewpoint changes to position PS2 while the reflection point DP remains unchanged, the occupant will see the virtual image corresponding to the AR guide as being at position MG2. In other words, when the occupant's viewpoint changes from position PS1 to position PS2, the position of the virtual image corresponding to the AR guide changes from position MG1 to position MG2.

[0030] Because positions MG1 and MG2 are offset by a distance LG in the longitudinal direction of the vehicle, depending on the size of the distance LG, the virtual image displayed superimposed on the scenery around the vehicle may feel unnatural. Therefore, it becomes necessary to change the reflection point DP to prevent the position of the virtual image corresponding to the AR guide from changing from position MG1 to position MG2.

[0031] However, changes in the position of the occupant's viewpoint can include both fast changes due to vehicle vibrations and slow changes due to occupant fatigue or changes in posture while driving. Therefore, if the display position is corrected based on all frequency components included in the change in the occupant's viewpoint, the correction of the display position may be delayed due to factors such as a time delay when correcting the virtual image. In this case, the corrected virtual image may cause discomfort. To suppress such discomfort caused by correction, the position correction unit 170 corrects the display position based on components having only frequencies below a predetermined frequency.

[0032] The position correction unit 170 may correct the display position when the amplitude of the specific component is greater than a predetermined threshold. As a result, if the display position of the AR guide does not deviate significantly from the scenery around the vehicle, the display position is not corrected, and the processing load associated with correcting the display position is reduced.

[0033] Alternatively, the position corrector 170 may set the first threshold as the predetermined threshold when the specific component is a component along the vertical direction of the vehicle. Furthermore, the position corrector 170 may set a second threshold greater than the first threshold as the predetermined threshold when the specific component is a component along the left-right direction of the vehicle. The reason for this can be explained as follows: The displacement of a virtual image caused by a component along the vertical direction is most significantly affected by changes in the distance between the virtual image and the vehicle. This is because the displacement of a virtual image caused by a component along the vertical direction is likely to cause discomfort to the occupant, and therefore requires a high degree of correction.

[0034] Furthermore, when the specific component is a component along the longitudinal direction, the position correction unit 170 may set a third threshold value greater than the second threshold value as the predetermined threshold value. The reason for this can be explained as follows: The displacement of the virtual image caused by the component along the longitudinal direction is least affected by the change in the distance between the virtual image and the vehicle. This is because the displacement of the virtual image caused by the component along the longitudinal direction is less likely to cause discomfort to the occupants and therefore less need for correction.

[0035] FIG. 3 shows how the correction amount is set by the position correction unit 170. FIG. 3 is a diagram illustrating an example of the relationship between the amplitude of a specific component and the correction amount of the display position. For example, as shown in FIG. 3, the position correction unit 170 may correct the display position when the amplitude (displacement) of the specific component is greater than a predetermined threshold TH. The position correction unit 170 may also set a larger correction amount of the display position as the amplitude of the specific component increases. This makes it possible to suppress any discomfort that may arise from the correction of the display position.

[0036] Furthermore, the position correction unit 170 may continuously and smoothly increase the correction amount of the display position in response to an increase in the amplitude of the specific component. Here, "smoothly increasing" means that when the correction amount of the display position is viewed as a function of the amplitude of the specific component, the derivative of the function with respect to the amplitude is continuous. This makes it possible to suppress any discomfort that may arise from the correction of the display position.

[0037] The output unit 140 outputs the AR guide generated by the image generation unit 110. The output AR guide is displayed in the display area 23 at a display position corrected by the position correction unit 170.

[0038] [Display control device processing procedure] Next, a processing procedure of the display control device according to this embodiment will be described with reference to the flowchart of Fig. 2. Fig. 2 is a flowchart showing the processing of the display control device according to one embodiment of the present invention.

[0039] In step S101, the detection unit 30 acquires viewpoint displacement information indicating a displacement of the viewpoint position of a passenger riding in the vehicle in the stationary system of the vehicle.

[0040] In step S103, the spectrum transform unit 150 performs a Fourier transform on the displacement of the occupant's viewpoint position, and decomposes it into frequency components.

[0041] In step S105, the extraction unit 160 extracts at least one specific component that has only frequencies equal to or lower than a predetermined frequency from the components for each frequency obtained by the spectrum conversion unit 150.

[0042] In step S107, the position corrector 170 determines whether the amplitude of the specific component is greater than a predetermined threshold value.

[0043] If it is determined that the amplitude of the specific component is greater than the predetermined threshold (YES in step S107), then in step S109, the position corrector 170 calculates the correction amount for the display position. On the other hand, if it is determined that the amplitude of the specific component is equal to or less than the predetermined threshold (NO in step S107), then in step S111, the position corrector 170 sets the correction amount for the display position to 0.

[0044] In step S113, the image generation unit 110 generates an AR guide (image). After that, the position correction unit 170 corrects the display position of the virtual image displayed in the display area.

[0045] In step S115, the output unit 140 outputs the AR guide whose display position has been corrected. The output AR guide is presented to the user via the display area 23. Thereafter, the processing of the display control device ends.

[0046] [Effects of the embodiment] As described above in detail, the display control method and display control device according to this embodiment acquire viewpoint displacement information indicating a displacement of the viewpoint position of a vehicle occupant in a stationary system of the vehicle, and extract, based on the viewpoint displacement information, at least one specific component having only a predetermined frequency or less from at least one component constituting the displacement. Based on the extracted specific component, correct the display position of a virtual image displayed in a display area that allows the surroundings of the vehicle to be seen.

[0047] This makes it possible to suppress the sense of incongruity felt by the occupant due to the virtual image being corrected and displayed so as to overlap with the foreground seen by the occupant, even if the position of the occupant's viewpoint changes rapidly.

[0048] In particular, if the position of the occupant's viewpoint changes at a frequency greater than a predetermined frequency due to vehicle vibrations or the like, the display position is not corrected. On the other hand, if the position of the occupant's viewpoint changes at a frequency equal to or less than the predetermined frequency due to occupant fatigue or changes in posture while driving, the display position is corrected. As a result, the sense of discomfort felt by the corrected virtual image, which is caused by factors such as a time delay during control to correct the display position, is reduced.

[0049] Furthermore, the display control method and display control device according to this embodiment may correct the display position when the amplitude of a specific component is greater than a predetermined threshold. As a result, if the display position of the AR guide does not deviate significantly from the scenery around the vehicle, the display position is not corrected, and the processing load associated with correcting the display position is reduced.

[0050] Furthermore, the display control method and display control device according to this embodiment may set a first threshold value as the predetermined threshold value when the specific component is a component along the vertical direction of the vehicle. Furthermore, when the specific component is a component along the longitudinal direction of the vehicle and the left-right direction perpendicular to the vertical direction, a second threshold value greater than the first threshold value may be set as the predetermined threshold value. The displacement of the virtual image caused by the component along the vertical direction is most significantly affected by the change in the distance between the virtual image and the vehicle. The displacement of the virtual image caused by the component along the vertical direction is likely to cause discomfort to the occupants and is therefore highly necessary to be corrected. Therefore, correction is performed only when there is a high need to correct the display position, thereby reducing the processing load.

[0051] Furthermore, the display control method and display control device according to this embodiment may set a third threshold value greater than the second threshold value as the predetermined threshold value when the specific component is a component along the longitudinal direction. The positional shift of the virtual image caused by the component along the longitudinal direction is least affected by the change in the distance between the virtual image and the vehicle. The positional shift of the virtual image caused by the component along the longitudinal direction is less likely to cause discomfort to the occupants and therefore less need for correction. Therefore, correction is performed only when there is a high need for correction of the display position, reducing the processing load.

[0052] Furthermore, the display control method and display control device according to the present embodiment may set a larger correction amount as the amplitude of the specific component increases, and change the display position by the correction amount, thereby suppressing any discomfort that may arise from correcting the display position.

[0053] Each function described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), circuit components arranged to perform the described functions.

[0054] Although the present invention has been described above based on the embodiments, it will be apparent to those skilled in the art that the present invention is not limited to these descriptions and that various modifications and improvements are possible. The descriptions and drawings that form part of this disclosure should not be understood as limiting the present invention. Various alternative embodiments, examples, and operating techniques will become apparent to those skilled in the art from this disclosure.

[0055] The present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention-specifying matters according to the scope of the claims that are appropriate from the above description. [Explanation of symbols]

[0056] 21 Display section 23 Display area 30 Detector 100 Controllers 110 Image generation unit 140 Output section 150 Spectral Conversion Unit 160 Extraction part 170 Position correction section

Claims

1. A display control method for controlling a device including a display unit, a detection unit, and a controller, comprising: the display unit has a display area that allows the surroundings of the vehicle to be seen through, The detection unit Obtain viewpoint displacement information indicating a displacement of a viewpoint position of a passenger riding in the vehicle in a stationary system of the vehicle; The controller setting a display position of the virtual image to be displayed in the display area as a set position when the viewpoint position is at a reference position fixed in the stationary system; extracting at least one specific component having only a frequency equal to or lower than a predetermined frequency from at least one component constituting the displacement based on the viewpoint displacement information; correcting a display position of a virtual image to be displayed in the display area based on the specific component; maintaining the set position without correcting the display position based on components having frequencies greater than the predetermined frequency; correcting the display position when the amplitude of the specific component is greater than a predetermined threshold; If the specific component is a component along a vertical direction of the vehicle, a first threshold is set as the predetermined threshold; When the specific component is a component along a longitudinal direction of the vehicle and a lateral direction perpendicular to the vertical direction, a second threshold value greater than the first threshold value is set as the predetermined threshold value; The display position does not include correction based on components having frequencies greater than the predetermined frequency. A display control method comprising:

2. 2. The display control method according to claim 1, The controller When the specific component is a component along the front-rear direction, a third threshold value greater than the second threshold value is set as the predetermined threshold value. A display control method comprising:

3. 3. The display control method according to claim 1 or 2, The controller The larger the amplitude of the specific component, the larger the correction amount is set, and the display position is changed from the set position by the amount of the correction amount. A display control method comprising:

4. A display control device including a display unit, a detection unit, and a controller, the display unit has a display area that allows the surroundings of the vehicle to be seen through, The detection unit Obtain viewpoint displacement information indicating a displacement of a viewpoint position of a passenger riding in the vehicle in a stationary system of the vehicle; The controller setting a display position of the virtual image to be displayed in the display area as a set position when the viewpoint position is at a reference position fixed in the stationary system; extracting at least one specific component having only a frequency equal to or lower than a predetermined frequency from at least one component constituting the displacement based on the viewpoint displacement information; correcting a display position of a virtual image to be displayed in the display area based on the specific component; maintaining the set position without correcting the display position based on components having frequencies greater than the predetermined frequency; correcting the display position when the amplitude of the specific component is greater than a predetermined threshold; If the specific component is a component along a vertical direction of the vehicle, a first threshold is set as the predetermined threshold; When the specific component is a component along a longitudinal direction of the vehicle and a lateral direction perpendicular to the vertical direction, a second threshold value greater than the first threshold value is set as the predetermined threshold value; The display position does not include correction based on components having frequencies greater than the predetermined frequency. A display control device comprising:

Images